Metrics for assessing the technical merit and quality of the science and engineering work include the following:
• Membership in professional societies. A high percentage (perhaps at least 75 percent) of the researchers of the organization should be members of professional organizations.
• Memberships in the National Academies and/or high recognition by professional societies. A very low percentage would be indicative of overall inadequacy of staff.
• Number of members of research staff who have been awarded National Medals of Science. There should certainly be at least two or three who have received this award. This is probably a good indicator of the overall quality of the research staff, the theory being that winners of these medals would not be part of a low-quality research organization.
• Whether the research organization maintains a database of research projects, findings and results. This database should also include lists of publications in refereed journals, citations, and awards. It should also include, based on the findings and results, an assessment of lessons learned by the researchers on each project. Lessons learned from failed projects can be as valuable as those learned from successful projects. Such a database could enable an assessment by funding organizations and scientific peer-review committees of the quality of basic research as well as applied research.
• Percentage of research staff members with doctorates and/or postdoctorate fellowships. Something greater than 50 percent is the metric for recognized high-quality research organizations.
• The balance between internally sponsored basic and applied research funding and customer funding that seeks applied research and/or engineering support to address specific problems. High-quality research organizations have 10 to 15 percent of their total funding in internally generated basic and applied research projects; another 25 to 30 percent is devoted to applied research funded by external organizations looking for concept solutions to problems; the remainder is allocated to scientific and engineering support to advanced development programs. When the internally generated basic and applied research effort falls significantly below 10 percent, the overall quality and stature of the research organization diminish significantly.
The following are management functions: providing the resources available to support high-quality work, effectively delivering the services and products required to
6National Research Council, Managing Air Force Basic Research, Washington, D.C.: National Academy Press, 1993.
7G. Decker, J. Davis, R.A. Beaudet, S. Dalal, and W.H. Forster, Improving Army Basic Research: Report of an Expert Panel on the Future of Army Laboratories, Santa Monica, Calif.: RAND Arroyo Center, 2012.