•   How much collaboration is there with outside organizations? How many staff exchanges are there?

•   Does management have a well-defined process and criteria for determining what work is performed in-house versus what work is sponsored via grants, contracts, or other mechanisms with external entities?

•   Does the management support a culture of creativity, diversity, and entrepreneurship?

Assessing the Quality of Scientific and Technical Work

Answers to the following questions will be useful in the assessment of the quality of an organization’s technical work:

•   Does the assessment include the quality of the staff, equipment, and facilities?

•   Does the assessment include the nature of the research portfolio as to alignment with the mission and the balance in regard to basic, applied, and development work and short-, intermediate, and long-term research?

•   Does the organization have a set of indicators that can serve as parameters when the time frame precludes immediate assessment? Does the organization benchmark itself against premier organizations?

•   Who is the expected audience for the assessment?

•   Is the review done by technical peers?

•   What are the criteria for ensuring the credibility and validity of the assessment?

•   What is the scope of the assessment? Does it include proposals for new work? Does it include assessment of completed work—internal review and authority to release a report, publications, patents, invited lectures, awards, and the like?

•   Who designs and manages the assessment?

Assessing Relevance and Impact

Addressing the following questions will be useful in the assessment of an organization’s relevance and impact:

•   Does the organization have a process for identifying its stakeholders and customers?

•   Does it have a regular process for reviewing its programs and plans with its stakeholders?

•   Does the organization have a process for learning of its customers’ current and likely future needs and expectations for the organization?

•   Does the organization have an explicit process for tracking the utilization of its results (e.g., is transition to the next R&D stage actively managed and measured)?

•   Does it have a formal program for recording the history of its work from concept to final utility or impact?

•   Does the organization have a program to conduct retrospective studies of its earlier work?

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement