4. University research organizations: Basic research is typically part of the core mission; assessment is typically self-commissioned, conducted by external peers and focused on the longer term.

Types of Assessment

Assessments can be grouped in three general categories:

1. Self-assessment typically looks at the effectiveness of the organization’s effort to improve the quality of its workforce and facilities, its preparedness to respond to current and future mission needs, and impact, as measured, for example, by return on investment.

2. Organization-commissioned external assessment is set by the organization; it usually looks at the quality of the work being performed and the strategies for maintaining and developing new core capabilities, as well as the impact of the organization on the broader community.

3. Independent external assessments are commissioned by and report to a stakeholder. The stakeholder sets the context, and frequently the assessment is focused on the impact and the return on investment.

TIMESCALES AND MULTIDIMENSIONALITY OF R&D ORGANIZATIONS

Research and development constitute a multidimensional process. An effective assessment includes consideration of all of these dimensions in order to provide a complete and comprehensive approach that enables valid, meaningful, and useful assessment of R&D organizations.

The timescale for the assessment may be short (1 to 3 years), medium (3 to 7 years), or long (7 years or longer). Typically, assessments involving shorter timescales focus more on the research process than on the research results. The nature of an R&D organization may reflect the sector within which it operates: mission-specific (generally government), industrial, national laboratories, or academic. The R&D performed in these different sectors may be done for quite different reasons, and assessment criteria may be different for these four settings.

The stages of R&D may be characterized as basic research, applied research, advanced technology development, preproduction, and, at times, production and product fielding. As illustrated in Figure 2-1, different considerations may apply to the assessment of different stages of R&D. Characteristics of the organization include quality, relevance, productivity, and impact as well as the characteristics of its management.

Assessment measures and criteria may be qualitative, quantitative, or anecdotal. Although the traditional demand has been for quantitative metrics for assessment, many characteristics of an organization are not well suited to yielding countable indicators. An assessment may be conducted at the levels of project/task, program, organizational element, laboratory, or overall organization. Collaborations within the organization and extramural collaborations may also be considered. The audience for the assessment findings may include



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement