both the relevance and the impact of the NIOSH program using 1–5 integer scales, and (2) providing input about emerging areas of research and recommendations for program improvement.

OVERVIEW OF THE EVALUATION FRAMEWORK

After examining different approaches to program evaluation (see Chapter 2), the framework committee decided to define the scope and stages of the evaluation process based on the logic model (Williams et al., 2009). The resulting evaluation framework described in this chapter breaks the logic models developed by NIOSH (Figure 3-1) into discrete program components to be assessed by each evaluation committee. Criteria for evaluation of each component of the framework are detailed below. In the evaluation framework (overview provided in Figure 3-2), the assessment of strategic goals and objectives, inputs, activities, and outputs (B to E) largely define the relevance of the program, while assessment of intermediate and end outcomes (F and G) largely define the program impact.

The following major components of each NIOSH program were assessed by the evaluation committees:

  • Major occupational safety and health challenges in the program area.

  • Goals and objectives as defined by NIOSH.

  • Inputs (e.g., budget; staff; facilities; and input from the program’s research management, the NIOSH Board of Scientific Counselors, and stakeholders).

  • Activities (efforts by NIOSH staff, contractors, and grantees; e.g., surveillance of injury, illness, and hazards; exposure assessment research; health-effects research; injury-risk factor research; intervention research; health services research; and technology transfer activities).

  • Outputs (NIOSH products; e.g., publications, reports, conferences, databases, tools, methods, guidelines, recommendations, education and training, and patents).

  • Intermediate outcomes (actions by external stakeholders in response to NIOSH products; e.g., policy change, training and education, self-reported use or repackaging of NIOSH data by stakeholders, adoption of NIOSH-developed technologies, implemented guidelines, and licenses).

  • End outcomes (e.g., reduction in work-related injuries, illnesses, or hazardous exposures in the workplace).

The framework committee understood that the efforts of any research program or the evaluation of that program will not be as linear as presented in either



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement