The following HTML text is provided to enhance online
readability. Many aspects of typography translate only awkwardly to HTML.
Please use the page image
as the authoritative form to ensure accuracy.
Evaluating Occupational Health and Safety Research Programs: Framework and Next Steps
has increased in prominence within the past 50 years. A major impetus to this growth was the need to assess the social programs instituted through the War on Poverty and Great Society policies of the 1960s (Shadish et al., 1991). Legislative requirements for the evaluation of many programs represented a turning point in the growth in the number of evaluations. Evaluation is now an established professional practice, reflected through organizations such as the American Evaluation Association and the European Evaluation Society (AEA, 2009; EES, 2009). Program evaluation is one element of results-oriented management, the approach to public management embodied in the past decade in the Government Performance and Results Act (OMB, 2009a) and the Office of Management and Budget’s (OMB’s) Program Assessment Rating Tool (OMB, 2009b).
Current efforts in program evaluation follow several schools of thought that differ in the evaluation processes used but are all focused on achieving a valid evaluation. The essence of evaluation is determining what is of value in a program. The work revolves around understanding program goals (if available), setting criteria for success, and gathering information to determine whether the criteria are being met as a result of program activities. Program evaluations focus on examining the characteristics of a portfolio of projects rather than assessing one project at a time and often use retrospective information about program outputs and outcomes. Program evaluation differs from a research project in being more tightly connected to practice; it is commissioned by a specific user or organization and designed to inform decision making. It also differs from performance measurement, which is an ongoing process that gathers indicators of what the program is accomplishing but may not assess why the indicators are changing.
Program evaluations can serve several functions. When the program is initially in development or is undergoing changes and is being evaluated with the goal of program improvement, the evaluation is termed a formative evaluation (Scriven, 1991). These evaluations are often initiated and used in-house. When the objective of the evaluation is to assess the program’s outcomes in order to determine whether the program is succeeding or has accomplished its goals, the evaluation is termed a summative evaluation (Scriven, 1967; Gredler, 1996). Users of summative evaluations are often decision makers outside of the program. Program evaluation often also helps communicate the program’s goals and accomplishments to external audiences. Evaluations provide information that contributes to decisions that shape program goals, strategic plans, and actions. In these cases, they serve instrumental functions. Often they also serve enlightenment functions, such as increasing general understanding of program operations, underlying assumptions, or social context (Weiss, 1977).
The practice of evaluating research programs has historically been somewhat separate from that of social program evaluation. Qualitative assessments of research programs in the United States date back to the 1950s (NAS, 1959). The evaluation