Evaluating Occupational Health and Safety Research Programs: Framework and Next Steps (2009)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

1 Introduction P reventing tractor rollovers on farmers, protecting construction workers from falls, improving the health of miners in dusty environments, reducing back injuries in nursing aides, developing substitute materials to eliminate hazard- ous chemical exposures, and designing work conditions to reduce fatigue and stress are among the issues critical to improving worker safety and health. In 2007, 5,657 fatal work injuries occurred in the United States, along with an estimated 4 million nonfatal occupational injuries and illnesses among workers in private industry (BLS, 2008, 2009). In addition to the human suffering involved, these statistics are associated with high economic costs. One estimate puts the costs of occupational injury and illnesses for all industries for 2005 at more than $160 billion (Leigh, 2008). The continued attention to further improve occupational health and safety through research is not only fully warranted, but such research requires critical evaluation of its relevance and impact. The core mission of the National Institute for Occupational Safety and Health (NIOSH) is to conduct research to improve and protect the health and safety of workers. In September 2004, NIOSH contracted with The National Academies to conduct a series of evaluations of individual NIOSH research programs. This set of inde- pendent evaluations focused on the relevance and impact of each of eight NIOSH programs on reducing work-related injuries, illnesses, and hazardous exposures.   program is defined as a set of A inputs and activities directed toward one or more common goals, typically under the direction of a manager or management team. 11 

12 E v a l u a t i n g O c c u p a t i o n a l H e a l t h a n d S a f e t y Research Programs From the outset of the evaluations, NIOSH leadership established the primary goal as program improvement, but the context for the evaluations also included the PART (Program Assessment Rating Tool) federal agency evaluation process. The first step in this multiphase effort was to develop an evaluation framework that could be applied across the set of program evaluations to enhance cross-study consistency. An Institute of Medicine (IOM)/National Research Council (NRC) committee (the framework committee) was appointed to develop the evaluation framework. The resulting evaluation framework was then used by eight separately appointed ad hoc committees (evaluation committees) to assess NIOSH programs in hearing loss; mining; agriculture, forestry, and fishing; respiratory diseases; personal protective technology; traumatic injury; construction; and health hazard evaluation. Each evaluation committee produced an individual report (IOM and NRC, 2006, 2008, 2009; NRC and IOM, 2007, 2008a,b, 2009a,b). This report provides the evaluation framework developed, implemented, and refined over the course of four years and eight evaluations. The framework uses a standard tool in program management and evaluation—the logic model—and provides details on the types of information that are needed and questions to be considered in each phase of the evaluation. This report has two goals: (1) to sum- marize the evaluation process and lessons learned in the development and use of the framework and (2) to provide recommendations for future evaluation efforts. The evaluation framework may prove applicable in evaluating other federal agency research programs. SCOPE OF THE TASK The framework and evaluation committees followed the same basic statement of task (Box 1-1). Although the statement of task was modified to clarify specific issues or to accommodate programs that were not specifically research programs, the basic objectives for the program evaluations remained the same: • An assessment of the relevance and impact of the NIOSH program’s contribution to reducing work-related hazardous exposures, illnesses, and injuries based on integer scales of 1 to 5, with text to support the rating; • Assessment of the program’s effectiveness in targeting new research areas and identification of emerging issues that the program should be prepared to address; and • Recommendations for program improvement.  The charges to the committees to evaluate the NIOSH Health Hazard Evaluation and Personal Protective Technology programs were each slightly modified to accommodate the unique standards- setting and investigative aspects of these programs. 

Introduction 13 BOX 1-1 Review of NIOSH Research Programs Statement of Task In response to a request from the National Institute for Occupational Safety and Health (NIOSH), the Institute of Medicine and the Division of Earth and Life Studies of the National Academies are conducting a series of evaluations of NIOSH research programs. Each evaluation is being conducted by an ad hoc committee, using a methodology and framework developed by the Committee to Review NIOSH Research Programs (framework committee). Each evaluation committee will review the program’s impact, relevance, and future di- rections. The evaluation committee will evaluate not only what the NIOSH research program is producing, but will also determine whether it is appropriate to credit NIOSH research with changes in workplace practices, hazardous exposures, and/or occupational illnesses and injuries, or whether the changes are the result of other factors unrelated to NIOSH. The program reviews should focus on evaluating the program’s impact and relevance to health and safety issues in the workplace and make recommendations for improvement. In conducting the review, the evaluation committee will address the following elements: 1.  Assessment of the program’s contribution through occupational safety and health research to reductions in workplace hazardous exposures, illnesses, or injuries through: a.  assessment of the relevance of the program’s activities to the improvement An of occupational safety and health; and b.  evaluation of the impact that the program’s research has had in reducing An work-related hazardous exposures, illnesses, and injuries.  evaluation committee will rate the performance of the program for its relevance The and impact using an integer score of 1 to 5. Impact may be assessed directly (e.g., reductions in illnesses or injuries) or, as necessary, using intermediate outcomes to estimate impact. Qualitative narrative evaluations will be included to explain the numerical ratings. 2.  Assessment of the program’s effectiveness in targeting new research areas and identifying emerging issues in occupational safety and health most relevant to future improvements in workplace protection. The committee will provide a qualitative narrative assessment of the program’s efforts and suggestions about emerging issues that the program should be prepared to address. In 2008, NIOSH requested that the Committee on the Review of NIOSH Research Pro- grams prepare a brief report to document the lessons learned in this evaluation effort and set forth the process that it recommends be used for future evaluations of occupational health and safety research programs. The report will build on the latest version of the framework used to guide the work of the eight evaluation committees completed to date, and may revise the framework as necessary based on lessons learned about the evaluation process (continued) 

14 E v a l u a t i n g O c c u p a t i o n a l H e a l t h a n d S a f e t y Research Programs BOX 1-1 Continued during these reviews. The committee will draw on the evaluation literature to provide ad- ditional context for the report. Recommendations for future program evaluations or more generalizable guiding principles for these types of efforts may be provided. Input for this report will involve discussions with NIOSH staff and with chairs and committee members of the evaluation committees. OVERVIEW OF NIOSH Created in 1970 by the Occupational and Safety Health Act, NIOSH is charged with the responsibility to “conduct . . . research, experiments, and demonstrations relating to occupational safety and health” and to develop “innovative methods, techniques, and approaches for dealing with occupational safety and health prob- lems” (Public Law 91-596). The focus of NIOSH’s responsibilities is occupational health and safety research, along with professional education and training. NIOSH is also involved in surveillance and in providing advice to the Secretary of Labor regarding standards needed to protect workers as well as other efforts that support research and the transfer of research into the workplace. NIOSH does not have the authority to establish or enforce regulations on workplace safety and health. Regulatory and enforcement authority at the federal level rests largely with the Occupational Safety and Health Administration and the Mine Safety and Health Administration, both in the Department of Labor. NIOSH was established as an agency within the Department of Health and Human Services (HHS). Originally the director of the institute reported to the Secretary of HHS. In the 1980s, NIOSH was reorganized as one of seven compo- nents of the Centers for Disease Control and Prevention. NIOSH offices are located across the country: Washington, DC; Pittsburgh, PA; Spokane, WA; Cincinnati, OH; Atlanta, GA; and Morgantown, WV. Because NIOSH deals with issues spe- cific to particular work sectors (e.g., mining, construction, agriculture) as well as crosscutting issues that span multiple sectors (e.g., hearing loss, personal protective technologies, respiratory diseases), many programs now use a matrix-management approach. (For discussion of the pros and cons of matrix management, see IOM and NRC, 2006.)  NIOSH has responsibility for certifying most types of occupational respirators and for updating the federal respirator certification and testing regulations (42 C.F.R. Part 84). 

Introduction 15 The NIOSH mission is “to provide national and world leadership to prevent work-related illness, injury, disability, and death by gathering information, con- ducting scientific research, and translating the knowledge gained into products and services.” To fulfill its mission, NIOSH has established three strategic goals (NIOSH, 2009a): • Goal 1: Conduct research to reduce work-related illnesses and injuries. C Track work-related hazards, exposures, illnesses, and injuries for prevention. o Generate new knowledge through intramural and extramural research programs. o Develop innovative solutions for difficult-to-solve problems in high- risk industrial sectors. • Goal 2: Promote safe and healthy workplaces through interventions, rec- ommendations, and capacity building. o Enhance the relevance and utility of recommendations and guidance.  o Transfer research findings, technologies, and information into practice.  o Build capacity to address traditional and emerging hazards.  • Goal 3: Enhance global workplace safety and health through international collaborations. o Take a leadership role in developing a global network of occupational  health centers. o Investigate alternative approaches to workplace illness and injury re-  duction and provide technical assistance to implement solutions. o Build global professional capacity to address workplace hazards  through training, information sharing, and research experience. In 1994, NIOSH embarked on a national partnership effort to identify research priorities to guide occupational health and safety research for the next decade (NIOSH, 2009b). Participants included external stakeholders  from many areas, including universities, large and small businesses, professional societies, government agencies, and worker organizations. The result of these efforts was the National Occupational Research Agenda (NORA). Through the NORA effort, 21 priority research areas were identified and research agendas developed. NORA was intended not only for use by NIOSH, but also for the entire occupational health and safety community. In the second decade of NORA, NIOSH continues to work with its partners to update the research agenda using an approach based  Stakeholders are defined as the broad group of individuals or organizations with an interest in the mission of the program. Partners are the subset of stakeholders that contribute directly to program activities. 

16 E v a l u a t i n g O c c u p a t i o n a l H e a l t h a n d S a f e t y Research Programs on industry sectors, with an increased emphasis on moving research results into practice in the workplace. The current NIOSH portfolio is organized into 8 sector programs and 24 crosscutting programs (Box 1-2). DEVELOPING AND IMPLEMENTING THE EVALUATION FRAMEWORK As noted above, before embarking on a series of program evaluations, the Na- tional Academies formed a committee to develop an evaluation framework and to continue over the course of the evaluations to monitor the use of the framework and adapt it as needed to make the evaluations more effective. Initial Framework Document The framework committee appointed by the National Academies consisted of 14 members with expertise in occupational medicine and health, industrial health and safety, industrial hygiene, epidemiology, civil and mining engineering, sociol- ogy, program evaluation, communication, toxicology, international occupational health, and industry and workforce issues. During the initial development of the evaluation criteria and process, the framework committee drew from the program evaluation literature and discussed evaluation strategies and precedents with NIOSH leadership and staff, industry leaders, RAND Corporation staff, occupational safety organizations, labor repre- sentatives, and staff from other federal agencies (see Appendix A). The framework committee decided to use the logic model as the basis for the evaluation framework. The logic model organizes the program and its efforts into inputs (e.g., budget, staffing, facilities), activities (e.g., research studies, surveillance, exposure measurement), outputs (e.g., reports, conferences, training, patents), and outcomes (e.g., collaborations, policy changes, reductions in injuries and hazard- ous exposures) (see Box 1-3 for definitions and additional examples). As will be further discussed in Chapter 2, logic models are widely used in program evaluation and planning. To provide guidance on evaluating and scoring the relevance and impact of each NIOSH program, the framework committee developed criteria and specific questions to be used in the assessment of each component of the logic model. Assessment of strategic goals and objectives, inputs, activities, and outputs largely  NIOSH contracted with RAND Corporation for the development of the logic models spe- cific to each program and assistance on identifying and assembling the materials needed for the evaluations. 

Introduction 17 BOX 1-2 NIOSH Program Portfolio The National Institute for Occupational Safety and Health (NIOSH) Program Portfolio has been organized into 8 National Occupational Research Agenda Sector Programs that represent industrial sectors, and 24 cross-sector programs. Sector Programs: • Agriculture, Forestry, and Fishing • Construction • Health Care and Social Assistance • Manufacturing • Mining; Oil and Gas Extraction Subsector • Services • Transportation, Warehousing, and Utilities • Wholesale and Retail Trade Cross-Sector Programs: • Authoritative Recommendations • Cancer, Reproductive and Cardiovascular Diseases • Communications and Information Dissemination • Economics • Emergency Preparedness and Response • Engineering Controls • Exposure Assessment • Global Collaborations • Health Hazard Evaluation • Hearing Loss Prevention • Immune and Dermal Diseases • Musculoskeletal Disorders • Nanotechnology • Occupational Health Disparities • Personal Protective Technology • Prevention Through Design • Radiation Dose Reconstruction • Respiratory Diseases • Small Business Assistance and Outreach • Surveillance • Training Grants • Traumatic Injury • Work Organization and Stress-Related Disorders • Worklife Initiative SOURCE: NIOSH (2009c). 

18 E v a l u a t i n g O c c u p a t i o n a l H e a l t h a n d S a f e t y Research Programs BOX 1-3 Logic Model Terms and Examples Planning Inputs: Stakeholder input, surveillance and intervention data, and risk assess- ments (e.g., input from Federal Advisory Committee Act panels or the National Occupational Research Agenda research partners, intramural surveillance information, Health Hazard Evaluations [HHEs]). Production Inputs: Intramural and extramural funding, staffing, management structure, and physical facilities. Activities: Efforts and work of the program, staff, grantees, and contractors (e.g., surveil- lance, health effects research, intervention research, health services research, information dissemination, training, technical assistance). Outputs: Direct products of NIOSH programs that are logically related to the achievement of desirable and intended outcomes (e.g., publications in peer-reviewed journals, recom- mendations, reports, website content, workshops and presentations, databases, educational materials, new technologies, patents, technical assistance). Intermediate Outcomes: Actions by stakeholders in response to NIOSH products or efforts (e.g., policy changes; production of standards or regulations; adoption of NIOSH-developed technologies; use of publications, technologies, methods, or recommendations by workers, industry, and occupational safety and health professionals in the field; citations of NIOSH research by industry and academic scientists). End Outcomes: Improvements in safety and health in the workplace—specifically, reductions in work-related injuries, illnesses, or deaths or reductions in hazardous exposures in the workplace—that can be attributed to NIOSH efforts. External Factors: Actions or forces beyond NIOSH’s control (e.g., by industry, labor, regula- tors, and other entities) with important bearing on moving research results into practice in the workplace. SOURCE: Adapted from IOM and NRC (2006). defined the relevance of the program; the committee examined the adequacy of the inputs and the scope and targeting of the activities and outputs in achieving the program’s goals. Assessment of the intermediate and end outcomes largely defined the program’s impact. Overlap necessarily occurred between the assessment of relevance and impact, particularly in the assessment of information transfer. 

Introduction 19 The framework committee met three times to develop the first version of the evaluation framework, which was released in December 2005. This version of the framework was then used by the evaluation committees examining the first four NIOSH programs to be considered: Hearing Loss Prevention; Mining; Agriculture, Forestry, and Fishing; and Respiratory Diseases. Issues in Developing and Implementing the Evaluation Framework The variations in the types of NIOSH programs undergoing evaluation raised several issues during the framework committee’s work and in the course of the initial set of program evaluations. These issues are further discussed in Chapter 4 and reflect the discussions among NIOSH staff and committee members at the November 2008 workshop held after the evaluations were completed. Primary issues included the following: • Differences in program mission, including whether a program is dedi- cated to multiple health and safety issues within a single industry sector (e.g., mining) or focuses on a set of health outcomes (e.g., hearing loss) in whichever industrial sector they occur (a cross-sector program); • Differences in program structure or management. Programs may use varying management styles and structures, with some programs using a matrix-management approach that coordinates across NIOSH divisions and facilities and others that are located within a single NIOSH division using a more traditional management structure; • Differences in program size; • Differences in program and subprogram maturity. An older program or subprogram may have better developed strategic goals and measurable outcomes; • Overlap among programs; • The weighting of different activities within and across study areas of a single program; • The influence of external factors on the implementation, translation, and impact of NIOSH’s efforts; • The definition and difference between intermediate and end outcomes; • Limitations in the availability of quantitative information (e.g., surveil- lance data), particularly related to priority setting and documentation of intermediate and end outcomes; and • The quantitative rating scales. The framework committee was concerned about developing scoring criteria that would maximize consistency across 

20 E v a l u a t i n g O c c u p a t i o n a l H e a l t h a n d S a f e t y Research Programs the reports. Many evaluation committee members were initially concerned that the required task of specifying scores for relevance and for impact on 5-point integer scales was too quantitative for a process that, by nature, was largely qualitative. An additional concern discussed by the framework committee and the evalu- ation committees was the challenge of evaluating and scoring a federal program within the broader context of agency evaluations being conducted by the Office of Management and Budget using the Program Assessment Rating Tool (PART). Concerns about how the evaluations would be used and their potential impact on the programs’ budgets were voiced. NIOSH senior management staff acknowledged the PART evaluation requirements, but also emphasized that NIOSH’s reason for conducting thorough and independent external evaluations was focused on quality improvement. The evaluation and framework committees kept both goals in mind and focused on conducting a fair and balanced evaluation. Revisions to the Evaluation Framework To monitor and refine the evaluation framework over the course of the evalu- ations, the framework committee continued to meet once or twice annually from 2006 through 2009 (see Appendix A). In 2006 and 2007, the framework committee discussed the first set of ongoing and recently completed evaluations with evalua- tion committee chairs and members, NIOSH staff, National Academies’ staff, and stakeholder groups. They explored the applicability of the framework document, the usefulness of the initial sets of materials provided by NIOSH (the program evi- dence packages), and issues associated with the quantitative ratings and the ratings criteria. Issues associated with the evaluation of program quality in the evaluation framework were also discussed. In response, the framework committee released a revised framework in August 2007 to improve clarity, accommodate the revised statement of task as negotiated with NIOSH, and make the document more usable to the evaluation committees. Changes included reorganization of the document to more closely follow the re- vised statement of task and to improve readability. Slight modifications were made to the criteria for scoring impact and relevance that made the wording more precise. Additions were made in the form of guidance to the evaluation committees regard- ing scoring a program for relevance based on the committee’s observed levels of “research priority” and “engagement in appropriate transfer activities.” The four evaluation committees examining the NIOSH Personal Protective Tech- nology, Traumatic Injury, Construction, and Health Hazard Evaluation programs used the August 2007 version of the framework. 

Introduction 21 As described below, the framework committee continued to receive input from the individual evaluation committees and followed their progress. The framework committee met in 2008 and 2009 with the goals of (1) convening a workshop on lessons learned in the evaluation process, and (2) compiling this report, which includes a further revised framework (version 2009) and recommendations for future evaluations. Evaluation Committees As of the publication date of this report, the evaluation of eight NIOSH pro- grams has been completed (IOM and NRC, 2006, 2008, 2009; NRC and IOM, 2007, 2008a,b, 2009a,b). Table 1-1 lists the programs evaluated, and summarizes the activities of the evaluation committees, the version of the framework document applied, and the dates of report release. Framework committee members served as evaluation committee members, liaisons, and report reviewers. Each evaluation committee met three to four times. In addition to meetings, some committees conducted site visits of research program facilities. The NIOSH program provided each committee with an evidence package prior to its first meeting. Additional information was acquired through site visits; input from external stakeholders through open sessions at committee meetings, online questionnaires, and writ- ten responses; and oral and written responses to questions posed to the NIOSH programs. Throughout the evaluation process, evaluation committee members provided information to the framework committee on the successes and limitations of the framework in conducting their specific program evaluation. Process for Developing This Report Subsequent to the eight evaluations, the framework committee reconvened for two meetings to discuss the lessons learned during the process and to revise the evaluation framework accordingly. This report is the result of those discussions and revisions. The report draws on the experience of those who participated in develop- ing the framework and those who used the framework to conduct an evaluation. Additionally, the framework committee held a workshop on November 24, 2008, that focused on lessons learned in the evaluation efforts (see Appendix A). NIOSH staff members, evaluation committee members and chairs, framework committee members, and IOM and NRC staff members participated in the workshop and shared their perspectives on lessons learned and their ideas for future evaluations. Committee chairs and NIOSH staff members had an additional opportunity for input after the workshop. 

22 E v a l u a t i n g O c c u p a t i o n a l H e a l t h a n d S a f e t y Research Programs TABLE 1-1  Evaluation Committee Timelines Evaluation Framework Program Name Used Meeting Datesa Report Release Hearing Loss 2005 January 5–6, 2006 August 2006 February 23–24, 2006 March 30–31, 2006 Mining Safety 2005 January 12–13, 2006 May 2007 and Health February 21–22, 2006 May 9–10, 2006 Agriculture, 2005 January 18–20, 2007 December 2007 Forestry, and March 28–29, 2007 Fishing May 30–31, 2007 Respiratory 2005 October 26–27, 2006 March 2008 Diseases December 5–6, 2006 March 22–23, 2007 Personal 2007b September 27–28, 2007 June 2008 Protective December 17–18, 2007 Technology March 6–7, 2008 (PPT) Traumatic 2007 March 29–30, 2007 August 2008 Injury May 31–June 1, 2007 September 6–7, 2007 Construction 2007 July 17–18, 2007 October 2008 September 25–27, 2007 December 10–12, 2007 Health Hazard 2007b October 18–19, 2007 October 2008 Evaluation December 10–11, 2007 (HHE) January 15–16, 2008 February 21–22, 2008 aThe list of meeting dates does not include site visits or committee conference calls. bThe PPT and HHE studies used slightly adapted statements of task because of the certification or investigative work of these programs. Therefore the committees had to slightly modify the application of the evaluation framework. 

Introduction 23 NIOSH Follow-Up on the Evaluation Reports During the time that the framework committee examined the evaluation pro- cess, NIOSH staff members have been responding to each of the evaluation reports. NIOSH program staff members were tasked with developing an action plan that addresses the evaluation report findings and recommendations. The action plans are presented to NIOSH senior management and then to the NIOSH Board of Scientific Counselors (BSC). The NIOSH BSC has committed to be actively in- volved in reviewing the action plans and to write a short assessment of each action plan. Additionally, the BSC is examining the recommendations from all eight of the reports to identify crosscutting issues (e.g., need for improved surveillance). Addressing the issues and strategies identified through this process might provide the greatest efficiency and effectiveness in continuing to improve the relevance and impact of the NIOSH programs. OVERVIEW OF THIS REPORT Chapter 2 of this report sets the National Academies’ approach to evaluating the NIOSH programs in the larger context of program evaluation. Chapter 3 pro- vides the revised evaluation framework. Because the framework has been integrated into this report (as opposed to previous versions that were stand-alone documents and appendixes to the evaluation reports), a number of the changes involved plac- ing introductory material in Chapter 1. Other changes were more substantive and were the result of the framework committee’s careful considerations of the lessons learned, which are detailed in Chapter 4. The concluding chapter provides the committee’s recommendations for NIOSH to consider in planning future evalua- tions. The committee believes the evaluation framework is sufficiently robust and the lessons learned are sufficiently generalizable to offer insights to other federal agencies as they consider program evaluation. REFERENCES BLS (Bureau of Labor Statistics). 2008. Workplace illnesses and injuries in 2007. http://www.bls.gov/ iif/oshwc/osh/os/osnr0030.pdf (accessed April 10, 2009). BLS. 2009. Fatal occupational injuries by industry and event or exposure. http://www.bls.gov/iif/oshwc/ cfoi/cftb0223.pdf (accessed April 10, 2009). IOM and NRC (Institute of Medicine and National Research Council). 2006. Hearing loss research at NIOSH. Committee to Review the NIOSH Hearing Loss Research Program. Rpt. No. 1, Reviews of Research Programs of the National Institute for Occupational Safety and Health. Washington, DC: The National Academies Press. 

24 E v a l u a t i n g O c c u p a t i o n a l H e a l t h a n d S a f e t y Research Programs IOM and NRC. 2008. The personal protective technology program at NIOSH. Committee to Review the NIOSH Personal Protective Technology Program. Rpt. No. 5, Reviews of Research Programs of the National Institute for Occupational Safety and Health. Washington, DC: The National Academies Press. IOM and NRC. 2009. Traumatic injury research at NIOSH. Committee to Review the NIOSH Trau- matic Injury Research Program. Rpt. No. 6, Reviews of Research Programs of the National Institute for Occupational Safety and Health. Washington, DC: The National Academies Press. Leigh, P. S. 2008. Cost of occupational injury and illness combining all industries. Presentation at the November 2008 Seminar for Western Center for Agriculture Health and Safety. http://agcenter. uc davis.edu/seminar/flyer/2009/Leigh_Nov3_2008.ppt#256 (accessed March 26, 2009). NIOSH (National Institute for Occupational Safety and Health). 2009a. NIOSH strategic plan outline, 2004–2009. http://www.cdc.gov/niosh /docs/strategic (accessed March 19, 2009). NIOSH. 2009b. NORA. http://www.cdc.gov/niosh/nora/ (accessed March 17, 2009). NIOSH. 2009c. NIOSH program portfolio. http://www.cdc.gov/niosh/programs/ (accessed March 17, 2009). NRC and IOM. 2007. Mining safety and health research at NIOSH. Committee to Review the NIOSH Mining Safety and Health Research Program. Rpt. No. 2, Reviews of Research Programs of the National Institute for Occupational Safety and Health. Washington, DC: The National Acad- emies Press. NRC and IOM. 2008a. Agriculture, forestry, and fishing research at NIOSH. Committee to Review the NIOSH Agriculture, Forestry, and Fishing Research Program. Rpt. No. 3, Reviews of Research Programs of the National Institute for Occupational Safety and Health. Washington, DC: The National Academies Press. NRC and IOM. 2008b. Respiratory diseases research at NIOSH. Committee to Review the NIOSH Re- spiratory Diseases Research Program. Rpt. No. 4, Reviews of Research Programs of the National Institute for Occupational Safety and Health. Washington, DC: The National Academies Press. NRC and IOM. 2009a. The health hazard evaluation program at NIOSH. Committee to Review the NIOSH Health Hazard Evaluation Program. Rpt. No. 7, Reviews of Research Programs of the National Institute for Occupational Safety and Health. Washington, DC: The National Acad- emies Press. NRC and IOM. 2009b. Construction research at NIOSH. Committee to Review the NIOSH Construc- tion Research Program. Rpt. No. 8, Reviews of Research Programs of the National Institute for Occupational Safety and Health. Washington, DC: The National Academies Press.