Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 15
Respiratory Diseases Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health 1 Introduction The National Institute for Occupational Safety and Health (NIOSH) was established by the Occupational Safety and Health Act of 1970 to “conduct … research, experiments, and demonstrations relating to occupational safety and health” and to develop “innovative methods, techniques, and approaches for dealing with [those] problems” (Public Law 91-596, 84 STAT. 1590, 91st Congress, S.2193, December 29, 1970). NIOSH, part of the Centers for Disease Control and Prevention within the Department of Health and Human Services, is authorized to conduct research, training, and education related to worker health and safety; perform on-site investigations to investigate hazards in the workplace; recommend occupational health and safety standards; and fund research by other agencies or private organizations. NIOSH has the legislative responsibility to develop the research base upon which it can then recommend occupational health and safety standards to OSHA, although it does not have the authority to promulgate binding standards or enforce regulations on workplace safety and health. Federal regulatory and enforcement authority for occupational safety and health rests with the Occupational Safety and Health Administration (OSHA) and the Mine Safety and Health Administration (NIOSH 2006a). The organizational configuration of NIOSH is shown in Figure 1-1. Preventing occupational respiratory disease has been a key part of the NIOSH research portfolio since the agency’s inception in 1970. Early on, NIOSH assumed responsibilities for health screening and research related to coal workers’ pneumoconiosis (CWP) that initially was mandated under the Federal Coal Mine Health and Safety Act of 1969. Respiratory disease research and surveillance related to min-
OCR for page 16
Respiratory Diseases Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health FIGURE 1-1 NIOSH organization chart, as of October 2006. Source: NIOSH 2006a.
OCR for page 17
Respiratory Diseases Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health ing activities remain major NIOSH activities. However, current work-related respiratory diseases and disorders that NIOSH is studying include a broad spectrum of other adverse health effects that can arise in a wide range of occupational settings. Such adverse health effects can range from mild, reversible conditions to progressive, fatal disorders and can be linked to short-term or long-term exposures. A major focus of NIOSH’s activities both historically and currently has been on occupational respiratory diseases. Deaths from work-related respiratory diseases and malignancies have been estimated to account for about 70% of all occupational disease mortality (Steenland et al. 2003). The American Thoracic Society stated that “A careful review of the literature demonstrates that approximately 15% of both asthma and COPD is likely to be work related” (Balmes et al. 2003). It has been estimated that this occupational asthma and chronic obstructive pulmonary disease (COPD) costs nearly $7 billion annually in the United States (Leigh et al. 2002). In 2000, there were an estimated 386,000 deaths worldwide from asthma, COPD, and pneumoconiosis and nearly 6.6 million disability-adjusted life years due to occupational exposure to airborne particulates (Driscoll et al. 2005a). As work-related diseases have been identified and safety measures taken to reduce the associated risks, the overall patterns of occupational respiratory diseases have changed. NIOSH data through 1999 indicate that morbidity and mortality from CWP declined appreciably over the preceding 20 years (NIOSH 2003). More recent data, however, indicate that the trend has slowed and even reversed, making this area one of active NIOSH investigation (NIOSH 2006a; data from E. L. Petsonk, NIOSH, as cited by Ward 2007). In the same time period, mortality due to silicosis has declined from well over 1,000 deaths annually in the late 1960s to fewer than 200 per year in the late 1990s (NIOSH 2003). Nonetheless, as with CWP “hot spots” (MMWR 2006a, 2007a), NIOSH has continued to report important clusters of silica-related disease (MMWR 1990, 2004). The identification of work-related outbreaks of respiratory disease—such as those related to artificial butter flavorings, respirable particles of nylon flock, and leather conditioning sprays—and of CWP hot spots point to the changing nature of occupational respiratory disease, with new issues arising from classic exposures as well as new diseases being detected from novel exposures. Additional challenges for NIOSH include the need to protect workers from potential occupational lung diseases due to exposures during national security emergencies, the potential for occupational exposures to weaponized biological agents, the emergence of naturally occurring infectious diseases, and the increased production and incorporation of nanomaterials that pose unknown risks to workers.
OCR for page 18
Respiratory Diseases Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health OVERVIEW OF THE RESPIRATORY DISEASES RESEARCH PROGRAM The formally defined Respiratory Diseases Research Program (RDRP) is a recent creation. As described in the evidence package, the RDRP is an “organizational component” that was designated in 2005 as a result of matrix management efforts intended to coordinate cross-institute programmatic activities as a result of the second National Occupational Research Agenda (NORA2) process. NIOSH describes the RDRP as “the range of individuals and groups supported by NIOSH to do work that is relevant to occupational respiratory disease” (NIOSH 2006a). As such, the program is intended to be a vehicle to describe the broad range of research on respiratory disease that occurs at NIOSH. For example, the RDRP is not an institute or physical location where the research occurs nor does it constitute a cohesive staff grouping under a single hierarchical reporting structure. The RDRP includes the multiple divisions and laboratories within NIOSH that deal with respiratory disease issues. For the sake of clarity, throughout this report the committee uses the term RDRP to refer to activities that occurred before as well as after establishment of the RDRP in 2005, even though it is recognized that the designation, as such, did not exist in the earlier time frame. History of the Program Originating Legislation and Facilities Table 1-1 shows some of the administrative developments important for development of the RDRP. The Federal Coal Mine Health and Safety Act of 1969 prescribed a number of research, surveillance, and regulatory-related activities that were later assigned to NIOSH when it was formed by the Occupational Health and Safety Act of 1970. Adoption of the Mine Safety and Health Act of 1977 also had an impact on the RDRP mission. Because of the location of NIOSH coal mining research facilities—first in Beckley, West Virginia, and later in Morgantown, West Virginia—initial RDRP research focused almost entirely on CWP. Beginning in 1976, with the formation of the Division of Respiratory Disease Studies (DRDS), the scope of research on respiratory disease based in Morgantown expanded significantly to other forms of pneumoconiosis that included silicosis, beryllium disease, asbestosis, and other fibrotic lung diseases, and a focus on organic dusts and other respiratory irritants that result in occupational asthma, airway obstruction, and hypersensitivity pneumonitis. Other elements of NIOSH that have had a role in respiratory disease activities are located in NIOSH facilities in Cincinnati, Ohio. These facilities include the Division of Applied Research and Technology, with expertise in exposure assessment, development of analytical methods, and control technologies; the Education
OCR for page 19
Respiratory Diseases Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health TABLE 1-1 Timeline of Administrative Developments Related to NIOSH Respiratory Diseases Research Year Administrative Development 1969 Federal Coal Mine Health and Safety Act 1970 Occupational Health and Safety Act Several RDRP components established in Cincinnati, Ohio. 1971 Appalachian Laboratory for Occupational Safety and Health (ALOSH) opens in Morgantown, West Virginia 1976 Division of Respiratory Disease Studies opens within ALOSH 1977 Federal Mine Safety and Health Act of 1977 1996 Health Effects Laboratory Division founded 1996 Pittsburgh Research Laboratory and Spokane Research Laboratory added to NIOSH 1996 Education and Information Division created 2000 Division of Applied Research and Technology formed from the Division of Physical Sciences and Engineering and the Division of Biological and Behavioral Sciences 2001 National Personal Protective Technology Laboratory founded 2006 Respiratory Diseases Research Program founded Source: NIOSH 2006a. and Information Division, responsible for health communication and coordinating development of recommendation documents; and the Division of Surveillance, Hazard Evaluations and Field Studies (DSHEFS), responsible for conducting short-and long-term field studies to evaluate health and safety issues including respiratory diseases. DSHEFS is the principal division that handles health hazard evaluations, although the DRDS in Morgantown has also been active in respiratory-disease-related health hazard evaluations. Other facilities have performed activities related to respiratory disease. One of them is the Health Effects Laboratory Division, which focuses on basic bench laboratory research, including research in basic toxicology and in engineering and exposure assessment. The Pittsburgh Research Laboratory and the Spokane Research Laboratory are mining-focused research groups concerned with research on mining engineering control technology for respiratory hazards. Most recently, the National Personal Protective Technology Laboratory in Bruceton, Pennsylvania, has responsibility for research on respiratory protection and certification of respirators (NIOSH 2006a). Strategic Planning Occupational lung diseases have long been a top priority for NIOSH and were the subject of a proposed national strategy in 1986 (NIOSH 1986a). In this
OCR for page 20
Respiratory Diseases Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health strategy, asbestosis, byssinosis (associated with cotton dust), silicosis, and CWP were cited as specific examples for action. The proposed strategy included the following elements: environmental hazard surveillance, medical hazard surveillance, hazard removal, control technologies, regulatory enforcement, worker education and training (includes respirators and other personal protective devices), and worker-oriented programs that include health promotion and smoking prevention (NIOSH 1986a). In 1996, NIOSH worked with the occupation safety and health community to develop a National Occupational Research Agenda (NORA) (NIOSH 1996a). Seen as a way to better guide and organize research activities in a fiscally constrained environment, NORA identified 21 research priorities grouped into three categories: disease and injury, work environment and workforce, and research tools and approaches (Table 1-2). Under NORA, there were three priorities related to respiratory disease under the disease and injury category: allergic and irritant dermatitis,1 asthma and COPD, and infectious disease. NIOSH’s objectives in creating the NORA approach were to Guide intramural and extramural funding decisions. Encourage and stimulate other government agencies to include NORA priorities in their internal and external research programs. Develop procedures and capacity to track the impact of NORA activities on health and safety outcomes using existing tracking models, if available. Provide for timely updates to the NORA priorities. Periodically review and communicate the overall role and effectiveness of NORA in occupational safety and health. During development of the NORA agenda, the importance of industrial sector-specific research (for example, construction or agriculture) was consistently raised (NIOSH 1996a). It was decided that the most effective way to integrate this research was through a matrix approach of coordinated research in some or all of the 21 priority areas, as appropriate for each sector. An example of this approach is provided in Table 1-3. Beginning in 2005, NIOSH introduced the second phase of the NORA process (NORA2), now organizing research priorities focused on eight key industry sectors. The sectors, listed in Table 1-4, are agriculture, forestry, and fishing; construction; health care and social assistance; manufacturing; mining; services; wholesale and retail trade; and transportation, warehousing, and utilities. Sector alignment is intended to provide direct occupational health and safety research and assistance 1 NIOSH groups allergic and irritant dermatitis in this classification.
OCR for page 21
Respiratory Diseases Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health TABLE 1-2 NORA Priority Research Areas Disease and Injury Allergic and irritant dermatitisa Asthma and chronic obstructive pulmonary diseasea Fertility and pregnancy abnormalities Hearing loss Infectious diseasesa Low back disorders Musculoskeletal disorders of the upper extremities Traumatic injuries Work Environment and Workforce Emerging technologies Indoor environment Mixed exposures Organization of work Special populations at risk Research Tools and Approaches Cancer research methods Control technology and personal protective equipment Exposure assessment methods Health services research Intervention effectiveness research Risk assessment methods Social and economic consequences of workplace illness and injury Surveillance research methods aRespiratory disease focus. Source: NIOSH 1996a. TABLE 1-3 Example of NORA’s Matrix Approach to Coordinating Research Sector Allergic and Irritant Asthma and Chronic Obstructive Fertility and Pregnancy Hearing Loss Agriculture Xa X X X Construction X X X Service X X X Mining X X X Manufacturing X X X X Xa = priority research area within a sector. Source: NIOSH 1996a.
OCR for page 22
Respiratory Diseases Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health TABLE 1-4 NORA2 Structure NORA Sector Programs Agriculture, forestry, and fishing Construction Health care and social assistance Manufacturing Mining Services Wholesale and retail trade Transportation, warehousing, and utilities NIOSH Cross-Sector Programs Authoritative recommendations Cancer, reproductive, and cardiovascular Communications and information dissemination Emergency preparedness/response Global collaborations Health hazard evaluation Hearing loss prevention Immune and dermal Musculoskeletal disorders Personal protective technology Radiation dose reconstruction Respiratory diseases Training grants Traumatic injury Work organization and stress-related disorders NIOSH Coordinated Emphasis Areas Economics Exposure assessment Engineering controls Work life initiative Occupational health disparities Small business assistance and outreach Surveillance to address specific issues as identified within each sector. Additionally, NORA2 identified 15 cross-sector programs and 7 coordinated emphasis areas, also shown in Table 1-4. The RDRP is one of the cross-sector programs. Each cross-sector program has a steering committee made up of staff from all NIOSH divisions and laboratories as well as the Office of the Director and the Office of Extramural Programs. According to NIOSH (2006a), the steering committee annually reviews
OCR for page 23
Respiratory Diseases Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health and ranks competitive intramural funding requests, reviews all in-house projects relevant to its program, and provides feedback to divisions about each project’s relevance and level of priority. Respiratory Diseases Research Program Structure As shown in Figure 1-2, the RDRP, as conceptualized, spans nearly all NIOSH units. In 2006, five divisions, three laboratories, and three offices were involved in the RDRP. The mission statement for the RDRP (Weissman 2006) is To provide national and international leadership for the prevention of work-related respiratory diseases, using a scientific approach to gather and synthesize information, create knowledge, provide recommendations, and deliver products and services to those who can effect prevention. FIGURE 1-2 Respiratory Diseases Research Program projects throughout NIOSH (does not include extramural, communications, research, and technology transfer offices). Source: NIOSH 2006a.
OCR for page 24
Respiratory Diseases Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health Within this overall mission five specific goals have been stated: Prevent and reduce work-related airways diseases. Prevent and reduce work-related interstitial lung diseases. Prevent and reduce work-related respiratory infectious diseases. Prevent and reduce work-related respiratory malignancies. Prevent respiratory and other diseases potentially resulting from occupational exposures to nanomaterials. To pursue these goals, the RDRP activities include multidisciplinary approaches to respiratory diseases through laboratory studies and short- and long-term field studies; surveillance and reporting on disease frequencies; assessment of control technologies and respiratory protection; communication, education, and training; and recommendations on exposure and other practices. Facilities and equipment exist for biomedical research, development of analytical methods, research on exposure assessment, research on engineering and industrial hygiene, respirator research, mining research, and epidemiologic investigations. Ultimately, NIOSH provides the scientific underpinning for development of new or revision of existing standards by OSHA. NIOSH’s Draft Fiscal Year 2008 Program Planning Guide, which describes the process for acquiring competitively awarded intramural research funds at NIOSH, outlines RDRP’s strategic and intermediate goals and their performance measures. The document has a high level of specificity, with measurable goals and target time frames. This document became available at the end of the committee’s deliberations; thus, the committee has not examined whether they are the correct performance measures or whether they are realistic. The committee notes, however, that because the RDRP is an “organizational unit” borne of the new matrix management approach that spans institutes and divisions, it is unclear whether funding and support are distributed based on whether a project satisfies RDRP’s strategic goals or whether RDRP’s goals do not matter because funding is distributed based on the goals of the divisions or institutes. Being “at the table” does not provide assurance to the RDRP or to the committee that appropriate weight is given to funding priorities. According to the evidence package, RDRP’s steering committee, with representatives from all divisions, “has been empowered to mold activities in respiratory diseases through its abilities to review and rank competitive intramural funding requests and through its access to the NIOSH Office of the Director.” (The committee did not attend or review minutes from steering committee meetings.) It is not clear what formal mechanisms exist to ensure that RDRP’s goals are supported by funding.
OCR for page 25
Respiratory Diseases Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health TABLE 1-5 NIOSH RDRP Funding by Fiscal Year, 1996-2005 Year NIOSH Funding, Millions of Dollars Intramural Extramural Total 1996 7.4 2.3 9.7 1997 7.1 3.1 10.2 1998 10.9 4.5 15.4 1999 13.2 7.3 20.5 2000 15.5 7.3 22.8 2001 16.2 8.6 24.8 2002 16.1 10.1 26.2 2003 16.4 10.3 26.7 2004 16.3 8.6 24.9 2005 17.2 11.4 28.6 Totals 136.3 73.5 209.8 Source: Ray Sinclair, NIOSH, unpublished material, April 07, 2007. Table 2-2 shows additional data for extramural funding through 2006, resulting in a total for extramural funding of $88 million for the period 1996-2006. Program Resources RDRP funding for the most recent fiscal year available (2005) was close to $29 million. Table 1-5 shows the funding levels for the RDRP intramural and extramural funded research for the years 1996-2005. Intramural funding supports staff salaries and benefits as well as goods and services related to staff research activities. Extramural activities are administered and funded through the NIOSH Office of Extramural Programs in Atlanta (greater detail, including by strategic goal, is provided in Chapter 2). Table 1-6 compares the fraction of the budget spent on intramural versus extramural research for the RDRP with several institutes or centers of the National Institutes of Health (NIH) and shows that the RDRP tends to spend about two-thirds of its budget on intramural projects. This is in contrast to research funding at NIH agencies, including the National Institute of Environmental Health Sciences, that tend to spend most of their research funding on extramural research. Given the relatively small research budget compared with other institutes whose research budgets are typically one to two orders of magnitude higher,2 the committee considered the limited support allocated to extramural programs unfortunate 2 For example, the fiscal year 2003 research budgets for NIEHS and NHLBI were approximately $500 million and $2.4 billion, respectively (DSA Pubdata, http://grants.nih.gov/grants/award/trends/icfund9803.html). In contrast, the 2003 research budget for RDRP was approximately $27 million (see Table 1-5).
OCR for page 26
Respiratory Diseases Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health TABLE 1-6 NIH Versus NIOSH Extramural Funding Funding Source Extramurala Percent of Annual Research Budget 2003 2002 2001 2000 1999 1998 National Center for Complementary and Alternative Medicine 82 78 77 79 — — National Cancer Institute 77 75 74 75 75 74 National Center on Minority Health and Health Disparities 55 43 0 — — — National Eye Institute 83 84 87 88 88 87 National Heart, Lung, and Blood Institute 87 87 86 87 86 88 National Institute on Aging 83 83 85 84 84 84 National Institute on Alcohol Abuse and Alcoholism 81 82 84 85 86 85 National Institute of Allergy and Infectious Diseases 79 83 85 87 84 86 National Institute of Arthritis and Musculoskeletal and Skin Diseases 87 87 87 90 90 89 National Institute of Child Health and Human Development 82 84 85 84 83 82 National Institute on Drug Abuse 84 84 86 88 88 88 National Institute on Deafness and Other Communication Disorders 84 85 85 86 87 87 National Institute of Dental and Craniofacial Research 75 74 74 74 75 76 National Institute of Diabetes and Digestive and Kidney Diseases 84 85 85 87 86 85 National Institute of Environmental Health Sciences 71 71 72 79 83 84 National Institute of Mental Health 83 83 84 84 83 82 National Institute of Neurological Disorders and Stroke 85 86 86 86 85 85 National Institute of Nursing Research 92 93 95 93 98 95 National Institute for Occupational Safety and Health Respiratory Diseases Research Program 39 39 35 32 36 29 aExtramural includes research grants, research and development, training grants, and fellowships. Source: Ray Sinclair, NIOSH, unpublished material, April 07, 2007, for NIOSH RDRP; DSA Pubdata (http://grants.nih.gov/grants/award/trends/icfund9803.html) for others. but probably necessary. Because the committee was not asked to review the extramural program, the committee is not in a position to evaluate the balance between extramural and intramural funding. However, as Table 1-5 shows, the proportion of extramural funding has generally increased with time, from 24% in 1996 to 40% in 2005, the latest year for which complete data are available.
OCR for page 27
Respiratory Diseases Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health STUDY CHARGE AND EVALUATION COMMITTEE In September 2004, NIOSH requested that the National Academies review various NIOSH programs with respect to the impact and relevance of their work in reducing workplace injury and illness and to identify future directions that their work might take. The Committee to Review the NIOSH Research Programs was established as an oversight committee and created a framework document, which will be used to evaluate individual programs (Appendix A) (NAS 2005). The RDRP was selected as one of the programs to undergo such a review. The National Research Council (NRC) convened the Committee to Review the NIOSH Respiratory Diseases Research Program in late 2006. The statement of task for the committee is as follows: In response to a request from the National Institute for Occupational Safety and Health (NIOSH), the Institute of Medicine (IOM) and the Division of Earth and Life Studies (DELS) of the National Academies (NA) are conducting a series of evaluations of NIOSH research programs. Each evaluation will be conducted by an ad hoc committee, using a methodology and framework developed by the Committee to Review NIOSH Research Programs (framework committee). Each evaluation committee will review the program’s impact, relevance, and future directions. The evaluation committee will evaluate not only what the NIOSH research program is producing, but will also determine whether it is appropriate to credit NIOSH research with changes in workplace practices, hazardous exposures, and/or occupational illnesses and injuries, or whether the changes are the result of other factors unrelated to NIOSH. The program reviews should focus on evaluating the program’s impact and relevance to health and safety issues in the workplace and make recommendations for improvement. In conducting the review, the evaluation committee will address the following elements: Assessment of the program’s contribution through occupational safety and health research to reductions in workplace hazardous exposures, illnesses, or injuries through: An assessment of the relevance of the program’s activities to the improvement of occupational safety and health, and An evaluation of the impact that the program’s research has had in reducing work-related hazardous exposures, illnesses, and injuries.
OCR for page 28
Respiratory Diseases Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health The evaluation committee will rate the performance of the program for its relevance and impact using a scale of 1 to 5. Impact may be assessed directly (e.g., reductions in illnesses or injuries) or, as necessary, using intermediate outcomes to estimate impact. Qualitative narrative evaluations should be included to explain the numerical ratings. Assessment of the program’s effectiveness in targeting new research areas and identifying emerging issues in occupational safety and health most relevant to future improvements in workplace protection. The committee will provide a qualitative narrative assessment of the program’s efforts and suggestions about emerging issues that the program should be prepared to address. The study committee was selected to include members with expertise in epidemiology, exposure assessment, industrial hygiene, inhalation toxicology, occupational medicine, and pulmonology. The committee reviewed the RDRP emphasizing, although not exclusively limited to, the time period since 1996, as this time frame encompasses both the original NORA and the recently completed NORA2. However, the committee is cognizant that improvements in workers’ health with regard to respiratory diseases that occurred during this period may be a result of NIOSH research activities that were completed much earlier. The committee met three times in the period October 2006 through March 2007. The first two meetings were data-gathering sessions that included presentations by NIOSH staff and other invited speakers in open session. At the end of each open session stakeholders and the general public had an opportunity to comment. In addition, several committee members and a member of the NRC staff attended a site visit at the NIOSH Morgantown, Cincinnati, and Pittsburgh facilities. The committee’s review of the NIOSH RDRP was based in part on materials provided by NIOSH at the onset and on specific materials the committee requested during the review (see Appendix C for a list of these materials). The committee wanted to hear from a broad range of stakeholders and created an online questionnaire for them to provide comments (see Appendix B). Presentations and discussions during the open session as well as the online questionnaire helped in shaping additional questions and giving background to areas under review. EVALUATION APPROACH Framework Document and Logic Model The committee evaluated the relevance of RDRP research to improvements in occupational safety and health and the impact that NIOSH research has had in
OCR for page 29
Respiratory Diseases Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health reducing occupationally related morbidity. In these efforts, the committee used the framework document (Appendix A) developed by the Committee for the Review of NIOSH Research Programs to provide structure and guidance to the individual evaluating committees (NAS 2005). Figure 1-3 outlines the general approach the framework document suggests using in evaluating the relevance and impacts of the RDRP. Relevance was evaluated in terms of the degree of research priority and connection to improvements in workplace protection. Factors taken into account include the frequency and severity of health outcomes and the number of people at risk, the structure of the program, and the degree of consideration of stakeholders’ input. The impact of the program’s research is evaluated in terms of its contributions to worker health and safety. The framework document recommends that the evaluation committee look at each strategic goal and present the committee’s assessments with regard to the relevance and impacts of the research completed or in progress for each separate goal. At the end of the review of the five strategic goals, the committee then develops a quantitative score for the relevance and impact of FIGURE 1-3 Flowchart for evaluation of the NIOSH research program.
OCR for page 30
Respiratory Diseases Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health the program as a whole and provides its quantitative and qualitative evaluation of the relevance and impacts of the RDRP. The evaluation is to conclude with detailed qualitative assessments as well as the assignment of scores between 1 and 5 for the relevance and impact of the RDRP research and other activities. Box 1-1 and Box 1-2 provide the scoring criteria for rating the programs. The study charge also directs the committee to review the progress the RDRP has made in identifying new research and provides the committee the opportunity to identify emerging research areas relevant to the program’s mission. According to the framework document, the committee’s identification of emerging research areas was done using members’ expert judgment rather than a formal research needs identification effort. The guidance in the framework document reflects the terminology and organization of a logic model adopted by NIOSH to characterize the steps in its work. The logic model used by the RDRP is shown in Figure 1-4, and examples of the terms used within the logic model are provided in Box 1-3. The committee reached consensus on its assessment of the individual activities of the RDRP and the program as a whole through deliberations at meetings and discussion on its written materials. This included a portion of one meeting spent devoted solely to develop- BOX 1-1 Criteria for Rating Relevance 5 = Research is in highest-priority subject areas and highly relevant to improvements in workplace protection; research results in, and NIOSH is engaged in, transfer activities at a significant level (highest rating). 4 = Research is in highest-priority subject areas and adequately connected to improvements in workplace protection; research results in, and NIOSH is engaged in, transfer activities. 3 = Research focuses on lesser priorities and is loosely or only indirectly connected to workplace protection; NIOSH is not significantly involved in transfer activities. 2 = Research program is not well integrated or well focused on priorities, is not clearly connected to workplace protection, and is inadequately connected to transfer activities. 1 = Research in the research program is an ad hoc collection of projects, is not integrated into a program, and is not likely to improve workplace safety or health.
OCR for page 31
Respiratory Diseases Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health BOX 1-2 Criteria for Rating Impact 5 = Research program has made a major contribution to worker health and safety on the basis of end outcomes or well-accepted intermediate outcomes. 4 = Research program has made a moderate contribution on the basis of end outcomes or well-accepted intermediate outcomes; or research program generated important new knowledge and is engaged in transfer activities, but well-accepted intermediate outcomes or end outcomes have not been documented. 3 = Research program activities or outputs are going on and are likely to improve worker health and safety (with explanation of why not rated higher). Transfer activity is planned. 2 = Research program activities or outputs are going on and may result in new knowledge or technology, but only limited application is expected. NIOSH is not significantly involved in transfer activities. 1 = Research activities and outputs are NOT likely to have any application. NA = Impact cannot be assessed; program is not mature enough. ing scores for relevance and impact. Assessments of “goals,” “inputs,” “activities,” and “outputs” were used to evaluate the relevance of the program’s research. End and intermediate outcomes were the principal focus for evaluation of the impact of the program’s research. The committee’s scores for relevance and impact are presented and discussed in Chapter 2 of this report. Information Sources The material for this review comes from: (1) the written documentation, known as the evidence package, provided by NIOSH to the evaluation committee before the first meeting of the evaluation committee (NIOSH 2006a); (2) presentations by NIOSH and stakeholders at the committee meetings; (3) responses to letters sent to NIOSH by the evaluation committee dated November 17, 2006, and December 6, 2006; (4) review of appendix material submitted with the original written documentation and a presentation at the October 26, 2006, meeting of the evaluation committee. The committee also drew on individual members’ knowl-
OCR for page 32
Respiratory Diseases Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health FIGURE 1-4 Logic model for the RDRP. Abbreviations: CDC, Centers for Disease Control and Prevention; NGOs, nongovernmental organizations; OSH, Occupational Safety and Health. Source: NIOSH 2006a. edge of NIOSH’s respiratory disease research and on standard literature searches (e.g., PubMed and other databases) where appropriate. ORGANIZATION OF THE REPORT The remainder of the report presents the findings from the committee’s evaluation. Chapter 2 presents the committee’s review of the NIOSH RDRP and the ratings for the program’s relevance and impact in reducing workplace injury and illness. In Chapter 3, the committee reviews the RDRP’s mechanisms for identifying emerging issues in occupational respiratory disease and identifies issues that may warrant future attention. In Chapter 4, the committee identifies opportunities to strengthen the NIOSH RDRP and increase the relevance and impact of the program’s efforts.
OCR for page 33
Respiratory Diseases Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health BOX 1-3 Logic Model Terms and Examples Planning Inputs: Stakeholder input, surveillance and intervention data, and risk assessments. Production Inputs: Intramural and extramural funding, staffing, management structure, and physical facilities. Activities: Efforts and work of the program, staff, grantees, and contractors. Outputs: A direct product of a NIOSH research program that is logically related to the achievement of desirable and intended outcomes. Intermediate Outcomes: Related to the program’s association with behaviors and changes at individual, group, and organizational levels in the workplace. An assessment of the worth of NIOSH research and its products by outside stakeholders. End Outcomes: Improvements in safety and health in the workplace. Defined by measures of health and safety and of impact on processes and programs. External Factors: Actions of forces beyond NIOSH’s control with important bearing on incorporation in the workplace of NIOSH’s outputs to enhance health and safety.