|
||||||||||||||||
|
||||||||||||||||
The following HTML text is provided to enhance online readability. Many aspects of typography translate only awkwardly to HTML. Please use the page image as the authoritative form to ensure accuracy. 1
|
||||||||||||||||
 |
 |
Page 15 |
 |
 |
 |
|
| ||||||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||||
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 15
Respiratory Diseases Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health
1
Introduction
The National Institute for Occupational Safety and Health (NIOSH) was established by the Occupational Safety and Health Act of 1970 to “conduct … research, experiments, and demonstrations relating to occupational safety and health” and to develop “innovative methods, techniques, and approaches for dealing with [those] problems” (Public Law 91-596, 84 STAT. 1590, 91st Congress, S.2193, December 29, 1970). NIOSH, part of the Centers for Disease Control and Prevention within the Department of Health and Human Services, is authorized to conduct research, training, and education related to worker health and safety; perform on-site investigations to investigate hazards in the workplace; recommend occupational health and safety standards; and fund research by other agencies or private organizations. NIOSH has the legislative responsibility to develop the research base upon which it can then recommend occupational health and safety standards to OSHA, although it does not have the authority to promulgate binding standards or enforce regulations on workplace safety and health. Federal regulatory and enforcement authority for occupational safety and health rests with the Occupational Safety and Health Administration (OSHA) and the Mine Safety and Health Administration (NIOSH 2006a). The organizational configuration of NIOSH is shown in Figure 1-1.
Preventing occupational respiratory disease has been a key part of the NIOSH research portfolio since the agency’s inception in 1970. Early on, NIOSH assumed responsibilities for health screening and research related to coal workers’ pneumoconiosis (CWP) that initially was mandated under the Federal Coal Mine Health and Safety Act of 1969. Respiratory disease research and surveillance related to min-
OCR for page 16
Respiratory Diseases Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health
FIGURE 1-1 NIOSH organization chart, as of October 2006. Source: NIOSH 2006a.
OCR for page 17
Respiratory Diseases Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health
ing activities remain major NIOSH activities. However, current work-related respiratory diseases and disorders that NIOSH is studying include a broad spectrum of other adverse health effects that can arise in a wide range of occupational settings. Such adverse health effects can range from mild, reversible conditions to progressive, fatal disorders and can be linked to short-term or long-term exposures.
A major focus of NIOSH’s activities both historically and currently has been on occupational respiratory diseases. Deaths from work-related respiratory diseases and malignancies have been estimated to account for about 70% of all occupational disease mortality (Steenland et al. 2003). The American Thoracic Society stated that “A careful review of the literature demonstrates that approximately 15% of both asthma and COPD is likely to be work related” (Balmes et al. 2003). It has been estimated that this occupational asthma and chronic obstructive pulmonary disease (COPD) costs nearly $7 billion annually in the United States (Leigh et al. 2002). In 2000, there were an estimated 386,000 deaths worldwide from asthma, COPD, and pneumoconiosis and nearly 6.6 million disability-adjusted life years due to occupational exposure to airborne particulates (Driscoll et al. 2005a).
As work-related diseases have been identified and safety measures taken to reduce the associated risks, the overall patterns of occupational respiratory diseases have changed. NIOSH data through 1999 indicate that morbidity and mortality from CWP declined appreciably over the preceding 20 years (NIOSH 2003). More recent data, however, indicate that the trend has slowed and even reversed, making this area one of active NIOSH investigation (NIOSH 2006a; data from E. L. Petsonk, NIOSH, as cited by Ward 2007). In the same time period, mortality due to silicosis has declined from well over 1,000 deaths annually in the late 1960s to fewer than 200 per year in the late 1990s (NIOSH 2003). Nonetheless, as with CWP “hot spots” (MMWR 2006a, 2007a), NIOSH has continued to report important clusters of silica-related disease (MMWR 1990, 2004).
The identification of work-related outbreaks of respiratory disease—such as those related to artificial butter flavorings, respirable particles of nylon flock, and leather conditioning sprays—and of CWP hot spots point to the changing nature of occupational respiratory disease, with new issues arising from classic exposures as well as new diseases being detected from novel exposures. Additional challenges for NIOSH include the need to protect workers from potential occupational lung diseases due to exposures during national security emergencies, the potential for occupational exposures to weaponized biological agents, the emergence of naturally occurring infectious diseases, and the increased production and incorporation of nanomaterials that pose unknown risks to workers.
OCR for page 18
Respiratory Diseases Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health
OVERVIEW OF THE RESPIRATORY DISEASES RESEARCH PROGRAM
The formally defined Respiratory Diseases Research Program (RDRP) is a recent creation. As described in the evidence package, the RDRP is an “organizational component” that was designated in 2005 as a result of matrix management efforts intended to coordinate cross-institute programmatic activities as a result of the second National Occupational Research Agenda (NORA2) process. NIOSH describes the RDRP as “the range of individuals and groups supported by NIOSH to do work that is relevant to occupational respiratory disease” (NIOSH 2006a). As such, the program is intended to be a vehicle to describe the broad range of research on respiratory disease that occurs at NIOSH. For example, the RDRP is not an institute or physical location where the research occurs nor does it constitute a cohesive staff grouping under a single hierarchical reporting structure. The RDRP includes the multiple divisions and laboratories within NIOSH that deal with respiratory disease issues. For the sake of clarity, throughout this report the committee uses the term RDRP to refer to activities that occurred before as well as after establishment of the RDRP in 2005, even though it is recognized that the designation, as such, did not exist in the earlier time frame.
History of the Program
Originating Legislation and Facilities
Table 1-1 shows some of the administrative developments important for development of the RDRP. The Federal Coal Mine Health and Safety Act of 1969 prescribed a number of research, surveillance, and regulatory-related activities that were later assigned to NIOSH when it was formed by the Occupational Health and Safety Act of 1970. Adoption of the Mine Safety and Health Act of 1977 also had an impact on the RDRP mission. Because of the location of NIOSH coal mining research facilities—first in Beckley, West Virginia, and later in Morgantown, West Virginia—initial RDRP research focused almost entirely on CWP. Beginning in 1976, with the formation of the Division of Respiratory Disease Studies (DRDS), the scope of research on respiratory disease based in Morgantown expanded significantly to other forms of pneumoconiosis that included silicosis, beryllium disease, asbestosis, and other fibrotic lung diseases, and a focus on organic dusts and other respiratory irritants that result in occupational asthma, airway obstruction, and hypersensitivity pneumonitis.
Other elements of NIOSH that have had a role in respiratory disease activities are located in NIOSH facilities in Cincinnati, Ohio. These facilities include the Division of Applied Research and Technology, with expertise in exposure assessment, development of analytical methods, and control technologies; the Education
OCR for page 19
Respiratory Diseases Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health
TABLE 1-1 Timeline of Administrative Developments Related to NIOSH Respiratory Diseases Research
Year
Administrative Development
1969
Federal Coal Mine Health and Safety Act
1970
Occupational Health and Safety Act
Several RDRP components established in Cincinnati, Ohio.
1971
Appalachian Laboratory for Occupational Safety and Health (ALOSH) opens in Morgantown, West Virginia
1976
Division of Respiratory Disease Studies opens within ALOSH
1977
Federal Mine Safety and Health Act of 1977
1996
Health Effects Laboratory Division founded
1996
Pittsburgh Research Laboratory and Spokane Research Laboratory added to NIOSH
1996
Education and Information Division created
2000
Division of Applied Research and Technology formed from the Division of Physical Sciences and Engineering and the Division of Biological and Behavioral Sciences
2001
National Personal Protective Technology Laboratory founded
2006
Respiratory Diseases Research Program founded
Source: NIOSH 2006a.
and Information Division, responsible for health communication and coordinating development of recommendation documents; and the Division of Surveillance, Hazard Evaluations and Field Studies (DSHEFS), responsible for conducting short-and long-term field studies to evaluate health and safety issues including respiratory diseases. DSHEFS is the principal division that handles health hazard evaluations, although the DRDS in Morgantown has also been active in respiratory-disease-related health hazard evaluations.
Other facilities have performed activities related to respiratory disease. One of them is the Health Effects Laboratory Division, which focuses on basic bench laboratory research, including research in basic toxicology and in engineering and exposure assessment. The Pittsburgh Research Laboratory and the Spokane Research Laboratory are mining-focused research groups concerned with research on mining engineering control technology for respiratory hazards. Most recently, the National Personal Protective Technology Laboratory in Bruceton, Pennsylvania, has responsibility for research on respiratory protection and certification of respirators (NIOSH 2006a).
Strategic Planning
Occupational lung diseases have long been a top priority for NIOSH and were the subject of a proposed national strategy in 1986 (NIOSH 1986a). In this
OCR for page 20
Respiratory Diseases Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health
strategy, asbestosis, byssinosis (associated with cotton dust), silicosis, and CWP were cited as specific examples for action. The proposed strategy included the following elements: environmental hazard surveillance, medical hazard surveillance, hazard removal, control technologies, regulatory enforcement, worker education and training (includes respirators and other personal protective devices), and worker-oriented programs that include health promotion and smoking prevention (NIOSH 1986a).
In 1996, NIOSH worked with the occupation safety and health community to develop a National Occupational Research Agenda (NORA) (NIOSH 1996a). Seen as a way to better guide and organize research activities in a fiscally constrained environment, NORA identified 21 research priorities grouped into three categories: disease and injury, work environment and workforce, and research tools and approaches (Table 1-2). Under NORA, there were three priorities related to respiratory disease under the disease and injury category: allergic and irritant dermatitis,1 asthma and COPD, and infectious disease. NIOSH’s objectives in creating the NORA approach were to
Guide intramural and extramural funding decisions.
Encourage and stimulate other government agencies to include NORA priorities in their internal and external research programs.
Develop procedures and capacity to track the impact of NORA activities on health and safety outcomes using existing tracking models, if available.
Provide for timely updates to the NORA priorities.
Periodically review and communicate the overall role and effectiveness of NORA in occupational safety and health.
During development of the NORA agenda, the importance of industrial sector-specific research (for example, construction or agriculture) was consistently raised (NIOSH 1996a). It was decided that the most effective way to integrate this research was through a matrix approach of coordinated research in some or all of the 21 priority areas, as appropriate for each sector. An example of this approach is provided in Table 1-3.
Beginning in 2005, NIOSH introduced the second phase of the NORA process (NORA2), now organizing research priorities focused on eight key industry sectors. The sectors, listed in Table 1-4, are agriculture, forestry, and fishing; construction; health care and social assistance; manufacturing; mining; services; wholesale and retail trade; and transportation, warehousing, and utilities. Sector alignment is intended to provide direct occupational health and safety research and assistance
1
NIOSH groups allergic and irritant dermatitis in this classification.
OCR for page 21
Respiratory Diseases Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health
TABLE 1-2 NORA Priority Research Areas
Disease and Injury
Allergic and irritant dermatitisa
Asthma and chronic obstructive pulmonary diseasea
Fertility and pregnancy abnormalities
Hearing loss
Infectious diseasesa
Low back disorders
Musculoskeletal disorders of the upper extremities
Traumatic injuries
Work Environment and Workforce
Emerging technologies
Indoor environment
Mixed exposures
Organization of work
Special populations at risk
Research Tools and Approaches
Cancer research methods
Control technology and personal protective equipment
Exposure assessment methods
Health services research
Intervention effectiveness research
Risk assessment methods
Social and economic consequences of workplace illness and injury
Surveillance research methods
aRespiratory disease focus.
Source: NIOSH 1996a.
TABLE 1-3 Example of NORA’s Matrix Approach to Coordinating Research
Sector
Allergic and Irritant
Asthma and Chronic Obstructive
Fertility and Pregnancy
Hearing Loss
Agriculture
Xa
X
X
X
Construction
X
X
X
Service
X
X
X
Mining
X
X
X
Manufacturing
X
X
X
X
Xa = priority research area within a sector.
Source: NIOSH 1996a.
OCR for page 22
Respiratory Diseases Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health
TABLE 1-4 NORA2 Structure
NORA Sector Programs
Agriculture, forestry, and fishing
Construction
Health care and social assistance
Manufacturing
Mining
Services
Wholesale and retail trade
Transportation, warehousing, and utilities
NIOSH Cross-Sector Programs
Authoritative recommendations
Cancer, reproductive, and cardiovascular
Communications and information dissemination
Emergency preparedness/response
Global collaborations
Health hazard evaluation
Hearing loss prevention
Immune and dermal
Musculoskeletal disorders
Personal protective technology
Radiation dose reconstruction
Respiratory diseases
Training grants
Traumatic injury
Work organization and stress-related disorders
NIOSH Coordinated Emphasis Areas
Economics
Exposure assessment
Engineering controls
Work life initiative
Occupational health disparities
Small business assistance and outreach
Surveillance
to address specific issues as identified within each sector. Additionally, NORA2 identified 15 cross-sector programs and 7 coordinated emphasis areas, also shown in Table 1-4. The RDRP is one of the cross-sector programs. Each cross-sector program has a steering committee made up of staff from all NIOSH divisions and laboratories as well as the Office of the Director and the Office of Extramural Programs. According to NIOSH (2006a), the steering committee annually reviews
OCR for page 23
Respiratory Diseases Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health
and ranks competitive intramural funding requests, reviews all in-house projects relevant to its program, and provides feedback to divisions about each project’s relevance and level of priority.
Respiratory Diseases Research Program Structure
As shown in Figure 1-2, the RDRP, as conceptualized, spans nearly all NIOSH units. In 2006, five divisions, three laboratories, and three offices were involved in the RDRP. The mission statement for the RDRP (Weissman 2006) is
To provide national and international leadership for the prevention of work-related respiratory diseases, using a scientific approach to gather and synthesize information, create knowledge, provide recommendations, and deliver products and services to those who can effect prevention.
FIGURE 1-2 Respiratory Diseases Research Program projects throughout NIOSH (does not include extramural, communications, research, and technology transfer offices). Source: NIOSH 2006a.
OCR for page 24
Respiratory Diseases Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health
Within this overall mission five specific goals have been stated:
Prevent and reduce work-related airways diseases.
Prevent and reduce work-related interstitial lung diseases.
Prevent and reduce work-related respiratory infectious diseases.
Prevent and reduce work-related respiratory malignancies.
Prevent respiratory and other diseases potentially resulting from occupational exposures to nanomaterials.
To pursue these goals, the RDRP activities include multidisciplinary approaches to respiratory diseases through laboratory studies and short- and long-term field studies; surveillance and reporting on disease frequencies; assessment of control technologies and respiratory protection; communication, education, and training; and recommendations on exposure and other practices. Facilities and equipment exist for biomedical research, development of analytical methods, research on exposure assessment, research on engineering and industrial hygiene, respirator research, mining research, and epidemiologic investigations. Ultimately, NIOSH provides the scientific underpinning for development of new or revision of existing standards by OSHA.
NIOSH’s Draft Fiscal Year 2008 Program Planning Guide, which describes the process for acquiring competitively awarded intramural research funds at NIOSH, outlines RDRP’s strategic and intermediate goals and their performance measures. The document has a high level of specificity, with measurable goals and target time frames. This document became available at the end of the committee’s deliberations; thus, the committee has not examined whether they are the correct performance measures or whether they are realistic. The committee notes, however, that because the RDRP is an “organizational unit” borne of the new matrix management approach that spans institutes and divisions, it is unclear whether funding and support are distributed based on whether a project satisfies RDRP’s strategic goals or whether RDRP’s goals do not matter because funding is distributed based on the goals of the divisions or institutes. Being “at the table” does not provide assurance to the RDRP or to the committee that appropriate weight is given to funding priorities. According to the evidence package, RDRP’s steering committee, with representatives from all divisions, “has been empowered to mold activities in respiratory diseases through its abilities to review and rank competitive intramural funding requests and through its access to the NIOSH Office of the Director.” (The committee did not attend or review minutes from steering committee meetings.) It is not clear what formal mechanisms exist to ensure that RDRP’s goals are supported by funding.
OCR for page 25
Respiratory Diseases Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health
TABLE 1-5 NIOSH RDRP Funding by Fiscal Year, 1996-2005
Year
NIOSH Funding, Millions of Dollars
Intramural
Extramural
Total
1996
7.4
2.3
9.7
1997
7.1
3.1
10.2
1998
10.9
4.5
15.4
1999
13.2
7.3
20.5
2000
15.5
7.3
22.8
2001
16.2
8.6
24.8
2002
16.1
10.1
26.2
2003
16.4
10.3
26.7
2004
16.3
8.6
24.9
2005
17.2
11.4
28.6
Totals
136.3
73.5
209.8
Source: Ray Sinclair, NIOSH, unpublished material, April 07, 2007. Table 2-2 shows additional data for extramural funding through 2006, resulting in a total for extramural funding of $88 million for the period 1996-2006.
Program Resources
RDRP funding for the most recent fiscal year available (2005) was close to $29 million. Table 1-5 shows the funding levels for the RDRP intramural and extramural funded research for the years 1996-2005. Intramural funding supports staff salaries and benefits as well as goods and services related to staff research activities. Extramural activities are administered and funded through the NIOSH Office of Extramural Programs in Atlanta (greater detail, including by strategic goal, is provided in Chapter 2). Table 1-6 compares the fraction of the budget spent on intramural versus extramural research for the RDRP with several institutes or centers of the National Institutes of Health (NIH) and shows that the RDRP tends to spend about two-thirds of its budget on intramural projects. This is in contrast to research funding at NIH agencies, including the National Institute of Environmental Health Sciences, that tend to spend most of their research funding on extramural research. Given the relatively small research budget compared with other institutes whose research budgets are typically one to two orders of magnitude higher,2 the committee considered the limited support allocated to extramural programs unfortunate
2
For example, the fiscal year 2003 research budgets for NIEHS and NHLBI were approximately $500 million and $2.4 billion, respectively (DSA Pubdata, http://grants.nih.gov/grants/award/trends/icfund9803.html). In contrast, the 2003 research budget for RDRP was approximately $27 million (see Table 1-5).
OCR for page 26
Respiratory Diseases Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health
TABLE 1-6 NIH Versus NIOSH Extramural Funding
Funding Source
Extramurala Percent of Annual Research Budget
2003
2002
2001
2000
1999
1998
National Center for Complementary and Alternative Medicine
82
78
77
79
—
—
National Cancer Institute
77
75
74
75
75
74
National Center on Minority Health and Health Disparities
55
43
0
—
—
—
National Eye Institute
83
84
87
88
88
87
National Heart, Lung, and Blood Institute
87
87
86
87
86
88
National Institute on Aging
83
83
85
84
84
84
National Institute on Alcohol Abuse and Alcoholism
81
82
84
85
86
85
National Institute of Allergy and Infectious Diseases
79
83
85
87
84
86
National Institute of Arthritis and Musculoskeletal and Skin Diseases
87
87
87
90
90
89
National Institute of Child Health and Human Development
82
84
85
84
83
82
National Institute on Drug Abuse
84
84
86
88
88
88
National Institute on Deafness and Other Communication Disorders
84
85
85
86
87
87
National Institute of Dental and Craniofacial Research
75
74
74
74
75
76
National Institute of Diabetes and Digestive and Kidney Diseases
84
85
85
87
86
85
National Institute of Environmental Health Sciences
71
71
72
79
83
84
National Institute of Mental Health
83
83
84
84
83
82
National Institute of Neurological Disorders and Stroke
85
86
86
86
85
85
National Institute of Nursing Research
92
93
95
93
98
95
National Institute for Occupational Safety and Health Respiratory Diseases Research Program
39
39
35
32
36
29
aExtramural includes research grants, research and development, training grants, and fellowships.
Source: Ray Sinclair, NIOSH, unpublished material, April 07, 2007, for NIOSH RDRP; DSA Pubdata (http://grants.nih.gov/grants/award/trends/icfund9803.html) for others.
but probably necessary. Because the committee was not asked to review the extramural program, the committee is not in a position to evaluate the balance between extramural and intramural funding. However, as Table 1-5 shows, the proportion of extramural funding has generally increased with time, from 24% in 1996 to 40% in 2005, the latest year for which complete data are available.
OCR for page 27
Respiratory Diseases Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health
STUDY CHARGE AND EVALUATION COMMITTEE
In September 2004, NIOSH requested that the National Academies review various NIOSH programs with respect to the impact and relevance of their work in reducing workplace injury and illness and to identify future directions that their work might take. The Committee to Review the NIOSH Research Programs was established as an oversight committee and created a framework document, which will be used to evaluate individual programs (Appendix A) (NAS 2005). The RDRP was selected as one of the programs to undergo such a review. The National Research Council (NRC) convened the Committee to Review the NIOSH Respiratory Diseases Research Program in late 2006. The statement of task for the committee is as follows:
In response to a request from the National Institute for Occupational Safety and Health (NIOSH), the Institute of Medicine (IOM) and the Division of Earth and Life Studies (DELS) of the National Academies (NA) are conducting a series of evaluations of NIOSH research programs. Each evaluation will be conducted by an ad hoc committee, using a methodology and framework developed by the Committee to Review NIOSH Research Programs (framework committee).
Each evaluation committee will review the program’s impact, relevance, and future directions. The evaluation committee will evaluate not only what the NIOSH research program is producing, but will also determine whether it is appropriate to credit NIOSH research with changes in workplace practices, hazardous exposures, and/or occupational illnesses and injuries, or whether the changes are the result of other factors unrelated to NIOSH.
The program reviews should focus on evaluating the program’s impact and relevance to health and safety issues in the workplace and make recommendations for improvement. In conducting the review, the evaluation committee will address the following elements:
Assessment of the program’s contribution through occupational safety and health research to reductions in workplace hazardous exposures, illnesses, or injuries through:
An assessment of the relevance of the program’s activities to the improvement of occupational safety and health, and
An evaluation of the impact that the program’s research has had in reducing work-related hazardous exposures, illnesses, and injuries.
OCR for page 28
Respiratory Diseases Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health
The evaluation committee will rate the performance of the program for its relevance and impact using a scale of 1 to 5. Impact may be assessed directly (e.g., reductions in illnesses or injuries) or, as necessary, using intermediate outcomes to estimate impact. Qualitative narrative evaluations should be included to explain the numerical ratings.
Assessment of the program’s effectiveness in targeting new research areas and identifying emerging issues in occupational safety and health most relevant to future improvements in workplace protection. The committee will provide a qualitative narrative assessment of the program’s efforts and suggestions about emerging issues that the program should be prepared to address.
The study committee was selected to include members with expertise in epidemiology, exposure assessment, industrial hygiene, inhalation toxicology, occupational medicine, and pulmonology. The committee reviewed the RDRP emphasizing, although not exclusively limited to, the time period since 1996, as this time frame encompasses both the original NORA and the recently completed NORA2. However, the committee is cognizant that improvements in workers’ health with regard to respiratory diseases that occurred during this period may be a result of NIOSH research activities that were completed much earlier. The committee met three times in the period October 2006 through March 2007. The first two meetings were data-gathering sessions that included presentations by NIOSH staff and other invited speakers in open session. At the end of each open session stakeholders and the general public had an opportunity to comment. In addition, several committee members and a member of the NRC staff attended a site visit at the NIOSH Morgantown, Cincinnati, and Pittsburgh facilities.
The committee’s review of the NIOSH RDRP was based in part on materials provided by NIOSH at the onset and on specific materials the committee requested during the review (see Appendix C for a list of these materials). The committee wanted to hear from a broad range of stakeholders and created an online questionnaire for them to provide comments (see Appendix B). Presentations and discussions during the open session as well as the online questionnaire helped in shaping additional questions and giving background to areas under review.
EVALUATION APPROACH
Framework Document and Logic Model
The committee evaluated the relevance of RDRP research to improvements in occupational safety and health and the impact that NIOSH research has had in
OCR for page 29
Respiratory Diseases Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health
reducing occupationally related morbidity. In these efforts, the committee used the framework document (Appendix A) developed by the Committee for the Review of NIOSH Research Programs to provide structure and guidance to the individual evaluating committees (NAS 2005). Figure 1-3 outlines the general approach the framework document suggests using in evaluating the relevance and impacts of the RDRP.
Relevance was evaluated in terms of the degree of research priority and connection to improvements in workplace protection. Factors taken into account include the frequency and severity of health outcomes and the number of people at risk, the structure of the program, and the degree of consideration of stakeholders’ input. The impact of the program’s research is evaluated in terms of its contributions to worker health and safety. The framework document recommends that the evaluation committee look at each strategic goal and present the committee’s assessments with regard to the relevance and impacts of the research completed or in progress for each separate goal. At the end of the review of the five strategic goals, the committee then develops a quantitative score for the relevance and impact of
FIGURE 1-3 Flowchart for evaluation of the NIOSH research program.
OCR for page 30
Respiratory Diseases Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health
the program as a whole and provides its quantitative and qualitative evaluation of the relevance and impacts of the RDRP. The evaluation is to conclude with detailed qualitative assessments as well as the assignment of scores between 1 and 5 for the relevance and impact of the RDRP research and other activities. Box 1-1 and Box 1-2 provide the scoring criteria for rating the programs.
The study charge also directs the committee to review the progress the RDRP has made in identifying new research and provides the committee the opportunity to identify emerging research areas relevant to the program’s mission. According to the framework document, the committee’s identification of emerging research areas was done using members’ expert judgment rather than a formal research needs identification effort.
The guidance in the framework document reflects the terminology and organization of a logic model adopted by NIOSH to characterize the steps in its work. The logic model used by the RDRP is shown in Figure 1-4, and examples of the terms used within the logic model are provided in Box 1-3. The committee reached consensus on its assessment of the individual activities of the RDRP and the program as a whole through deliberations at meetings and discussion on its written materials. This included a portion of one meeting spent devoted solely to develop-
BOX 1-1
Criteria for Rating Relevance
5
=
Research is in highest-priority subject areas and highly relevant to improvements in workplace protection; research results in, and NIOSH is engaged in, transfer activities at a significant level (highest rating).
4
=
Research is in highest-priority subject areas and adequately connected to improvements in workplace protection; research results in, and NIOSH is engaged in, transfer activities.
3
=
Research focuses on lesser priorities and is loosely or only indirectly connected to workplace protection; NIOSH is not significantly involved in transfer activities.
2
=
Research program is not well integrated or well focused on priorities, is not clearly connected to workplace protection, and is inadequately connected to transfer activities.
1
=
Research in the research program is an ad hoc collection of projects, is not integrated into a program, and is not likely to improve workplace safety or health.
OCR for page 31
Respiratory Diseases Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health
BOX 1-2
Criteria for Rating Impact
5
=
Research program has made a major contribution to worker health and safety on the basis of end outcomes or well-accepted intermediate outcomes.
4
=
Research program has made a moderate contribution on the basis of end outcomes or well-accepted intermediate outcomes; or research program generated important new knowledge and is engaged in transfer activities, but well-accepted intermediate outcomes or end outcomes have not been documented.
3
=
Research program activities or outputs are going on and are likely to improve worker health and safety (with explanation of why not rated higher). Transfer activity is planned.
2
=
Research program activities or outputs are going on and may result in new knowledge or technology, but only limited application is expected. NIOSH is not significantly involved in transfer activities.
1
=
Research activities and outputs are NOT likely to have any application.
NA
=
Impact cannot be assessed; program is not mature enough.
ing scores for relevance and impact. Assessments of “goals,” “inputs,” “activities,” and “outputs” were used to evaluate the relevance of the program’s research. End and intermediate outcomes were the principal focus for evaluation of the impact of the program’s research. The committee’s scores for relevance and impact are presented and discussed in Chapter 2 of this report.
Information Sources
The material for this review comes from: (1) the written documentation, known as the evidence package, provided by NIOSH to the evaluation committee before the first meeting of the evaluation committee (NIOSH 2006a); (2) presentations by NIOSH and stakeholders at the committee meetings; (3) responses to letters sent to NIOSH by the evaluation committee dated November 17, 2006, and December 6, 2006; (4) review of appendix material submitted with the original written documentation and a presentation at the October 26, 2006, meeting of the evaluation committee. The committee also drew on individual members’ knowl-
OCR for page 32
Respiratory Diseases Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health
FIGURE 1-4 Logic model for the RDRP. Abbreviations: CDC, Centers for Disease Control and Prevention; NGOs, nongovernmental organizations; OSH, Occupational Safety and Health. Source: NIOSH 2006a.
edge of NIOSH’s respiratory disease research and on standard literature searches (e.g., PubMed and other databases) where appropriate.
ORGANIZATION OF THE REPORT
The remainder of the report presents the findings from the committee’s evaluation. Chapter 2 presents the committee’s review of the NIOSH RDRP and the ratings for the program’s relevance and impact in reducing workplace injury and illness. In Chapter 3, the committee reviews the RDRP’s mechanisms for identifying emerging issues in occupational respiratory disease and identifies issues that may warrant future attention. In Chapter 4, the committee identifies opportunities to strengthen the NIOSH RDRP and increase the relevance and impact of the program’s efforts.
OCR for page 33
Respiratory Diseases Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health
BOX 1-3
Logic Model Terms and Examples
Planning Inputs: Stakeholder input, surveillance and intervention data, and risk assessments.
Production Inputs: Intramural and extramural funding, staffing, management structure, and physical facilities.
Activities: Efforts and work of the program, staff, grantees, and contractors.
Outputs: A direct product of a NIOSH research program that is logically related to the achievement of desirable and intended outcomes.
Intermediate Outcomes: Related to the program’s association with behaviors and changes at individual, group, and organizational levels in the workplace. An assessment of the worth of NIOSH research and its products by outside stakeholders.
End Outcomes: Improvements in safety and health in the workplace. Defined by measures of health and safety and of impact on processes and programs.
External Factors: Actions of forces beyond NIOSH’s control with important bearing on incorporation in the workplace of NIOSH’s outputs to enhance health and safety.