Mining Safety and Health Research at NIOSH Reviews of Research Programs of the National Institute for Occupational Safety and Health (2007) / Chapter Skim
Currently Skimming:
A A Framework for the Review of Research Programs of the National Institute for Occupational Safety and Health
A A Framework for the Review of Research Programs of the National Institute for Occupational Safety and Health
Pages 193-238
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 193... ...
Appendixes
|
From page 195... ...
It is a working document that will be subject to change by the Framework Committee aimed at improving its relevance on the basis of responses received from evaluation committee members, NIOSH, stakeholders, and the general public before and during the course of the assessments conducted by independent evaluation committees of up to 15 research programs and healthoutcomes programs.
|
From page 196... ...
mInIng safety H e a lt H R e s e a R c H nIosH and at 9 All public comments submitted to the Committee for the Review of NIOSH Research Programs will be included in the Public Access File for this study as pro vided in the National Academies Terms of Use (www.nationalacademies.org/legal/ terms.html)
|
From page 197... ...
Evaluation of NIOSH Research Programs -- the Process 212 III.A. Analysis of External Factors Relevant to the NIOSH Program 212 III.A.1.
|
From page 198... ...
mInIng safety H e a lt H R e s e a R c H nIosH and at 9 Table 1 NORA High-Priority Research Areas by Category 203 Table 2 Examples of NIOSH Program Research and Transfer Activities 220 Table 3 Examples of a Variety of Scientific Information Outputs 223 Table 4 Evaluation Committee Worksheet to Assess Research Programs and Subprograms 231
|
From page 199... ...
NORA2 (National Occupational Research Agenda 2005-forward) OSHA (Occupational Safety and Health Administration)
|
From page 200... ...
In developing a framework, the FC was asked to address the following: 1. Evaluation committee assessment of progress in reducing workplace illnesses and injuries facilitated by occupational safety and health research through (a)
|
From page 201... ...
I.A. NIOSH Strategic goals and Operational Plan As a prelude to understanding the NIOSH strategic goals and operational plan, NIOSH research efforts should be understood in the context of the Occupational Safety and Health Act (OSHAct)
|
From page 202... ...
In 1994, NIOSH embarked on a national partnership effort to identify research priorities to guide occupational health and safety research for the next decade. The National Occupational Research Agenda (NORA)
|
From page 203... ...
I.B. Information from Other Evaluations The FC is aware that several NIOSH programs have already been subjected to evaluation by internal and external bodies.
|
From page 204... ...
, training stakeholder inputs, releases extramural, including technologies, licenses, and education community, SH surveillance and domestic and patents practitioners intervention international efforts, such as work effectiveness data, conducted at ERCs, earmarks, ARCs and WHO risk assessments Global Network of Collaborating Centers Transfe r Prevention through effective research, transfer, and evaluation External Factors: Economic and social conditions and regulatory environment FIgURE 1 The NIOSH operational plan presented as a logic model A-1
|
From page 205... ...
• Internal NIOSH processes and research: ♦ Intramural surveillance, research, and transfer activities. ♦ Process to solicit and approve intramural research proposals.
|
From page 206... ...
♦ Interventions, recommendations, and information-dissemination and technology-transfer activities designed to get research findings used to improve occupational safety and health. ♦ Outcomes of research, alerts, standard-setting, investigations, and consultations; for example -- documented reductions in risk after program-supported interventions, employer and industry behav ior changes made in response to research outputs, and worker behavior changes in response to research outputs.
|
From page 207... ...
. • Intermediate outcomes (responses by NIOSH stakeholders to NIOSH products, such as public or private policy change, training and educa tion in the form of workshop or seminar attendance, self-reported use or repackaging of NIOSH data by intermediary stakeholders, adop tion of technologies developed by NIOSH, implemented guidelines, licenses, and reduction of workplace hazardous exposures and other risk factors)
|
From page 208... ...
0 FIgURE 2 Flow chart for the evaluation of the NIOSH research program
|
From page 209... ...
. For purposes of this reiew, the results of inputs and external factors are the program research actiities, outputs, and associated transfer actiities that may result in intermediate outcomes and possibly eentual end outcomes.
|
From page 210... ...
10. Identify significant emerging research areas (Section III.C)
|
From page 211... ...
In some cases, they may be quantifiable. It is possible, however, to evaluate the impact of a NIOSH research program whether the outcomes are intermediate outcomes or end outcomes.
|
From page 212... ...
. Identification of external factors by the ECs is essential to providing a context for NIOSH program evaluation.
|
From page 213... ...
• NIOSH resources are inadequate to tackle the key questions. Evaluation of the impact of NIOSH research outputs on outcomes may require consideration of external factors that might have impeded or aided implementation, measurement, and so on.
|
From page 214... ...
. It will be important for the ECs to get a general sense of the history of the NIOSH research program and its impact, but their efforts should be focused on the impact and relevance of NIOSH programs from 1996 on.
|
From page 215... ...
Research program relevant evaluations that should be requested include the NIOSH annual program review by the Leadership Team; the NORA research program proposal pre-award external review, NORA post-award program external review, and external scientific program review.
|
From page 216... ...
Production inputs include intramural and extramural funding, staffing, management structure, and physical facilities. Inputs for program evaluation include existing intramural and extramural information and, potentially, surveys or case studies that might have been devel oped specifically to assess progress in reducing workplace illnesses and injuries and to provide information relevant to targeting research appropriately to future
|
From page 217... ...
Planning inputs Planning inputs can be qualitative or quantitative. Sources of qualitative inputs include • Federal Advisory Committee Act panels (Board of Scientific Counsel ors, Mine Safety and Health Research Advisory Committee, National Advisory Committee on Occupational Safety and Health, and so on)
|
From page 218... ...
, and HHEs and related staff. Consideration should also be given to budget inputs for program evaluation and to leveraged funds provided by partners, such as National Institutes of Health and the Environmental Protec tion Agency joint requests for applications or program announcements and OSHA, MSHA, and Department of Defense contracts with NIOSH to conduct work.
|
From page 219... ...
. Transfer activities include information dissemination, training, technical assistance, and education designed to translate research outputs into content and formats designed for application in the workplace to produce improvements in occupational safety and health.
|
From page 220... ...
Diffusion and dissemination research Training effectiveness Information-dissemination effectiveness Diffusion of technology Health-services and other research Access to occupational health care Infrastructure research -- delivery of occupational-health services, including international health and safety Socioeconomic consequences of work-related injuries and illnesses Worker compensation Technology-transfer and other transfer activities Information dissemination Training programs The ECs should review the list of research and transfer activities (projects) for the research program under review that have been completed, are in progress, or have been planned.
|
From page 221... ...
Questions to Guide the Ealuation Committee in Assessing Transfer Actiities 1. Is there a coherent planned program of transfer activities?
|
From page 222... ...
Assessment of the transfer activities must include considerations of program planning, coherence, quality, and impact.
|
From page 223... ...
Peer-reviewed publications by external researchers funded by NIOSH Total number of NIOSH-funded original research articles by external researchers Total number of NIOSH-funded review articles by external researchers (including best-practices articles) Complete citation for each written report Complete copies of the "top five" articles Collaboration with other government or academic researchers NIOSH reports in the research program Total number of written reports Complete citation for each written report Complete copies of the "top five" reports Sponsored conferences and workshops Total number of sponsored conferences Total number of sponsored workshops For each sponsored conference or workshop, describe: Title, date, and location Partial vs complete sponsorship (if partial, who were cosponsors?
|
From page 224... ...
Complete citation Percentage of target audience that has used product 1, 5, and 10 years later Up to three examples of implementation in the field Identification of "top five" patents to date Miscellaneous Any other important program outputs The EC should ask NIOSH to provide information on all relevant outputs for the specific program for the chosen time period. Questions to Guide the Ealuation Committee 1.
|
From page 225... ...
; attendance at training and education programs sponsored by other organizations; use of publications by workers, industry, and occupational safety and health professionals in the field; and citations of NIOSH research by industrial and academic scientists. More difficult-to-collect intermediate outcomes that may be valid indicators of quality or utility include self-report measures by users and relevant nonusers of NIOSH outputs.
|
From page 226... ...
4. Has the program resulted in new personal protective equipment that is feasible for use or has been adopted in the workplace to reduce risk factors or exposures?
|
From page 227... ...
End outcomes are defined by measures of health and safety and of impact on process and programs. The FC recognizes that a major challenge in assessing the causal relationship between NIOSH research and specific occupational health and safety outcomes is that NIOSH does not have direct responsibility or authority for implementing its research findings in the workplace.
|
From page 228... ...
. • NIOSH intramural surveillance systems, such as the National Electron ic Injury Surveillance System (NEISS)
|
From page 229... ...
4. To what degree has the NIOSH program or subprogram been respon sible for improvements in occupational safety or health?
|
From page 230... ...
Assessment Evaluation by the EC may consist of a discussion of other outcomes, includ ing positive changes that have not yet occurred; other social, economic, security, or environmental outcomes; and the impact that NIOSH has had on international occupational safety and health. It might also consider the incorporation of in ternational research results into the NIOSH program of knowledge transfer for industry sectors.
|
From page 231... ...
and evidence that external factors affected reduction Subprogram Activity Addressing Charge 1 Category Program 1 … … n 1.3 Contributions of NIOSH research Research and transfer activities to changes in work-related practices Transfer 1.4 Contributions of NIOSH research Research and transfer activities to reductions in workplace exposure, Transfer illness, or injuries 1.5 Evidence of external factors Research preventing application of NIOSH research results Transfer 1.6 Contribution of NIOSH research Research to enhancement of capacity in government or other research Transfer institutions 1.7 Contributions of NIOSH research Research to productivity, security, or environmental quality (beneficial Transfer side effects) Addressing Charge 2 2.1 Relevance of current and recently Research completed research and transfer activities to future improvements Transfer in workplace safety and health 2.2 Progress in targeting research to Research areas of study most relevant to future improvements in Transfer occupational safety and health
|
From page 232... ...
and charge 2 (relevance) , the ECs will use their expert judgment, their responses to the questions in Table 4, and any other appropriate information to arrive at one overall rating for the impact of the research program and one for its relevance to the improvement of occupational safety and health.
|
From page 233... ...
Rating of Impact 5= Research program has made a major contribution to worker health and safety on the basis of end outcomes or well-accepted intermediate outcomes.
|
From page 234... ...
3= Research focuses on lesser priorities and is loosely or only indirectly connected to workplace protection; NIOSH is not significantly involved in transfer activities. 2= Research program is not well integrated or well focused on priorities and is not clearly connected to workplace protection and inadequately connected to transfer activities.
|
From page 235... ...
In addition, appropriate federal advisory committees and other stakeholder groups should be consulted to provide qualitative information. The EC members should use their expert judgment both to evaluate what NIOSH has identified as emerging research targets (charge 2)
|
From page 236... ...
Iv. EvALUATION COMMITTEE REPORT TEMPLATE The following outline flows from the FC's review of the generalized logic model prepared by NIOSH, the request for information from NIOSH programs, and the assessment model described earlier in this report.
|
From page 237... ...
: The EC should assess the progress that the NIOSH program has made in targeting new research in the fields of occupational safety and health. There should be a discussion of the assessment process and results.
|
From page 238... ...
G Assessment of relative importance of external factors in permitting or preventing intermediate or end outcomes; attention paid to accounting for and planning within the constraints of external fac tors (not simply assigning lack of progress to external factors)
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.