5
Planning for the Future

In the years since the objectives and the methodology of the Survey of Respirator Use and Practices (SRUP) were developed, and since the results of the survey were made available, the approach of the federal government to personal protective equipment (PPE) technology has changed rather dramatically. As the principal agency with responsibility for PPE technology, the National Personal Protective Technology Laboratory (NPPTL) has come of age. In recent years, the NPPTL has forged new directions and adopted new statements of emphasis, reflected in the new National Institute for Occupational Safety and Health (NIOSH) NPPTL strategic objectives outlined in the NPPTL Strategic Plan.1,2 In light of these changes, NPPTL’s changing informational needs were considered. In this section, the committee answers the questions: What should the future data-gathering activities agenda for NPPTL be, and what should the process be for determining what these activities are and how they should be conducted?

ESTABLISHING INFORMATIONAL NEEDS

NPPTL is a new agency, and directions for the agency were formulated well after the goals and objectives of the SRUP had been established in the late 1990s. Future data collection objectives and methods should be developed keeping in mind this evolving environment and the unique focus of PPE expertise at NPPTL.

FOCUS ON OUTCOMES

A key to developing a plan for future data gathering is to accommodate the new NIOSH-NPPTL focus on outcomes. This new focus is depicted in the Value Creation System adopted as part of the NPPTL Strategic Plan (see Figure 5.1). The system provides architecture for defining, measuring, and improving the performance of the organization and focuses on outputs, intermediate outcomes, and public benefit outcomes:

  • Outputs are products and services that are the direct results of NPPTL activities; these include measurable activities such as peer-reviewed publications, protective equipment standards, and respirator certification approvals.

  • Intermediate outcomes are benefits that occur in the process of developing outputs, to include partnerships with NPPTL technology developers and suppliers.

  • Public-sector outcomes are defined to include certified PPE and PPE ensembles that provide more reliable and higher levels of employee protection and a subsequent reduction in occupational illnesses and injuries.

It is in this context that NPPTL will measure program effectiveness in the future. This will require assessing two difficult-to-measure public benefit outcomes that were specifically identified in the NPPTL Strategic Plan: (1) reductions in the incidence of work-related injury and illness related to PPE use (a real measure of the efficacy of respirators and respirator programs), and (2) reductions in economic, human, and lost opportunity costs from work-related injury and illness.

1

NPPTL, NIOSH’S National Personal Protective Technology Laboratory: Providing Personal Protective Technology Innovations for the 21st Century, NIOSH Publication No. 2004-111, May 2004.

2

NPPTL, Options for Future Surveys and NPPTL Surveillance Strategy, December 1, 2005.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 39
Measuring Respirator Use in the Workplace 5 Planning for the Future In the years since the objectives and the methodology of the Survey of Respirator Use and Practices (SRUP) were developed, and since the results of the survey were made available, the approach of the federal government to personal protective equipment (PPE) technology has changed rather dramatically. As the principal agency with responsibility for PPE technology, the National Personal Protective Technology Laboratory (NPPTL) has come of age. In recent years, the NPPTL has forged new directions and adopted new statements of emphasis, reflected in the new National Institute for Occupational Safety and Health (NIOSH) NPPTL strategic objectives outlined in the NPPTL Strategic Plan.1,2 In light of these changes, NPPTL’s changing informational needs were considered. In this section, the committee answers the questions: What should the future data-gathering activities agenda for NPPTL be, and what should the process be for determining what these activities are and how they should be conducted? ESTABLISHING INFORMATIONAL NEEDS NPPTL is a new agency, and directions for the agency were formulated well after the goals and objectives of the SRUP had been established in the late 1990s. Future data collection objectives and methods should be developed keeping in mind this evolving environment and the unique focus of PPE expertise at NPPTL. FOCUS ON OUTCOMES A key to developing a plan for future data gathering is to accommodate the new NIOSH-NPPTL focus on outcomes. This new focus is depicted in the Value Creation System adopted as part of the NPPTL Strategic Plan (see Figure 5.1). The system provides architecture for defining, measuring, and improving the performance of the organization and focuses on outputs, intermediate outcomes, and public benefit outcomes: Outputs are products and services that are the direct results of NPPTL activities; these include measurable activities such as peer-reviewed publications, protective equipment standards, and respirator certification approvals. Intermediate outcomes are benefits that occur in the process of developing outputs, to include partnerships with NPPTL technology developers and suppliers. Public-sector outcomes are defined to include certified PPE and PPE ensembles that provide more reliable and higher levels of employee protection and a subsequent reduction in occupational illnesses and injuries. It is in this context that NPPTL will measure program effectiveness in the future. This will require assessing two difficult-to-measure public benefit outcomes that were specifically identified in the NPPTL Strategic Plan: (1) reductions in the incidence of work-related injury and illness related to PPE use (a real measure of the efficacy of respirators and respirator programs), and (2) reductions in economic, human, and lost opportunity costs from work-related injury and illness. 1 NPPTL, NIOSH’S National Personal Protective Technology Laboratory: Providing Personal Protective Technology Innovations for the 21st Century, NIOSH Publication No. 2004-111, May 2004. 2 NPPTL, Options for Future Surveys and NPPTL Surveillance Strategy, December 1, 2005.

OCR for page 39
Measuring Respirator Use in the Workplace FIGURE 5.1 NPPTL Value Creation System. SOURCE: NIOSH NPPTL Strategic Plan, May 2004, p. 11. SECTORAL APPROACH In considering a data collection program for the future, the fact that NPPTL is aligning its activities along the sectoral divisions adopted by NIOSH was also considered. The NIOSH program portfolio has been organized into eight National Occupational Research Agenda (NORA) sectors that are rough approximations of industry sectors: agriculture, forestry, and fishing; construction; health care and social assistance; manufacturing; mining; services; transportation, warehousing, and utilities; and wholesale and retail trade. In addition, NIOSH has organized its efforts along 15 cross-sector programs, selected to emphasize health outcomes, statutory programs, and global efforts. These programs include two of direct relevance to formulating a PPE data-gathering program for the future: personal protective technology and respiratory diseases. As described in Chapter 3, these initiatives are under the umbrella of NORA. Through NORA, NIOSH is reaching out to a community that includes public interest, employer, and employee groups as it develops its approach to a research agenda for the future. In the summer of 2006, NIOSH was hard at work to develop a baseline of occupational safety and health information used and disseminated by business associations, professional associations, and labor unions within the eight NORA industrial sectors. The goal of this project is to develop and administer a survey of business associations, professional associations, and labor organizations in the eight NIOSH industrial sectors to determine (1) occupational safety and health (OSH) information presently being disseminated by these associations and unions to their members; (2) channels of communication within the associations and unions used to disseminate OSH information; (3) their needs for specific types of OSH information, especially those needs not presently being served; (4) sources of OSH information presently used by business associations, professional associations, and labor unions; (5) OSH concerns of the associations and labor unions; (6) awareness and perception of NIOSH as a source of OSH information; (7) use of NIOSH information services (web site, printed publications, 800 number, etc.); (8) usefulness of NIOSH information to address their OSH concerns; and (9) credibility of NIOSH as a trusted source of OSH information.3 The results of this information gathering could be used by NPPTL to sharpen its research focus and develop a more effective information dissemination program. Ultimately, NPPTL’s data needs for the future have increasingly become tactically focused on program objectives. They are more in alignment with NIOSH and NPPTL agency policy directions and more reflective of the new emphasis throughout the government on measurability, in sharp contrast to the focus on generalized public health surveillance issues of interest in the NIOSH work that preceded the SRUP (see Chapter 2). The data needs will focus more on outcomes that will inform NPPTL’s regulatory or certification and research priorities. As it adopts this new emphasis, it is expected that NPPTL data collection will extend beyond basic research, embodied in the SRUP, which provides important baseline data focused on employer respiratory protection programs. Likewise, while NPPTL may make recommendations to the Occupational Safety and Health Administration (OSHA) and the Mine Safety and Health Administration (MSHA) based on SRUP data to assist in enforcement targeting or regulatory changes to improve employer respiratory protection program compliance, enforcement data are unlikely to constitute the principal focus of future NPPTL data collec- 3 Communication with Maryann D’Alessandro, NPPTL, July 14, 2006.

OCR for page 39
Measuring Respirator Use in the Workplace tion. Indeed, NPPTL might more appropriately partner with other NIOSH divisions or extramural partners to evaluate issues related to respiratory protection programs within the context of the organization of work and management systems or programs. NEW INFORMATION NEEDS In this environment, the emerging information-gathering objectives have been stated with a programmatic emphasis.4 Some of the emerging objectives address issues that can be resolved in a survey-based data collection, reminiscent of the objectives that underscored the 2001 SRUP, but most require a new approach. RECOMMENDATION 1: NPPTL should continue to address and explicitly articulate data needs to evaluate and improve the respirator certification program so as to ensure the efficient availability and advancement of protective technologies for employees. RECOMMENDATION 2: Discussion and explicit articulation of information needs related to PPE performance and utilization should be the subject of continuous review and periodic updating by NPPTL as PPE technology evolves and the method of meeting those needs changes. It is impractical to mount and sustain an intensive effort such as that devoted to this review on an ongoing basis. However, NPPTL’s surveillance efforts necessitate a continual review and updating of data needs to support the agency’s program objectives. Imposing a continuous program of top-to-bottom consideration of reliable indicators of program objectives is a hallmark of performance-based management. The continuous review of current and emerging program objectives should be conducted with recognition of the increasing research role of NPPTL. NIOSH is, in essence, a research organization and it should approach the development of program management data with that in mind. NPPTL data-gathering activities in the future should place a sharp focus on meeting the needs of the research program, not on making findings that would inform OHSA and MSHA on the adequacy of compliance programs. Several research questions are postulated in this section and summarized in Table 5.1 in an effort to give some substance to a research agenda that focuses on the use of PPE in the field. RECOMMENDATION 3: NPPTL’s future data-gathering activities should seek, within a sound scientific approach, to derive explanations for observations on the use of PPE in the field. Furthermore, there should be a consistent framework within which these needs are considered. This framework should be aligned with the core mission of NPPTL as described above. There are many possible ways to depict this framework; however, the following is recommended: RECOMMENDATION 4: In the future, the resource-intensive data-gathering framework for NPPTL surveillance efforts should focus on the evolving mission categories of surveillance, certification, research, technology, and standards. To assist NPPTL in evaluating the various options for data collection in light of the aforementioned mission categories, the committee considered the kinds of data that would provide a useful tool for measuring program performance and guide the NPPTL research agenda. Among these data elements are the following: Surveillance How often is PPE used? What is the motivation for its use or nonuse? Is PPE used in its intended way? Are training programs adequate? Is information about PPE adequate? Does the use of PPE interfere with job performance? What kinds of jobs are performed while wearing PPE? Are the correct types of PPE being used? How important are economics in the selection of PPE? Certification Is the certification program adequate? How does certification testing match real-life job demands? What significance is paid to certified products in actual use? Should products be recertified? Can certification testing be extended to best use instead of just protection? Should certification testing be extended to best use instead of just protection? Are there minimum protection factors that can be assigned to specific pieces of PPE? Research How does PPE affect employee productivity? Are there better ways to train employees? What physical or physiological burdens are associated with PPE use? Can computer modeling be used to predict employee response to PPE? Do combinations of PPE types affect employees differently? 4 D’Alessandro, Maryann, Emerging NIOSH approaches to PPE, presentation at the meeting of the panel, open session, March 23, 2006.

OCR for page 39
Measuring Respirator Use in the Workplace What should physicians look for in a candidate for PPE wear? Technology Are there improved filter technologies? Can PPE be designed for specific types of tasks? What defines a better material? How can respirators be fully integrated with other items of PPE and with the wearer? Standards How specific should standards be? Do standards restrict the introduction of new technologies? What information is necessary before a new standard can be written? Should the standard lead technology or vice versa? Are present NIOSH standards adequate? Based on this assessment, Table 5.1 was prepared to reflect a sense of the information sources associated with these mission-related data requirements. This is an initial effort. The illustrative list of categorized data needs displayed in Table 5.1 lends itself to periodic reexamination and to prioritization within the context of the emerging strategic plans of NIOSH and the NPPTL. Table 5.1 contains a more detailed list of data that needs to be collected to achieve NIOSH and NPPTL missions, including the aforementioned broader list of data needs. The columns of Table 5.2 indicate (by an “X”) suggested types of data-gathering options available to collect the types of information indicated in the rows of Table 5.1. The options include probability surveys, administrative records, and qualitative methods. Description and comments on the most likely of these data-gathering options are presented in the next section. However, a quick reading of the table indicates that data gathered from individuals in a probability survey and from focus groups yields data representative of most data sources. POTENTIALLY USEFUL INFORMATION-GATHERING APPROACHES With this new emphasis, at the request of NPPTL, a number of information-gathering options were identified for meeting future information needs on PPE use (Table 5.2). These options included the following: Follow-on employer survey utilizing the 2001 SRUP design Redesigned SRUP employer survey Employee-within-establishment survey Combined employer and employee survey Focus groups Combined survey and focus group approach Household survey Compliance data Use of existing administrative data sources Alternative means of data collection FOLLOW-ON EMPLOYER SURVEY UTILIZING THE 2001 SRUP DESIGN Repeating a survey in the same manner in which it was originally conducted is not recommended, even though a follow-on survey based on the same survey design as the 2001 SRUP would have the benefit of providing some trend information on employer respirator programs and practices and would permit sharpening some of the concepts and making methodological improvements based on lessons learned in the original survey.5 There are a number of reasons for the committee’s lack of enthusiasm for this option. Most notable among them are numerous shortcomings in the 2001 SRUP (see Chapter 4), although many of these could potentially be avoided in a repeat application of the SRUP design (see discussion of the redesigned SRUP employer survey). Another drawback is that the survey was quite inefficient. A fairly large proportion of the data collection effort was spent on collecting information from industries in which there was relatively little respirator use, and consequently, the report had a significant number of “blank” data cells. While this shotgun approach permitted development of an estimate of total number of establishments and employees in the private sector using respirators of various types and information about the risks and respirator programs of the companies, the approach is too expensive and too limiting for assessing the new NPPTL programmatic issues. REDESIGNED SRUP EMPLOYER SURVEY Employer or establishment surveys are useful instruments for examining workplace issues. For an employer survey following the basic SRUP design concept, NPPTL may wish to identify other groups at risk for targeting research interventions to prevent occupational injury and disease that were not covered by the SRUP—the informal workforce, the self-employed (now more than a quarter of the construction workforce), and temporary and personnel leasing agencies employing persons in high-risk occupations such as construction. Increasing attention should be paid to the impact of workforce trends on the PPE research agenda, such as increased employee turnover rates in many industry 5 The utility of trend information will be somewhat diminished by the fact that the Bureau of Labor Statistics converted the industrial classification scheme from the Standard Industrial Classification used in the SRUP to the North American Industrial Classification System for subsequent Surveys of Occupational Injuries and Illnesses. Other changes include changes in the occupation coding system and changes in population weighting based on more recent counts of employment.

OCR for page 39
Measuring Respirator Use in the Workplace TABLE 5.1 Potential Sources of Information to Meet Specific Information Needs     Potential Information Sources     Information Needs Employees or Individuals Employers Manufacturers of PPE Researchers (academia, AIHA) Educators (individuals that develop training programs and train employees) Practitioners (e.g., CSP, CIH, consultants)                   Leadership or Culture             1 Do executive management and supervisors strongly support the proper use of PPE? X X       X 2 Are there any disciplinary actions that take place should an employee not use the assigned PPE? X X       X 3 Do supervisors, etc. wear the same PPE as employees reporting to them? Are supervisors and employees able to communicate when wearing PPE? How are communication challenges addressed? X X       X 4 Are supervisors required to comply with the same use requirements? Are productivity and safety performance given the same relative weights? X X     X X                   Surveillance             5 How frequently is PPE used? X X     X X 6 What is the motivation for use or any nonuse (e.g., cost, comfort, reduced productivity)? X X   X X X 7 Is PPE used in its intended way? X X   X X X 8 Is information about PPE adequate (e.g., proper fit, donning, limitations, inspection criteria)? X X X X X X 9 Does the use of PPE interfere with job performance? X     X X X 10 What tasks are performed while wearing PPE? X X       X 11 Are the correct types of PPE being used? X X   X X X 12 How would you rate wearability and comfort? X X   X   X 13 How compatible is the equipment with work tasks? What is a typical learning curve for new users? X X       X 14 Do you believe that the PPE device provides adequate protection against the hazard? X X   X   X

OCR for page 39
Measuring Respirator Use in the Workplace     Potential Information Sources     Information Needs Employees or Individuals Employers Manufacturers of PPE Researchers (academia, AIHA) Educators (individuals that develop training programs and train employees) Practitioners (e.g., CSP, CIH, consultants) 15 Are there instances in which its level of protection is so high, that PPE must be removed to perform a task (e.g., hearing protection must be removed to hear instructions)? X         X 16 Are there instances in which the PPE is so uncomfortable or cumbersome that it must be removed to perform a task? X         X 17 What is the duration and frequency of use? X X       X 18 How frequently does equipment have to be replaced? X X X X X X 19 Do employees ever share PPE? Are they aware of issues of cleaning and contamination? X X       X 20 Are training programs adequate (e.g., simply read manufacturer’s literature, hands-on training)? X X   X X X 21 Is retraining on PPE conducted periodically (e.g., annually, when process(es) change, when determined that an employee has insufficient training)? X X   X X X 22 Are there better ways to train employees?   X   X X X 23 Does the company measure the effectiveness of its training program(s)? X X       X 24 Are PPE maintenance, cleaning, storage, and other program components implemented and effective? X X       X                   Selection             25 Are hazard and risk analyses performed before deciding which PPE is necessary?   X   X   X 26 Who selects the supplier? Do employees get to choose between various brands or suppliers? X X       X 27 What is the most important criterion in the selection of PPE (e.g., cost, style, level of protection, availability)? X X X     X 28 What weight is given to certified products in actual use? X X X X X X

OCR for page 39
Measuring Respirator Use in the Workplace     Potential Information Sources     Information Needs Employees or Individuals Employers Manufacturers of PPE Researchers (academia, AIHA) Educators (individuals that develop training programs and train employees) Practitioners (e.g., CSP, CIH, consultants) 29 Do you have a designated person in each department who helps employees with reviewing their equipment and purchasing what they need or want? X X       X 30 Is the adequacy of selected PPE validated with monitoring or observation? X X       X                   Certification             31 Is the PPE certification program adequate to distinguish reliably between respirators providing adequate protection and those that do not?     X X X   32 How does PPE certification testing match with real-life job demands?     X X X X 33 Should products be recertified on a periodic basis?     X X X   34 Can certification testing be extended to best use instead of just protection (e.g., recommended conditions of use and use life, storage and shelf life)?       X X X 35 Are there minimum protection factors that can be assigned to specific pieces of PPE?     X X X X 36 Should manufacturers’ labeling be standardized to facilitate communication and comparisons? What about symbolic information? X X X X X X 37 What are the impacts of overlapping government approvals (e.g., Food and Drug Administration for surgical N95 respirators)?   X X     X 38 What are the impacts of gaps in certification (e.g., no regulation for consumer use respirators)? X X X     X                   Standards             39 How specific should standards be?     X X X   40 Do standards restrict introduction of new technologies?     X X    

OCR for page 39
Measuring Respirator Use in the Workplace     Potential Information Sources     Information Needs Employees or Individuals Employers Manufacturers of PPE Researchers (academia, AIHA) Educators (individuals that develop training programs and train employees) Practitioners (e.g., CSP, CIH, consultants) 41 What information is necessary before a new standard can be written?     X X   X 42 Should the standard lead technology or vice versa?     X X   X 43 Does conformance with the standards significantly reduce exposures or improve employee safety and health?     X X X X 44 How is conformance verified or confirmed?     X X   X 45 Does the standard improve consistency of performance across sites or manufacturers?     X X   X 46 Does the standard facilitate performance comparisons across sites or products?   X X     X 47 Do instructions, labeling, or design increase ease of use or reduce the learning curve for conforming products? X X         48 Are conforming products easier to fit, do they retain fit better, or do they provide better fit for difficult-to-fit subpopulations? X X   X   X                   Research             49 How does PPE affect employee productivity and the learning curve for new users? X X   X X X 50 What physical or physiological burdens are associated with PPE use? X X X X X X 51 Can computer modeling be used to predict employee response to PPE?     X X   X 52 Do combinations of PPE types affect employees differently? X X X X X X 53 What are some of the human factors that should be taken into consideration by physicians and other health professionals in assessing the protection afforded by PPE?     X X X X 54 What are the most common causes of equipment failure? Are these failures easily repairable in the field application? X X X X   X

OCR for page 39
Measuring Respirator Use in the Workplace     Potential Information Sources     Information Needs Employees or Individuals Employers Manufacturers of PPE Researchers (academia, AIHA) Educators (individuals that develop training programs and train employees) Practitioners (e.g., CSP, CIH, consultants) 55 How compatible is the equipment with work tasks? X X         56 How durable is the equipment? How long is the shelf life? X X X     X                   Advances in Technology             57 Are there improved filter technologies?     X X     58 Can PPE be designed for specific types of tasks? Can task-based performance be assessed?     X X     59 Material advances for PPE applications (sorbents, catalysts, antimicrobial materials, polymerics)     X X     60 How can PPE be fully integrated with other items of PPE and with the wearer?     X X   X 61 Are technical advances in PPE driven or inhibited by the standards and certification process?     X X     NOTE: AIHA = American Industrial Hygiene Association; CIH = Certified Industrial Hygienist; CSP = Certified Safety Professionals. TABLE 5.2 Comparison of Information-Gathering Methods by Data Sources   Probability Surveys Records Qualitative Methods Data Sources Establishment Individual (e.g., employee, household) Employee Within Establishment OSHA Compliance Data Focus Groups Observation Employees   X X   X X Establishments X   X X X   Manufacturers X       X   Researchers   X     X   Educators   X     X   Practitioners   X     X  

OCR for page 39
Measuring Respirator Use in the Workplace sectors; decreased establishment lifetimes; increased multiemployer worksites with subcontracting and outsourcing, including virtual establishments where essentially all of the production work is outsourced; and increased immigrant employees. These all present surveillance challenges. Our national data collection systems can no longer assume long-term employment; stable establishments operating in the location where production occurs; establishments that directly employ individuals rather than subcontracting, leasing, or using temporary agencies; establishments that perform work in a single industry code; establishments that exist long enough to collect data rather than project-based limited-liability corporations; and establishments that have land line telephones rather than usually unlisted cell phones and Internet phones. In redesigning the SRUP, NPPTL should consider limitations for specific respirator user subpopulations in the survey design. Respirators and other PPE are appropriately used where other control options are not feasible or while those controls are being implemented. Therefore, they tend to be used in higher-risk and less controlled work environments. These high-risk subpopulations (firefighters, health employees, miners, and construction employees) are an appropriate focus of NPPTL research. With limited resources, surveillance that allows reliable estimates of use in every industry sector may not be necessary. Preliminary targeted data collection such as focus groups or surveys of high-risk or heavy users of specific products may provide sufficient information for some program management purposes. Targeted assessments might also identify specific surveillance challenges where the definition of establishment based on unemployment insurance and payroll locations is not descriptive of employer, worksite, or work risks or tasks. Single employers that are multiple establishments, mobile employers, project-based limited-liability corporations, personnel services, and virtual employers present challenges in national surveillance that might be addressed in targeted settings. Clearly, NPPTL should take care to avoid the methodological shortcomings of the 2001 SRUP. In selecting a sample frame, NPPTL should consider the entire workforce, not simply those employed in establishments covered by unemployment insurance laws. To the extent feasible, any future SRUP sample frame of establishments should include those sectors that are specifically excluded from federal OSHA regulatory oversight, including the self-employed, informal-sector employees who are paid cash, and domestic employees. Others who may be covered by regulation plans but not by OSHA or MSHA including railroad and airline employees, seafarers, longshore and harbor employees, and public employees should also be covered. For example, self-employed construction workers constitute approximately one-quarter of the construction workforce and are very likely a high-risk population with a high rate of respirator use. Similarly, firefighters and police are likely to have high rates of respirator use but were excluded from the SRUP. This exclusion would be particularly inappropriate given NPPTL’s expanded mandate related to PPE for chemical, biological, and radiological hazards and the concentrated use of these devices by public-sector first responders, the military, and other public employees. Recent changes in Bureau of Labor Statistics (BLS) survey coverage have permitted consideration of expansion of the survey to firefighting, corrections, public-sector waste management, and other key service sectors in the 28 states that collect their own Survey of Occupational Injuries and Illnesses (SOII) data. At the time of the SRUP, BLS had not surveyed the public sector because there was no OSHA recordkeeping requirement that would elicit a sampling frame. However, BLS is now testing its ability to use the public-sector establishments in the Longitudinal Data Base (LDB).6 NIOSH and BLS are conducting a survey of U.S. workplaces to evaluate employers’ perspectives regarding policies, training, and other related issues on workplace violence prevention, including risk factors associated with workplace violence and prevention strategies. The public-sector firms in the LDB are being sampled for this survey. It is hoped that this experience will determine if it is possible to extend coverage to the pubic sector in all states in future special surveys.7 NPPTL may wish to more selectively target future data collections to emphasize sectors of interest: Agriculture, Forestry, and Fishing. The use of respiratory protective equipment in wildland firefighting is a significant issue in this sector. Wildland firefighting crews are subject to inhalation hazards from combustion by-products and carbon monoxide, and there is a perceived reluctance on the part of the wildland firefighting community to accept the need for respiratory protection.The National Fire Protec- 6 The LDB is the universe from which BLS draws the Current Employment Statistics survey sample. The LDB contains data on the approximately 8 million U.S. business establishments covered under the Unemployment Insurance System, representing 97 percent of all employers in the U.S. economy. There are a few sections of the economy that are not covered, including the self-employed, unpaid family employees, railroads, religious organizations, small agricultural employers, and elected officials. Data for employers generally are reported at the worksite level. The Quarterly Census of Employment and Wages program collects data from employers, on a quarterly basis, in cooperation with state employment security agencies. The LDB contains employment and wage information from employers, as well as name, address, and location information. It also contains identification information such as Unemployment Insurance account number and reporting unit or worksite number. 7 A recent study has estimated the extent of respirator use in the U.S. Postal Service and among state and local governments. In support of a recent rulemaking, OSHA made estimates based on respirator use patterns in “like ” private-sector establishments. Extrapolating the 2001 SRUP results, OSHA estimated that more than 1,000 establishments in the Postal Service and nearly 6,900 establishments in state and local governments had respirator users. OSHA, 29 CFR Parts 1910, 1915, and 1926, Assigned Protection Factors: Proposed Rule, Federal Register, June 6, 2003, pp. 34073-34074.

OCR for page 39
Measuring Respirator Use in the Workplace tion Association has considered a performance standard for wildland firefighting respirators. Service Sector. A great deal of work has gone into understanding the hazards and PPE protection requirements for emergency response personnel in the development of chemical, biological, radiological, and nuclear (CBRN) improvements after September 11, 2001, and some of that work has focused attention on the problem of inhalation of combustion by-products during initial fire knockdown and cleanup. Other issues are associated with exposure to respiratory hazards, such as smoke, soot, and airborne particles following the use of self-contained breathing apparatus. In the firefighting sector, there are some alternative sources of information that can be evaluated, including International Association of Fire Fighters and other firefighter data. Employees in waste management and remediation experience exposure to inhalation hazards; those in correctional institutions are exposed to biological particulate and aerosol threats from inmates and detainees; and ambulance service personnel are exposed to sick and contaminated persons during treatment and transport. Mining. The issues involved with respirator use in mining are somewhat more complex than in most other sectors. Mining is generally acknowledged to have the most advanced programs supporting respirator and breathing apparatus usage and the highest compliance rates. Yet current data indicate that the mining industry is associated with the highest rates of pneumoconiosis mortality. This introduces the complication of measuring respirator use in an environment in which mortality rates are the result of past exposures rather than current practices. Other issues are unique to mining. The tragedy at the Sago Mine in West Virginia that occurred in January 2006 underscored the importance of training and emergency preparedness, as well as the ready availability of PPE in mines. Additional details about self-contained self-rescuers in mining are also of interest, questions such as how many, where they are located, their capacity, their condition, their frequency of use, and their use in-place within the mine as opposed to use in escape. Construction. The NIOSH strategic goal with regard to construction is to enhance surveillance to support identification of emerging technologies and associated hazards, evaluation of intervention effectiveness, and identification of future priorities for protecting construction employees. Based on a peer review of construction surveillance conducted in 2004, NPPTL has adopted the approach of bringing together focus groups and targeting interventions rather than wide-scale survey operations for this industry. The focus groups are yielding valuable information about respirator usage from the perspective of the user, as well as the companies. The uniqueness of construction compounds the difficulty of obtaining valid information on respirator usage in a survey operation. For example, there is considerable movement of employees between employers in construction. Except in the case of a hiring process managed in a controlled environment such as a union hiring hall, this turnover vitiates direct association of a particular company’s respirator program characteristics with individual protection outcomes. High employee turnover and intermittent employment also present challenges for medical clearance for respirator use, training, and other aspects of effective respirator programs. Manufacturing. The manufacturing sector continues to present a variety of challenges for respiratory protection research. The approach of NIOSH to this industry relies on exploiting data from existing programs, such as the Sentinel Event Notification System for Occupational Risks (SENSOR) program, and focusing on specific identifiers available in the NIOSH Employee Health Chartbook to better identify risks in manufacturing.Emerging new hazards, such as those that may be associated with nanotechnologies, require vigilance in adapting surveillance systems to meet changing demands. Health Care. In health care, the requirement for respiratory protection is most often associated with protection from infectious diseases, important for both the health care provider and the patients. In addition, there is concern with smoke and fumes generated in the operating rooms of hospitals. The risk is evidenced by the estimate that the proportion of health care employees using respirators was higher than in all industries except mining, construction, and manufacturing.9 Additionally, NPPTL has identified risks in emergency departments to be of special interest. Parallel approval mechanisms for surgical N95 respirators by the Food and Drug Administration (FDA),10 which may be confused with NIOSH-NPPTL approval but are intended to protect patients, should be the subject of a scientific assessment that would compare the rigor and implications of the two systems of approval and certification. The fact that FDA approval is based on test data submissions by PPE manufacturers also offers an opportunity for public policy comparisons. Given the strategic goals of NPPTL to consider possible interference within PPE “ensembles” and the requirement to consider ensembles for CBRN, the new survey should also consider other types of PPE. In particular, PPE that may interfere with respiratory protection should be considered including hearing protection, eye and face protection, and hoods or helmets intended to be used simultaneously. Ideally this surveillance will assist in defining future research priorities related to evaluating ensembles. 8 NIOSH, NPPTL, Peer Review of Construction Surveillance, 2004. 9 BLS/NIOSH, Respirator Usage in Private Sector Firms, 2001, Table 7, pp. 26-27. 10 See http://www.fda./CDRH/PPE/.

OCR for page 39
Measuring Respirator Use in the Workplace that a trained moderator can explore related but unanticipated topics as they arise in the discussion. However, the results obtained from focus groups must be considered with some caution. They are limited because the sample is neither randomly selected nor representative of a target population, so the results cannot be generalized to the general population and the data cannot be treated statistically as would a survey sample. For example, the selection of management and union member employees from the three construction industries may have biased the results because the focus members were not representative of the bulk of employees in construction. The results of the focus groups should be taken to represent only the participants of the group. The quality of the data is further influenced by the environment for the group and by the skills and motivation of the moderator. The results are also subject to the influence of the make-up of the particular focus group. Volunteer respondents may very likely have more interest in a topic and perhaps more knowledge about it. Focus groups are useful methods for identifying topics and issues that should be examined with more statistical rigor, particularly when policy decisions are at issue. Nonetheless, focus groups have their place in a repertoire of tools for understanding the use of PPE in the workplace. COMBINED SURVEY AND FOCUS GROUP APPROACH Yet another approach to gathering information of use in guiding NIOSH programs is represented in the NIOSH approach to evaluate the impact of the Fire Fighter Fatality Investigation and Prevention Program (FFFIPP) and the effects of the FFFIPP recommendations and information products that are periodically distributed to the nation’s 30,000 fire departments.15 This study examined career and volunteer, large and small size, and urban and rural fire departments to determine the extent to which firefighter reports, recommendations, and other information products are being implemented by fire departments. This evaluation also measured the effects of FFFIPP on the knowledge, behavior, attitudes, and safety practices of fire department management. The survey combines a fire department survey (Tier 1) and focus groups (Tier 2). The fire department survey used a cross-sectional design with restricted random sampling16 to include 215 fire departments where there had been an investigation of a fatal incident. The survey included a random sample of 300 fire departments where there has not been any such investigation and, with certainty, the 10 largest fire departments. The random selection of additional fire departments was designed to balance various factors such as the number of volunteer versus career, or rural versus urban firefighters, and other considerations. The evaluation team qualitatively supplemented findings from the Tier 1 survey with a series of six focus groups with firefighters from across the country in order to explore ways in which FFFIPP may have affected firefighter knowledge, attitudes, behaviors, and safety practices. Focus groups are also used as an important tool in the development of questionnaires and other facets of the design of population surveys. For instance, they might be used during the planning phase of a survey to identify questionnaire content, to develop and test survey questions, or to understand respondent reluctance to participate. HOUSEHOLD SURVEYS While not exactly a “rare” population, the 3.3 million employees17 who would be likely to use respirators for required purposes are a very small proportion of the overall workforce—just 3 percent of all private-sector employees. Finding them in a large-scale household survey would be a daunting and expensive undertaking. The survey operation would have to screen out 97 employees to identify the 3 who are in the population of interest. The likelihood of finding the employees themselves, so that information about respirator use can be obtained directly (self-response), adds a further dimension of difficulty to the possibility of employing a household-type survey for this data collection. Thus, a standalone household survey of employee respirator use would be inefficient and probably cost-prohibitive. Nonetheless, the allure of gathering data that may be crossed by other characteristics of interest generally gathered in household surveys (occupation, demographic characteristics, employment status, and hours worked) suggests that it might be fruitful to explore the possibility of adding a brief set of questions about employee respirator use onto an existing household survey, particularly if the screening can plausibly be conducted by telephone. Another approach would be to screen the sample from a large existing health survey for affected employees. One survey that commends itself to service as the collection vehicle for a supplementary set of inquiries about respirator usage is the National Health and Nutrition Examination Survey (NHANES). Conducted annually by the National Center for Health Statistics, NHANES gathers data from a nationally representative sample of about 5,000 persons each year to assess the health and nutrition status of adults and children in the United States. The survey is quite unique in that it combines interviews with physical exams. The interviews have questions regarding the demographic, socioeconomic, 15 http://a257.g.akamaitech.net/7/257/2422/06jun20041800/edocket.access.gpo.gov/2004/04-28609.htm. 16 Restricted random sampling is the arrangement of sample units within blocks or clusters. 17 As estimated by the SRUP. This estimate, however, omits public-sector and other types of employees who would be accessible in a household survey and increase the percentage of eligible households, although it is unknown by what amount.

OCR for page 39
Measuring Respirator Use in the Workplace and health status of the population. Although the limited size of the sample limits its utility in addressing the prevalence of respirator use in the workplace, the possibility of collecting information on respirator usage, and occupational exposure, in conjunction with demographic and health information, recommends this survey for consideration. A shortcoming of employing an existing survey to collect data on employees who use respirators is that the sample designs will not be efficient for providing a good representation of employees from industries of particular interest. It would not be possible to oversample selected industry sectors. This would limit the amount of possible subgroup analyses. If such a survey were used, it might best be done in conjunction with the special surveys focused on particular industries (described above). If it were possible to collect reliable data from a household-type survey, it would be useful to extend those data by linking household or individual data to an employer survey. A complex but interesting example might be the way the National Health Interview Survey and the Medical Expenditure Panel Survey interlink several surveys including household, insurer, and health care provider. Other large household surveys that might be screened in this way would be the Behavioral Risk Factor Surveillance System survey, the Current Population Survey, and the American Community Survey. If the objective of the survey of households were simply to screen and identify a population that is likely to represent employed workers in respirator-using industries, another technique would be to use a data collection mechanism developed by the National Center for Health Statistics known as the State and Local Area Integrated Telephone Survey (SLAITS). Designed to provide in-depth state and local health care information, this mechanism uses the sampling frame from the National Immunization Survey, an ongoing telephone survey that screens nearly one million households per year to produce estimates of vaccination coverage levels among children. SLAITS provides a mechanism, using a random digit dial telephone design, to collect data quickly on a broad range of topics at the national, state, and local levels. A partial list of examples of research areas includes health insurance coverage, access to care, perceived health status, utilization of services, and measurement of child well-being.18 In the United Kingdom, the Health and Safety Executive conducted a Workplace Health and Safety Survey of households in 2005. The survey was administered by telephone, with households selected by random digit dialing and a respondent selected randomly from household members who worked at some time in the 12 months prior to interview. The survey collected information on the number of employees exposed to different workplace hazards and employee’s concerns, training levels, and view of trends in risk. Although the survey collected information on exposure to dust and fumes, and about training and information on these hazards, no information was collected about the use of respirators or other PPE. The results from this survey are based on responses from 10,016 British employees, a response rate of only 26 percent.19 USE OF OSHA COMPLIANCE DATA It is fitting to consider the administrative data collected as a result of the OSHA compliance program as a potential source of information on respirator use in the workplace. After all, most of the questions posed in the various statements of purpose for the NIOSH-BLS survey pertained to whether or not the employers were compliant with OSHA regulations. It would seem that an ideal way to measure compliance would be to use the results of compliance inspections. The data in the OSHA Integrated Management Information System reflect that from October 2004 through September 2005, the agency issued more than 4,500 federal citations for respiratory protection (Table 5.3). The year before, citations issued by OSHA for violation of the 1910.134 respiratory standards were the fifth highest on the list of 378 different types of violations.20 Special emphasis programs at the area office or regional level might also allow narrow targeting. Unfortunately, for a number of reasons the use of OSHA compliance data is not a very promising avenue of approach to illuminate issues of respirator use in the workplace. The 1998 NIOSH Respirator Surveillance Team concluded as much when it stated that OSHA compliance data regarding respirators are too limited to be of significant utility in addressing the issues detailed in Chapter 3. For one thing, much like BLS data, OSHA compliance data do not address respirator use per se. The inspections reflect the faithfulness of the establishment’s implementation of a program in compliance with the regulation. Most importantly, OSHA compliance data are unlikely to be representative of the population of companies that should have respirator programs. OSHA targets its programmed inspections by first identifying all establishments that were in four-digit Standard Industrial Classification categories with reported lost work time injury rates above a state’s manufacturing average. Then, it chooses sites randomly within this set, excluding where there are fewer than 11 employees or where an inspection was conducted in the last three years.21 Inspections may also be targeted based on “special emphasis programs at the area office level, or based on reported lost work time injuries” (which may 18 http://www.cdc.gov/nchs/slaits.htm. 19 Hodgson, J.T., J.R. Jones, S.D. Clarke, A.J. Blackburn, S. Webster, C.S. Huxtable, and S. Wilkinson, Workplace Health and Safety Survey Programme: 2005 Employee Survey First Findings Report, Health and Safety Executive, Caerphilly, UK, May 2005, p. i. 20 Doney, op. cit., p. 275. 21 Mendeloff, John, and Wayne B. Gray, Inside the black box: How do OSHA inspections lead to reductions in workplace injuries? Law and Policy April:221-223, 2005.

OCR for page 39
Measuring Respirator Use in the Workplace TABLE 5.3 Federal OSHA Citations by SIC Division, October 2004-September 2005 No. Cited No. Inspected Penalty (dollars) Industry Classification 2,449 1,104 930,785 Division D/Manufacturing 939 348 152,134 Division I/Services 592 309 231,139 Division C/Construction 206 92 78,053 Division E/Transportation, Communications, Electric, Gas, and Sanitary Services 116 50 40,173 Division F/Wholesale Trade 98 43 8,070 Division G/Retail Trade 44 16 28,063 Division B/Mining 23 8 3,750 Division J/Public Administration 22 10 3,365 Division H/Finance, Insurance, and Real Estate 15 4 2,987 Division A/Agriculture, Forestry, And Fishing SOURCE: http://www.osha.gov/pls/imis/industryprofile.stand?p_esize=&p_stand=1910.134&p_state=FEFederal&p_type=1. or union representative about an alleged violation or hazard at the workplace, although these may not involve site visits. While these selection criteria may yield a population of great interest to OSHA, the process is designed to produce a population that is not representative of the overall population of employers; thus, the compliance data cannot be used to make generalizations applicable to the overall population of respirator-using establishment. Since these data are not collected for research purposes and OSHA inspectors are provided no training on proper coding or data quality assurance, considerable effort would be required to evaluate the reliability of this dataset. USE OF EXISTING ADMINISTRATIVE DATA SOURCES Several of the alternative sources of data that hold promise for illuminating the issues in respirator protection have been discussed in the proceeding sections, including sources such as the International Association of Fire Fighters surveillance data and the SENSOR program data. Many of these data sources are summarized in easily accessible form in the NIOSH Employee Health Chartbook. The 1998 NIOSH Division of Respiratory Disease Studies (DRDS) Respirator Surveillance Team report concluded that currently available sources are fragmentary and not likely to be useful for purposes of assessing respirator use in industry,22 and there is no indication that these fragmentary, and thus unrepresentative, sources have evolved since then in ways that would provide a more comprehensive view of respirator usage in the workplace. However, the DRDS team found that OSHA, MSHA, and company annual reports indicated the possibility that a special effort to obtain unpublished information that may be available from these parties would partially meet the need for assessing respirator use. The team suggested eliciting this unpublished, nonproprietary information in focus groups or like venues. ALTERNATIVE MEANS OF COLLECTION OF DATA Several other possible means of data collection were contemplated. They are listed here for information only. NPPTL may wish to consider the following: Some form of postmarket surveillance of respirator use or performance similar to the FDA postmarket surveillance (21 CFR 822) used for medical devices, which is based on section 351 of the Public Health Service Act.23 This sort of reporting requirement might require modification of regulations related to respirator certification. A database of user complaints to identify end user problems with specific respirator performance, durability, instructions, labeling, and so forth. Improved technology and software for web-linked databases would make such an effort relatively inexpensive. This would cost considerably less than end user or user establishment surveys and could provide ongoing feedback to NPPTL. The ability of NPPTL management to observe trends may be more valuable than a much more detailed survey that might not be repeated. Some form of user survey (include coupon or lottery cards in each box with web address and prizes for those that go to the site and fill out the questionnaire in order to enter the contest or obtain the prize). These would be inserted into boxes by manufacturers as a condition of future certifications. This would be a low-cost supplement to other surveillance databases that could be linked through a specific respirator model or certification number. Telephone follow-up of specific PPE users treated in emergency rooms and reported in a data collection system akin to the Consumer Product Safety Commission’s National Electronic Injury Surveillance System (NEISS) data- 22 DRDS, op. cit., 1998. 23 See http://www.fda.gov/cdrh/devadvice/352.html.

OCR for page 39
Measuring Respirator Use in the Workplace base. NEISS is a national probability sample of hospitals in the United States and its territories. Patient information is collected from each NEISS hospital for every emergency visit involving an injury associated with consumer products. From this sample, the total number of product-related injuries treated in hospital emergency departments nationwide can be estimated. In this manner, follow-up information can be captured within a couple of days after emergency department visits, and detailed telephone interviews initiated. NIOSH supports the collection of work injury and employment data from a subset of these emergency departments through a memorandum of understanding with the U.S. Consumer Product Safety Commission, but PPE would have to be addressed in targeted follow-back with injured employees.24 This would constitute a small sample, but it would yield potentially rich data related to acute injury. Collection of data from respirator users on federal contracts or procurements. This could be inserted as a contractual obligation. Mandatory response has definite advantages if the limitations or characteristics of the sample can be defined clearly. Analysis of PPE-specific data from Chemical Safety Board investigations of industrial disasters that often involve significant PPE use information. Using state or federal workers’ compensation claims or data from large private owners involved in owner-controlled insurance plans on multiemployer sites to identify injured employees who could be asked about injuries involving the use of or failure to don PPE. Targeting specific large projects and/or randomly selecting and investigating every employer on a specific city block or census code. The United Kingdom’s Safety Executive uses this “blitz” approach to identify small employers with some success. Conducting unannounced observational visits to worksites to ascertain worker practices with regard to wearing and maintaining respirators. This concept is currently the approach of an organization such as the Joint Commission on Accreditation of Healthcare Organizations. This organization no longer asks questions in advance or announces visitation in advance. Observers arrive spontaneously to actually determine what is being done. They look for gaps between what was to be done and what is done. While these options offer the benefit of relative availability and the allure of relatively inexpensive access, it should be clearly noted that findings representative of the general population from such sources are highly unlikely, thus seriously limiting their utility in determining what is happening in the general populations of employers and employees. A COMPREHENSIVE SURVEY PROGRAM FOR THE FUTURE To meet future NPPTL information needs that are best met by gathering data from employer and/or employee surveys, it is plausible and necessary to build on experience from the SRUP and other comparable recent surveys to develop a more useful, flexible, and scientifically rigorous survey data-gathering capacity. As evident from the committee’s review of information needs related to PPE use, a well-designed series of periodic surveys could provide NPPTL with most if not all of the population-based data it will need in the next several decades. Data on respirator use are best provided by employers and employees in the context of the work setting. RECOMMENDATION 5: NPPTL should develop for implementation an ongoing survey of employers and employees whose overarching goal is to obtain needed information on the use of respirators and other PPE in the United States. To ensure the scientific quality and broad utility of this series of surveys, NPPTL should ensure that this system of periodic surveys Is sufficiently flexible to meet NPPTL’s changing information needs; and Attains to the highest standards of current survey research by Following a responsive sample design, Ensuring the reporting accuracy of all of its key survey measures, Widely disseminating its findings, Making its data easily available to outside researchers, and Documenting all of its methods in detail. Several survey design options should be considered when determining the best approach to obtaining data on PPE use from employers and employees. Employer data might be collected by mail, Internet, telephone, or in person from a sample of employers only, or from the first-stage sample of employers in a two-stage sample of employees. Employee information might be gathered from employees in the second stage in this same two-stage design. When there are significant barriers to collecting employer and employee information from the same sample, a split sample of employers might be used with one employer subsample designated for employer data gathering and the other subsample being used to collect data from employees. The design features described below could, for example, be used as the basis for developing a series of design-related periodic surveys that provide both detailed benchmark data from large in-depth survey rounds conducted every 5-10 years and more basic trend data from smaller rounds conducted in the intervening years. Another alternative might be to use the initial cross-sectional survey of employees as 24 The NEISS Sample, see http://www.cpsc.gov/neiss/2001d011-6b6.pdf.

OCR for page 39
Measuring Respirator Use in the Workplace the basis for a panel design in which employees would be recontacted periodically and interviewed. While this introduces the costs and complications of panel maintenance (e.g., panel conditioning, respondent tracking, attrition), it might provide some data on change over time. It would also permit follow-up on health changes over time, a benefit not available from the other methods. If a major survey can be conducted only every 5-10 years, a panel might have some appeal. Regardless of actual details of the final design of the system for these surveys, the underlying assumption in each periodic survey is that the information needed by NPPTL is best obtained from employers or from the employees themselves.25 Two key requirements of these periodic surveys must be that (1) their designs are rigorously developed and (2) they are flexible in both design and implementation. While the basic designs can be sketched here, a number of important design issues would emerge in developing the design of these future surveys. Among them would be to rigorously answer important design questions concerning the best and most practical Employer or establishment frame; Sample size; Allocation of the sample among stages and strata; Mode or mixture of modes to collect the needed data; Wording of individual survey questions (particularly those used to produce the key outcome measures of interest); Survey instrument(s) design; Means to obtain high-quality measurements of PPE use; Estimation, reporting, documentation, and dissemination approaches; and Methods for assessing data quality. Resolving these important design questions in a deliberate fashion would require a planning period of two to three years to answer these questions rigorously before regular survey implementation can commence. In real terms, this planning effort should be led by appropriate NPPTL staff, but with significant external input from sample and questionnaire design experts in survey research methods as well as outside experts in all subject matter areas that are relevant to the purposes and content of these surveys. After the specific needs to be met by these surveys have been articulated, a detailed list of survey objectives would be set, and a preliminary draft of key survey questions would be developed. Activities during this planning period might include any or all of the following: statistical design studies to address sampling issues, in-depth cognitive testing of all key survey questionnaires and forms, rigorously designed and conducted field tests to resolve key survey design issues (e.g., the most accurate way to measure PPE use, alternative contact persons, what level of financial or token incentives to offer employers and/or employees), issues in using multiple modes of data collection, and large-scale field tests to act as a “dress rehearsal” for the final design plan(s). Finally, during the implementation phase of these survey designs, there should be an occasional formal assessment of both the designs and the data-gathering processes associated with them to identify and systematically resolve any unanticipated issues in conducting these surveys. In sum, the design is a project in itself. A second key requirement of the recommended periodic surveys is that they must be adaptable to changes in and evolution of NPPTL information needs over time. More specifically, the framework for selecting the sample for each round must be able to accommodate specific statistical needs at that time. For instance, if a round must focus more intensively on findings in certain work sectors, the sample must be stratified by those sectors so that larger sampling rates can be directly applied to disproportionately sample each sector of particular interest. The design must also readily allow for reasonable statistical representation of those from whom data are collected. Moreover, the data collection plan must move easily between collecting information from employers, employees, or both. Design for a Survey of Employers Only Sample Design When data are sought only from employers, conventional list sampling methods are frequently appropriate. In this type of survey design there is a one-to-one link between those sampled and those observed. For surveys of business establishments, this means that a sample of employers is randomly chosen from an appropriate list frame (of employers) and that employer-level data are collected from those selected in the sample. The sample size in these and all samples should be adequate to meet clearly prespecified quantitative measures of needed statistical precision and/or statistical power for key survey estimates (e.g., relative standard error, margin of error, power to detect subgroup differences of some specified amount). Some form of stratified simple random sampling is often most useful to identify those employers from whom data will be gathered. Strata used in this way might be defined (1) by employer characteristics that are highly correlated with the key study outcome measures 25 More in-depth treatment of the survey methods issues and techniques mentioned in this section is given by the following: Kish, Leslie, Survey Sampling, New York: John Wiley and Sons, 1965; Lohr, S. Sampling: Design and Analysis, California: Duxbury Press, Pacific Grove, 1999; Kalsbeek, W., and G. Heiss, Building bridges between populations and samples in epidemiological studies, Annual Review of Public Health 21:1-23, 2000; Groves, R.M., F.J. Fowler, M.P. Couper, J.M. Lepkowski, E. Singer, and R. Tourangeau, Survey Methodology, New York: Wiley & Sons, 2004; Kalton, G., and I. Flores-Cervantes, Weighting methods, Journal of Official Statistics 19(2):81-97, 2003.

OCR for page 39
Measuring Respirator Use in the Workplace and/or (2) by the concentration of one or more population subgroups whose findings are of particular interest and thus may be disproportionately sampled. Sample allocation in a stratified sample might be proportionate or optimal (following Neyman allocation), if overall population estimates are the main priority in analysis, or disproportionate to facilitate focusing findings on important population subgroups by oversampling those subgroups. Stratification might also be used to facilitate screening down to a particular type of business establishment, such as establishments with organization of work or safety culture characteristics associated with noncompliant PPE use. For example, when focus in a round is limited to establishments whose employees actually use PPE, one might isolate those types of establishments that are most likely to use PPE and then disproportionately sample them. Stratification in this case reduces the cost of screening out those employers with employees who do not use PPE. However, it does not effectively segregate establishments that do not use respirators, but should. In a pilot sample, stratification based on other criteria, such as risk rather than respirator use, should be considered. The quality of the business establishment frame is an important determinant of the representativeness of the sample and the utility of the findings obtained from the sample. The respondents to the SOII were used for employer sampling in the SRUP, but as this review has found, the choice of this sample frame limited coverage of some groups of interest and, importantly, limited sponsor access to the survey data due to the need to protect the confidentiality of the list of businesses. In future surveys, NPPTL should explore using other employer frame sources. These include business lists available from private vendors, such as Dun and Bradstreet and InfoUSA. If data access is not an issue, NPPTL may wish to explore contracting with the U.S. Bureau of the Census to have access to the lists maintained by that agency.26 Some criteria for determining which list to use should include coverage of the targeted population of interest, availability of information on workforce size and measures used for stratification, availability of current and accurate locator information, relative ease and cost of use, and whether or not access to individual records is required. Balanced against these considerations is the potential benefit of again using the list of SOII respondents as a sample frame if it becomes possible to capture the potential benefits of correlated ancillary information (from the SOII) that can improve the precision of survey findings through the use of ratio or regression estimation. This design might be expanded to three (or more) stages if it were more practical to sample employers and employees separately within a sample of local area units such as counties or census tracts, which in this case would become the target areas for the sampling, also known as primary sampling units (PSUs). When in-person visits and/or interviews are required by the survey design, sampling area units first in this way would decrease the cost of data gathering by geographically confining the field operation and thus reducing travel costs. Data Collection Plan In terms of methodology, any future establishment survey should be based on a set of protocols. RECOMMENDATION 6: The most efficient data collection approach for the establishment survey is a mixed-mode design involving three phases: (1) an initial round of telephone screening calls, (2) a mail survey approach, and (3) telephone follow-up calls to nonrespondents. The initial round of telephone screening calls has the following objectives: (1) to identify and remove from the sample those establishments that are no longer in business, (2) to identify establishments that are PPE users, (3) to identify and close out as “completed” non-PPE user establishments, and (4) to make contact with the best-qualified respondent at PPE user establishments. The calls should be made by well-trained telephone interviewers in the survey contractor’s centralized call center. A computer-assisted telephone interview (CATI) system should be used to control the interview process, schedule calls, and manage the caseload. The interviewer should ask to speak to the person most knowledgeable about possible PPE use at the establishment. RECOMMENDATION 7: Instead of addressing the survey to someone with unknown expertise in the area of interest, surveillance personnel at NPPTL should conduct a quick telephone screening of sampled establishments in advance of the mail survey to identify the best-qualified respondent and to learn from that person whether the establishment is eligible for the full survey (that is, required use of respirators in the past 12 months). Once contact is made with this person, the interviewer would determine the PPE status of the establishment and, for PPE establishments, explain the survey in more detail and encourage the person to complete and return the questionnaire when it arrives. The mail survey phase should be patterned after Dillman’s Tailored Design Method, discussed earlier. Three mailings should be used: (1) an initial questionnaire mailing, sent promptly after the advance screening call to the respondent 26 For an evaluation of the major government and the Dun and Bradstreet lists as sample frames, see Cox, Brenda, Quality of U.S. Business Establishment Frames: Discussion, Proceedings, Survey Methodology Section, Washington, D.C.: American Statistical Association, 1997. For a comparison of the Dun and Bradstreet and InfoUSA lists, see Long, David A., and Tammy Ouelette, Private Employers and TANF Recipients, Final Report, Abt Associates, Washington, D.C.: May 25, 2004, Exhibit 5.2.

OCR for page 39
Measuring Respirator Use in the Workplace identified in that call; (2) a thank-you or reminder postcard mailed to all initial questionnaire recipients a week after the initial mailing; and (3) a replacement questionnaire mailing sent to nonrespondents two to four weeks after the initial questionnaire mailing. Follow the suggestions in Chapter 4 of the Dillman text concerning the content and format of the mailings and the type of postage to use. Most important, follow the precepts of the Tailored Design Method to the maximum extent possible. The purpose of the nonrespondent follow-up calls would be to motivate the person to complete and return the questionnaire.27 These calls should be made by the same telephone interviewer staff that made the initial round of advance screening calls (ideally, by the same interviewer who made the screening call to the establishment initially). Again, a CATI system should be used to control and manage the process. As noted in Chapter 4, the calls should be made beginning one week after the replacement questionnaire mailing to increase the likelihood that the respondent still has the questionnaire. When a respondent promises to complete and return the questionnaire, the interviewer should enter a follow-up date in the case management system 10 days or so hence and then call back the person if the questionnaire has still not been received at that time. Persons who refuse to participate during the nonrespondent follow-up call should be recontacted by specially trained “converter interviewers” in an effort to overcome the refusal. Analysis and Weights From any randomly chosen sample one can obtain estimates of population parameters of interest such as the mean, total, or proportion associated with certain employer or employee characteristics of the population targeted by the sample. Sample designs that are developed to produce these population estimates include some important statistical features, including stratification and cluster selection as well as the inclusion probabilities of individual survey respondents. From the latter feature, sampling weights are computed for each survey respondent as the inverse of their selection probability that is adjusted to account for differential frame coverage and nonresponse.28 These design features are then incorporated into the analysis of the survey data. A weight would be developed for a sampling unit, whether an employer or employee. To compute national- or industry-level estimates from the survey data, a weight for each employee or employer in the sample would be developed. One multiplicative component of these weights is the base weight, computed as the inverse of the inclusion or selection probability, to account for disproportionality in sample selection. The remaining weight components might include some type of adjustment to correct for differential frame coverage and/or nonresponse. Some or all of the adjustment weights may be used depending on the information available to create weights and the degree of error introduced by nonresponse or non-coverage. For example, an additional weight could be used to account for instances where some records, such as specific employers, appear on the sampling frame more than once. If these were not removed prior to sampling, this multiplicity can be adjusted by using another weight to account for the number of appearances on the sampling frame. To account for sample imbalance due to nonresponse and remaining sources, weighting class and calibration adjustments might be used, respectively. A weighting class adjustment uses available information for both respondents and nonrespondents to account for differential nonresponse across members of the sample.29 Information available from the SOII on industry type could be used to create adjustment weighting classes and adjustment weights. In addition, information may be available for the entire population from BLS or other sources to calibrate the final set of weights to the study population (through poststratification or ranking ratio adjustments). The objective of adjustment weights is to reduce bias introduced by survey nonsampling errors. The weights reduce bias but increase sampling variance. Weight trimming or truncation attempts to reduce the increased variability introduced by extremely large weights. A sensitivity analysis could be used to determine the right balance of weight trimming to adopt in the survey. The final weight associated with each survey respondent is a multiple of the base weight times each of the adjustments. Following an analysis plan to produce the findings sought in meeting survey objectives, the final weight is then used in an analysis that accounts for all features of the sample design: stratification, cluster sampling, and sample weights. These features are generally available in software designed for analysis of survey data such as SUDAAN, STATA, and the analysis of survey data procedures in SAS Version 8.0 and beyond. A helpful comparative summary of survey analysis software appears through a link at the web page for the Survey Research Methods Section of the American Statistical Association.30 27 If feasible, the interviewer should first try to complete the questionnaire over the telephone during the nonrespondent call, rather than rely on the person to complete and return it. However, the length and content of the questionnaire may make this infeasible. Determining the feasibility of this approach should be one of the goals of a pilot study. 28 Sample weights might also be “trimmed” to control variation in the weights, and they might be calibrated to the target population to compensate for other important imbalances that can affect the statistical bias of weighted estimates. 29 Similar to a nonresponse or undercoverage adjustment, a weight might be considered to adjust for those firms that have gone out of or come into existence during the data collection period that are not accounted for in the frame. 30 See http://www.hcp.med.harvard.edu/statistics/survey-soft/.

OCR for page 39
Measuring Respirator Use in the Workplace Design for a Survey of Employees Only Sample Design When data in a survey round are to be collected from employees only, or from both employers and employees within the same sample design, it is prudent to sample separately at those two levels of the workforce. This implies cluster sampling in two or more stages of selection, with sampling units being employers or worksites in one stage and employees in the next (and final) stage. Two-stage sampling is the simplest application of this family of designs, where a random sample of employers or worksites serving as the PSU (i.e., the sampling unit in the first stage) is chosen and a random sample of employees is selected separately in the second stage. Frame selection issues and the notion of limiting survey data collection to a sample of area units in this design setting would be similar to those previously discussed in the employer-only design setting. In a two-stage sample of employees a stratified sample of business establishments would be chosen first. The issues related to defining strata and allocating the sample among them would also be similar to those presented for an employer-only survey. Selection of employers in each stratum in a two-stage design would best be done with probability proportional to size (PPS) if an employee sample is sought and if a suitable measure of the current number of employees is available on the employer sampling frame. The sample of employees is chosen in the next sampling stage by separately choosing an employee sample within each participating member of the employer sample. Sampling in this stage requires a list of employees that the employer can and is willing to produce. This list must include sufficient information to stratify the employee selection (if needed, e.g., by length of employment) and to contact selected employees for study participation and data collection. Employee selection within each employer might typically utilize basic selection methods, such as simple random sampling or systematic sampling, unless employees must be disproportionately sampled (e.g., by demographics such as amount of formal education). Data Collection Plan As indicated above, employees can best be surveyed using a two-stage sampling design in which employers are sampled at the first stage and employees are sampled in the second stage. This approach has been used successfully to survey employees in the Department of Labor’s Occupation Information Network (O*NET) project. A summary of the O*NET data collection protocol used by the contractor is presented in Box 5.2. This model could easily be adapted for use in a PPE employee survey. The first step would be a “verification” call to the sampled business establishments to eliminate those that are no longer in business and to identify a contact person for the establishment as before. The next step would be a screening call to the contact person to learn if the establishment uses any PPE. If the business does not use PPE, data collection is complete for the establishment at this point. For PPE users, the caller briefly explains the employee survey and tells the contact person that an information package will arrive shortly for his or her consideration. After the screening call, the next step would be the mailing of an information package to the contact person, including a NIOSH transmittal letter, a brochure, endorsement letters, and possibly a token gift.31 A week or so later, a “recruiting” call would be made to confirm eligibility and enlist the contact person’s agreement to participate. If the contact person agrees, the caller would ask the contact person to complete a roster of employees who use PPE and would make an appointment to call the contact person back for the sampling call. During the subsequent sampling call the survey contractor’s telephone interviewer would work with the contact person to select a sample of PPE users from the contact person’s roster. The caller would inform the contact person of the roster line numbers of the selected employees and explain that a set of questionnaire packages will be sent to him or her for distribution to the sampled employees. The next step would be the mailing of questionnaire packages to the contact person. The packages would be identified only by roster line number; the survey contractor never has the employees’ names. The contact person is reminded to distribute the questionnaire packages to the sampled employees on the roster and to encourage them to complete and return the questionnaire to the survey contractor. Consideration should be given to providing a gift to the contact person to help motivate him or her to follow through with these tasks.32 The transmittal letter to the sampled employees would encourage them to complete their questionnaires on their own time and return them directly to the survey contractor in the business reply envelope provided. The employees are assured that their data will be treated confidentially and never provided to their employer or to anyone else in a manner that would permit identification of a respondent. At this point, the case would enter the follow-up phase. The telephone interviewers would make a series of followup calls to their contact persons to keep them updated on the progress of data collection from the sampled employees (which roster lines have returned questionnaires and which haven’t) and to request that the contact person follow up with 31 An example of a token gift would be a refrigerator magnet such as NIOSH’s “Have your checked your seal?” or “We have you covered” magnets. 32 The O*NET project includes a framed Certificate of Appreciation from the U.S. Department of Labor in the questionnaire mailing to the point of contact.

OCR for page 39
Measuring Respirator Use in the Workplace BOX 5.2 Surveying Employees in an Establishment Survey: The O*NET Example The Occupational Information Network (O*NET) is a system for collecting, organizing, describing, and disseminating information on occupational requirements and employee attributes. Administered by the U.S. Department of Labor, the database is designed to be the most comprehensive standard source of occupational and skill information in the United States. The O*NET data collection program employs a two-stage design to survey establishments and employees within those establishments. The survey consists of (1) a statistical sample of establishments expected to employ employees in each specific occupation and (2) a sample of employees in the targeted occupations within each sampled establishment. The sampled employees are asked to complete one of several survey questionnaires covering various aspects of their occupation, a brief demographic questionnaire, and a short occupation-specific task inventory. The primary stage sample of establishments is selected from lists supplied by InfoUSA (prior to October 2003) and Dun and Bradstreet (subsequent to October 2003). The establishments are selected from industries expected to employ the occupations. Up to 10 occupations are then randomly assigned to each establishment for possible data collection. The complex data collection environment of the O*NET survey requires special, intensive data collection procedures: Step 1. Verification call to initial establishment contact (receptionist) Step 2. Screening call to possible contact person Step 3. Send information package to contact person Step 4. Recruiting call to contact person Step 5. Sampling call to contact person Step 6. Send questionnaire package to contact person Step 7. Send Toolkit for Business (a package of information on how to use O*NET data) Step 8. 7-day follow-up call to contact person Step 9. Send thank-you or reminder postcards Step 10. 21-day follow-up call to contact person Step 11. 31-day follow-up call to contact person Step 12. Send replacement questionnaires Step 13. 45-day follow-up call to contact person The contractor had used this intensive methodology to survey more than 69,000 sampled establishments and 118,000 selected employees as of September 2004. The overall response rate was 70 percent for establishments and 65 percent for employees. Comparisons were made between respondents and nonrespondents on a variety of frame variables, and weight adjustments were made to compensate for the small differences found. Response rates for most questionnaire items were very high, minimizing the potential for item nonresponse bias. employees who have not returned their questionnaires. In addition to the follow-up phone calls, thank-you or reminder postcards are sent to the contact person about a week after the sampling call, with a note requesting that the contact person distribute postcards to all of the sampled employees. After several weeks, replacement questionnaires are sent to the contact person for distribution to nonresponding employees Analysis and Weights The previously described principles for producing sample weights and implementing the analysis plan for any survey round would apply here as well. Design for a Simultaneous Survey of Employers and Employees Sample Design If it was found that information from both employers and employees was needed at the same time (i.e., during the same survey round), two potential design strategies might be followed. One design option (Design 1) would produce two separate samples: one of employers following selection methods described for employer-only survey rounds and a second of employees chosen from a separate random sample of employers. The samples of employers for each

OCR for page 39
Measuring Respirator Use in the Workplace component sample might be chosen by randomly splitting an initial probability sample of employers. The two employer samples might also be independently selected random samples, although drawing independent samples raises the unwanted possibility (probably slight) of employer overlap in the two samples. Each of these alternative approaches has its own advantages and limitations. In particular, the independent samples permit different stratification plans for employer and employee surveys. If particular categories of employees are to be oversampled, the independent samples approach permits oversampling (by selected establishments disproportionately) without the use of (possibly) unduly large employee cluster sizes. The other general design option (Design 2) would follow the two-stage sample design described for the employee-only survey, but data would be collected both from a contact person in the business establishment and from employees. The respondent to the employer questionnaire and the contact person to facilitate employee sampling may or may not be the same person. Since the PPS sampling of employers in the employee-only survey may not be the most appropriate here, the sampling method within the employer sampling strata may be simple random sampling with disproportionate allocation among strata. The sampling rates would be higher for employers with larger workforces. The disproportionate allocation would result in variable employee sample sizes within selected employers. Data Collection Plan While the employer-only and employee-only surveys described above involve somewhat different data collection protocols, they are similar in that both rely heavily on telephone and mail survey resources. Thus, some efficiency could be gained from conducting the two surveys simultaneously and from the same sample design as assumed in the Design 1 option above. For example, the same telephone staff could be used to conduct the initial telephone screening call and nonrespondent follow-up calls for the employer component of a combined survey. Other field work that would realize similar efficiencies would include screening employers for PPE use, recruiting eligible employers to participate, conducting within-establishment employee sampling, and completing follow-up calls for the employee interview component. In addition, the mail operations of both surveys could be performed by the same staff, using similar procedures. If the survey contractor has an adequate survey operations capacity, efficiencies could also be realized in staff training and supervision, common management information and survey control systems, and production operations such as mailing and processing completed questionnaires. All of these efficiencies, when taken together, would imply lower overall survey management costs resulting from a shorter performance period relative to conducting the two surveys at different times. The employer and employee surveys described above could be combined into a single survey as described for Design 2 by modifying the employee survey protocol to include data collection from the selected employer contact person, as well as from a sample of their employees. In this setting, the contact person would complete the employer questionnaire and also coordinate data collection from the employees, as outlined above. While this design would require fewer resources than two separate surveys, it is of concern that contact persons may be reluctant to participate for fear that the employees may give a different picture of PPE usage at the establishment than that reflected in the questionnaire completed by the person responding for the employer. If this model is considered, it would have to be carefully field-tested during the planning phase to determine its feasibility.33 Analysis and Weights The previously described principles for producing sample weights and implementing the analysis plan for any survey round would apply here as well. Data Quality In any of the above design alternatives, attention needs to be given to checking the quality of implementation and the quality of the data. Among other steps that might be taken would be a reinterview program at both the screening and the main data collection stages. In these procedures, samples of establishments responding to the screener and main survey are selected and reinterviewed, with the reinterview and survey data being compared to estimate screening error rates and the reliability of reports. For the screening, a sample of firms that screened out as having no respirator use would be recontacted and rescreened. The sample might be disproportionally selected from those industry types with high respirator use. This would permit an estimate of firms that were screened out in error. For the main interview, a sample of establishments would be recontacted and administered a subset of selected items from the interview. The reinterview approach would not be as easily done at the employee level, but other means should be explored in pilot testing to assess the quality of the employee data. COMMITTEE COMMENT Proper respirator use in the workplace is truly a matter of life and death. It is important to understand those practices and technologies that promote proper use, as well as those that inhibit the proper wearing of this important protective 33 The response rate concern could be ameliorated somewhat if the establishment questionnaire collected different information about PPE usage than the employee questionnaire.

OCR for page 39
Measuring Respirator Use in the Workplace equipment. While the committee applauds the initiative of NIOSH and the BLS in conducting the pioneering SRUP in 2001 in order to bring workplace practices to light, it concludes that further studies must go beyond this initial effort to discover information useful in the promotion of wear and care of respirators. For this reason, the committee strongly recommends that future surveys elicit information from both employers and employees. The procedures for an employee-within-establishment survey have been implemented successfully elsewhere and should be replicable for gathering this type of information from the ultimate users as well as the employers who bear responsibility for ensuring proper use. The recommendations in this report, in concert with the lessons learned from the pioneering SRUP, should enable NPPTL to mount a data collection operation that employs the most advanced and scientifically sound methods and helps augment the base of knowledge needed to advance the ultimate goal of the organization to improve health and safety in the workplace.