National Academies Press: OpenBook

Assessment of the NIOSH Head-and-Face Anthropometric Survey of U.S. Respirator Users (2007)

Chapter: 3 The NIOSH-Sponsored Anthrotech Study's Sampling Strategy

« Previous: 2 Anthropometric Measurements
Suggested Citation:"3 The NIOSH-Sponsored Anthrotech Study's Sampling Strategy." Institute of Medicine. 2007. Assessment of the NIOSH Head-and-Face Anthropometric Survey of U.S. Respirator Users. Washington, DC: The National Academies Press. doi: 10.17226/11815.
×

3
The NIOSH-Sponsored Anthrotech Study’s Sampling Strategy

The validity of the National Institute for Occupational Safety and Health (NIOSH)-sponsored Anthrotech report depends largely on the sample population that it assembled; therefore, the committee examined the details of that sampling in this chapter.

THE ANTHROTECH SAMPLING PLAN

The goal of the NIOSH-sponsored Anthrotech study was to survey a group of individuals reflecting the racial, ethnic, age, and gender diversity of the respirator wearing workforce in the United States. The sampling plan used in the study (Appendix C) called for approximately 4,000 subjects that would be divided evenly into men and women, three age categories (18-29, 30-44, and 45-65), and four racial/ethnic categories (White, African American, Hispanic, and “Other”) (Table 3-1) (Zhuang, 2001). It was unclear to the committee why the NIOSH protocol directed Anthrotech to sample by “age, race, and gender,” and not use ethnic and racial categories from the United States Census, which uses White, African American, Hispanic, Asian/Pacific, and American Indian (U.S. Census Bureau, 2000; Anthrotech, 2004). The committee recognizes that issues of race and ethnicity are complex, and the terms are often inconsistently applied. However, to remain consistent with the designations used in the NIOSH-sponsored Anthrotech report the committee uses the terms as they are used in the original Anthrotech report.

Suggested Citation:"3 The NIOSH-Sponsored Anthrotech Study's Sampling Strategy." Institute of Medicine. 2007. Assessment of the NIOSH Head-and-Face Anthropometric Survey of U.S. Respirator Users. Washington, DC: The National Academies Press. doi: 10.17226/11815.
×

TABLE 3-1 Proposed Demographic Sample Distribution

Race

Male Age Group

Female Age Group

18-29

30-44

45-65

Total

18-29

30-44

45-65

Total

White

166

166

166

498

166

166

166

498

African

166

166

166

498

166

166

166

498

American

Hispanic

166

166

166

498

166

166

166

498

Other

166

166

166

498

166

166

166

498

Total

664

664

664

1,992

664

664

664

1,992

SOURCE: Anthrotech, 2004.

NIOSH requested that Anthrotech obtain measurements from 166 subjects in each age, gender, and racial/ethnic category. The proposed number of 166 subjects per category was based on a standard developed by the International Organization for Standardization (ISO). Standard 15535 recommends n = (1.96 × CV/a)2 × 1.534, where n is the sample size per cell, CV is the coefficient of variation for a specific measurement (standard deviation divided by mean times 100), and a is the level of precision desired. The investigators chose to define CV based on the menton-sellion length (CV=6.5/121.9=5.3%), as it was one of the more biologically variable measurements. Thus, if the CV is met for this measurement, it would likely also be met for the other measurements (Zhuang, 2001). The 95 percent confidence limits (reflected in the term 1.96), the choice of a 1 percent level of precision (1 percent of the mean), and the value 1.534 (a constant computed using a Bonferroni type of argument to adjust for the multiple comparison) were based on procedures outlined in the ISO 15535—General Requirements for Establishing an Anthropometic Database. However, the committee is unsure about the accuracy of the calculation for this task, since there was no discussion in the NIOSH-sponsored Anthrotech report and only limited discussion in the protocol, if the 95 percent confidence limits were appropriate for this specific task (Zhuang, 2001; Anthrotech, 2004).

To ensure that the sample adequately captured the variation of measurements within each of the racial groups, Anthrotech oversampled the African American, Hispanic, and “Other” groups—that is, the sample included a larger percentage of these minorities than is found in the general population. During the analysis, each subcategory was then weighted to accurately represent its proportion within the total population. The advantages of this approach are that it provides the needed level of preci-

Suggested Citation:"3 The NIOSH-Sponsored Anthrotech Study's Sampling Strategy." Institute of Medicine. 2007. Assessment of the NIOSH Head-and-Face Anthropometric Survey of U.S. Respirator Users. Washington, DC: The National Academies Press. doi: 10.17226/11815.
×

sion for each group separately, and it provides the flexibility to reweight the data in the future as population demographics change.

Unfortunately, the NIOSH-sponsored Anthrotech study did not appropriately subdivide the sample population based on race and ethnicity. The “Other” category was overly diverse, including Asian, American Indian, and Alaskan Native subjects, while not including other groups that might be designated “Other.” Further, the groups included in “Other” have vastly different facial characteristics, so it is possible that most or all of those groups may have been undersampled. Given the diversity of the respirator-wearing workforce, it is important that the “Other” group be included in analyses of the entire target population. However, its small sample size and great diversity may make it difficult, if not impossible, to perform useful analysis of the “Other” category as a distinct subgroup.

SAMPLING PROTOCOL

Anthrotech’s sampling protocol involved three steps to obtain a characteristic sample population: (1) identify a subset number of representative geographical locations, (2) choose representative worksites, and (3) identify individuals willing to have their facial dimensions measured.

Because of the need to have investigators present at the worksites, as well as the difficulty and expense of moving raters from one site to another, the NIOSH-sponsored Anthrotech study limited the sample geographically to eight states: California, Illinois, Kentucky, New York, Ohio, Pennsylvania, Texas, and Virginia (Anthrotech, 2004).

Within each state, Anthrotech investigators selected worksites that were deemed representative. The NIOSH-sponsored Anthrotech report indicates that each potential study site was telephoned to ascertain the willingness to participate in the study. “More often than not,” the report says, “the organization or company invited chose not to participate.” When an organization did agree to participate, Anthrotech provided the company with a packet of information explaining the study protocol and purposes.

The report provides no further details on either the criteria Anthrotech used to select the various establishments, response rate from these establishments, or any comparison of participating versus nonparticipating sites. This committee found that the 41 worksites listed in the final report were more often larger establishments. It is not clear whether this

Suggested Citation:"3 The NIOSH-Sponsored Anthrotech Study's Sampling Strategy." Institute of Medicine. 2007. Assessment of the NIOSH Head-and-Face Anthropometric Survey of U.S. Respirator Users. Washington, DC: The National Academies Press. doi: 10.17226/11815.
×

was due to a decision by Anthrotech to approach larger companies or whether larger employers were more likely to agree to participate. Anthrotech did note some difficulty identifying companies that had the time and energy to participate (Anthrotech, 2004). The committee is unsure, and there is no discussion in the report, on how this selection bias may have skewed the final datasets.

Once a work site was identified, Anthrotech investigators worked with a single individual who provided access to test subjects. This point of contact assisted with both scheduling and recruitment. Unfortunately, the criteria for selection of the test subjects are ill defined in the report, so it is difficult to ascertain if the selection process was random, or how potential issues that arose during the selection of the subjects may have biased the sample.

Anthrotech may have introduced an additional source of bias by providing monetary incentives to participate at some worksites. This is difficult to determine as the number and size of incentives is not reported. However, monetary incentives are well-known to create biases in survey data, partly because these monetary incentives appeal more to some kinds of potential participants than others and partly by increasing bias in responses. The first of these is not likely to be great here, and the second should not apply to facial measurements.

SAMPLE DISTRIBUTION

In some instances Anthrotech was not able to meet the sampling quotas, while in others the quotas were surpassed (e.g., the final male-female ratio was closer to 60:40 rather than the desired 50:50). Due to the weighting of the subpopulations, the NIOSH-sponsored Anthrotech report regards the final sample as accurate, although the report does not define exactly what is meant by “accurate.”

Researchers initially measured 4,026 subjects. The final sample was 3,998, after eliminating 27 subjects who were younger than 18 or older than 66, and an additional subject that failed to provide an age (Table 3-2). This final total was very close to the original target number of 24 × 166 = 3,984. The final distribution was also closer to the actual demographic makeup of the U.S. workforce than were the planned equal numbers per category. This has certain advantages, as the imbalance among

Suggested Citation:"3 The NIOSH-Sponsored Anthrotech Study's Sampling Strategy." Institute of Medicine. 2007. Assessment of the NIOSH Head-and-Face Anthropometric Survey of U.S. Respirator Users. Washington, DC: The National Academies Press. doi: 10.17226/11815.
×

TABLE 3-2 Number of Individuals Measured in Each Demographic Category

Race

Male Age Group

Female Age Group

18-29

30-44

45-65

Total

18-29

30-44

45-65

Total

White

271

611

485

1,367

151

194

174

519

African

101

255

278

634

51

213

325

589

American

Hispanic

155

182

75

412

53

36

37

126

Other

24

47

59

130

52

65

103

220

Total

551

1,095

897

2,543

307

508

639

1,454

NOTE: The total count in this table is 3,997 because one subject did not provide age.

SOURCE: Anthrotech, 2004.

categories adds to the variance of comparison among groups. Both objectives may be important, but the committee notes that little has yet been published comparing groups.

Geographic Distribution

As with demographic variation, Anthrotech intended to have an even distribution across geographic regions. However, the final distribution was more heavily weighted (~60 percent) toward Texas, Ohio, and Illinois (the midsection of the United States) (Table 3-3). While the NIOSH-sponsored Anthrotech study found no evidence that geographic regions affected face size and shape, the 2000 U.S. Census shows that most residents from Illinois and Ohio come from white European ancestors, whereas states in the southeast and southwest have more residents with ancestry other than Caucasian (U.S. Census Bureau, 2006). Additionally, many immigrants tend to concentrate in distinct areas of the United States (e.g., Asian immigrants in California, and Hmong and Somali immigrants in Minnesota) (Camarota and McArdle, 2003; Ronningen, 2004). Thus the limited geographic distribution may have unintentionally introduced a racial and ethnic bias.

Suggested Citation:"3 The NIOSH-Sponsored Anthrotech Study's Sampling Strategy." Institute of Medicine. 2007. Assessment of the NIOSH Head-and-Face Anthropometric Survey of U.S. Respirator Users. Washington, DC: The National Academies Press. doi: 10.17226/11815.
×

TABLE 3-3 Geographic Distribution

State

n

%

California

229

5.69

Illinois

1,564

39.12

Kentucky

93

2.31

New York

120

2.98

Ohio

751

18.78

Pennsylvania

29

0.72

Texas

857

21.44

Virginia

355

8.82

Total

3,998

100.00

SOURCE: Anthrotech, 2004.

Occupational Distribution

The test sample was achieved by surveying workers from a variety of different occupations, including construction, firefighting, health care, law enforcement, manufacturing, and others (Table 3-4) (Anthrotech, 2004). However, the study protocol did not explain why these industries were chosen as the representative sample population. Further, an analysis of the results indicates that in some of the categories, such as law enforcement and health care, the representation of men and women is out of the range of what might be expected. Consequently, the committee questions if the tallies were recorded correctly or if there was an unintended selection bias.

These data presented in the NIOSH-sponsored Anthrotech report (Table 3-4) differ from those published that are reproduced below in Table 3-5 (Zhuang and Bradtmiller, 2005). NIOSH explained that the discrepancy resulted from a data sorting error in the Excel spreadsheet. The data presented in Table 3-5 is consistent with collected data from the survey (Personal communication, Z. Zhuang, National Personal Protective Technology Laboratory, October 27, 2006).

LIMITATIONS OF THE SAMPLE POPULATION

Overall the committee found limited analysis in the NIOSH-sponsored Anthrotech study comparing the categorized subgroups, so it is not known whether age, race, and gender affect specific measurements, composite measure, or ultimately the overall utility of anthro-

Suggested Citation:"3 The NIOSH-Sponsored Anthrotech Study's Sampling Strategy." Institute of Medicine. 2007. Assessment of the NIOSH Head-and-Face Anthropometric Survey of U.S. Respirator Users. Washington, DC: The National Academies Press. doi: 10.17226/11815.
×

TABLE 3-4 Occupational Distribution

Occupation

Men

Women

Total

n

%

n

%

n

%

Construction

594

23.35

47

3.23

641

16.03

Firefighting

429

16.86

60

4.13

489

12.23

Health care

776

30.50

75

5.16

851

21.29

Law enforcement

381

14.98

1,100

75.65

1,481

37.04

Manufacturing

121

4.76

7

0.48

128

3.20

Others

243

9.55

165

11.35

408

10.21

Total

2,544

100.00

1,454

100.00

3,998

100.00

SOURCE: Anthrotech, 2004.

TABLE 3-5 Occupational Distribution

 

Men

Women

Total

Occupation

n

%

n

%

n

%

Construction

594

23.4

47

3.2

641

16.0

Firefighting

774

30.4

74

5.1

848

21.2

Health care

381

15.0

1,100

75.7

1,481

37.1

Law enforcement

121

4.8

7

0.5

128

3.2

Manufacturing

429

16.9

60

4.1

489

12.2

Others

244

9.6

166

11.4

410

10.3

Total

2,543

100.0

1,454

100.0

3,997

100.0

SOURCE: Zhuang and Bradtmiller, 2005.

pometric face panels. Further, Anthrotech was unable to find a reference reflecting the ethnic and geographic distribution of respirator users, so the data set were weighted against the U.S. workforce as a whole. Anthrotech’s plan calls for reweighting of this sample, should data characterizing the population of respirator users become available in the future.

If survey data are to serve as a sound foundation for conclusions, both the sample and the analysis of the sample must meet certain criteria. These include

  • precise definition of the target population (the population to be covered by inferences from the data);

  • precise operational definition of the sampling frame (the persons to be sampled, including any clustering variables such as employer);

  • some form of random sampling from the frame;

  • understanding the nature (and if possible the general size and direction) of biases likely to impair inferences from the data to the target population;

Suggested Citation:"3 The NIOSH-Sponsored Anthrotech Study's Sampling Strategy." Institute of Medicine. 2007. Assessment of the NIOSH Head-and-Face Anthropometric Survey of U.S. Respirator Users. Washington, DC: The National Academies Press. doi: 10.17226/11815.
×
  • specific uses of the data, with careful consideration of the need for statistical power, particularly in light of the expected biases in the data;

  • to the extent feasible, study of the data to learn about important influences on the findings; and

  • understanding the nature, sources, and size of variations in measurement.

The committee found no evidence that these matters received adequate consideration in the design and analysis of the NIOSH-sponsored Anthrotech survey.

Defining the Target Population

A clear definition of the target population is critical to survey design, which was not done in the NIOSH-sponsored Anthrotech study. The survey intends to reflect “respirator users,” but that term is ambiguous: Does “respirator user” mean workers who typically use respirators? What about part-time workers, occasional users, workers who only wear respirators in emergency situations, and other workers who should be wearing respirators? These questions may impact the target sample population, because the demographics (and facial measurements) of workers in these additional categories may differ from those of regular, full-time users.

Identification of Subjects

Anthrotech identified establishments that were willing to participate as measuring sites a number of ways, including searching public databases, the Internet, and telephone directories. However, the report did not provide sufficient information about how workers who use respirators were recruited for participation in this study (Anthrotech, 2004). Were they self-selected, selected by management, or randomly selected? Does the sample include overrepresentation of workers whom the management knew had adapted well to the respirators they wore? If so, the hard-to-fit may be seriously underrepresented.

Suggested Citation:"3 The NIOSH-Sponsored Anthrotech Study's Sampling Strategy." Institute of Medicine. 2007. Assessment of the NIOSH Head-and-Face Anthropometric Survey of U.S. Respirator Users. Washington, DC: The National Academies Press. doi: 10.17226/11815.
×

Other Potential Biases

An understanding of potential biases is critically important in the interpretation of survey data. For example, responders are likely to differ from non-responders, so non-response rates are important. However, the committee recognizes that it would be difficult to even define non-response with a sample such as that in the Anthrotech study.

Additionally, random sampling at each stage of a multilevel scheme is important. Equal probabilities are not needed, but each person in the sampling frame must have a known, nonzero probability of being selected, and correct calculation of variances requires that pairwise probabilities also be known (e.g., the probabilities of each possible pair of workers within and between establishments will differ if sampling is clustered by establishment). It is not clear from the survey protocol or report if NIOSH and Anthrotech considered using a random sampling approach (Zhuang, 2001; Anthrotech, 2004).

Conclusion

The proposal and NIOSH-sponsored Anthrotech Report did not adequately define or represent an appropriate target population.


Recommendation 3-1: Define Target Populations More Precisely.


Future anthropometric face panel studies undertaken by, or on behalf of, NIOSH should have a statistically rigorous and valid sampling plan and implementation strategy that precisely define the target population and also ensure that the samples of selected subjects included in the studies are representative of the predefined sample population (e.g., the current workforce that wears respirators, workers who should be using respirators, the general United States workforce).

Suggested Citation:"3 The NIOSH-Sponsored Anthrotech Study's Sampling Strategy." Institute of Medicine. 2007. Assessment of the NIOSH Head-and-Face Anthropometric Survey of U.S. Respirator Users. Washington, DC: The National Academies Press. doi: 10.17226/11815.
×

REFERENCES

Anthrotech. 2004. A head-and-face anthropometric survey of U.S. respirator users: Final report. Prepared by B. Bradtmiller and M. Friess for NIOSH/NPPTL.

Camarota, S., and N. McArdle. 2003. Where immigrants live: An examination of state residency of the foreign born by country of origin in 1990 and 2000. Washington, DC: Center for Immigration Studies.

Ronningen, B. J. 2004. Estimates of selected immigrant populations: 2004. St. Paul, MN: Minnesota State Demographic Center.

U.S. Census Bureau. 2006. Ancestry: 2000. http://www.census.gov/prod/ 2004pubs/c2kbr-35.pdf (accessed December 15, 2006).

Zhuang, Z. 2001. Anthropometric survey of respirator users: Study protocol. Pittsburgh, PA: National Institute for Occupational Safety and Health.

Zhuang, Z., and B. Bradtmiller. 2005. Head-and-face anthropometric survey of U.S. respirator users. J Occup Environ Hyg 2(11):567-576.

Suggested Citation:"3 The NIOSH-Sponsored Anthrotech Study's Sampling Strategy." Institute of Medicine. 2007. Assessment of the NIOSH Head-and-Face Anthropometric Survey of U.S. Respirator Users. Washington, DC: The National Academies Press. doi: 10.17226/11815.
×
Page 43
Suggested Citation:"3 The NIOSH-Sponsored Anthrotech Study's Sampling Strategy." Institute of Medicine. 2007. Assessment of the NIOSH Head-and-Face Anthropometric Survey of U.S. Respirator Users. Washington, DC: The National Academies Press. doi: 10.17226/11815.
×
Page 44
Suggested Citation:"3 The NIOSH-Sponsored Anthrotech Study's Sampling Strategy." Institute of Medicine. 2007. Assessment of the NIOSH Head-and-Face Anthropometric Survey of U.S. Respirator Users. Washington, DC: The National Academies Press. doi: 10.17226/11815.
×
Page 45
Suggested Citation:"3 The NIOSH-Sponsored Anthrotech Study's Sampling Strategy." Institute of Medicine. 2007. Assessment of the NIOSH Head-and-Face Anthropometric Survey of U.S. Respirator Users. Washington, DC: The National Academies Press. doi: 10.17226/11815.
×
Page 46
Suggested Citation:"3 The NIOSH-Sponsored Anthrotech Study's Sampling Strategy." Institute of Medicine. 2007. Assessment of the NIOSH Head-and-Face Anthropometric Survey of U.S. Respirator Users. Washington, DC: The National Academies Press. doi: 10.17226/11815.
×
Page 47
Suggested Citation:"3 The NIOSH-Sponsored Anthrotech Study's Sampling Strategy." Institute of Medicine. 2007. Assessment of the NIOSH Head-and-Face Anthropometric Survey of U.S. Respirator Users. Washington, DC: The National Academies Press. doi: 10.17226/11815.
×
Page 48
Suggested Citation:"3 The NIOSH-Sponsored Anthrotech Study's Sampling Strategy." Institute of Medicine. 2007. Assessment of the NIOSH Head-and-Face Anthropometric Survey of U.S. Respirator Users. Washington, DC: The National Academies Press. doi: 10.17226/11815.
×
Page 49
Suggested Citation:"3 The NIOSH-Sponsored Anthrotech Study's Sampling Strategy." Institute of Medicine. 2007. Assessment of the NIOSH Head-and-Face Anthropometric Survey of U.S. Respirator Users. Washington, DC: The National Academies Press. doi: 10.17226/11815.
×
Page 50
Suggested Citation:"3 The NIOSH-Sponsored Anthrotech Study's Sampling Strategy." Institute of Medicine. 2007. Assessment of the NIOSH Head-and-Face Anthropometric Survey of U.S. Respirator Users. Washington, DC: The National Academies Press. doi: 10.17226/11815.
×
Page 51
Suggested Citation:"3 The NIOSH-Sponsored Anthrotech Study's Sampling Strategy." Institute of Medicine. 2007. Assessment of the NIOSH Head-and-Face Anthropometric Survey of U.S. Respirator Users. Washington, DC: The National Academies Press. doi: 10.17226/11815.
×
Page 52
Next: 4 Data Analysis and Fit-Test Panels »
Assessment of the NIOSH Head-and-Face Anthropometric Survey of U.S. Respirator Users Get This Book
×
Buy Paperback | $43.00 Buy Ebook | $34.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

NIOSH and the Occupational Safety and Health Administration (OSHA) share responsibility for overseeing respiratory protection in the workplace and have established regulations for this purpose. Specifically, NIOSH has issued regulations which define respirator testing and certification. OSHA has issued regulations which define conditions under which employers are required to maintain respiratory protection programs in general industry, shipyards, marine terminals, and construction.

In 2005, NIOSH contracted with the Institute of Medicine (IOM) to study the NIOSH-sponsored Anthrotech study along with its supporting information and reports, and to examine and report on the adequacy and relevance of the study protocol, the analyses conducted, the resulting anthropometric dataset, and the appropriateness of the respirator fit-test panels derived from the new dataset.

Assessment of the NIOSH Head-and-Face Anthropometric Survey of U.S. Respirator Users focuses on the establishment of the scientific base required for certification standards of respirators, not their use in the workplace. This report describes and analyzes the anthropometric measurements performed by Anthrotech for its NIOSH-sponsored study. This report looks at the survey methods used by Anthrotech in the study, and examines how Anthrotech analyzed its data to derive fit-test panels and suggests some ways that analysis could be improved. This report discusses future directions, pointing toward further analyses of the data and offers suggestions for moving from research to practice.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!