1. is clear that judgement is involved, the demands made of each study are quite severe and the summary evaluations that are presented are quite conservative. Serious disagreements in how these data should be interpreted may come from those who question the legitimate use of epidemiology in any forum such as this, but epidemiologists can be expected to see these judgements as soundly based and determined on reasonable and well articulated criteria. The scheme developed to evaluate strength of evidence is clear and consistent with similar efforts by agencies such as the International Agency for Research on Cancer or organizations such as the American Conference of Governmental Industrial Hygienists.
  2. NIOSH appropriately does not directly address the question of incidence in the general population versus specific groups of workers. There are a variety of data sources that might be used to estimate population prevalence (for example, NHIS, NHANES, and other surveys from NCHS, SSA Disability Reports, data accumulated as byproducts from health insurance) but none provide incidence data and none serve as a useful comparison for NIOSH's evaluation. NIOSH is careful to provide explicit information on the comparison populations (for example no or low exposure groups in cohort studies, unaffected controls in case control studies). Since the assessment examined specific physical factors, several of which may have been included in a particular study, the reviewers take care to determine that a proper unexposed or low exposed group is defined in terms of the specific exposure factor studied in cohort or cross-sectional studies. In case control studies the assessments made of both cases and controls are with regard to the specific factor under study.
  3. I want to answer a question not asked: Is the approach taken by NIOSH to evaluate and summarize the epidemiologic literature on work-related musculoskeletal disorders of the neck, upper extremity, and low back appropriate and the most desirable? One might ask why NIOSH did not attempt several meta-analyses rather than use their more qualitative review? Meta-analysis is not appropriate when the question under study is as broad as the one NIOSH addressed. In my judgement Shapiro provides the answer which, in his words is: "I question whether quantitative methods can ever be as thoroughgoing, probing and informative as qualitative methods."[2]. In addition, in his final comments from a symposium on "Meta-analysis of Observational Studies" he summarizes apparent agreement with two other leading epidemiologists that synthetic meta-analyses (efforts to arrive at a single risk estimate by combining multiple studies) are of questionable validity and more likely to be misleading than helpful. One might also ask why did NIOSH not exclude case series from their review? The answer is provided by Checkoway, et al. in their classic textbook [3]. In discussing causal inference, he says: "Attempts to codify guidelines for assessing research quality are invariably detrimental to the practice and application of epidemiologic methods." He goes on to illustrate, through a hypothetical example, how a case series can easily provide information at least as important as a well-designed epidemiologic study. There is no "correct" way to carry out a literature review particularly with as large a scope as the one undertaken by NIOSH. The authors of the NIOSH report are to be commended for developing a methodology that is reasonable, understandable, clearly presented, open and conservative. It is hard to imagine a more effective way to summarize this literature.


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement