Review of the Environmental Protection Agency's Draft IRIS Assessment of Formaldehyde (2011)

As noted in Chapter 1, the committee was asked to review and comment on specific aspects of the draft IRIS assessment of formaldehyde. This chapter provides general comments on the methods and structure of the document. The committee’s rationale for providing general comments is that the specific elements of the charge are inseparable from the approach used by the U.S. Environmental Protection Agency (EPA) for the development of the assessment and presentation of its findings.¹ In responding to questions posed in its charge and developing its report, the committee noted some recurring methodologic problems that cut across components of the charge.

The general problems that the committee identified are not unique to the draft IRIS assessment of formaldehyde. Committees of the Board on Environmental Studies and Toxicology (BEST) of the National Research Council (NRC) have reviewed a number of IRIS assessments in the last decade, including three (NRC 2005, 2006, 2010) in the last 5 years. Some of the general problems identified by the present committee have been commented on by the other BEST committees. For example, the 2006 NRC report on dioxin and related compounds commented on the need for formal, evidence-based approaches for noncancer effects, the need for transparency and clarity in the selection of data sets for analysis, and the need for greater attention to uncertainty and variability (NRC 2006). The 2010 NRC review of the draft IRIS assessment of tetrachloroethylene found similar problems and provided a chapter, “Moving Beyond the Current State of Practice,” that addressed methodologic issues and the failure to establish clear and transparent methods for carrying out and presenting the assessment (NRC 2010). That report also provided a broad set of recommendations on characterization of uncertainty.

The present chapter addresses the general assessment methods and covers identification of the studies considered, their evaluation, and the weight-of-evidence assessment. These issues are also addressed within the context of the specific health outcomes evaluated (see Chapters 4-5).

IRIS has the overall purpose of evaluating human health effects that may arise from exposure to environmental contaminants (EPA 2010a). An IRIS assessment addresses noncancer and cancer effects as appropriate and provides descriptive and quantitative information:

• “Noncancer effects: Oral reference doses and inhalation reference concentrations (RfDs and RfCs, respectively) for effects known or assumed to be produced through a nonlinear (possibly threshold) mode of action. In most instances, RfDs and RfCs are developed for the noncarcinogenic effects of substances” (EPA 2010a).

• “Cancer effects: Descriptors that characterize the weight of evidence for human carcinogenicity, oral slope factors, and oral and inhalation unit risks for carcinogenic effects. Where a nonlinear mode of action is established, RfD and RfC values may be used” (EPA 2010a).

A sequence of activities is involved in conducting IRIS assessments and in calculating RfCs and unit risk estimates. Figure 2-1 is a generic schema that describes the steps used to generate the draft IRIS assessment and the actions needed at each step. The figure is the committee's representation of that process, as gleaned from the assessment. Although the draft IRIS assessment does not explicitly acknowledge these steps, they are implicit in the approach and are ordered as shown.

FIGURE 2-1 Elements of the IRIS process. Abbreviations: IRIS, Integrated Risk Information System; RfC, reference concentration; and UR, unit risk.

In fact, the general approach receives little description in the draft assessment. The methods for conducting the assessment are described in a two-page introduction to a document whose main chapters and appendixes occupy about 1,000 pages. The committee notes that the introductory chapter of the formaldehyde assessment is almost identical with that of other IRIS assessments (see, for example, the IRIS summary for chloroprene, EPA 2010b). The two pages constitute the sole description of the methods used by the authors and cite numerous EPA guidance documents, some dating to 1986 (see Appendix B for some of the most relevant portions of those documents). Some of the guidance documents are cited at appropriate points in the draft assessment, but their specific roles in the preparation of the draft are not clear.

In general, the committee found that the draft assessment was not prepared in a transparent, consistent fashion with clear linkages to an underlying framework as it moves from review of the relevant evidence to calculation of the RfCs and unit risk estimates and characterization of their uncertainty and variability. The committee did not find sufficient documentation of methods and criteria for identifying the epidemiologic and experimental evidence to be reviewed, for evaluating individual studies, for assessing weight of evidence, for selecting individual studies for derivation of toxicity and risk estimates, or for characterizing uncertainty and variability. Summary sections that synthesize the evidence are variable and too often brief or not present, and strength of evidence is not characterized with standardized descriptors.

The committee emphasizes that its criticism regarding the lack of documentation is not a recommendation for adding lengthy summaries of the individual guidance documents to the introductory chapter. It is suggesting that clear concise descriptions of key criteria used to include studies in the analysis, to exclude studies, or to advance studies for calculation of RfCs and unit risk estimates are needed. Nuances concerning specific health outcomes could be addressed in the introductory sections on those outcomes. The following sections provide comments on the general steps of the process. Again, specific aspects are addressed in Chapters 4-5 for each health outcome.

The ability to identify and filter studies is crucial for any literature review that is synthesizing the potential effects of a suspected hazard. The evaluation of all relevant studies in an IRIS review process is analogous to the collection of relevant studies for a meta-analysis. A general approach to literature review is provided in Chapter 1 of the draft IRIS assessment of formaldehyde. EPA used a Chemical Abstracts Service Registry Number (CASRN) and “at least one common name” (EPA 2010c, p. 1-2) to search for relevant publications. The specific databases searched are not listed. PubMed searches are critical for identifying the literature on risks to health; the committee notes that PubMed does not specifically capture the CASRN.

The state of the art of literature searches now involves providing an extensive description of the databases searched and the search terms used. Chapter 1 of the draft IRIS assessment does not provide a list of the search terms used, such as terms that were used for the various health outcomes relevant to formaldehyde. The draft assessment also does not describe the results of searches, so the numbers of articles identified and excluded are unavailable to readers.

The draft IRIS assessment evaluates many individual studies in a variety of disciplines. A description of the methods for evaluating individual studies is not provided, and it appears to this committee that studies were not reviewed with a common template for assessing their strengths and weaknesses. The committee notes that the template for evaluation would vary appropriately with the type of research study being considered. Such a strategy is not uniformly evident, and the evidence considered is not presented consistently in informative tables.

In some cases, there is a tendency to describe the studies ultimately selected for the derivation of the RfC in favorable terms. For example, a cross-sectional study by Krzyzanowski et al. (1990)—a study selected for the calculation of an RfC for respiratory effects—is referred to as “well-designed and executed” (EPA 2010c, p. 4-41) without emphasis on the inherent weaknesses of its cross-sectional design. The committee found one study selected for advancement for calculation of an RfC (Ritchie and Lehnen 1987) to be potentially subject to severe bias and would not have recommended it for advancement. Specifically, selection of the study population was based on a visit to a physician and referral for formaldehyde-concentration measurement, and the concentration-response gradient was considered by the committee to be implausibly steep (see Chapter 4 for further discussion).

In evaluating the evidence of causation, the draft IRIS assessment cites various EPA guidelines that apply weight-of-evidence approaches in assessing the strength of evidence. Those guidelines have been developed over a period of nearly 2 decades, and consequently consistency of methods is lacking from outcome to outcome (Appendix B). The implementation of the guidelines appears to be subjective and not standardized. The committee found variable detail in how the weight-of-evidence criteria had been applied. Uniformly developed discussions applying the weight-of-evidence criteria cannot be identified at appropriate points in the text. In some sections, the discussion of biologic evidence is particularly weak (for example, in the case of asthma pathogenesis) and often not reflective of the current state of knowledge.

In summary, when the review of studies used in the draft IRIS assessment of formaldehyde is compared with the current standard for evidence-based reviews and causal inference, limitations in each step used to generate the draft IRIS assessment are evident. For example, the methods are not clearly described, the review approaches are not transparent, and there is no indication that evidence-grading strategies were uniformly applied. In addition, the selection approach to identifying studies for RfC calculation appears ad hoc. The committee emphasizes that it is not recommending that EPA add an extensive discussion of its guidelines to the draft IRIS assessment. It is recommending that key factors used to exclude, include, or advance studies be discussed.

EPA (U.S. Environmental Protection Agency). 2010a. Frequent Questions: What is IRIS? Integrated Risk Information System, U.S. Environmental Protection Agency, Washington DC [online]. Available: http://www.epa.gov/IRIS/help_ques.htm#whatiris [accessed Dec. 28, 2010].

EPA (U.S. Environmental Protection Agency). 2010b. Toxicological Review of Chloroprene (CAS No. 126-99-8). In Support of Summary Information on the Integrated Risk Information System (IRIS). EPA/635/R-09/01F. U.S. Environmental Protection Agency, Washington, DC. September 2010 [online]. Available: http://www.epa.gov/iris/toxreviews/1021tr.pdf [accessed Jan. 6, 2010].

EPA (U.S. Environmental Protection Agency). 2010c. Toxicological Review of Formaldehyde (CAS No. 50-00-0) – Inhalation Assessment: In Support of Summary Information on the Integrated Risk Information System (IRIS). External Review Draft. EPA/635/R-10/002A. U.S. Environmental Protection Agency, Washington, DC [online]. Available: http://cfpub.epa.gov/ncea/iris_drafts/recordisplay.cfm?deid=223614 [accessed Nov. 22, 2010].

Krzyzanowski, M., J.J. Quackenboss, and M.D. Lebowitz. 1990. Chronic respiratory effects of indoor formaldehyde exposure. Environ. Res. 52(2):117-125.

NRC (National Research Council). 2005. Health Implications of Perchlorate Ingestion. Washington, DC: National Academies Press.

NRC (National Research Council). 2006. Health Risks from Dioxin and Related Compounds: Evaluation of the EPA Reassessment. Washington, DC: National Academies Press.

NRC (National Research Council). 2010. Review of the Environmental Protection Agency's Draft IRIS Assessment of Tetrachloroethylene. Washington, DC: National Academies Press.

Ritchie, I.M., and R.G. Lehnen. 1987. Formaldehyde-related health complaints of residents living in mobile and conventional homes. Am. J. Public Health. 77(3):323-328.