1

Introduction

In 2011, the National Research Council (NRC) released the report Review of the Environmental Protection Agency’s Draft IRIS Assessment of Formaldehyde. The report provided a scientific review of the toxicological review of formaldehyde drafted by the US Environmental Protection Agency (EPA) for its Integrated Risk Information System (IRIS). Chapter 7 of the NRC report also suggested changes in the general process used to develop IRIS assessments and suggested a roadmap for making revisions if EPA decided to do so. In response, EPA announced plans to work with its Science Advisory Board to address the committee’s suggestions and recommendations. In 2011 testimony before a subcommittee of the US House of Representatives, the assistant administrator of EPA’s Office of Research and Development outlined the approach that EPA planned to take in response to the NRC recommendations.1 On December 23, 2011, the Consolidated Appropriations Act, 2012 (Public Law 112- 74) was signed into law; the House report (112-151) accompanying the act stated that “EPA shall incorporate, as appropriate, based on chemical-specific datasets and biological effects, the recommendations of Chapter 7 of the National Research Council’s Review of the Environmental Protection Agency’s Draft IRIS Assessment of Formaldehyde into the IRIS process.” To ensure that EPA adequately considers the NRC recommendations, Congress requested that NRC assess the scientific, technical, and process changes that EPA is making. As a result of that request, NRC convened the Committee to Review the IRIS Process, which drafted the present report.

THE INTEGRATED RISK INFORMATION SYSTEM AND THE 2011 FORMALDEHYDE REPORT

EPA created the IRIS program in 1985 to provide information on human health effects that could arise from chronic exposures to environmental contaminants. A primary goal of IRIS is to increase the consistency of assessments being conducted throughout the agency. Accordingly, the IRIS program develops toxicologic assessments of various chemicals that include hazard and dose-response characterizations. The assessments also include toxicity values (reference values and unit risks) that can be used with exposure estimates to derive quantitative risk estimates.

Over the years, federal and state agencies and other entities have come to rely on IRIS assessments for setting regulatory standards and establishing exposure guidelines. IRIS assessments provide information needed to evaluate risks at the local level and are also considered authoritative internationally. In recent years, however, questions have been raised about the scientific basis of reference values and unit risks reported in some IRIS assessments. The IRIS program has also been criticized for the time that it takes the agency to complete assessments. Some assessments have taken more than a decade to develop and have gone through multiple cycles of revision and review, which further delay their acceptance. For example, the develop-

_____________________________

1EPA’s IRIS Program: Evaluating the Science and Process behind Chemical Risk Assessment, 2011: Hearing before the Subcommittee on Investigations and Oversight of the Committee on Science, Space, and Technology, House of Representatives, 112th Cong., 1st Sess., July 14, 2011.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 10
1 Introduction In 2011, the National Research Council (NRC) released the report Review of the Environ- mental Protection Agency’s Draft IRIS Assessment of Formaldehyde. The report provided a sci- entific review of the toxicological review of formaldehyde drafted by the US Environmental Pro- tection Agency (EPA) for its Integrated Risk Information System (IRIS). Chapter 7 of the NRC report also suggested changes in the general process used to develop IRIS assessments and sug- gested a roadmap for making revisions if EPA decided to do so. In response, EPA announced plans to work with its Science Advisory Board to address the committee’s suggestions and rec- ommendations. In 2011 testimony before a subcommittee of the US House of Representatives, the assistant administrator of EPA’s Office of Research and Development outlined the approach that EPA planned to take in response to the NRC recommendations.1 On December 23, 2011, the Consolidated Appropriations Act, 2012 (Public Law 112- 74) was signed into law; the House report (112-151) accompanying the act stated that “EPA shall incorporate, as appropriate, based on chemical-specific datasets and biological effects, the recommendations of Chapter 7 of the National Research Council’s Review of the Environmental Protection Agency’s Draft IRIS As- sessment of Formaldehyde into the IRIS process.” To ensure that EPA adequately considers the NRC recommendations, Congress requested that NRC assess the scientific, technical, and pro- cess changes that EPA is making. As a result of that request, NRC convened the Committee to Review the IRIS Process, which drafted the present report. THE INTEGRATED RISK INFORMATION SYSTEM AND THE 2011 FORMALDEHYDE REPORT EPA created the IRIS program in 1985 to provide information on human health effects that could arise from chronic exposures to environmental contaminants. A primary goal of IRIS is to increase the consistency of assessments being conducted throughout the agency. Accordingly, the IRIS program develops toxicologic assessments of various chemicals that include hazard and dose-response characterizations. The assessments also include toxicity values (reference values and unit risks) that can be used with exposure estimates to derive quantitative risk estimates. Over the years, federal and state agencies and other entities have come to rely on IRIS as- sessments for setting regulatory standards and establishing exposure guidelines. IRIS assess- ments provide information needed to evaluate risks at the local level and are also considered au- thoritative internationally. In recent years, however, questions have been raised about the scientific basis of reference values and unit risks reported in some IRIS assessments. The IRIS program has also been criticized for the time that it takes the agency to complete assessments. Some assessments have taken more than a decade to develop and have gone through multiple cycles of revision and review, which further delay their acceptance. For example, the develop- 1 EPA’s IRIS Program: Evaluating the Science and Process behind Chemical Risk Assessment, 2011: Hearing before the Subcommittee on Investigations and Oversight of the Committee on Science, Space, and Technology, House of Representatives, 112th Cong., 1st Sess., July 14, 2011. 10

OCR for page 10
Introduction 11 ment of the most recent draft IRIS assessment of formaldehyde began in October 2004, and the draft was released to the public in June 2010. The US Government Accountability Office con- cluded in 2008 that “the IRIS database is at serious risk of becoming obsolete because EPA has not been able to routinely complete timely, credible assessments or decrease its backlog of 70 ongoing assessments” (GAO 2008, p.1). Over the last decade, EPA risk-assessment guidance documents have reiterated EPA’s pol- icy of evaluating and integrating evidence with an approach that is consistent, comprehensive, balanced, and reproducible (EPA 2002, 2003, 2004, 2005). However, NRC committees have conducted several reviews of some of the more complex and challenging IRIS assessments in the last decade and have identified methodologic problems and pointed out deficiencies in EPA’s approaches. For example, the NRC committee that reviewed the dioxin reassessment found prob- lems with the noncancer assessment and stated that “EPA does not use a rigorous approach for evaluating evidence from studies and the weight of their evidence” (NRC 2006, p. 47). The NRC committee that reviewed the tetrachloroethylene assessment offered similar criticisms and em- phasized that “the overall impression is that data are presented to support a positive association between tetrachloroethylene and cancer, and that studies that found no such association are criti- cized or minimized” (NRC 2010, p. 85). In June 2010, EPA released the IRIS formaldehyde assessment. Recognizing the complex nature of the assessment and its importance as the basis of risk calculations and regulatory deci- sions for this high-production chemical, EPA asked NRC to review the assessment and answer questions related specifically to the derivation of the inhalation reference concentration (RfC) and unit risk values. The NRC committee convened to conduct that task released its report in 2011. It identified problems similar to those expressed by earlier NRC committees and conclud- ed that “the draft was not prepared in a consistent fashion; it lacks clear links to an underlying conceptual framework; and it does not contain sufficient documentation on methods and criteria for identifying evidence from epidemiologic and experimental studies, for critically evaluating individual studies, for assessing the weight of evidence, and for selecting studies for derivation of the RfCs and unit risk estimates ” (NRC 2011, p. 4). As noted, the committee provided specif- ic recommendations for revision of the IRIS formaldehyde assessment. It also made general sug- gestions for improvement of the IRIS process and provided a roadmap for its revision as guid- ance if EPA decided to move forward with changes in the process. IMPROVEMENTS IN THE INTEGRATED RISK INFORMATION SYSTEM After release of the NRC formaldehyde report (NRC 2011), Congress held several hearings to examine the objectivity and credibility of IRIS assessments and the program.2,3 On July 12, 2011, EPA (2011) emphasized its commitment to respond to the recommendations in the NRC formaldehyde report and to improve the IRIS program further. Figure 1-1 highlights EPA’s ac- tions and demonstrates its commitment to improve IRIS since release of the NRC formaldehyde report. As requested by Congress, EPA gave relevant congressional committees a progress report in June 2012 that described a phased approach to implementing the NRC recommendations. EPA noted that its first action was “streamlining documents, increasing transparency and clarity, and using more tables and figures to present information and data in assessments” (EPA 2012a, p. 12). Specific improvements that were described in the EPA progress report included developing 2 EPA’s IRIS Program: Evaluating the Science and Process behind Chemical Risk Assessment, 2011: Hearing before the Subcommittee on Investigations and Oversight of the Committee on Science, Space, and Technology, House of Representatives, 112th Cong., 1st Sess., July 14, 2011. 3 Chemical Risk Assessment: What Works for Jobs and Economy? 2011: Hearing Before the Subcom- mittee on Environment and the Economy, Committee on Energy and Commerce, House of Representatives, 112th Cong., 1st Sess., October 6, 2011.

OCR for page 10
12 Review of EPA’s Integrated Risk Information System (IRIS) Process a new document structure with an executive summary to highlight major toxicologic findings and a preamble to outline approaches for identifying and evaluating studies, weighing evidence, se- lecting studies for deriving toxicity values, and deriving toxicity values. EPA also noted that lit- erature-search strategies and criteria for evaluating studies would be explicitly described in new assessments and that it was developing a framework for reaching conclusions on noncancer ef- fects, although it indicated that this effort would be implemented in a later phase. EPA empha- sized, however, that it would be using more systematic approaches for analyzing data. Finally, EPA noted that it was expanding efforts for early peer and stakeholder consultation by hosting public workshops on various issues and that it was forming the Chemical Assessment Advisory Committee (CAAC) under the auspices of its Science Advisory Board. The purpose of the CAAC is to advise the agency on specific assessments and possibly on broader issues. Regarding peer consultation, EPA held a public stakeholder meeting in November 2012 to hear the needs of IRIS users and their views on improvements needed in the IRIS program (EPA 2012b). Release of NRC formaldehyde report April 2011 EPA announces commitment to July 2011 responding to NRC recommendations Congress directs EPA IRIS program to incorporate, as appropriate, NRC December 2011 recommendations into chemical assessments EPA provides Congress with progress report June 2012 on implementing NRC recommendations Kenneth Olden becomes director, EPA July 2012 National Center for Environmental Assessment EPA holds first stakeholder meeting to hear November 2012 views on needed improvements in IRIS EPA provides draft template, preamble, handbook, and chemical-specific examples to NRC January 2013 IRIS committee and the public First meeting of Chemical Assessment April 2013 Advisory Committee of the Science Advisory Board EPA provides IRIS committee with updates on improvements and additional chemical-specific July 2013 examples EPA announces additional improvements in IRIS August 2013 EPA holds systematic review workshop August 2013 FIGURE 1-1 Timeline of events since release of the NRC report, Review of the Environmental Protection Agency’s Draft IRIS Assessment of Formaldehyde. The timeline does not include all the meetings that have been held concerning IRIS and chemical-specific assessments.

OCR for page 10
Introduction 13 On January 30, 2013, EPA provided materials to the present committee that outlined its ac- tions in response to each of the recommendations offered in Chapter 7 of the NRC formaldehyde report (NRC 2011) and indicated the status of implementation of each change. It also provided the new IRIS document template, the draft preamble, a draft handbook for the IRIS assessment development, and several chemical-specific examples of its new approaches. The committee was later given a fact sheet issued on July 31, 2013 (EPA 2013) that restated EPA’s commitment to improving IRIS assessments and that highlighted some of the improvements that it had described previously, such as the new document structure and the use of systematic-review methods. It also noted changes to enhance productivity and transparency, such as focusing staff on fewer assess- ments and introducing stopping rules for the inclusion of new data or scientific issues. EPA de- scribed several other program improvements, including planning and scoping meetings for each assessment; making public at early stages critical pieces of draft assessments, such as literature searches, evidence tables, and dose-response figures; providing a forum to receive public com- ments; and ensuring opportunities for the public to make comments on draft assessments once released. Finally, the committee was updated on August 20, 2013, on the implementation of sev- eral NRC recommendations and was given additional chemical-specific examples of assessment documents. The materials provided to the committee are listed in Table 1-1 and discussed and reviewed in greater depth in the chapters that follow. SYSTEMATIC REVIEW In 2011, the Institute of Medicine (IOM) released the report Finding What Works in Health Care: Standards for Systematic Review. As defined by the IOM report, systematic review is “a scientific investigation that focuses on a specific question and uses explicit, prespecified scien- tific methods to identify, select, assess, and summarize the findings of similar but separate stud- ies” (IOM 2011, p. 1). Although the report was written in the context of comparative- effectiveness research, systematic-review methods have been used for decades in a variety of fields, from agriculture to education (Light and Pillemer 1984; Chalmers et al. 2002). Systematic reviews might or might not result in a quantitative summary of the data, such as an effect esti- mate. A specific approach to summarize quantitatively somewhat more homogeneous infor- mation is often referred to as meta-analysis. TABLE 1-1 Materials Received from the US Environmental Protection Agency Document Content Date Received Part 1: Status of Implementation Status report on implementation January 30, 2013 of Recommendations IRIS toxicologic review template Preamble to IRIS toxicologic reviews Example of directions for contractors Information on Comment Tracker Database Information on scoping for IRIS assessments Draft handbook for IRIS assessment development Part 2: Chemical-Specific Examples Seven chemical-specific examples of implementation January 30, 2013 of various NRC recommendations EPA Fact Sheet Summary of enhancements of IRIS program July 31, 2013 E-mail from IRIS program director Responses to NRC committee inquiries August 20, 2013 Attachments: Tables documenting implementation of NRC recommendations from draft ammonia, trimethylbenzene, and benzo[a]pyrene assessments IRIS Toxicological Review of Draft IRIS assessment of benzo[a]pyrene and August 20, 2013 Benzo[a]pyrene supplemental materials IRIS Toxicological Review of Methanola Draft IRIS noncancer assessment of methanol September 30, 2013 a The IRIS methanol assessment was not sent directly to the committee but was released while the commit- tee was conducting its review and therefore was used as an example of implemented changes.

OCR for page 10
14 Review of EPA’s Integrated Risk Information System (IRIS) Process The materials and examples provided by EPA indicate that the agency is incorporating sys- tematic-review principles as it makes changes in the IRIS process. Figure 1-2 shows systematic review as the present committee envisions its use in the context of the IRIS process. As noted above, one might be able to use meta-analysis to summarize quantitatively the results of the sys- tematic reviews of each data stream (human, animal, and mechanistic). Meta-analysis is also one analytic approach that could be used to integrate evidence across data streams for hazard identi- fication (see Chapter 6) and that could be used for combining data or dose-response estimates of individual studies to derive toxicity values (see Chapter 7). The terms weight of evidence (WOE) and WOE analysis are sometimes used interchangea- bly with systematic review, and there is often confusion surrounding the meanings of the terms. However, the committee views a WOE analysis as a judgment-based process for evaluating the strength of evidence to infer causation, that is, as one approach to integrating evidence for hazard identification. The meaning and use of WOE analysis are discussed further in Chapter 6 of this report. THE COMMITTEE, ITS TASK, AND ITS APPROACH The committee, which was convened, included experts in epidemiology, toxicology, dose- response modeling, risk assessment, systematic review, and risk communication (see Appendix A for biographic information on the committee). As noted earlier, it was asked to assess the sci- entific, technical, and process changes that EPA is making in the IRIS process. The verbatim statement of task is provided in Box 1-1. The committee held six committee meetings to accomplish its task. Open sessions were held during the first and second meetings in which the committee heard from the sponsor on changes being made in the IRIS process and from staff of the National Toxicology Program on changes being made in its chemical-assessment program. During the third meeting, the commit- tee held a workshop that involved participants from academe, government agencies, and private organizations to address approaches used to evaluate and integrate evidence for use in an IRIS assessment (see Appendix B for the workshop agenda). In each open session, interested parties were allowed to address the committee. The committee reviewed materials provided by EPA that document changes that it has made or is planning to make in the IRIS process and materials submitted by interested parties. Scoping Human Human Human Develop Problem Protocols for Animal Identify Animal Evaluate Animal Integrate Formulation Systematic Evidence Studies Evidence Reviews Mechanistic Mechanistic Mechanistic Systematic Reviews Broad Literature Search Dose- Response Assessment Hazard and Derivation Identification of Toxicity Values FIGURE 1-2 Systematic review in the context of the IRIS process. The committee views public input and peer review as integral parts of the IRIS process, although those activities are not specifically noted in the figure.

OCR for page 10
Introduction 15 BOX 1-1 Statement of Task A committee of the National Research Council (NRC) will assess the scientific, tech- nical, and process changes being implemented by the U.S. Environmental Protection Agency (EPA) for its Integrated Risk Information System (IRIS). Specifically, the commit- tee will review the IRIS process and the changes being implemented or planned by EPA and will recommend modifications or additional changes as appropriate to improve the scientific and technical performance of the IRIS program. The committee will focus on the development of the IRIS assessments rather than the review process that follows draft development. Because several reviews of IRIS assessments have expressed con- cerns about EPA’s weight-of-evidence analyses, the committee will review current meth- ods for evidence-based reviews and recommend approaches for weighing scientific evi- dence for chemical hazard and dose-response assessments. ORGANIZATION OF REPORT The present report is organized into eight chapters and three appendixes. Chapter 2 pro- vides an overview of some general issues associated with IRIS assessments. Chapter 3 describes the need for problem formulation to define the systematic-review questions and protocol devel- opment to describe the methods used in the systematic reviews. Chapters 4 and 5 address evi- dence identification and evidence evaluation, respectively. Chapter 6 discusses evidence integra- tion for hazard identification, and Chapter 7 evaluates methods for deriving toxicity values. Chapter 8 presents some overall findings and considerations for future directions. Appendix A provides biographic information on the committee, Appendix B is the agenda of the committee’s workshop, and Appendix C provides some background information on Bayesian analysis. REFERENCES Chalmers, I., L.V. Hedges, and H. Cooper. 2002. A brief history of research synthesis. Eval. Health Prof. 25(1):12-37. EPA (U.S. Environmental Protection Agency). 2002. Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by the Environmental Protection Agency. EPA/260R-02-008. Office of Environmental Information, U.S. Environmental Protection Agency, Washington, DC [online]. Available: http://www.epa.gov/quality/informationguidelines/do cuments/EPA_InfoQualityGuidelines.pdf [accessed Aug. 13, 2013]. EPA (U.S. Environmental Protection Agency). 2003. A Summary of General Assessment Factors for Evaluat- ing the Quality of Scientific and Technical Information. EPA 100/B-03/001. Science Policy Council, U.S. Environmental Protection Agency, Washington, DC [online]. Available: http://www.epa.gov/stpc/ pdfs/assess2.pdf [accessed Aug. 13, 2013]. EPA (U.S. Environmental Protection Agency). 2004. An Examination of EPA Risk Assessment Principles and Practices. EPA/100/B-04/001. Office of the Science Advisor, U.S. Environmental Protection Agency Washington, DC [online]. Available: http://www.epa.gov/osa/pdfs/ratf-final.pdf [accessed Aug. 13, 2013]. EPA (U.S. Environmental Protection Agency). 2005. Guidelines for Carcinogen Risk Assessment. EPA/630/P- 03/001F. Risk Assessment Forum, U.S. Environmental Protection Agency, Washington, DC [online]. Available: http://www.epa.gov/raf/publications/pdfs/CANCER_GUIDELINES_FINAL_3-25-05.PDF [accessed Aug. 13, 2013]. EPA (U.S. Environmental Protection Agency). 2011. EPA Strengthens Kay Scientific Database to Protect Public Health. EPA News: July 12, 2011 [online]. Available: http://yosemite.epa.gov/opa/admpress. nsf/48f0fa7dd51f9e9885257359003f5342/a3fcd60838197067852578cb00666c4d!OpenDocumen t [accessed Aug. 13, 2013].

OCR for page 10
16 Review of EPA’s Integrated Risk Information System (IRIS) Process EPA (U.S. Environmental Protection Agency). 2012a. EPA’s Integrated Risk Information Program, Pro- gress Report and Report to Congress. Office of Research and Development, U.S. Environmental Pro- tection Agency [online]. Available: http://www.epa.gov/iris/pdfs/irisprogressreport2012.pdf [ac- cessed Aug. 13, 2012]. EPA (U.S. Environmental Protection Agency). 2012b. Integrated Risk Information System Program: Sum- mary Report from November 2012 Public Stakeholder Meeting [online]. Available: http://www.epa. gov/iris/publicmeeting/stakeholders-kickoff/ [accessed Aug. 13, 2013]. EPA (U.S. Environmental Protection Agency). 2013. Enhancements to EPA’s Integrated Risk Information System Program [online]. Available: http://www.epa.gov/iris/pdfs/irisprocessfactsheet2013.pdf [ac- cessed Aug. 13, 2013]. GAO (U.S. Government Accountability Office). 2008. Toxic Chemicals: EPA’s New Assessment Process Will Increase Challenges EPA Faces in Evaluating and Regulating Chemicals. GAO-08-743T. Wash- ington, DC: U.S. Government Accountability Office [online]. Available: http://www.gao.gov/assets/ 120/119860.pdf [accessed Aug. 13, 2013]. IOM (Institute of Medicine). 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: National Academies Press. Light, R.J., and D.B. Pillemer. 1984. Summing Up: The Science of Reviewing Research. Cambridge MA: Harvard University Press. NRC (National Research Council). 2006. Health Risks from Dioxin and Related Compounds: Evaluation of the EPA Reassessment. Washington, DC: National Academies Press. NRC (National Research Council). 2010. Review of the Environmental Protection Agency’s Draft IRIS Assessment of Tetrachloroethylene. Washington, DC: National Academies Press. NRC (National Research Council). 2011. Review of the Environmental Protection Agency’s Draft IRIS Assessment of Formaldehyde. Washington, DC: National Academies Press.