National Academies Press: OpenBook

Progress Toward Transforming the Integrated Risk Information System (IRIS) Program: A 2018 Evaluation (2018)

Chapter: Appendix E: Committee Findings Regarding 2014 Recommendations

« Previous: Appendix D: Posters by US Environmental Protection Agency
Suggested Citation:"Appendix E: Committee Findings Regarding 2014 Recommendations." National Academies of Sciences, Engineering, and Medicine. 2018. Progress Toward Transforming the Integrated Risk Information System (IRIS) Program: A 2018 Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/25086.
×
Suggested Citation:"Appendix E: Committee Findings Regarding 2014 Recommendations." National Academies of Sciences, Engineering, and Medicine. 2018. Progress Toward Transforming the Integrated Risk Information System (IRIS) Program: A 2018 Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/25086.
×
Item Chapter Recommendations from 2014 NRC Reporta Finding Evidence
1A 2 EPA needs to complete the changes in the IRIS process that are in response to the recommendations in the [2011] NRC formaldehyde report. The 2014 report reviewed and encapsulated recommendations from the 2011 report, so the present committee focused its review on assessing progress made in implementing recommendations made by the 2014 report. Workshop presentations, posters, and discussion
Recent IRIS documents (such as plans, protocols, and assessments) and tools.
1B 2 [EPA needs to] specifically complete documents, such as the draft handbook, that provide detailed guidance for developing IRIS assessments. When those changes and the detailed guidance, such as the draft handbook, have been completed, there should be an independent and comprehensive review that evaluates how well EPA has implemented all the new guidance. The present committee is completing its report while those revisions are still in progress. The revised handbook was not provided to the committee. EPA staff indicated that the handbook is under internal agency review and that its public release is expected in 2018. The agency further indicated that standard operating procedures might evolve as the IRIS program gains additional experience in performing systematic review and using emerging methods. The committee expects handbook revisions to be a continuing process, and EPA similarly characterizes the IRIS handbook as “evergreen.” The committee observed that guidance for conducting newly planned IRIS assessments is contained in protocols, and EPA stated that some material currently in protocols might reside in the handbook. The amount of and need for overlap in the protocols and handbook could not be judged without seeing the handbook. Slides 21–22
Systematic Review Protocol for the IRIS Chloroform Assessment (Inhalation)b
2 2 EPA should provide a quality-management plan that includes clear methods for continuing assessments of the quality of the process. The roles of the various internal entities involved in the process, such as the CASTs, should be described. The assessments should be used to improve the overall process and the performance of EPA staff and contractors. IRIS management has taken multiple steps to ensure high-quality management, including the creation of expertise-specific work groups, systematic-review work groups, and other intermediate structures to improve the quality of the IRIS assessments. EPA has also used the SAB Chemical Assessment Advisory Committee to review IRIS assessments. Funding for contractors has decreased. Slides 7–10, 151
The GAO audit of the IRIS program indicates that improvements in program management have occurred (Slide 10)
3 2 When extracting data for evidentiary tables, EPA should use at least two reviewers to assess each study independently for risk of bias. The reliability of the independent coding should be calculated; if there is good agreement, multiple reviewers might not be necessary. EPA uses two people to extract data and, when needed, involves a third person to resolve conflicts. EPA also uses two people to complete the risk-of-bias evaluation. Slide 39
Systematic Review Protocol for the IRIS Chloroform Assessment (Inhalation) (p. 17, line 1; p. 30, lines 18–20)b
4A 2 EPA should continue its efforts to develop clear and transparent processes that allow external stakeholder input early in the IRIS process. EPA has adopted the process of soliciting public comment early through the release of assessment plans and protocols for public comment. IRIS Web site
Slides 24–25, 29
4B 2 [EPA] should develop communication and outreach tools that are tailored to meet the needs of the various stakeholder groups. For example, EPA might enhance its engagement with the scientific community through interactions at professional-society meetings, advertised workshops, and seminars. In contrast, greater use of social media might help to improve communications with environmental advocacy groups and the public. Although this recommendation was not discussed specifically with EPA, the agency has worked in the past with the National Academies to identify experts that could provide input at IRIS workshops. The IRIS Web site provides features for sharing information via social-media tweets and Facebook. The calendar feature clearly indicates the schedule for public engagement events on IRIS assessments. EPA staff also discussed data-and tool-sharing with stakeholders to increase understanding and accessibility of systematic-review practices used to develop IRIS assessments. IRIS Web site
Slide 15
Suggested Citation:"Appendix E: Committee Findings Regarding 2014 Recommendations." National Academies of Sciences, Engineering, and Medicine. 2018. Progress Toward Transforming the Integrated Risk Information System (IRIS) Program: A 2018 Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/25086.
×
Item Chapter Recommendations from 2014 NRC Reporta Finding Evidence
5 2 Similar to other EPA technical-assistance programs, EPA should consider ways to provide technical assistance to under-resourced stakeholders to help them to develop and provide input to the IRIS program. This recommendation was not discussed specifically with EPA.
6 2 The stopping rules should be explicit and transparent, should describe when and why the window for evidence inclusion should be expanded, and should be sufficiently flexible to accommodate truly pivotal studies. Such rules could be included in the preamble. The issue of stopping rules was not specifically discussed, but the IRIS program has completed a rapid review of chloroprene, and this is consistent with this recommendation. Systematic Review of Chloroprene Studies Published Since 2010 IRIS Assessmentc
7 2 Regarding promotion of efficiencies, EPA should continue to expand its efforts to develop computer systems that facilitate storage and annotation of information relevant to the IRIS mission and to develop automated literature and screening procedures, sometimes referred to as text-mining. EPA has made considerable progress in developing and upgrading the Health and Environmental Research Online (HERO) database and the Health Assessment Workspace Collaborative (HAWC) computer system to facilitate storage and annotation of data. Those systems are not subject to third party control. EPA is also using other software systems, including the Sciome Workbench for Interactive computer-Facilitated Text-mining (SWIFT) and related products for text-mining. Workshop Demonstrations
Slides 36, 92–116
8 2 More details need to be provided on the recognition and applications of expert judgment throughout the assessment-development process, especially in the later stages of the process. The points at which expert judgment is applied should be identified, those applying the judgment should be listed, and consideration should be given to harmonizing the use of expert judgment at various points in the process. EPA has developed guided expert judgment to synthesize evidence on the basis of modified Bradford Hill criteria and for integrating evidence across data streams. The agency has developed working groups with expertise (such as PBPK) that can be applied to the assessment process. The draft chloroform protocol identified some situations when expert judgment will be used, including evaluation of studies to identify characteristics that indicate how informative the results are (p. 16, line 21) to perform outcome-specific study evaluations (p. 16, line 24). Systematic Review Protocol for the IRIS Chloroform Assessment (Inhalation)b

Slides 8, 48, 69–86
9 3 EPA should establish a transparent process for initially identifying all putative adverse outcomes through a broad search of the literature. The agency should then develop a process that uses guided expert judgment to identify the specific adverse outcomes to be investigated, each of which would then be subjected to systematic review of human, animal, and in vitro or mechanistic data. EPA has developed assessment plans that provide information about the scoping and problem formulation process. The plans are developed by using expert judgment and input from EPA regional offices and other stakeholders. Each assessment plan identifies the specific aims of the systematic review and the PECO statement. IRIS Assessment Plan for Chloroform (Scoping and Problem Formulation Materials)d
10 3 For all literature searches, EPA should consult with an information specialist who is trained in conducting systematic reviews. EPA staff indicated that they use an information specialist. EPA protocol provides the name of the HERO librarian (see chloroform protocol, page vii);b that person has an MS in library and information science
11 3 EPA should include protocols for all systematic reviews conducted for a specific IRIS assessment as appendixes to the assessment. The IRIS program has developed draft systematic-review protocols that are undergoing public comment before being made final. The protocols contain many of the elements identified by the 2014 Systematic Review Protocol for the IRIS Chloroform Assessment (Inhalation), Appendix Ab
Suggested Citation:"Appendix E: Committee Findings Regarding 2014 Recommendations." National Academies of Sciences, Engineering, and Medicine. 2018. Progress Toward Transforming the Integrated Risk Information System (IRIS) Program: A 2018 Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/25086.
×
report as meeting best practices defined by IOM. Furthermore, the chloroprene reassessment included as appendixes the literature-search strategy and approaches for evaluating risk of bias in epidemiology and other human studies. The study objective, PECO statement, and methods used to search and screen the literature and evaluate studies were included in the main body of the report. That approach is consistent with this recommendation. The committee expects that some items found in the protocol can be addressed in the handbook. Including the analysis plan in the systematic-review protocols might lead to additional amendments to the protocol that could be minimized if they used a separate analysis plan. Systematic Review of Chloroprene Studies Published Since 2010 IRIS Assessmentc
12 4 The trajectory of change needs to be maintained. The IRIS program has been responsive to the recommendations made in the 2014 report and is continuing the trajectory of change. The changes appear to have accelerated with the recruitment of new NCEA and IRIS leadership. Workshop presentations, posters, and discussion
Recent IRIS documents (such as plans, protocols, and assessments) and tools
13 4 The current process can be enhanced with more explicit documentation of methods. Protocols for IRIS assessments should include a section on evidence identification that is written in collaboration with information specialists trained in systematic reviews and that includes a search strategy for each systematic-review question being addressed in the assessment. Specifically, the protocols should provide a line-by-line description of the search strategy, the date of the search, and publication dates searched and, as noted in Chapter 3, explicitly state the inclusion and exclusion criteria for studies. EPA systematic-review protocols contain descriptions of how evidence will be identified, including relevant search terms and databases to be queried. The protocols also include descriptions of inclusion and exclusion criteria. Systematic Review Protocol for the IRIS Chloroform Assessment (Inhalation), Table 2 (p. 9), p. 12, Appendix Ab

Systematic Review of Chloroprene Studies Published Since 2010 IRIS Assessmentc
14 4 Evidence identification should involve a predetermined search of key sources, follow a search strategy based on empirical research, and be reported in a standardized way that allows replication by others. The search strategies and sources should be modified as needed on the basis of new evidence on best practices. Contractors who perform the evidence identification for the systematic review should adhere to the same standards and provide evidence of experience and expertise in the field. EPA systematic-review protocols contain descriptions of how evidence will be identified, including relevant search terms and databases to be queried. Systematic Review Protocol for the IRIS Chloroform Assessment (Inhalation), Appendix Ab
15 4 EPA should consider developing specific resources, such as registries, that could be used to identify and retrieve information about toxicology studies reported outside the literature accessible by electronic searching. In the medical field, clinical-trial registries and US legislation that has required studies to register in ClinicalTrials.gov have been an important step in ensuring that the total number of studies that are undertaken is known. This recommendation goes beyond the scope of the IRIS program and therefore was not addressed by the committee during its review. Systematic-review protocols indicate that IRIS assessments include only publicly accessible, peer-reviewed information, which should be available through the databases identified by the IRIS program. Systematic Review Protocol for the IRIS Chloroform Assessment (Inhalation) (p. 11, line 14)b
Suggested Citation:"Appendix E: Committee Findings Regarding 2014 Recommendations." National Academies of Sciences, Engineering, and Medicine. 2018. Progress Toward Transforming the Integrated Risk Information System (IRIS) Program: A 2018 Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/25086.
×
Item Chapter Recommendations from 2014 NRC Reporta Finding Evidence
16 4 EPA is encouraged to use at least two reviewers who work independently to screen and select studies, pending an evaluation of validity and reliability that might indicate that multiple reviewers are not warranted. It is important that the reviewers use standardized procedures and forms. EPA uses two persons to screen studies. Screeners use a structured form based on the PECO in DistillerSR. Systematic Review Protocol for the IRIS Chloroform Assessment (Inhalation) (p. 12, lines 11–16)b Slide 39
17 4 EPA should engage information specialists trained in systematic reviews in the process of evidence identification, for example, by having an information specialist peer review the proposed evidence-identification strategy in the protocol for the systematic review. The IRIS assessment team includes an information specialist. The specific tasks completed by that person are not clear. It is hoped that the handbook will clearly define the roles that the person has in the IRIS process. Systematic Review Protocol for the IRIS Chloroform Assessment (Inhalation)b
18 4 EPA should encourage and support research on reporting biases and other methodologic topics relevant to the systematic-review process in toxicology. EPA is supporting and encouraging research through its collaborative efforts described at the workshop. The committee expects EPA research in this field to emerge as the IRIS program continues to develop expertise in systematic-review method development. Slides 79, 91, 149, 145, 150
19 4 EPA should continue to document and standardize its evidence-identification process by adopting (or adapting, where appropriate) the relevant IOM standards described in Table 4-1. It is anticipated that its efforts will further strengthen the overall consistency, reliability, and transparency of the evidence-identification process. Appropriate tools and methods for evidence identification were described and are being used. Systematic Review Protocol for the IRIS Chloroform Assessment (Inhalation)b
Workshop presentations
20A 5 To advance the development of tools for assessing risk of bias in different types of studies (human, animal, and mechanistic) used in IRIS assessments, EPA should explicitly identify factors, in addition to those discussed in this chapter, that can lead to bias in animal studies—such as control for litter effects, dosing, and methods for exposure assessment—so that these factors are consistently evaluated for experimental studies. The draft chloroform protocol describes the domains to be considered in the evaluation of epidemiology studies and animal toxicity studies. Domain ratings and their descriptions have also been provided. EPA also presented heat maps of risk-of-bias analyses for studies performed by the IRIS program. Systematic Review Protocol for the IRIS Chloroform Assessment (Inhalation) (Tables 5–6)b

Slides 53, 55
20B 5 Likewise, EPA should consider a tool for assessing risk of bias in in vitro studies. The 2014 report noted that few tools were available for assessing risk of bias in in vitro studies. Fully developed tools that meet the needs of the IRIS program are not available. EPA is exploring adaptations of existing tools for its purpose. Slide 78
21A 5 When considering any method for evaluating individual studies, EPA should select a method that is transparent, reproducible, and scientifically defensible. EPA has adopted systematic-review methods that are transparent and scientifically defensible. Systematic Review Protocol for the IRIS Chloroform Assessment (Inhalation)b
Systematic Review of Chloroprene Studies Published Since 2010 IRIS Assessmentc
Slides 50–63
Suggested Citation:"Appendix E: Committee Findings Regarding 2014 Recommendations." National Academies of Sciences, Engineering, and Medicine. 2018. Progress Toward Transforming the Integrated Risk Information System (IRIS) Program: A 2018 Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/25086.
×
21B 5 Whenever possible, there should be empirical evidence that the methodologic characteristics that are being assessed in the IRIS protocol have systematic effects on the direction or magnitude of the outcome. The methodologic characteristics that are known to be associated with a risk of bias should be included in the assessment tool. Additional quality-assessment items relevant to a particular systematic-review question could also be included in the EPA assessment tool. EPA is using and adapting risk-of-bias tools appropriately. Systematic Review Protocol for the IRIS Chloroform Assessment (Inhalation)b
Systematic Review of Chloroprene Studies Published Since 2010 IRIS Assessmentc
Slides 52–63
Posters D-4, D-5, D-9
22 5 EPA should carry out, support, or encourage research on the development and evaluation of empirically based instruments for assessing bias in human, animal, and mechanistic studies relevant to chemical-hazard identification. Specifically, there is a need to test existing animal-research assessment tools on other animal models of chemical exposures to ensure their relevance and generalizability to chemical-hazard identification. Furthermore, EPA might consider pooling data collected for IRIS assessment to determine whether, among various contexts, candidate risk-of-bias items are associated with overestimates or underestimates of effect. EPA is supporting and encouraging research through its collaborative efforts described in the workshop. Slides 145, 149
23 5 Although additional methodologic work might be needed to establish empirically supported criteria for animal or mechanistic studies, an IRIS assessment needs to include a transparent evaluation of the risk of bias of studies used by EPA as a primary source of data for the hazard assessment. EPA should specify the empirically based criteria it will use to assess risk of bias for each type of study design in each type of data stream. EPA has adapted existing risk-of-bias tools for its use. Draft protocols describe the domains to be considered in the evaluation of epidemiology studies and animal toxicity studies. Domain ratings and their descriptions have also been provided. EPA also presented heat maps of risk-of-bias analyses for studies performed by the IRIS program. Tools have not been developed for mechanistic studies. Systematic Review Protocol for the IRIS Chloroform Assessment (Inhalation) (Tables 5 and 6)b

Slides 53, 78
24 5 To maintain transparency, EPA should publish its risk-of-bias assessments as part of its IRIS assessments. It could add tables that describe the assessment of each risk-of-bias criterion for each study and provide a summary of the extent of the risk of bias in the descriptions of each study in the evidence tables. EPA presented example heat maps of risk-of-bias analyses for studies performed by the IRIS program. The heat maps have been included in a recent assessment. Systematic Review of Chloroprene Studies Published Since 2010 IRIS Assessment (Figure 2)c
25 5 EPA should develop terminology for potential sources of bias with definitions that can be applied during systematic reviews. EPA has adapted existing risk-of-bias tools for its use. The draft chloroform protocol describes the domains to be considered in the evaluation of epidemiology studies and animal toxicity studies. Reporting bias was not included as a domain for epidemiology studies, and its omission is not consistent with standard systematic-review methods for assessing risk of bias. Systematic Review Protocol for the IRIS Chloroform Assessment (Inhalation) (Tables 5–6)b

Slides 55, 57
Suggested Citation:"Appendix E: Committee Findings Regarding 2014 Recommendations." National Academies of Sciences, Engineering, and Medicine. 2018. Progress Toward Transforming the Integrated Risk Information System (IRIS) Program: A 2018 Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/25086.
×
Item Chapter Recommendations from 2014 NRC Reporta Finding Evidence
26 5 Funding sources should be considered in the risk-of-bias assessment conducted for systematic reviews that are part of an IRIS assessments EPA documents funding source, but it is unclear how the data are used. Workshop discussion
27A 5 EPA should contact investigators to obtain missing information that is needed for the evaluation of risk of bias and other quality characteristics of included studies. Investigators are contacted on a case-by-case basis that depends partly on the expected effect of the missing data. IRIS systematic-review protocols also indicate that decisions are made on an assessment-specific basis. If the information is not reported, it is generally not useful to reach out to the study authors. However, if missing study details could change confidence in study conclusions, efforts should be made to contact the study authors. Outreach to study authors is documented and considered unsuccessful if researchers do not respond to multiple e-mail or phone requests within a reasonable period. Systematic Review Protocol for the IRIS Chloroform Assessment (Inhalation) (Table 6 (p. 25); p. 18, line 41)b
27B 5 The committee expects that, as happened in the clinical literature in which additional reporting standards for journals were implemented (Turner et al. 2012), the reporting of toxicologic research will eventually improve as risk-of-bias assessments are incorporated into the IRIS program. However, a coordinated approach by government agencies, researchers, publishers, and professional societies will be needed to improve the completeness and accuracy of reporting toxicology studies in the near future. This recommendation goes beyond the scope of the IRIS program and therefore was not addressed during this review.
28 5 The risk-of-bias assessment of individual studies should be carried forward and incorporated into the evaluation of evidence among data streams. The results of the evaluation of individual studies are a critical component of the current evidence synthesis processes and integration frameworks. Risk of bias is one factor that EPA uses to determine an overall study confidence rating for epidemiology and animal toxicity studies. High- or medium-confidence studies are favored for quantitative dose–response analysis. Slides 66, 54, 71–73, 81
29 6 EPA should continue to improve its evidence-integration process incrementally and enhance the transparency of its process. It should either maintain its current guided-expert-judgment process but make its application more transparent or adopt a structured (or GRADE-like) process for evaluating evidence and rating recommendations along the lines that NTP has taken. If EPA does move to a structured evidence-integration process, it should combine resources with NTP to leverage the intellectual resources and scientific experience in both organizations. The committee does not offer a preference but suggests that EPA consider which approach best fits its plans for the IRIS process. The IRIS process continues to use a guided expert judgment process, but structured sets of categorical criteria for decision-making within that process are more explicitly defined. Slides 67, 79–86
Suggested Citation:"Appendix E: Committee Findings Regarding 2014 Recommendations." National Academies of Sciences, Engineering, and Medicine. 2018. Progress Toward Transforming the Integrated Risk Information System (IRIS) Program: A 2018 Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/25086.
×
30 6 EPA should expand its ability to perform quantitative modeling of evidence integration; in particular, it should develop the capacity to do Bayesian modeling of chemical hazards. That technique could be helpful in modeling assumptions about the relevance of a variety of animal models to each other and to humans, in incorporating mechanistic knowledge to model the relevance of animal models to humans and the relevance of human data for similar but distinct chemicals, and in providing a general framework within which to update scientific knowledge rationally as new data become available. The committee emphasizes that the capacity for quantitative modeling should be developed in parallel with improvements in existing IRIS evidence-integration procedures and that IRIS assessments should not be delayed while this capacity is being developed. EPA illustrated its use of meta-analysis of human and animal studies for evidence integration. Bayesian methods are being explored to help to characterize uncertainty and to combine evidence to identify hazard. New methods and assays are increasingly being evaluated quantitatively. Slide 130
Posters provided examples that show how EPA uses new approach methods as part of a chemical assessment process
31 6 EPA should develop templates for structured narrative justifications of the evidence-integration process and conclusion. The premises and structure of the argument for or against a chemical’s posing a hazard should be made as explicit as possible, should be connected explicitly to evidence tables produced in previous stages of the IRIS process, and should consider all lines of evidence (human, animal, and mechanistic) used to reach major conclusions. The 2017 Toxicological Profile for Benzo[a]pyrene shows well-developed evidence tables that support the structured narrative and conclusion regarding carcinogenicity. For other effects, the evidence is described as ranging from “strongest evidence for human hazards” to “less robust evidence.” Workshop discussion and the chloroform protocol show progress in template development. EPA staff stated that the approach to standardization of hazard descriptors for noncancer effects is being tested and discussed in the agency. Slides 80–86
2017 IRIS Toxicological Profile for Benzo[a]pyrenee
Systematic Review Protocol for the IRIS Chloroform Assessment (Inhalation)b
32 6 Guidelines for evidence integration for cancer and noncancer end points should be more uniform. Although EPA has not developed these guidelines, the issue goes beyond the IRIS program with respect to agency procedures. However, the IRIS program has developed frameworks for evidence integration and is testing and discussing how conclusions should be summarized.
33 7 EPA should develop criteria for determining when evidence is sufficient to derive toxicity values. One approach would be to restrict formal dose-response assessments to when a standard descriptor characterizes the level of confidence as medium or high (as in the case of noncancer end points) or as “carcinogenic to humans” or “likely to be carcinogenic to humans” for carcinogenic compounds. Another approach, if EPA adopts probabilistic hazard classification, is to conduct formal dose-response assessments only when the posterior probability that a human hazard exists exceeds a predetermined threshold, such as 50% (more likely than not likely that the hazard exists). Progress has been made. Quantitative toxicity values are restricted to studies with strongest conclusions for a human health effect (for cancer, a descriptor of Known) or a moderately strong conclusion for a human health effect (for cancer, a descriptor of Likely). Criteria are not provided for inclusion of studies that are considered on a case-by-case basis when a weaker conclusion regarding a human health effect (for cancer, a descriptor of Suggestive) is reached. IRIS has not produced final descriptors for noncancer effects and mechanistic studies other than review and application of PK/PBPK models. Systematic Review of Chloroprene Studies Published Since 2010 IRIS Assessmentc
2017 IRIS Toxicological Profile for Benzo[a]pyrenee
Slides 131–133
Suggested Citation:"Appendix E: Committee Findings Regarding 2014 Recommendations." National Academies of Sciences, Engineering, and Medicine. 2018. Progress Toward Transforming the Integrated Risk Information System (IRIS) Program: A 2018 Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/25086.
×
Item Chapter Recommendations from 2014 NRC Reporta Finding Evidence
34 7 EPA should continue its shift toward the use of multiple studies rather than single studies for dose-response assessment but with increased attention to risk of bias, study quality and relevance in assessing human dose-response relationships. For that purpose, EPA will need to develop a clear set of criteria for judging the relative merits of individual mechanistic, animal, and epidemiologic studies for estimating human dose-response relationships. Progress has been made toward using multiple studies or end points and comparing multiple candidate toxicity values. IRIS assessments provide one or more candidate toxicity values for use by risk managers. The IRIS program considers the quality of studies when deciding which studies will be advanced for quantitative dose–response modeling; studies rated as having medium or high confidence will be advanced for dose–response considerations. Other study attributes—such as relevance of a species to humans, relevance of an exposure route, and susceptibility—might also be considered.
EPA is developing new tools for making and visualizing comparisons. EPA recognizes that there is no one-size-fits-all sets of criteria for inclusion of mechanistic studies, but the criteria for evaluating PK/PBPK models and how they are applied in dose–response and toxicity-value determinations are a good start.
Slides 62, 130–135, 142–146

2012 IRIS Toxicological Review of Tetrachloroethylened

Workshop demonstrations of HAWC and SWIFT
35 7 EPA should use formal methods for combining multiple studies and the derivation of IRIS toxicity values with an emphasis on a transparent and replicable process. IRIS has begun to develop and apply tools in response to this recommendation. EPA presented two demonstrations for meta-regression and Bayesian approaches that showcase the agency efforts. EPA has not presented criteria for when and how new tools should be used. Tool development and application will be a continuing process that requires sustained resources and continued capacity-building. Slide 140
Case studies provided for alternative dose estimates (posters D-2, D-10)
36 7 EPA should clearly present two dose-response estimates: a central estimate (such as a maximum likelihood estimate or a posterior mean) and a lower-bound estimate for a POD from which a toxicity value is derived. The lower bound becomes an upper bound for a cancer slope factor but remains a lower bound for a reference value. EPA indicated that this approach is now standard procedure. Several examples were presented that show comparisons between BMDs and BMDLs and demonstrate how key studies compare with other supporting studies Slides 134, 135; posters
37 7 As the IRIS program evolves, EPA should develop and expand its use of Bayesian or other formal quantitative methods in data integration for dose-response assessment and derivation of toxicity values. Demos show the beginning stage of IRIS efforts on applications of Bayesian methods.
EPA has not yet developed criteria for when and how new tools should be used.
New research is under way to address New Approach Methods, such as data-mining, cheminformatics, high-throughput exposure modeling and toxicokinetics, and visualization tools.
Case studies (Poster D-10)

Slides 136, 139, 140, 143–146
Suggested Citation:"Appendix E: Committee Findings Regarding 2014 Recommendations." National Academies of Sciences, Engineering, and Medicine. 2018. Progress Toward Transforming the Integrated Risk Information System (IRIS) Program: A 2018 Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/25086.
×
38 7 Uncertainty analysis should be conducted systematically and coherently in IRIS assessments. To that end, EPA should develop IRIS-specific guidelines to frame uncertainty analysis and uncertainty communication. Moreover, uncertainty analysis should become an integral component of the IRIS process. Efforts are beginning with development of model-averaging approaches and adoption of WHO/IPCS guidance for reporting toxicity values and their uncertainty.
IRIS specific guidance has yet to be developed because tools and approaches remain under development.
Cooper et al. (2016)g
Slides 137, 138
Case studies (Poster D-9)

aNRC. 2014. Review of EPA’s Integrated Risk Information System (IRIS) Process. Washington, DC: The National Academies Press.

bEPA. 2018. Systematic Review Protocol for the IRIS Chloroform Assessment (Inhalation) [CASRN 67-66-3]. EPA/635/R-17/486. Integrated Risk Information System, National Center for Environmental Assessment, Office of Research and Development, U.S. Environmental Protection Agency, Washington, DC.

cOrme-Zavaleta, J. 2018. Response to the Request for Correction (RFC). Letter to Robert Holden, Liskow & Lewis, New Orleans, LA, from Jennifer Orme-Zavaleta, Principal Deputy Assistant Administrator for Science, Office of Research and Development, Washington, DC, January 25, 2018; Attachment 1. EPA Response to the Denka Performance Elastomers (DPE) Request for Correction of the Toxicological Review of Chloroprene (CAS No. 126-99-8) In Support of Summary Information on the Integrated Risk Information System (IRIS); Attachment 2. Systematic Review of Chloroprene [CASRN 126-99-80] Studies Published Since 2010 IRIS Assessment to Support Consideration of the Denka Request for Correction (RFC). January 2018 [online]. Available: https://www.epa.gov/sites/production/files/2018-01/documents/epa_repsonse_to_mr._holdren_jan_25_2018_complete.pdf [accessed February 9, 2018].

dEPA. 2017a. IRIS Assessment Plan for Chloroform [CASRN 67-66-3]. EPA/635/R-17/330. Integrated Risk Information System, National Center for Environmental Assessment, Office of Research and Development, U.S. Environmental Protection Agency, Washington, DC.

eEPA. 2017b. Toxicological Review of Benzo[a]pyrene. EPA/635/R-17/003Fa. Integrated Risk Information System, National Center for Environmental Assessment, Office of Research and Development, U.S. Environmental Protection Agency, Washington, DC [online]. Available: https://cfpub.epa.gov/ncea/iris/iris_documents/documents/toxreviews/0136tr.pdf [accessed February 14, 2018].

fEPA. 2012. Toxicological Review of Tetrachloroethylene (Perchloroethylene) [CAS No. 127-18-4]. EPA/635/R-80/011F. U.S. Environmental Protection Agency, Washington, DC. February 2012 [online]. Available: https://cfpub.epa.gov/ncea/iris/iris_documents/documents/toxreviews/0106tr.pdf [accessed February 27, 2018].

gCooper, G.S., R.M. Lunn, M. Ågerstrand, B.S. Glenn, A.D. Kraft, A.M. Luke, and J.M. Ratcliffe. 2016. Study sensitivity: Evaluating the ability to detect effects in systematic reviews of chemical exposures. Environ. Int. 92-93:605-610.

Suggested Citation:"Appendix E: Committee Findings Regarding 2014 Recommendations." National Academies of Sciences, Engineering, and Medicine. 2018. Progress Toward Transforming the Integrated Risk Information System (IRIS) Program: A 2018 Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/25086.
×
Page 110
Suggested Citation:"Appendix E: Committee Findings Regarding 2014 Recommendations." National Academies of Sciences, Engineering, and Medicine. 2018. Progress Toward Transforming the Integrated Risk Information System (IRIS) Program: A 2018 Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/25086.
×
Page 111
Suggested Citation:"Appendix E: Committee Findings Regarding 2014 Recommendations." National Academies of Sciences, Engineering, and Medicine. 2018. Progress Toward Transforming the Integrated Risk Information System (IRIS) Program: A 2018 Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/25086.
×
Page 112
Suggested Citation:"Appendix E: Committee Findings Regarding 2014 Recommendations." National Academies of Sciences, Engineering, and Medicine. 2018. Progress Toward Transforming the Integrated Risk Information System (IRIS) Program: A 2018 Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/25086.
×
Page 113
Suggested Citation:"Appendix E: Committee Findings Regarding 2014 Recommendations." National Academies of Sciences, Engineering, and Medicine. 2018. Progress Toward Transforming the Integrated Risk Information System (IRIS) Program: A 2018 Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/25086.
×
Page 114
Suggested Citation:"Appendix E: Committee Findings Regarding 2014 Recommendations." National Academies of Sciences, Engineering, and Medicine. 2018. Progress Toward Transforming the Integrated Risk Information System (IRIS) Program: A 2018 Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/25086.
×
Page 115
Suggested Citation:"Appendix E: Committee Findings Regarding 2014 Recommendations." National Academies of Sciences, Engineering, and Medicine. 2018. Progress Toward Transforming the Integrated Risk Information System (IRIS) Program: A 2018 Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/25086.
×
Page 116
Suggested Citation:"Appendix E: Committee Findings Regarding 2014 Recommendations." National Academies of Sciences, Engineering, and Medicine. 2018. Progress Toward Transforming the Integrated Risk Information System (IRIS) Program: A 2018 Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/25086.
×
Page 117
Suggested Citation:"Appendix E: Committee Findings Regarding 2014 Recommendations." National Academies of Sciences, Engineering, and Medicine. 2018. Progress Toward Transforming the Integrated Risk Information System (IRIS) Program: A 2018 Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/25086.
×
Page 118
Suggested Citation:"Appendix E: Committee Findings Regarding 2014 Recommendations." National Academies of Sciences, Engineering, and Medicine. 2018. Progress Toward Transforming the Integrated Risk Information System (IRIS) Program: A 2018 Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/25086.
×
Page 119
Progress Toward Transforming the Integrated Risk Information System (IRIS) Program: A 2018 Evaluation Get This Book
×
Buy Ebook | $34.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Over the past several years, the US Environmental Protection Agency (EPA) has been transforming the procedures of its Integrated Risk Information System (IRIS), a program that produces hazard and dose‒response assessments of environmental chemicals and derives toxicity values that can be used to estimate risks posed by exposures to them. The transformation was initiated after suggestions for program reforms were provided in a 2011 report from the National Academies of Sciences, Engineering, and Medicine that reviewed a draft IRIS assessment of formaldehyde. In 2014, the National Academies released a report that reviewed the IRIS program and evaluated the changes implemented in it since the 2011 report.

Since 2014, new leadership of EPA’s National Center for Environmental Assessment (NCEA) and IRIS program has instituted even more substantive changes in the IRIS program in response to the recommendations in the 2014 report. Progress Toward Transforming the Integrated Risk Information System Program: A 2018 Evaluation reviews the EPA’s progress toward addressing the past recommendations from the National Academies.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!