Making Full Use of Scientific Information in Risk Assessment
This appendix is written in response to Appendix N-1 written by Adam Finkel, which is included in the CAPRA report at the request of the committee. That appendix advocates a principle of "plausible conservatism" for choosing and altering default assumptions and in making cancer risk estimates. It describes this principle as an alternative to the use of best available science and calculation of central tendency risk estimates. This appendix proposes an alternative view to Appendix N-1. We present a different framing of the issue of making full use of science in risk assessment, as opposed to increasing the use of conservative value judgments as described in Appendix N-1.
EPA already practices what we interpret as plausible conservatism in the selection of default options. As set forth in the 1986 Guidelines for Carcinogen Risk Assessment, EPA has selected its default options to be scientifically plausible and protective of human health. EPA's cancer potency estimates are intended to be plausible upper bounds on risk. Neither we nor others on the CAPRA Committee have asserted that these EPA risk assessment procedures are inappropriate. Rather CAPRA has sought to strengthen EPA's risk assessment process through further refinements. One of the potential refinements is an explicit standard for departure from defaults. We have concerns that using plausible conservatism as the standard for departure from defaults, as advocated in Appendix N-1, may not be useful and appropriate.
A major theme of the CAPRA report is that of an iterative approach to risk assessment. EPA should carry out risk assessments at multiple levels, with more detail and more use of site and substance-specific data in the upper tiers of an
iterative process. While simple procedures and single-number estimates are appropriate for screening purposes in lower tiers of risk assessment, explicit disclosure of uncertainty and results from multiple scientifically plausible models are encouraged as part of upper tier risk assessment.
It is assumed in Appendix N-1 that the fundamental output of a risk assessment is a single estimate of risk: one number. We take a very different view, that risk assessment is a process for summarizing the available scientific information in both qualitative and quantitative form, for risk managers and for interested members of the public. Thus, regulatory decisions on managing risks should not be driven solely by single number risk estimates, but rather by a more comprehensive characterization of available scientific information, including uncertainties. We believe the CAPRA report strongly supports this latter interpretation.
An important aspect of risk management is the management of research directed at improving risk assessments by reducing uncertainty, permitting conservative assumptions to be superseded by more accurate models and observational data. The tiered approach to risk assessment and explicit consideration of both model and parameter uncertainties will facilitate identification of the opportunities for research that are most important for achieving the nation's health protection, environmental, and economic goals. We view debate over which conservative assumption to use in risk assessment as a poor substitute for an effective process to identify and pursue research that will improve regulatory decisions by reducing both the uncertainties and the need for the conservative assumptions.
Organization Of This Appendix
In this appendix, we discuss: 1) the role of risk assessment in supporting societal decisions on managing risk; 2) the use of "plausible conservatism" in selecting default options and alternatives to default options; 3) the use of an iterative approach in which specific science displaces default options; 4) the need for risk characterizations to be matched to their intended uses, and why a single quantitative estimate of risk may not be adequate; 5) why the process for conducting science-based risk assessments should be integrated and comprehensive; and 6) how risk assessments can serve an important role in guiding research to improve future risk assessments.
The Role of Risk Assessment in Supporting
Societal Decisions on Managing Risk
The development of risk assessments is one part of a larger process by which societal decisions and actions concerning risks are made. Risk assessments are that phase of the overall process in which all of the available information concerning exposure to the agent(s), the agent's(s') ability to cause adverse responses, and exposure-dose-response relationships, is synthesized into a risk
characterization whose degree of comprehensiveness is matched to the intended use of the risk characterization. When specific data are not available, default options based on general scientific knowledge and risk assessment policy are used. The risk characterization product of the risk assessment is then used as input along with a diverse array of other information to make a wide range of risk-based decisions, as for example, whether to limit exposure to the agent(s) (and if so, to what extent). These risk-based decisions may on occasion involve a comparison of risks between agents causing similar adverse responses or, more broadly, disease. In other cases, the risk characterization may be used as input to decisions as to how to allocate economic or other societal resources. Clearly, risk characterizations must be as accurate as possible because of the potential importance of the decisions concerning health (and disease) and allocation of scarce societal resources.
This appendix proposes that risk characterizations should be developed by a well-documented process that makes full use of the available scientific data. When specific data are not available, the process should use default options and other assumptions that are clearly identified. The end-product risk characterization should be reported with a degree of comprehensiveness matched to its intended use and in a form that can be readily understood by decision makers and interested members of the public. One intent of the process is to avoid the introduction of unidentified bias that would either under-estimate or over-estimate the risk being characterized. The approach we advocate emphasizes scientific plausibility with regard to the use of alternative models and appropriate disclosure of uncertainties.
The approach we are advocating contrasts sharply with the approach advocated by Dr. Finkel, which introduces into the risk assessment process an additional standard: whether the alternative based on the scientific information yields a plausible, conservative estimate of risk. A default option would be displaced only if it is found to be no longer plausible, or if a plausible alternative gives a higher estimate of risk. Thus, judgments on the extent of conservatism would largely determine the result from the risk assessment process. It is our opinion that value judgments as to the degree of conservatism should not have such a large influence on the output of the risk assessment process. We believe that EPA should make these value judgments consistently according to established guidelines where such judgments are necessary (e.g., choice of default options), and should disclose the use of such judgments fully to risk managers and to the public.
The value judgments are most appropriately dealt with as part of the risk management or risk decision-making phase of the overall process. In particular, risk managers should not be restricted by value judgments made during risk assessment. Risk managers can and should override conservative default value judgments in the risk assessment process whenever they believe it is appropriate public policy to do so. Such departures should be clearly identified as policy and not as science. Risk managers must assume full responsibility for making such
overrides and for explaining their reasoning to the interested and affected members of the public.
Use of "Plausible Conservatism" in Selecting Default Options and
Alternatives to Default Options
It has been noted that inference guidelines, or default options as they are typically called in this report, are generic guidelines used when the necessary scientific information is not available. These guidelines are based on general scientific knowledge and applied to assure consistency in the development of multiple risk assessments. It is our understanding that EPA has selected default options that are scientifically plausible and conservative in the sense that they are intended to avoid underestimating health risks. Hence, these generic guidelines generally follow the principle of "plausible conservatism" as we believe it is described in Appendix N-1. We do not object to this approach for selecting the default options.
We do object to the use of "plausible conservatism" as a criterion in deciding when specific science can be used to replace a default option. The use of "plausible conservatism" as the test for displacing default options places an excessively high hurdle for the new science. The use of "plausible conservatism'' will therefore discourage the conduct of research to generate the scientific information that might displace the use of the default option. The result will be to freeze risk characterizations at the level determined by the conservative default options.
As specific science is developed and used to replace default options, the result will typically be a reduction both in the estimates of risk and the extent of uncertainty in the risk estimates. The replacement of default options with specific science was illustrated in Chapter 6 using formaldehyde as an example. In this case the initial risk estimate, which was based on a default option for relating exposure to response, i.e., the cancer risk, had a plausible upper bound estimate of 0.016 (1.6 × 10-2) at 1 ppm. The lower limit may be zero. Thus, there was a wide range of uncertainty, from 0 to 0.016. In successive iterations as new scientific information was incorporated on delivered dose to target tissue using data on DNA-protein cross-links, first from rats and then from monkeys, the upper bound on risk at 1 ppm was reduced to 2.8 × 10-3 and then to 3.3 × 10-4. For neither of these iterations can a lower bound estimate of zero be excluded. Thus, at the last iteration the range of uncertainty has been reduced to 0 to 3.3 × 10-4. This is a substantial reduction from the 0 to 1.6 × 10-2 calculated based on the default options.
In this example the departure from the default options was far more plausible than the original default options. The DNA-protein cross-links provide a direct measurement of a biomarker for the extent to which the formaldehyde is penetrating into tissues where cancers might be induced.
In many other situations, the difference in plausibility between the default and the alternative using specific scientific information may be less apparent. It is our judgment that weighing the plausibility of alternatives is a highly judgmental evaluation that must be carried out by scientists. We believe it would be a mistake to try to define a sharp threshold for plausibility. Such a sharp threshold will stifle research and impede communication about uncertainties. When an alternative approach is judged plausible, but the default option also plausible, it will be appropriate for the risk estimates from both approaches to be conveyed to the risk manager, as CAPRA has recommended.
Better criteria for departures from defaults are needed. However, we believe that scientific judgment will remain at the heart of the process for determining that a default option should be displaced, either for a specific substance, or for a class of substances. Chapter 6 provides several examples of instances in which departures from defaults have been accepted, or considered and rejected as not yet adequately supported by scientific information, based on outside scientific peer review through the EPA Science Advisory Board. In our opinion EPA's process for making such judgments works reasonably wellalthough there is clearly room for improvement. More research directed at the important uncertainties should permit more departures from defaults, based upon adequate support from the scientific information obtained through the research.
We view the extent of conservatism in risk assessment guidelines as a policy issue to be determined by EPA, most appropriately through notice and comment rulemaking in the same manner as when EPA risk assessment guidelines were adopted in 1986. The proposal in Appendix N-1 does not give precise guidance for establishing default options or for departing from these defaults. Scientists may disagree as to whether a model is plausible or not plausible, and lack of plausibility will be very difficult to establish outside the range of observed data. The usual choice will be between simple models whose structure is assumed, (e.g., low dose linearity) vs. more complex models based on knowledge of biological and pathobiological processes. Both alternatives may be judged plausible. However, the biologically based models may be more valuable because they incorporate more information and provide a better basis for discriminating on the extent of the risk posed by different chemicals at relevant levels of human exposure.
We are also concerned that recommendations from CAPRA on policy issues could be inappropriate and subject to misinterpretation. Therefore, we believe it is inappropriate for the National Research Council to recommend default options to EPA. NRC recommendations might be perceived as being based on solely on science, but such would not be the case; such recommendations would reflect value judgments that scientists are no more qualified to make than other citizens. However, it is appropriate for NRC to point out where default options are needed, so that these policy questions can be addressed by the regulatory agency. For example, should the same cancer potency be used for all chemicals in a class
(discussed at the end of Chapter 6)? Should the same cancer potency be applied to all people, or should sensitive subgroups be treated separately (discussed in Chapter 10)? It is our position that judgments on what are the appropriate defaults should be made by the regulatory agency, and not by the members of an NRC committee.
There are broader questions of risk assessment and risk management policy that CAPRA has declined to address. There is much dispute and inconsistency on the appropriate basis for regulating toxic chemicals, especially carcinogens. Some within the scientific community believe that Congress and the regulatory agencies have gone much too far in regulating some chemicals (e.g., synthetic pesticide residues in processed food) and not far enough in regulating other chemicals (indoor radon and other indoor air toxicants). We believe that such disputes and inconsistencies should be addressed using risk assessment for communication, to inform those with decision responsibility what science can and cannot say about the magnitude of the risks posed by chemicals to health and the environment. Scientists should not attempt to resolve risk management disputes by influencing the choice of default options or the criteria for departure from default options.
The Use of an Iterative Approach, in which Specific Science Displaces Default Options and Provides a Means to Improve Risk Assessments and Reduce Uncertainty in Risks
The CAPRA report advocates the conduct of iterative risk assessments matched to decision-making needs. This approach recognizes that EPA must deal with at least 189 hazardous air pollutants, many with limited data and, perhaps, posing low risks. EPA needs an approach for carrying out iterative risk assessment on hazardous air pollutants, and Chapter 12 builds upon EPA's planned methodology to describe such an approach. As a part of this approach, EPA must develop a system for prioritizing these chemicals so that the limited funds available may be used most effectively to protect human health. Because of differences in the available data and the differences in the magnitude of the risk posed by different chemicals, EPA should not deal with each chemical the same way. The highly quantitative formal techniques described in CAPRA Chapters 9, 10, 11 are not intended for every chemical, but only for supporting the most important and difficult regulatory decisions, for which advanced analytical concepts and procedures may be needed. The sophistication and complexity of these methods add to the difficulty of communicating to regulatory decision makers and to the public. EPA needs a risk assessment process that can deal effectively, cheaply, and quickly with most of the chemicals, while permitting more sophisticated and data-intensive risk assessment in situations where the additional time, expense, analytical sophistication, and risk communication difficulties are warranted by the importance of the regulatory decisions.
Risk Characterizations Must Be as Clear and Comprehensive as Practical, Given Their Intended Uses; and A Single Quantitative Estimate of Risk May Not Be Adequate
The risk assessment process and the resulting risk characterization should be matched to the intended use of the risk characterization. (Recall the discussion in the preceding section of the need for an iterative approach.) Obviously, the degree of comprehensiveness that can be achieved for a given risk characterization will be dependent on the extent of the scientific information available.
For the chemicals with the least amount of data the risk characterization may be a qualitative, narrative summary of the limited available information. For chemicals with more extensive data, such as several bioassays, the risk characterization may include a plausible upper bound risk estimate, using the 95% upper confidence limit computed from the bioassay data set that yields the highest risk estimate (e.g., most sensitive strain, sex, species, and tumor end point) and a conservative and relatively crude exposure estimate.
For the most extensive data sets, it may be possible to provide multiple risk calculations corresponding to alternative models and data sets corresponding to individuals and populations. These data may be organized in the form of one or more probability distributions, from which a probability distribution on risk is computed. The probability distribution on risk may be summarized by using expected values or other summary statistics computed through Monte Carlo analysis or other probabilistic analysis techniques. Such central tendency estimates will be helpful supplements to upper and lower bound calculations (more generally, statistical confidence limits) to assist decision makers and the public in understanding the implications of the probability distributions. Such analysis based on the most extensive available data for cancer potency and exposure has not, to our knowledge, been carried out in support of a major regulatory decision, but the procedures involved are illustrated in Appendices (Texaco and ENSR articles) and in the scientific literature (Wallsten and Whitfield, 1989; Howard et al., 1972).
The proposal in Appendix N-1 for plausible conservatism seems to assume that the output of risk assessment is a single risk number that can be used for regulatory decision making. We oppose this aspect of his proposal, especially for the upper tiers of risk assessment. The goal for risk assessment should be to inform decision makers and the public, not to give them a number.1 To the
1In Appendix N-1, Dr. Finkel uses an example of when to leave for the airport to illustrate his advocacy of conservative estimates, and we use the same example to make the point that single-number estimates may be inadequate as a summary of information for purposes of decision making. The decision on when to leave for the airport depends on the information about how long it will take to get to the airport, an uncertain quantity. It is our judgment that most decision makers would not wish to have this uncertainty summarized as a single estimated travel time, as he has asserted. Rather, we believe that decision makers prefer to have a description of the possibilities and their likelihood. For example, an estimate of the travel time under normal conditions might be supplemented by a description of possible delays and the probabilities that such delays might occur. Such an analysis might be quite simple, with only a few sources of delay considered, or quite complex, requiring a computer to calculate the probability distribution on the time from departure to boarding the airplane. In presenting the analysis, an assessor might highlight the most important uncertainties (e.g., "The normal driving time is approximately 30 minutes, with a probability of 20% that traffic delays might add between 10 and 30 minutes. The probability that travel time by taxi to the airport would exceed one hour is judged to be less than 5%.").
extent that risk assessment provides only one number, based on conservative assumptions, then the group that determines which conservative assumptions shall be used will determine regulatory policy. Thus, the discretion of the risk manager will be preempted by the risk assessment process.
The EPA Science Advisory Board Report on Dioxins (EPA, 1989) stressed the importance of replacing linear extrapolation with a biologically based model, and that the default of linearity might cause risk to be overestimated or underestimated. The SAB encouraged EPA to consider revisions in the regulatory standard based on policy and on the scientific uncertainties. SAB did not support changing the single number risk estimate on the basis of the scientific information then available.
It can be argued theoretically that for decision making, the best single number will be the expected valuethe average over the probability distribution. However, we believe that the distribution is better than the any single number. If an average value is to be used, misinterpretation should be minimized, and for more than a decade EPA's risk estimates have generally been upper bounds. (Only a few risk estimates based on human epidemiology have represented conceptual departuresfor example, lung cancer from indoor radon, where the health risk estimate comes from extrapolation of observed lung cancer incidence in uranium miners.)
In Appendix N-1, the example of a substance that may pose an unacceptably high risk of X, or zero, depending on which of two incompatible biologic theories is true. Such a situation is clearly one in which risk managers will wish to learn about this critically important uncertainty as to which theory is correct. Within the risk management context if not within risk assessment, it may be useful to characterize the judgment of knowledgeable scientists in terms of a subjective probability. Suppose there is a consensus among scientists that the probability is p that the risk is at or near zero. In our judgment, the decision maker will wish to understand this characterization of the risk: a probability p that the risk is at or near zero and a probability 1-p that the risk is at the high level X. We believe it inappropriate to summarize this situation by presenting only the expected value of (1-p)X as the estimate of risk for the decision maker. The probability distribution should be used for the risk characterization, not one
risk estimate. Decision makers and the public should have little difficulty understanding this simple characterization.
The Process for Conducting Science-Based Risk Assessments Should Be
Integrated and Comprehensive
The process being advocated for the conduct of science-based risk assessments builds on the general principles outlined in the 1983 NRC Committee Report (the Red Book). We reaffirm these general principles and build on them in proposing a process for the conduct of risk assessments. The general principles we believe to be appropriate include:
Risk Assessments Can Serve an Important Role in Guiding Research
to Improve Future Risk Assessments
It is our opinion that risk assessments can have a major role in guiding research to improve the scientific basis for future risk assessments. This will require a new attitude recognizing that the risk assessment process should yield not only a risk characterization but also identify the unanswered questions which, if addressed with research, could have the potential for reducing the uncertainty in the estimates of risk as schematically related in Figure N-1. This process of identifying research needs (opportunities) may be informal or formalized as in the use of sensitivity analyses. Having identified the major sources of uncertainty, the question may be asked as to whether the issue can be addressed with current research technologies and, if so, the potential cost and time required to carry out the research. These costs and time estimates can then be balanced against the potential value of the information in making decisions on proceeding with the targeted research effort.
A recent OTA report, Researching Health Risks (OTA, 1993), addressed the issue of conducting targeted research of this kind both as related to specific chemicals but also as a means of improving risk assessment methodology. Obviously, the two go hand-in-hand with research on specific chemicals (in which they serve as useful probes) addressing generic toxicological/risk assessment issues while also providing highly relevant information applicable to the specific chemical.
The most important risk management decisions will involve large potential impacts on public health and large economic consequences from control actions. Such decisions should involve a careful review of the underlying science. Risk managers may wish to consider whether to act with present information, which may involve large uncertainties in the public health consequences, or to delay the decision for a period of time which research is carried out to reduce these uncertainties and therefore provides a better basis for decision. It is our belief that Congress could do much more to encourage EPA, other federal agencies such as NIEHS, and private sector organizations to plan and carry out research to reduce important uncertainties on the health consequences of toxic air contaminants. Such research might take a decade or more to complete, but research started now might provide significant new information supporting departures from defaults that could save billions of dollars in control costs while providing even better protection of public health.
Scientific knowledge of the mechanism by which toxic substances cause cancer and other chronic health impacts is evolving rapidly. However, much of this research is aimed at understanding and treating the health impacts, rather than understanding the relationship of the health impacts to the relatively low levels of exposure to toxic substances in the ambient air. The most important uncertainties are those for which the value of information is high, because reso-
lution of the uncertainties are likely to change decisions, leading to substantial benefits in improved public health and reduced control costs (OTA, 1993). More targeted research designed to avoid costly regulations based on conservative default options in risk assessment should pay very large economic dividends, while at the same time allowing better management of the substances that do present substantial risks to public health.
EPA (U.S. Environmental Protection Agency). 1986. Guidelines for carcinogen risk assessment. Fed. Regist. 51:33992-34003.
EPA (U.S. Environmental Protection Agency). 1989. Letter report to EPA administrator, William Reilly, from the Science Advisory Board, Nov. 28. SAB-EC-90-003. U.S. Environmental Protection Agency, Washington, D.C.
Howard, R.A., J.E. Matheson, and D.W. North. 1972. The decision to seed hurricanes. Science 176:1191-1202.
NRC (National Research Council). 1983. Risk Assessment in the Federal Government: Managing the Process. Washington, D.C.: National Academy Press.
OTA (U.S. Office of Technology Assessment). 1993. Researching Health Risks. U.S. Office of Technology Assessment, Washington, D.C.
Whitfield, R.G., and T.S. Wallsten. 1989. A risk assessment for selected lead-induced health effects: An example of a general methodology. Risk Anal. 9:197-207.