In 1983, the National Research Council (NRC) issued the seminal report Risk Assessment in the Federal Government: Managing the Process, which provided a framework for the conduct of risk assessment. It defined risk assessment as “the qualitative or quantitative characterization of the potential health effects of particular substances on individuals or populations” (NRC 1983, p. 38) and indicated four components of risk assessment: hazard identification, dose-response assessment, exposure assessment, and risk characterization. Over the last 2 decades, the practice of risk assessment has evolved in the federal government. Some agencies, such as the Environmental Protection Agency (EPA) and the National Aeronautics and Space Administration (NASA), have formalized guidelines for various types of risk assessment (EPA 1996, 1998, 2005; NASA 2002); others have no formal process. Thus, the practice of risk assessment in the federal government varies considerably. To improve the overall practice of risk assessment, the Office of Management and Budget (OMB) issued the Proposed Risk Assessment Bulletin, which sets forth specific standards for risk assessments used and disseminated by federal agencies, and asked NRC to review it. In response to OMB’s request, NRC convened the Committee to Review the OMB Risk Assessment Bulletin, which prepared this report.
THE OFFICE OF MANAGEMENT AND BUDGET
OMB is responsible for ensuring that information, analyses, and regulatory actions issued by federal agencies meet high quality standards.
Executive Order 12866 (58 Fed. Reg. 51735 ) directs OMB to provide guidance to federal agencies on regulatory development and emphasizes the need for each agency to “base its decisions on the best reasonably obtainable scientific, technical, economic, and other information.” The Paperwork Reduction Act (44 U.S.C. § 3504 ) requires OMB to “develop and oversee the implementation of policies, principles, standards, and guidelines to...apply to Federal agency dissemination of public information.” The Information Quality Act (Public Law 106-554 § 515(a) ), a supplement to the Paperwork Reduction Act, directs OMB to develop guidelines that “provide policy and procedural guidance to Federal agencies for ensuring and maximizing the quality, objectivity, utility and integrity of information” released by federal agencies.
In 2002, in response to the many directives, OMB finalized Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies, which defines quality, objectivity, utility, and integrity and requires federal agencies to issue their own information-quality guidelines (67 Fed. Reg. 8452 ). In 2003, OMB issued Circular A-4, Regulatory Analysis, which defines the key elements of a regulatory analysis and provides specific guidance on conducting cost-benefit and cost-effectiveness analyses (OMB 2003). In 2005, OMB published Final Information Quality Bulletin for Peer Review, which requires peer review of important scientific information by qualified experts before release by federal agencies (70 Fed. Reg. 2664 ). In its continuing effort to improve the quality of information and analyses disseminated by federal agencies, OMB issued the draft bulletin (OMB 2006) providing guidance for the conduct of risk assessment.
THE BULLETIN AND SUPPLEMENTARY INFORMATION
OMB’s draft bulletin consists of a preamble (22 pages) containing primarily “supplementary information” followed by the actual bulletin (3 1/2 pages) (see Appendix B). The bulletin provides key definitions, goals for risk assessment, general risk assessment and reporting standards, and special standards for influential risk assessment. The bulletin also presents information on applicability, updates, certification, deferrals and waivers, responsibilities of executive offices, effective date, and judicial review. The supplementary information includes background on
risk assessment, OMB’s legal authority to issue the bulletin, and details on the requirements listed in the bulletin.
It is important to understand the relationship between the preamble and the bulletin. When a federal agency publishes a rule, it also provides a statement of the reason and basis for the rule. Those who are subject to the regulation are bound by the words in the regulation itself. If the regulation is challenged and the dispute reaches the courts, the courts look first at the text of the regulation; if the text is unclear, the courts will generally refer to the preamble to determine what the promulgating agency meant in the regulation. As indicated in the draft OMB bulletin, judicial review and hence judicial interpretation are explicitly precluded, and challenges to agency action under the bulletin would go to OMB, which would invariably be guided by the statements in the preamble as to scope and content. The committee therefore reviewed both the bulletin and the supplementary information and discusses both in its report. Generic references to the bulletin here typically refer to both the bulletin and the supplementary information.
COMMITTEE’S TASK AND APPROACH
The committee members were selected for their expertise in risk assessment, clinical medicine, toxicology, industrial hygiene, statistics, engineering, epidemiology, ecology, decision and uncertainty analysis, and cost-benefit analysis. Specifically, the committee was asked to conduct a scientific review of OMB’s draft risk assessment bulletin and to complete the following tasks: (1) determine whether the application of the draft bulletin will meet OMB’s stated objective to “enhance the technical quality and objectivity of risk assessments prepared by federal agencies”; (2) comment, in general terms, on how the bulletin will affect the practice of risk assessment in the federal government; (3) identify critical elements that might be missing from the bulletin; (4) determine whether OMB appropriately incorporated recommendations from previous reports of NRC and other organizations into the draft bulletin; and (5) assess whether there are scientific or technical circumstances that might limit applicability of the bulletin (see Appendix C for a verbatim statement of task). The study sponsors were EPA, NASA, the U.S. Department of Agriculture, the Department of Defense, the Department of Energy, the Department of Health and Human Services, and the Department of Labor.
To accomplish its task, the committee held three meetings from May to August 2006. The first included a public meeting during which the committee heard presentations from the sponsors; invited speakers from private industry, universities, trade associations, and environmental groups also provided a diverse perspective on risk assessment and the draft bulletin (see Appendix D for an agenda of the public meeting). At that meeting, OMB asked the committee to comment in its report on specific aspects of the proposed bulletin, including the definition of risk assessment, the goals, the proposed standards, possible omissions or errors, and examples to serve as models for risk assessment. The committee reviewed numerous documents cited in the supplementary information and reviewed public comment submitted to OMB on the bulletin. The committee also requested information from the federal agencies on their risk assessment practices and their view of the impact of the bulletin on current practices. Appendix E presents the questions submitted to the agencies and their responses.
Much of the language used and the examples provided in the bulletin and the supplementary information are related to human health risk assessment and not engineering, ecologic, or behavioral risk assessments. Specifically, little is said about the failure of engineered systems, the degradation of ecosystems, and the risk of malicious human behavior. The committee recognizes that each class of risks has generated risk assessment methods to address its unique set of issues (see Box 1-1). However, the committee was tasked with reviewing the bulletin and not providing a comprehensive treatment of risk assessment; therefore, the committee comments focus primarily on human health risk assessment.
ORGANIZATION OF THE REPORT
The committee’s report is organized into seven chapters. Chapter 2 evaluates the consistency of the bulletin with previous NRC and other expert reports. Chapter 3 addresses issues surrounding the definition of risk assessment provided in the bulletin and the goals listed in it. The risk assessment standards articulated in the bulletin are reviewed in Chapter 4. Chapter 5 highlights key omissions from the bulletin. Chapter 6 discusses the impact of the bulletin on risk assessment practices in the federal government. The committee’s conclusions and recommendations are summarized in Chapter 7.
Types of Risk Assessment
Historically, risk assessment has been dominated by two parallel methodologic developments: (1) public-health risk assessment, with a major focus on the health effects of chronic exposures to chemicals, contaminants, and pollutants in the water we drink, the air we breathe, and the food we eat, and (2) engineered-systems risk assessment, with a primary focus on immediate and delayed effects due to the failure of systems, such as aerospace vehicles, chemical process plants, and nuclear power plants. More recently, there has been heightened interest in other risks, including ecologic risks, such as the degradation of ecologic systems due to nonnative invasive species, global warming, and genetically modified organisms; risks related to severe natural phenomena, such as hurricanes, earthquakes, fires, and floods; and risks associated with malicious human acts, such as terrorism. Each domain raises its own intellectual challenges, sometimes involving extension of public-health and engineered-systems methods, at other times requiring dedicated methods. Differences and similarities of risk assessment for public health and engineered systems provide insight into the issues faced in the development of scientifically sound methods.
Risk assessment, in both cases, involves a search for “causal links” or “causal chains” verified by “objective” analytic and experimental techniques, such as quantifying the behavior of various elements (for example, pumps, valves, operators, maintenance supervisors, and physicians) in terms of failure-rate data or exposure and dose-response data. Risk assessments for engineered systems focus on the questions, What can go wrong? How likely is it to happen? (Kaplan and Garrick 1981). The analysis is typically organized around fault and event trees, delineating the impacts of initiating events and failure rates. Public-health risk assessment focuses on the question, What are the consequences in terms of exposure assessment and dose-response assessment, using quantitative estimates of behaviors like ingestion and metabolism. Each field has generated its own analytic methods and experimental protocols, with the common goal of quantifying overall system performance in terms of valued consequences.
EPA (U.S. Environmental Protection Agency). 1996. Guidelines for Reproductive Toxicity Risk Assessment. EPA/630/R-96/009. Risk Assessment Forum, U.S. Environmental Protection Agency, Washington, DC [online]. Available: http://www.epa.gov/ncea/raf/pdfs/repro51.pdf [accessed July 27, 2006].
EPA (U.S. Environmental Protection Agency). 1998. Guidelines for Ecological Risk Assessment. EPA/630/R-95/002F. Risk Assessment Forum, U.S. Environmental Protection Agency, Washington, DC [online]. Available: http://cfpub.epa.gov/ncea/cfm/recordisplay.cfm?deid=12460 [accessed July 27, 2006].
EPA (U.S. Environmental Protection Agency). 2005. Guidelines for Carcinogen Risk Assessment. EPA/630/P-03/001B. Risk Assessment Forum, U.S. Environmental Protection Agency, Washington, DC [online]. Available: http://www.epa.gov/iris/cancer032505.pdf [accessed July 27, 2006].
Kaplan, S., and B.J. Garrick. 1981. On the quantitative definition of risk. Risk Anal. 1(1):11-27.
NASA (National Aeronautics and Space Administration). 2002. Probabilistic Risk Assessment Procedures Guide for NASA Managers and Practitioners, Version 1.1. Prepared for Office of Safety and Mission Assurance NASA Headquarters, Washington, DC [online]. Available: http://www.hq.nasa.gov/office/codeq/doctree/praguide.pdf [accessed Oct. 11, 2006].
NRC (National Research Council). 1983. Risk Assessment in the Federal Government: Managing the Process. Washington, DC: National Academy Press.
OMB (U.S. Office of Management and Budget). 2003. Regulatory Analysis. Circular A-4 to the Heads of Executive Agencies and Establishments, September 17, 2003 [online]. Available: http://www.whitehouse.gov/omb/circulars/a004/a-4.pdf [accessed Oct. 12, 2006].
OMB (U.S. Office of Management and Budget). 2006. Proposed Risk Assessment Bulletin. Released January 9, 2006. Washington, DC: Office of Management and Budget, Executive Office of the President [online]. Available: http://www.whitehouse.gov/omb/inforeg/proposed_risk_ assessment_bulletin_010906.pdf [accessed Oct. 11, 2006].