A RISK-CHARACTERIZATION FRAMEWORK FOR DECISION-MAKING AT THE FOOD AND DRUG ADMINISTRATION
NATIONAL RESEARCH COUNCIL
OF THE NATIONAL ACADEMIES
THE NATIONAL ACADEMIES PRESS
Washington, D.C.
www.nap.edu
THE NATIONAL ACADEMIES PRESS
500 Fifth Street, NW
Washington, DC 20001
NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance.
This project was supported by Contract HHSP23320042509XI/HHSP233200800001T between the National Academy of Sciences and the U.S. Department of Health and Human Services. Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the authors and do not necessarily reflect the view of the organizations or agencies that provided support for this project.
International Standard Book Number-13: 978-0-309-21280-9
International Standard Book Number-10: 0-309-21280-4
Additional copies of this report are available from
The National Academies Press
500 Fifth Street, NW Box 285 Washington, DC 20055 800-624-6242 202-334-3313 (in the Washington metropolitan area) http://www.nap.edu
Copyright 2011 by the National Academy of Sciences. All rights reserved.
Printed in the United States of America.
THE NATIONAL ACADEMIES
Advisers to the Nation on Science, Engineering, and Medicine
The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Ralph J. Cicerone is president of the National Academy of Sciences.
The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Charles M. Vest is president of the National Academy of Engineering.
The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Harvey V. Fineberg is president of the Institute of Medicine.
The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Ralph J. Cicerone and Dr. Charles M. Vest are chair and vice chair, respectively, of the National Research Council.
COMMITTEE ON RANKING FDA PRODUCT CATEGORIES BASED ON HEALTH CONSEQUENCES, PHASE II
Members
ROBERT S. LAWRENCE (Chair),
Johns Hopkins Bloomberg School of Public Health, Baltimore, MD
JEFFREY B. BENDER,
University of Minnesota, St. Paul
FRANCISCO DIEZ-GONZALEZ,
University of Minnesota, St. Paul
KATHRYN M. EDWARDS,
Vanderbilt University, Nashville, TN
SUSAN S. ELLENBERG,
University of Pennsylvania, Philadelphia
PAUL S. FISCHBECK,
Carnegie Mellon University, Pittsburgh, PA
KAREN E. JENNI,
Insight Decisions LLC, Denver, CO
HELEN H. JENSEN,
Iowa State University, Ames
L. ROBIN KELLER,
University of California, Irvine
JAMES D. MCKEAN,
Iowa State University, Ames
DAVID O. MELTZER,
University of Chicago, Chicago, IL
SANFORD A. MILLER,
University of Maryland, College Park
RICHARD PLATT,
Harvard Pilgrim Health Care Institute and Harvard Medical School, Boston, MA
JOHN T. WATSON,
University of California, San Diego
Staff
ELLEN MANTUS, Project Director
NORMAN GROSSBLATT, Senior Editor
HEIDI MURRAY-SMITH, Program Officer
MIRSADA KARALIC-LONCAREVIC, Manager,
Technical Information Center
RADIAH ROSE, Manager,
Editorial Projects
KERI SCHAFFER, Research Associate
PANOLA GOLSON, Program Associate
Sponsor
U.S. DEPARTMENT OF HEALTH AND HUMAN SERVICES
BOARD ON ENVIRONMENTAL STUDIES AND TOXICOLOGY1
Members
ROGENE F. HENDERSON (Chair),
Lovelace Respiratory Research Institute, Albuquerque, NM
PRAVEEN AMAR,
Northeast States for Coordinated Air Use Management, Boston, MA
TINA BAHADORI,
American Chemistry Council, Washington, DC
MICHAEL J. BRADLEY,
M.J. Bradley & Associates, Concord, MA
DALLAS BURTRAW,
Resources for the Future, Washington, DC
JAMES S. BUS,
Dow Chemical Company, Midland, MI
JONATHAN Z. CANNON,
University of Virginia, Charlottesville
GAIL CHARNLEY,
HealthRisk Strategies, Washington, DC
FRANK W. DAVIS,
University of California, Santa Barbara
RICHARD A. DENISON,
Environmental Defense Fund, Washington, DC
H. CHRISTOPHER FREY,
North Carolina State University, Raleigh
J. PAUL GILMAN,
Covanta Energy Corporation, Fairfield, NJ
RICHARD M. GOLD,
Holland & Knight, LLP, Washington, DC
LYNN R. GOLDMAN,
George Washington University, Washington, DC
LINDA E. GREER,
Natural Resources Defense Council, Washington, DC
WILLIAM E. HALPERIN,
University of Medicine and Dentistry of New Jersey, Newark
PHILIP K. HOPKE,
Clarkson University, Potsdam, NY
HOWARD HU,
University of Michigan, Ann Arbor
ROGER E. KASPERSON,
Clark University, Worcester, MA
THOMAS E. MCKONE,
University of California, Berkeley
TERRY L. MEDLEY,
E.I. du Pont de Nemours & Company, Wilmington, DE
JANA MILFORD,
University of Colorado at Boulder, Boulder
FRANK O’DONNELL,
Clean Air Watch, Washington, DC
RICHARD L. POIROT,
Vermont Department of Environmental Conservation, Waterbury
DANNY D. REIBLE,
University of Texas, Austin
ROBERT F. SAWYER,
University of California, Berkeley
KATHRYN G. SESSIONS,
Health and Environmental Funders Network, Bethesda, MD
JOYCE S. TSUJI,
Exponent Environmental Group, Bellevue, WA
MARK J. UTELL,
University of Rochester Medical Center, Rochester, NY
Senior Staff
JAMES J. REISA, Director
DAVID J. POLICANSKY, Scholar
RAYMOND A. WASSEL, Senior Program Officer for Environmental Studies
SUSAN N.J. MARTEL, Senior Program Officer for Toxicology
ELLEN K. MANTUS, Senior Program Officer for Risk Analysis
EILEEN N. ABT, Senior Program Officer
RUTH E. CROSSGROVE, Senior Editor
MIRSADA KARALIC-LONCAREVIC, Manager,
Technical Information Center
RADIAH ROSE, Manager,
Editorial Projects
OTHER REPORTS OF THE BOARD ON ENVIRONMENTAL STUDIES AND TOXICOLOGY
Review of the Environmental Protection Agency’s Draft IRIS Assessment of Formaldehyde (2011)
The Use of Title 42 Authority at the U.S. Environmental Protection Agency (2010)
Review of the Environmental Protection Agency’s Draft IRIS Assessment of Tetrachloroethylene (2010)
Hidden Costs of Energy: Unpriced Consequences of Energy Production and Use (2009)
Contaminated Water Supplies at Camp Lejeune—Assessing Potential Health Effects (2009)
Review of the Federal Strategy for Nanotechnology-Related Environmental, Health, and Safety Research (2009)
Science and Decisions: Advancing Risk Assessment (2009)
Phthalates and Cumulative Risk Assessment: The Tasks Ahead (2008)
Estimating Mortality Risk Reduction and Economic Benefits from Controlling Ozone Air Pollution (2008)
Respiratory Diseases Research at NIOSH (2008)
Evaluating Research Efficiency in the U.S. Environmental Protection Agency (2008)
Hydrology, Ecology, and Fishes of the Klamath River Basin (2008)
Applications of Toxicogenomic Technologies to Predictive Toxicology and Risk Assessment (2007)
Models in Environmental Regulatory Decision Making (2007)
Toxicity Testing in the Twenty-first Century: A Vision and a Strategy (2007)
Sediment Dredging at Superfund Megasites: Assessing the Effectiveness (2007)
Environmental Impacts of Wind-Energy Projects (2007)
Scientific Review of the Proposed Risk Assessment Bulletin from the Office of Management and Budget (2007)
Assessing the Human Health Risks of Trichloroethylene: Key Scientific Issues (2006)
New Source Review for Stationary Sources of Air Pollution (2006)
Human Biomonitoring for Environmental Chemicals (2006)
Health Risks from Dioxin and Related Compounds: Evaluation of the EPA Reassessment (2006)
Fluoride in Drinking Water: A Scientific Review of EPA’s Standards (2006)
State and Federal Standards for Mobile-Source Emissions (2006)
Superfund and Mining Megasites—Lessons from the Coeur d’Alene River Basin (2005)
Health Implications of Perchlorate Ingestion (2005)
Air Quality Management in the United States (2004)
Endangered and Threatened Species of the Platte River (2004)
Atlantic Salmon in Maine (2004)
Endangered and Threatened Fishes in the Klamath River Basin (2004)
Cumulative Environmental Effects of Alaska North Slope Oil and Gas Development (2003)
Estimating the Public Health Benefits of Proposed Air Pollution Regulations (2002)
Biosolids Applied to Land: Advancing Standards and Practices (2002)
The Airliner Cabin Environment and Health of Passengers and Crew (2002)
Arsenic in Drinking Water: 2001 Update (2001)
Evaluating Vehicle Emissions Inspection and Maintenance Programs (2001)
Compensating for Wetland Losses Under the Clean Water Act (2001)
A Risk-Management Strategy for PCB-Contaminated Sediments (2001)
Acute Exposure Guideline Levels for Selected Airborne Chemicals (nine volumes, 2000-2010)
Toxicological Effects of Methylmercury (2000)
Strengthening Science at the U.S. Environmental Protection Agency (2000)
Scientific Frontiers in Developmental Toxicology and Risk Assessment (2000)
Ecological Indicators for the Nation (2000)
Waste Incineration and Public Health (2000)
Hormonally Active Agents in the Environment (1999)
Research Priorities for Airborne Particulate Matter (four volumes, 1998-2004)
The National Research Council’s Committee on Toxicology: The First 50 Years (1997)
Carcinogens and Anticarcinogens in the Human Diet (1996)
Upstream: Salmon and Society in the Pacific Northwest (1996)
Science and the Endangered Species Act (1995)
Wetlands: Characteristics and Boundaries (1995)
Biologic Markers (five volumes, 1989-1995)
Science and Judgment in Risk Assessment (1994)
Pesticides in the Diets of Infants and Children (1993)
Dolphins and the Tuna Industry (1992)
Science and the National Parks (1992)
Human Exposure Assessment for Airborne Pollutants (1991)
Rethinking the Ozone Problem in Urban and Regional Air Pollution (1991)
Decline of the Sea Turtles (1990)
Copies of these reports may be ordered from the National Academies Press
(800) 624-6242 or (202) 334-3313
Preface
The U.S. Food and Drug Administration (FDA) has regulatory responsibility for a vast number of products—foods, drugs, cosmetics, biologics, veterinary products, medical devices, tobacco, and products that emit radiation—and often faces difficult management decisions as to how to ensure the safety of the products that it regulates. FDA recognized that collecting and evaluating information on the risks posed by the regulated products in a systematic manner would aid in its decision-making process. Consequently, FDA and the Department of Health and Human Services (DHHS) asked the National Research Council (NRC) to develop a conceptual model that could evaluate products or product categories that FDA regulates and provide information on the potential health consequences associated with them. As a result of that request, NRC formed the Committee on Ranking FDA Product Categories Based on Health Consequences.
The project was to be conducted in two phases. For the first phase, the committee was to produce a brief letter report that described a conceptual model that could be used to rank product categories. For the second phase, the committee was to develop, refine, and apply the model to conduct a risk ranking of FDA product categories. The letter report was completed and provided to FDA in February 2009 and is provided in Appendix A. On receipt of the letter report, FDA and DHHS re-evaluated the project and determined that the original scope was too ambitious, that a ranking of products or product categories would be premature, and that the project should be revised to focus more on developing a framework that could be used to evaluate and characterize the public-health consequences associated with FDA-regulated products or product categories in the context of various decision scenarios. Furthermore, FDA and DHHS wanted a framework that would provide a common set of metrics that would enable each center to evaluate the public-health consequences using a common terminology and approach that would allow comparisons within and among disparate programs. The present report reflects the change in scope; it describes the risk-characterization framework proposed by the committee, illustrates the use of
that framework with several case studies, and provides the committee’s conclusions and recommendations.
This report has been reviewed in draft form by persons chosen for their diverse perspectives and technical expertise in accordance with procedures approved by the NRC’s Report Review Committee. The purposes of the independent review are to provide candid and critical comments that will assist the institution in making its published report as sound as possible and to ensure that the report meets institutional standards of objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process. We thank the following for their review of this report: Susan Alpert, Medtronic; Robert L. Buchanan, University of Maryland; Julie A. Caswell, University of Massachusetts; Louis Anthony Cox, Jr., Cox Associates; Robert L. Davis, Kaiser Permanente; James K. Hammitt, Harvard University and Toulouse School of Economics; Lee-Ann Jaykus, North Carolina State University; Gregory M. Paoli, Risk Sciences International, Inc.; Birgit Puschner, University of California, Davis; Jim E. Riviere, North Carolina State University; Joseph V. Rodricks, ENVIRON; and Hugh H. Tilson, University of North Carolina.
Although the reviewers listed above have provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations, nor did they see the final draft of the report before its release. The review of the report was overseen by the review coordinator, Lauren Zeise, California Environmental Protection Agency, and the review monitor, Chris G. Whipple, ENVIRON. Appointed by NRC, they were responsible for ensuring that an independent examination of the report was carried out in accordance with institutional procedures and that all review comments were carefully considered. Responsibility for the final content of the report rests entirely with the committee and the institution.
The committee gratefully acknowledges staff from several centers at FDA for attending the committee’s public session to discuss their current approaches for decision-making at the various centers.
The committee is also grateful for the assistance of the NRC staff in preparing this report. Staff members who contributed to the effort are Ellen Mantus, project director; James Reisa, director of the Board on Environmental Studies and Toxicology; Norman Grossblatt, senior editor; Heidi Murray-Smith, program officer; Mirsada Karalic-Loncarevic, manager of the Technical Information Center; Radiah Rose, manager of editorial projects; Keri Schaffer, research associate; and Panola Golson, program associate.
I would especially like to thank all the members of the committee for their efforts throughout the development of this report.
Robert S. Lawrence, Chair
Committee on Ranking FDA Product Categories
Based on Health Consequences, Phase II
BOXES, FIGURES, AND TABLES
BOXES
S-1 |
Description of Case Studies, |
|||
2-1 |
Summary Measures vs Detailed Characterization of Public-Health Consequences, |
|||
2-2 |
What about Benefits?, |
|||
2-3 |
Methods for Estimating Uncertain Quantities, |
|||
2-4 |
Importance of Characterizing Uncertainty, |
|||
4-1 |
Risk-Ranking Models for Foods, |
FIGURES
S-1 |
Factors in FDA decision-making, |
|||
1-1 |
Worldwide distribution of establishments regulated by the U.S. Food and Drug Administration, |
|||
2-1 |
A framework for risk-based decision-making that maximizes the utility of risk assessment, |
|||
3-1 |
Estimating the size of the unprotected population given uncertainty in vaccination rates and vaccine effectiveness, |
|||
3-2 |
Relationship among various factors used to estimate the number of deaths from rotavirus disease and adverse effects of vaccination for the hypothetical decision to leave RRV-TV on the market, |
|||
4-1 |
General approach used by the committee to estimate annual number of deaths and other adverse effects associated with leafy greens, |
|||
6-1 |
Median (red dash) and range (black vertical line) of estimated melamine dose for an exposed infant of various ages given uncertainties in melamine concentration in formula, infant weight, and formula consumption, |
|||
6-2 |
Distribution of estimated melamine dose for an exposed infant of various ages given uncertainties in melamine concentration in formula, infant weight, and formula consumption, |
|||
6-3 |
Distribution of the number of infants exposed to various melamine doses, |
|||
6-4 |
Dose-response relationship for health effects of melamine consumption used for illustrative calculations, |
|||
6-5 |
Distribution of hogs consuming feed with various melamine concentrations, |
6-6 |
Distribution of number of hogs (left scale) and pounds of pork product (right scale) with various melamine tissue concentrations, |
|||
6-7 |
A decision tree for a testing-with-intervention decision involving infant formula and hog feed, |
|||
6-8 |
Potential outcomes of a decision to test infant formula but not animal feed, assuming a test without error, and interventions if the test result is positive, |
|||
6-9 |
Tree for the decision to test formula but not feed, including uncertainty about test accuracy, |
TABLES
2-1 |
Summary of Attributes and Metrics, |
|||
3-1 |
Risk Attributes for Mitigation-Selection Decision for Biologics, |
|||
3-2 |
Estimated Numbers and Rates of Deaths and Illnesses from Rotavirus Disease in the Absence of Vaccination, |
|||
3-3 |
Example Calculation of Estimated Number of Deaths and Illnesses from Rotavirus Disease and RRV-TV-Induced Intussusceptions for Option to Leave the Vaccine on Market (Based on Median Estimates of all Factors), |
|||
4-1 |
Risk Attributes for Targeting Decision for Foods, |
|||
4-2 |
Agents Linked to Shrimp-Associated Outbreaks and Cases in the United States, 2001-2005, |
|||
5-1 |
Risk Attributes for Strategic-Investment Decision for Medical Devices, |
|||
5-2 |
Mortality Estimates, |
|||
6-1 |
Risk Attributes for Targeting Decision that Spans Food and Drug Administration Centers, |
|||
6-2 |
Formula Consumption by Age, |
|||
6-3 |
Estimated Dose for Infants Consuming Formula with Melamine at 100 ppm in Dry Formula, |
|||
6-4 |
Estimates of Mortality and Morbidity from Melamine Contamination in Infant Formula for the Hypothesized Dose-Response Relationship Incorporating Uncertainty in Exposure and Dose, |
|||
6-5 |
Concentration of Hog Production in 2006, |
|||
6-6 |
Melamine Doses to Humans Who Consume Pork with Various Melamine Tissue Concentrations, |