Prerequisites for Implementing the Vision in Regulatory Contexts
The committee’s vision sets the stage for transformative changes in toxicity testing in the regulatory agencies and the larger scientific community. Although advances in the state of the science are indispensable to realization of the vision, corresponding institutional changes are also important. The changes will promote acceptance of the principles and methods envisioned. Acceptance will depend on several factors, some having scientific origins. For example, the new testing requirements will be expected to reflect the state of the science and to be founded on peer-reviewed research, established protocols, validated models, case examples, and other scientific features. Other factors stem from administrative procedures associated with rule-making, such as documenting scientific sources; providing opportunities for scientific experts, stakeholders, and the interested public to participate; and consulting with sister agencies and international organizations.
This chapter explores the conditions required for using the new testing strategy for regulatory purposes. It focuses on the federal agencies and identifies institutional outlooks and orientation—both tangible, such as budget and staffing, and intangible, such as leadership and commitment—that can determine the pace and degree to which the vision is incorporated into agency culture and practice. The chapter also addresses the fundamental issues related to the use and the validity of the new concepts, technologies, and resulting data for the specific purpose of developing federal regulations.
The committee’s vision anticipates continual change over the next 2-3 decades. Beyond the scientific and procedural considerations summarized in this chapter, the state of the economy, changing environmental conditions and social perspectives, and other dynamics that shape the political climate will influence legislative changes and federal budgets that, in turn, will determine the future of toxicity testing in the regulatory context.
INSTITUTIONAL CHANGE TO MEET THE VISION
Attitudes and Expectations
Full realization of the vision depends on the promotion of new testing principles and methods in the scientific community at large. As in the past, some changes will originate outside the regulatory agencies and work their way into agency practice, and others will originate in the agencies and work their way into the larger scientific community. In both cases, far-reaching shifts in orientation and perception will be critical. For risk assessors and researchers, the shifts will be from familiar types of studies and established procedures involving overt effects in laboratory animals and cross-species extrapolation to new approaches that focus on how chemicals, both endogenous and exogenous, interact in
human disease processes (Lieber 2006). Many analysts in and outside the agencies will have to apply their expertise in new ways.
The need for a change in attitude and orientation extends far beyond risk assessors and the toxicity-testing community. Most difficult, perhaps, will be the new level of scientific understanding needed to enable many participants, especially nonscientists, to become sufficiently informed to engage in discussion of the new methods. Law-makers who determine policy and appropriate funds, federal executives who determine research priorities, politically accountable managers and decision-makers who use databased risk assessment for making regulatory decisions, courts that review those decisions, and the public, which has an interest in the need for and nature of regulations, will need to become acquainted with new terminology and concepts.
Nonscientists will grasp some aspects of the new science—such as having regulations based on data derived from human cells, cell lines, and tissues rather than on laboratory animals—more easily than other aspects, such as the molecular basis of chemical changes that lead to adverse health effects. Ideally, individual or institutional “champions” will emerge to foster and guide the implementation process.
Developing and Cultivating Expertise
Effective implementation depends on competent scientists and informed agency management. Those factors are crucial: agency progress depends on the expertise and experience of the technical staff and a supportive management structure. Incorporating new tests and testing strategies into risk-assessment practices and agency testing guidelines will go no further or faster than staffing permits.
For several decades, academic institutions have prepared scientists for toxicity testing and risk analysis through training in
chemistry, biology, toxicology, pharmacology, and the related medical and engineering disciplines. Agency scientists receive their basic undergraduate and postgraduate education and training from external institutions and bring their training to bear on their work for the agencies. For many, pre-agency experience includes postdoctoral fellowships, internships, or first jobs at universities, industry laboratories, consulting laboratories, and other outside organizations. The kind of expertise currently available in the agencies therefore reflects in large measure expertise in the larger scientific community. That tradition has contributed to a large and stable cadre of well-trained scientists in the federal agencies that have science-based responsibilities. Thus, implementing the vision will require an infusion of new scientists who have education and experience in the new technologies and special training for current scientific staff and managers.
Scientists in academe, industry, and consulting laboratories and organizations have had a productive exchange with those in regulatory agencies through professional conferences and workshops, joint research projects, and peer-review activities. Fostering and accelerating those activities will be critical for implementing the vision and will require congressional and management support of targeted investment in developing and sustaining agency expertise. Scientists gravitate to attractive, well-funded, and well-staffed programs. To hire and retain high-caliber scientists in the numbers and disciplines needed, agencies will need congressional and management support of the vision reflected in budget allocations and hiring authorizations.
Policies to Foster Development and Use of New Tests
Institutional change does not come easily. The history of toxicity testing indicates that the pace and extent of change will depend in part on policies and incentives. Some policies and incen-
tives to encourage the use and development of the new tests by agencies are discussed here.
First, continued progress in the use of the new technologies constitutes the greatest incentive to reconfiguring agency testing programs in line with the vision. Policies to support and reward effective use of new testing concepts and methods should be implemented. Apart from historical high-visibility examples, such as the Human Genome Project, current broad-scale examples include the development and use of mechanistic data and the expanding list of –omics applications.
Second, policies to encourage the use of data generated with the new testing paradigm in chemical assessments by the agencies will be important. That will involve the evolution of agencies’ risk-assessment methods and guidelines as the new tests are developed and used. For decades, the federal agencies have promulgated formal risk-assessment guidelines, based in part on consultation with outside scientists and the public, that codify generally accepted concepts and methods to be followed in assessing the hazards, dose-response relationships, exposures, and risks related to environmental agents (for example, EPA 1991, 1996, 1998a, 2005). Policies to include the new technologies in agency assessments can foster and accelerate their acceptance and institutionalization.
Third, congressional funding of agencies to implement the vision is essential to support relevant research and staffing, encourage work with external scientists outside the agencies, recognize accomplishments by scientists and their management, and support other policies to promote change.
Fourth, dependence of market access on the conduct of specific toxicity tests can be a policy incentive. For example, the European Union’s Registration, Evaluation and Authorisation of Chemicals (REACH) program requires generation of a basic set of toxicity data on new industrial chemicals before the chemicals can enter the market; the program also sets deadlines for receipt of
basic toxicity data on existing industrial chemicals. Another example is the registration of pesticides in the United States.
Fifth, scientific progress in toxicity testing depends on work in academic and private-sector laboratories and in the federal sector. Congressional and agency policies and activities must ensure that sufficiently informative data generated from effective new methods are used in the regulatory process and that the large expenditures of money are not in vain.
Sixth, policies designed to overcome tendencies to resist novel approaches and maintain the status quo will be important. Implementing the vision requires periodic re-examination of testing programs and strategies in each agency and possibly a return to Congress to address outdated and ineffective programs that might impede implementation of novel tests and improved testing strategies.
REGULATORY USE OF NEW METHODS
The committee’s vision sets the stage for transformative change in developing data to meet regulatory objectives codified in laws passed by Congress. Although the term toxicity testing rarely, if ever, appears in the major statutes administered by the U.S. Environmental Protection Agency (EPA), the availability of reliable data on “adverse effects” and health or environmental “risk” is an underlying assumption in them. The Clean Water Act, the Clean Air Act, the Toxic Substances Control Act (TSCA), and pesticide and Superfund legislation are based on the availability of data for risk assessment and regulatory decision-making for chemicals in their jurisdictions.
The data can have several sources. Some statutes—such as the Federal Insecticide, Fungicide, and Rodenticide Act (FIFRA), the Food Quality Protection Act, and TSCA—authorize EPA to require the producers of some chemicals to develop and submit
specific categories of data to the agency. Other statutes—such as the Clean Air Act, the Clean Water Act, and the Safe Drinking Water Act—require toxicity data to be considered but depend mainly on information available in the scientific literature or government laboratory reports.1 Regardless of the statute or the data source, toxicity data are indispensable for well-reasoned conclusions on the nature and dimensions of risk and for well-grounded decisions on the necessity of regulation to protect the public health or the environment and on the nature and scope of any such regulations.
As discussed in previous chapters, the committee’s vision will result in the generation of data on perturbations in toxicity pathways with the use of high- and medium-throughput assays. A few of the test methods considered in this report have a long history and a place in the current regulatory testing programs and current risk-assessment guidelines and practices. Others are in early stages of development and have yet to be considered for regulatory use. Still others that will be used eventually are not yet on the drawing board or even imagined. Debate on the scientific validity of nonapical test methods and the application of the resulting data should be expected, and controversy could stall or bar the use of new test methods by regulatory agencies.
The discussion here addresses the prospect of controversy and focuses on the validity and defensibility of the new approaches. The primary measure of validity for regulatory purposes is scientific validity. Evidence of reliability and credibility that satisfies established scientific criteria is the principal basis for adopting and adapting new testing concepts and methods for regulatory use.2 However, there are also policy and procedural
In some cases, these statutes authorize EPA to apply TSCA and FIFRA testing requirements to chemicals in their jurisdiction.
Validity in this sense does not require de novo testing or further confirmation of previously validated scientific tests (see Chapter 5). Rather, it involves producing documentary evidence that the tests have been validated consistently with
aspects to validation, so the discussion also addresses administrative policies and procedures and other nonscientific considerations related to promulgating and defending government testing practices and requirements.3
Scientific Prerequisites of Validity
The federal agencies have a 75-year history of developing and promulgating toxicity-testing requirements for external entities, such as pesticide and drug manufacturers, and internal guidance for government laboratories (see Chapter 1). Documenting the validity, reliability, and relevance of test methods to the satisfaction of the scientific community has been and will continue to be an essential first step in identifying appropriate methods for use in the regulatory context. That documentation can also provide information and a tutorial for decision-makers, the public, and the courts.
Individual agency testing requirements do not arise de novo. For example, EPA’s Office of Pesticide Programs promulgates test guidelines and requirements only after a comprehensive development and review process involving public comment, harmonization with other international organizations, and peer review by experts in the field.4 Documentary evidence of validity has many sources and takes several forms. It includes evidence that customary criteria of scientific acceptance, such as peer review and publication in scholarly journals, have been satisfied. Use by other laboratories, other government agencies, or international organizations, such as the Organisation for Economic Co-operation and
standard scientific criteria. The objective is to avoid bringing unproven tests and the resulting data into the regulatory system.
Development, is an indication of scientific acceptability. As new methods emerge, case studies and peer-reviewed testing guidelines, standardized operating procedures, and practice can be used to document validity.
Establishing and documenting the validity of the new nonapical test methods and the validity of markers of adverse responses corresponding to perturbations of toxicity pathways will be important milestones in implementing the committee’s vision for regulatory use. Some considerations for accomplishing this are discussed below.
Adopting and Adapting New Test Systems and Methods
The vision prompts questions regarding the extent to which scientific progress using primarily human cells, cell lines, and cellular components in vitro can replace and, ideally, surpass in vivo mammalian systems as predictors of toxic effects in humans. Testing with cellular systems derived from human tissue and from nonmammalian systems is backed by an impressive scientific literature and has a long history that includes major contributions to cancer research and the Human Genome Project.
Regulatory agencies also use in vitro systems for toxicity testing and risk assessment. In vitro mode-of-action data were central elements when EPA proposed revisions to the cancer guidelines more than 10 years ago and in the final guidelines (EPA 2005). Mode-of-action data are featured in a wide array of risk assessments in EPA, other government institutions, and the private sector (for example, Meek et al. 2003; CalEPA 2004; NTP 2005; IARC 2006). EPA’s exploration of mode-of-action approaches illustrates the use of information on biologic perturbations involved in key toxicity pathways.
With few exceptions, such studies are used in the regulatory context mainly to supplement or complement data from in vivo
studies. As a result, despite the established value of in vitro systems for many purposes, increased reliance on them for regulatory testing may require further evidence of validity. As discussed in this report, a particularly important aspect of establishing validity concerns metabolism. Many of the issues are highlighted in the following statement:
Several major problems are encountered in studying metabolism-related toxicity in vitro: (a) modeling human metabolism…; (b) maintaining tissue-specific function in vitro; (c) selecting an appropriate xenobiotic metabolizing system; (d) keeping enzyme activity stable over time; and (e) the adverse effects to toxicity-indicator cells of subcellular metabolizing fractions…. Two further problems [are] the testing of mixtures of chemicals that might require different enzyme systems … and … the inactivation of exogenous biotransformation systems, due to exposure to certain solvents and test substance (Coecke et al. 2006).
Unresolved scientific issues of that type are potential barriers to full validation and acceptance of some new concepts and methods for use in the regulatory context. Such issues show that although the vision conforms to the current movement from in vivo to in vitro test systems, a new set of scientific and related issues may replace interspecies extrapolation as a source of controversy. For example, using human cell lines in culture instead of laboratory animals to identify early perturbations in a cellular-response network avoids the uncertainties associated with the customary animal-to-human extrapolation. But such human-to-human methods introduce new issues and related uncertainties, such as extrapolation from isolated cells in tissue culture to intact humans and from the genetic backgrounds of the cultured cells to the genetic backgrounds of individuals or populations of interest for risk-assessment purposes.
Incorporation of emerging methods depends in part on the status of the new methods in the scientific community, which in turn depends on the reliability of new test systems in identifying compounds with known biologic activities. The generic question is “readiness” for regulatory use. Methods still under development are not necessarily barred, but until they are fully tested and documentable, questions regarding extrapolation, relevance, and possible controversy with respect to use for regulatory purposes can be expected.
Identifying and Defining Markers and Indicators of Adverse Responses
The vision calls for replacing current tests for apical end points, such as tumors and birth defects, with mechanistically based testing that identifies early markers of disease and potential risk. The new tests focus on perturbations that are expected to produce adverse responses. This aspect of the vision presents validation issues that require two kinds of documentation, one scientific and one policy-related.
As discussed above, assessment of scientific validity will require evidence, such as peer-reviewed publications and other indicators of acceptance in the scientific community. Similar documentation will be required for other new end-point categories identified as early indicators of perturbations of critical pathways that have the potential to cause toxic effects.
The policy question is an old one: What constitutes an adverse effect? The regulatory trigger for many statutes administered by EPA is an adverse effect or some variation. For example, the Safe Drinking Water Act calls for establishing contaminant concentrations at which “no known or anticipated adverse effects on the health of persons occur and which allows an adequate margin of safety.” A FIFRA provision calls for preventing “unrea-
sonable adverse effects on the environment,” a phrase that includes nontarget animals as well as humans. As a result, identifying adverse effects is the objective of many current testing practices and regulations and will be critical for the use of new test methods and data.
Historically, both in legislation and in practice, testing and regulation have focused on apical end points, particularly clinically, anatomically, or histopathologically observable end points, such as tumors, birth defects, and neurologic impairments. That precedent could provide a basis of resistance to a move from traditional apical end points to perturbations of toxicity pathways. However, despite the historical emphasis, scientific and regulatory sources make clear that adverse effects embrace a wide array of end-point categories. Table 6-1 provides some definitions that are consistent with the vision’s approach to toxicity testing.
In this case, establishing validity for regulatory purposes involves documenting (1) sources that justify a broad interpretation of adverse effects as a concept and (2) published papers and other materials that show the relationship between responses in toxicity pathways and disease. Case studies that link specific chemicals, mechanistic end points, and disease would be useful.
Policy and Procedural Prerequisites of Validity
Ideally, new test systems and agency guidelines that incorporate them will co-evolve. In that regard, opportunities for public participation are as important as scientific measures of validity. For the courts, in laboratories subject to government testing requirements, and in the public forum, the perceived legitimacy of new testing approaches depends also on nonscientific factors.
TABLE 6-1 Definitions of Adverse Effect
“Adverse effect: A biochemical change, functional impairment, or pathologic lesion that affects the performance of the whole organism, or reduces an organism’s ability to respond to an additional environmental challenge.”
“Adverse effect: Change in the morphology, physiology, growth, development or life span of an organism, system or (sub) population that results in an impairment of functional capacity, an impairment of the capacity to compensate for additional stress, or an increase in susceptibility to other external influences.”
Renwick et al. 2003
“… adverse effects are changes that are undesirable because they alter valued structural or functional attributes of the entities of interest…. The nature and intensity of effects help distinguish adverse changes from normal … variability or those resulting in little or no significant change.”
“The spectrum of undesired effects of chemicals is broad. Some effects are deleterious and others are not…. [Regarding drugs], some side effects … are never desirable and are deleterious to the well-being of humans. These are referred to as the adverse, deleterious, or toxic effects of the drug.”
Klaassen and Eaton 1991
“All chemicals produce their toxic effects via alterations in normal cellular biochemistry and physiology…. It should also be recognized that most organs have a capacity for function that exceeds that required for normal homeostasis, sometimes referred to as functional reserve capacity.”
Klaassen and Eaton 1991
Establishing a Record
For any of the components of the vision, documentary evidence of scientific validity reviewed above makes up the substantive portion of the record, but evidence of public participation is also important. Current EPA practice often includes extensive discussion with scientists in universities, industry, advocacy groups, and other government agencies at public conferences and workshops. Informal or formal notice-and-comment rule-making procedures and external peer review are critical steps in the development and issuance of new testing and risk-assessment guidance (EPA 1998b, 2005).
Audience and Communication Issues
The committee’s vision is the product of extensive scientific thought supported by a substantial body of scientific evidence. The scientific principles and methods involved in the implementation of the committee’s vision are well known in the scientific community, a major constituency in the discussion of the scientific validity of data derived from toxicity tests for regulatory use. Scientists have long recognized the importance of effective communication of scientific results to a wide variety of stakeholders in toxicity testing, including other scientists, regulatory authorities, industry, the mass media, nongovernment organizations, and the public (NRC 1989; Leiss 2001; Krewski et al. 2006; ATSDR 2007). However, because of the transformative nature of the committee’s vision for toxicity testing, communication of the scientific basis of the vision and its implications for risk assessment of environmental agents will be challenging.
Here, there is a need for clarity in communicating the essence of the committee’s vision to affected parties. The nature and scientific complexity of the unfamiliar and more sophisticated methods
promoted in the vision may require new communication approaches. The scientific community may be best positioned to understand the scientific basis on which the committee’s vision rests but may need time to appreciate its implications fully. Acceptance of the committee’s vision in the scientific community will require further elaboration of the technical details of its implementation and generation of new scientific evidence to support the move away from apical end points to perturbations of toxicity pathways. The broad participation of the scientific community in the elaboration of the committee’s vision for toxicity testing is essential for its success.
Even more challenging will be the nonscientists’ understanding and acceptance of the committee’s vision. Regulatory authorities will need to consider how current risk-assessment practices can be adapted to make use of the types of toxicity-testing data underlying the committee’s vision to arrive at human exposure guidelines for environmental agents judged, on the basis of the new test results, to have toxic potential. Law-makers will need to determine whether the regulatory statutes that form the basis of such guidelines need to be modified to reflect the greater reliance on indicators of toxicity-pathway perturbations than on overt health outcomes. For regulatory and legal experts to support the implementation of the committee’s vision, it is essential that the fundamental biologic tenets underlying it be clearly articulated and reinforced by the development of the scientific data needed to support the shift away from a focus on apical outcomes to biologic perturbations of key toxicity pathways. The communication challenge will be to portray the benefits of adopting the committee’s vision in scientifically valid terms without confusing the vision with over-reliance on intricate scientific detail.
Adoption of the committee’s vision will require acceptance by politicians and the public alike. There will undoubtedly be a lack of support for its implementation if the scientific essence of the vision (the notion of toxicity pathways and the effects of per-
turbing them) is not communicated in understandable terms. Data will need to be generated to demonstrate that avoidance of such perturbations will provide a level of protection against the potential health risks posed by environmental agents at least as great as the current level. It will also be important to demonstrate that adoption of the committee’s vision will permit an assessment of the potential risks associated with many more agents than is possible with current toxicity-testing practices and that this expanded coverage of the universe of environmental agents can be achieved cost-effectively.
The vision for toxicity testing in the 21st century articulated here represents a paradigm shift from the use of experimental animals and apical end points toward the use of more efficient in vitro tests and computational techniques. Implementation of the vision, which will provide much broader coverage of the universe of environmental agents that warrant our attention from a risk-assessment perspective, will require a concerted effort on the part of the scientific community. A substantial commitment of resources will be required to generate the scientific data needed to support that paradigm shift, which can be achieved only with the steadfast support of regulators, law-makers, industry, and the general public. Their support will be garnered only if the essence of the committee’s vision can be communicated to all stakeholders in understandable terms.
ATSDR (Agency for Toxic Substances and Disease Registry). 2007. A Primer on Health Risk Communication Principles and Practices. U.S. Department of Health and Human Services, Agency for Toxic Substances and Disease Registry, Division of Health Education, Atlanta, GA [online]. Available: http://www.atsdr.cdc.gov/risk/riskprimer/index.html [accessed March 20, 2007].
CalEPA (California Environmental Protection Agency). 2004. Public Health Goal for Arsenic in Drinking Water. Office of Environmental Health Hazard As-
sessment, California Environmental Protection Agency. April 2004 [online]. Available: http://www.oehha.ca.gov/water/phg/pdf/asfinal.pdf [accessed March 20, 2007].
Coecke, S., H. Ahr, B.J. Blaauboer, S. Bremer, S. Casati, J. Castell, R. Combes, R. Corvi, C.L. Crespi, M.L. Cunningham, G. Elaut, B. Eletti, A. Freidig, A. Gennari, J.F. Ghersi-Egea, A. Guillouzo, T. Hartung, P. Hoet, M. Ingelman-Sundberg, S. Munn, W. Janssens, B. Ladstetter, D. Leahy, A. Long, A. Meneguz, M. Monshouwer, S. Morath, F. Nagelkerke, O. Pelkonen, J. Ponti, P. Prieto, L. Richert, E. Sabbioni, B. Schaack, W. Steiling, E. Testai, J.A. Vericat, and A. Worth. 2006. Metabolism: A bottleneck in in vitro toxicological test development. The report and recommendations of ECVM Workshop 54. ATLA 34(1):49-84.
EPA (U.S. Environmental Protection Agency). 1991. Guidelines for Developmental Toxicity Risk Assessment. EPA/600/FR-91/001. Risk Assessment Forum, U.S. Environmental Protection Agency, Washington, DC [online]. Available: http://www.epa.gov/NCEA/raf/pdfs/devtox.pdf [accessed March 7, 2007].
EPA (U.S. Environmental Protection Agency). 1996. Guidelines for Reproductive Toxicity Risk Assessment. EPA/630/R-96/009. Risk Assessment Forum, U.S. Environmental Protection Agency, Washington, DC [online]. Available: http:://www.epa.gov/ncea/raf/pdfs/repro51.pdf [accessed July 27, 2006].
EPA (U.S. Environmental Protection Agency). 1998a. Guidelines for Neurotoxicity Risk Assessment. EPA/630/R-95/001F. Risk Assessment Forum, U.S. Environmental Protection Agency, Washington, DC [online]. Available: http://www.epa.gov/ncea/raf/pdfs/neurotox.pdf [accessed March 7, 2007].
EPA (U.S. Environmental Protection Agency). 1998b. Health Effects Test Guidelines: OPPTS 870.4300 Combined Chronic Toxicity/Carcinogenicity. EPA 712-C-98-212. Office of Prevention, Pesticides and Toxic Substances, U.S. Environmental Protection Agency, Washington, DC [online]. Available: http://www.epa.gov/opptsfrs/publications/OPPTS_Harmonized/870_Health_Effects_Test_Guidelines/Series/870-4300.pdf [accessed March 20, 2007].
EPA (U.S. Environmental Protection Agency). 2005. Guidelines for Carcinogen Risk Assessment. EPA/630/P-03/001F. Risk Assessment Forum, U.S. Environmental Protection Agency, Washington, DC [online]. Available: http://www.epa.gov/iris/cancer032505.pdf [accessed July 27, 2006].
EPA (U.S. Environmental Protection Agency). 2006. Pesticides: Science and Policy. Office of Pesticides, U.S. Environmental Protection Agency [online]. Available: http://www.epa.gov/pesticides/science/index.htm [accessed March 21, 2007].
IARC (International Agency for Research on Cancer). 2006. Cobalt in Hard Metals and Cobalt Sulfate, Gallium Arsenide, Indium Phosphide and Vanadium Pentoxide. IARC Monographs on the Evaluation of Carcinogenic Risks to Humans, Vol. 86. Lyon, France: IARC Press.
IRIS (Integrated Risk Information System). 2007. Glossary of IRIS Terms. Integrated Risk Information System, U.S. Environmental Protection Agency [online]. Available: http://www.epa.gov/iris/gloss8.htm [accessed March 20, 2007].
Klaassen, C.D., and D.L. Eaton. 1991. Principles of toxicology. Pp. 12-49 in Casarett and Doull’s Toxicology: The Basic Science of Poisons, 4th Ed., M.O. Amdur, J. Doull, and C.D. Klaassen, eds. New York: Pergamon Press.
Krewski, D., L. Lemyre, M.C. Turner, J.E.C. Lee, C. Dallaire, L. Bouchard, K. Brand, and P. Mercier. 2006. Public perception of population health risks in Canada: Health hazards and sources of information. Hum. Ecol. Risk Assess. 12(4):626-644.
Leiss, W. 2001. In the Chamber of Risks: Understanding Risk Controversies. Montreal: McGill-Queen’s University Press.
Lieber, M.M. 2006. Towards an understanding of the role of forces in carcinogenesis: A perspective with therapeutic implications. Riv. Biol. 99(1):131-160.
Meek, M.E., J.R. Bucher, S.M. Cohen, V. Dellarco, R.N. Hill, L.D. Lehman-McKeeman, D.G. Longfellow, T. Pastoor, J. Seed, and D. Patton. 2003. A framework for human relevance analysis of information on carcinogenic modes of action. Crit. Rev. Toxicol. 33(6):591-653.
NRC (National Research Council). 1989. Improving Risk Communication. Washington, DC: National Academy Press.
NTP (National Toxicology Program). 2005. Report on Carcinogens, 11th Ed. U.S. Department of Health and Human Services, Public Health Service, National Toxicology Program [online]. Available: http://ntpserver.niehs.nih.gov/ntp/roc/toc11.html [accessed March 20, 2007].
Renwick, A.G., S.M Barlow, I. Hertz-Picciotto, A.R. Boobis, E. Dybing, L. Edler, G. Eisenbrand, J.B. Greig, J. Kleiner, J. Lambe, D.J. Muller, M.R. Smith, A. Tritscher, S. Tuijtelaars, P.A. van den Brandt, R. Walter, and R. Kroes. 2003. Risk characterization of chemicals in food and diet. Food Chem. Toxicol. 41(9):1211-1271.
Sergeant, A. 2002. Ecological risk assessment: History and fundamentals. Pp. 369-442 in Human and Ecological Risk Assessment: Theory and Practice, D.J. Paustenbach, ed. New York: John Wiley and Sons.