The Quality of Risk Assessment Includes Both Process and Product Attributes

The decision-making environment associated with health and environmental risk management compels the various users of risk assessment to value and scrutinize the assessment process. In addition, risk assessment is understood to result in a set of final products whose specific attributes are critical for meeting their objectives. In a sense, it may be neither possible nor appropriate to separate the process from the product. The situation is somewhat analogous to that of other products whose quality is more readily scrutinized with respect to the process that is used rather than through scrutiny of detectable qualities of the final product. For example, the safety aspects of the quality of complex engineered systems, medical devices, and foods are increasingly scrutinized with respect to the quality of the process that generates and maintains them rather than judged solely on the basis of measurable qualities of the final product. Similarly, the final products of a risk assessment have a mixture of detectable and undetectable qualities, and both the final product and the underlying process must be considered in judging the overall quality.

Given the demands of health and environmental decision-making, perhaps the most appropriate element of quality in risk-assessment products is captured in their ability to improve the capacity of decision-makers to make informed decisions in the presence of substantial, inevitable and irreducible uncertainty. A secondary but surely important quality is the ability of the assessment products to improve other stakeholders’ understanding and to foster and support the broader public interests in the quality of the decision-making process (for example, fairness, transparency, and efficiency). Those attributes are difficult to measure, and some elements of quality often cannot be judged until some time after the completion of the risk assessment.

Formative and Iterative Design of Risk Assessments

For the committee’s purposes, the term design implies adopting a user-centered perspective to craft both an assessment process and a decision-support product that achieves the objectives of supporting high-quality decision-making while working within inevitable constraints. Accordingly, an important part of the early design process is the understanding and weighing of all the objectives, recognition of constraints, and explicit acknowledgment of the need for tradeoffs.

Design will inevitably occur throughout the risk-assessment process, and flexibility and iteration will be important aspects of the overall process design. Like any complex product designed in a complex environment, the process and product may need to be redesigned as objectives and constraints inevitably change and in response to new knowledge. While recognizing the iterative nature of risk-assessment planning, the committee strongly encourages increasing attention to design in the formative stages of a risk assessment. Such a shift in attention is recognized by the Environmental Protection Agency (EPA 2004a). It is also captured in guidance documents for ecologic risk assessment and cumulative risk assessment (EPA 1992, 1998, 2003). In those applications, EPA has adopted two tasks labeled planning and scoping and problem formulation. The two tasks are examples of early design activities, and the committee believes that they should be formalized, applied more consistently in risk-assessment activities, and, perhaps most important, result in concrete outputs detailing the rationale and findings of the early design process. The tasks are described in more detail later in this chapter.

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement