Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
RESEARCH ON RISK Al DECISION MAKING People in a large number of professions and disciplines are currently engaged in research related to risk and decision making: Toxicologists devise laboratory experiments to identify potential carcinogens, mutagens, and other toxic substances; Climatologists build models to predict the effects of atmospheric CON concentrations on global weather patterns; Epidemiologists use large data bases to isolate statistical associations between various risk factors (e.g., pollution and diet) and various indices of morbidity and mortality; Physicists, chemists, and biologists study fun- damental physical processes to facilitate the identification and assessment of risks; Ecologists investigate the tensions between the needs of humans and the needs of other organisms in an ecosystem; Economists explore the effects of regulation on inflation, employment, innovation, competition, and productivity; 47
48 Legal scholars assess how various liability doc- trines affect both the compensation of victims and the incentives for injurers to engage in risk-generating conduct; Psychologists develop and test theories about how people form perceptions about risks and about how people make personal decisions about risks in their daily lives; Communication specialists examine the potential effects of educational media campaigns on self- hazardous life-styles; Market researchers assess the consequences of advertising on the consumption of hazardous products and substances; Sociologists study the influences of peer pres- sure on teenage smoking and drinking habits; Political scientists describe and evaluate how different political and economic systems gener- ate and cope with risks; Philosophers and political theorists study the value trade-offs and ethical considerations in risk and decision making; Demographers and biostatisticians compile and analyze risk indices to identify crucial trends in risks over time; Defense analysts weigh the deterrent effects of weapon systems against the risks of escalation in armed conflict; Classical and Bayesian statisticians study how inferences about uncertainties should be made, how new information about risks should be in- corporated into old beliefs, and how information about risks from disparate sources should be combined in a formal decision analysis; Organizational theorists study how the incen- tives and rewards faced by employees in business firms and public agencies cause people to gener- ate and cope with risks; Engineers design safer consumer products and cleaner production processes; they worry about the cost and complexity of safety devices verse the risks of accidents; Geographers study techniques for managing natural hazards and natural disasters; and IS
49 · Decision and management scientists develop methods for formalizing value trade-offs in decisions about risks. This list, although incomplete, conveys the range and diversity of expertise now involved in research involving consideration of risk and resulting deci- sions. It also makes plain the near impossibility of establishing priorities for research. Accordingly, the agenda for research offered below is not a ranking but rather a sampling of research that is generally appli- cable to the improvement of risk analyses in individual disciplines. A message also carried by the agenda is that per- formance of risk analysis calls not only for knowledge of its methods but also for disciplinary understanding of the particular problems at issue, be they certifying airplanes or assessing the mutagenic effects of a new chemical; it also demands an awareness of economic and social implications of a given risk and various options for dealing with it. The practice of risk analyses re- quires the knowledge of many fields; it is in fact an undertaking of multiple disciplines melded together in an interdisciplinary analysis. Obviously, with the diversity of research actors, coordination of research and communication of its re- sults is a necessity if risk analyses are to have avail- able the best tools and expertise. To be more specific, research on particular issues in risk analysis that coordinates, suggests, and links insights from different disciplines may be especially effective. Research that coordinates the techniques of one field with the issues facing another does not now have a home within univer Consequently this work, falling at the interface between disciplines, is often neglected; yet it is the essence of risk analy- sis. Moreover, the coordinating function, to be truly effective, must be supported by effective communication of results to the involved parties. Our nation needs vigorous and coordinated programs of research on risk and decision making. Risk in its many forms is such a pervasive problem, is subject to so ~ _ ~ such a signifi- th~ a t ton- . a _ _ sities or research institutions. many unknowns and uncertainties. and is cant source of social concern that it demands Lion of the nation's research communities and research
50 sponsors. Research is needed not only on the identifi- cation, assessment, and quantification of various risks, but also on improved methods of analysis for decision making and on improved market, legislative, regulatory, and judicial decision-making processes. Such research should draw on experiences within the United States as well as those in other countries. There will not be, of course, any scientific resolu- tion to many of the controversies about risk and deci- sion making. Scientific research can narrow the range of disagreement about the magnitudes of certain risks, but even in those cases the result may trigger more explicit and heated controversy about value trade-offs At the 8 ame time. the whole and ethical considerations. field of conflict resolution--so integral to risk and decision making--is a fertile and valuable research . . Domain tar oenav~ora', social, and management scien- tists. Research and experimentation with improved tech- niques for mediation and bargaining could be especially useful in resolving disputes about environmental poli- cies that affect health, safety, and the environment. There is also a need for expansion of interdiscinli nary research projects on risk and decision making. The relative neglect of interdisciplinary research is under- standable, given the inherent difficulties with such efforts: differences in problem definition, language barriers between participants, problems in finding spon sors who are sympathetic to joint projects, the complex- ities in establishing appropriate peer review mechanisms for interdisciplinary work, and the extra time and ex- pense associated with combating all of these diffi- culties. Despite these problems, the committee believes that an expanded interdisciplinary research program on risk and decision making should be undertaken. We identify numerous specific research topics that are both critically important and that require the joint efforts of multiple disciplines. There are several general rationales for giving some priority to these kinds of projects: The fact that most risk problems require knowl- edge of specialists from a variety of natural, social, and management sciences; The desire of many decision makers to have an effective synthesis of scientific inputs into
51 decision making, a difficult yet appropriate demand that certainly requires multiple talents; · me ability of research teams with diverse memo hers to communicate findings to decision makers who lack scientific training and to the general public; and · The tendency of research projects within a single discipline to become preoccupied with technical intricacies that may not be central to the needs of decision makers who are faced with immediate problems involving risks. Several billion dollars are spent every year on research that is related to society's efforts to cope with health, safety, and environmental risks. Nearly all of this money is devoted to natural science and engineering research, especially biomedical research and, to a lesser extent, safety engineering. Relatively little attention has been focused on social science research, on research to develop better analytical methods for risk assessment, risk evaluation, and deci- sion making. We list social science and analytical research topics that might be explored. Research on risk analysis is not within the scope of any single agency, but coordination and the communica- tion of results may be. Alternative mechanisms may be an interagency committee or a unit separate from the governmental structure, perhaps hocused within a univer- sity. Within that framework the committee proposes an agenda for research, one that is not comprehensive but rather selective and indicative. The agenda follows in its outline that of the first section of this report and takes up in order research on: actual and perceived risks; risk generating and risk coping processes; and approaches to and methods of risk analysis. ACTUAL AND PERCEIVED RISKS A risk analysis, while asking distinct questions and applying specific knowledge, still must be provided with a perspective--the overall health of society as measured by indicators of the population's health, safety, the state of the environment, or the economy. This is par- ticularly true when the analysis must consider not only
52 the risk itself but perceptions of it. Whether the sub- ject of the analysis is the effects of changing the upper limits on nitrogen oxide emissions from cars, the effects of low-level radiation from nuclear power plants, or putting a new airport at a particular loca- tion, the common need is a yardstick against which to measure the relative risk. No such yardstick is now available; even the very extensive data collected by the committee was incomplete, especially for measures of morbidity and environmental quality. A source book of health, safety, and environmental indicators, regularly revised, would be extremely valuable; it certainly would have helped the work of the committee. An analogous volume is Social Indicators. a her of "selected data on social conditions and trends in the United States" with some international comparisons, which is assembled and periodically updated by the social indicators staff of the Center for Demographic Studies in the Bureau of the Census in collaboration with the Interagency Committee on Social Indicators. Many of the indices in this compilation of statistics are relevant to concerns about risk, but many more appropriate indices about risks are missing. A compilation of risk indicators could include data about public perceptions of risks as well as data on actual risk indices. In addition to a regularly published comprehensive collection of existing health, safety, and environmental indicators, serious thought should be given to how to collect, analyze, evaluate, and disseminate additional information about trends and patterns in mortality, morbidity, and environmental quality. Deficiencies in available data should be assessed, including the likeli- hood of misinterpretations from such data. As noted above, data on morbidity and on environ- mental quality are sparse. Furthermore, data on global conditions and on comparisons of conditions in different countries are very spotty. A more complete, current, and accurate data base would be very useful. It is an ambitious task, but a beginning could be made by outlin- ing the scope of the volume, the data not now collected that should be, and the data that are currently incom- plete or insufficiently validated. The forms in which current data are now provided should be critically examined. Are the caveats in the data clearly stated? Societies are not static and neither is their health. In looking at existing and prospective risks
53 the direction of change must be considered as well as the current state. For that reason, an effort to encourage the relevant disciplines, such as epidemi- ology, demography, actuarial analysis, psychology, and statistics, to assess likely trends in known data and their underlying forces would be invaluable in improving the analysis of risks. A dynamic analysis of this sort would, among other things, better tell how relevant current data are to possible future risks. As examples, the following are important questions for risk analysts to discuss: · Why have cardiovascular death rates declined so dramatically over the last two decades? · How much are cancer rates likely to increase or decrease in the future? What are the causes of higher mortality among blacks compared with whites, males compared with females, Americans compared with Swedes, etc.? Risk Perceptions and Behavior Two research communities are concerned with perceptions of risk and choices among them. Behavioral or descrip- tive decision analysts are concerned with how both lay people and experts actually perceive risks--how they learn about risks, how they behave, and how they explain or rationalize their behavior. This community includes cognitive psychologists, psychometricians, learning theorists, and some economists and operations research- ers. Normative or prescriptive decision analysts are ~r~~-~ concerned with how people should behave or might want to behave if they were consciously made aware of underlying desiderata for reasoned behavior. The field has a the- oretical and an applied side. Theory asserts how ideal- ized people should behave to satisfy certain rational desiderata, while the applied side is concerned with guiding people to behave more "rationally." Prescrip- tive decision analysis includes in its community many (but not all) theoretical economists, decision analysts, operations researchers, and management scientists. Behavioral decision analysts have demonstrated in the laboratory and through surveys and clinical studies not only that individuals do not behave according to the rationality assumptions of the prescriptive theories,
54 but also that many individuals continue to behave the same way even after they are made aware of their so- called inconsistencies. "Behavior" as used in this context has three facets: how people perceive risks and think about uncertainties; how with new information they modify these perceptions, that is, how they learn; and how they choose among alternatives when uncertainties are present. There is a need to foster a closer link between these two communities. Prescriptive theories that are designed to guide behavior should be modified to account for real psychological concerns. - ~ ~ ~ In dealing with issues ot risk, many people do not think probabilistically, even when they know how to, and the heuristics they use to guide their behavior often are clearly inappropri- ate. How can some of the insights from prescriptive decision analysis be introduced into the general con- sciousness of people in ways that will produce more reasonable heuristics for behavior? The aim is to teach people how to think about risks more clearly but not to indoctrinate them into narrow channels that eliminate real psychological concerns. Further Descriptive Research on Perceptions It may be appropriate to extend in several directions the research being done on the ways in which people think about risks and about such risk-taking behavior as choice of insurance coverage, choice of occupation, life-style habits, life-saving precautions, and choice of investment. Several general questions are worth pursuing: In perceptions about risks and in risk-taking behavior, what are the differences (if any) between cultures--between the United States and other countries; and, within the United States, between the sexes, races, religious affilia- tions, rural and city dwellers, socioeconomic groups, people with different levels of educa- tion, etc.? What accounts for these differences? Bow do children think and learn to think about risks? Do competitive sports have an influence on risk attitudes? What are the influences of early education, number of siblings, etc.? , _ _ _
55 How do perceptions of risk relate to beliefs in astrology and how do attitudes toward risk re- late to other attitudes, e.g., internal versus external control, fatalism? Is there a rela- tionship between those beliefs and attitudes and the perceptions of risks What kinds of people are more concerned about what kinds of risks? For example, to what extent and why do risk perceptions differ, for example, between business leaders and govern- mental officials or between technical experts and lay people? What are the attributes of a risk that people are particularly concerned about (e.g., control- lability, catastrophic potential, etc.~? What is the role of the media in shaping and responding to people's perceptions about risks? To what extent are an individual's attitudes toward entrepreneurial risks related to his or her attitudes toward health, safety, and envi- ronmental risks? What are the historical trends in people's atti- tudes toward value trade-offs--between dollars and health, economic growth and environment, and so forth? THE GENERATION OF RISKS As we have argued, the responses to risks, the coping mechanisms, depend on the class of risks and in turn on how a particular risk is generated. In some instances, such as hang gliding, the policy response is to do noth- ing; in others, such as child abuse, the response is vigorous. The design of coping mechanisms and the selection of alternatives is aided by an understanding not only of how risks are generated but also of the milieus in which people tolerate some risks but not others, deliberately expose themselves to risks or with vigor or considerable cost avoid them. Both political and economic milieus may affect attitudes toward taking risks. Thus, in understanding responses to risk or to its avoidance, it would help to more precisely tabulate the different health, safety, and environmental problems in nations with different political systems (democracies 1
56 versus autocracies), different economic systems (capi- talist versus socialist), or different national mores (e.g., West German versus American). The corollary question is whether some political or economic systems are better able to cope with certain risks than with others. If so, which risks and why? The essential purpose of the research, then, is to clarify the dif- ferent responses to risks in different political, eco- nomic, or national systems and also to analyze the nature of the risks in those countries. What are con- sidered to be serious risks in the U.S.S.R., for exam- ple, that are accepted in the United States? Or which risks that concern policy makers in the United States are given a gloss in France? What is the pattern of causal relationships that link wealth and health? Does "richer mean safer" in countries as wealthy as the United States? How impor- tant are the various correlates of high standards of living, including education levels, leisure, and pat- terns of employment, in determining levels of health? To what extent would a change in the rate of economic growth affect the rate of progress against mortality and morbidity? COPING WITH RISKS There are many ways to cope with risk, of which Figure 2 lists only a few. Some are used routinely, such as in- suring against a fire or car accident; others, such as effluent fees, are intensely discussed in the academic literature but have yet to be used in the United States; others that are applied to one sort of risk may be ap- plicable to others. A critical function of research should be to explore the dimensions of both existing and untried coping strategies, determining the type of risks they may best fit, the implications for decision makers, and associated benefits and costs. Social Experiments It is, of course, very difficult to predict with reason- able accuracy the consequences of innovative coping strategies, such as the sale of pollution licenses and changes in tort law. Consequently, research is needed
~7 on ways in which new coping strategies can be experi- mentally tested, especially those for possible new environmental threats, such as acid precipitation, changing climates due to rising levels of carbon diox- ide, the erosion of ozone by manufactured chemicals, or indoor pollution. The design of a social experiment in risk-coping should include procedures for monitoring and evaluating the effects of innovations over time with adaptive feedback controls. International Comparisons The same research agenda for comparing the attitudes toward and generation of risks in nations differing in their politics, economics, and mores can be applied to an international survey of coping strategies. Certainly in developing a richer array of coping strategies, as suggested above, the United States would benefit from the experience of other industrialized countries; as with the generation of risks, however, little effort to date has been made to compare, contrast, and critically evaluate the tactics in different countries for coping with risks. Some coping strategies that are employed extensively in other nations, such as effluent fees for pollution control and legal restrictions on self-hazardous behav- ior, are not used extensively in the United States. What factors explain the difference in usage, what can be learned from foreign experience, and how can the suc- cessful strategies from abroad be implemented in the United States? A Particular Strategy: Monitoring Chemicals The society is exposed to tens of thousands of chemicals and each year thousands of new chemicals are intro- duced. Some of these chemicals are noxious without important benefits and clearly have been removed or have not been introduced. Others are clearly benign with copious benefits. Others present problems: Are their uncertain risks and costs worth their uncertain bene- fits? One question is what to do about a given chem- ical. A larger and more important question in how to design an entire system to review old and new chemicals
58 --a system that must exploit and perhaps modify given institutional structures to ensure that trivial risks do not overload the system and that proper attention is paid to large risks, whether they are historical (often known risks) or are more uncertain (see Weinstein, 1979, 1981~. That is simply a recognition of the nature of the problem: a huge industry is in place, the national economy depends in part on both the health of the industry and the availability of its products, and new chemicals are constantly introduced and others removed. me issue thus is more than testing a particular chem- ical; it involves dealing with an industry that is an economic pillar in a manner that strengthens the economy and protects the population. The coping tactics applied to monitoring chemicals are necessarily iterative ones, a constant seeking of an equilibrium among economic, health, safety, and environmental concerns. Such a strategy means setting priorities, accumula tong more experimental evidence, rearranging prorates, and so on. Such a strategy must involve a host of actors, including government agencies, government research facilites, business firms and their research facilities, universities, and industry-sponsored re- search facilities. It must be designed so that the incentives of the marketplace work, on balance, for the common good. The problem can be characterized as a sequential design for the interaction of experimentation and action in a decentralized environment. Science, assessment, experimentation, evaluation, action, monitoring--all must interact within the confines of our institutional structures. Is this a researchable topic? Certainly what is now done can be described. The specific procedures of the current strategy can be critiqued and ameliorative, incremental changes can be recommended. What we are doing can be contrasted with what other countries are doing. While their institutions differ, ~ epidemiological data are still relevant. One might also simplify the problem by ignoring institutional con- straints and investigate how a control system might work in a more abstract setting--the hope being that some of the analysis would generate realistic heuristic in- sights. One might even suggest experimental modifica- tions of the current strategy and carefully monitor and evaluate the performance of the modified system. - __~e~ en -~ ~ ~ g 1
59 Underlying what has been said so far is our ~oci- ety's capacity to deal with aggregated risk: that is, an entire industry rather than any one of its products. New strategies may be needed to deal with that diffi- culty. Research is needed to consider what these strategies might be, the suitability of our present institutional structures for applying them effectively, their applicability to a particular problem involving aggregate risks, and their probable costs and benefits. One might also foreshadow the discussion below on the assessment and evaluation stages of risk analysis by noting that the separation of the two stages is quite natural in considering a single chemical, but not at all in coping with a risk analysis en bloc for hundreds or even thousands of chemicals. Also, examination of such "macro risks" demands extensive interdisciplinary inter- actions. Taxation Strategies Economists have published research on the theoretical advantages of tax schemes for coping with various risks (e.g., effluent charges for pollution control and con- sumption fees for hazardous products). The issue now is whether and how this theoretical work will actually be applicable within political constraints. Can taxation strategies be used effectively to cope with the emerging problem of hazardous wastes? What are the political or institutional impediments to more widespread use of these approaches in the United States, and how can they be made more politically feasible? Mediation and Arbitration Power plants, refineries, dumps for toxic chemicals, airports, and incinerators frequently have to be built despite the fact that increased risks may be imposed on the nearby residents. Vociferous arguments occur about the siting of facilities that are deemed on balance beneficial to the majority but unfair to a minority. Ultimately such decisions must be made through a politi- cal and judicial process, but analysis may be of assist- ance in several ways. In particular, is it possible to
60 elicit reliable opinions about costs, benefits, equi- ties, and values from those having a stake in the result? This is a growing research area among econo- mists and policy analysts. The choice of the site and the type and the size of a facility pose difficult technical problems. If there were no conflicts between interest groups, then optimi- zation techniques, such as operations research methods, could be used. There is a tendency for conflict to drive out analysis, but a case could be made for the reverse: Adding conflict to an already technically difficult choice makes the problem still harder, and analysis might be useful in designing compromise solu- tions. mis is ~ however, rarely done in practice. In recent years, several environmental disputes have been mediated successfully because the mediator has been astute enough to suggest a new compromise alternative that was not on the table in front of the disputants (see for example Center for Environmental Conflict Resolution, 1978~. Part of the mediators' task is to turn a win-lose confrontation mentality into a win-win joint problem-solving task; very rarely has formal analysis been used in this type of joint problem solving. If towns or regions were monolithic, it might be possible to compensate a locality that accepts a locally noxious facility. Auctions for example, could be used , ~_ ~_ ^ : ~ _ ~ ~-~ ~ to determine the amount or rlnancla' compensation co calm locality accepting the facility. Compensation can come in many guises. One of them is to couple alternatives that raise risks with those that lower them so that the net effect is lessened risk for the locality that ac- cepts a noxious facility. Applied research talent is needed to exploit these ideas, to show the practicality of such interventions in settling disputes. Innovation, Productivity, and Competition Political debates about regulatory reform frequently contain assertions about the effects of federal regula- tion on industrial research and development, produc- tivity, and competition. Reliable economic research does not currently exist to refute or establish most of these claims. Since repeal or modification of many regulations may occur during the tenure of the Reagan
61 administration, a unique opportunity exists to asses 8 the consequences of regulatory reform. Of course, any macroeconomic effects of regulatory changes may be lost in "measurement noise." However, microeconomic studies of particular firms and industries can bolster cause- and-effect hypotheses. Some issues deserving special attention are: the effects of regulatory changes on small firms and industry market shares; the effects of regulatory uncertainty on research, development, and capital investment decisions; and the positive and negative effects of regulation on measures of economic productivity. RISK ANALYS IS While, as emphasized in the first part of this report, analysis has a limited role in decision making, it still can be a powerful tool for categorizing risks, assessing them, evaluating a given set of alternative coping strategies, and devising new alternatives. The actual role of risk analysis in particular situations is variable, as are the methods used. There is also the fact that in some decision making on risk, analysis has no role; in other situations analysis is useful only if it is designed so that it provides information that the decision maker cannot obtain. Research that is based largely on retrospective looks at analyses that have been done and their uses would be helpful to both analysts and to decision makers in determining when formal analyses may be useful, operating under what criteria, and involving what sort of dialogue between the analysts and those using their advice. Such research would inform the committee's belief that formal analyses can often be useful to decision makers at various stages of interaction, either by providing critical information or by helping decision makers to structure their thinking. As part of learning how to do better risk analyses, existing studies could be reviewed, compared, evaluated, and criticized--and then perhaps redone. Comparisons of studies within a fairly narrow field could be used to discover and elucidate valuable features elements to be included, good methods, etc., versus omissions, pit- falls, inappropriate methods, etc. Comparisons of rela- tively good studies across fields could be used to !
62 discover, elucidate, and illustrate the common features as well as the reasons for differences in emphasis, me goal of studies that redo a risk method, and so on. analysis would be to exploit improvements in methods, better data collection and problem formation, the use of experts, and the communication of results; the purpose is to show how improvements can be made by explicitly building on the partial successes and failures of the past. me studies that have been done of risk analyses are often regarded, justifiably or not, as tainted by adversial biases; that is, they were done by those unhappy with the conclusions of the original work that triggered the follow-up study. It may be useful to expend considerable effort on analyses of a few specific risks to advance the state of the art of risk analysis. The purpose is to produce analyses of such high quality that they become models in the field. Specific problems should be chosen to illus- trate various important features of analysis and with a view toward the transferability of approach, technique, style, organization, and process. There are now very few calculations of risk in the open literature and still fewer with detailed commentaries. Some of these prototypical analyses might concen- trate on risk assessments (i.e., on estimating the mag- nitudes of health, safety, and environmental hazards); other analyses might concentrate on separating and link- ing assessment with those evaluations that consider policy alternatives, value trade-offs, political con- straints, implementation concerns, and so on; other analyses might concentrate on cases in which the separa- tion between assessment and evaluation is inappropriate or not achievable without crippling distortions. Some "models" might illustrate the usefulness of simple, back-of-the-envelope quick analysis, whereas others might focus on more elaborate time-consuming studies. At the core of risk assessment and evaluation are experts--generating facts, expressing judgments, influ- encing values. As expertise becomes more important in social decision making, research is needed about what it is, what its systematic biases are, how it interacts with other sources of knowledge or opinion (such as intuition, common sense, adversarial presentation, etc.), how it is acquired and lost, how experts communi- cate with lay persons and with other experts, and whether expertise can be isolated from other ingredients . .
63 of opinion so that others can evaluate those ingredients separately by the different criteria appropriate to each. Some of the ethical issues that arise in coping with risks, alluded to earlier, include: . . . Distributional equity with respect to the poten- tial costs and benefits of policies that gener- ate or mitigate risks or allow risks to remain unabated; Distributional equity between the interests of people living today and members of future gener- ations with respect to potential costs and bene- fits of policies that generate or mitigate risks or allow risks to remain unabated; · The imposition of risks that will fall on a statistically chosen few for the good of many; Proper balancing between incommensurables like expenditures and the saving of anonymous lives; and · The balance between individual freedom of choice and social interventions. Surveys have asked Americans how they feel about such issues, but the questions usually are vague and ineffec- tive in probing deep ethical values. Can a better job be done in understanding how people really feel about these issues? Can simple questions be devised whose responses correlate well with the results of deeper, more probing, more time-consuming studies? A specific research agenda in thin area might include the compari- son of the results from standard surveys and in-depth analysis of the same issue. The outcome may aid in determining the most efficient and economical methods of probing the distribution of ethical values in society. If so, will they lead to a better understanding of the distribution of ethical values in society through survey techniques or through studies of small groups of indi- viduals? SYNTHESIZING RISK ASSESSMENTS Separating assessment from evaluation in risk analyses is difficult but also more commonly done than supposed. It may be done poorly. Nevertheless, the effort is
64 generally made to separate from evaluation those com- ponents that are a part of risk assessment, such as the identification of hazards, the review of scientific theories and facts, and the compilation of experimental and epidemiological data. Such separation is attempted because it aids systematic and rational attempts to deal with complex issues. The synthesis of a risk assessment into a form that is coherent and useful to the evaluator and to the policy maker is, as also noted earlier, invariably more difficult than the original separation, given the dis- puted interpretations, conflicting or incomplete data, and disparate methodologies that typify many assess- ments. Accordingly, research into understanding and ameliorating those difficulties is needed. A clear presentation of indisputable facts dealing with a given uncertainty or uncertainties might suffice tor decision and policy purposes. In some specific cir- cumstance, it may be clear that a given chemical is severely carcinogenic and that appropriate, inexpensive substitutes exist; no profound analysis may be necessary to conclude that the chemical should not be used. In other circumstances the evidence may indicate clearly that the chemical has earned an impeccably clean bill of health. In some circumstances, however, the evidence may pull in different directions: theory may pull one way; in vitro tests another way; animal studies of dif- ferent qualities and credibilities may pull in several contradictory ways; human controlled experiments may be sparse and only indirectly relevant; epidemiological studies may be marred by a host of extraneous complicat- ing factors that make cause and effect almost impossible to sort out. In short, there are cases in which the facts do not speak for themselves. Policy makers may want to know how the experts interpret the data; they may want all the theory and data relating to the uncer- tainties in question to be synthesized in a way that will be suitable as an input to the policy-evaluation process. Statisticians are trained to work with other scien- tists to tease out the inferential meaning of exneri mental evidence. ~ structured problems for which there are . . ~cat r- There are even textbook cases of well- formal ways of combining evidence from different sources. But there are a host of real problems that plague risk assessors, for which a reasoned syntheses of the data is extremely
65 difficult, even for the most sophisticated statisti- cian. Synthesis, if it is to be done at all, must incorporate the judgments of experts who have studied the empirical facts and who are knowledgeable about the science involved. The skills of the scientist, the statistician, and the detective have to be combined-- creatively and artistically. How can this task be done better? Can the art, if not the science, of synthesis be taught? At the very least, there is need for docu- mentation of case studies of good and bad syntheses. Can people even recognize or agree on good and bad cases of synthesis? Considerable research has been done in the area of eliciting judgments about uncertainties from a single individual. Some research has been done in statis- tically validating individual judgments about uncer- tainties with reality (e.g., how well do meteorologists or stock ,..arket analysts calibrate over time); some research has examined how independent judgments from - several experts can be combined and how well these composite assessments calibrate with reality (e.g., pooling betting odds of experts on horse races or other sporting events); some research examines formal itera- tive techniques that drive experts toward agreement (e.g., Delphi techniques); a great deal of sociological research has examined small group behavior. Despite all this research on component aspects of the problems faced by risk-assessment committees, there are very few case studies of committee processes and deliberations that can be used as appropriate models. There has been considerable research on the Delphi technique, but the reviews are mixed at best and few substitutes have appeared on the scene. The deliberation of experts in committees is a pervasive issue in risk assessment, and much more research should be addressed to the process itself. In what form, for example, should substantive dis- agreements among committee members be reported to the evaluator? How can the significance of such disagree- ments to the central policy questions be stated? Should, in fact, an assessment committee consider the importance of its disagreements to the resolution of policy issues? Research also is needed on techniques for separating assessment from evaluation, both the effectiveness of current techniques and the development of alternatives.
66 Finally, studies are needed of situations in which sepa ration is impossible or inappropriate. What are the most effective ways to conduct analyses for which decom- position is not done? What are examples of such studies done well? Or done poorly? How can such studies be structured to derive optimum gain from peer review? 1-
