National Academies Press: OpenBook
« Previous: Front Matter
Suggested Citation:"Summary." National Research Council. 2012. Using Science as Evidence in Public Policy. Washington, DC: The National Academies Press. doi: 10.17226/13460.
×

Summary

Fresh from the contributions made by science to the World War II success, at mid-century the nation adopted a broad policy to invest heavily in science and technology as a foundation for economic growth, social welfare, and national security. The emphasis was on the physical and biological sciences, but the social sciences were mobilized with respect to selected foreign and domestic challenges—area studies for the former and large-scale empirical projects on social welfare for the latter.

The 1966 study Equality of Educational Opportunity (known as the Coleman report) is a convenient marker for the arrival of “big” social science. It was designed to inform national and state policy relevant to reducing racial disparities in public education. Other large-scale research projects followed: on a negative income tax, housing allowances, and health insurance, among others. Evaluation research was announced as a new research specialty. Later in the century emphasis was placed on performance metrics, social indicators, ranking schemes, comparative assessment, and related tools and concepts based in social science. Private sector organizations—university centers and institutes, think tanks, survey houses, and for-profit consulting firms—rapidly expanded in numbers and scope, as did graduate-level schools to prepare professionals for careers in public policy. The federal statistical system made available its significant information base for policy analysis in these nongovernmental settings. The federal government recruited social scientists in executive agencies and on congressional staffs.

Suggested Citation:"Summary." National Research Council. 2012. Using Science as Evidence in Public Policy. Washington, DC: The National Academies Press. doi: 10.17226/13460.
×

By the end of the 20th century, a multibillion dollar policy enterprise was in place, drawing on private philanthropic support as well as federal and state funding. The task of this loosely interconnected policy enterprise is to describe social conditions, advise on policy interventions, test alternative program designs, and evaluate outcomes. This work is funded on the promise that good science will be used to decide what social conditions need attention, what should be a public responsibility or better left to the market or not-for-profit actors, and what interventions—to grow the economy, improve welfare, protect security—are efficient and effective.

As the policy enterprise expanded and extended its reach, interest mounted in whether its knowledge products were being adequately used. Continued investment in producing the knowledge suggested it was used and valued, but how valuable and for whom was uncertain. This uncertainty was addressed in a 1978 National Research Council report, Knowledge and Policy: The Uncertain Connection. The report found that, despite numerous social science studies of policy interventions and steps to increase their relevance to and use for policy making, “we lack systematic evidence as to whether these steps are having the results their sponsors hope for.…”

More than three decades later our report returns to the “uncertain connection,” to again ask what is known about how scientific knowledge is used in public policy and how it can be more effectively used. The Committee on the Use of Social Science Knowledge in Public Policy was charged by the National Research Council “to review the knowledge utilization and other relevant literature to assess what is known about how social science knowledge is used in policy making … [and] to develop a framework for further research that can improve the use of social science knowledge in policy making.”

The first charge, to assess what is known, led us to an early and obvious point. Knowledge from all the sciences is relevant to policy choices: the physical sciences inform energy policy on renewable efficiencies; the biological sciences inform public health policy on infectious diseases; the engineering sciences inform national defense policy on weapon design; the social sciences inform economic policy on international trade trends. Understanding whether, why, and how this scientific knowledge is used, however, is uniquely suited to the methods and theories of the social sciences. Making “use” of scientific knowledge is what people and organizations do. And what people and organizations do is the focus of social science.

To date, there has not been much success in explaining the use of science in public policy. We base this statement on three findings. First,

Suggested Citation:"Summary." National Research Council. 2012. Using Science as Evidence in Public Policy. Washington, DC: The National Academies Press. doi: 10.17226/13460.
×

although there are heuristically valuable typologies of ways science is used in policy, the typologies have not (and perhaps cannot) guide empirical research programs. Second, the research specialty labeled “knowledge utilization” has focused on challenges highlighted by the “two communities” metaphor (researchers and policy makers, each with their distinctive cultures) and proposed various innovations to improve communication and interaction between science and policy—brokering, translation, interaction models. There is little systematic research on whether these innovations are improving the use of science in policy, although there are clear indications that they are being usefully applied in practice settings. In fact, it is not even clear that the two communities metaphor is the most fruitful way to frame a study of knowledge use in policy. Third, although the relatively recent approach known as evidence-based policy and practice, focused on improving understanding of “what works,” has influenced the production of scientific knowledge, it has made little contribution to understanding the use of that knowledge. In some of its more prominent formulations the issue of “use,” because it involves political and value considerations, is said to be outside the scope of evidence-based policy.

If more than three decades of worrying about science use in public policy has not produced satisfactory explanations, it may be that we have been looking in the wrong place—for a coherent typology of use or ways to bridge the gap between two communities. The committee turned its attention to a research framework that draws on recent developments in social science perhaps better suited to explaining the use of science in public policy.

The first step in constructing the framework reprised a familiar point. Science, when it has something to offer, should be at the policy table. But it shares that table with an array of nonscientific reasons for making a policy choice: personal and political beliefs and values are present, as are lessons from experience, trial and error learning, and reasoning by analogy. Obviously, political matters and pressures weigh heavily when policy choices are made. Nevertheless science is a unique voice. What science has to say about policy choices results from investigations governed by systematic and rule-governed efforts that guard against self-deception—against believing something is true because one wants it to be. Because science is designed to be disinterested, if a policy question involves what are the “real” conditions or what will “probably” happen if one policy is implemented instead of another, science is generally a more dependable and defensible guide than informed hunches, analogies, or personal experience. Also, at least in

Suggested Citation:"Summary." National Research Council. 2012. Using Science as Evidence in Public Policy. Washington, DC: The National Academies Press. doi: 10.17226/13460.
×

a democracy, political leaders are obliged to give reasons for their policy choices—the theory of democratic accountability underpins this obligation. These reasons often require science-based description of conditions needing attention and explanations of what is likely to happen (or did happen) because of a policy intervention.

Science has five tasks related to policy: (1) identify problems, such as endangered species, obesity, unemployment, and vulnerability to natural disasters or terrorist acts; (2) measure their magnitude and seriousness; (3) review alternative policy interventions; (4) systematically assess the likely consequences of particular policy actions—intended and unintended, desired and unwanted; and (5) evaluate what, in fact, results from policy.

Across all of these tasks, there are political and value considerations that are outside the scope of science. We acknowledge that and build it into the recommended research framework.

A FRAMEWORK FOR RESEARCH ON USE

Policy is made in many settings. It evolves from a many faceted social process involving multiple actors engaged in assembling, interpreting, and debating what evidence is relevant to the policy choice at hand, and then, perhaps, using that evidence to claim that a particular policy choice is better than its alternatives. This process is best understood as a form of policy argument or practical reasoning that is persuasive with respect to the benefit or harm of policy actions. Policy argument includes generalizations, extrapolations, assumptions, analogies, metaphors, anecdotes, and other elements of reasoning that differ from and can contradict scientific reasons. From this perspective, scientific knowledge is “evidence” when that knowledge is used in support of statements relevant to policy claims. “Evidence” does not reside only in the world where science is produced; it emerges in the political world of policy making, where it is interpreted, made sense of and is used, perhaps persuasively, in policy arguments. Evidence-influenced politics is suggested as a more informative metaphor, descriptively and prescriptively, than evidence-based policy.

Our research framework argues for more careful study of policy argumentation, as well as for increased roles for the psychology of decision making and for systems perspectives. The social sciences offer important knowledge about how mental models, belief systems, organizational rules, societal norms, and other factors influence the behavior of decision makers. They also offer important knowledge about how people learn, when

Suggested Citation:"Summary." National Research Council. 2012. Using Science as Evidence in Public Policy. Washington, DC: The National Academies Press. doi: 10.17226/13460.
×

they optimize and when they satisfice; why they organize themselves, form institutions, communicate, establish norms, and develop routines; how they assess risks; and how they make decisions, individually and collectively. This array of scientific specialties has never fully addressed a key issue: when, why, how, even whether science is used in public policy making. Research can explain the cognitive operations and biases that policy makers and scientists bring to their work and the context-specific situations, practices, logics (ways of reasoning and understanding), and cultural assumptions of the settings in which they operate. Relevant research fields include social psychology, behavioral economics, decision theory, and organizational sociology. We urge scholars in these and related specialties to investigate the use of scientific knowledge in policy making.

Policy interventions unfold in large, complex, dynamic social systems. A systems perspective helps decision makers and researchers think broadly about the many effects a policy may produce and the ways in which a planned social intervention interacts with other existing interventions and institutional practices. Rarely can the study of the individual components of a system lead to a full understanding of the system. There are systems effects on individual actors and the system as a whole, including emergent, indirect, and delayed effects, as well as unintended and unpredictable consequences from the interactivity of a system’s elements. The social sciences bring a variety of approaches and methodologies to the study of complex systems. Examples of the use of systems thinking in the study of national security, obesity prevention, and the evaluation of complex social interventions illustrate its potential utility in policy making more broadly.

THE NEXT GENERATION OF RESEARCHERS AND PRACTITIONERS

The three actors central to advancing and applying the research framework are established scholars in the fields and specialties identified above, Ph.D. candidates in those fields and specialties, and administrators and faculty responsible for curricula in schools and programs characterized by the term “policy education.”

Established scholars have long-range research agendas and are not easily persuaded to drop them to pursue new questions. New research fields nevertheless emerge when even a few established scholars focus their theories and methods on a major question getting little attention. Among decision-making theorists, cognitive psychologists, and scholars of system

Suggested Citation:"Summary." National Research Council. 2012. Using Science as Evidence in Public Policy. Washington, DC: The National Academies Press. doi: 10.17226/13460.
×

properties are some, we expect, who will find that posing the issue of science use as needing their attention will be attractive. It is exciting to be in on the ground floor of a new field of research, especially when there is a large and influential audience waiting for guidance on how to strengthen science use in policy.

A companion effort focuses on students at the Ph.D. stage. There are numerous examples of philanthropic and federal funding that shaped the choice of dissertation topics and the early research trajectory of young scholars—resulting in new scholarly fields of inquiry. With heightened political attention to the “broader impacts” of science, answers are being sought, for example, for better ways to link natural and social sciences in addressing policy challenges, to better understand how variability in the quality of scientific evidence affects its use, and to the value of investing in intermediaries promising to promote the use of science as evidence.

The third audience is those responsible for the curriculum in public policy schools and programs in U.S. universities, which annually graduate thousands of students, many of whom find positions in the policy enterprise. It would be useful to know in some detail, first, what these students are and are not being taught as it bears on the use of science in public policy. That would require an investigation far more extensive than the committee could undertake. We did conduct a limited review sufficient to reach conclusions relevant to what our findings imply for policy education.

Our point is simple: policy education should equip its graduates to promote the use of science in policy-making settings. Graduates need, obviously, a working familiarity with the substance of policy issues and competency to locate, assess, and introduce validated research on those issues. But more is needed. Success at promoting science depends on grasping the complexity of the policy world, and on understanding the assumptions underlying divergent policy framings, expert judgments, and consensus-building techniques, as well as standard analytic methods and approaches. Policy students can be taught to appreciate policy making through policy argument or practical reasoning and to understand that the relevance of and weight given to science depends on the policy context. They can recognize the limits of the persuasive power of scientific reasoning, the substantial institutional barriers and cultural resistance to new scientific knowledge, and the role of moral and ethical beliefs.

For a century or more the social sciences have contributed to policy making in many ways, especially in informing policy design and evaluation. We see a fresh way they can further contribute: by specifically focusing on whether, why, and how science is, or is not, used as evidence in public policy.

Suggested Citation:"Summary." National Research Council. 2012. Using Science as Evidence in Public Policy. Washington, DC: The National Academies Press. doi: 10.17226/13460.
×
Page 1
Suggested Citation:"Summary." National Research Council. 2012. Using Science as Evidence in Public Policy. Washington, DC: The National Academies Press. doi: 10.17226/13460.
×
Page 2
Suggested Citation:"Summary." National Research Council. 2012. Using Science as Evidence in Public Policy. Washington, DC: The National Academies Press. doi: 10.17226/13460.
×
Page 3
Suggested Citation:"Summary." National Research Council. 2012. Using Science as Evidence in Public Policy. Washington, DC: The National Academies Press. doi: 10.17226/13460.
×
Page 4
Suggested Citation:"Summary." National Research Council. 2012. Using Science as Evidence in Public Policy. Washington, DC: The National Academies Press. doi: 10.17226/13460.
×
Page 5
Suggested Citation:"Summary." National Research Council. 2012. Using Science as Evidence in Public Policy. Washington, DC: The National Academies Press. doi: 10.17226/13460.
×
Page 6
Next: 1 Introduction »
Using Science as Evidence in Public Policy Get This Book
×
Buy Paperback | $38.00 Buy Ebook | $30.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Using Science as Evidence in Public Policy encourages scientists to think differently about the use of scientific evidence in policy making. This report investigates why scientific evidence is important to policy making and argues that an extensive body of research on knowledge utilization has not led to any widely accepted explanation of what it means to use science in public policy. Using Science as Evidence in Public Policy identifies the gaps in our understanding and develops a framework for a new field of research to fill those gaps.

For social scientists in a number of specialized fields, whether established scholars or Ph.D. students, Using Science as Evidence in Public Policy shows how to bring their expertise to bear on the study of using science to inform public policy. More generally, this report will be of special interest to scientists who want to see their research used in policy making, offering guidance on what is required beyond producing quality research, beyond translating results into more understandable terms, and beyond brokering the results through intermediaries, such as think tanks, lobbyists, and advocacy groups. For administrators and faculty in public policy programs and schools, Using Science as Evidence in Public Policy identifies critical elements of instruction that will better equip graduates to promote the use of science in policy making.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!