Click for next page ( R2


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page R1
Emerging and Readily Available Technologies and National Security — A Framework for Addressing Ethical, Legal, and Societal Issues Jean-Lou Chameau, William F. Ballhaus, and Herbert S. Lin, Editors Committee on Ethical and Societal Implications of Advances in Militarily Significant Technologies That Are Rapidly Changing and Increasingly Globally Accessible Computer Science and Telecommunications Board Board on Life Sciences Committee on Science, Technology, and Law Center for Engineering, Ethics, and Society Advisory Group

OCR for page R1
THE NATIONAL ACADEMIES PRESS  500 Fifth Street, NW  Washington, DC 20001 NOTICE: The project that is the subject of this report was approved by the Gov- erning Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engi- neering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance. Support for this project was provided by the Defense Advanced Research Proj- ects Agency under Award Number HR0011-11-C-0038. Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the views of the organizations or agencies that provided support for the project. International Standard Book Number-13:  978-0-309-29334-1 International Standard Book Number-10:  0-309-29334-0 Library of Congress Control Number: 2013958004 This report is available from Computer Science and Telecommunications Board National Research Council 500 Fifth Street, NW Washington, DC 20001 Additional copies of this report are available from the National Academies Press, 500 Fifth Street, NW, Keck 360, Washington, DC 20001; (800) 624-6242 or (202) 334-3313; http://www.nap.edu. Copyright 2014 by the National Academy of Sciences. All rights reserved. Printed in the United States of America

OCR for page R1
The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal govern- ment on scientific and technical matters. Dr. Ralph J. Cicerone is president of the National Academy of Sciences. The National Academy of Engineering was established in 1964, under the char- ter of the National Academy of Sciences, as a parallel organization of outstand- ing engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. C. D. Mote, Jr., is president of the National Academy of Engineering. The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Harvey V. Fineberg is president of the Institute of Medicine. The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in pro- viding services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Ralph J. Cicerone and Dr. C. D. Mote, Jr., are chair and vice chair, respectively, of the National Research Council. www.national-academies.org

OCR for page R1
COMMITTEE ON ETHICAL AND SOCIETAL IMPLICATIONS OF ADVANCES IN MILITARILY SIGNIFICANT TECHNOLOGIES THAT ARE RAPIDLY CHANGING AND INCREASINGLY GLOBALLY ACCESSIBLE WILLIAM F. BALLHAUS, The Aerospace Corporation (retired), Co-Chair JEAN-LOU CHAMEAU, California Institute of Technology, Co-Chair MARCUS FELDMAN, Stanford University BRAN FERREN, Applied Minds BARUCH FISCHHOFF, Carnegie Mellon University MICHAEL GAZZANIGA, University of California, Santa Barbara HANK GREELY, Stanford University MICHAEL IMPERIALE, University of Michigan Medical School ROBERT H. LATIFF, University of Notre Dame JAMES MOOR, Dartmouth College JONATHAN MORENO, University of Pennsylvania JOEL MOSES, Massachusetts Institute of Technology KENNETH OYE, Massachusetts Institute of Technology ELIZABETH RINDSKOPF PARKER, University of the Pacific McGeorge School of Law SARAH SEWALL, Harvard University ALFRED SPECTOR, Google, Inc. JOHN H. TILELLI, JR., Cypress International, Inc. STEPHEN J.A. WARD, University of Oregon Staff HERBERT S. LIN, Study Director and Chief Scientist, Computer Science and Telecommunications Board (CSTB) JON EISENBERG, Director, CSTB ENITA WILLIAMS, Associate Program Officer, CSTB (through April 2013) SHENAE BRADLEY, Senior Program Assistant, CSTB ERIC WHITAKER, Senior Program Assistant, CSTB RACHELLE HOLLANDER, Director, Center for Engineering, Ethics, and Society FRAZIER BENYA, Program Officer, Center for Engineering, Ethics, and Society JO L. HUSBANDS, Senior Program Officer, Board on Life Sciences ANNE-MARIE MAZZA, Director, Committee on Science, Technology, and Law iv

OCR for page R1
COMPUTER SCIENCE AND TELECOMMUNICATIONS BOARD ROBERT F. SPROULL, University of Massachusetts, Amherst, Chair LUIZ ANDRÉ BARROSO, Google, Inc. ROBERT F. BRAMMER, Brammer Technology, LLC EDWARD FRANK, Apple, Inc. JACK L. GOLDSMITH III, Harvard Law School SEYMOUR E. GOODMAN, Georgia Institute of Technology LAURA HAAS, IBM Corporation MARK HOROWITZ, Stanford University MICHAEL KEARNS, University of Pennsylvania ROBERT KRAUT, Carnegie Mellon University SUSAN LANDAU, Google, Inc. PETER LEE, Microsoft Corp. DAVID LIDDLE, US Venture Partners BARBARA LISKOV, Massachusetts Institute of Technology JOHN STANKOVIC, University of Virginia JOHN SWAINSON, Dell, Inc. PETER SZOLOVITS, Massachusetts Institute of Technology ERNEST J. WILSON, University of Southern California KATHERINE YELICK, University of California, Berkeley JON EISENBERG, Director LYNETTE I. MILLETT, Associate Director and Senior Program Officer VIRGINIA BACON TALATI, Program Officer SHENAE BRADLEY, Senior Program Assistant RENEE HAWKINS, Financial and Administrative Manager HERBERT S. LIN, Chief Scientist, CSTB ENITA WILLIAMS, Associate Program Officer (through April 2013) ERIC WHITAKER, Senior Program Assistant For more information on CSTB, see its Web site at http://www.cstb.org, write to CSTB, National Research Council, 500 Fifth Street, NW, Washing- ton, DC 20001, call (202) 334-2605, or e-mail the CSTB at cstb@nas.edu. v

OCR for page R1
BOARD ON LIFE SCIENCES JO HANDELSMAN, Yale University, Chair ENRIQUETA C. BOND, Burroughs Wellcome Fund ROGER D. CONE, Vanderbilt University Medical Center SEAN EDDY, Howard Hughes Medical Institute SARAH C.R. ELGIN, Washington University DAVID R. FRANZ, Consultant, Frederick, Maryland LOUIS J. GROSS, University of Tennessee, Knoxville ELIZABETH HEITMAN, Vanderbilt University Medical Center JOHN G. HILDEBRAND, University of Arizona, Tucson RICHARD A. JOHNSON, Arnold & Porter, LLP JUDITH KIMBLE, University of Wisconsin, Madison CATO T. LAURENCIN, University of Connecticut ALAN I. LESHNER, American Association for the Advancement of Science KAREN NELSON, J. Craig Venter Institute ROBERT M. NEREM, Georgia Institute of Technology CAMILLE PARMESAN, University of Texas at Austin ALISON G. POWER, Cornell University MARGARET RILEY, University of Massachusetts JANIS WEEKS, University of Oregon MARY WOOLLEY, Research!America FRAN SHARPLES, Director SAYYEDA “AYESHA” AHMED, Senior Program Assistant CARL-GUSTAV ANDERSON, Program Associate BETHELHEM M. BANJAW, Financial Associate KATHERINE BOWMAN, Senior Program Officer INDIA HOOK-BARNARD, Program Officer JO L. HUSBANDS, Scholar/Senior Project Director ANGELA KOLESNIKOVA, Temporary Program Assistant JAY LABOV, Senior Scientist/Program Director for Biology Education KEEGAN SAWYER, Associate Program Officer MARILEE SHELTON-DAVENPORT, Senior Program Officer vi

OCR for page R1
COMMITTEE ON SCIENCE, TECHNOLOGY, AND LAW DAVID KORN, Massachusetts General Hospital, Co-Chair RICHARD A. MESERVE, Carnegie Institution for Science, Co-Chair BARBARA E. BIERER, Harvard Medical School ELIZABETH H. BLACKBURN, University of California, San Francisco JOHN BURRIS, Burroughs Wellcome Fund CLAUDE CANIZARES, Massachusetts Institute of Technology ARTURO CASADEVALL, Albert Einstein College of Medicine JOE S. CECIL, Federal Judicial Center ROCHELLE COOPER DREYFUSS, New York University School of Law DREW ENDY, Stanford University MARCUS FELDMAN, Stanford University JEREMY FOGEL, Federal Judicial Center ALICE P. GAST, Lehigh University BENJAMIN W. HEINEMAN, JR., Harvard Law School D. BROCK HORNBY, U.S. District Court, District of Maine WALLACE LOH, University of Maryland, College Park MARGARET MARSHALL (retired), Massachusetts Supreme Judicial Court ALAN B. MORRISON, George Washington University Law School CHERRY MURRAY, Harvard School of Engineering and Applied Sciences ROBERTA NESS, University of Texas School of Public Health HARRIET RABB, Rockefeller University DAVID RELMAN, Stanford University RICHARD REVESZ, New York University School of Law DAVID S. TATEL, U.S. Court of Appeals for the District of Columbia Circuit ANNE-MARIE MAZZA, Director STEVEN KENDALL, Associate Program Officer vii

OCR for page R1
CENTER FOR ENGINEERING, ETHICS, AND SOCIETY ADVISORY GROUP JOHN AHEARNE, Sigma Xi, The Scientific Research Society, Chair ALICE AGOGINO, University of California, Berkeley STEPHANIE J. BIRD, Ethics Consultant and Co-Editor of Science and Engineering Ethics GLEN DAIGGER, CH2M HILL GERALD E. GALLOWAY, JR., University of Maryland, College Park DEBORAH JOHNSON, University of Virginia WILLIAM KELLY, American Society for Engineering Education FELICE LEVINE, American Educational Research Association MICHAEL LOUI, University of Illinois at Urbana-Champaign DONNA RILEY, Smith College CHRIS SCHAIRBAUM, Texas Instruments, Inc. CAROLINE WHITBECK, Case Western Reserve University WILLIAM WULF, University of Virginia RACHELLE D. HOLLANDER, Center Director FRAZIER BENYA, Program Officer SIMIL RAGHAVAN, Associate Program Officer VIVIENNE CHIN, Administrative Assistant viii

OCR for page R1
Preface In 2010, the Defense Advanced Research Projects Agency (DARPA) asked the National Academies to develop and articulate a framework for policy makers, institutions, and individual researchers that would help them think through ethical, legal, and societal issues (ELSI) as they relate to research and development on emerging and readily available technologies with military relevance.1 The study was motivated in part by DARPA’s experience earlier in the previous decade with programs that encountered difficulties related to privacy concerns and the realiza- tion that a more systematic approach to ethical, legal, and societal issues was an important ingredient for success in its mission of avoiding and creating surprise through R&D. Box P.1 contains the full charge to the Committee on Ethical and Societal Implications of Advances in Militarily Significant Technologies That Are Rapidly Changing and Increasingly Globally Accessible. Coming from the Department of Defense (DOD), this concern—stated so explicitly—is relatively new. The DOD has long required a legal review of whether weapons are in conformance with the law of armed conflict, but this requirement applies only to weapons near procurement and 1 DARPA’s original charge to the committee used the term “democratized technologies” rather than “emerging and readily available technologies.” Democratized or, equivalently, emerging and readily available technologies are those with rapid rates of progress and low barriers to entry. However, the committee believed that the term “democratized” is easily misunderstood, and this report uses the term “emerging and readily available technologies” (ERA technologies). More discussion of this topic is contained in Chapters 1 and 3. ix

OCR for page R1
x PREFACE Box P.1 The Project Statement of Task The National Academies will develop a consensus report on the topic of ethi- cal, legal, and societal issues relating to research on, development, and use of increasingly globally accessible and rapidly changing technologies with potential military application, such as information technologies, synthetic biology, and nano- technology. This report will articulate a framework for policy makers, institutions, and individual researchers to think about such issues as they relate to these tech- nologies of military relevance and to the extent feasible make recommendations for how each of these groups should approach these considerations in their research activities. A workshop to be held as early as practical in the study will be convened to obtain perspectives and foster discussion on these matters. A final report will be issued within 21 months of the project start, providing the National Research Council’s and National Academy of Engineering’s findings and recommendations. not to R&D more generally. It is true that certain technologies—genome research, synthetic biology, and nanotechnology, for example—have in the eyes of the U.S. government warranted some degree of explicit attention to ethical, legal, and societal issues. In addition, there is a long history of academic work on ELSI concerns related to various civilian-oriented tech- nologies. But for the most part, these technologies have been exploited for civilian purposes, and work on ethical, legal, and societal issues has been confined largely to that context. ELSI concerns are inherently challenging, complex, and multidimen- sional, and their resolution often involves seeking common ground among individuals with deeply held but often unarticulated assumptions about ethics, culture, and epistemology. In some cases, finding common ground may be impossible to achieve in any reasonable time frame. Nevertheless, at the very least, ethical, legal, and societal issues are important enough to deserve serious exploration and attention, even if such common ground cannot be found, and in the committee’s view, DARPA deserves great credit for being willing to raise such issues. How ELSI expertise and scholarship developed in the context of civilian-oriented science and technology can be applied to the military context is a central theme of this report. But the lessons offered from that expertise and scholarship will require some modification for and adapta- tion to the military context—that is, they cannot be adopted wholesale, given that the military context does have a number of unique attributes. Skeptics of the Department of Defense’s attention to ELSI concerns

OCR for page R1
PREFACE xi may well claim that any attempt to argue for uniqueness and processes different from those used for civilian-oriented research is tantamount to shoving hard issues under the table while maintaining a veneer of concern, but the committee does not share this point of view. That is, the committee recognizes the existence of real tensions between military mis- sions (and the technology for supporting those missions) and traditional ELSI concerns. These tensions cannot be eliminated, but it is the com- mittee’s hope that this report can help senior leadership and program managers of agencies that support R&D for military and other national security purposes—including but not limited to DARPA—do a better job of managing these tensions. In addition, the report may also be of value to individual researchers, whether in the defense community or not, who work on the technologies discussed in this report and who may also be interested in the ELSI dimensions of their work. The committee assembled for this project included individuals with expertise in risk analysis, perception, and communication; ethics; human rights; military operations; military acquisitions; national security law; organizational behavior; media/communications; bioethics; biomedical sciences; and information technology. The committee first met in August 2011 and five times subsequently. Its earlier meetings were devoted primarily to workshops and plenary sessions for gathering input from a broad range of experts on a variety of topics related to ethical, legal, and societal issues associated with tech- nology of different kinds used in different contexts; later meetings were devoted primarily to committee deliberations. (See Appendix A for brief biographies of committee members and staff and Appendix B for the agendas for the committee’s information-gathering sessions.) The com- mittee heard presentations related to military ethics and law, emerging contexts for military operations, future military missions and technolo- gies for use in these missions, biomedical ethics and engineering ethics, risk assessment and communication, emerging technologies and ELSI concerns, mechanisms used by government agencies to address ethical, legal, and societal issues, approaches to embedding ethics in research and development, and non-U.S. perspectives on ethics in science and technol- ogy. In addition, the committee received input on specific emerging and readily available technologies, including information technology, neuro- science, prosthetics and human enhancement, synthetic biology, cyber weapons, robotics and automated weapons, and nonlethal weapons. Additional input included perspectives from professional conferences, the extant literature regarding ELSI concerns and science and technology, and government reports studied by committee members and staff.

OCR for page R1
xii PREFACE ACKNOWLEDGMENTS The complexity of the issues explored in this report meant that the committee had much to learn from its briefers. The committee is grateful to many parties for presentations on the following dates: • August 30-31, 2011. Shannon French (Case Western Reserve Univer- sity), Ward Thomas (College of the Holy Cross), Judith Miller (formerly of the Department of Defense), Peter Schwartz (Global Business Network), Scott Wallace (U.S. Army (ret.)), George Lucas (U.S. Naval Academy), Patrick Lin (California Polytechnic State University), R. Alta Charo (Uni- versity of Wisconsin Law School), and Joseph Herkert (Arizona State University). • November 2-3, 2011. Peter Lee (Microsoft Research), Keith Miller (University of Illinois, Springfield), Gloria Mark (University of ­ alifornia, C Irvine), Simson Garfinkel (Naval Postgraduate School), Scott Grafton (Uni- versity of California, Santa Barbara), Craig Stark (University of California, Irvine), Martha Farah (University of Pennsylvania), Stuart Harshbarger (Contineo Robotics), Daniel Palanker (Stanford University), Gerald Loeb (University of Southern California), Nicholas Agar (Victoria University of Wellington, New Zealand), James Hughes (Trinity College), George Church (Harvard University), Drew Endy (Stanford University), Nita A. Farahany (Vanderbilt University), Judith Reppy (Cornell University), and George Khushf (University of South Carolina). • January 12-13, 2012. Deborah Johnson (University of Virginia), Sheila Jasanoff (Harvard University, Kennedy School of Government), David Rejeski (Woodrow Wilson Center), Malcolm Dando (University of Bradford, United Kingdom), Kelly Moore (National Science Founda- tion), Jean McEwen (National Human Genome Research Institute), Valery Gordon (National Institutes of Health), Fred Cate (Indiana University School of Law), Ray Colladay (DARPA (ret.)), Mark Seiden (Yahoo!), Randall Dipert (University of Buffalo), Neil Rowe (Naval Postgraduate School), Ron Arkin (Georgia Institute of Technology), Peter Singer (Brook- ings Institution), Jürgen Altmann (Technische Universität Dortmund, Ger- many), Denise Caruso (Carnegie Mellon University), and Peter Hancock (University of Central Florida). • April 12-13, 2012. Heather Douglas (University of Waterloo, Can- ada), Alex John London (Carnegie Mellon University), Nils-Eric Sahlin (Lund University, Sweden), Paul Fischbeck (Carnegie Mellon University), Wandi de Bruin (Carnegie Mellon University), Arthur Lupia (University of Michigan), Adam Finkel (Carnegie Mellon University), William Brinkman (U.S. Department of Energy, Office of Science), Carmen Maher (U.S. Food and Drug Administration, Office of the Chief Scientist), Edward Knipling (U.S. Department of Agriculture), Diana Hoyt (National Aeronautics and

OCR for page R1
PREFACE xiii Space Administration), Qiu Renzong (Chinese Academy of Social Sci- ence, China), Frans Brom (Utrecht University, The Netherlands), Steven Lee (Hobart and William Smith Colleges), and Montgomery McFate (U.S. Naval War College). • June 4, 2012. George Perkovich (Carnegie Endowment for Inter- national Peace), David Fidler (Indiana University), and Neil Davison (International Committee of the Red Cross). The committee also appreciates the support of Norman Whitaker from the Defense Advanced Research Projects Agency in the conduct of this project. In addition, the committee acknowledges the intellectual contributions of NRC and NAE staff: Herbert S. Lin (study director and chief scientist of the Computer Science and Telecommunications Board (CSTB)), Jon Eisenberg (director, CSTB), Enita Williams (associate program officer, CSTB), Rachelle Hollander (director, Center on Engineering Eth- ics), Frazier Benya (program officer, Center for Engineering, Ethics, and Society), Anne-Marie Mazza (director, Committee on Science, Technology, and Law), and Jo Husbands (senior program officer, Board on Life Sci- ences). Shenae Bradley and Eric Whitaker (both senior program assistants for CSTB) provided administrative support. Special thanks are also due to Patricia Wrightson (associate director of the Board on Global Science and Technology), who contributed time and expertise as a staff consultant. Jean-Lou Chameau, Co-Chair William F. Ballhaus, Co-Chair Committee on Ethical and Societal Implications of Advances in Militarily Significant Technologies That Are Rapidly Changing and Increasingly Globally Accessible

OCR for page R1

OCR for page R1
Acknowledgment of Reviewers This report has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise, in accordance with procedures approved by the National Research Council’s Report Review Committee. The purpose of this independent review is to provide candid and critical comments that will assist the institution in making its pub- lished report as sound as possible and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process. We wish to thank the following individuals for their review of this report: Carlos Betha, United States Air Force Academy, Kathleen Clark, Washington University School of Law, Nancy Connell, University of Medicine and Dentistry of New Jersey, David Fidler, Indiana University Maurer School of Law, Shannon French, Case Western Reserve University, Paul Gaffney, Monmouth University, Elizabeth Heitman, Vanderbilt University Medical Center, Deborah Johnson, University of Virginia, David Korn, Harvard University, Miltos Ladikas, University of Central Lancashire, Maria Lapinski, Michigan State University, Patrick Lin, California Polytechnic State University, Lester L. Lyles, United States Air Force (retired), xv

OCR for page R1
xvi ACKNOWLEDGMENT OF REVIEWERS Richard O’Meara, Rutgers University, David Relman, Veterans Administration Palo Alto Health Care System, Robert F. Sproull, Oracle (retired), Detlof von Winterfeldt, University of Southern California, and John Weckert, Charles Sturt University. Although the reviewers listed above have provided many construc- tive comments and suggestions, they were not asked to endorse the con- clusions or recommendations, nor did they see the final draft of the report before its release. The review of this report was overseen by Julia Phillips from Sandia National Laboratories and Kenneth Keller from the Johns Hopkins University Bologna Center. Appointed by the National Research Council, they were responsible for making certain that an independent examination of this report was carried out in accordance with institu- tional procedures and that all review comments were carefully consid- ered. Responsibility for the final content of this report rests entirely with the authoring committee and the institution.

OCR for page R1
Contents SUMMARY 1 1 FRAMING THE ISSUES 15 1.1 National Security and the Role of Technology, 15 1.2 Ethical, Legal, and Societal Issues in Science and Technology, 17 1.3 ELSI Considerations for Science and Technology in a National Security Context, 23 1.4 Emerging and Readily Available Technologies of Military Significance, 28 1.5 Ethics of Armed Conflict, 34 1.6 What Is and Is Not Within the Scope of This Report, 35 1.7 A Roadmap to This Report, 43 2 FOUNDATIONAL TECHNOLOGIES 45 2.1 Information Technology, 46 2.1.1 Scientific and Technological Maturity, 46 2.1.2 Possible Military Applications, 49 2.1.3 Ethical, Legal, and Societal Questions and Implications, 52 2.2 Synthetic Biology, 57 2.2.1 Scientific and Technological Maturity, 58 2.2.2 Possible Military Applications, 60 xvii

OCR for page R1
xviii CONTENTS 2.2.3 Ethical, Legal, and Societal Questions and Implications, 61 2.3 Neuroscience, 65 2.3.1 Scientific and Technological Maturity, 67 2.3.2 Possible Military Applications, 68 2.3.3 Ethical, Legal, and Societal Questions and Implications, 74 3 APPLICATION DOMAINS 79 3.1 Robotics and Autonomous Systems, 79 3.1.1 Robotics—The Technology of Autonomous Systems, 80 3.1.2 Possible Military Applications, 82 3.1.3 Ethical, Legal, and Societal Questions and Implications, 83 3.2 Prosthetics and Human Enhancement, 92 3.2.1 The Science and Technology of Prosthetics and Human Enhancement, 92 3.2.2 Possible Military Applications, 93 3.2.3 Ethical, Legal, and Societal Questions and Implications, 94 3.3 Cyber Weapons, 97 3.3.1 The Technology of Cyber Weapons, 97 3.3.2 Possible Military Applications, 98 3.3.3 Ethical, Legal, and Societal Questions and Implications, 100 3.4 Nonlethal Weapons, 103 3.4.1 The Technology of Nonlethal Weapons, 104 3.4.2 Possible Applications, 105 3.4.3 Ethical, Legal, and Societal Questions and Implications, 106 4 SOURCES OF ELSI INSIGHT 115 4.1 Insights from Synthesizing Across Emerging and Readily Available Technologies, 115 4.2 Ethics, 118 4.2.1 Philosophical Ethics, 118 4.2.2 Disciplinary Approaches to Ethics, 120 4.3 International Law, 129 4.3.1 The Laws of War, 132 4.3.2 International Human Rights Law, 138 4.3.3 Arms Control Treaties, 140

OCR for page R1
CONTENTS xix 4.4 Social and Behavioral Sciences, 142 4.4.1 Sociology and Anthropology, 143 4.4.2 Psychology, 147 4.5 Scientific and Technological Framing, 153 4.6 The Precautionary Principle and Cost-Benefit Analysis, 154 4.7 Risk Communication, 157 4.8 Using Sources of ELSI Insight, 161 5 AN ANALYTICAL FRAMEWORK FOR IDENTIFYING 163 ETHICAL, LEGAL, AND SOCIETAL ISSUES 5.1 Stakeholders, 164 5.1.1 Those Involved in or Connected to the Conduct of Research, 165 5.1.2 Users of an Application, 168 5.1.3 Adversaries, 168 5.1.4 Nonmilitary Users, 171 5.1.5 Organizations, 173 5.1.6 Noncombatants, 174 5.1.7 Other Nations, 175 5.2 Crosscutting Themes, 175 5.2.1 Scale, 175 5.2.2 Humanity, 177 5.2.3 Technological Imperfections, 179 5.2.4 Unanticipated Military Uses, 180 5.2.5 Crossovers to Civilian Use, 181 5.2.6 Changing Ethical Standards, 182 5.2.7 ELSI Considerations in a Classified Environment, 183 5.2.8 Opportunity Costs, 185 5.2.9 Sources of Insight from Chapter 4, 185 5.3 An Example of Using the Framework, 186 5.3.1 A Hypothetical Scenario for Analysis, 186 5.3.2  Process for Identifying Ethical, Legal, and Societal A Issues, 186 5.3.3 Questions Related to Stakeholders and Crosscutting Themes, 192 5.3.4 Developing a Future Course of Action, 198 5.4 The Framework in Context, 199 5.4.1 A Summary of the Framework’s Questions, 199 5.4.2 Utility of the Framework, 205 5.4.3 Identifying Fraught Technologies, 208 5.4.4 Frequently Heard Arguments, 209

OCR for page R1
xx CONTENTS 6 GOING BEYOND INITIAL A PRIORI ANALYSIS 212 6.1 Unanticipated Impacts, 212 6.2 Limits of A Priori Analysis, 213 6.2.1 The Limited Utility of Technology Forecasting, 213 6.2.2 Sources of Uncertainty in Technology Forecasting, 214 6.3 Broadening Predictive Analysis of Ethical, Legal, and Societal Issues, 219 6.3.1 Use of Deliberative Processes, 220 6.3.2 Anticipatory Governance, 226 6.3.3 Adaptive Planning, 227 7 MECHANISMS FOR ADDRESSING ETHICAL, LEGAL, 230 AND SOCIETAL ISSUES 7.1  Characterizing Possible Mechanisms for Addressing Ethical, Legal, and Societal Issues, 230 7.2  What Mechanisms Have Been Used to Address Ethical, Legal, and Societal Issues?, 233 7.2.1 Self-regulation and Self-awareness, 233 7.2.2 Established Institutional Mechanisms, 235 7.2.3 Existing DARPA Efforts to Manage ELSI Concerns, 237 7.3  Considerations for Mechanisms Used to Address Ethical, Legal, and Societal Issues in the Context of Military R&D, 240 8 FINDINGS AND RECOMMENDATIONS 245 8.1 Synthesis, 245 8.2 Findings, 246 8.3 Recommendations, 251 8.3.1 Recommendations for Agencies, 251 8.3.2 Recommendation for Research-Performing Institutions and Individual Researchers, 265 8.4 Concluding Observations, 266 APPENDIXES A Committee Members and Staff 271 B Meeting Agendas and Participants 283 C Research and Development Organizations Within the 298 Department of Defense D Established Institutional Mechanisms for Addressing Ethical, 306 Legal, and Societal Issues