National Academies Press: OpenBook
Page 1
Suggested Citation:"Understanding Biosecurity." National Research Council. 2010. Understanding Biosecurity: Protecting Against the Misuse of Science in Today's World. Washington, DC: The National Academies Press. doi: 10.17226/13341.
×
Page 1
Page 2
Suggested Citation:"Understanding Biosecurity." National Research Council. 2010. Understanding Biosecurity: Protecting Against the Misuse of Science in Today's World. Washington, DC: The National Academies Press. doi: 10.17226/13341.
×
Page 2
Page 3
Suggested Citation:"Understanding Biosecurity." National Research Council. 2010. Understanding Biosecurity: Protecting Against the Misuse of Science in Today's World. Washington, DC: The National Academies Press. doi: 10.17226/13341.
×
Page 3
Page 4
Suggested Citation:"Understanding Biosecurity." National Research Council. 2010. Understanding Biosecurity: Protecting Against the Misuse of Science in Today's World. Washington, DC: The National Academies Press. doi: 10.17226/13341.
×
Page 4
Page 5
Suggested Citation:"Understanding Biosecurity." National Research Council. 2010. Understanding Biosecurity: Protecting Against the Misuse of Science in Today's World. Washington, DC: The National Academies Press. doi: 10.17226/13341.
×
Page 5
Page 6
Suggested Citation:"Understanding Biosecurity." National Research Council. 2010. Understanding Biosecurity: Protecting Against the Misuse of Science in Today's World. Washington, DC: The National Academies Press. doi: 10.17226/13341.
×
Page 6
Page 7
Suggested Citation:"Understanding Biosecurity." National Research Council. 2010. Understanding Biosecurity: Protecting Against the Misuse of Science in Today's World. Washington, DC: The National Academies Press. doi: 10.17226/13341.
×
Page 7
Page 8
Suggested Citation:"Understanding Biosecurity." National Research Council. 2010. Understanding Biosecurity: Protecting Against the Misuse of Science in Today's World. Washington, DC: The National Academies Press. doi: 10.17226/13341.
×
Page 8
Page 9
Suggested Citation:"Understanding Biosecurity." National Research Council. 2010. Understanding Biosecurity: Protecting Against the Misuse of Science in Today's World. Washington, DC: The National Academies Press. doi: 10.17226/13341.
×
Page 9
Page 10
Suggested Citation:"Understanding Biosecurity." National Research Council. 2010. Understanding Biosecurity: Protecting Against the Misuse of Science in Today's World. Washington, DC: The National Academies Press. doi: 10.17226/13341.
×
Page 10
Page 11
Suggested Citation:"Understanding Biosecurity." National Research Council. 2010. Understanding Biosecurity: Protecting Against the Misuse of Science in Today's World. Washington, DC: The National Academies Press. doi: 10.17226/13341.
×
Page 11
Page 12
Suggested Citation:"Understanding Biosecurity." National Research Council. 2010. Understanding Biosecurity: Protecting Against the Misuse of Science in Today's World. Washington, DC: The National Academies Press. doi: 10.17226/13341.
×
Page 12
Page 13
Suggested Citation:"Understanding Biosecurity." National Research Council. 2010. Understanding Biosecurity: Protecting Against the Misuse of Science in Today's World. Washington, DC: The National Academies Press. doi: 10.17226/13341.
×
Page 13
Page 14
Suggested Citation:"Understanding Biosecurity." National Research Council. 2010. Understanding Biosecurity: Protecting Against the Misuse of Science in Today's World. Washington, DC: The National Academies Press. doi: 10.17226/13341.
×
Page 14
Page 15
Suggested Citation:"Understanding Biosecurity." National Research Council. 2010. Understanding Biosecurity: Protecting Against the Misuse of Science in Today's World. Washington, DC: The National Academies Press. doi: 10.17226/13341.
×
Page 15
Page 16
Suggested Citation:"Understanding Biosecurity." National Research Council. 2010. Understanding Biosecurity: Protecting Against the Misuse of Science in Today's World. Washington, DC: The National Academies Press. doi: 10.17226/13341.
×
Page 16
Page 17
Suggested Citation:"Understanding Biosecurity." National Research Council. 2010. Understanding Biosecurity: Protecting Against the Misuse of Science in Today's World. Washington, DC: The National Academies Press. doi: 10.17226/13341.
×
Page 17
Page 18
Suggested Citation:"Understanding Biosecurity." National Research Council. 2010. Understanding Biosecurity: Protecting Against the Misuse of Science in Today's World. Washington, DC: The National Academies Press. doi: 10.17226/13341.
×
Page 18
Page 19
Suggested Citation:"Understanding Biosecurity." National Research Council. 2010. Understanding Biosecurity: Protecting Against the Misuse of Science in Today's World. Washington, DC: The National Academies Press. doi: 10.17226/13341.
×
Page 19
Page 20
Suggested Citation:"Understanding Biosecurity." National Research Council. 2010. Understanding Biosecurity: Protecting Against the Misuse of Science in Today's World. Washington, DC: The National Academies Press. doi: 10.17226/13341.
×
Page 20
Page 21
Suggested Citation:"Understanding Biosecurity." National Research Council. 2010. Understanding Biosecurity: Protecting Against the Misuse of Science in Today's World. Washington, DC: The National Academies Press. doi: 10.17226/13341.
×
Page 21
Page 22
Suggested Citation:"Understanding Biosecurity." National Research Council. 2010. Understanding Biosecurity: Protecting Against the Misuse of Science in Today's World. Washington, DC: The National Academies Press. doi: 10.17226/13341.
×
Page 22
Page 23
Suggested Citation:"Understanding Biosecurity." National Research Council. 2010. Understanding Biosecurity: Protecting Against the Misuse of Science in Today's World. Washington, DC: The National Academies Press. doi: 10.17226/13341.
×
Page 23
Page 24
Suggested Citation:"Understanding Biosecurity." National Research Council. 2010. Understanding Biosecurity: Protecting Against the Misuse of Science in Today's World. Washington, DC: The National Academies Press. doi: 10.17226/13341.
×
Page 24

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

UNDERSTANDING BIOSECURITY Protecting Against the Misuse of Science in Today’s World

Foreword  1 Understanding Biosecurity  2 The Biosecurity Challenge  4 New Science Seeks to Conquer “Old” Diseases  4 Emerging Science Makes Security a Moving Target  4 Addressing the Challenge  7 The Role of Scientists  7 The Role of Research Institutions  10 The Role of Journal Editors  10 The Role of Professional Societies  11 The Role of the Federal Government  13 The Role of International Coordination  15 Doing Science Responsibly  17 NOTES  18 WEB RESOURCES For Further INFORMATION  18

Foreword As a scientist who has worked for over 40 years to find cures for infectious disease, I find the idea that terrorists would use biological agents as a weapon to be anathema. It violates the fundamental values of the life sciences that I and my colleagues hold dear: that science is a vital tool for improving life and the health of our planet and enhancing our understanding of the natural world. My own work has focused on cholera, a disease responsible for the death of thousands of people around the world every year. During the past 40 years, research carried out through international collaboration of scientists has saved many thousands of lives. Research with biological toxins and pathogens is both necessary and important, and our nation’s health and security depend upon our understanding their mechanisms of virulence. At the same time, we are firm in the belief that this research should be conducted safely and responsibly. To date, the incidence of either laboratory workers or members of the public being infected, whether from laboratory accidents or intentional action, is extremely small. Nonetheless, through the years, safety and security practices and procedures have been developed that have successfully prevented accidental or intentional misuse of biological materials. Scientists have not only demonstrated concern about these issues, they also recognize they have the most at stake should an incident occur. They are best able to identify potential risk, whether from a laboratory door left unsecured or the unusual behavior of a laboratory worker. It is for these reasons that we focus on promoting a culture of responsibility, enabling and empowering scientists to be vigilant stewards of their science. Care must be taken to avoid being overzealous and implementing procedures that don’t make life sciences research more secure, only more difficult to conduct—with results that may diminish rather than strengthen security. Rita R. Colwell Chair, National Academy of Sciences Committee on Laboratory Security and Personnel Reliability Assurance Systems for Laboratories Conducting Research on Biological Select Agents and Toxins; University of Maryland, College Park, and Johns Hopkins University Bloomberg School of Public Health, Baltimore; and President and Chief Executive Officer, CosmosID, Inc., Bethesda, Maryland. Adapted from Responsible Research with Biological Select Agents and Toxins (NRC, 2009)

UNDERSTANDING BIOSECURITY Protecting Against the Misuse of Science in Today’s World T he life sciences offer move the life sciences forward for 2001 shortly after the 9/11 terrorist tremendous promise for legitimate purposes while reducing attacks on the World Trade Center meeting many 21st-century the risks that some materials, towers and the Pentagon, a series challenges. Powerful new tools knowledge, tools, and technologies of letters containing lethal anthrax allow the collection and analysis of could also be used to do harm. In powder were sent through the mail vast amounts of information about biological systems—from single cells to global cycles—invigorating the life sciences community and spurring innovation in numerous The Scientific Enterprise Is Built  areas. Building on the knowledge on a Foundation of Trust... and experience of generations of their predecessors, life scientists . . . for example, that researchers adhere to professional standards of science. are developing information and Science is really a partnership between scientists and the greater society. technologies to improve human Scientists rely on society for support of their work, whether from taxpayer health, agriculture, energy, the funding of research or from public policy and regulation that encourage and environment, and applications in sustain research. The public expects that scientific results—for example, the many other areas. Whether they vaccines that eliminated smallpox and have nearly conquered polio—will deliver improvements in health and the quality of life. Because of that are motivated to enhance the partnership—and their own commitment to advancing science for the public quality of life, inspired by the spark good—scientists must conduct research ethically and responsibly. of discovery and innovation, or driven by the essential quest for A critical cornerstone of the modern scientific enterprise is the essential deeper knowledge of our world, exchange of information that enables scientists to replicate, verify, and build life scientists today face remarkable on the results of fruitful lines of research. This fundamental principle and opportunities. practice also depends on public support. Open access allows scientists to evaluate, interpret, adapt, and extend results from many fields of inquiry for But with opportunities come use in their own work. It can speed the delivery of life-saving knowledge to responsibilities. An important medical practice. Yet open access may allow both beneficent and malignant aspect of scientists’ responsibility to uses of scientific information. As a result, scientists are called on to keep society is captured in the concept watch against the potential for misuse of their work. Policy makers also need of biosecurity—the challenge to to pursue security goals without restraining the ability of science to advance and deliver on its promises to society.

National Academy of Engineering, Institute of Medicine, and National Research Council—to introduce some of the issues at the intersection of science and security. Because this booklet is limited to the scope of the National Academies’ work, it is not a guide to the outstanding efforts of many other organizations and institutions in the United States and overseas, although some additional resources are provided at the end of the booklet for further reference. This booklet was developed by the Academies to serve as an educational resource for students and the scientific community, illuminate the importance of biosecurity, and explore how scientists, organizations, and governments at many levels can work together to minimize the threat.  to New York City, Washington, Biologists working in a lab designed to handle pathogenic materials. D.C., and other locations. Those SOURCE: NIAID/NIH. letters, which resulted in five deaths and put thousands of people at community, policy makers, and the risk, represent one example of the public need to be educated about “The standards of potential misuse of biology. Recent the risks and their responsibilities science extend beyond scientific progress, combined with to help mitigate them. Research responsibilities that are the globalized nature of today’s institutions, scientific journals, society, has in some ways expanded professional societies, governments, internal to the scientific vulnerabilities to such misuse and and international bodies all have a community. Researchers the potential magnitude of effects. role in addressing the biosecurity also have a responsibility challenge. It is particularly important to reflect on how their As scientific research evolves, that today’s science students—the so does its biosecurity context. next generation of life scientists—are work and the knowledge Policy, governance, and oversight aware of the biosecurity context of they are generating frameworks that affect research in their work. might be used in the the life sciences are also in a state of change. To inform decision This booklet draws from the work broader society.” making and support a productive of the National Academies—the –On Being a Scientist (NRC, dialogue, life scientists, the security National Academy of Sciences, 2009)

The Biosecurity Challenge The benefits of biotechnology and advancing our understanding of benefits must also be weighed its applications in biomedicine, health and disease. The research against potential risks. Genetic agriculture, and other areas have was designed to illuminate the research could propel rapid progress been great. Two scientific inquiries factors that made that strain—and against deadly infectious diseases, as illustrate the rapid progress toward possibly related strains—so lethal. new information is discovered and new and beneficial knowledge that Once the virus’s genetic sequence as access to that information fosters biotechnology offers. They also was published, however, it became additional innovative research. highlight the unexpected biosecurity freely accessible to everyone, challenges new knowledge may including terrorists and developers Emerging Science Makes bring. of biological weapons. Some Security a Moving Target members of the sequencing team Historically, biological weapons New Science Seeks to then went on to reconstruct the generally utilized naturally occurring Conquer “Old” Diseases virus, an even more controversial viruses and bacteria. Advances in The virulent strain of influenza that step. For some, the existence of the biotechnology, however, such as the engulfed the globe in 1918-1919, deadly virus in laboratories raised advent of genetic engineering, have known as the “Spanish flu,” is concerns about the possibility of raised concerns that developing estimated to have killed between its accidental introduction into the “designer diseases” and pathogens  20 million and 50 million people in human population. The actual risks one of the most devastating health posed by the reconstruction of the Emergency hospitals were quickly disasters in history. The 1918 flu 1918 influenza virus are unclear— set up to handle the huge influx of virus has been the focus of a great many factors combine to determine patients during the 1918 “Spanish” deal of research over the years, influenza outbreak. Scientists have whether an infectious agent causes reconstructed the virus that caused and in 2005 a group of researchers a pandemic infection—and potential the pandemic. broke new ground in the quest to understand the factors that contributed to the 1918 pandemic. The group did painstaking work to collect samples of genetic material from the 1918 flu strain, traveling to remote reaches of Alaska to exhume the bodies of flu victims buried in permafrost. The team used modern molecular tools to decode the virulent strain. Many consider this research to be a significant achievement for

with increased weapons utility is becoming more feasible. An unintended outcome of research by a team of Australian scientists drew attention to this potential in the late 1990s. In an effort to fight out-of-control mouse infestations in Australia, the researchers bioengineered a strain of Ectromelia virus (mousepox) that would render infected mice permanently infertile. But the genetic manipulation they used—incorporating the immunomodulatory cytokine IL-4 into the mousepox genome— mice that were completely resistant Scientist loading DNA into a gel. unexpectedly turned the virus SOURCE: Maggie Bartlett , NHGRI. to the parent virus. into an extremely effective mouse killer. The altered virus proved to Publication of the experiment’s be so virulent that 60 percent of results in 2001 sparked concern and overcome such engineered viruses. infected mice died within weeks. debate throughout the scientific The authors were sensitive to the Surprisingly, the researchers found and political spheres. Some saw biosecurity issues involved in their that the IL-4 gene-expressing  the paper as dangerous because work and had consulted with their mousepox killed even vaccinated it illustrated how to construct an peers about whether the results IL-4–expressing orthopox virus, should be submitted for publication. potentially providing a “roadmap” When the authors did submit the for sophisticated bioterrorists to paper to the Journal of Virology, engineer a more virulent strain reviewers and editors at the journal “It is reasonable of smallpox that could affect did not express concern about to anticipate that even vaccinated individuals. possible misuse of information in Others supported the research the manuscript, and the article humans are capable of and its publication, pointing to was accepted for publication. After engineering infectious its significant contributions to publication, concerns were raised agents with virulence knowledge of health and disease. about biosecurity. The editor in chief Some argued that it was important conducted a retrospective review equal to or perhaps far to recognize the possibility that and concluded that the journal was worse than any observed even vaccinated individuals may correct in its decision to publish. naturally.” be killed by a virus that has been Ultimately, the controversy led to spontaneously or intentionally termination of the research because –Globalization, Biosecurity, altered. Knowledge of these of concerns about its dual use and the Future of the Life experiments allows the scientific potential. Sciences (NRC, 2006) community to explore how to

The Biosecurity Challenge: Examples of Research with Dual Use Potential A concept related to biosecurity—dual use—refers to the fact that research intended for legitimate purposes may also have a potential to be misused in the development of bioweapons. The debates sparked by the publication of data related to the reconstruction of the 1918 “Spanish” influenza virus1 (as explained on page 4) illustrated how scientific achievements may also generate security concerns. Additional recent research endeavors that have been identified as having the potential for misuse include: The technique for modifying tobacco (shown above) and other plants, such as lettuce, to express the cholera • Synthesis of infectious poliovirus. Researchers sought 2 vaccine offers a likely prospect for quick and efficient to resolve the unusual nature of poliovirus, which production of the vaccine. However, some critics suggest behaves as both a chemical and a “living” entity. that it may also have a potential for misuse, illustrating the central conflict involved in biosecurity concerns. They succeeded in recreating the virus by chemically The tobacco seeds on the left have been successfully synthesizing a cDNA of its genome. Some critics engineered, or “transformed,” to express cholera assert that the publication of their methods vaccine; those on the right are untransformed. provided a recipe for terrorists by showing how SOURCE: Professor Henry Daniell, College of Medicine, University of Central Florida. one could create any virus from chemical reagents purchasable on the open market. The researchers  acknowledged this potential but noted that a threat of bioterrorism arises only if mass vaccinations to produce large quantities of cholera toxin cheaply against polio end. and relatively easily, paving the way for fast and efficient vaccine production. Concerns have arisen • Development of “stealth” viruses that could evade that it might also have a potential for misuse. the human immune system. These viruses are being developed to serve as molecular means • Development of new technologies for delivering drugs by for introducing curative genes into patients with aerosol spray in individual doses. Some have expressed inherited diseases. However, the research has raised concern that this development, intended to improve questions about whether they could potentially the ease of use and rate of compliance among be induced to express dangerous proteins, such as diabetic users of insulin, could be adapted to allow toxins.3 aerosol sprays to cover wider areas in an attack.6 • A method for the construction of “fusion toxins” derived Nonlaboratory research may also lend itself to possible from two distinct nontoxic chemical predecessors.4 misuse. Investigation of the potential effects of a This technique was originally investigated for the deliberate release of botulinum toxin into the U.S. purpose of killing cancer cells, but some argue that milk supply recommended aggressive pursuit of early it might be redirected to develop novel toxins that detection measures and new research on means could target the normal cells of almost any tissue to inactivate the toxin. Publication of the studies when introduced into a human host. pinpointed weaknesses in the system that critics argue • Genetic engineering of the tobacco plant to produce could help direct a terrorist to the most vulnerable subunits of cholera toxin.5 Because tobacco is easy points in the milk supply.7 to engineer, it is a likely candidate for producing plant-based vaccines. The technique could be used

ADDRESSING THE Challenge Scientists, security professionals, and contributing to the advancement and professionals, and looks to a society as a whole support adopting of biowarfare or bioterrorism. nationally convened advisory group active approaches to biosecurity Individuals cannot be expected for recommendations on oversight challenges, without unnecessarily to ensure that the knowledge strategies and guidelines. The hindering scientific innovation. Vital, they generate will never be used National Research Council report practical steps to improve oversight for malevolent purposes, but the A Survey of Attitudes and Actions on of research and provide education report concludes that scientists can Dual Use Research in the Life Sciences: can give people the tools to conduct and should take steps to minimize A Collaborative Effort of the National science responsibly. Reports and this possibility. It is each scientist’s Research Council and the American activities from many sources have responsibility to consider the dual Association for the Advancement examined current regulations and use potential of his or her own work, of Science (2009) found a basis of practices and recommended ways make informed decisions on whether support among U.S. life scientists to improve approaches to oversight, or how to proceed, and conduct for measures that could lead to a governance, and education for the work using the principles of system of self-governance on dual biosecurity. responsible scientific practice. use research. The Role of Scientists The report proposes a “cradle-to- To help frame these cradle-to-grave  In some ways, scientists are the grave” system in which reviews of reviews, Biotechnology Research in an “front line” of defense against experiments at various stages in the Age of Terrorism (2004) identifies an misuse of biological research. research life cycle—from proposal initial seven classes of experiments Working together, scientists can and grant evaluation to publication that raise concerns about their develop effective guidelines and and communication—would identify potential for misuse. Proposed standards that deter misuse without and assess potential biosecurity research in these categories should inhibiting the exploration of new risks. The report recommends prompt thoughtful consideration and important lines of research. Such that this system involve a mix of of whether such experiments guidelines have become a model for voluntary self-governance by the should be conducted or their full dealing with other potential threats. scientific community and expansion results published. “Experiments of They can also help decision makers of the existing regulatory process, Concern” include those that would: who focus on security concerns in part based on the approaches • demonstrate how to render a understand both the potential to oversight of research using vaccine ineffective; biosecurity implications and the recombinant DNA. That model benefits of new developments. of self-governance engages • provide pathogens with the personal responsibility and resistance to therapeutically As noted in the National Research accountability of the researcher, useful antibiotics or antivirals; Council report Biotechnology Research establishes local oversight by the • enhance the virulence of in an Age of Terrorism (2004), research institution through a a pathogen or render a biologists have a moral duty to avoid committee of peer researchers nonpathogen virulent;

• The Biosecurity Challenge: Using Disease as a Weapon Human beings have been using biological materials to cause harm for centuries. In 600 B.C.E. the Athenian leader Solon poisoned the water supply in the city of Kirrha with the noxious roots of the Helleborus plant—a primitive but effective biological toxin. In 1763 during continued conflict after the French and Indian War in North America, the commander of besieged British troops at Fort Pitt infected Native Americans with smallpox by giving them blankets used by sufferers of the disease. But the entire concept of bioweapons changed in the late 19th and early 20th centuries when biologists developed techniques to identify, isolate, and culture pathogens under precisely controlled conditions. This ability spawned both weapons A government scientist tests samples for anthrax programs and continuing international efforts to following the letter attacks in 2001. prevent the use of bioweapons. SOURCE: U.S. Army Medical Research Institute of Infectious Diseases. As many as two dozen nations have engaged in  bioweapons research. All the major combatants in The fundamental international agreements not to use World War II had bioweapons research programs. disease as a weapon are the Geneva Protocol, signed in Only Japan actually used a biological weapon, in the 1925, which bans first use of biological weapons, and Sino-Japanese war that extended into World War II, the Biological and Toxin Weapons Convention (BWC), by air-dropping plague-infested fleas in parts of China signed in 1972. The BWC was the first disarmament to create epidemics. The United States established an agreement to ban an entire class of weapons. Building offensive bioweapons program during World War II to on those accords, United Nations Security Council deter the use of bioweapons and to retaliate against Resolution 1540 (2004) adds a further binding an attack if necessary. Because bioweapons, if used, international commitment. It requires that all U.N. could spread to military and civilians alike and affect member states adopt measures to prevent terrorist both enemies and allies, President Richard Nixon shut groups, clandestine procurement networks, and other down the United States’ offensive bioweapons program nonstate actors from acquiring biological weapons or in 1969. Bioweapons stockpiles were destroyed and the means of their delivery. facilities were converted to other purposes, including research on defenses. Unfortunately, some countries have not honored B. anthracis spores, their international commitments. Despite having taken from the letter sponsored the 1972 BWC, for example, the Soviet sent to the New York Union continued to carry out clandestine bioweapons Post and postmarked programs. Russia continued to stockpile anthrax, September 18, 2001, are prepared for smallpox, plague, and other pathogens until 1992, analysis. when Russian President Boris Yeltsin banned further SOURCE: U.S. Department bioweapons activities. of Justice.

• increase transmissibility of a often be detected by watching for “Biological scientists have pathogen; warning signs before they actually an affirmative moral duty commit a malevolent or violent act. • alter the host range of a to avoid contributing Active and sustained monitoring pathogen; and management by scientists could to the advancement • enable the evasion of diagnosis or recognize many of these signals, of biowarfare or detection; providing the basis for prevention. bioterrorism.” • enable the weaponization of a To minimize potential security biological agent or toxin. and safety risks, the report urges –Biotechnology Research in an implementation of programs and Age of Terrorism (NRC, 2004) Following a recommendation in practices to develop and support a the report, the U.S. government culture of trust and responsibility, established the National Science Advisory Board for Biosecurity, which developed additional criteria to identify experiments that could be considered “dual use research of Scientific Self-Governance and the concern.”8 (For more information on the advisory board, see page 13.) Asilomar Conference A common message from many recent National Research Council reports The accelerating pace of research is that the scientific community should take preemptive steps to protect progress is expanding the concept the integrity of science and minimize its risks. A 1975 conference and of biosecurity risks in the life sciences  the guidelines resulting from it provide an example of how scientists beyond pathogens, according to the responded to one case where a promising technology also appeared to National Research Council report pose potential risks. Globalization, Biosecurity, and the Future of the Life Sciences (2006). The 1975 Asilomar conference—named for the California conference Rapid international diffusion of center in which it was held—became a landmark example of the scientific results produced by the life sciences community’s ability to lead the way in developing new technologies and the relevance to the life sciences responsibly. The focus of the conference was recombinant DNA, then a novel technology of unexplored potential and uncharacterized risks. of research in other disciplines pose These discussions inspired the development of guidelines, issued the further challenges. following year by the National Institutes of The National Research Council Health, to prevent the report Responsible Research with unintended creation of Biological Select Agents and Toxins harmful organisms in (2009) affirms that misuse of work with recombinant biological materials is taboo in every DNA. The influence of corner of the scientific community. the Asilomar conference But disastrous results could follow has continued to inspire actions by only a few individuals responsible conduct and ignoring that taboo. As stated in self-governance among the report, such individuals can scientists to this day. SOURCE: From the Dr. Donald S. Fredrickson Papers, NLM.

within which potential personnel issues can be identified and addressed. The Role of Research Institutions (Private Laboratories, Universities, and Others) Institutions that house biological research facilities—such as universities, government research campuses, and the laboratories of private companies—have a responsibility to protect the safety and security of their workers, those living in the surrounding environment, and the public at large. These institutions must be aware of and comply with rules SOURCE: NIH/NIEHS (Steve McCaw, Image Associates). governing research activities and be prepared to actively educate Research institutions can also play security community to participate in their employees and students an important role in facilitating fellowships at universities. 10 about relevant requirements and useful exchanges among scientists responsible practices for safety and and others. For example, The Role of Journal Editors security. constructive dialogue between Peer-reviewed journals serve scientists and members of the as a hub for the exchange of The research institutions in which national security community is scientific information. The editors most scientists conduct their work needed to create a system that of these journals, in turn, serve have established formal oversight is responsive to the risks but as gatekeepers that determine mechanisms, some of which stem also credible to and feasible for what should—or should not—be from specific legal obligations. researchers. To help address this published. Recognizing that Bodies such as Institutional Biosafety gap, the National Research Council publishing information with dual Committees, Institutional Review report Science and Security in a Post use potential carries risks, journal Boards, and Institutional Animal 9/11 World (2007) recommends editors have taken steps to mitigate Care and Use Committees may, to that universities work closely the risk that information they some extent, oversee research that with federal agencies to develop publish will be misused. In 2003 also involves biosecurity. Depending opportunities for scientists to following a workshop held by the on the institution’s structure, such participate in policy fellowships at National Academies and the Center committees review research at intelligence and national security for Strategic and International many stages, such as assessing agencies and, conversely, to Studies, a group of scientists, grant applications and reviewing develop opportunities for members journal editors, and security experts experimental approaches. of the intelligence and national drafted a “Statement on Scientific

“It is an unfortunate reality that almost all advances in life sciences technology pose potential ‘dual use’ risks. But better science is the best protection against potential threats.” –Globalization, Biosecurity, and the Future of the Life Sciences (NRC, 2006) are explored in detail in the report A Survey of Attitudes and Actions on Dual Use Research in the Life Sciences Recognizing that publishing information with dual use potential carries (2009). risks, journal editors have taken steps to mitigate that risk—for example, screening submitted articles for potential dual use implications. In addition, policies in the United SOURCES: Service, R.F. 2006. Biosecurity: synthetic biologists debate policing themselves, States must be considered in their 11 Science 312:1116, reprinted with permission from AAAS. Davidson, E.M. and R. Cook- Deegan, 2007. Science and security: practical experiences in dual-use review, Science international context. Journals in 316:1432-1433, reprinted with permission from AAAS. Cover. 2007. Proceedings of the the United States publish papers National Academy of Sciences U. S. A. 104(12). Cover. 2008. Nature 454(7201). from scientists in other countries, and thousands of journals publish life science research outside of U.S. Publication and Security.” In the journals have now adopted formal borders. Biotechnology Research in an statement, editors of leading policies on biosecurity. Age of Terrorism (2004) identifies the journals in the life sciences accepted Scientific journals are not the need for cooperation among journal responsibility for screening only place where science is editors to develop international manuscripts to reduce the risk of communicated. All scientists are guidelines for the publication of misuse of scientific information, responsible for monitoring their manuscripts containing potentially indicating that manuscripts would communication to maximize the sensitive information. be rejected if “the potential harm of publication outweighs the potential benefits and minimize the risks of The Role of Professional societal benefits.” The statement their research. Biosecurity needs Societies was simultaneously published in change rapidly as science develops, and much depends on both the Scientific professional societies Science, Nature, and Proceedings of scientists who conduct research and provide crucial venues for scientists the National Academy of Sciences the editors who publish it. Their to discuss current issues and the (PNAS) and by the American Society roles and attitudes on biosecurity evolving contexts of their work. for Microbiology. These and other These organizations also furnish

networks through which information legal issues” related to dual use biodefense and emerging disease can be shared among scientists research to American and foreign research.10 working in academia, government, scientists in the United States. • The Federation of American and industry. As such, professional Specific proposals emphasized Societies for Experimental Biology societies are uniquely qualified to establishing federal funding (FASEB) has issued a statement help raise awareness of biosecurity for such programs, developing on biosecurity education, calling concerns among scientists and guidance for scientists, and for integrating dual use education provide guidance to limit the risks. creating educational materials for into the training scientists receive scientists and nonscientists on the in the responsible conduct of Biotechnology Research in an Age dual use dilemma.9 research.11 of Terrorism (2004) called on • The American Society for professional societies to create Microbiology (ASM) has called The idea to incorporate dual use educational programs about the for biosecurity awareness to be a issues into professional codes potential risks of dual use research. component of formal training for of conduct has also gathered In addition, Seeking Security: microbiologists, just as biosafety considerable attention in recent Pathogens, Open Access, and Genome practices have been. The society years. The InterAcademy Panel on Databases (2004) calls for codes holds annual conferences on International Issues is a network of conduct for scientists to protect of more than one hundred of against misuse of scientific progress the world’s academies of science. that would cause environmental or The group issued a Statement on medical harm and to conduct their Biosecurity in 2005 that provides research in ways that minimize the 12 principles that academies and other risk of misuse of life science research scientific bodies should consider in for destructive purposes. preparing codes of conduct. While codes of conduct cannot prevent Some professional societies have those who are dedicated to carrying incorporated education about out malevolent acts from doing biosecurity issues into their so, they can raise awareness and missions and are raising awareness and facilitating discussion about biosecurity through training Highly engineered Biosafety Level 4 (BSL-4) laboratories are equipped— programs and curricula, standards and their personnel trained—to and guidelines, and professional handle highly infectious biological development activities and agents. Here a scientist showers in a decontamination booth before conferences. For example: exiting the sealed confines of the lab. • The American Association for Increasingly, scientists receive information and training from the Advancement of Science professional societies as well as (AAAS) convened a workshop academic institutions not only in safety measures but also in in November 2008 to evaluate recognizing and preventing education on dual use research. situations that may give rise to The group called for teaching biosecurity concerns. SOURCE: Centers for Disease Control and “the scientific, ethical, and Prevention.

sensitize practicing scientists to the persons,” including convicted fully understand the concerns of risks of working with hazardous felons, illegal aliens, or those the national security community pathogens. deemed ineligble for other reasons. about communication of scientific Facilities must prepare security plans information or the significance The Role of the Federal that include controlling access to of limits and responsibilities that Government laboratories that conduct select security regulations place on The U.S government addresses agent research and responding scientists’ activities. biosecurity concerns in part by in the event of theft or accidental setting basic policy for the conduct release. Select agent laboratories Several National Research Council of science and by producing are also subject to inspection by the reports, including Science and guidance and regulations to government agencies that oversee Security in a Post 9/11 World (2007), govern specific types of research. the program. Seeking Security: Pathogens, Open Communication between the Access, and Genome Databases scientific and security communities The National Science Advisory (2004), and Biotechnology is important to ensure that policies Board for Biosecurity Research in an Age of Terrorism balance increased security with There is a gap—in language, goals, (2004), emphasize the need continued scientific progress. and understanding—between for closer partnerships among Scientists thus have an essential role the scientific community and those involved in addressing the in using their knowledge to inform the national security community. biosecurity challenge. As noted decisions. Some in the national security above, Biotechnology Research in community do not fully understand an Age of Terrorism recommended The Select Agent Program scientists’ need for openly sharing that a science advisory board be One of the primary mechanisms in established to serve as a point 13 data and ideas, the importance of place in the United States to oversee foreign students and scholars, or of continuing dialogue between the use of dangerous pathogens in the extensive nature and benefits scientists and security professionals research laboratories is the Select of international collaboration in to provide case-specific advice on Agent program, administered by science. Among university scientists, the oversight and dissemination of the Centers for Disease Control on the other hand, many do not life sciences research information. and Prevention and the U.S. Department of Agriculture. Begun in 1997, the regulations governing the program were strengthened significantly in the aftermath of the anthrax mailings in 2001 and now provide a rigorous formal oversight system to decide who can possess microorganisms and toxins that could be used as weapons and how facilities that possess them will be protected. Individuals are subject to a background check to identify those on the list of “restricted

SARS: Science in a Globalized Society Today’s society is truly a global village. Commerce and Scientists in laboratories around the world raced travel create a constant flow of people and materials to investigate the virus. Its genetic sequence was around the planet. The severe acute respiratory determined and within weeks was promptly published syndrome (SARS) outbreak of 2002-2003 underscored in the public domain. Dozens of companies and the world’s vulnerabilities to emerging disease laboratories used this information to try to uncover outbreaks. Although it is not an example of the misuse the disease’s pathogenic mechanism, develop of science—the virus occurred naturally—the SARS diagnostic tools and vaccines, and determine what experience demonstrated how valuable unrestricted measures would be most effective to contain the access to scientific information can be in a public spread of the virus. health crisis. Fortunately, the outbreak was contained and the The SARS outbreak was rapid and widespread. The feared pandemic did not occur. The factors responsible virus that causes it is thought to have originated in for its containment, as well as factors that might an animal host in China. Shortly after the first few contribute to its reemergence, are still being studied. human cases occurred, people in five countries became It is clear to many that research to investigate SARS infected within a 24-hour period; within six months, was and continues to be necessary for the public good. the disease had reached more than 30 countries, On the other hand, the complete genome sequence of ultimately killing more than 700 of the estimated the virus that causes SARS, which is publicly accessible, 8,000 people who were infected. During the outbreak, could conceivably be used by a very sophisticated 14 SARS was seen as having the potential to cause a more bioterrorist to synthesize a new version of the virus. severe pandemic than even the 1918 flu. The globalized nature of the scientific enterprise, in combination with the speed with which people—and pathogens—travel around the globe, increases the complexity of the biosecurity challenge, as well as the need for international collaboration in addressing it. In response to the SARS infectious disease alert of 2003 in Asia, people moving around were screened for the disease, and measures were put in place at airports and other transport hubs to provide quarantine conditions for the public arriving from areas of infection.

and made recommendations on “Top-down” efforts, such as specific issues, including synthetic formal international agreements, biology and proposals for personnel can set policies and standards, reliability programs for those doing and international institutions can research on dangerous pathogens. engage multiple stakeholders to set policy and guidance. “Bottom-up” The Role of International networks of scientists and scientific Coordination organizations can work together to In the global biological research determine appropriate practices and enterprise, organisms, information, mechanisms for research oversight tools, and people are constantly and self-governance. Both types of The World Health Organization crossing international borders, efforts could be strengthened; many promotes policies and actions in both on the ground and in National Research Council reports all countries to protect against potential biosecurity events and cyberspace. Global cooperation and activities have explored ways threats. Its Laboratory biosafety among researchers speeds scientific for the international community to manual encourages countries to advances—including progress in engage effectively on these issues. develop national codes of practice for safe handling of pathogenic defenses against public health microorganisms in laboratories. threats as well as bioterrorism. Although prudence requires good The third edition of the manual, published in 2004, stresses Individual scientists often have stewardship of harmful biological throughout the importance of many opportunities to conduct, materials, tools, and knowledge personal responsibility. This version in the United States, as stated in publish, and share their work in of the international biohazard warning symbol appears in the other countries, increasing the 2004 edition. ability of researchers to make 15 new connections and build The National Science Advisory Board on one another’s work. Even “Given the for Biosecurity (NSABB), chartered within the United States, the in 2004, has its headquarters at scientific workforce is increasingly fundamentally the National Institutes of Health. international: At the National international character Providing federal guidelines and Institutes of Health, for example, of research in the then relying on self-governance by about half the technical staff is made research institutions are key to the life sciences, any up of non-U.S. citizens. board’s proposed oversight system serious attempt to for dual use research. This system International coordination and prevent the misuse of would require principal investigators cooperation will be necessary to research must include to make an initial assessment of the make any effort to mitigate the potential for misuse of their research risks of bioterrorism effective, efforts at improving and would add biosecurity to the according to Biotechnology Research and harmonizing scope of reviews conducted by in an Age of Terrorism (2004) and standards and practices Institutional Biosafety Committees. Globalization, Biosecurity, and the The NSABB has developed criteria Future of the Life Sciences (2006). internationally.” for identifying “dual use research There are two main approaches –Biotechnology Research of concern,” proposed guidance to achieving international in an Age of Terrorism on education and oversight issues, coordination on biosecurity: (NRC, 2004)

Science and Security in a Post 9/11 World (2007), unnecessarily closing ourselves off from the world in a futile effort to protect ourselves will only isolate us from an increasingly integrated and competitive global community. Recent international efforts have sought to supplement the deep and long-standing foundation of scientific self-regulation, voluntary standards, and associational accreditation with mandatory requirements on specific aspects of laboratory safety. A number of countries impose export controls on dual use equipment, pathogens, and toxins that could legally binding—such as those to emphasize different strategies, be used for biological warfare. In made by the Australia Group, an participants supported developing addition, agreements that are not informal network of 40 member a toolkit of multiple options for countries—work to harmonize addressing education, oversight, 16 national controls on the export of and governance issues. For the dual use materials and equipment. bottom-up approach, participants However, international regulation identified key roles for international “The participants [in the of biology is complicated by the scientific organizations, such Forum] came from all lack of a multilateral consensus as as the International Council for over the world because to the basic security framework that Science, the InterAcademy Panel on would allow consistent application International Issues, the Academy of the life sciences are of controls. Sciences for the Developing World, a genuinely global and science unions, because these enterprise, and thus any In 2005 and 2008 the National organizations are often perceived as Academies partnered with other policies must include neutral networks that can engage scientific organizations in an scientists from many countries. For international as well as international dialogue. As described the top-down aspects, participants national measures.” in The 2nd International Forum particularly cited the Biological and on Biosecurity: Summary of an Toxin Weapons Convention not only –The 2nd International International Meeting, Budapest, because it provides the fundamental Forum on Biosecurity: Hungary, March 30 to April 2, 2008, norm against the misuse of the life Summary of an participants at the forum identified sciences but also for its value as a International Meeting, Budapest, Hungary, views shared among many countries convening mechanism to address March 30 to April 2, 2008 and issues on which they differ. topics relevant to the scientific (NRC, 2009) Because countries may choose community.

doing science responsibly Biology is advancing rapidly and the life sciences achieve a greater community are all grappling with powerful materials, tools, and appreciation of the dangers and a biosecurity issues. Although much knowledge are widely accessible greater willingness to shoulder the more can be done, a network of throughout the world—indeed, responsibility to prevent misuse. resources exists to guide scientists the free exchange of scientific On a global scale, a new ethos is in making informed decisions. information was an essential required. Within their institutions, researchers factor in enabling the many have oversight bodies to turn to; scientific achievements of the 20th Life scientists also need to put forth, within their professional societies, century. In parallel, current and for the education of policy makers they have access to training and future biosecurity threats could and the public, cogent arguments other resources; within their state come as much from terrorists for the benefits of research for and federal governments, there are operating outside of traditional both health and security. The same regulations and sources of guidance government frameworks as from developments serve biosecurity on broad issues. Together with the governments of other nations. needs by providing therapies that networks at the international level, Recognizing this reality, it is would dramatically decrease the these resources—which continue increasingly important that life success of a potential biological to grow in number and type—can scientists, and the organizations— attack. Concerted communication give scientists the information 17 funding agencies, professional of this fact is needed to assure that and tools they need to advance societies, scientific journals, and policy makers and the security knowledge while doing their part others—that support their activities, community do not impede essential to protect the safety and security take steps to ensure that the fruits research. of those around them: in short, to of their work are not exploited conduct science responsibly. for malevolent purposes. This Researchers, policy makers, will require that those working in regulators, and the security

NOTES WEB RESOURCES For 1 Gibbs, M.J. et al. 2001. Recombination in the Further INFORMATION hemagglutinin gene of the 1918 “Spanish Flu.” Science 293(5536):1842-1845. National Academies Biosecurity Website http://nationalacademies.org/biosecurity 2 Cello, J. et al. 2002. Chemical synthesis of poliovirus This site aggregates information on studies and other cDNA: generation of infectious virus in the absence of activities at The National Academies on a wide array of natural template. Science 297(5583):1016-1018. issues related to biosecurity, both in the United States and internationally. 3 Aldous, P. 2001. Biologists urged to address risk of data aiding bioweapon design. Nature 414(6861):237-238 U.S. Government Interagency Biosecurity Website as cited in Zilinskas, R.A. and J.B. Tucker. 2002. Limiting http://www.whitehouse.gov/administration/eop/ostp/ the contribution of the open scientific literature to the nstc/biosecurity biological weapons threat. Journal of Homeland Security. The White House has convened an interagency working Available online at www.homelandsecurity.org/journal/ group to coordinate biosecurity outreach and education Articles/tucker.html. across federal agencies. This website, developed by the 4 Arora, N. and S.H. Leppa. 1994. Fusions of anthrax working group, presents the biosecurity activities of the toxin lethal factor with Shiga toxin and diptheria toxin agencies and provides links to agencies, regulations, enzymatic domains are toxic to mammalian cells. and reports being used to develop policies. Its goal is to Infection and Immunity 62(11):4955-4961. reduce public confusion about U.S. biosecurity activities and, in the spirit of Open Government, have a transparent 5 Wang, X.G. et al. 2001. Purified cholera toxin B subunit process for the interagency working group. from transgenic tobacco plants possesses authentic antigenicity. Biotechnology and Bioengineering 72(4): National Science Advisory Board for Biosecurity 490-494. http://oba.od.nih.gov/biosecurity/biosecurity.html This official site of the National Science Advisory Board on 6 Boyce, N. 2002. Should Scientists Publish Work Biosecurity contains information on their meetings and 18 That Could Be Misused? US News and World Report publications as well as background information on dual 132(22):60. use issues. 7 Wein, L.M. and Y. Liu. 2005. Analyzing a bioterror attack on the food supply: the case of botulinum toxin in milk. American Association for the Advancement of Proceedings of the National Academy of Sciences U. S. A. Science (AAAS) 102(28):9984-9989. http://cstsp.aaas.org/dualuse.html This site, maintained by the AAAS Center for Science, 8 NSABB. June 2007. Proposed Framework for the Technology, and Security Policy, contains information Oversight of Dual Use Life Sciences Research: Strategies about dual use issues relevant to teachers and students. for Minimizing the Potential Misuse of Research Information. Available online at http://oba.od.nih.gov/ The Virtual Biosecurity Center (VBC) biosecurity/biosecurity_documents.html. http://virtualbiosecuritycenter.org A project of the Federation of American Scientists 9 http://www.aaas.org/news/releases/2008/1229dual_ involving several scientific organizations, including AAAS use.shtml. and the National Academies, along with the Organisation 10 http://www.asmbiodefense.org. for Economic Co-operation and Development, this site aggregates a wide spectrum of material on many aspects http://opa.faseb.org/pdf/2009/FASEB_Statement_on_ 11 of biosecurity. Dual_Use_Education.pdf. Biosecurity Codes Website http://www.biosecuritycodes.org/ This site, developed by the International Futures Program of the Organisation for Economic Cooperation and Development, discusses and supplies numerous examples of biosecurity codes of conduct and international and national legislation. It also links visitors to resources throughout the international biosecurity community.

This booklet was prepared by the National Research Additional Reports from the National Council based on the following reports: Academies Responsible Research with Biological Select Agents Department of Homeland Security Bioterrorism Risk and Toxins (2009) Assessment: A Call for Change (2008) Sponsored by: National Institutes of Health Sponsored by: Department of Homeland Security A Survey of Attitudes and Actions on Dual Use An International Perspective on Advancing Research in the Life Sciences: A Collaborative Effort Technologies and Strategies for Managing Dual- of the National Research Council and the American Use Risks: Report of a Workshop (2005) Association for the Advancement of Science (2009) Sponsored by: Department of Homeland Security, Sponsored by: Carnegie Corporation of New Centers for Disease Control and Prevention, York, Alfred P. Sloan Foundation, and the National Food and Drug Administration, National Institute Academies Presidents’ Circle Communications of Allergy and Infectious Diseases, National Initiative Science Foundation, and Intelligence Technology Innovation Center The 2nd International Forum on Biosecurity: Summary of an International Meeting, Budapest, Microbial Threats to Health: The Threat of Hungary, March 30 to April 2, 2008 (2008) Pandemic Influenza (2005) Sponsored by: Carnegie Corporation of New York Excerpted and updated from report sponsored and Alfred P. Sloan Foundation by: Centers for Disease Control and Prevention’s National Center for Infectious Diseases, Science and Security in a Post 9/11 World: A Report Department of Defense, U.S. Agency for Based on Regional Discussions Between the Science International Development, U.S. Department of and Security Communities (2007) Agriculture’s Food Safety and Inspection Service, Sponsored by: National Science Foundation and National Institutes of Health’s National Institute 19 National Institutes of Health of Allergy and Infectious Diseases, National Institutes of Health’s Fogarty International Center, Globalization, Biosecurity, and the Future of the Life Ellison Medical Foundation, U.S. Food and Drug Sciences (2006) Administration, and U.S. Joint Institute for Food Sponsored by: Department of Homeland Security, Safety Research Centers for Disease Control and Prevention, Food and Drug Administration, National Institute of On Being a Scientist: A Guide to Responsible Allergy and Infectious Diseases, National Science Conduct in Research, 3rd ed. (2009) Foundation, and Intelligence Technology Innovation Sponsored by: National Science Foundation Center Biotechnology Research in an Age of Terrorism Reports from the National Academies are available (2004) from the National Academies Press, 500 Fifth Street, Sponsored by: Alfred P. Sloan Foundation and NW, Washington, DC 20001; 800-624-6242; Nuclear Threat Initiative www.nap.edu. Reports are available online in a fully searchable format. Seeking Security: Pathogens, Open Access, and Genome Databases (2004) Sponsored by: National Science Foundation

About the National Academies The National Academies—the National Academy of Sciences, National Academy of Engineering, Institute of Medicine, and the National Research Council—provide a public service by working outside the framework of government to ensure independent advice on matters of science, technology, and medicine. They enlist committees of the nation’s top scientists, engineers, and other experts—all of whom volunteer their time to study specific concerns. The results of these deliberations are authoritative, peer-reviewed reports that have inspired some of the nation’s most significant efforts to improve the health, education, and welfare of the population.

Understanding Biosecurity: Protecting Against the Misuse of Science in Today's World Get This Book
×
Buy Paperback (pack of 5) | $2.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Drawing on the work of the National Academies, this booklet introduces some of the issues at the intersection of science and security. The life sciences offer tremendous promise for meeting many 21st century challenges. But with opportunities come responsibilities. An important aspect of scientists' responsibility to society is captured in the concept of biosecurity-- the challenge to move the life sciences forward for legitimate purposes while reducing the risks that some materials, knowledge, tools, and technologies could also be used to do harm. This booklet was developed to serve as an educational resource for students and the scientific community, to illuminate the importance of biosecurity, and to explore how scientists, organizations, and governments at many levels can work together to minimize the threat.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!