National Academies Press: OpenBook
Suggested Citation:"Front Matter." Institute of Medicine. 2000. Description and Analysis of the VA National Formulary. Washington, DC: The National Academies Press. doi: 10.17226/9879.
×

DESCRIPTION AND ANALYSIS OF THE VA National Formulary

David Blumenthal and Roger Herdman, Editors

VA Pharmacy Formulary Analysis Committee

Division of Health Care Services

INSTITUTE OF MEDICINE

NATIONAL ACADEMY PRESS
Washington, D.C.

Suggested Citation:"Front Matter." Institute of Medicine. 2000. Description and Analysis of the VA National Formulary. Washington, DC: The National Academies Press. doi: 10.17226/9879.
×

NATIONAL ACADEMY PRESS
2101 Constitution Avenue, N.W.Washington, D.C.20418

NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance.

Support for this study was provided by the Department of Veterans Affairs (Contract No. V-101-(93)P-1637, Task Order 10). The views presented are those of the Institute of Medicine VA Pharmacy Formulary Analysis Committee and are not necessarily those of the funding organization.

Description and Analysis of the VA National Formulary is available for sale from the

National Academy Press
, 2101 Constitution Avenue, N.W. , Box 285 , Washington, DC20055 ; call (800) 624-6242 or (202) 334-3313 (in the Washington metropolitan area), or visit the NAP's on-line bookstore at www.nap.edu.

The full text of this report is available on line at www.nap.edu.

Copyright 2000 by the National Academy of Sciences. All rights reserved.

Printed in the United States of America.

The serpent has been a symbol of long life, healing, and knowledge among almost all cultures and religions since the beginning of recorded history. The image adopted as a logotype by the Institute of Medicine is based on a relief carving from ancient Greece, now held by the Staatliche Musseen in Berlin.

Suggested Citation:"Front Matter." Institute of Medicine. 2000. Description and Analysis of the VA National Formulary. Washington, DC: The National Academies Press. doi: 10.17226/9879.
×

“Knowing is not enough; we must apply. Willing is not enough; we must do.”

—Goethe

INSTITUTE OF MEDICINE

Shaping the Future for Health

Suggested Citation:"Front Matter." Institute of Medicine. 2000. Description and Analysis of the VA National Formulary. Washington, DC: The National Academies Press. doi: 10.17226/9879.
×

THE NATIONAL ACADEMIES

National Academy of Sciences

National Academy of Engineering

Institute of Medicine

National Research Council

The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Bruce M. Alberts is president of the National Academy of Sciences.

The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. William A. Wulf is president of the National Academy of Engineering.

The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Kenneth I. Shine is president of the Institute of Medicine.

The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy's purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Bruce M. Alberts and Dr. William A. Wulf are chairman and vice chairman, respectively, of the National Research Council.

Suggested Citation:"Front Matter." Institute of Medicine. 2000. Description and Analysis of the VA National Formulary. Washington, DC: The National Academies Press. doi: 10.17226/9879.
×

VA PHARMACY FORMULARY ANALYSIS COMMITTEE

DAVID BLUMENTHAL (Chair), Director,

Institute for Health Policy,

Professor of Medicine and Health Care Policy,

Harvard Medical School

REINHARDT HENRY BODENBENDER, Director of Medical Services,

Paralyzed Veterans of America, Washington, D.C.

J. LYLE BOOTMAN, Professorand Dean, Executive Director,

Center for Health Outcomes and PharmacoEconomic Research, University of Arizona College of Pharmacy

JOHN P. BURKE, Professor of Medicine, Chief,

Department of Clinical Epidemiology, Latter Day Saints Hospital, Salt Lake City, Utah

ELIZABETH DICHTER, Executive Vice President,

PCS Health Systems, Scottsdale, Arizona

THOMAS R. FULDA, Program Director,

Drug Utilization Review, U.S. Pharmacopoeia, Rockville, Maryland

MARTHA N. HILL, Professor, Director of the Center for Nursing Research,

Johns Hopkins University School of Nursing

JOHN D. JONES, Director of Pharmacy Networks and Legal Affairs,

Prescription Solutions, Costa Mesa, California

JAMES JONATHAN LIPSKY, Director of Clinical Pharmacology, Professor of Medicine and Pharmacology,

Mayo Clinic and Foundation, Rochester, Minnesota

ALBERT SIU, Professorand Chief of General Internal Medicine,

Mount Sinai School of Medicine

FRANK A. SLOAN, Professor of Health Policy and Management, Professor of Economicsand Director,

Center for Health Policy, Law, and Management, Duke University

RICHARD A. WANNEMACHER, JR., Associate National Legislative Director,

Disabled American Veterans (DAV), Washington, D.C.

OTTO F. WOLKE, President of Schellen & Partners USA, Inc.,

Danville, Pennsylvania

ALLAN ZIMMERMAN, Senior Executive Vice Presidentand General Manager,

National Prescription Administrators, East Hanover, New Jersey

Consultants

JEFFREY S. BROWN,

Schneider Institute for Health Policy, The Heller School, Brandeis University

ARNOLD M. EPSTEIN,

Harvard School of Public Health, Department of Health Policy and Management

RICHARD FRANK,

Harvard Medical School, Department of Health Care Policy

HAIDEN A. HUSKAMP,

Harvard Medical School, Department of Health Care Policy

Suggested Citation:"Front Matter." Institute of Medicine. 2000. Description and Analysis of the VA National Formulary. Washington, DC: The National Academies Press. doi: 10.17226/9879.
×

IOM Staff

ROGER HERDMAN, Study Director

CHRISTINE COUSSENS, Research Associate

RITA GASKINS, Senior Project Assistant

Suggested Citation:"Front Matter." Institute of Medicine. 2000. Description and Analysis of the VA National Formulary. Washington, DC: The National Academies Press. doi: 10.17226/9879.
×

Preface

Medications have captured the imagination of the American people and the attention of policy makers. New scientific findings have demonstrated the extraordinary power and promise of drags to prevent and cure disease, relieve suffering, and improve the quality of life for many Americans. Media reports, the World Wide Web, and aggressive advertising by the pharmaceutical industry have kept consumers informed of every advance, large or small, in the development of new medications. Pharmaceuticals have become as vital to good health care as hospital and physician services. Propelled by new, expensive products and consumer demand, national spending on medications is increasing at double-digit rates and may soon rival physician and even inpatient hospital expenditures.

This latter prospect has naturally attracted the interest of policy makers responsible for the publicly funded programs that have become so important to so many Americans. Most Medicaid programs have adopted measures, including formularies, to control the costs of pharmaceuticals. The Medicare program does not cover outpatient drugs, but debate rages over whether it should. If, as seems likely, Medicare does provide pharmaceutical benefits for its 39 million beneficiaries, then programs to control expenditures will be part and parcel of this reform.

As the steward of the nation's largest and most comprehensive publicly owned and managed health care program, the U.S. Department of Veterans Affairs (VA) and its Veterans Health Administration (VHA) are also confronting the opportunities and challenges associated with the national surge in the use and costs of medications. The VHA is the largest single purchaser of pharma

Page viii Cite
Suggested Citation:"Front Matter." Institute of Medicine. 2000. Description and Analysis of the VA National Formulary. Washington, DC: The National Academies Press. doi: 10.17226/9879.
×

ceuticals in the United States. It cares for a population that is disproportionately elderly and ill, including many veterans with service-related disabilities. With pharmaceutical expenditures rising at more than 10% a year, the VHA in 1997 implemented a new National Formulary intended to help control costs and improve the quality of prescribing in the VHA's 172 hospitals, more than 600 ambulatory facilities, 132 nursing homes, and other health care facilities.

The VA National Formulary generated controversy, which motivated congressional scrutiny and a directive to the VA to commission this report reviewing the experience with the National Formulary and formulary system. This Institute of Medicine committee was pleased to assist the Congress with this review, in part because the committee saw in the VHA example an opportunity to understand and anticipate problems that all publicly funded programs are likely to encounter in this new age of pharmaceuticals.

The Congress asked the committee to review the restrictiveness of the National Formulary, its impact on the costs and quality of care in the VHA, and how it compared to formularies and drug management practices in the private sector and in other public programs, especially Medicaid. Detailed in the pages that follow, the committee's findings and conclusions on these questions are, the committee believes, highly instructive, though not always in the ways that we anticipated.

The committee found that formularies and formulary systems (the many policies and procedures necessary to manage implementation of formularies) are an essential part of modern health care systems and that the VHA therefore was justified in creating its National Formulary. Further, we found that the VA National Formulary is not overly restrictive, and the limited available evidence suggests that it has probably meaningfully reduced drug expenditures without demonstrable adverse effects on quality.

This is the good news. The committee also concluded that there are manifold opportunities to improve the management of the formulary system used by the VHA. The National Formulary lacks essential systems to assure that new drugs are expeditiously reviewed for inclusion, that a responsive process for assuring access to medically necessary exceptions to the formulary is consistently in place system-wide, that therapeutic interchange is accomplished in a flexible and consistent way, sensitive to patient risks, across the far-flung VHA system, and that views of critical constituencies of both providers and patients are represented in the management of the National Formulary.

Perhaps most troubling, the committee found a dearth of information to evaluate the full impact of the National Formulary on veterans —their health and satisfaction—and on the VHA. Although the VHA has made efforts to improve quality monitoring generally, no data exist that allow an assessment of the effect of the VA National Formulary on the structure, processes, and outcomes of care received by VHA patients. Committee consultants made valiant efforts to assess quality effects using data on rates of hospitalization for conditions associated with drugs most closely regulated by the National Formulary—those in closed

Suggested Citation:"Front Matter." Institute of Medicine. 2000. Description and Analysis of the VA National Formulary. Washington, DC: The National Academies Press. doi: 10.17226/9879.
×

and preferred classes. The committee considers such analyses helpful, but insufficient to reach a satisfactory conclusion on quality effects. Available cost data do not allow a full evaluation of the indirect effects of the VA National Formulary on the use of other health care resources such as other drugs and services. Such effects have been significant in the Medicaid experience and could greatly reduce net savings on drugs themselves. There are also few data on the administrative costs of the National Formulary, including its important but subtle effects on the time health professionals must spend with and for patients in managing changes in medications.

Gaps in the information available for policy making concerning the VA National Formulary are not unique to this system. The committee found a comparable paucity of data on the quality effects of formularies in Medicaid and the private sector. Reliable information on the policies of the fast-changing private companies that manage formularies for private health care plans—so-called pharmacy benefits management companies or PBMs—were also lacking when the committee initiated its review. The committee is indebted to representatives of several PBMs who organized a survey of their companies and supplied valuable new data on their policies and procedures.

Given the rapid pace of policy development with respect to Medicare and other public and private programs, the lack of data to support decision making is disturbing. It is difficult to imagine a modern health care system that does not employ a formulary. It is also difficult to imagine how such formularies can now be effectively designed and managed given our limited understanding of their effects on cost and quality of care and patient and provider satisfaction. The VHA, because of its central control and comprehensive benefits, has a unique opportunity to study its own experience with formularies and formulary systems. The committee strongly urges the Congress and the VHA to take advantage of this opportunity.

The committee's work benefited enormously from the contributions of a dedicated and talented staff. Roger Herdman, its study director, and Christine Coussens, his assistant, accomplished an extraordinary amount of work in a very modest period of time—reviewing the copious literature on the history and current status of formularies, conducting field work to understand the National Formulary and its functioning, designing and supervising the collection of primary data on PBMs, and supervising able contractors. Drs. Herdman and Coussens were invariably rigorous in their review of data and diplomatic in their management of the committee. The committee is also indebted to Haiden Huskamp and Richard Frank, who conducted an excellent analysis of the cost effects of the National Formulary, and to Jeffrey Brown, who expertly reviewed and summarized the Medicaid experience with drug management.

The importance and salience of drug benefits in modern health care systems is more clear than ever from the committee's work. The central role of formularies in providing high-quality and affordable drug coverage is also clear. It will

Suggested Citation:"Front Matter." Institute of Medicine. 2000. Description and Analysis of the VA National Formulary. Washington, DC: The National Academies Press. doi: 10.17226/9879.
×

be hard to resolve the inevitable controversies over optimal approaches to manage drug benefits without an urgent national program of research and development on the design of formularies and formulary systems themselves.

David Blumenthal, M.D., M.P.P.

Chair

Suggested Citation:"Front Matter." Institute of Medicine. 2000. Description and Analysis of the VA National Formulary. Washington, DC: The National Academies Press. doi: 10.17226/9879.
×

REVIEWERS

The report was reviewed by individuals chosen for their diverse perspectives and technical expertise in accordance with procedures approved by the National Research Council's Report Review Committee. The purpose of this independent review is to provide candid and critical comments to assist the authors and the Institute of Medicine in making the published report as sound as possible and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The contents of the review comments and the draft manuscript remain confidential to protect the integrity of the deliberative process. The committee wishes to thank the following individuals for their participation in the report review process:

Jerome Avorn, Brigham and Women's Hospital, Boston

Enriqueta C. Bond, Burroughs Wellcome Fund, Durham, North Carolina

Christine K. Cassel, Mount Sinai Medical Center, New York City

David H. Kreling, School of Pharmacy, University of Wisconsin

Joseph P. Newhouse, Harvard University

Judith Wagner, Congressional Budget Office, Washington, D.C.

Albert L. Wertheimer, Merck and Co., West Point, Pennsylvania

Mel Worth, Senior Scholar in Residence, Institute of Medicine, Washington, D.C.

While the individuals listed above provided many constructive comments and suggestions, responsibility for the final content of the report rests solely with the authoring committee and the Institute of Medicine.

Suggested Citation:"Front Matter." Institute of Medicine. 2000. Description and Analysis of the VA National Formulary. Washington, DC: The National Academies Press. doi: 10.17226/9879.
×
Suggested Citation:"Front Matter." Institute of Medicine. 2000. Description and Analysis of the VA National Formulary. Washington, DC: The National Academies Press. doi: 10.17226/9879.
×
   

 National Formulary Classes,

 

34

   

 Formulary Relationship to Pharmaceutical Companies,

 

35

   

 Report Outline,

 

36

 2

 

IS THE VA NATIONAL FORMULARY OVERLY RESTRICTIVE AND DOES IT PREVENT PHYSICIANS FROM MEETING THE HEALTH CARE NEEDS OF VETERANS?

 

37

   

 Background,

 

37

   

 Elements of Restrictiveness,

 

38

   

 Size and Coverage of the National Formulary,

 

40

   

 Adequacy of the VA National Formulary,

 

42

   

 Closed and Preferred Drug Classes in the VA,

 

44

   

 Restrictiveness of Class Closure,

 

46

   

 Excluded Drugs and Other Limits,

 

48

   

 National Formulary Effects on Drug Use,

 

49

   

 Addition of New Drugs and Formulary Reappraisal,

 

50

   

 National Policy Regarding New Drugs,

 

50

   

 VISN Addition of New FDA Approvals,

 

51

   

 Evaluation of the VA Process for Adding New FDA-Approved Priority Drugs,

 

52

   

 Addition of Existing Drugs by the VA,

 

54

   

 Additions of Newly Approved or Existing Drugs in Other Health Care Systems,

 

56

   

 The Nonformulary Process,

 

57

   

 Therapeutic Interchange,

 

61

   

 OTC Drug Coverage, Generic Substitution, and Physician Satisfaction,

 

65

   

 Generic Substitution,

 

66

   

 Veterans' Complaints,

 

67

   

 VA Physician Satisfaction,

 

68

   

 Summary,

 

70

 3

 

WHAT ARE THE POTENTIAL COSTS TO VA HEALTH CARE ASSOCIATED WITH THE NATIONAL FORMULARY FOR DRUGS?

 

72

   

 Introduction,

 

72

   

 The VHA National Formulary as It Affects Costs,

 

74

   

 Background,

 

74

   

 Basic Economics of the VA National Formulary,

 

76

   

 Features of the National Formulary,

 

77

   

 VISN Implementation Issues,

 

78

   

 Other Influences on VHA Pharmaceutical Spending,

 

78

   

 Analytical Framework,

 

79

   

 Data,

 

79

Suggested Citation:"Front Matter." Institute of Medicine. 2000. Description and Analysis of the VA National Formulary. Washington, DC: The National Academies Press. doi: 10.17226/9879.
×
   

 Empirical Methods,

 

81

   

 Results of the IOM Committee Analysis of Cost Effects,

 

81

   

 How Have the VHA and the Veteran User Population Changed During the Study Period?

 

81

   

 How Has the National Formulary Affected Prices for Closed and Preferred Classes?

 

81

   

 How Has the National Formulary Affected Prescribing Patterns Within the Closed and Preferred Classes?

 

82

   

 How Has the National Formulary Affected Pharmaceutical Spending Per Veteran User for Closed and Preferred Classes?

 

100

   

 Is There Evidence That Changes in Formulary Policy Have Resulted in Increased Utilization Elsewhere in the VHA System?

 

109

   

 Exploration of Changes in Hospital Discharges Per VHA User,

 

109

   

 Costs Associated with Implementing and Managing the National Formulary,

 

113

   

 Estimated National Formulary Savings,

 

114

   

 Conclusions,

 

118

 4

 

WHAT ARE THE EFFECTS OF THE NATIONAL FORMULARY AND RELATED POLICIES ON QUALITY OF CARE?

 

120

   

 Background Information,

 

120

   

 Sources of Quality Data,

 

121

   

 Quality of Care in the VHA and Effects of the National Formulary,

 

122

   

 Pharmacy, Clinical, and Formulary Program Elements Relevant to Quality of Care,

 

123

   

 Clinical Pharmacy,

 

123

   

 P&T Committees,

 

124

   

 The VA PBM Complex,

 

127

   

 Additions to, and Quality of, the National Formulary,

 

130

   

 Policies and Procedures,

 

131

   

 Drug Class Reviews,

 

131

   

 Clinical Guidelines and Drug Utilization Reviews,

 

133

   

 The Nonformulary Process,

 

134

   

 Therapeutic Interchange, Policy, and Results,

 

138

   

 Effects of the National Formulary on Use of Drugs by the VHA,

 

139

   

 Adverse Drug Events,

 

143

   

 Changes in Inpatient Hospital Discharge Associated with the National Formulary,

 

144

   

 Patient Complaints—Advocate, Veterans of Foreign Wars, and Survey Data,

 

145

   

 Physician Complaints and Survey Data,

 

146

   

 Summary Statement,

 

147

Page xvii Cite
Suggested Citation:"Front Matter." Institute of Medicine. 2000. Description and Analysis of the VA National Formulary. Washington, DC: The National Academies Press. doi: 10.17226/9879.
×

DESCRIPTION AND ANALYSIS OF THE

VA National Formulary

Page xviii Cite
Suggested Citation:"Front Matter." Institute of Medicine. 2000. Description and Analysis of the VA National Formulary. Washington, DC: The National Academies Press. doi: 10.17226/9879.
×
Suggested Citation:"Front Matter." Institute of Medicine. 2000. Description and Analysis of the VA National Formulary. Washington, DC: The National Academies Press. doi: 10.17226/9879.
×
Page R1
Suggested Citation:"Front Matter." Institute of Medicine. 2000. Description and Analysis of the VA National Formulary. Washington, DC: The National Academies Press. doi: 10.17226/9879.
×
Page R2
Suggested Citation:"Front Matter." Institute of Medicine. 2000. Description and Analysis of the VA National Formulary. Washington, DC: The National Academies Press. doi: 10.17226/9879.
×
Page R3
Suggested Citation:"Front Matter." Institute of Medicine. 2000. Description and Analysis of the VA National Formulary. Washington, DC: The National Academies Press. doi: 10.17226/9879.
×
Page R4
Suggested Citation:"Front Matter." Institute of Medicine. 2000. Description and Analysis of the VA National Formulary. Washington, DC: The National Academies Press. doi: 10.17226/9879.
×
Page R5
Suggested Citation:"Front Matter." Institute of Medicine. 2000. Description and Analysis of the VA National Formulary. Washington, DC: The National Academies Press. doi: 10.17226/9879.
×
Page R6
Suggested Citation:"Front Matter." Institute of Medicine. 2000. Description and Analysis of the VA National Formulary. Washington, DC: The National Academies Press. doi: 10.17226/9879.
×
Page R7
Page viii Cite
Suggested Citation:"Front Matter." Institute of Medicine. 2000. Description and Analysis of the VA National Formulary. Washington, DC: The National Academies Press. doi: 10.17226/9879.
×
Page R8
Suggested Citation:"Front Matter." Institute of Medicine. 2000. Description and Analysis of the VA National Formulary. Washington, DC: The National Academies Press. doi: 10.17226/9879.
×
Page R9
Suggested Citation:"Front Matter." Institute of Medicine. 2000. Description and Analysis of the VA National Formulary. Washington, DC: The National Academies Press. doi: 10.17226/9879.
×
Page R10
Suggested Citation:"Front Matter." Institute of Medicine. 2000. Description and Analysis of the VA National Formulary. Washington, DC: The National Academies Press. doi: 10.17226/9879.
×
Page R11
Suggested Citation:"Front Matter." Institute of Medicine. 2000. Description and Analysis of the VA National Formulary. Washington, DC: The National Academies Press. doi: 10.17226/9879.
×
Page R12
Page xiii Cite
Suggested Citation:"Front Matter." Institute of Medicine. 2000. Description and Analysis of the VA National Formulary. Washington, DC: The National Academies Press. doi: 10.17226/9879.
×
Page R13
Suggested Citation:"Front Matter." Institute of Medicine. 2000. Description and Analysis of the VA National Formulary. Washington, DC: The National Academies Press. doi: 10.17226/9879.
×
Page R14
Suggested Citation:"Front Matter." Institute of Medicine. 2000. Description and Analysis of the VA National Formulary. Washington, DC: The National Academies Press. doi: 10.17226/9879.
×
Page R15
Suggested Citation:"Front Matter." Institute of Medicine. 2000. Description and Analysis of the VA National Formulary. Washington, DC: The National Academies Press. doi: 10.17226/9879.
×
Page R16
Page xvii Cite
Suggested Citation:"Front Matter." Institute of Medicine. 2000. Description and Analysis of the VA National Formulary. Washington, DC: The National Academies Press. doi: 10.17226/9879.
×
Page R17
Page xviii Cite
Suggested Citation:"Front Matter." Institute of Medicine. 2000. Description and Analysis of the VA National Formulary. Washington, DC: The National Academies Press. doi: 10.17226/9879.
×
Page R18
Next: Executive Summary »
Description and Analysis of the VA National Formulary Get This Book
×
Buy Paperback | $78.00 Buy Ebook | $64.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The VA National Formulary generated controversy, which motivated congressional scrutiny and a directive to the VA to commission this report reviewing the experience with the National Formulary and formulary system. This Institute of Medicine committee was pleased to assist the Congress with this review, in part because the committee saw in the VHA example an opportunity to understand and anticipate problems that all publicly funded programs are likely to encounter in this new age of pharmaceuticals. The Congress asked the committee to review the restrictiveness of the National Formulary, its impact on the costs and quality of care in the VHA, and how it compared to formularies and drug management practices in the private sector and in other public programs, especially Medicaid. Detailed in the pages that follow, the committee's findings and conclusions on these questions are, the committee believes, highly instructive, though not always in the ways that we anticipated.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!