Click for next page ( R2


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page R1
ASSESSI No MEDICAL I ~ ~ ~ ~ war ~ ~ Committee for Evaluating Medical Technologies in Clinical Use Division of Health Sciences Policy Division of Health Promotion and Disease Prevention INSTITUTE OF MEDICINE NATIONAL ACADEMY PRESS Washington, D. C. 1985 E,~

OCR for page R1
National Academy Press 2101 Constitution Avenue, NW Washington, DC 20418 NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the Na- tional Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competencies and with regard for appropriate balance. This report has been reviewed by a group other than the authors according to the procedures approved by a Report Review Committee consisting of members of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The Institute of Medicine was chartered in 1970 by the National Academy of Sciences to enlist distinguished members of the appropriate professions in the examination of policy matters pertaining to the health of the pub- lic. In this, the Institute acts under both the Academy's 1863 congressional charter responsibility to be an advisor to the federal government and its own initiative in identifying issues of medical care, research, and education. This study was supported by the Henry J. Kaiser Family Foundation, Kaiser Foundation Hospitals, the Na- tional Research Council Fund, and by the Office of Medical Applications of Research, National Institutes of Health, Contract No. ASU000001-03. The National Research Council Fund is a pool of private, discretionary, nonfederal funds that is used to support a program of academy-initiated studies of national issues in which science and technology figure significantly. Library of Congress Cataloging-in-Publication Data Institute of Medicine (U.S.). Division of Health Science Policy. Assessing medical technologies. Outgrowth of a conference held by the Institute of Medicine in 1980. Prepared by the Committee for Evaluating Medical Technologies in Clinical Use. Includes bibliographies and index. 1. Medical careUnited States. 2. Technology assessmentUnited States. United StatesEvaluation. 3. Medical technology 4. Medical care. 5. Tech- nology assessment. 6. Medical technologyEvaluation. I. Institute of Medicine (U.S.~. Division of Health Promotion and Disease Prevention. II. Institute of Committee for Evaluating Medical III. Title. fDNLM: Medicine (U. S. ) . Technologies in Clinical Use. 1. Evaluation Studies. 2. Technology Assessment, Biomedical. 3. Technology, Medicalstandards. WE 365 I59a] RA445.I58 1985 610'.28 85-15284 ISBN 0-309-03583-X Copyright (D 1985 by the National Academy of Sciences No part of this book may be reproduced by any mechanical, photographic, or electronic process or in the form of a phonographic recording, nor may it be stored in a retrieval system, transmitted, or otherwise copied for public or pri- vate use, without written permission from the publisher, except for the purposes of official use by the United States Government. Printed in the United States of America

OCR for page R1
Committee for Evaluating Medical Technologies in Clinical Use FREDERICK MOSTELLER Roger I. Lee Professor, Harvard School of Public Health, Boston, Massachusetts (Chairman) H. DAVID BANTA Deputy Director, Pan American Health Organization, Washington, D.C. STUART BONDURANT Professor of Medicine and Dean, University of North Carolina School of Medicine, Chapel Hill, North Carolina MORRIS F. COLLEN Former Director, Technology Assessment, Department of Medical Methods Research, Kaiser Permanente Medical Care Program, Oakland, California JOANNE E. FINLEY Fellow, Department of Psychiatry, University of Pennsylvania Graduate Hospital Division (Former New Jersey State Commissioner of Health) BARBARA I. McNEIL Professor of Radiology and Clinical Epidemiology, Harvard Medical School, and Director, Center for Cost-Effective Care, Brigham and Womens Hospital Boston, Massachusetts LAWRENCE C. MORRIS, JR. Senior Vice President, Health Benefits Management, Blue Cross and Blue Shield Association, Chicago, Illinois LINCOLN E. MOSES Professor of Statistics, Stanford University, Stanford, California . SEYMOUR PERRY Senior Fellow and Deputy Director, Institute for Health Policy Analysis Georgetown Medical Center, Washington, D.C. DOROTHY P. RICE Regent's Lecturer, Department of Social and Behavioral Sciences, School of Nursing, University of California, San Francisco, California HERMAN A. TYROLER Alumni Distinguished Professor, Department of Epidemiology, School of Public Health, University of North Carolina, Chapel Hill, North Carolina DONALI) A. YOUNG Executive [Director' Prospective Payment Assessment Commission, Washington, D.C. Study Staff ENRIQUETA C. BOND, Study Director and Director, Division of Health Promotion and Disease Prevention BARBARA FILNER, Director, Division of Health Sciences Policy CAREN CARNEY, Research Assistant CLIFFORD GOODMAN, National Research Council Fellow LINDA DePUGH, Administrative Secretary NAOMI HUDSON, Administrative Secretary WALLACE K. WATERFALL, Editor

OCR for page R1

OCR for page R1
Acknowlecigments As with many reports issued by the Institute of Medicine (IOM), this report rep- resents the collaborative efforts of the committee, a number of TOM members, the project staff, anti many persons outside the TOM. The individuals listen] below drafted sections of the report, wrote background] papers for committee consider- ation, reviewed (lrafts, submitted suggestions for improving technology assess- ment, and, in general, gave their active assistance and collaboration. In Appendix B are collected a number of papers written or contributed to the project as further- ing our undertaking of technology assessment. We wish especially to express our gratitude to National Research Council Fellow, Clifford Goodman, who has worked closely with the committee for the past 2 years. The major contributors to each chapter are further identified in appropriate sections of the report. John Bailar, Harvard School of Public Health John Ball, American College of Physicians Benjamin Barnes, Harvard School of Public Health *Jeremiah A. Barondess, Cornell University Medical College *Lionel M. Bernstein, Knowledge Systems, Inc. Jonathan Brown, Harvard School of Public Health *Thomas C. Chalmers, Mount Sinai Medical Center David Cohen, Cleveland Metropolitan General Hospital *Jerome R. Cox, Jr., St. Louis School of Engineering and Applied Sciences, Washington University Betty L. Dooley, Center for Health Policy Studies Jack Dunn, Employee Benefit Specialist, Ford Motor Company * Denotes TOM member v

OCR for page R1
*David Eddy, Duke University *Richard H. Egdahl, Boston University Medical Center Penny H. Feldman, Harvard School of Public Health *Harvey V. Fineberg, Harvard School of Public Health G.D. Friedman, The Permanente Medical Group Clifton R. Gaus, Foundation for Health Services Research George Greenberg, U.S. Department of Health and Human Services Paul F. Griner, The Henry J. Kaiser Family Foundation *Ruth S. Hanft, Consultant Richard J. Havlik, National Heart, Lung, and Blood Institute Paul Jones, Case Western Reserve University School of Medicine *Albert R. Jonsen, University of California, San Francisco Edward H. Kass, Harvard Medical School Jeffrey P. Koplan, Centers for Disease Control John Laszlo, Duke University Medical Center *Robert H. Moser, American College of Physicians *Duncan B. Neuhauser, Case Western Reserve University School of Medicine Joel J. Nobel, ECRI (formerly the Emergency Care Research Institute) *Gerald T. Perkoff, University of Missouri-Columbia School of Medicine Michael R. Reich, Harvard School of Public Health Janet Reis, Northwestern University Center for Health Services and Policy Research Stanley J. Reiser, The University of Texas Health Science Center at Houston Pierre F. Renault, National Institutes of Health J. Sanford Schwartz, University of Pennsylvania *Sam Shapiro, The Johns Hopkins University Donald Shepard, Harvard School of Public Health Herbert Sherman, Harvard School of Public Health Jonathan Showstack, University of California, San Francisco *Reuel A. Stallones, University of Texas Health Science Center at Houston C. Frank Starmer, St. Louis School of Engineering and Applied Sciences, Washington University Seymour Sudman, University of Illinois Bruce Vladeck, United Hospital Fund of New York Milton C. Weinstein, Harvard School of Public Health *Frank Young, Food and Drug Administration $ Denotes TOM member V1

OCR for page R1
Preface Twentieth century advances in scientific knowledge have been responsible for profound improvements in human health. Many diseases have been eradicates! and others now can be successfully treated or prevented. However, new technolo- gies ant] procedures have cleveloped so rapidlyand there are such economic and social incentives to use them that the evaluation of their safety, efficacy, and cost-effectiveness as well as the consideration of their social and ethical conse- quences has lagger] far behincl. This situation does not serve the patient, the physi- cian, or our society well, and there is an increasing conviction that we need a coor- dinatecI system for medical technology assessment and a national program of support for it. This stucly was the outgrowth of a 1980 Institute of Medicine conference on linking the clinical use of biomedical technologies and the collection of evaluative data. * The conferees considered] several methods for evaluating biomedical tech- nologies and for applying the information obtained to physician eclucation, clini- cal practice, resource allocation, quality assurance, and reimbursement. They also consiclered such issues as the funding of evaluative research and translating re- search findings on safety, efficacy, and cost-effectiveness into policy ant} practice. This study was undertaken to address in greater detail the following questions raised at the conference: What are the strengths and limits of methods for technology assessment ant] how can it be improved? This question is primarily considered in Chapter 3 of the report. Useful methods do exist for assessing medical technologies. * Supported by the Charles H. Revson Foundation. ~ V11

OCR for page R1
.. V111 PREFACE How does knowledge from technology assessment translate into better clini- cal care? This issue is discussed in Chapter 4, which indicates that we have much to learn in this area. What gaps exist in the current system for technology assessment? Chapter 2 explores the terrain of technology assessment, identifying who is cloing it, how much they are spending on it, ant! for what purposes. These considerations are carrier] further in Chapters 6 and 7. Who shouIc3 pay for technology assessment? The stage for this important question is set in Chapters 2 ant! 5. The question is also examined in Chapter 7, along with the recommendations of the study committee. PROCEDURE FOR THE STUDY Primary responsibility for conclucting the stucly was vested in a committee of 12 experts in the fields of technology assessment, biostatistics, epidemiology, public health policy, clinical practice, and thir(l-party payment mechanisms. Committee members defined the task more precisely in preliminary meetings and then en- rollec] the assistance of outside experts in crafting materials for their review ant] incorporation into the report. Committee members, staff, ant] outside experts worked collaboratively to complete the report. Dr. Enriqueta Bond lee] the staff, organized much of the committee's work, and tirelessly wrote parts of ant} contributes] to the editing of the report. Clifforc] Goodman contributed much beyond the sections signed by him and wrote Appen- clix A to this book, which gives basic systematic information about many of the institutions performing technology assessment. Wallace Waterfall aided the com- mittee enormously by editing a massive manuscript flowing from many hands. Without the excellent secretarial assistance of Naomi Hudson and Linda DePugh, the book wouIcI never have been completed. AUDIENCE FOR THE STUDY Various chapters of the report are expected to be of particular interest to one or another segment of the total audience. To facilitate the reacler's choosing his or her own path through the text, each chapter is intended to include enough informa- tion from the others to keep at hand the context of the whole. First, the book is a terrain map of medical technology assessment that can serve as a textbook for stu- dents and educators. Second, researchers and funding agencies can find a research agenda cleveloped throughout the book, but especially in Chapters 3, 4, and 7. Next, those who carry out ant] use the results of technology assessment will find value in the scope of technology assessment described in Chapter 2 and Appendix A, the methods identified in Chapter 3, and recommendations for developing a better system of technology assessment outlined in Chapter 7. Policymakers will also be interested in these recommendations as well as the Summary, Chapter 2, ant] Appendix A. Finally, the book should serve as a resource to the Institute of

OCR for page R1
PREFACE i: Medicine in its efforts to contribute to the nation's system for technology assess- ment by the establishment of a Council for Technology as definer! by P.~. 98-551 . Recommendations in this report are of three types. Most of the individual chap- ters include recommendations very specific to the chapter topics. Chapter 7 has recommendations pertinent to establishing a system of technology assessment. And the text in several chapters has sentences that are highlighter] in boiciface type as contributions to a research agenda for technology assessment. Appendix B of this report presents a selection from among the many papers pre- parec] for the committee. Paper titles and authors are as follows: Guicle to Com- parative Clinical Trials, by Clifford S. Goodman; Information Needs for Technol- ogy Assessment, by Morris F. Collen; Towarc! Evaluating Cost-Effectiveness of Medical and Social Experiments, by Frederick Mosteller and Milton C. Weinstein; Technology Assessment in Prepaic] Group Practice, by Morris F. Collen; A Ran- domizec] Controlled Trial to Evaluate the Effects of an Experimental Prepaid Group Practice on Medical Care Utilization and Cost, by Gerald T. Perkoff; The Metro Firm Trials: An Innovative Approach to Ongoing Ranclomizecl Clinical Trials, by David I. Cohen and Duncan Neuhauser; Values and Preferences in the Delivery of Health Care, by Barbara J. McNeil; New Fecleralism and State Sup- port of Technology Assessment, by George D. Greenberg ant! Penny H. Feldman; ant] Government Payers for Health Care, by Donald A. Young. FREDER~cK Most Chairman .x

OCR for page R1

OCR for page R1
Contents SUMMARY .................................... The Scope of U.S. Medical Technology Assessment, 2 Varieties and Expense of Assessment, 3 Federal Government, 3; Drug Industry, 3; Medical Device Industry, 4; Other Private Sector Assessment Activities, 4 Lags in Assessment, 5 Recommendations, 6 MethocIs of Medical Technology Assessment, 6 Effects of Evaluation on Diffusion of Technology, Diffusion and Its Determinants, 3 Types of Evaluation That Precede Accepted Medical Practice, Recommendations, 9 Reimbursement and Technology Assessment, 9 Assessment in the New Era of Cost Containment, 10 Paying for Technology Assessment, ~ O Recommendations, TI Medical Technology Assessment AbroacI, IT Recommendations, 12 Conclusions ant] Recommendations, 13 The Challenge, 13 Key Functions for Assessment, 13 Builcling a System, 14 Financing, 14 Recommendations, 14 X1

OCR for page R1
~ X11 1. INTRODUCTION Polio, 17 Surgical Procedure, 17 Examples of Technology Assessment, 19 Electronic Fetal Monitoring, 19 Computed Tomography Scanning, 20 Drug Treatment for Hypertension, 21 Hysterectomy, 22 Medical Information System, 23 Some Lessons from the Examples, 24 Comprehensive Technology Assessment, 25 Different Parties, Different Aims in Assessments, 26 References, 30 2. THE SCOPE OF U.S. MEDICAL TECHNOLOGY ASSESSMENT An Overview, 32 Varieties of Medical Technology Assessment, 34 National Expenditures for Health Research and Development, Clinical Trials, and Technology Assessment, 37 Federal Government, 39 National Institutes of Health, 40 Office of Medical Applications of Research, 41 Food and Drug Administration, 41 Office of Technology Assessment, 41 Prospective Payment Assessment Commission, 42 Health Care Financing Administration, 42 National Center for Health Services Research and Health Care Technology Assessment, 43 Office of Health Technology Assessment, 44 National Center for Health Statistics, 44 Veterans Administration, 45 Centers for Disease Control, 45 Department of Defense, 45 Drug Inclustry, 46 Drug Industry Research and Development, 46 Assessment of Drugs, 47 Expenditures for Drug Assessment, 43 Medical Device Industry, 49 Medical Device Industry Research and Development, 49 Assessment of Medical Devices, 50 Expenditures for Medical Device Assessment, 52 Other Private Sector Assessment Activities, 53 Insurers, 53 Medical Associations, 54 CONTENTS . . . . . .16 .32

OCR for page R1
CONTENTS Policy Research Groups, 55 Industry Associations, 55 Provider Institutions, 56 Academic Institutions, 58 Employers, 59 Conclusions, 59 Recommendations, 61 Assessment Concerns, 61 Coordination of Assessment Information, 61 Responsibility for Conduct of Assessments, 61 Evaluation of Assessment Programs, 62 Financial Support for Medical Technology Assessment, 62 Amount of Support, 62; Sources of Support, 64; A Worthy Investment, 64 Notes, 64 References, 66 3. METHODS OF TECHNOLOGY ASSESSMENT Randomizec! Clinical Trials, 73 Practical Problems of Comparing Treatments, 74 The Protocol, 74 Random Assignment to Treatment, 75 Limitations of RCTs, 76 Strengthening RCTs, 78 A Final Remark, 79 Evaluating Diagnostic Technologies, 80 Present Evaluation Methods, 80 Limitations of Present Evaluation Methods, 85 Strengthening the Method, 87 Summary, 88 The Series of Consecutive Cases as a Device for Assessing Outcomes of Intervention, 89 Description of a Series, 90 Needed Information, 90 Adjustment for Interfering Variables, 90 Capabilities and Limitations, 91 Interpretation of Series, 92 Integrity of Counting, 92; Consequences of No Protocol, 92; Consequences of No Randomization, 93 What About the Clear-Cut Series?, 94 Some Additional Issues, 95 Subgroups, 95; Temporal Drift, 96; Grab Samples, 96 Strengthening the Method, 97 ~ x~t .70

OCR for page R1
XIV CONTENTS The Case Study as a Too! for Meclical Technology Assessment, 97 What Is a Case Study?, 97 Case Studies from the Office of Technology Assessment, 99 Weaknesses of Case Studies, 100 Strengths of Case Studies, 100 Strengthening Case Studies, 101 Conclusions, 101 Registers and Data Bases, 101 Uses of Registers and Data Bases, 104 Capabilities and Limitations of Registers and Data Bases, ~ 06 Strengthening Uses of Registers and Data Bases, 109 Sample Surveys, 109 Uses of Sample Surveys, Il0 NCHS Surveys, Il0 Health Care Financing Administration Data, ~13 Capabilities ant! Limitations of Sample Surveys, Il4 Strengthening Uses of Sample Surveys, Il5 Epiclemiologic Methods, Il6 Uses of Epidemiologic Methods, Il7 Cohort Studies, 117; Case-Control Studies, 118; Cross-Sectional Studies, 118 Capabilities and Limitations of Epidemiologic Methods, Il9 Strengthening Uses of Epidemiologic Methods, Il9 Surveillance, 120 Uses of Surveillance, 121 Capabilities and Limitations of Surveillance, 123 Strengthening Uses of Surveillance, 124 Quantitative Synthesis Methods Meta-Analysis, 125 Voting Methods, 126 Cumulation of Significance Levels (p-Values), 127 Other Statistical Synthesis Procedures, 127 Example of Synthesis, 128 Group Judgment Methods, 129 Formal Group Methods, 130 Delphi Technique, 130; Nominal Group Technique, 131 Group Methods Designed for Health Issues, 131 NIH Consensus Development Conferences, 131; State-of-the-Art Diagnosis and Care of COPD, 133; Medical Practice Information Project, 133; Rand-UCLA Health Services Utilization, 134; Computer Knowledge Bases, 134 Strengthening Group Judgment Efforts, 135 Cost-Effectiveness and Cost-Benefit Analyses, 136 Uses of CEA/CBA, 140 Capabilities and Limitations, 141

OCR for page R1
CONTENTS Strengthening Use of CEA/CBA, 143 Recommendations, 144 Technology Assessment: The Role of Mathematical Modeling, 144 Background, 145 An Example, 146 The Role of Moclels, 146 Mental Models, 146; Mathematical Models, 147 Uses of Mathematical Models in Technology Assessment, 147 Estimating Outcomes, 147; Additional Uses, 148 Types of Mathematical Models, 148 Validation of Mathematical Models, 149 Limitations, 150 Strengthening the Technique, 152 Conclusion, 153 Social ant] Ethical Issues in Technology Assessment Experience in Aciciressing Ethical Issues, 157 Ethics of Investigation, 158 Initiation, 158; Conduct, 158; Termination, 158 ., 154 Conclusions and Recommendations, 159 Appendix 3-A: Example of Cost-Benefit Analysis, 160 Appendix 3-B: An Example of a Mathematical Moclel of Medical Technology, 164 References, 166 4. EFFECTS OF CLINICAL EVALUATION ON THE DIFFUSION OF MEDICAL TECHNOLOGY....... Evaluation, 176 Diffusion, 177 Determinants of Diffusion, 178 Prevailing Theory, 178; The Innovation, 178; The Clinical Situation, 178; Advocacy, 178; The Potential Adopter, 179; The Practice Setting, 179; The Decision-Making Process, 179; Environmental Constraints and Incentives, 179; Evaluation and Methods of Evaluation, 179; Channels of Communication, 180 Measures of Diffusion and Sources of Data, 181 Diffusion as Affected by Evaluation, 182 Evidence About the Effects of Evaluation on Diffusion, 185 Evaluation and Physician Behavior, 185 The Impact of Primary Evaluations on Physicians, 185; Effects of Clinical Trials on Physician Awareness, 185; Effects of RCTs on Physician Practices, 187; Types of Evaluation That Precede Accepted Medical Practices, 193; The Impact of Synthetic Evaluations, 194 Evaluation and Policy on Regulation and Reimbursement, 195 Conclusions and Recommendations, 196 xv . ... ...176

OCR for page R1
XVI Appendix 4-A: Summaries of Studies, 198 References, 207 REIMBURSEMENT AND TECHNOLOGY ASSESSMENT Assessment in the Era of Retrospective Payments, 212 The Need for Assessment in the New Era of Cost Containment, 214 Leverage of the Reimbursement System over Institutions, 215 Capital Acquisition, 216 Stimulus for Technology Assessment in Reimbursement Policy, 217 DRG Price Development and Technology Assessment, 217 Cost-Containment Information: Feedback for Change, 219 Mechanisms to Promote Appropriate Use of Technology, 220 Paying for Technology Assessment, 221 Conclusions and Recommendations, 224 Appendix 5-A: Legal Considerations in Private Health Insurer's Identification and Exclusion of Unnecessary, Incompetent, and Unproved Services, 224 References, 226 6. MEDICAL TECHNOLOGY ASSESSMENT IN DEVELOPED COUNTRIES: TRENDS AND OPPORTUNITIES FOR COLLABORATION Trends in Assessment and Regulation, 229 Research and Development, 230 Clinical Trials, 231 Consensus Activities, 232 Use of Technology in Different Countries, 232 Assessment of Drugs in Different Countries, 234 France, 235 West Germany, 236 Health Care Systems in Different Countries, 237 Britain, 237 Japan, 238 International Organizations, 239 International Pharmaceutical Firms, 239 International Health Information, 239 World Health Organization Network on Technology Assessment, 240 Postmarketing SurveilIanee of Drugs, 240 Conclusions and Recommendations, 241 References, 242 CONTENTS .211 .228

OCR for page R1
CONTENTS 7. CONCLUSIONS AND RECOMMENDATIONS The Challenge, 246 Key Functions Needing Improvement, 246 :Information Monitoring ant] Acquisition, 246 Combining Information from Different Sources, 247 Dissemination of Tnformation, 247 Identification of Gaps in Knowledge That Require Research, 247 Data and Tnformation Acquisition, 247 Priority Setting, 247 Manpower for Technology Assessment, 248 Research ant] Development of Methods for Assessment, 248 Builcling a System, 249 Advantages and Disadvantages of Various Institutional Arrangements, 249 Private-Public Body, 249 Reestablish the NCHCT, 250 Development of a System by the DHHS Secretary, 250 A Separate Federal Agency, 250 Not the Regulatory Approach, 251 Financing, 252 Recommendations, 252 Closing Comment, 253 References, 254 APPENDIX A Profiles of 20 Technology Assessment Programs APPENDIX B Selected Papers ........ Guicle to Comparative Clinical Trials, 490 Tnformation Needs for Technology Assessment, 502 Toward Evaluating Cost-Effectiveness of Medical ant] Social Experiments, 506 Technology Assessment in Prepaic} Group Practice, 515 A Randomized Controller] Trial to Evaluate the Effects of an Experimental Prepaic] Group Practice on Meclical Care Utilization and Cost, 525 The Metro Firm Trials: An Innovative Approach to Ongoing Randomized Clinical Trials, 529 Values and Preferences in the Delivery of Health Care, 535 New Federalism ant] State Support for Technology Assessment, 542 Government Payers for Health Care, 554 INDEX . ~ XV11 .244 .255 490 .565

OCR for page R1