Ranking
Vaccines
A Prioritization Framework
Phase I: Demonstration of Concept and a Software Blueprint
Committee on Identifying and Prioritizing
New Preventive Vaccines for Development
Board on Population Health and Public Health Practice
Board on Global Health
Guruprasad Madhavan, Kinpritma Sangha, Charles Phelps,
Dennis Fryback, Tracy Lieu, Rose Marie Martinez, and
Lonnie King, Editors
INSTITUTE OF MEDICINE
OF THE NATIONAL ACADEMIES
THE NATIONAL ACADEMIES PRESS
Washington, D.C.
www.nap.edu
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page R1
Committee on Identifying and Prioritizing
New Preventive Vaccines for Development
Board on Population Health and Public Health Practice
Board on Global Health
Guruprasad Madhavan, Kinpritma Sangha, Charles Phelps,
Dennis Fryback, Tracy Lieu, Rose Marie Martinez, and
Lonnie King, Editors
OCR for page R1
THE NATIONAL ACADEMIES PRESS 500 Fifth Street, NW Washington, DC 20001
NOTICE: The project that is the subject of this report was approved by the Gov-
erning Board of the National Research Council, whose members are drawn from
the councils of the National Academy of Sciences, the National Academy of Engi-
neering, and the Institute of Medicine. The members of the committee respon-
sible for the report were chosen for their special competences and with regard
for appropriate balance.
This study was supported by Contract No. HHSP23337024T, TO #45 between
the National Academy of Sciences and the National Vaccine Program Office of the
Department of Health and Human Services. Any opinions, findings, conclusions,
or recommendations expressed in this publication are those of the author(s) and
do not necessarily reflect the view of the organizations or agencies that provided
support for this project.
International Standard Book Number-13: 978-0-309-25525-7
International Standard Book Number-10: 0-309-25525-2
Additional copies of this report are available from the National Academies Press,
500 Fifth Street, NW, Keck 360, Washington, DC 20001; (800) 624-6242 or (202)
334-3313; www.nap.edu.
For more information about the Institute of Medicine, visit the IOM home page
at: www.iom.edu.
Copyright 2012 by the National Academy of Sciences. All rights reserved.
Printed in the United States of America
The serpent has been a symbol of long life, healing, and knowledge among almost
all cultures and religions since the beginning of recorded history. The serpent
adopted as a logotype by the Institute of Medicine is a relief carving from ancient
Greece, now held by the Staatliche Museen in Berlin.
Suggested citation: IOM (Institute of Medicine). 2012. Ranking vaccines: A pri-
oritization framework: Phase I: Demonstration of concept and a software blueprint.
Washington, DC: The National Academies Press.
OCR for page R1
“Knowing is not enough; we must apply.
Willing is not enough; we must do.”
— Goethe
Advising the Nation. Improving Health.
OCR for page R1
The National Academy of Sciences is a private, nonprofit, self-perpetuating
society of distinguished scholars engaged in scientific and engineering research,
dedicated to the furtherance of science and technology and to their use for the
general welfare. Upon the authority of the charter granted to it by the Congress
in 1863, the Academy has a mandate that requires it to advise the federal govern-
ment on scientific and technical matters. Dr. Ralph J. Cicerone is president of the
National Academy of Sciences.
The National Academy of Engineering was established in 1964, under the
charter of the National Academy of Sciences, as a parallel organization of out-
standing engineers. It is autonomous in its administration and in the selection of
its members, sharing with the National Academy of Sciences the responsibility
for advising the federal government. The National Academy of Engineering also
sponsors engineering programs aimed at meeting national needs, encourages
education and research, and recognizes the superior achievements of engineers.
Dr. Charles M. Vest is president of the National Academy of Engineering.
The Institute of Medicine was established in 1970 by the National Academy of
Sciences to secure the services of eminent members of appropriate professions
in the examination of policy matters pertaining to the health of the public. The
Institute acts under the responsibility given to the National Academy of Sciences
by its congressional charter to be an adviser to the federal government and, upon
its own initiative, to identify issues of medical care, research, and education. Dr.
Harvey V. Fineberg is president of the Institute of Medicine.
The National Research Council was organized by the National Academy of
Sciences in 1916 to associate the broad community of science and technology
with the Academy’s purposes of furthering knowledge and advising the federal
government. Functioning in accordance with general policies determined by the
Academy, the Council has become the principal operating agency of both the
National Academy of Sciences and the National Academy of Engineering in pro-
viding services to the government, the public, and the scientific and engineering
communities. The Council is administered jointly by both Academies and the
Institute of Medicine. Dr. Ralph J. Cicerone and Dr. Charles M. Vest are chair and
vice chair, respectively, of the National Research Council.
www.national-academies.org
OCR for page R1
Committee on Identifying and Prioritizing
New Preventive Vaccines for Development
LONNIE KING (Chair), Dean, College of Veterinary Medicine, and
Executive Dean, Health Sciences College, Ohio State University
PAUL CITRON, Retired Vice President, Technology Policy and Academic
Relations, Medtronic, Inc.
RITA COLWELL, Chair, Canon U.S. Life Sciences, Inc.; Distinguished
Professor, University of Maryland, College Park and Johns Hopkins
University Bloomberg School of Public Health; and Former Director,
National Science Foundation
KATHRYN EDWARDS, Sarah H. Sell Professor of Pediatrics, Vanderbilt
University School of Medicine
JOSHUA EPSTEIN, Professor of Emergency Medicine, Johns Hopkins
University, and External Professor, Santa Fe Institute
DENNIS FRYBACK, Professor Emeritus of Population Health Sciences
and Industrial Engineering, University of Wisconsin–Madison
PATRICIA GARCIA, Dean and Professor, School of Public Health,
Cayetano Heredia University, Peru
DEMISSIE HABTE, President, Ethiopian Academy of Sciences, Ethiopia
VICTORIA HALE, Founder and Chief Executive Officer, Medicines 360
TRACY LIEU, Professor, Department of Population Medicine, Harvard
Medical School
WILLIAM PAUL, NIH Distinguished Investigator and Chief, Laboratory
of Immunology, National Institute of Allergy and Infectious Diseases,
National Institutes of Health
CHARLES PHELPS, University Professor and Provost Emeritus,
University of Rochester
RINO RAPPUOLI, Global Head, Vaccines Research, Novartis Vaccines,
Italy
ARTHUR REINGOLD, Edward Penhoet Distinguished Professor of
Global Health and Infectious Diseases, University of California,
Berkeley
VINOD SAHNEY, Senior Fellow, Institute for Health Care Improvement
ROBERT STEINGLASS, Immunization Team Leader, Maternal and
Child Health Integrated Program, John Snow, Inc.
Staff
GURUPRASAD MADHAVAN, Study Director
KINPRITMA SANGHA, Research Associate
MALCOLM BILES, Senior Program Assistant
v
OCR for page R1
HOPE HARE, Administrative Assistant
DORIS ROMERO, Financial Associate
ROSE MARIE MARTINEZ, Senior Director, Board on Population
Health and Public Health Practice
PATRICK KELLEY, Senior Director, Board on Global Health
KATHLEEN STRATTON, Scholar (until September 2011)
AMY PRYZBOCKI, Financial Associate (until December 2011)
SAMANTHA ARNETT, Christine Mirzayan Science and Technology
Policy Fellow (Winter 2011)
Consultants, Modeling and Software Development
SCOTT LEVIN, Assistant Professor of Emergency Medicine, Johns
Hopkins University
MATTHEW TOERPER, Senior Software Engineer, Johns Hopkins
University
PANAYIOTIS KARABETIS, Partner and Lead Information Designer,
VIM Interactive
MICHAEL KAPETANOVIC, Partner and Project Manager, Reef Light
Interactive
Consultants, Concept Evaluation
JON ANDRUS, Deputy Director, Pan American Health Organization
CLAIRE BROOME, Adjunct Professor, Department of Global Health,
Rollins School of Public Health, Emory University
JOACHIM HOMBACH, Acting Head, Initiative for Vaccine Research,
World Health Organization
PHILIP HOSBACH, Vice President of Immunization Policy and
Government Relations, Sanofi Pasteur
ROBERT LAWRENCE, Center for a Livable Future Professor, and
Professor of Environmental Health Sciences, Health Policy, and
International Health, John Hopkins Bloomberg School of Public
Health
ADEL MAHMOUD, Professor, Woodrow Wilson School of Public and
International Affairs and Department of Molecular Biology, Princeton
University; Former President, Merck Vaccines
GREGORY POLAND, Mary Lowell Leary Professor of Medicine and
Director, Mayo Vaccine Research Group, Mayo Clinic
JAIME SEPULVEDA, Executive Director, Global Health Sciences,
University of California, San Francisco
EDWARD SHORTLIFFE, President and Chief Executive Officer,
American Medical Informatics Association; Adjunct Professor,
Department of Biomedical Informatics, Columbia University
ALASTAIR WOOD, Partner, Symphony Capital, LLC
vi
OCR for page R1
Reviewers
This report has been reviewed in draft form by individuals chosen for their
diverse perspectives and technical expertise, in accordance with proce-
dures approved by the National Research Council’s Report Review Com-
mittee. The purpose of this independent review is to provide candid and
critical comments that will assist the institution in making its published
report as sound as possible and to ensure that the report meets institutional
standards for objectivity, evidence, and responsiveness to the study charge.
The review comments and draft manuscript remain confidential to protect
the integrity of the deliberative process. We wish to thank the following
individuals for their review of this report:
Yves Bergevin, United Nations Population Fund
John Boslego, PATH
Margaret Brandeau, Stanford University
Michael Drummond, University of York, United Kingdom
Maria Freire, The Albert and Mary Lasker Foundation
Maryellen Giger, The University of Chicago
Robert Greenes, Arizona State University
Prasad Kulkarni, Serum Institute of India, Ltd.
Shabir Madhi, University of Witwatersrand, South Africa
Charles Manski, Northwestern University
Osman Mansoor, United Nations Children’s Fund
Regina Rabinovich, Bill & Melinda Gates Foundation
John Robbins, National Institutes of Health
Thomas Saaty, University of Pittsburgh
Jeanne Stephenne, GlaxoSmithKline Biologicals, Belgium
Detlof von Winterfeldt, University of Southern California
Although the reviewers listed above have provided many construc-
tive comments and suggestions, they were not asked to endorse the con-
vii
OCR for page R1
viii Reviewers
clusions or recommendations, nor did they see the final draft of the report
before its release. The review of this report was overseen by Stephen Fien-
berg, Carnegie Mellon University, and Alfred Berg, University of Wash-
ington School of Medicine. Appointed by the National Research Council
and the Institute of Medicine, they were responsible for making certain
that an independent examination of this report was carried out in accor-
dance with institutional procedures and that all review comments were
carefully considered. Responsibility for the final content of this report rests
entirely with the authoring committee and the institution.
OCR for page R1
Foreword
Ranking Vaccines: A Prioritization Framework previews a product that is
unique in the annals of the Institute of Medicine: an early-stage decision-
support software for prioritizing new vaccines.
Decision makers in the area of vaccine development—including
developers, investors, practitioners, and policy makers—are constantly
challenged by rapidly changing demographics, epidemiology, econom-
ics, technologies, and health systems. Thus, a comprehensive yet adapt-
able framework is needed to assist decision making. The Strategic Multi-
Attribute Ranking Tool for Vaccines, or SMART Vaccines, described in this
report, provides one such framework.
SMART Vaccines was conceived with the appreciation that chang-
ing circumstances, technological developments, and resource availability
influence priorities for new vaccines. This tool should make it possible
for decision makers in a variety of circumstances to weigh competing val-
ues, test assumptions, and explore alternative scenarios to help guide the
priority-setting process. Like all decision tools, SMART Vaccines is an aid
for decision making, not a substitute for sound judgment.
Beyond its potential applications in independent and collaborative
decision making, SMART Vaccines can facilitate focused and informed dis-
cussion among various stakeholders. In this role, it can provide a common
platform for diverse constituents to arrive at mutually agreeable priorities
and help foster collaborations among them. In addition, SMART Vaccines
is being designed so that it can be adapted and configured to help set priori-
ties related to health interventions other than vaccines.
We intend the initial prototype to serve as a springboard to further
development. With iterative enhancements, SMART Vaccines should
become a dynamic, living guide that can be applied both domestically and
internationally and reapplied according to changing health needs, scien-
tific knowledge, and financial constraints.
ix
OCR for page R1
x Foreword
I congratulate the members and staff of the Committee on Identify-
ing and Prioritizing New Preventive Vaccines for Development for leading
this exciting initiative and bringing the project to this promising stage of
development.
Harvey V. Fineberg, M.D., Ph.D.
President, Institute of Medicine
OCR for page R1
Preface
Vaccines have profoundly improved the practice and the quality of public
health. New opportunities for developing or improving vaccines are prom-
ising, even exciting, in this “decade of vaccines.”
However, designing a national and global vaccine development strat-
egy is a Herculean task. Such an effort would involve a concrete, crosscut-
ting understanding of the health, demographic, economic, business, scien-
tific, technological, policy, social, and operational dimensions of vaccines.
The first step toward tackling this complex mission will be to pri-
oritize which vaccines most need to be developed for both domestic and
international use. This is a basic task but not an easy one, as the resulting
decisions may have significant health, economic, and global consequences.
Unfortunately, no universally accepted method or model exists to help
guide these important decisions.
To make progress in this area, the Institute of Medicine, at the request
of the National Vaccine Program Office of the U.S. Department of Health
and Human Services, created a 16-member Committee on Identifying and
Prioritizing New Preventive Vaccines for Development. A central commit-
ment of the committee was to ensure that stakeholders were significantly
involved in informing the work and the deliberations of the committee.
As part of fulfilling its charge, the committee developed and tested a
model designed to assist in the prioritization of new vaccines. The commit-
tee also prototyped the beta version of a software named Strategic Multi-
Attribute Ranking Tool for Vaccines, or SMART Vaccines. This is a unique
product within the National Academies and is expected to be an evolving
tool.
In this report we describe the committee’s thought process and mod-
eling strategy, and introduce the software blueprint of SMART Vaccines
Beta through illustrative screenshots. Since this is a work in progress and
subject to additional improvements, we have chosen not to release SMART
xi
OCR for page R1
xii Preface
Vaccines Beta along with this report. Further work in the next phase of this
study is expected to result in SMART Vaccines 1.0, which would be made
available for public use.
Through this effort we hope to inspire a community of users who will
improve, enhance, and potentially manage the capabilities of this product
in an open-source environment and who will generate the required data for
operating a multi-stakeholder vaccine prioritization software.
On behalf of the committee, I would like to thank a number of indi-
viduals and organizations who gave their time, advice, and expertise to our
work.
The committee is indebted to the Institute of Medicine study staff,
whose diligence, creativity, and excellent organizational skills were criti-
cal to our success. The committee gratefully acknowledges the outstanding
work of Guru Madhavan, the study director; the invaluable contributions
of Kinpritma Sangha, our research associate; and the able administrative
assistance from Malcolm Biles.
We recognize Rose Marie Martinez, director of the Board on Pop-
ulation Health and Public Health Practice; Patrick Kelley, director of the
Board on Global Health; and Kathleen Stratton, who skillfully led previous
Institute of Medicine studies on vaccines, for their thoughtful insights. We
deeply appreciate the wise counsel of Clyde Behney, deputy executive offi-
cer of the Institute of Medicine, and Marc Gold, associate general counsel
of the National Academy of Sciences, as well as the assistance of other staff
members throughout this project.
The committee is very appreciative of our modeling consultants,
Scott Levin and Matthew Toerper from the Johns Hopkins University, and
our software developers, Pete Karabetis of VIM Interactive and Michael
Kapetanovic of Reef Light Interactive. The committee also thanks Robert
Pool, Laura DeStefano, and Hannan Braun for their terrific editorial assis-
tance and Samantha Arnett, the National Academies’ Christine Mirzayan
Science and Technology Policy Fellow, for her research assistance.
Our special thanks go to Jon Andrus, Claire Broome, Joachim Hom-
bach, Philip Hosbach, Robert Lawrence, Adel Mahmoud, Gregory Poland,
Jaime Sepulveda, Edward Shortliffe, and Alastair Wood whose thoughtful
comments and critical feedback during our concept evaluation sessions
have helped us improve SMART Vaccines Beta.
Finally, we would like to thank the National Vaccine Program Office
of the Department of Health and Human Services for its sponsorship, sup-
port, and encouragement.
Lonnie King, Chair
July 2012
OCR for page R1
Contents
DISCLAIMER xvii
SUMMARY 1
1 INTRODUCTION: FROM SMALLPOX TO
SMART VACCINES 11
Study Scope and Process, 14
Vaccine Market Dynamics, 16
Prioritization Efforts in the Vaccine Enterprise, 17
Capturing the Value of Vaccination, 21
Previous IOM Efforts in Vaccine Prioritization, 22
Previous WHO Efforts in Vaccine-Related Prioritization, 23
New Technologies and Development Strategies, 24
Stakeholder Priorities, 26
2 MODELING STRATEGY: FROM SINGLE ATTRIBUTE
TO MULTIPLE ATTRIBUTES 27
New Vaccine Development (1985–1986) and Vaccines for the
21st Century (2000), 27
Modeling Beyond Cost-Effectiveness, 29
Values and Objectives in Priority Setting, 29
Multi-Criteria Decision-Making Methods, 30
Axiomatic Foundation, 31
Priority Scaling, 31
Sensitivity Analysis, 32
Transparency, 32
Mathematical Programming or Optimization, 33
Analytic Hierarchy Process, 34
Multi-Attribute Utility Theory, 35
xiii
OCR for page R1
xiv Contents
The Rank Order Centroid Approximation, 36
Data Demands, 37
The Modeling Framework for SMART Vaccines Beta, 37
Foundation for the Computational Submodel, 39
Foundation for the Value Submodel, 53
User Entries and Prioritization Categories, 56
Demographic Inputs, 56
Disease Epidemiology and Clinical Inputs, 57
Economic Inputs, 58
Vaccine Inputs, 58
Disease Burden Summary Measures, 59
Other Attributes, 60
Ranking Method, 60
The Meaning and Interpretation of Weights, 62
The Risk of Double Counting, 63
Discounting and Inflation, 65
Time Horizon and Uncertainty, 66
3 DATA EVALUATION AND SOFTWARE DEVELOPMENT 67
Selection of Vaccine Candidates, 67
Data Sourcing and Analysis, 68
SMART Vaccines Submodels, 73
Development of the Computational Submodel, 74
Evaluation of the Computational Submodel, 76
Simulation of the Value Submodel, 78
The Value Experiment and Scenarios, 84
Software Development: Operational Features of SMART
Vaccines Beta, 88
Step 0: Terms of Agreement, 88
Step 1: Values, 88
Step 2: Demographics, 90
Step 3: Disease Burden, 92
Step 4: Vaccines, 97
Step 5: Value Assessment, 102
Step 6: Value Assessment and Score, 103
Consideration of Uncertainty, 103
Uncertainty About the Likelihood of Successful Licensure, 103
Other Uncertainties, 105
Current Capability for Sensitivity Analysis, 107
Beta Concept Evaluation, 108
OCR for page R1
xv
Contents
4 OBSERVATIONS AND LOOKING FORWARD 109
Data Requirements, 109
Looking Ahead, 111
Model Attributes, 111
Model Evaluation, 112
Trade-Off Considerations, 112
Enhancing the Software Capabilities, 113
REFERENCES 115
APPENDIXES
A Mathematical Functions 121
B Candidate Disease Profiles and Data 127
C Stakeholder Speakers 153
D Biographical Information 155
OCR for page R1
OCR for page R1
Disclaimer
This report presents SMART Vaccines, a prioritization model and
blueprint of associated software in development. This work is being
developed by the Institute of Medicine Committee on Identifying
and Prioritizing New Preventive Vaccines for Development with the
assistance of consultants from Johns Hopkins University and VIM
Interactive. This report does not intend to actually provide a ranking
of vaccine priorities. It describes the committee’s modeling strategy
and assumptions in order to demonstrate a proof of concept.
This consensus study is being conducted in two phases. The
Phase I statement of task asked for a model to be developed that
prioritizes the development of new preventive vaccines, tested with
two or three vaccine candidates. In Phase II the committee will ob-
tain feedback from the stakeholders on the Phase I model and use it
to enhance SMART Vaccines in addition to adding three test vaccine
candidates. Thus this report describes a product that is purposefully
midstream in development.
The committee has chosen to employ a modeling approach
based on multi-attribute utility theory, supported by a computa-
tional engine and a user-friendly interface. SMART Vaccines Beta
processes available or expert-informed data for three conditions
(influenza, tuberculosis, and group B streptococcus) in two nations
(the United States and South Africa). Thus the examples that appear
in this report are limited to comparing hypothetical vaccines only.
SMART Vaccines is intended to serve only as a decision-support
tool for vaccine prioritization and not to be used as a decision mak-
er. Final decisions should not be made based on the scores provided
by SMART Vaccines. The Institute of Medicine does not warrant the
completeness of the model, the accuracy of the software in devel-
opment, or the reliability of any data presented in this report.
July 2012
xvii
OCR for page R1