National Academies Press: OpenBook
« Previous: 3 Data Synthesis, Software Redesign, and Evaluation
Suggested Citation:"4 Observations and Looking Forward." Institute of Medicine. 2013. Ranking Vaccines: A Prioritization Software Tool: Phase II: Prototype of a Decision-Support System. Washington, DC: The National Academies Press. doi: 10.17226/13531.
×

4

Observations and Looking Forward

Previous efforts in the realm of prioritizing new vaccines have been limited because of the small number of attributes and value preferences those models were able to accommodate. Moving from the traditional single-attribute modeling framework to a multi-attribute modeling framework involved substantial public input and a wide range of expertise in order to develop the computational model and the user interface that support SMART Vaccines.

As noted in the 2012 Institute of Medicine (IOM) report Ranking Vaccines: A Prioritization Framework (IOM, 2012) and earlier in this edition, the committee’s charge did not include the production of a “list” of vaccines for development. Instead, through this work the committee developed SMART Vaccines as a dynamic, customizable decision-support tool to guide planning for the future development of preventive vaccines. It can also be used to select among existing vaccines for investment in a public health setting or to support other vaccine-related investment and implementation decisions. SMART Vaccines should always be used as a tool to discuss values and preferences across various user groups and not as the ultimate determinant of a decision that needs to be made.

Guiding Principle and Strategy

To inform future efforts in this area, the committee adopted a guiding principle: SMART Vaccines will have the greatest potential and value if it is programmed as a dynamic, continuously evolving software applica-

Suggested Citation:"4 Observations and Looking Forward." Institute of Medicine. 2013. Ranking Vaccines: A Prioritization Software Tool: Phase II: Prototype of a Decision-Support System. Washington, DC: The National Academies Press. doi: 10.17226/13531.
×

tion and made freely available in an open-source environment to all decision makers and developers around the world.

As a strategy relating to this principle, the committee believes that it would be most beneficial if the National Vaccine Program Office of the Department of Health and Human Services identifies a host for SMART Vaccines and its future versions. Moreover, any decision-support system such as SMART Vaccines has no intrinsic value without accurate and relevant data. Consequently, the committee places the highest importance on creation of a data architecture and expanding data collection for use in SMART Vaccines.

A Vision for the Future

Consistent with this guiding principle and strategy, the committee believes that SMART Vaccines will achieve optimal value if the following events occur:

1. SMART Vaccines and its future versions are hosted in an open-source setting on a widely trusted website with a distinct identity and appropriately protected from unwarranted modifications or intrusions.

2. The host organization creates, maintains, funds, and facilitates a community of users to curate and manage further development of SMART Vaccines and supporting data.

3. The community of users includes decision makers involved in research, development, regulation, and implementation of new vaccines as well as developers with expertise in such areas as modeling, epidemiology, demography, software engineering, database management, and visual design.

4. The community of users—independently or in collaboration with the host organization—establishes an advisory group to help plan future versions and facilitate the adoption of SMART Vaccines.

5. The community of users, together with the host and sponsors, develops mechanisms to encourage the development and updating of data for populations at regional, national, and sub-national levels; for the disease burdens these populations confront; for the costs of preventing and treating those diseases in each distinct environment; and for the productivity losses associated with these disease burdens. These data are accessible in a standardized format, shared with other users through the common website that

Suggested Citation:"4 Observations and Looking Forward." Institute of Medicine. 2013. Ranking Vaccines: A Prioritization Software Tool: Phase II: Prototype of a Decision-Support System. Washington, DC: The National Academies Press. doi: 10.17226/13531.
×

    hosts SMART Vaccines, and improved through an editing process agreed upon and overseen by the user community. These processes could ultimately help guide improvements in global communication and coordination of data and initiatives of common interest and shared importance.

6. The community of users studies the application of SMART Vaccines for retrospective analysis, validation, or confirmation of previous decisions relating to new vaccine development. The results would have both an educational and a continuous learning benefit.

Future Steps

No software product remains static—it either evolves or becomes obsolete, as examples from every realm of software development demonstrate. This remains true even when a single software system dominates a market, as in word processing, spreadsheet, graphical presentation, relational database management, or statistical analysis. In light of the continuous cycles of improvement required for SMART Vaccines, the committee developed a set of steps to guide future development, which are listed in priority order here:

Establishing a Data Architecture

If SMART Vaccines is to become a valuable component of decision making in the vaccine enterprise, concerted data collection efforts will need to be initiated and sustained. The most useful first step would be to establish a data warehouse containing the best sources and estimates for various populations. While demographic datasets are widely available and adaptable for SMART Vaccines, data on disease burden, economic factors, and vaccine characteristics are largely unavailable and need to be estimated with expert guidance. Once experts provide such information for settings around the world, then it will be beneficial to develop training and guidance tools regarding common definitions, calculation formulas, estimation principles, and standardized nomenclature because this will enhance comparability among users.

Conducting Usability Studies

The most useful evolution of SMART Vaccines will occur if there is input from early adopters involved in the on-site testing and assessment of Ver-

Suggested Citation:"4 Observations and Looking Forward." Institute of Medicine. 2013. Ranking Vaccines: A Prioritization Software Tool: Phase II: Prototype of a Decision-Support System. Washington, DC: The National Academies Press. doi: 10.17226/13531.
×

img

FIGURE 4-1
A rendering of SMART Vaccines imagined as a potential tablet application. Future versions of SMART Vaccines in an open-source setting have the potential to be instantiated in a range of different formats, including different languages, and using different visual design features.

sion 1.0 as a support tool in actual decision making. Such input will help point to additional modifications that can improve SMART Vaccines and will help inform the requirements for multiple or customizable visual interfaces for users from various sectors and countries. The usability testing may also shed light on the value of developing versions of SMART Vaccines in alternate software and visual design platforms and also in various of the official languages of the United Nations besides English—for example, Arabic, Chinese, French, Russian, or Spanish. An artist’s rendering of SMART Vaccines in the form of a potential tablet application is shown in Figure 4-1.

Developing Standard Profiles to Enhance
Cross-User Comparisons

Multi-attribute value models (including multi-attribute utility theory as used in SMART Vaccines) do not provide a uniform scale of comparison for different users. As has been explained before with the temperature anal-

Suggested Citation:"4 Observations and Looking Forward." Institute of Medicine. 2013. Ranking Vaccines: A Prioritization Software Tool: Phase II: Prototype of a Decision-Support System. Washington, DC: The National Academies Press. doi: 10.17226/13531.
×

ogy, each user’s value structure creates its own value metric. While some of the vaccine examples employed in this and in the Phase I report allow such a comparison, they are limited to the United States and South Africa. Expanding the availability of such “comparison cases” to many other settings will improve the ability of users to understand their own and others’ SMART Scores in various settings. Thus, the committee believes that there will be great value in the further development of a series of “standard” vaccine descriptors for which the disease burdens, prevention and treatment costs, mortality and morbidity, productivity losses, and vaccine effectiveness are well understood and there are no misinterpretations of results.

Refining the Software Platform and Model

No model is perfect, and the modeling work the committee has undertaken will require continuous refinements. Thus, future work will have the greatest impact if the model is enhanced and improved using software languages that are widely accessible and platform-neutral. The current model is programmed in MATLAB, which can be compiled for use on standard operating systems from Microsoft, Apple, and Linux.

SMART Vaccines will work best if it incorporates or allows the use of a variety of commonly used data-management platforms, either commercially developed platforms or other open source spreadsheet programs for data entry and export. The current version allows a Microsoft Excel–based spreadsheet structure for data entry and export, an approach widely used with both Microsoft products and the Open Source Initiative.

Users of SMART Vaccines will find numerous existing databases that provide information about populations, their disease burdens, and the desirable attributes of yet-to-exist vaccines which can form the basis for initial uses of SMART Vaccines. Future enhancements can facilitate the easy introduction and use of such extant data sources and accommodate them as they change structure and content.

Different diseases are best modeled with different population dynamics. Future enhancements can enlarge the number of ways in which population growth is treated. The most complex of these could include, for example, a population with evolving death rates and fertility rates and even a fertility rate that changes in response to alterations in the mortality rate caused by various diseases.

As decision sciences and modeling techniques continue to improve, future versions of the software may incorporate improvements in the multi-attribute utility theory modeling or the ranking approach used by SMART Vaccines 1.0, and they could offer options for alternative value modeling, such as mathematical programming or analytic hierarchy process.

Suggested Citation:"4 Observations and Looking Forward." Institute of Medicine. 2013. Ranking Vaccines: A Prioritization Software Tool: Phase II: Prototype of a Decision-Support System. Washington, DC: The National Academies Press. doi: 10.17226/13531.
×

Expanding Outreach and Training

The use of SMART Vaccines will increase as more people become aware of its capabilities, which should in turn increase the breadth, depth, and overall competence and expertise within the community of users. Thus, the committee has considered a variety of outreach platforms and tools, and it believes that outreach through presentations, publications, online guides, tutorials, and academic courses will help expand awareness of the software’s capabilities and increase its future potential.

Enhanced Applications of SMART Vaccines

Comparisons of New and Existing Vaccines, Public Health Prevention, and Treatment

This committee was charged with creating a modeling framework that, when fully instantiated, would allow users with different perspectives to create their own rankings of the potential value of new preventive vaccines. While SMART Vaccines was designed to accomplish this specific goal, the committee believes that it can be applied to a broader array of related applications with little or no change in the software structure by assembling the data necessary to describe the particular options being evaluated. Several of these potential applications are described briefly in the following paragraphs.

Choosing Among Existing Vaccines

SMART Vaccines has the potential to help health administrators in various settings compare existing vaccines in order to determine those that best fit their own demographic, economic, and contextual needs. This task is actually easier than ranking new vaccines, because the characteristics of existing vaccines—such as cost, distribution, storage and administration requirements, potential side effects, likely population coverage, and efficacy—are already available and do not have to be estimated.

Alternative Public Health Measures

Although the word “vaccines” appears explicitly in the name “SMART Vaccines,” the software need not be limited to comparing vaccines. Consider, for example, methods of reducing the disease burden of malaria. In principle, one could use SMART Vaccines to evaluate a variety of measures for achieving that goal and could thus compare the performance of vari-

Suggested Citation:"4 Observations and Looking Forward." Institute of Medicine. 2013. Ranking Vaccines: A Prioritization Software Tool: Phase II: Prototype of a Decision-Support System. Washington, DC: The National Academies Press. doi: 10.17226/13531.
×

ous vaccines with the medical treatment of malaria and with the prevention of malaria through the use of mosquito netting or mosquito abatement programs. While SMART Vaccines cannot incorporate the exact details of such programs, the committee believes that adept users will be able to guide decision making among these types of choices using the existing capabilities of SMART Vaccines. Thus, “clean water” or “mosquito netting” or “improved sewage disposal” can be entered as “new preventive vaccines” with considerable efficacy and so expand the potential uses of the software application.

Veterinary Vaccines

While this report focuses on human diseases and human vaccine prioritization and development, vaccination is also used to protect hundreds of millions of livestock, poultry, aquatic life, and companion animals worldwide. Interestingly, the second disease completely eradicated worldwide, after smallpox, was rinderpest in 2011. This paramyxovirus does not infect humans, but it was nonetheless responsible for countless human deaths that resulted from the losses of millions of head of livestock, which led to famine and disease.

Currently, veterinary vaccines account for approximately 20 percent of the total vaccine market. SMART Vaccines can be modified for use in the world of veterinary medicine. Intuitively, it seems likely that the greatest interest would be in analyzing the economic attributes of animal vaccines; however, the broad area of animal health would benefit from a tool that guides the making of policy and various other decisions that affect animals. Furthermore, healthier food animals lead to safer food, and a number of animal diseases pose risks of transmission to humans. Thus, an investment in animal vaccines will also have a real benefit to human health, which leverages and adds value to the investment in veterinary vaccines. Except for the new and different data demands, there is no difference between comparing human vaccines and comparing vaccines for domesticated animals. And improving vaccines for food animals has the additional advantage that it could help reduce the use of antibiotics, thereby reducing the risk of developing highly resistant organisms in both animal and human populations.

Disease-Resistant Plants

The hybridization and genetic modification of plants to enhance disease resistance is conceptually similar to vaccinating humans or animals against

Suggested Citation:"4 Observations and Looking Forward." Institute of Medicine. 2013. Ranking Vaccines: A Prioritization Software Tool: Phase II: Prototype of a Decision-Support System. Washington, DC: The National Academies Press. doi: 10.17226/13531.
×

disease. The multi-attribute model embedded in SMART Vaccines could be used to compare alternative approaches to creating disease resistance in plant; for example, it could be used to compare the potential risks of creating disease immunity in plants by hybridization versus through gene modification.

Facilitating Discussions Among Various User Groups

From discussions with various user groups throughout Phase I and Phase II, the committee has concluded that interest groups often fail to fully communicate with each other concerning values, choices, and impediments to reaching their goals. The resulting shortcomings in understanding can be seen, for example, in questions that arise during the design phase about the best compromises in vaccine attributes or product profiles. What is more valuable to stakeholders: oral administration or thermal stability of the vaccine? Do vaccines fit within the existing immunization schedules and programs? What social customs might enhance or limit population acceptance of certain vaccines? The answers to these questions will differ widely from setting to setting, but the committee believes that these types of trade-offs have not been fully discussed or understood by various user communities. SMART Vaccines allows the formal consideration of these trade-offs at the local level. Thus, the software can allow users to comprehend and communicate their own preferences more clearly.

Vaccine manufacturers face other constraints and may have different objectives. They seek profitable products, because sales revenues are typically the only source for financing research, development, testing, and production. Intellectual property laws provide temporary market protection for recouping research and development costs, but ultimately the risks surrounding scientific research and eventual commercial products drive many corporate decisions about new product development. Knowing which attributes are more or less valuable to potential user communities could enhance vaccine developers’ decision-making process and ensure that they better understand the values that users place on various potential vaccines as a way of improving their decision making.

In short, it is the committee’s hope that SMART Vaccines will serve as a valuable tool to enhance and clarify discussions among various user communities, which will, over time, lead to improved public health outcomes.

Suggested Citation:"4 Observations and Looking Forward." Institute of Medicine. 2013. Ranking Vaccines: A Prioritization Software Tool: Phase II: Prototype of a Decision-Support System. Washington, DC: The National Academies Press. doi: 10.17226/13531.
×

Next Phase

In an immediate follow-up Phase III activity, an Institute of Medicine and National Academy of Engineering committee is expected to evaluate the utility of SMART Vaccines and offer guidance on a data warehouse for the software. Specifically, the committee is expected to produce some use case scenarios in collaboration with actual end users of the software. In doing so, it will also attempt to develop a framework for a data warehouse including estimation strategies to create future datasets for the software.

Suggested Citation:"4 Observations and Looking Forward." Institute of Medicine. 2013. Ranking Vaccines: A Prioritization Software Tool: Phase II: Prototype of a Decision-Support System. Washington, DC: The National Academies Press. doi: 10.17226/13531.
×

This page intentionally left blank.

Suggested Citation:"4 Observations and Looking Forward." Institute of Medicine. 2013. Ranking Vaccines: A Prioritization Software Tool: Phase II: Prototype of a Decision-Support System. Washington, DC: The National Academies Press. doi: 10.17226/13531.
×
Page 61
Suggested Citation:"4 Observations and Looking Forward." Institute of Medicine. 2013. Ranking Vaccines: A Prioritization Software Tool: Phase II: Prototype of a Decision-Support System. Washington, DC: The National Academies Press. doi: 10.17226/13531.
×
Page 62
Suggested Citation:"4 Observations and Looking Forward." Institute of Medicine. 2013. Ranking Vaccines: A Prioritization Software Tool: Phase II: Prototype of a Decision-Support System. Washington, DC: The National Academies Press. doi: 10.17226/13531.
×
Page 63
Suggested Citation:"4 Observations and Looking Forward." Institute of Medicine. 2013. Ranking Vaccines: A Prioritization Software Tool: Phase II: Prototype of a Decision-Support System. Washington, DC: The National Academies Press. doi: 10.17226/13531.
×
Page 64
Suggested Citation:"4 Observations and Looking Forward." Institute of Medicine. 2013. Ranking Vaccines: A Prioritization Software Tool: Phase II: Prototype of a Decision-Support System. Washington, DC: The National Academies Press. doi: 10.17226/13531.
×
Page 65
Suggested Citation:"4 Observations and Looking Forward." Institute of Medicine. 2013. Ranking Vaccines: A Prioritization Software Tool: Phase II: Prototype of a Decision-Support System. Washington, DC: The National Academies Press. doi: 10.17226/13531.
×
Page 66
Suggested Citation:"4 Observations and Looking Forward." Institute of Medicine. 2013. Ranking Vaccines: A Prioritization Software Tool: Phase II: Prototype of a Decision-Support System. Washington, DC: The National Academies Press. doi: 10.17226/13531.
×
Page 67
Suggested Citation:"4 Observations and Looking Forward." Institute of Medicine. 2013. Ranking Vaccines: A Prioritization Software Tool: Phase II: Prototype of a Decision-Support System. Washington, DC: The National Academies Press. doi: 10.17226/13531.
×
Page 68
Suggested Citation:"4 Observations and Looking Forward." Institute of Medicine. 2013. Ranking Vaccines: A Prioritization Software Tool: Phase II: Prototype of a Decision-Support System. Washington, DC: The National Academies Press. doi: 10.17226/13531.
×
Page 69
Suggested Citation:"4 Observations and Looking Forward." Institute of Medicine. 2013. Ranking Vaccines: A Prioritization Software Tool: Phase II: Prototype of a Decision-Support System. Washington, DC: The National Academies Press. doi: 10.17226/13531.
×
Page 70
Next: References »
Ranking Vaccines: A Prioritization Software Tool: Phase II: Prototype of a Decision-Support System Get This Book
×
Buy Paperback | $50.00 Buy Ebook | $39.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

SMART Vaccines--Strategic Multi-Attribute Ranking Tool for Vaccines--is a prioritization software tool developed by the Institute of Medicine that utilizes decision science and modeling to help inform choices among candidates for new vaccine development. A blueprint for this computer-based guide was presented in the 2012 report Ranking Vaccines: A Prioritization Framework: Phase I.

Ranking Vaccines: A Prioritization Software Tool,Phase II extends the proof-of-concept presented in the Phase I report, which was based on multi-attribute utility theory. This report refines a beta version of the model developed in the Phase I report and presents its next iteration, SMART Vaccines 1.0.

Ranking Vaccines: Phase II discusses the methods underlying the development, validation, and evaluation of SMART Vaccines 1.0. It also discusses how SMART Vaccines should--and, just as importantly, should not--be used. The report also offers ideas for future enhancements for SMART Vaccines as well as for ideas for expanded uses and considerations and possibilities for the future.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!