4

Observations and Looking Forward

Previous efforts in the realm of prioritizing new vaccines have been limited because of the small number of attributes and value preferences those models were able to accommodate. Moving from the traditional single-attribute modeling framework to a multi-attribute modeling framework involved substantial public input and a wide range of expertise in order to develop the computational model and the user interface that support SMART Vaccines.

As noted in the 2012 Institute of Medicine (IOM) report Ranking Vaccines: A Prioritization Framework (IOM, 2012) and earlier in this edition, the committee’s charge did not include the production of a “list” of vaccines for development. Instead, through this work the committee developed SMART Vaccines as a dynamic, customizable decision-support tool to guide planning for the future development of preventive vaccines. It can also be used to select among existing vaccines for investment in a public health setting or to support other vaccine-related investment and implementation decisions. SMART Vaccines should always be used as a tool to discuss values and preferences across various user groups and not as the ultimate determinant of a decision that needs to be made.

Guiding Principle and Strategy

To inform future efforts in this area, the committee adopted a guiding principle: SMART Vaccines will have the greatest potential and value if it is programmed as a dynamic, continuously evolving software applica-



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 61
4 Observations and Looking Forward Previous efforts in the realm of prioritizing new vaccines have been lim- ited because of the small number of attributes and value preferences those models were able to accommodate. Moving from the traditional single- attribute modeling framework to a multi-attribute modeling framework involved substantial public input and a wide range of expertise in order to develop the computational model and the user interface that support SMART Vaccines. As noted in the 2012 Institute of Medicine (IOM) report Ranking Vaccines: A Prioritization Framework (IOM, 2012) and earlier in this edi- tion, the committee’s charge did not include the production of a “list” of vaccines for development. Instead, through this work the committee devel- oped SMART Vaccines as a dynamic, customizable decision-support tool to guide planning for the future development of preventive vaccines. It can also be used to select among existing vaccines for investment in a public health setting or to support other vaccine-related investment and imple- mentation decisions. SMART Vaccines should always be used as a tool to discuss values and preferences across various user groups and not as the ultimate determinant of a decision that needs to be made. Guiding Principle and Strategy To inform future efforts in this area, the committee adopted a guiding prin- ciple: SMART Vaccines will have the greatest potential and value if it is programmed as a dynamic, continuously evolving software applica- 61

OCR for page 61
62 RANKING VACCINES: A Prioritization Software Tool tion and made freely available in an open-source environment to all decision makers and developers around the world. As a strategy relating to this principle, the committee believes that it would be most beneficial if the National Vaccine Program Office of the Department of Health and Human Services identifies a host for SMART Vaccines and its future versions. Moreover, any decision- support system such as SMART Vaccines has no intrinsic value without accurate and relevant data. Consequently, the committee places the high- est importance on creation of a data architecture and expanding data collection for use in SMART Vaccines. A Vision for the Future Consistent with this guiding principle and strategy, the committee believes that SMART Vaccines will achieve optimal value if the following events occur: 1.  MART Vaccines and its future versions are hosted in an open- S source setting on a widely trusted website with a distinct identity and appropriately protected from unwarranted modifications or intrusions. 2.  he host organization creates, maintains, funds, and facilitates a T community of users to curate and manage further development of SMART Vaccines and supporting data. 3.  he community of users includes decision makers involved in T research, development, regulation, and implementation of new vaccines as well as developers with expertise in such areas as modeling, epidemiology, demography, software engineering, data- base management, and visual design. 4.  he community of users—independently or in collaboration with T the host organization—establishes an advisory group to help plan future versions and facilitate the adoption of SMART Vaccines. 5.  he community of users, together with the host and sponsors, T develops mechanisms to encourage the development and updat- ing of data for populations at regional, national, and sub-national levels; for the disease burdens these populations confront; for the costs of preventing and treating those diseases in each distinct environment; and for the productivity losses associated with these disease burdens. These data are accessible in a standardized for- mat, shared with other users through the common website that

OCR for page 61
Observations and Looking Forward 63 hosts SMART Vaccines, and improved through an editing process agreed upon and overseen by the user community. These pro- cesses could ultimately help guide improvements in global com- munication and coordination of data and initiatives of common interest and shared importance. 6.  he community of users studies the application of SMART Vac- T cines for retrospective analysis, validation, or confirmation of pre- vious decisions relating to new vaccine development. The results would have both an educational and a continuous learning benefit. Future Steps No software product remains static—it either evolves or becomes obsolete, as examples from every realm of software development demonstrate. This remains true even when a single software system dominates a market, as in word processing, spreadsheet, graphical presentation, relational data- base management, or statistical analysis. In light of the continuous cycles of improvement required for SMART Vaccines, the committee developed a set of steps to guide future development, which are listed in priority order here: Establishing a Data Architecture If SMART Vaccines is to become a valuable component of decision mak- ing in the vaccine enterprise, concerted data collection efforts will need to be initiated and sustained. The most useful first step would be to estab- lish a data warehouse containing the best sources and estimates for various populations. While demographic datasets are widely available and adapt- able for SMART Vaccines, data on disease burden, economic factors, and vaccine characteristics are largely unavailable and need to be estimated with expert guidance. Once experts provide such information for settings around the world, then it will be beneficial to develop training and guid- ance tools regarding common definitions, calculation formulas, estimation principles, and standardized nomenclature because this will enhance com- parability among users. Conducting Usability Studies The most useful evolution of SMART Vaccines will occur if there is input from early adopters involved in the on-site testing and assessment of Ver-

OCR for page 61
64 RANKING VACCINES: A Prioritization Software Tool FIGURE 4-1 A rendering of SMART Vaccines imagined as a potential tablet application. Future versions of SMART Vaccines in an open-source setting have the potential to be instantiated in a range of dif- ferent formats, including different languages, and using different visual design features. sion 1.0 as a support tool in actual decision making. Such input will help point to additional modifications that can improve SMART Vaccines and will help inform the requirements for multiple or customizable visual interfaces for users from various sectors and countries. The usability test- ing may also shed light on the value of developing versions of SMART Vac- cines in alternate software and visual design platforms and also in various of the official languages of the United Nations besides English—for exam- ple, Arabic, Chinese, French, Russian, or Spanish. An artist’s rendering of SMART Vaccines in the form of a potential tablet application is shown in Figure 4-1. Developing Standard Profiles to Enhance Cross-User Comparisons Multi-attribute value models (including multi-attribute utility theory as used in SMART Vaccines) do not provide a uniform scale of comparison for different users. As has been explained before with the temperature anal-

OCR for page 61
Observations and Looking Forward 65 ogy, each user’s value structure creates its own value metric. While some of the vaccine examples employed in this and in the Phase I report allow such a comparison, they are limited to the United States and South Africa. Expanding the availability of such “comparison cases” to many other set- tings will improve the ability of users to understand their own and others’ SMART Scores in various settings. Thus, the committee believes that there will be great value in the further development of a series of “standard” vac- cine descriptors for which the disease burdens, prevention and treatment costs, mortality and morbidity, productivity losses, and vaccine effective- ness are well understood and there are no misinterpretations of results. Refining the Software Platform and Model No model is perfect, and the modeling work the committee has undertaken will require continuous refinements. Thus, future work will have the great- est impact if the model is enhanced and improved using software languages that are widely accessible and platform-neutral. The current model is pro- grammed in MATLAB, which can be compiled for use on standard operat- ing systems from Microsoft, Apple, and Linux. SMART Vaccines will work best if it incorporates or allows the use of a variety of commonly used data-management platforms, either commer- cially developed platforms or other open source spreadsheet programs for data entry and export. The current version allows a Microsoft Excel–based spreadsheet structure for data entry and export, an approach widely used with both Microsoft products and the Open Source Initiative. Users of SMART Vaccines will find numerous existing databases that provide information about populations, their disease burdens, and the desirable attributes of yet-to-exist vaccines which can form the basis for initial uses of SMART Vaccines. Future enhancements can facilitate the easy introduction and use of such extant data sources and accommodate them as they change structure and content. Different diseases are best modeled with different population dynamics. Future enhancements can enlarge the number of ways in which population growth is treated. The most complex of these could include, for example, a population with evolving death rates and fertility rates and even a fertility rate that changes in response to alterations in the mortality rate caused by various diseases. As decision sciences and modeling techniques continue to improve, future versions of the software may incorporate improvements in the multi- attribute utility theory modeling or the ranking approach used by SMART Vaccines 1.0, and they could offer options for alternative value modeling, such as mathematical programming or analytic hierarchy process.

OCR for page 61
66 RANKING VACCINES: A Prioritization Software Tool Expanding Outreach and Training The use of SMART Vaccines will increase as more people become aware of its capabilities, which should in turn increase the breadth, depth, and overall competence and expertise within the community of users. Thus, the committee has considered a variety of outreach platforms and tools, and it believes that outreach through presentations, publications, online guides, tutorials, and academic courses will help expand awareness of the software’s capabilities and increase its future potential. Enhanced Applications of SMART Vaccines Comparisons of New and Existing Vaccines, Public Health Prevention, and Treatment This committee was charged with creating a modeling framework that, when fully instantiated, would allow users with different perspectives to create their own rankings of the potential value of new preventive vac- cines. While SMART Vaccines was designed to accomplish this specific goal, the committee believes that it can be applied to a broader array of related applications with little or no change in the software structure by assembling the data necessary to describe the particular options being evaluated. Several of these potential applications are described briefly in the following paragraphs. Choosing Among Existing Vaccines SMART Vaccines has the potential to help health administrators in various settings compare existing vaccines in order to determine those that best fit their own demographic, economic, and contextual needs. This task is actually easier than ranking new vaccines, because the characteristics of existing vaccines—such as cost, distribution, storage and administration requirements, potential side effects, likely population coverage, and effi- cacy—are already available and do not have to be estimated. Alternative Public Health Measures Although the word “vaccines” appears explicitly in the name “SMART Vaccines,” the software need not be limited to comparing vaccines. Con- sider, for example, methods of reducing the disease burden of malaria. In principle, one could use SMART Vaccines to evaluate a variety of measures for achieving that goal and could thus compare the performance of vari-

OCR for page 61
Observations and Looking Forward 67 ous vaccines with the medical treatment of malaria and with the preven- tion of malaria through the use of mosquito netting or mosquito abatement programs. While SMART Vaccines cannot incorporate the exact details of such programs, the committee believes that adept users will be able to guide decision making among these types of choices using the existing capabilities of SMART Vaccines. Thus, “clean water” or “mosquito net- ting” or “improved sewage disposal” can be entered as “new preventive vaccines” with considerable efficacy and so expand the potential uses of the software application. Veterinary Vaccines While this report focuses on human diseases and human vaccine priori- tization and development, vaccination is also used to protect hundreds of millions of livestock, poultry, aquatic life, and companion animals world- wide. Interestingly, the second disease completely eradicated worldwide, after smallpox, was rinderpest in 2011. This paramyxovirus does not infect humans, but it was nonetheless responsible for countless human deaths that resulted from the losses of millions of head of livestock, which led to famine and disease. Currently, veterinary vaccines account for approximately 20 percent of the total vaccine market. SMART Vaccines can be modified for use in the world of veterinary medicine. Intuitively, it seems likely that the greatest interest would be in analyzing the economic attributes of animal vaccines; however, the broad area of animal health would benefit from a tool that guides the making of policy and various other decisions that affect animals. Furthermore, healthier food animals lead to safer food, and a number of animal diseases pose risks of transmission to humans. Thus, an investment in animal vaccines will also have a real benefit to human health, which leverages and adds value to the investment in veterinary vaccines. Except for the new and different data demands, there is no difference between comparing human vaccines and comparing vaccines for domesticated ani- mals. And improving vaccines for food animals has the additional advan- tage that it could help reduce the use of antibiotics, thereby reducing the risk of developing highly resistant organisms in both animal and human populations. Disease-Resistant Plants The hybridization and genetic modification of plants to enhance disease resistance is conceptually similar to vaccinating humans or animals against

OCR for page 61
68 RANKING VACCINES: A Prioritization Software Tool disease. The multi-attribute model embedded in SMART Vaccines could be used to compare alternative approaches to creating disease resistance in plant; for example, it could be used to compare the potential risks of creating disease immunity in plants by hybridization versus through gene modification. Facilitating Discussions Among Various User Groups From discussions with various user groups throughout Phase I and Phase II, the committee has concluded that interest groups often fail to fully com- municate with each other concerning values, choices, and impediments to reaching their goals. The resulting shortcomings in understanding can be seen, for example, in questions that arise during the design phase about the best compromises in vaccine attributes or product profiles. What is more valuable to stakeholders: oral administration or thermal stability of the vac- cine? Do vaccines fit within the existing immunization schedules and pro- grams? What social customs might enhance or limit population acceptance of certain vaccines? The answers to these questions will differ widely from setting to setting, but the committee believes that these types of trade-offs have not been fully discussed or understood by various user communities. SMART Vaccines allows the formal consideration of these trade-offs at the local level. Thus, the software can allow users to comprehend and commu- nicate their own preferences more clearly. Vaccine manufacturers face other constraints and may have different objectives. They seek profitable products, because sales revenues are typi- cally the only source for financing research, development, testing, and pro- duction. Intellectual property laws provide temporary market protection for recouping research and development costs, but ultimately the risks sur- rounding scientific research and eventual commercial products drive many corporate decisions about new product development. Knowing which attributes are more or less valuable to potential user communities could enhance vaccine developers’ decision-making process and ensure that they better understand the values that users place on various potential vaccines as a way of improving their decision making. In short, it is the committee’s hope that SMART Vaccines will serve as a valuable tool to enhance and clarify discussions among various user com- munities, which will, over time, lead to improved public health outcomes.

OCR for page 61
Observations and Looking Forward 69 Next Phase In an immediate follow-up Phase III activity, an Institute of Medicine and National Academy of Engineering committee is expected to evaluate the utility of SMART Vaccines and offer guidance on a data warehouse for the software. Specifically, the committee is expected to produce some use case scenarios in collaboration with actual end users of the software. In doing so, it will also attempt to develop a framework for a data warehouse includ- ing estimation strategies to create future datasets for the software.

OCR for page 61