4
Reflections and Looking Forward
The Phase II report Ranking Vaccines: A Prioritization Software Tool (IOM, 2013, p. 9), specified a guiding principle that formed the foundation for the Phase III committee’s thinking: “SMART Vaccines will have the greatest potential and value if it is programmed as a dynamic, continuously evolving software application and made freely available in an open-source environment to all decision makers and developers around the world.”
The Phase II report also stressed the importance of the National Vaccine Program Office identifying a future home for SMART Vaccines and of creating a data architecture to enhance the creation of useful data for SMART Vaccines users. The report then listed a sequence of events that the Phase II committee believed would increase the long-term value of SMART Vaccines. Those events are summarized here:
- SMART Vaccines is hosted in an open-source setting in a widely trusted website with a distinct identity, protected from unwarranted modification or intrusion.
- The host organization creates, maintains, and funds a user community to create and manage data and to facilitate further software and data development.
- Ideally, the user community includes decision makers from a wide spectrum of the vaccine community and includes expertise in such relevant areas as epidemiology, demographic sciences, software engineering, database management, and visual design.
- The community of users establishes an advisory group to help plan future improvements and enhancements of SMART Vaccines.
- The community of users and the host organization facilitate the development and updating of data, which would optimally be accessible in standardized format for users around the world.
- The community of users retrospectively studies previous choices made about vaccine development as part of a continuous learning and improvement process.
The Phase III committee endorses these concepts and elaborates upon them in the following sections.
Phase I of this project involved developing the SMART Vaccines concept, and Phase II saw the robust testing of version 1.0 of the software. In Phase III this committee provided an enhanced user interface in version 1.1 and tested the software in actual use cases in collaboration with three user groups based in the Canada, India, and the United States. The committee also added population data for 34 Organisation for Economic Co-operation and Development (OECD) member countries in addition to data from India, New York State, and South Africa, and it provided initial boundary estimates for attributes with values calculated within the software so as to reduce the start-up barrier to its further use.
As Phase III comes to a close, it has become all the more apparent to the committee that a transition strategy to a permanent home for SMART Vaccines is necessary for the ongoing use, enhancement, and even the survival of the software as a tool for strategic planning. The committees of the various phases that worked on SMART Vaccines have consistently understood the importance of a permanent home and the need for an active user community working to improve the software and the data library to support it.
There are many examples of existing user consortia involved with software applications—for improving university operations, for enhancing the quality of life for the elderly, for improving ocean ecology, for using and improving statistical analysis and other business software, and more—and these examples suggest that the formation of a similar group devoted to the use and enhancement of SMART Vaccines would neither be difficult to achieve nor complicated to maintain.
The permanent home for SMART Vaccines could be a single institution such as a private foundation or a research university, or it could be
a partnership involving combinations of foundation and university participation. It might even involve a broader public–private partnership that includes for-profit vaccine manufacturers. This committee’s task did not include identifying the permanent host, but it did include a discussion of issues that will enhance “further development, maintenance, and dissemination of the software and the data warehouse.” With that in mind, we turn next to a discussion of a process that could to a permanent host and a discussion of the key attributes that a desirable permanent home would possess.
A Process Leading to a Permanent Home
The committee believes that the National Vaccine Program Office and the Fogarty International Center of the National Institutes of Health—the federal sponsors of the Phase III project—will be best served if they promptly create a process to facilitate the transition of SMART Vaccines to a permanent home.1 They could do this by convening a group of relevant stakeholders with the goal of having this group recommend a process for selecting the permanent home. This approach might involve, among other things, a process for receiving proposals and a strategic mechanism for determining which of these groups—individually or as a consortium—would be suited to become a permanent home for SMART Vaccines. The relevant stakeholder groups would include public health groups that focus on vaccine research, development, and policy—both domestic and international—as well as representation from producers and suppliers of vaccines, higher educational institutions with a prominent focus on global public health, international governmental and nongovernmental health organizations, and philanthropic donors with strong interests in public health and vaccine policy.
Ensuring the Growth and Value of SMART Vaccines
The Phase II report indicated—and this committee strongly agrees—that the ultimate future applications and benefits of SMART Vaccines depend on the strengths of the organization or consortium that
____________
1 The committee believes that the existing operational structure of the National Academies does not make it a plausible permanent home for a product such as SMART Vaccines that must necessarily evolve through time, a characteristic that would require a qualified host organization to support and oversee software modification, enhancement, and data curation and to facilitate an active community of users.
becomes the permanent host. The committee’s discussion of these issues contains seven main elements, many of which are interrelated. We describe these issues here to help illuminate any future discussions that take place leading to the choice of a permanent home.
The Host Organization
SMART Vaccines offers a unique ability for individual users to specify what matters to them—and by how much—rather than limiting the user to a single metric, such as has been the case with previous vaccine-ranking tools from the Institute of Medicine and elsewhere. Because of the inherent flexibility in the viewpoints able to be expressed in SMART Vaccines analyses, a host organization with a particular point of view might lead others with a different point of view to avoid participation in a community of users, and this might also run the risk that SMART Vaccines would become identified with a particular viewpoint.
Thus, the committee emphasizes the importance of choosing a host organization that both is neutral among many users’ competing viewpoints—and is clearly viewed as such—and is well equipped with organizational and technological capabilities. The committee believes not only that the best hosting organization will have a significant international presence and reputation, but also that the hosting organization will best serve the user community if it is—or partners with—a research-intensive institution of higher education.
The Importance of a Higher Education Presence in the Host Organization
The committee believes that a higher education presence in the permanent host for SMART Vaccines could provide important benefits to the user community around the world. In particular, the hosting arrangement should include a major research university or a consortium of universities with a significant global public health focus. Many other government and private-sector organizations possess some of these traits, and consortium arrangements between such organizations could likely provide a similar set of strengths, but they would require an agreement among stakeholders on the management and governance structure.
In addition to important strengths in public health, a major research university presence would bring access to many other important knowledge domains, including vaccinology, immunology, epidemiology, public policy analysis and modeling, demography, health care systems engineer-
ing, health economics, business management, information management, and computer sciences. Many universities around the world also serve as World Health Organization collaborating centers. Some research universities also have partnerships with or have access to technology parks with strengths in new product incubation whose models may inform the development of the successive versions of SMART Vaccines.
Research universities generally possess the requisite independence and neutrality that is desirable in the permanent SMART Vaccines hosting site. One exception to this characterization of research universities would arise if the host university had a major presence in vaccine development, particularly if it was closely linked to the development and testing of the vaccines of one or a small number of vaccine manufacturers. In such a situation, a consortium of universities could reduce the actual and apparent conflicts of interest that might arise otherwise.
Research universities bring another important attribute to the table: They have standing educational programs that could mesh well with the presence of SMART Vaccines. One can easily envision, for example, graduate-level courses on strategic planning in public health or health policy that involve SMART Vaccines and the concepts therein (e.g., multi-criteria decision analysis, multi-attribute utility theory, and systems analysis) in a fundamental way, which would serve as a valuable complement to the cost-effectiveness and cost-benefit analyses traditionally found in such courses.
In addition, a university’s undergraduate and graduate students, either through formal course-work or hired on an hourly basis, offer an attractive option for data development. With proper guidance, training, and templates, students could provide a ready mechanism to crowdsource the development of data libraries. Furthermore, SMART Vaccines would offer an almost endless set of opportunities for graduate dissertation material in a wide array of fields.
From Phase I onward, the committees helping to develop SMART Vaccines have emphasized the importance of developing a network of users. The committee urges that a community of users, developers, and decision makers be created, fostered, and supported (most likely by the host of SMART Vaccines) to facilitate further use of the tool, data development and curation, and to guide additional software improvements and enhancements. User consortia such as the one the committee envi-
sions for SMART Vaccines have numerous functions. They include the following:
Manage Product Development and Enhancements
No software system is immune from bugs, and the best way to find them is to use the software extensively. Thus, one of the essential functions of any software user community is to find, report, and fix bugs, and the SMART Vaccines user community will be no different. Furthermore, the Phase II report discussed the option of a user community establishing an oversight group to manage and guide the further development and extension of SMART Vaccines. This committee reaffirms the desirability and importance of such a guiding group.
Practical Contributions from Users
Some software tools are built from the ground up to allow users to develop and share enhancements that run as programs or subroutines within the larger software environment. Even early programming languages such as FORTRAN relied on subroutine libraries that were developed, tested, curated, and made available by various organizations (sometimes competing with one another). The underlying philosophy of the Linux operating system is to base Linux on the contributions of a community of users. Many other software-based systems similarly benefit from user-provided contributions, some of which eventually are embedded into the primary source code of the software, while other contributions remain independent programs that operate within the overall software environment.
Exchange of Ideas and Training
Most software consortia have newsletters, blogs, discussion forums, and social media platforms to allow users to share ideas and help each other solve problems. Ideally the host institution or consortium for SMART Vaccines would provide and maintain the mechanisms for doing that for this software.
Depending on the ultimate size and global spread of the SMART Vaccines user community, it may become useful to create formal training tools to provide instruction both in using the software and in carrying out the data development necessary to expand the populations and the diseases that can be used in SMART Vaccines. This training could be carried out via workshops, webinars, tutorial videos, or formal courses either within higher education settings or affiliated with relevant professional meetings in public health, medical decision analysis, or public policy.
In larger user communities, conferences and seminars are organized on a regular basis, sometimes being hosted by the sponsoring organization or independent groups. Even with SMART Vaccines, it seems likely that at some point in the future it will be desirable to have regular workshops where the user community can share best practices and new ideas.
Data Development, Curation, and Sharing
Another way the user community could help increase the usefulness of SMART Vaccines would be to expand, refresh, and, curate data for different populations around the world, including data on demographics, disease burden, and costs of care. As part of this function, the user community could also verify the quality of datasets submitted by individual users or students using SMART Vaccines in order to provide a sense of the accuracy and trustworthiness of datasets developed around the world.
The data curation for SMART Vaccines—which, as noted earlier, will include such functions as quality control, storage, and access—will likely involve an integrated data warehouse. Such a warehouse will likely include a relational database management system to provide summary analytics and to allow specialized data outputs on all desired dimensions, including reports specific to a given population and reports summarizing disease burden data across all populations represented in the database.
Outreach and Awareness Enhancement
The committee places a strong emphasis on the importance of additional outreach and communication efforts to achieve the best use of SMART Vaccines. Based on previous outreach activities for SMART Vaccines in the form of seminars, workshops, conference symposia, and other ad hoc presentations, the committee knows that people in the vaccine community are eager to learn more about the possible uses of SMART Vaccines and in many cases to pursue further exposure to and use of the system. Those presentations have led many in the vaccine community, both in the United States and internationally, to express interest in contributing to the further development of SMART Vaccines and its potential applications. In addition, the committee has been encouraged by the interest of academic communities which have appreciated the teaching and learning value of SMART Vaccines for students interested in public health, health policy, business, engineering, and biomedicine. These communities could help refine the tool, and suggest further uses.
Early Partners and Value Demonstration
In most endeavors like SMART Vaccines, outreach activities lead to some organizations becoming early adopters (or pioneers) of new software systems. These early users commonly have an important role in determining the eventual success of software systems. They lead the discussion on the software value, which in turn attracts other users. They demonstrate ways in which value creation overcomes the initial barriers to entry that exist when one is starting to use a new system. In the case of SMART Vaccines, the main initial hurdle is developing data to characterize not only the demography of the relevant populations, but also the disease burdens and the costs of treating those diseases. Thus, finding and encouraging a set of early partners is one of the key goals of the initial seminars, workshops, symposia, and other communication modes in the outreach efforts for SMART Vaccines.
Data Development: An Opportunity Awaiting
As daunting as the data requirements for SMART Vaccines might seem, the committee believes that to carry out any vaccine prioritization task sensibly, decision makers will necessarily need to have these same data in hand. Without these basic data, decisions cannot be made as carefully or with an empirical basis.
The data requirements that may seem to loom large in the eyes of potential users are not created by the software itself—it merely brings them to the forefront. One cannot make intelligent, data-informed decisions about vaccine priorities without these data. Once the data are assembled, SMART Vaccines provides a useful tool, which has a significant, data-driven basis, for managing the data and for enhancing decision making. Earlier in this report, a discussion appeared concerning the desirable approaches to providing a data warehouse that allows for the introduction of new data, quality validation, and access to the data from various perspectives (e.g., by country or by disease). All of the data development suggestions that follow will benefit from a carefully constructed and well managed data warehouse capability.
The Phase III committee sought to reduce the data input burden on users by pre-loading the software with the population data for 34 member countries of the OECD as well as India, South Africa, and New York State. The committee also sought to provide users with various resources to simplify finding and entering data in other categories. Ultimately, the com-
mittee finds the data warehouses developed by the World Bank,2the Global Burden of Disease project of the Institute for Health Metrics and Evaluation,3and the International Labour Organization4to be desirable templates for informing data synthesis, tutorials, and visualization to support vaccine priority-setting efforts in general.
To further expand the data library for SMART Vaccines at a strategic level, rather than at the tactical level discussed in Chapter 2, the committee envisions three basic approaches to data development, which can be carried out singly or in various combinations. These approaches are
1. Each user develops the population-specific data required. This is basically how the three use-case groups described earlier in this report assembled their data—with some assistance from the committee.
2. Centralized data development with external funding. One way to fill out the databases for use in SMART Vaccines would be the use of centralized data development. This would require significant external funding to contract for the data development, either globally or region by region. Such funding might come from governments, private-sector resources, or philanthropy. Separate organizations might fund different parts of the data development, e.g., data development in Europe, Central and South America, the Middle East, and various regions of Asia. The committee observes that the use of centralized funding to create the data under contract would be a viable option.
3. Crowdsourcing. A crowdsourcing approach would rely on volunteer providers of data, with smaller collections of data sent to a central repository. The benefits and weaknesses of this approach versus the first two are obvious. Crowdsourcing would likely occur at lower cost, but it would also likely occur more sporadically and almost certainly with lower data quality than centralized or user-specific approaches might provide. Crowdsourcing is more likely to succeed in the presence of the following three things:
(a) Tutorials, videos, and training tools.
(b) A higher education connection. Students taking classes that use SMART Vaccines provide a natural base for crowd-
____________
2 See www.data.worldbank.org.
3 See www.healthdata.org/gbd. See, for example, Murray et al. (2012).
4 ILOSTAT at www.ilo.org.
sourcing data. Academic courses using SMART Vaccines could (and preferably would) have as a learning component the development of a new data segment. Thus, for example, a course offered in a given country might focus specifically on data development concerning the burden of diseases and costs of treatment in that country. Many U.S. research universities have international partners that would be natural for U.S. students to work with in order to develop data for other populations.
(c) An active user group that can provide guidance and training for participants involved in crowdsourcing with the ability to analyze and rate the quality of newly developed data for other users.
Based on the feedback from the three user groups’ and on numerous presentations made by the committee concerning SMART Vaccines, both in the United States and abroad, the committee observes that the current Matlab-compiled software implementation is an impediment to users. The committee believes that a fully Web-based version is an essential next step in the development of SMART Vaccines.
Three features of the current software environment underpin this observation. First, SMART Vaccines is currently platform-dependent—it runs only in a Windows environment, which precludes its use in Apple, Linux, or other operating systems.
Second, the current version requires the downloading of a Matlab compiler, which then creates the operational version of SMART Vaccines on the user’s computer. This is time consuming, particularly in environments with less-robust Internet connectivity, and it adds a layer of extra effort and complexity that a Web-based system would avoid.
This in turn leads to the third issue, which the committee has repeatedly experienced in user testing and in other public demonstrations of SMART Vaccines. Particularly in governmental office buildings, firewalls, and other intranet security systems prohibit the downloading or installation of outside software and, in some cases, even the simple act of connecting an externally provided computer to the system. A Web-based version of SMART Vaccines would avoid these difficulties because it would ideally operate through the Web browser on any user’s computer with Internet connectivity.
Intellectual Property Considerations
Just as did the Phase I and II committees, the Phase III committee has emphasized the benefits of a consortium of users dedicated to enhancing SMART Vaccines software and data systems. To develop such a consortium properly, the intellectual property associated with SMART Vaccines must have proper licensing terms and conditions both to assure that it is available to the community of users in ways that enhance its use and also to prevent unauthorized uses or modifications of the software and its potential derivative products.
Specifically, the committee envisions a future software environment that allows for open use of SMART Vaccines and its associated data library, but with control over the official versions of the source code and the data governed by appropriate licensing terms and conditions. This type of arrangement is in wide use for open source software, with dozens of various specific models of licenses existing, each with modestly different arrangements.5
The Linux operating system software, for example, uses the GNU model, where users around the world are free to use and modify the original code, and can apply to the group that owns the Linux copyright and trademark. The APACHE Software Foundation6 has a similar license and arrangements, but without the single-person control over dispute resolution (e.g., deciding whether or not to alter the official version of the software code) that is embedded in Linux and other software systems.
This committee does not have the expertise or charge within its task to prescribe the precise legal structure for management, support and improvement of the SMART Vaccines software and its associated data libraries, but it emphasizes the need for appropriate legal protection without curtailing the broad use and refinement of the software within a user-driven consortium.
____________
5 See www.opensource.org/licenses.
6 The APACHE Software Foundation is a “U.S. 501(c)(3) non-profit corporation [that] provides organizational, legal, and financial support for a broad range of over 140 open source software projects. The Foundation provides an established framework for intellectual property and financial contributions that simultaneously limits potential legal exposure for our project committers. Through a collaborative and meritocratic development process known as The Apache Way, Apache™ projects deliver enterprise-grade, freely available software products that attract large communities of users. The pragmatic Apache License makes it easy for all users, commercial and individual, to deploy Apache products.” See www.apache.org.
Future Improvements and Research
The committee suggests several areas for additional research and development to improve SMART Vaccines.
Setting Boundaries
In the multi-attribute utility model used in SMART Vaccines—which is described fully in the Phase I report Ranking Vaccines: A Prioritization Framework (IOM, 2012)—a number of attributes have numerical values that can take on a wide range, depending on the populations of interest. The multi-attribute utility theory process works best when the potential boundaries of each attribute are clearly defined for each population of interest. The upper boundary should represent a best-case scenario (spectacular success) in a vaccine, and the lower boundary should represent the worst-case scenario (failure or no effect).
As a specific example, consider the attribute “premature deaths averted,” which depends sensitively not only on the size of the population, but also on the intrinsic disease burden faced in each population. A lower bound on deaths averted would be zero (the vaccine has no effect), but the upper bound would depend on the disease burden confronting the population. In the United States, with a population of 310 million (2010 estimates), we know that the largest number of vaccine-preventable deaths currently arises from lower respiratory infections (at a rate of 18.5 per 100,000 or 57,300 deaths annually). In SMART Vaccines 1.1, we would specify half of that amount—28,650 deaths averted—as an upper bound representing the best-case scenario. This target for success would apply to all vaccines, and they would all be rated on how much of that potential “target” they could achieve.
Consider, by contrast, the deaths from infectious diseases in Bangladesh, which has a population of approximately 155 million. Lower respiratory infections are also the leading killer in Bangladesh, with 65.5 deaths per 100,000, for a total annual death rate of 101,525. Half of that—about 50,000 deaths—would be the upper-bound target in the Bangladesh. Australia, with a population of 24 million, also has more annual infectious disease deaths from lower respiratory infections than any other type of infection, but it has an annual rate of only 14.4 per 100,000 and for a total of 3,456 deaths, so half of that—1,728—would become its upper bound.
Thus, each population therefore requires a different upper bound simply because of its overall size, but also because of overall death rates from the disease that causes the most fatalities (which, in most cases in the World Health Organization database comes from lower respiratory infections). The death rates from lower respiratory infections exceed 200
per 100,000 in numerous African nations as well as some others such as Afghanistan. At the lower end, these rates drop into the single digits per 100,000 in some highly industrialized nations as well as in some less highly industrialized nations such as the Bahamas, Bahrain, and Costa Rica.
Clearly, boundary setting must have a population-specific focus. As discussed in Chapter 2, if the boundaries are set much too wide, the effect of that attribute on the SMART Score is blunted, and if the boundaries are set too narrow, the reverse occurs. Thus, the committee sees significant value in additional research to establish the best ways to set these boundaries within SMART Vaccines for all attributes with population-sensitive values and, hence, population-sensitive boundaries.
Granularity of User-Defined Attributes
Those attributes that do not depend on population values are assigned levels of success by the user. In the current version of SMART Vaccines, some of these attributes have simple Yes/No options (e.g., whether the vaccine “fits within existing immunization schedules” or “benefits military populations”). Some others are graded on a five-point Likert scale (e.g., “likelihood of financial profitability for the manufacturer”). These choices were made by Phase I and Phase II committee members working to develop the initial versions of SMART Vaccines. Further analysis could help determine the best granularity options on each of the user-defined attributes.
Age Granularity for Disease Burden and Costs of Care
For some priority-setting exercises, particularly in selecting among a number of existing vaccines that are all aimed at the same disease, the available options within SMART Vaccines do not perfectly accommodate the desired granularity of illness burden or treatment costs according to the age of affected people. Earlier committees made choices on these dimensions to balance user friendliness against data burden. Research can help clarify the best choices, which may differ from setting to setting.
Display Design
Using human factors engineering and cognitive psychology, one can study whether the current ways of presenting data and the SMART Scores visually in SMART Vaccines are the best options for users. Should there be additional development to look for alternative graphical methods for presenting these values? Further research can illuminate this question.
Indirect Benefits of Immunization
Vaccination programs can produce many benefits that standard cost-effectiveness or cost-benefit analysis cannot readily capture. SMART Vaccines captures some of these with user-defined attributes—such as the potential of the vaccine to target a disease that raises fear and stigma in the public, the possibility of completely eradicating a disease, or the ability to raise public health awareness—yet other indirect benefits remain unmeasured and could apply to some, if not all vaccines, particularly those affecting children. The current analytical model captures the direct economic benefits arising from the elimination of lost work days, for example, but it does not introduce the benefits of reduced illness burden (particularly chronic illnesses) on children’s abilities to advance further along their educational paths and thus enjoy such benefits as increased lifetime earnings and greater general productivity. These benefits in turn will provide positive spillovers to subsequent generations. Another possible effect of vaccines is the way that declining infant mortality may affect fertility rates. Extensive research has also shown strong empirical links between the two, but the causality can go in either direction, and research on this issue remains unsettled (NRC, 1998). Further research will help elucidate the best ways to measure these broader benefits in subsequent versions of SMART Vaccines.
Moving from SMART Scores to a Priority List
Unlike previous IOM reports (IOM, 1985, 1986, 2000), SMART Vaccines does not create a priority list, but rather it provides a tool that can be used to create many lists from different perspectives using multiple criteria. This does not mean, however, that SMART Vaccines could not be used to create a priority list if a group or organization chose to do so. This would require the stakeholder group or organization to determine a set of attributes to use in ranking vaccines and also determine the weights attached to each of those attributes. Given such a set of attributes and weights, SMART Vaccines would then readily create a list of vaccine priorities that bore the stamp of the sponsoring organization. The complexity comes in how the group goes about creating the desired set of attributes and their weights.
Mechanisms to convert individual preferences to group preferences have multiple complications associated with them. Numerous approaches have been devised to aggregate individual rank-order preferences into a group rank-order preference system. Systems for doing this always have some defect or another. Economist Kenneth Arrow demonstrated that it was impossible to create a system that unambiguously aggregates individual
preferences into societal preferences, unless the system either contained a dictator or the choice set was limited to only two options (Arrow, 1950). Subsequently, Gibbard (1973) and Satterthwaite (1975) showed that any system seeking to combine individual preferences into social preferences is subject to manipulation—unless, as with Arrow’s analysis, there exists a dictator. Subsequent analysis (Reny, 2001) has demonstrated the strong ties between these two obviously related understandings of the problems in creating societal preference rankings.
To move from individual preferences (e.g., those of members of some committee) to a group preference (i.e., the preferences of the committee itself) in the most useful way will require further specific research to understand the strengths and weaknesses of various approaches. Further research and stakeholder input can help illuminate not only the best mechanisms to assemble individual preference ranks into group preferences, but also the question of what group should be polled. Should it be a committee of experts or a population sample survey? Studying such issues should help enhance the value of SMART Vaccines in the future.
Expanded Uses of SMART Vaccines
Beyond the current sole purpose of SMART Vaccines—to prioritize new preventive vaccines for development—the committee believes that it would be fruitful to identify desirable expansions of the software, each of which would have the potential to widen the community of users, data development, and potential philanthropic and other support. The real benefit would come from increasing the scope and scale of the improved decision making made possible with SMART Vaccines.
Choosing Among Existing Vaccines
One alternative use, identified in the Phase II report, emerged as the primary use in one of the user groups: the New York State Department of Health. The team used SMART Vaccines to choose among competing vaccines that accomplish the same goal, i.e., the vaccination of infants against rotavirus or, in a second case, vaccinating against influenza. In the case of influenza, several vaccines exist in the market, many with important differences in their product profiles. The New York team sought to provide better advice to health providers in New York State about which of these vaccines to choose in various settings. In the rotavirus case, the key difference between the two existing vaccines was the number of doses required (two or three), but there were also differences in cost, coverage potential,
and vaccine efficacy. The team’s interest was choosing among competing existing vaccines that help prevent the same disease.
Comparing Vaccines with Other Public Health Interventions
Many diseases with a potential for prevention through vaccination programs can also be addressed through other approaches to reducing or eliminating the disease burden. While these other, non-vaccine approaches can be accommodated to some extent with the current version of SMART Vaccines, some software enhancements would make this task easier and more fruitful.
Examples of these situations abound. The example of malaria is often cited in public health. One can reduce malaria infections by reducing mosquito populations (through insecticide sprays, for instance) or by the use of window screens and mosquito netting. And in the many cases of infectious diseases that are waterborne, providing clean water supplies or methods of removing infection vectors from drinking water may offer alternatives to vaccination programs.
Resurrecting Shelved Vaccines
One application that the committee has contemplated—which was suggested during a stakeholder feedback session—is to bring together vaccine manufacturers from around the world to discuss the possibility of resurrecting vaccines that were previously aborted from the development process. The approach could very well be valuable because (a) most vaccine manufacturers have a set of vaccines in a partial development state that were not moved forward for reasons other than potential efficacy, (b) the manufacturers may know a lot more now about the science of these vaccines and the potential for success, and (c) they also may have updated knowledge about the underlying disease burdens and potential markets for these vaccines. Thus, it may be useful to apply SMART Vaccines to a set of aborted vaccines to see which of these is most likely to merit further development.
Animal Health and Veterinary Vaccines
In 2013, the global veterinary vaccine market was valued at $5.8 billion, with an expected growth rate of 8.1 percent over the following 5 years. Decisions about the development of new preventive vaccines are important for animal health and must, as with human vaccines, be based on multiple attributes. A subset of attributes already included in SMART Vaccines
could offer a helpful template and tool for improved decision making in the rapidly developing global animal health markets. The software would need to be revised in various ways, such as moving from human “workforce productivity loss” to economic issues associated instead with raising domesticated animals. It would also probably be important to take into account the spillover effects on human health of animal vaccines, because numerous public health concerns arise because of infections present in the animal food supply chain of humans around the world (or in other domesticated animal populations such as camels, which were implicated in the recent MERS outbreak).
From SMART Vaccines to SMART Health
The SMART Vaccines approach could, if desired, be applied to much broader prioritization questions in the area of public health. If, for example, one was considering investment either in research and development or provision of services through health care systems, then the SMART Vaccines approach—using multi-attribute utility theory to help clarify tradeoffs, benefits, costs, and risks from various health care interventions—could assist decision making across a wide spectrum of organizations. Doing this would require a significant expansion of the current software, and the committee lists this option only for completeness, with no implication that the current version could fulfill this role. But if a group of interested parties desired to do so, then it would be quite natural to extend the SMART Vaccines model to a broader SMART Health.
With SMART Vaccines, interested entities can make a substantial change to how they approach decision making. This software product is a significant and novel creation from the National Academies, which has not produced software from scratch in any of its previous studies. As a multi-stakeholder decision-support system, the software has the potential to change the practices of many parties in the vaccine enterprise—suppliers, users, and supporters of vaccine deployment, both domestically and internationally. Many of these organizations already have processes in place to help prioritize their decisions about the development and deployment of vaccines. Thus, embracing SMART Vaccines as a tool to assist in these processes would require an investment of both time and other resources that these organizations may see as potentially unnecessary. For those organizations without formal decision models to assist in their prioritization efforts, sim-
ilar resource investments will be required, but they would not be viewed as duplicating existing tools and processes.
Having studied the work of Harvard Business School’s John Kotter, a specialist in understanding organizational change (Kotter, 1995), the committee recognizes that successful change requires multiple steps, taken in the proper sequence. These steps include (1) creating a sense of urgency; (2) forming a powerful guiding coalition; (3) creating a vision, explaining it to others, and empowering people to act upon it; (4) creating some short-term “wins” that others can see and emulate; and (5) embedding the changes in the culture of the institution.
The sense of urgency must ultimately come from within the various stakeholder organizations, but the committee believes that this sense of urgency can be helped along by the observation that the various organizations—supply, demand, and facilitation of vaccine development and deployment—all use different approaches to prioritization, each of which involves different metrics and tools. Thus, these organizations have no common language to speak, no common data to share and discuss, and limited ways of bridging the gaps (perhaps even chasms) in their collective understanding about the best pathways forward in vaccine development and deployment. SMART Vaccines can serve as the basis for narrowing or removing these gaps. It need not replace the tools and approaches used by the many stakeholder organizations, but it can serve as a way to help them understand each other’s goals, aspirations, and constraints. Finally, the economic challenges and profound changes seen in today’s health care system should also create a sense of urgency to improve disease prevention strategies—which this committee believes will help move SMART Vaccines into widespread use.
Finding a permanent home supported by a user group with a formalized leadership structure for advancing SMART Vaccines would at least begin to fulfill the second step identified by Kotter—creating a powerful guiding coalition. As to the “vision” issues, the committee members believe that the proposed future pathway for SMART Vaccines—a software system maintained, enhanced, and improved by an active user community—provides the basis for a shared vision of how to reach an improved future. The proposed user community could also help create the necessary “wins” and share them with other users, an essential feature of successful change as understood by Kotter and others. The process for embedding the changes into the institutional practices of the many stakeholder organizations will necessarily remain the task of those organizations themselves as they continue to manage and lead change through our dynamic and challenging times.
