C
Data Sources and Methodology for SMART Vaccines

Kinpritma Sangha, M.P.H.
Institute of Medicine




SMART Vaccines is a data-intensive tool and compilation of pertinent, high-quality data can be burdensome. At its core, SMART Vaccines relies on three types of data for producing final scores:

Readily available “known-knowns.” Some examples of this type of data would include population data for most nations of the world (and sometimes for smaller areas within nations such as states and counties in the United States). These data are available from several sources such as the World Health Organization and provide the lowest level of challenge for future users of SMART Vaccines. In high-income countries, economic and epidemiologic data resources are also available. These types of data include disease burden, typical patterns and cost of treating these diseases, productivity loss arising from these diseases, and similar data needed as inputs into SMART Vaccines. However, these types of data may be available only sporadically or not at all in low-income countries with less investment in data gathering infrastructures.

Theoretically knowable “unknown knowns.” Some data that are available readily in high-income countries would be in theory be available in low- income countries as well, but do not exist. In these cases, potential users of SMART Vaccines will have to decide whether to make new investments in data gathering infrastructures or to rely on low-cost approximations for initial priority setting exercises using SMART Vaccines.

In many settings, the most useful way to get sufficient data to begin



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 99
C Data Sources and Methodology for SMART Vaccines Kinpritma Sangha, M.P.H. Institute of Medicine SMART Vaccines is a data-intensive tool and compilation of pertinent, high-quality data can be burdensome. At its core, SMART Vaccines relies on three types of data for producing final scores: Readily available “known-knowns.” Some examples of this type of data would include population data for most nations of the world (and some- times for smaller areas within nations such as states and counties in the United States). These data are available from several sources such as the World Health Organization and provide the lowest level of challenge for future users of SMART Vaccines. In high-income countries, economic and epidemiologic data resources are also available. These types of data include disease burden, typical patterns and cost of treating these diseases, produc- tivity loss arising from these diseases, and similar data needed as inputs into SMART Vaccines. However, these types of data may be available only sporadically or not at all in low-income countries with less investment in data gathering infrastructures. Theoretically knowable “unknown knowns.” Some data that are available readily in high-income countries would be in theory be available in low- income countries as well, but do not exist. In these cases, potential users of SMART Vaccines will have to decide whether to make new investments in data gathering infrastructures or to rely on low-cost approximations for initial priority setting exercises using SMART Vaccines. In many settings, the most useful way to get sufficient data to begin 99

OCR for page 99
100 RANKING VACCINES: A Prioritization Software Tool the priority setting process is to obtain best expert estimates, often relying on different individuals for different types of estimates. This could be done in a more formalized structure using group consultation techniques such as the Delphi methods or its variations for estimating unknown data. The sensitivity analysis capabilities of SMART Vaccines provide another way to help users understand the importance of improving the quality of various types of data. If the user finds that within their context— and using their own weights—and across a wide range of possible data lev- els that the priority rankings do not change, then it is an indication that further investments to improve those data elements is not necessary. Alter- natively, if the priority rankings are sensitive to the levels of certain data elements, then it signals the importance of investing resources to improve those data. Currently unknowable “known unknowns.” These are data whose nature is well known but they do not exist, including the vaccines that do not yet exist, and the nature and hence the dangers of diseases that do not yet exist. Data Categories Specific data needed for SMART Vaccines can be categorized into four broad categories: population, epidemiology, economics, and vaccine- related characteristics. As mentioned in this report, the data are merely estimates derived from available information in order to offer guidance for further data collection. The primary sources for SMART Vaccines data were publications reporting primary epidemiologic and economic data or else reporting suf- ficient information to derive estimates of the primary data. Studies report- ing data on a national scale were given precedence over those that analyzed populations on a state, county, or provincial basis. In some instances when national estimates were unavailable, estimates from a smaller subset of a population were extrapolated to the entire population. Indirect estimates from mathematical modeling studies were consulted when firsthand data were unavailable. Specific source references are embedded in the data spreadsheets. As part of the Phase II data collection efforts, the disease burden and vaccines data for Phase I candidates were revised to closely reflect the pop- ulation measures of the 2009 U.S. and South African populations as used in the software. Data will naturally differ from year to year as updated sources become available. The examples provided here represent only a subset of all data sources and estimation approaches; they are not representative of

OCR for page 99
Appendix C 101 the data from all the surveillance measures that are available to the user. All datasets and specific source references are embedded in the data spread- sheets that are available online with the report and the software. To ensure best estimates for each disease and associated vaccine, the data inputs were verified and standardized to the 2009 U.S. and South African populations. Disease burden data were verified internally within the model and also against existing estimates. Number of deaths and cases were standardized to the 2009 U.S. and South African populations using estimates for each age group (available from life tables) to obtain the data needed for the model in the following four age groups: less than 1 year old, 1 to less than 20 years old, 20 to less than 65 years old, and 65 years or older. Next, given the number of cases and deaths within a population, the incidence and case fatality rates were calculated using the following for- mula: Incidence = (number of cases/total population)*100,000, and Case fatality rate = (number of deaths/number of cases). The calculated inci- dence and case fatality rates are checked with original number of deaths and cases, followed by verification using existing publications. To the extent possible, costs were also standardized to 2009 U.S. dollars; however, in many instances cost estimates from another year were used. Standard Data Sources Centers for Disease Control and Prevention (CDC): CDC is a reliable source of disease- and vaccine-specific information within the United States. Its surveillance systems follow national guidelines within its scope of work to maintain standardized data collection. For instance, two surveillance systems collect information about rotavirus disease and the rotavirus vac- cine: the National Respiratory and Enteric Virus Surveillance System and the New Vaccine Surveillance Network. Similarly, there are multiple sur- veillance systems for influenza, including FLU VIEW (a weekly influenza surveillance report) and International Influenza Surveillance. Information from surveillance systems on mortality and morbidity are available through the Morbidity and Mortality Weekly Report (MMWR). The MMWR series, prepared by the CDC, contains disease burden data from 1990 to present. World Health Organization (WHO): The WHO Global Health Observatory (GHO) data repository provides epidemiologic and health indicator data for WHO’s 194 member states. The GHO data repository contains more than 50 datasets on priority health topics, including the mortality and bur- den of diseases, immunization, and health systems. Annual summaries of health-related data are also available for member states. WHO Choosing

OCR for page 99
102 RANKING VACCINES: A Prioritization Software Tool Interventions that are Cost Effective (WHO-CHOICE) assembles regional databases on the costs, population health impact, and cost-effectiveness of key health interventions. Using WHO-defined regions, WHO-CHOICE developed standard tools and methods to generate regional databases for collecting costs and health data. The costs and effectiveness of a wide range of health interventions are determined with probabilistic uncertainty anal- ysis. Currently there is also a contextualization tool that makes it possible to adapt regional results to the country level. Information about health interventions, demography, epidemiology, and cost effectiveness analyses for certain diseases are also available by WHO regions. Global Burden of Disease (GBD): GBD (most recent version released in 2012) is an international collaboration that describes the global distribution and causes of a wide array of major diseases, injuries, and health risk factors. GBD provides data on, among other things, age and sex-specific mortality, global and regional mortality from 235 causes of death, disability-adjusted life years for 291 diseases, and healthy life expectancy for 187 countries. Healthcare Cost and Utilization Project (HCUPnet): HCUPnet is a free online data system that provides access to health statistics and information on U.S. hospital inpatient and emergency department utilization, both at the national and the state level. The Nationwide Inpatient Sample (NIS) is particularly important because it contains information found in a typi- cal hospital discharge or billing record. Using this information, HCUPnet provides data for specific conditions and their associated durations of stay, hospital costs, national costs, percent of patients who died in the hospi- tal, and discharge status. However, the national averages are not suited for regional analyses because of geographic differences among and within states both in health care utilization and in costs. Custom Data Sources When high-quality data were unavailable or when data sources varied in quality, accuracy, and comprehensiveness, proxy measures were used to produce estimates. Estimates of vaccine manufacturing costs were largely informed by vaccine industry experts on the committee. When the age- specific disease burden for certain diseases was unavailable in South Africa, epidemiologists based in South Africa were contacted via e-mail or tele- phone for their assistance in developing estimates. Furthermore, disease burden data from South Africa provinces were used in place of national assessments, and vaccine costs and analyses from other low- and middle-

OCR for page 99
Appendix C 103 income countries were used to substitute for costs of vaccines in South Africa. Data Collection Methodology Population Characteristics: SMART Vaccines was designed to accommo- date demographic differences within a country, a region, or a consortium of countries. The U.S. and South African population data were obtained from the WHO country statistics database, which houses actuarial data for various countries. Population data for each country are given by sex and are divided into 5-year age intervals, with the exception of children under 5, who are further divided into children up to 1 year in age and children from 1 to 5 years old. The data provided for each age interval are the total popula- tion (N), the number of people left alive at age x (lx), person-years lived between ages x and x+n (nLx), and life expectancy at age x (ex). The stan- dard life expectancy (sx) is the life expectancy for Japanese women—the group that is known to have the world’s longest life expectancy and so is used in calculations for disability-adjusted life-years. The health utilities index 2 (HUI2) is used to estimate the quality of life for people in the vari- ous age intervals in order to calculate quality-adjusted life-years. Because HUI2 data are unavailable for South Africa, U.S. estimates were used. The hourly wage rate is used in estimating the value of time lost to illness; the hourly wage rate of parents is used for children less than 15 years old. The wage rate for South Africa was approximated in U.S. dollars by using the prevailing currency exchange rate. Special populations can include groups manifesting specific char- acteristics that ought to be considered in developing or delivering a vaccine—for instance, immunocompromised individuals with multiple co- morbidities, or people in a particular state or province within a country. A population table divided into males and females will need to be filled in for this group as well. To illustrate this further, let us consider a special population that includes HIV-positive individuals. If nearly 2 percent of a country’s total population is living with HIV, then the absolute number of people living with HIV can be obtained by multiplying the total population by 0.02. If we assume, for instance, that the life expectancy of an HIV-positive individual is 10 years less than that of a healthy individual, then it is a straightforward matter to fill in the life table for this special population using the standard life table. Similarly, an HIV-positive individual will likely have a lower qual- ity of life and hence a lower HUI2. However, these assumptions will change

OCR for page 99
104 RANKING VACCINES: A Prioritization Software Tool depending on the country in question. In a low-income country, HIV is a debilitating condition physically, economically, and socially, whereas it can be managed with anti-retroviral drugs in the United States. Similarly, a health official may be interested in focusing on a particu- lar province within a country as a special population. In addition to speci- fying the total population, there would be another step to define the special population—the total number of people (N) in the province, the number of people left alive at age x (lx), the person-years lived between ages x and x+n, (nLx), and the life expectancy at age x (ex). A major concern in cre- ating subsets of a special population from the total population is double counting.1 To avoid this error, it is important to subtract the special (prov- ince) subset from the total (country) population to represent the different population, disease and vaccine characteristics between the two groups. When using SMART Vaccines to prioritize several vaccines in a pop- ulation, the demographic data defining that population remain constant, while parameters specifying disease burden and vaccine-related informa- tion vary. Disease Characteristics: To illustrate the methods used to collect data on disease characteristics, we will use the case of influenza in the United States as an example. The first step in estimating disease burden is to esti- mate age- and sex-specific incidence and case-fatality rates for the selected population. In this case, contrasting claims for the magnitude of influenza incidence and case-fatality rate must be reconciled in order to capture the appropriate and complete burden of disease. This is done by calculating the number of deaths by age and sex using incidence and case fatality rate, and the figures must sum to the total number of deaths for that age–sex group estimated via the standard data sources described above. Because vaccine-preventable diseases affect sex and age groups dif- ferently, it is important to make the distinction between, for example, the disease burden caused in infants and the disease burden in the elderly. This is an important consideration for decision makers thinking about invest- ment in pediatric vaccines versus adult vaccines. Thus, disease burden is specified by sex and in the following age intervals: infants (less than 1 year), children (1 to <20 years), adults (20 to <65 years), and elderly (65 years or older).These age intervals are selected to reflect the availability of data because most disease burden is measured in aggregated age groups of infants, children, adults and elderly. These categories also relieve user 1  A discussion on potential double counting can be found in Phase I report, pp. 63–65 (IOM, 2012).

OCR for page 99
Appendix C 105 burden because the software imports the previously specified age-specific population data to create the aggregated age intervals for the disease bur- den profile. Despite the difference in disease burden due to age, influenza affects both sexes equally, thus the incidence and case fatality rate are assumed to be the same for both males and females. However, conditions affecting cer- tain ages or sexes disproportionately will generally have higher mortality for certain groups; for instance, human papillomavirus has higher mortal- ity rates in women because of cervical cancer, while rotavirus causes high mortality in children in low-income countries. For influenza, the epidemiologic information was obtained from a CDC-based publication examining disease burden and costs associated with seasonal influenza in the United States (Molinari et al., 2007). To estimate age and sex-specific deaths for influenza, consider influenza in female infants (less than 1 year old) in 2009 in the United States: The inci- dence is 20,300 cases per 100,000 people, there are 2,183,518 female infants (less than 1 year old) and the case fatality rate (the proportion of deaths within those affected with the disease cases) is 0.000040. This information is then used by the software to calculate the number of deaths due to influ- enza in female infants in the United States by using the following formula: incidence*the population (N) in the age group*case fatality rate = [(20,300/ 100,000)*2,183,518*0.000040] = 17.73 female infant deaths. Because disease burden includes both mortality and morbidity, SMART Vaccines allows the user to specify morbidities associated with the disease in question. Morbidities are any conditions causing health and economic burden due to the disease and can be specified simply by stating the consequent condition with its severity, such as mild (influenza without outpatient visit), moderate (influenza without outpatient visit), and severe (influenza with inpatient visit). Morbidities can also include other complications that are a conse- quence of the principal condition, such as meningitis, sinusitis, and otitis media due to pneumococcal infection or fever and abdominal pain due to rotavirus. Morbidities are obtained from the disease burden publications relevant for each condition and vaccine (Molinari et al., 2007; O’Brien et al., 2009; Payne et al., 2008). Health Utility Index 2 and Disability Weights: A reduction in the health- related quality of life for the amount of time an individual is sick is repre- sented using the HUI2. For instance, if a healthy individual with an average HUI2 score of 0.99 is home sick with the influenza for 3 days, his or her quality of life may drop down to, say, 0.90 or perhaps even 0.80 in severe

OCR for page 99
106 RANKING VACCINES: A Prioritization Software Tool cases. To account for this reduction in quality of life due to morbidity, disu- tility tolls are calculated. Disutility tolls represent the difference between the HUI2 of the healthy state prior to illness (0.99) and the state during sickness (0.90), or an overall toll of 0.09. HUI2 values were used because these are available for the general U.S. population and for the disease and vaccine disutility tolls. The disu- tility tolls were taken from articles in the refereed literature that directly measured disutilities related to the target conditions. In the absence of such data either reported directly or else indirectly supported in the literature (e.g., used in a cost-effectiveness analysis with substantiated disutility esti- mates), estimates were obtained using nearest analogy health states to seg- ment the data (Fryback, 2009; Fryback et al., 2007). Individuals’ answers to the quality of well-being scale were used to identify relevant health states. The HUI2 values were regressed on an indicator for the health condition and age, the regression coefficient for the indicator being a rough estimate of the toll. The disutility tolls are also rough estimates. There are several other health utility measures that are available, such as HUI2, HUI3, and EQ-5D, and information from them can be used as well. The only caveat is that users should be consistent throughout the model because the health utility and disutility information is used to calcu- late QALYs, which will not be appropriately computed if different indexes are used. For example, if HUI3 data are used for the population tables, it is best to also use HUI3-related disutility tolls for the disease and vaccine morbidities, as listed in the disease morbidity and vaccine complications in the Phase I report, Ranking Vaccines: A Prioritization Framework (IOM, 2012). To estimate HUI2 tolls for influenza, individuals’ answers on the quality of well-being scale were used to identify relevant health states—e.g., a day with influenza was equated to a positive answer to Question 2(w), which asks respondents which of the past 3 days they have had fever, chills, or sweats. The HUI2 was regressed on an indicator for the health condition and age, and the regression coefficient for the indicator served as a rough estimate of the toll.  To estimate disability weights, datasets from the GBD were used (Salomon et al., 2012). The GBD provides disability weights by categories, such as communicable diseases, cancers, and chronic diseases. For this report, proxies for DALY weights were identified from the GBD 2012 list of conditions that were sufficiently similar to the morbidities in question. Specifically, the proxies were estimated based on the severity of the dis- ease. For instance, GBD 2012 lists the following for infectious diseases: (1) acute episode, mild = 0.005, (2) acute episode, moderate = 0.053, and (3)

OCR for page 99
Appendix C 107 acute episode, severe = 0.210. Using these estimates as well as the judg- ment of clinicians, the DALY weight for influenza was derived based on the severity of the condition—mild, moderate, and acute. Economic Characteristics: The economic burden of each disease was esti- mated at the population level. Both payer and societal perspectives were used to calculate direct medical costs, vaccine delivery costs, and work- force productivity costs. Costs were estimated for four outcomes: (1) death due to the disease, (2) outpatient visit due to the disease, (3) hospitalization (with the disease as primary diagnosis), and (4) medication costs (includ- ing over-the-counter drugs). For deaths that occurred in the hospital, the cost for an inpatient admission that resulted in a fatality was obtained using the 2010 HCUPnet. Cost per case was averaged using the disease-associated diagnostic catego- ries, such as the International Classification of Diseases, Ninth Revision (ICD-9) codes for primary diagnosis in the United States (AHRQ, 2012). Hospital costs for a death that occurred due to influenza were estimated to be $6,000. For hospitalization, the cost for an inpatient admission that resulted in a discharge was also obtained using information from the 2010 HCUPnet. Again, cost per case was averaged using the disease-associated diagnostic categories, such as the ICD-9 codes for primary diagnosis in the United States as listed in HCUPnet (AHRQ, 2012) In the case of influenza, hospi- talization costs were not included in the analysis. Outpatient visits include costs for visits to the physician that did not include hospital admission. Direct medical expenses for outpatient visits included physician costs and outpatient and pharmaceutical needs, such as lab tests, imaging tests, and consults. The outpatient visit costs for influenza were estimated at $250. For cases in which the patient did not seek medical attention, the direct costs were assumed for over-the-counter medications. For example, the average over-the-counter influenza medications cost $3 in the United States (Molinari et al., 2007). Vaccine Characteristics: If the vaccine under consideration currently exists, data for coverage costs were obtained from CDC and WHO for the United States and South Africa. Vaccine effectiveness and length-of-immunity information were derived from published literature in vaccine random- ized control trials. Because effectiveness and length of immunity depend on the location of the population as well as age, sex, and environment, these estimates are specific to regions and demographics. SMART Vaccines also allows the option to “turn on” herd immunity; if herd immunity is applied,

OCR for page 99
108 RANKING VACCINES: A Prioritization Software Tool indirect protection is conferred when vaccine coverage is greater than or equal to 80 percent of the target population. Once vaccine coverage reaches the threshold of 80 percent, there are no additional benefits from increas- ing vaccination coverage because those who are unvaccinated receive protection (according to vaccine effectiveness) from those who have been vaccinated. Data for cost per dose, doses required per person, and cost to admin- ister per dose were purely hypothetical and based upon proxy vaccines. Cost per dose is generally the price paid by the government to purchase wholesale vaccines from manufacturers. Cost to administer a dose are the costs involved in provision of the vaccine. Because SMART Vaccines con- siders new preventive vaccines for which these data do not exist, those vac- cines that are judged to be sufficiently similar to the one under consider- ation are used to derive this information. Estimates from industry experts on the committee provided estimates for research and development costs, time to adoption, licensure costs, and one-time start-up costs. Vaccine safety is not quantified in the current version of SMART Vaccines, it is only included in the qualitative attributes because vaccines considered within the software are hypothetical and may not have this information readily available. References AHRQ (Agency for Healthcare Research and Quality). 2012. HCUPnet, healthcare cost and utilization project. http://hcupnet.ahrq.gov. Fryback, D. G. 2009. United States national health measurement study, 2005– 2006. http://www.icpsr.umich.edu/cocoon/NACDA/STUDY/23263 xml: Inter-university Consortium for Political and Social Research (ICPSR) [distributor]. Fryback, D. G., N. C. Dunham, M. Palta, J. Hanmer, J. Buechner, D. Cherepanov, S. Herrington, R. D. Hays, R. M. Kaplan, and T. G. Ganiats. 2007. U.S. norms for six generic health-related quality-of-life indexes from the national health measurement study. Medical Care 45(12):1162–1170. IOM (Institute of Medicine). 2012. Ranking Vaccines: A prioritization framework: Phase I: Demonstration of concept and a software blueprint. Washington, DC: The National Academies Press. Molinari, N.-A. M., I. R. Ortega-Sanchez, M. L. Messonnier, W. W. Thompson, P. M. Wortley, E. Weintraub, and C. B. Bridges. 2007. The annual impact of seasonal influenza in the U.S.: Measuring disease bur- den and costs. Vaccine 25(27):5086–5096.

OCR for page 99
Appendix C 109 O’Brien, K. L., L. J. Wolfson, J. P. Watt, E. Henkle, M. Deloria-Knoll, N. McCall, E. Lee, K. Mulholland, O. S. Levine, and T. Cherian. 2009. Burden of disease caused by streptococcus pneumoniae in children younger than 5 years: Global estimates. Lancet 374(9693):893–902. Payne, D. C., M. A. Staat, K. M. Edwards, P. G. Szilagyi, J. R. Gentsch, L. J. Stockman, A. T. Curns, M. Griffin, G. A. Weinberg, C. B. Hall, G. Fairbrother, J. Alexander, and U. D. Parashar. 2008. Active, population- based surveillance for severe rotavirus gastroenteritis in children in the United States. Pediatrics 122(6):1235–1243. Salomon, J. A., T. Vos, D. R. Hogan, et al. 2012. Common values in assess- ing health outcomes from disease and injury: Disability weights mea- surement study for the global burden of disease study 2010. Lancet 380(9859):2129–2143.

OCR for page 99