Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
CROSSCUTTING DATA ISSUES FOR LEADING HEALTH INDICATOR SETS 63 5 Crosscutting Data Issues for Leading Health Indicator Sets DATA SOURCES FOR PROPOSED LEADING HEALTH INDICATOR SETS As noted previously, data are the foundation of Healthy People (U.S. Department of Health and Human Services, 1998a). The availability of data is requisite for implementation and evaluation of the entire Healthy People 2010 initiative and, in particular, for the leading health indicator component to ensure ongoing assessment of each indicator. Regular monitoring of and reporting on the relevant data sets will ensure that information about the selected set and its indicators is available to inform the public about and maintain the awareness of the indicators in the mind of the public. Currently, more than 200 data sets contribute to an electronic database, the DATA2000 Monitoring System, that was established for Healthy People 2000 and that is maintained by the Centers for Disease Control and Prevention. The DATA2000 Monitoring System is a component of the Centers' WONDER system and contains the national baseline and monitoring data for all measurable objectives established for Healthy People 2000. It is expected that the DATA2000 Monitoring System will be updated to include all of the measurable objectives contained in Healthy People 2010. It will also be imperative to ensure that data for each of the indicators in the selected set are incorporated into the updated version of the DATA2000 Monitoring System. The general public currently has access to the DATA2000 Monitoring System through the Internet (http:// www.cdc.gov/nchswww/datawh/cdcwond/d2000ms/d2000ms.htm), and detailed statistical summary reports are published in the National Center for Health Statistics' Healthy People 2000 Statistical Notes (http:// www.cdc.gov/nchswww/default.htm). These, however, are not considered viable strategies for effective, broad- based communication of important results pertaining to progress toward the specific leading health indicators in the selected leading health indicator set. The content and format of the Statistical Notes publications are best suited for professionals involved in the public and private health care delivery systems. As such, they are very unlikely to inform the public about leading health indicators. Furthermore, although access to the Internet is quickly expanding, many populations such as elderly people, racial or ethnic minority groups, and groups of lower socioeconomic status may be less likely to know about the Internet, may have no or limited access to the Internet, and may not consider results from the DATA2000 Monitoring System to be of personal relevance or pertinence. Consequently, alternative strategies for the reporting of results from the DATA2000 Monitoring System will have to be developed. The U.S. Department of Health and Human Services maintains the primary data sets that contribute to the DATA2000 Monitoring System. These include the following:
CROSSCUTTING DATA ISSUES FOR LEADING HEALTH INDICATOR SETS 64 1. vital statistics, 2. National Health and Nutrition Examination Survey, 3. National Health Interview Survey, 4. Youth Risk Behavior Survey, 5. Primary Care Provider Survey, 6. National Survey of Worksite Health Promotion Activities, 7. National Survey of Family Growth, 8. Behavioral Risk Factor Surveillance Survey, 9. National Household Survey on Drug Abuse, 10. National Hospital Discharge Survey, 11. National Notifiable Disease Surveillance System, and 12. National Immunization Survey. A brief description of the 12 data sets is provided in the publication Leading Indicators for Healthy People 2010 (U.S. Department of Health and Human Services, 1998aa). These descriptions provide information about the purpose of the survey, the general content and format of the survey method of data collection, and the periodicity of data collection efforts. More detailed information is available from the U.S. Department of Health and Human Services and the particular agencies responsible for each data collection effort. It is expected that these 12 data sets will play a lead role in providing ongoing monitoring of the proposed leading health indicators for the U.S. Department of Health and Human Services. Each of the 12 data sets have been or will be modified to comply with current federal policies about the collection and reporting of race and ethnicity data. This is of critical importance to ensure that the appropriate subgroup analyses can be completed for the leading health indicators. Additional federal databases may contain data that inform the proposed leading health indicator within the chosen set, such as motor vehicle accident rates, injuries, and deaths as a subset of preventable morbidity and mortality (National Traffic Safety Board), occupational injuries and deaths as a subset of preventable morbidity and mortality (U.S. Department of Labor), environmental data (Environmental Protection Agency), tobacco consumption patterns (Internal Revenue Service), the level of dissemination of information to patients about primary prevention behaviors such as exercise and immunizations (National Ambulatory Medical Care Survey), and levels of education (U.S. Department of Education). Succinct summaries of these and many other potential data sources that can be used to inform the proposed leading health indicators can be found in a number of recent publications including A Compendium of Selected Public Health Data Sources (U.S. Department of Health and Human Services, 1996), Key Monitoring Indicators of the Nation's Health and Health Care and Their Support by NCHS Data Systems (Lewin-VHI, 1995), and Data Sources for Monitoring Progress Toward the Year 2000 Objectives for the Nation (Research Triangle Institute, 1990a, b). Identification of the specific data sets that will be used to quantify the baselines and targets for the leading health indicators in the selected set is beyond the scope of this report for several reasons: (1) data will be available from multiple sources and it will be necessary to select the best data set(s) to inform each of the indicators; (2) the final format, structure, and language for each of the indicators and the selection of effective communications strategies will be determined only after one of the three proposed indicator sets is chosen; (3) the availability of data at the national, state, local, and community levels may be subject to change; and (4) reliability and validity of data for each of the indicators must be determined. Each of these issues is discussed in greater detail below. First, among the eligible data sets, more than a single data set can capture information about specific indicators by different methodologies and sampling techniques. For example, information about cancer
CROSSCUTTING DATA ISSUES FOR LEADING HEALTH INDICATOR SETS 65 screening behaviors can be provided by the Behavioral Risk Factor Surveillance Survey, National Health and Nutrition Examination Survey, Primary Care Provider Survey, and the Medicare Current Beneficiary Survey (U.S. Department of Health and Human Services, 1998aa). Similarly, the practice of healthy behaviors such as rates of exercise, non use of tobacco, and non use of illicit substances by the general population or specific age cohorts can be quantified by results from the National Health and Nutrition Examination Survey, Youth Risk Behavior Survey, Behavioral Risk Factor Surveillance Survey, National Survey of Worksite Health Promotion Activities, National Survey of Family Growth, and National Household Survey on Drug Abuse. Actual selection of the appropriate data sets for each leading health indicator within a set will best be accomplished in cooperation with federal personnel associated with the design and administration of these diverse surveys and the National Center for Health Statistics because these individuals will have the greatest familiarity with the strengths and limitations of each data set. In addition, results from the research recommended by this committee will be used to finalize the actual format, structure, and language for the each of the leading health indicators and develop effective strategies to support their dissemination to the public. Results from this research will not be available for several months. In the absence of information about issues such as placement of the indicators within a set (which might imply prioritization to the public) or the final language chosen to ensure broad communication of the indicators and their measures to the general public and diverse population groups, it is premature to select a specific survey or surveys to provide the requisite data on measurable indicators. Similarly, it would be premature to recommend new data collection initiatives. The ability of existing surveys to include data availability at the national, state, local, and even census tract levels may also be subject to change. Such changes might be predicated on the mandate to ensure adequate representation of target populations in meaningful subanalyses according to racial and ethnic minority group, gender group, age group, income group and disability status. Survey content and sampling frame are two dimensions that could be significantly altered to respond to this mandate. Similarly, the age adjustment modification from 1940 to 2000 has the potential to affect the distribution of populations within the sampling frames for each survey. These two factors pose significant challenges to the selection of specific surveys for use in the measurement of individual leading health indicators. Finally, reliable and valid data on indicators that might reflect social norms or a bias to give the best or most socially acceptable answers on topics such as income, level of educational attainment, disabilities associated with mental health, and substance abuse are not readily available from the 12 major sources of data. This suggests that these surveys will require evaluation and modification to ensure the validity of the data for these indicators or that new data collection efforts will be required to begin to collect this information. It is likely that survey modification will best be undertaken for some indicators, whereas new data collection efforts will be appropriate for others. It is premature to select data sources for the proposed leading indicators until decisions are made about the best way to resolve these issues and until the appropriate modifications to data sets are complete. GENERAL DATA ISSUES FOR PROPOSED LEADING HEALTH INDICATOR SETS Data Quality As new and existing data sources are considered for the leading health indicators for Healthy People 2010 a number of overriding issues will warrant careful consideration. First, the quality of the data will merit intense scrutiny, particularly for those efforts that are based on new or expanded data collection initiatives. Compromises to data quality can occur at multiple levels including the operationalization of
CROSSCUTTING DATA ISSUES FOR LEADING HEALTH INDICATOR SETS 66 measures, uniformity in structure and content of specific questions, adherence to standardized methods of data collection and recording, and adherence to standardized methods for data management and analysis. Limitations of Self-Reported Data It will also be important to recognize that the majority of existing potential data sets that might be used to measure the leading health indicators rely on either telephone, in-person, or mail surveys. Although, these are scientifically sound, credible, and well-established methods of data collection, they may be vulnerable to biases inherent in self-reporting. Consequently, specific indicators of health promotion and health protective behaviors as well as some social determinants of health such as income and education that may be susceptible to self-reporting biases may benefit from new data collection efforts that do not rely on self-reporting or third-party confirmation of self-reported findings. Data Validity and Reliability In addition, attention must be given to factors that might affect the validity and reliability of the data collected for specific indicators. It will be essential to establish the construct validity of data for each of the leading health indicators, especially those that will rely on new data collection efforts. Further consideration must also be given to the reliability of the data collected on the leading health indicators. The issue of the validity of responses over time will be of particular significance to the success of efforts intended to establish ongoing monitoring and feedback to the public regarding each indicator. It is likely that many of the proposed indicators will rely on data from sources that are well established and that have been evaluated for their scope of representation as well as validity and reliability. However, if specific questions in existing data collection efforts require modification or if new data collection efforts must be established to inform the selected leading health indicator set accurately, the representativeness of sampling frames along with the validity and reliability of new questions will have to be determined before application of the data to that indicator set. Periodicity of Data Availability The periodicity or frequency with which data for certain indicators will be available will also be of concern. Data collection efforts may not be performed at sufficiently frequent intervals, and thus, information will not be available to measure specific indicators on a routine basis. Inconsistencies in the frequency of data collection affects both federal and nonfederal sources. For example, the National Health and Nutrition Examination Survey was most recently conducted during the interval from 1988 to 1994; the next National Health and Nutrition Examination Survey will begin in 1999. Consequently, this survey will be of limited value in providing ongoing information for some of the proposed leading health indicators as will the National Health Interview Survey, which has a lag time for data reporting and which contains supplementary questions that are not asked on an annual basis. Variable reporting frequencies have the potential to undermine dissemination efforts to ensure that the leading health indicators will achieve and maintain a constant level of public awareness and provide ongoing feedback about progress toward achieving the indicator targets.
CROSSCUTTING DATA ISSUES FOR LEADING HEALTH INDICATOR SETS 67 Timeliness of Data Availability The timeliness of data availability will be essential to ensure that the indicators are updated and presented at a regular frequency to the public to maintain a constant level of awareness that will, in turn, motivate changes in communities, health systems, and individual behaviors. Such regular data updates will also establish the necessary feedback that will underscore the need for the intensification or redirection of efforts to achieve specific targets for the various indicators. Current schedules for the collection, analysis and reporting of data for existing data sets may present a challenge to the achievement of regular updates on the status of the leading health indicators. For example, the provision of final vital records data may take up to 3 or 4 years following the year in which the data were actually collected. In fact, final vital statistics data for 1995 became available only in 1998. If this remains the case, vital statistics will be of limited utility in providing the necessary updates on leading health indicators associated with mortality, natality, and social variables. Consequently, those involved in analysis of and reporting on data for the leading health indicators will have to rely on alternate sources of data that can be readily available within the shortest amount of time possible without compromises to validity or reliability. Representativeness of Data It will also be essential to consider the representativeness of survey data obtained from statistical samples that will be used to monitor each of the leading health indicators. This is a particular threat to analyses that will be required to characterize the status of specific population groups for the selected indicators. Most national sample surveys have complex sampling frames, and this complexity must be addressed during the data analysis phase to ensure valid estimates on the performance of specific leading health indicators for the general population as well as diverse population groups. Personnel involved with the actual analysis of data for multiple demographic and health status groups as well as multiple jurisdictions (the national, state, local, and community levels) will have to be well versed in the specific strengths and weaknesses of the specific sampling frames for all existing and new surveys that will contribute data to be used in the monitoring of the leading health indicators. Small-Area Analysis Leading health indicators based on population-based survey data, health care expenditure data, or vital records will be of great value with respect to characterization of the overall health status of the nation. However, the indicators are also expected to be informative at the state, local, and community levels for diverse populations including racial and ethnic minorities, and groups categorized by gender, age, socioeconomic status, and disability status. Some of the data sets that contribute data to the DATA2000 Monitoring System may be pertinent to smaller geographic areas, such as measures of common occurrences (e.g., all causes of mortality or live births) and those that are collected at frequent intervals (national and state estimates provided by the monthly Behavioral Risk Factor Surveillance Survey). However, statistical justification will be extremely weak if efforts are made to infer to smaller localities findings representative of specific leading health indicators from national sample survey data. Assurances about the statistical stability of specific indicators also will not be possible when the number of events in a given time period and jurisdiction is small. It will be important to inform the public about the limitations of such data sets to prevent the inappropriate generalization of national findings to state, local, or even community levels. It will
CROSSCUTTING DATA ISSUES FOR LEADING HEALTH INDICATOR SETS 68 be equally important to identify alternate sources of similar information that will accurately support local level analyses and to develop new data collection efforts that will be conducted as local or community-based surveys with effective linkages to data management systems for the leading health indicators. Some mechanisms that can address these limitations and that can handle requests for data about specific leading health indicators from local jurisdictions and interest groups include the following: 1. federal support and technical assistance for efforts to conduct national surveys that are relevant to the leading health indicator sets at the state, local, and community levels and for diverse population groups; 2. federal support and technical assistance for identification of and statistical improvements to existing state, local, and community risk factor, vital records, or survey data that might be relevant to one or more indicators; 3. provision of indicator information from geographically, socially, or demographically similar population surveys; 4. development and dissemination of statistical "tool kits" that would assist state, local, and community health authorities and interested groups to extrapolate national statistical information to the demography of the local population; and 5. education in analytic techniques that would support summarization of existing state, local, and community information over longer but more statistically secure intervals, such as "rolling averages." To summarize, significant work needs to be undertaken by the U.S. Department of Health and Human Services following selection of a set of leading health indicators for Healthy People 2010. First, data sets that will be appropriate for the measurement of each of the indicators within the chosen set must be selected and evaluated on the basis of a number of dimensions including the quality of the data, limitations of self-reported data, data validity and reliability, periodicity and timeliness of data availability, the representativeness of the data, and the ability of the data to be used for small-area analyses. Of equal importance will be the determination of appropriate intervals for collection, methods of analysis, and frequency of reporting on results for each of the indicators. It would be best if the collection of data on each of the indicators were to occur on an ongoing basis rather than to be tied to surveys that obtain data at only a simple point in time. Furthermore, methods of data analysis should be defined at the outset and should be adhered to during the course of implementation of the selected leading health indicator set. Consistency of analysis will help to ensure that the same information about each indicator is available to be reported to the public. Finally, careful consideration of the appropriate time intervals of reporting on the indicators to the public will be required. Reporting at intervals that are too frequent may make it difficult for the public to perceive any significant or meaningful change in the indicators. In contrast, delayed reporting on the indicators to every 1 or 2 years may increase the likelihood that the public will lose awareness of the indicators and motivation to act on them.
