3
Potential PPG Measures for 1997–2002

To facilitate the review of the hundreds of candidate performance measures discussed at the regional meetings and provided to the panel, the panel divided into working groups corresponding to the health areas within its purview. Each working group followed the same general procedure for reviewing the candidate measures (using the measure assessment guidelines described in Chapter 1):

  1. classify all of the proposed measures using the framework developed by the full panel;
  2. select measures that appear to be clear and measurable;
  3. review and select measures remaining after step 2. for adequacy of data source(s);
  4. review and select measures remaining after step 3. for validity, reliability, and responsiveness;
  5. select from the remaining outcome measures1 those that can provide valid assessment of actions that might be taken at the state level within 3–5 years;
  6. select examples of relevant process and capacity measures for each health area (see discussion below).
  7. Many of the performance measures discussed below can and should be used

    1  

    As defined in this report, health outcome measures for performance partnership grants include health status, social functioning, and consumer satisfaction. For PPG purposes, risk status measures are considered to be "intermediate outcome" measures when there is a demonstrable link between the action taken to reduce a risk (e.g., vaccinations) and the desired health status outcome.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 18
3 Potential PPG Measures for 1997–2002 To facilitate the review of the hundreds of candidate performance measures discussed at the regional meetings and provided to the panel, the panel divided into working groups corresponding to the health areas within its purview. Each working group followed the same general procedure for reviewing the candidate measures (using the measure assessment guidelines described in Chapter 1): classify all of the proposed measures using the framework developed by the full panel; select measures that appear to be clear and measurable; review and select measures remaining after step 2. for adequacy of data source(s); review and select measures remaining after step 3. for validity, reliability, and responsiveness; select from the remaining outcome measures1 those that can provide valid assessment of actions that might be taken at the state level within 3–5 years; select examples of relevant process and capacity measures for each health area (see discussion below). Many of the performance measures discussed below can and should be used 1   As defined in this report, health outcome measures for performance partnership grants include health status, social functioning, and consumer satisfaction. For PPG purposes, risk status measures are considered to be "intermediate outcome" measures when there is a demonstrable link between the action taken to reduce a risk (e.g., vaccinations) and the desired health status outcome.

OCR for page 18
in evaluating performance in subpopulations, such as high-risk populations. Such populations can be defined demographically, such as minorities, children, or elderly persons; by conditions, such as the lack of health insurance or homelessness; or by geographic area, such as central cities, high-risk neighborhoods, or rural communities. Specific sets of target populations can vary across states. Rather than trying to anticipate multiple submeasures that can be developed for each potential measure, the panel chose to develop broad population measures that can be tailored by each state to focus on its specific population group priorities. Clearly, validity, sample size, and other statistical issues need to be examined separately for every subpopulation. The health outcome measures presented in this report are not meant as a mandated list. Few states are likely to have data necessary to support all of them. Furthermore, state agencies have major priorities in addition to those indicated by the outcome measures listed here (e.g., injury prevention, oral health, hearing and vision, environmental health) and are responsible for administering major programs relevant to public health (e.g., Medicaid) that are not covered by this panel's mandate. Therefore, the health outcome measures presented in this report should be considered an important subset, but not an exhaustive listing, of those that will be of interest to state agencies. Indeed, it is the panel's hope that performance measure evaluation will evolve so that new health outcome measures are continuously defined, studied, and adopted. Similarly, the process and capacity measures presented in this report are for illustrative purposes only. Since states can pursue many reasonable strategies to improve health outcomes, a prohibitively long list of process and capacity measures would be required to cover all of their reasonable program options. The panel concluded, therefore, that the most useful approach would be to provide good, representative examples of relevant process and capacity measures in each program area. In order to illustrate the myriad strategies available to attain a single health outcome or risk status objective, Table 3-1 provides a list of possible program strategies and corresponding process measures aimed at reducing the incidence of smoking. A major goal of this report is to provide an analytic framework for use by the states and DHHS in assessing the appropriateness of specific outcome, process, and capacity measures proposed for PPG agreements in the future. It is anticipated that many of the measures described in this report can, in time, be modified or replaced by others that meet the panel's selection guidelines. Although the panel began its work with expectations that it would identify a set of core measures to support the PPG process in all the states, the panel has concluded that such a set of measures cannot be selected at this time. This conclusion is based on two findings. First, data sets to generate comparable state-level estimates exist for only a few health outcome measures; for the most part, data are not comparable across states. Second, as noted above, there are many reasonable process and capacity measures that states could adopt for PPG purposes

OCR for page 18
TABLE 3-1 Examples of Program Strategies and Related Process Measures for Reducing the Incidence of Tobacco Smoking Program Strategy Process Measure Limit illegal youth purchases of smoking tobacco Percentage of vendors who illegally sell smoking tobacco to minors   Percentage of communities with ordinances and regulations restricting smoking tobacco sales   Number of vending machines selling smoking tobacco in locations accessible by youth   Presence or absence of state or local tobacco retailer licensing system Increase the price of tobacco products Amount of excise tax (cents) per pack of cigarettes Restrict smoking tobacco advertising Percentage of communities with ordinances or regulations restricting smoking tobacco advertising   Number of billboards advertising smoking tobacco close to schools and playgrounds   Number of sport or entertainment events sponsored by tobacco companies Restrict indoor tobacco smoking Percentage of worksites (day cares, schools, restaurants, public places) that are smoke free (or have limited smoking to separately ventilated areas) Educate children about hazards of smoking tobacco Proportion of elementary, junior high, and high schools with age-appropriate smoking prevention activities and comprehensive curricula Increase access or availability of smoking cessation programs Proportion of current tobacco smokers visiting a health care provider during the past 12 months who received advice to quit   Proportion of managed care organizations (or schools or obstetric and gynecological service providers) that have active smoking prevention and cessation plans Market effective antismoking messages to the general public Percentage of adults who can recall seeing an antismoking message during the 12 months following a media campaign

OCR for page 18
and the selection of any subset of such measures would be arbitrary. Therefore, for the future, the panel recommends that DHHS (1) assist states in standardizing both health outcome measures (especially in the areas of substance abuse and mental health) and methods for collecting data and (2) sponsor empirical outcome studies related to state agency ''best practices" so that a more definitive list of recommended process and capacity measures can be developed. The rest of this chapter presents and discusses potential outcome measures of performance and examples of process and capacity measures identified by the panel and others for each of the PPG subject areas; see Appendix C for detailed descriptions, rationale, and data sources. Chronic Diseases Prevention of chronic disease morbidity and mortality is the primary goal of many health programs, and the outcomes of these programs must be monitored. For the most part, however, chronic disease incidence and mortality data are not useful for PPG health outcome measures because the expected time period between most prevention activities and the effect of those activities on disease incidence or mortality greatly exceeds the 3–5 years of the performance grant concept. It also exceeds the time that health departments and others are generally willing to wait to assess the effectiveness of their interventions. However, the panel recommends that states continue to measure mortality from various chronic diseases (cancers, cardiovascular disease, diabetes, etc.) through the state's vital record system. The panel also recommends that states work to develop systems to better measure the incidence and prevalence of chronic diseases. With the exception of cancers in certain geographic areas, such information is generally not now available. Since the duration of latency of most chronic diseases prevents incidence or mortality from being a useful short-term health outcome measure, potential chronic disease measures are focused on risk reduction and screening (based on the relationship of those activities to disease reduction and more effective treatment, respectively), supplemented by process measures aimed at evaluating program activities for reducing the incidence or severity of chronic diseases. The two major strategies for this approach are reduction of the major risks leading to the development of chronic diseases and improvement of the delivery of clinical preventive services for early detection of chronic diseases. The list of major risk reduction strategies for chronic diseases is short: prevention of tobacco use, improved nutrition, increased exercise, reduction of sun exposure, reduction of alcohol and other drug use, and perhaps, avoidance of environmental carcinogens (e.g., radon). (Measures for the reduction of alcohol and other drug use are presented in a separate section of this report.) Prevention of tobacco use can be divided into reduction of personal tobacco use and reduction of exposure to second-hand smoke. Nutrition can be divided into two parts:

OCR for page 18
how much people eat (total calories) and what people eat (e.g., amounts of dietary fat, fruits, and vegetables). The list of commonplace clinical preventive (or screening) services for chronic diseases that have been empirically shown to improve population outcomes or for which there is consensus regarding efficacy is also fairly short: screening for hypertension, cholesterol, breast cancer, cervical cancer, colon cancer, and osteoporosis. In contrast to some other areas, there is a fair amount of standardization of existing measures and data collection methodologies across states in the area of chronic disease. As a consequence, the panel suggests a relatively precise set of measures for which data are widely available. For purposes of clarity, measures were defined according to the language used by the currently available survey questionnaires, as well as the populations surveyed by the commonly used methodologies. Given this construct, several possible measures, including ones for dietary fat content, sun exposure, and osteoporosis screening were not included at this time because of a current lack of data or methodology for collecting needed information. The suggested measures do not include chronic disease treatment, such as for cardiovascular disease, chronic obstructive lung disease, cancer, etc. Screening for complications of diabetes was one exception: the panel included it because of the body of evidence showing the effectiveness of such screening, the existence of a large federal diabetes program, and the similarity in barriers and strategies for implementing these services and common general clinical preventive services. Potential Risk Status Measures Risk status measures represent intermediate health outcomes (see fn. 1). Tobacco   Individual adult Percentage of (a) persons aged 18–24 and (b) persons aged 25 and older currently smoking tobacco Individual youth Percentage of persons aged 14–17 (grades 9–12) currently smoking tobacco Individual pregnant woman Percentage of women who gave birth in the past year and reported smoking tobacco during pregnancy Individual working adult Percentage of employed adults whose workplace has an official policy that bans smoking

OCR for page 18
Nutrition   Content Percentage of persons aged 18 and older who eat five or more servings of fruits and vegetables per day2 Content Percentage of persons aged 14–17 (grades 9–12) who eat five or more servings of fruits and vegetables per day3 Total calories Percentage of persons aged 18 and older who are 20 percent or more above optimal body mass index4 Exercise   Individual adult Percentage of persons aged 18 and older who do not engage in physical activity or exercise Individual youth Percentage of persons aged 14–17 (grades 9–12) who do not engage in physical activity or exercise Screenings and Tests   Hypertension Percentage of persons aged 18 and older who had their blood pressure checked within past 2 years5 Cholesterol Percentage of women aged 45 and older and men aged 35 and older who had their cholesterol checked within past 5 years6 Breast Cancer Percentage of women aged 50 and older who received a mammogram within past 2 years7 Colon Cancer Percentage of adults aged 50 and older who had a fecal occult blood test within past 12 months or a flexible sigmoidoscopy within past 5 years8 2 The numerical value in this measure is the level that is generally regarded as appropriate by the medical community; it does not represent a level that has been independently determined or endorsed by the panel. 3 See fn. 2. 4 See fn. 2. 5 See fn. 2. 6 See fn. 2. 7 See fn. 2. 8 See fn. 2.

OCR for page 18
Cervical Cancer Percentage of women aged 18 and older who received a Pap smear within past 3 years9 Diabetes HbA1C Percentage of persons with diabetes who had HbA1C checked within past 12 months10 Foot exam Percentage of persons with diabetes who had a health professional examine their feet at least once within past 12 months11 Eye exam Percentage of persons with diabetes who received a dilated eye exam within past 12 months12 Examples of Process Measures Nutrition Program Strategy: Enable children to learn healthy dietary habits Process Measure: Percentage of schools with menus that meet dietary guidelines for fat content and five or more servings of fruits and vegetables daily13 Physical Activity Program Strategy: Increase opportunities for sedentary working adults to exercise Process Measure: Percentage of worksites with worksite wellness programs that include physical exercise Smoking Program Strategy: See Table 3-1 Screening Program Strategy: Educating patients regarding need for and appropriate timing of screening tests Process Measure: Percentage of persons with diabetes receiving diabetes health education 9 See fn. 2. 10 See fn. 2. 11 See fn. 2. 12 See fn. 2. 13 See fn. 2.

OCR for page 18
Screening Program Strategy: Improving access to screening services Process Measure: Percentage of managed care organizations in which patients can schedule mammograms at convenient times for them Screening Program Strategy: Implementing tracking and recall systems Process Measure: Proportion of providers with chart-based or other real-time system for identifying women in need of mammography Examples of Capacity Measures Resources Number of full-time health department employees for chronic disease prevention Number of public service messages prepared by state agency shown annually for chronic disease prevention Proficiencies Number of key surveillance systems and data sets (i.e., death certificates, cancer registry data, birth certificates, Behavioral Risk Factor Surveillance System (BRFSS), Youth Risk Behavior Surveillance System (YRBSS), hospital discharge data, Medicaid and Medicare encounter information and other relevant local data sets) that are established and maintained Percentage of local health departments receiving technical assistance and training Percentage of labs that meet quality standards Planning Percentage of population served by systematic community planning process, with leadership provided by the official health agency and participation of all relevant groups (e.g., consumers, providers, advocators) Percentage of population covered by written comprehensive chronic disease prevention plan(s) containing priorities and objectives based on needs, resources, and local demands Community Involvement Percentage of health care providers working under agreements established

OCR for page 18
with public health departments to provide population-based prevention programming to reduce major risk factors for premature morbidity and mortality Proportion of health department programs that operate within the framework of a community coalition or have a community advisory group STDS, HIV, and Tuberculosis The long-term goal for prevention efforts directed against sexually transmitted diseases (STDs) are similar to those directed against human immunodeficiency virus (HIV) infections and tuberculosis, namely, the reduction of the suffering, complications, and loss of life that these infections cause. HIV infection, tuberculosis, and many of the STDs have a natural history that resembles noninfectious chronic diseases. For some of the STDs, and certainly for HIV infection and tuberculosis, the interval between the acquisition of infection and the development of serious consequences may be years (e.g., between cervical human papilloma virus infection and cervical cancer, between HIV infection and serious immune deficiency (AIDS), and between tuberculosis infection and cavitary lung disease). Monitoring the long-term consequences of these infections is important, but their tracking does not provide a useful short-term indication of the performance of prevention efforts. However, not all of the manifestations of these infections are delayed in onset. Acute symptomatic diseases caused by some of the STDs, many of the bacterial forms of which are completely curable by antibiotics, occur shortly after the onset of infection, and reporting these acute syndromes can provide a valid indicator of the true incidence of new infections. For tuberculosis, a small proportion of new cases develop pulmonary disease early in the course of the infection. There does not appear to be a similarly easily identifiable acute condition that occurs early in the course of HIV infection. Also, even some of the serious complications of STDs may occur relatively soon after the onset of infection (e.g., pelvic inflammatory disease and epididymitis due to gonococcal or chlamydial infection). When HIV infection (and some STDs) occur during pregnancy, the vertical transmission of the infectious agent to the fetus or newborn may also result in serious consequences relatively early in the course of the maternal infection. Indicators that measure the prevention of this vertical transmission provide potentially valid measures of prevention efforts. For these reasons, measuring progress in this public health area is more complex than it is for other areas (e.g., immunization). Similarly, selecting useful performance measures is difficult and complex, for several reasons related to the communicable nature and the typical courses of these diseases: The duration of the infectious state once the infection has occurred in an individual is often very long. (The typical duration is unique to each disease.)

OCR for page 18
A substantial proportion of people with new STD, HIV, or tuberculosis infections remain free of symptoms for long periods of time. Even people with a newly acquired STD who do develop symptoms that prompt medical treatment will typically experience a presymptomatic interval (technically, the incubation period) during which they may be infectious to others. Effective treatment is available for many of these infections, which not only benefits the individual treated by curing the infection, but also prevents the spread of the infection to others in the population. Community spread of these infections appears to be maintained by a population of "core transmitters." The prevention value of early diagnosis and treatment of core transmitters is substantially higher than similar efforts for the general population. The predominant proportion of the spread of STDs and HIV infection in a community involves intimate sexual practices that are the object of considerable stigma in modern American society. Tuberculosis is associated with marginal and disenfranchised populations, thus bringing its own stigma. Stigma influences medical practice and reporting behaviors. A recent Institute of Medicine (1997a) report emphasizes three major strategies for preventing STDs: reducing the risk of exposure, reducing the probability that an exposed person becomes infected, and reducing the duration of the infectious state among persons who become infected. These three general strategies apply to not only STDs, but also to HIV infection and tuberculosis, although the emphasis on each approach varies by disease. The outcome indicator best suited to measuring the results of the first two strategies seeks to measure directly, or indirectly, the incidence of disease (i.e., the rate of new infections in a defined population in a defined period of time). Prevalence monitoring (i.e., the counting of existing infections in a defined population) best measures the third approach. Indicators of incidence and of prevalence are interrelated because, all other things being equal, prevalence depends on the incidence and the duration of infection. Potential health outcome indicators include those that attempt to measure incidence or monitor prevalence in a defined population. Indicators that attempt to measure important risk factors closely linked to disease incidence or prevalence, such as sexual behaviors, drug and alcohol use, or behaviors related to seeking medical treatment, are candidates for related outcome indicators. When these reductions can be measured in the core transmitter populations, they may be good candidates for risk status or intermediate outcome measures, as long as the data source(s) for such measures are based on sufficiently large samples to enable valid inferences to be drawn. Lastly, there are a group of indicators contingent on adequate and early treatment of cases, which can be closely linked to the prevention of further transmission, including the vertical transmission to fetuses or

OCR for page 18
newborns, either before or during childbirth. Some of these indicators may prove to be useful intermediate outcome indicators. There are no reliable direct measures of the incidence of STDs, HIV infection, and tuberculosis in the general population. Rarely are patients or health providers able to determine the exact onset of an infection. The rate of reported cases of these infections as a part of routine communicable disease surveillance is influenced not only by the true incidence of the disease, but also by the likelihood that the infected individual seeks medical care, is tested or screened, receives the correct diagnosis, and finally, is reported in the surveillance system. Consequently, state communicable disease reporting systems, particularly when associated with laboratory reporting, can be used to monitor incidence rates for some diseases, but only with a full appreciation of the potential pitfalls of these systems. In the future, some states may be able to reliably measure the incidence of reported genital herpes. The panel has selected several illustrative examples of incidence measures that may be useful to assess how a particular state is performing in its prevention efforts for STDs and HIV infection. Unfortunately, the panel is unable to suggest any incidence measure for tuberculosis since the long latency period of the disease, combined with an absence of early or intermediate symptoms, makes any incidence measure of confirmed cases inappropriate for use in performance agreements that cover 3–5 years. Monitoring of prevalence over an extended period of time in defined populations is a very attractive potential outcome measure. In reality, trends in empirically measured prevalence may be heavily influenced by factors other than the true prevalence—such as media campaigns aimed at encouraging groups to be tested, improved laboratory screening tests, and changes in medical practice. But, very focused prevalence monitoring may provide a useful outcome indicator for the effectiveness of prevention efforts, particularly when the monitoring occurs consistently over time at sites that serve the core populations. One additional outcome-related indicator seems advisable for prevention programs for STDs, HIV infection, and tuberculosis because they are so inextricably linked to the quality of clinical care: measurement of client satisfaction with the services being provided. Measurement of client satisfaction is particularly relevant to core transmitters. Again, special surveys of client satisfaction will have to be conducted of these populations for states interested in using this outcome measure. Just as reduction in the prevalence of tobacco use is a valuable risk-related outcome indicator of prevention progress against lung cancer and heart disease, changes in sexual behavior, alcohol use, and condom use—particularly among core transmitters—provide potentially valuable risk-related outcome measures for STDs and HIV infection. One key difference exists, however: for tobacco use, the value of reducing smoking is quite similar for most of the population; for STDs and HIV infection, however, the general population benefit from sexual behavior risk reduction will be much greater when it occurs in the core transmitter

OCR for page 18
Potential Health Status Outcome Measures Reported incidence rate of representative vaccine-preventable diseases Potential Risk Status Measures Risk status measures represent intermediate health outcomes (see fn. 1). Age-appropriate vaccination rates for target age groups for each major vaccine group: children aged 2 years; children entering school at approximately 5 years of age mumps, measles, and rubella diphtheria-tetanus-pertussis polio hemophilus influenza B hepatitis B varicella adults aged 65 years and older diphtheria-tetanus hepatitis B influenza pneumococcal pneumonia Examples of Process Measures Program Strategy: improve access to immunization services Process Measure: Percentage of population who do not cite financial resources as a barrier to immunization Program Strategy: Increase parent education and awareness Process Measure: Percentage of parents with children under 18 who believe that the benefits of immunization outweigh the risks Process Measure: Percentage of parents with children under 18 who report receiving immunization reminders from their immunization providers For immunization, program capacity is, in effect, one of the core public health functions. A state's ability to monitor vaccine compliance, facilitate vaccine

OCR for page 18
access, and respond to disease outbreaks depends on both its ability to coordinate data collection with private practitioners and health delivery systems and federal data collection efforts. There are currently at least ten different sources of immunization data collected by various federal agencies. Unfortunately, many of these sources suffer from various methodological limitations that compromise their value and appropriateness for the purpose of state performance measurement (e.g., inadequate sampling frames and sample sizes that preclude reliable state-level estimates for all but the largest states and metropolitan areas; unclear data validity and reliability). For infant and childhood vaccination rate estimates, the newly developed National Immunization Survey (NIS) conducted by the National Program Immunication Office in conjunction with the National Center for Health Statistics appears to meet many of the needs of the PPG program. Other potential sources of data include statewide registries, day care and Head Start programs, school reports, the Medicare statistical system, and for health maintenance organization's (HMOs). However, these sources are not yet developed or fully implemented and standardized across states; they require validity and reliability verification and possibly new data collection and reporting structures; and they are subject to selection bias. There are no current valid and reliable data collected on adolescent vaccination. Similarly, vaccination data from BRFSS on high-risk nonelderly adults are limited to influenza and pneumococcal pneumonia. Because most candidates for influenza and pneumonia vaccines are over 65 years old, the Medicare Statistical System can provide some estimate of vaccination, although the Medicare data on immunizations administered by hospitals and HMOs are not universally available.19 In addition, the quality of Medicare data for residents of skilled nursing homes is not clear. State BRFSS data on adult immunizations may not be adjusted for risk, (e.g., age and presence of respiratory conditions), limiting the value of the data for benchmarking across states. The most efficient and cost-effective way to increase the available data on vaccine incidence may be to modify and expand the NIS, taking advantage of the large sampling frame to collect data on adolescents and adults. This use may require a somewhat expanded sampling frame to ensure adequate statistical power and will require the development of additional survey modules. Finally, whenever possible, coordination of data collection effort among other performance measurement efforts is highly desirable to maximize efficiency and minimize data burden and cost. Many of the suggestions to the panel focused on programmatic process 19   The current draft of Health Employer Data and Information Set (HEDIS 3.0) requires HMOs seeking accreditation from the National Committee for Quality Assurance to report on influenza immunizations for their Medicare members and for high-risk adults under age 65.

OCR for page 18
measures. Such measures may be appropriate to include as a portion of state performance measures under limited and specific conditions: e.g., to assess progress toward development and implementation of DHHS-state agreements on programmatic initiatives, such as vaccination surveillance or administration programs for women, infants, and children; statewide registry development and implementation; insurance coverage policies; and educational programs. However, the specification of such measures must await DHHS-state agreements and should be restricted to carefully specified circumstances until relevant outcome measures are available. Substance Abuse The substance abuse measures suggested to the panel had a number of common themes and fell under several distinct categories: treatment effectiveness; treatment completion; medical screening; use during pregnancy; HIV/STDs; overall use and consequences; youth use and consequences; other prevention activities; access and special needs; and general and infrastructure issues. A list of 120 suggested measures was distilled from these categories, from which the panel identified 11 health outcome (including risk reduction) measures that best met the selection criteria presented in Chapter 2, at least for some states.20 Although data for any one of the measures may not be available for more than a few states, such measures could be used as part of performance agreements for any state as long as the particular measure reflects a priority of that state and the specific data sources, populations, definitions, time frames, etc., are based on the data available in that state. Given the lack of nationwide data for most of these measures, as well as the variability in how states use their federal substance abuse block grant funds, some flexibility in the final PPG measures negotiated with each state will be needed. As in the other areas discussed in this report, the panel expects that states will select from among the suggested measures listed below, to the extent that the measures can be supported by their data resources and reflect program priorities. States also should be encouraged to propose other measures that meet the panel's guidelines. It should be noted that several of the measures listed in other sections (e.g., chronic disease; prevention of disabilities; STDs, HIV infection, and tuberculosis) may also be appropriate measures in the substance abuse area. For example, the following measures could be negotiated as part of an individual state's PPG agreement if relevant to their substance abuse efforts: 20   Tobacco is of increasing concern to substance abuse agencies; specific measures involving tobacco are discussed in the separate sections on chronic diseases and disabilities.

OCR for page 18
for chronic disease: smoking during pregnancy for STDs, HIV infection, and tuberculosis: injection drug use during pregnancy for disabilities: alcohol, tobacco, or other drug use during pregnancy States are encouraged to coordinate the measures they select in the various areas, as well as their data collection activities to measure them. Many of the health status and risk reduction measures listed below are not affected, in the short run, solely by the actions taken by a state agency for alcohol and drug abuse. Nevertheless, these substance abuse outcome and risk reduction measures—when combined with related process and capacity measures that are under the direct control of the agencies—can provide useful insights regarding progress in reducing problems caused by alcohol and drug abuse. Over the long run (5–10 years), state agencies responsible for alcohol and other drug abuse should be able to demonstrate their agency's impact on reducing such abuse and on the resulting problems caused by these substances. Potential Health Status Outcome Measures Death rate of persons aged 15–65 attributed to (a) alcohol, (b) other drug use, and (c) combined agents Percentage of emergency room encounters for alcohol or other drug-related causes21 Potential Social Functioning Outcome Measures Prevalence rate of substance abuse clients who report experiencing diminished severity of problems after completing treatment as measured by the Addiction Severity Index (ASI) or a similar measure Ratio of substance abuse clients involved with the criminal justice system before and after completing treatment Potential Risk Status Measures Risk status measures represent intermediate health outcomes (see fn. 1). 21   Statewide estimates are not available from the available data system that supports this measure (DAWN); however, this measure should be included among those states that may choose from if they want to focus their efforts on a defined geographical area(s) as part of their performance agreements with DHHS.

OCR for page 18
Prevalence rate of adolescents aged 14–17 engaged in heavy drinking or other drug use22 Prevalence rate of persons aged 18 and older engaged in heavy drinking or other drug use23 Percentage of women who gave birth in the past year and reported using alcohol or other drugs during pregnancy Mean age at first use of ''gateway" drugs (tobacco, marijuana, alcohol) Percentage of adolescents aged 14–17 stating disapproval of marijuana use Percentage of adolescents aged 14–17 who report parents or guardians who communicate non-use expectations Percentage of drug abuse clients who engage in risk behaviors related to HIV/AIDS after completing treatment plan Examples of Process Measures Process Measure: Percentage of pregnant women screened for substance abuse Process Measure: Percentage of drug abuse clients screened for STDs, HIV infection, and tuberculosis during treatment Examples of Capacity Measures Resources Percentage of at-risk population(s) who have access to and receive specialized services Planning The presence or absence of statewide prevention and treatment needs assessment study completed within last 2 years Percentage of providers that use uniform criteria to assess and match clients to appropriate services There will need to be continued support for data definition and collection efforts at both the state and national level in order for states to report on the 22   Although the estimated incidence rate would be a more appropriate measure of state agency performance, the most suitable data source for this measure is the YRBSS, which is a population survey and, therefore, only provides estimates of prevalence. 23   Although the estimated incidence rate would be a more appropriate measure of state agency performance, the most suitable data source for this measure is the BRFSS, which is a population survey.

OCR for page 18
proposed substance abuse measures. In particular, support for state needs assessment studies that can support valid and reliable incidence measures of alcohol and other drug use, improved student surveys, expanded behavioral risk surveys, uniform client data sets, emergency room reporting, and client follow-up data will be critical to the ability of states to report in this important area. Some of the measures recommended in the substance abuse area are based on self-report data, either by clients during and after treatment or by a sample of the total population. Although questions are often raised about the validity and reliability of self-report data in this area, studies that have used collateral sources to verify client self-reports have found a high degree of consistency between the clients' statements and statements from significant others (Hoffman and Harrison, 1991). One extensive review of a variety of research on the validity of substance abuse clients' self-reports (Sobell and Sobell, 1986) found that as long as clients' confidentiality was ensured and questions were objective and clear, client self-reports are sufficiently valid and reliable to be used in outcome studies. An area of particular concern to the panel is what may happen to the ability of states to report on these measures as they move toward greater use of managed care and as previously separate funding sources are merged. As an example, many managed care firms rely heavily on consumer satisfaction surveys to measure quality of care. Such surveys are not meaningful for most substance abuse clients, most of whom dislike treatment even if their problems are reduced. The effects of major changes, such as the move to managed care and the merging of various funding streams, on the quality, cost, and effectiveness of client services will be impossible to measure if adequate attention and resources are not devoted to preserving or building data systems before such changes are implemented. It is important to include people at the state and local levels who are most knowledgeable about substance abuse in those decisions. Sexual Assault, Disabilities, and Emergency Medical Services The panel was charged by DHHS to propose candidate performance measures in three specific areas of prevention: sexual assault, disabilities, and emergency medical services. The panel has addressed this charge but recognizes that there are many other areas of prevention of concern to public health agencies such as injury prevention. The performance measures presented here may serve as useful models for measures that could be developed for other important areas of public health. Sexual Assault Candidate measures suggested to the panel focused on a broad range of issues related to sexual assault, from prevention to the provision of services to

OCR for page 18
those who become victims of these violent acts. A basic problem with developing meaningful indicators for sexual assault prevention programs is the difficult nature of data collection and the high degree of underreporting of assault and abuse. Another fundamental problem in measuring sexual assault is that it is currently viewed by many as a criminal justice issue rather than a public health issue. The panel identified only one measure that could be used with currently available data. Potential Health Status Outcome Measure Incidence rate of sexual assault reported by females This outcome measure is very unlikely to be affected solely by actions taken by the state health agency. Nevertheless, this measure—when combined with related process and capacity measures that are under the direct control of state health agencies—can provide useful insights regarding progress in reducing the incidence of this behavior. Over the long run (5–10 years), effective programs of prevention should be able to accomplish a measurable reduction in the rate of sexual assault. Examples of Process Measures Process Measure: Percentage of sexual assault victims receiving acute medical and psychosocial services from specially trained personnel Process Measure: Percentage of victims receiving postvictimization services Examples of Capacity Measures Resources Percentage of at-risk population(s) who have access to and receive specialized services Percentage of counties with rape crisis centers offering hot-line and other services Proficiencies Number of counties with a sexual assault surveillance system Percentage of medical and criminal justice professionals involved with sexual assault cases who have had specialized training in these fields Percentage of elementary and secondary schools providing educational instruction on the problem of sexual assault

OCR for page 18
Percentage of perpetrators of sexual assault who receive professional counseling designed to prevent reoccurrence Disabilities Performance indicators suggested to the panel reflected the focus of existing CDC programs—measures targeted toward preventing those disabilities that have their onset during childhood. The suggested indicators did not address disabilities that occur as a result of occupation or chronic illness; mental retardation resulting from congenital conditions was also not discussed. The panel believes that these are critical omissions from disability prevention and will address them during the second phase of its study. The performance measures in this report focus on preventing initial impairment of function and preventing secondary disability due to complications from lack of or inadequate rehabilitation. Many disabilities are secondary to central nervous system illness and injury and although the panel does not offer specific measures, it believes such risk status indicators as rates of helmet use by operators of motorcycles, motorbikes, and bicycles could be useful to states in monitoring their progress toward meeting important health outcome goals. Seat belt use would also be an appropriate risk status indicator. However, data systems to support these measures are not available in most states. Potential Health Status Outcome Measure Percentage of newborns with neural tube defects Potential Social Functioning Outcome Measure Percentage of persons aged 18–65 with disabilities who are in the workforce Potential Risk Status Measures Risk status measures represent intermediate health outcomes (see fn. 1). Percentage of children aged 6 or younger with blood lead greater than 10 micrograms per deciliter24 Percentage of women who gave birth in the past year and reported using alcohol, tobacco, or other drugs during pregnancy The panel is aware that none of these measures is likely to be affected solely 24   See fn. 2.

OCR for page 18
by actions taken by a state health agency. Nevertheless, the panel believes that these outcome measures—when combined with related process and capacity measures that are under the direct control of state health agencies—can provide useful insights regarding progress in, for example, increasing the rate of disability of persons in the workforce. The panel also believes that over the long run (5–10 years) state health agencies should be able to demonstrate their impact on increasing the percentage of people with disabilities who are working. Examples of Process Measures Program Strategy: Reduce the incidence of neural tube defects Process Measure: Percentage of (high-risk) women screened for maternal serum alpha feto protein (MSAFP) Process Measure: Percentage of high-risk women taking periconceptual folic acid supplementation Program Strategy: Reduce the incidence of elevated blood lead Process Measure: Number of counties with housing regulations designed to reduce lead hazards in low-income housing rentals Process Measure: Percentage of parents living in homes built before 1950 who can cite lead from paint as a potential health hazard to their children Examples of Capacity Measures Resources Percentage of disabled population(s) who have access to and receive specialized services Proficiencies Number of counties offering parents early childhood education programs focused on disabilities prevention Planning Number of counties actively monitoring the incidence of disabilities and the impact of disabilities State and federal programs do not have a consistent definition of disability. This lack confounds efforts to accurately identify and measure indicators related to prevention and service delivery. The panel notes with interest the efforts by

OCR for page 18
the Social Security Administration through its Medical Evaluation Study to better quantify the number of people with disabilities in this country. This study will allow the development of better indicators to measure efforts to prevent and mitigate disabilities. Emergency Medical Services Because the panel's charge by DHHS in this area was to focus on the emergency medical services system, indicators that relate solely to medical care in the emergency departments of hospitals were not considered unless they directly affected the quality of care in the prehospital setting or the transport and bypass decisions.25 Among the measures suggested to the panel, the range of possible indicators covered aspects of emergency services from the initial call to treatment in specialized centers; the panel found, however, that data to support these measures were not available in most states. The panel selected measures that would be available in all jurisdictions and related to nationally accepted indicators of good emergency medical system performance. However, federal funding represents only a small portion of the funding for the emergency medical systems that exist and the cost of expanding them. Potential Health Status Outcome Measure Percentage of persons who suffer out-of-hospital cardiac arrest who survive Examples of Process Measures Process Measure: Percentage of trauma patients meeting state, or regional, triage guidelines that are transported to a trauma or burn center designated by the state or regional authority or meeting other nationally recognized criteria Process Measure: Average time from initial call to arrival of the patient at the destination hospital Process Measure: Percentage of emergency medical service systems with medical direction Process Measure: Percentage of inappropriate calls to 911 or the emergency medical services system Process Measure: Percentage of patients who receive appropriate early defibrillation Process Measure: Percentage of the population served by poison control centers 25   Indicators related to prevention of injuries were also not included as part of the charge to the panel.

OCR for page 18
Examples of Capacity Measures Resources Percentage of the state population with access to a trauma care system that includes: legal authority to designate trauma centers authority to establish triage procedures that allow prehospital personnel to bypass nearer facilities trauma center identification and designation field categorization and triage protocols interhospital transfer agreements linkage to the rehabilitation system system evaluation activities Number of counties with 911 or enhanced 911 systems Number of counties with injury prevention programs Proficiencies Number of counties with 911 systems that have personnel who are able to communicate with users in their language and in a culturally competent way Planning Number of counties that maintain databases of prehospital care reports Number of counties that support a statewide trauma registry In the second phase of the panel's work, development of valid outcome measures for monitoring EMS system performance will be addressed. The panel recognizes that better systems of collecting prehospital care data with linkage to posthospital outcomes will be necessary. These measures will also need to be sensitive to the diversity of coverage areas from urban to rural, and from basic to advanced levels of care.