National Academies Press: OpenBook
« Previous: 2: CURRENT DATA SOURCES THAT CAN SUPPORT PPG MEASURES
Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×

3
Potential PPG Measures for 1997–2002

To facilitate the review of the hundreds of candidate performance measures discussed at the regional meetings and provided to the panel, the panel divided into working groups corresponding to the health areas within its purview. Each working group followed the same general procedure for reviewing the candidate measures (using the measure assessment guidelines described in Chapter 1):

  1. classify all of the proposed measures using the framework developed by the full panel;
  2. select measures that appear to be clear and measurable;
  3. review and select measures remaining after step 2. for adequacy of data source(s);
  4. review and select measures remaining after step 3. for validity, reliability, and responsiveness;
  5. select from the remaining outcome measures1 those that can provide valid assessment of actions that might be taken at the state level within 3–5 years;
  6. select examples of relevant process and capacity measures for each health area (see discussion below).
  7. Many of the performance measures discussed below can and should be used

    1  

    As defined in this report, health outcome measures for performance partnership grants include health status, social functioning, and consumer satisfaction. For PPG purposes, risk status measures are considered to be "intermediate outcome" measures when there is a demonstrable link between the action taken to reduce a risk (e.g., vaccinations) and the desired health status outcome.

Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×

in evaluating performance in subpopulations, such as high-risk populations. Such populations can be defined demographically, such as minorities, children, or elderly persons; by conditions, such as the lack of health insurance or homelessness; or by geographic area, such as central cities, high-risk neighborhoods, or rural communities. Specific sets of target populations can vary across states. Rather than trying to anticipate multiple submeasures that can be developed for each potential measure, the panel chose to develop broad population measures that can be tailored by each state to focus on its specific population group priorities. Clearly, validity, sample size, and other statistical issues need to be examined separately for every subpopulation.

The health outcome measures presented in this report are not meant as a mandated list. Few states are likely to have data necessary to support all of them. Furthermore, state agencies have major priorities in addition to those indicated by the outcome measures listed here (e.g., injury prevention, oral health, hearing and vision, environmental health) and are responsible for administering major programs relevant to public health (e.g., Medicaid) that are not covered by this panel's mandate. Therefore, the health outcome measures presented in this report should be considered an important subset, but not an exhaustive listing, of those that will be of interest to state agencies. Indeed, it is the panel's hope that performance measure evaluation will evolve so that new health outcome measures are continuously defined, studied, and adopted.

Similarly, the process and capacity measures presented in this report are for illustrative purposes only. Since states can pursue many reasonable strategies to improve health outcomes, a prohibitively long list of process and capacity measures would be required to cover all of their reasonable program options. The panel concluded, therefore, that the most useful approach would be to provide good, representative examples of relevant process and capacity measures in each program area. In order to illustrate the myriad strategies available to attain a single health outcome or risk status objective, Table 3-1 provides a list of possible program strategies and corresponding process measures aimed at reducing the incidence of smoking.

A major goal of this report is to provide an analytic framework for use by the states and DHHS in assessing the appropriateness of specific outcome, process, and capacity measures proposed for PPG agreements in the future. It is anticipated that many of the measures described in this report can, in time, be modified or replaced by others that meet the panel's selection guidelines.

Although the panel began its work with expectations that it would identify a set of core measures to support the PPG process in all the states, the panel has concluded that such a set of measures cannot be selected at this time. This conclusion is based on two findings. First, data sets to generate comparable state-level estimates exist for only a few health outcome measures; for the most part, data are not comparable across states. Second, as noted above, there are many reasonable process and capacity measures that states could adopt for PPG purposes

Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×

TABLE 3-1 Examples of Program Strategies and Related Process Measures for Reducing the Incidence of Tobacco Smoking

Program Strategy

Process Measure

Limit illegal youth purchases of smoking tobacco

Percentage of vendors who illegally sell smoking tobacco to minors

 

Percentage of communities with ordinances and regulations restricting smoking tobacco sales

 

Number of vending machines selling smoking tobacco in locations accessible by youth

 

Presence or absence of state or local tobacco retailer licensing system

Increase the price of tobacco products

Amount of excise tax (cents) per pack of cigarettes

Restrict smoking tobacco advertising

Percentage of communities with ordinances or regulations restricting smoking tobacco advertising

 

Number of billboards advertising smoking tobacco close to schools and playgrounds

 

Number of sport or entertainment events sponsored by tobacco companies

Restrict indoor tobacco smoking

Percentage of worksites (day cares, schools, restaurants, public places) that are smoke free (or have limited smoking to separately ventilated areas)

Educate children about hazards of smoking tobacco

Proportion of elementary, junior high, and high schools with age-appropriate smoking prevention activities and comprehensive curricula

Increase access or availability of smoking cessation programs

Proportion of current tobacco smokers visiting a health care provider during the past 12 months who received advice to quit

 

Proportion of managed care organizations (or schools or obstetric and gynecological service providers) that have active smoking prevention and cessation plans

Market effective antismoking messages to the general public

Percentage of adults who can recall seeing an antismoking message during the 12 months following a media campaign

Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×

and the selection of any subset of such measures would be arbitrary. Therefore, for the future, the panel recommends that DHHS (1) assist states in standardizing both health outcome measures (especially in the areas of substance abuse and mental health) and methods for collecting data and (2) sponsor empirical outcome studies related to state agency ''best practices" so that a more definitive list of recommended process and capacity measures can be developed.

The rest of this chapter presents and discusses potential outcome measures of performance and examples of process and capacity measures identified by the panel and others for each of the PPG subject areas; see Appendix C for detailed descriptions, rationale, and data sources.

Chronic Diseases

Prevention of chronic disease morbidity and mortality is the primary goal of many health programs, and the outcomes of these programs must be monitored. For the most part, however, chronic disease incidence and mortality data are not useful for PPG health outcome measures because the expected time period between most prevention activities and the effect of those activities on disease incidence or mortality greatly exceeds the 3–5 years of the performance grant concept. It also exceeds the time that health departments and others are generally willing to wait to assess the effectiveness of their interventions. However, the panel recommends that states continue to measure mortality from various chronic diseases (cancers, cardiovascular disease, diabetes, etc.) through the state's vital record system. The panel also recommends that states work to develop systems to better measure the incidence and prevalence of chronic diseases. With the exception of cancers in certain geographic areas, such information is generally not now available.

Since the duration of latency of most chronic diseases prevents incidence or mortality from being a useful short-term health outcome measure, potential chronic disease measures are focused on risk reduction and screening (based on the relationship of those activities to disease reduction and more effective treatment, respectively), supplemented by process measures aimed at evaluating program activities for reducing the incidence or severity of chronic diseases. The two major strategies for this approach are reduction of the major risks leading to the development of chronic diseases and improvement of the delivery of clinical preventive services for early detection of chronic diseases.

The list of major risk reduction strategies for chronic diseases is short: prevention of tobacco use, improved nutrition, increased exercise, reduction of sun exposure, reduction of alcohol and other drug use, and perhaps, avoidance of environmental carcinogens (e.g., radon). (Measures for the reduction of alcohol and other drug use are presented in a separate section of this report.) Prevention of tobacco use can be divided into reduction of personal tobacco use and reduction of exposure to second-hand smoke. Nutrition can be divided into two parts:

Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×

how much people eat (total calories) and what people eat (e.g., amounts of dietary fat, fruits, and vegetables).

The list of commonplace clinical preventive (or screening) services for chronic diseases that have been empirically shown to improve population outcomes or for which there is consensus regarding efficacy is also fairly short: screening for hypertension, cholesterol, breast cancer, cervical cancer, colon cancer, and osteoporosis.

In contrast to some other areas, there is a fair amount of standardization of existing measures and data collection methodologies across states in the area of chronic disease. As a consequence, the panel suggests a relatively precise set of measures for which data are widely available. For purposes of clarity, measures were defined according to the language used by the currently available survey questionnaires, as well as the populations surveyed by the commonly used methodologies. Given this construct, several possible measures, including ones for dietary fat content, sun exposure, and osteoporosis screening were not included at this time because of a current lack of data or methodology for collecting needed information.

The suggested measures do not include chronic disease treatment, such as for cardiovascular disease, chronic obstructive lung disease, cancer, etc. Screening for complications of diabetes was one exception: the panel included it because of the body of evidence showing the effectiveness of such screening, the existence of a large federal diabetes program, and the similarity in barriers and strategies for implementing these services and common general clinical preventive services.

Potential Risk Status Measures

Risk status measures represent intermediate health outcomes (see fn. 1).

Tobacco

 

Individual adult

Percentage of (a) persons aged 18–24 and (b) persons aged 25 and older currently smoking tobacco

Individual youth

Percentage of persons aged 14–17 (grades 9–12) currently smoking tobacco

Individual pregnant woman

Percentage of women who gave birth in the past year and reported smoking tobacco during pregnancy

Individual working adult

Percentage of employed adults whose workplace has an official policy that bans smoking

Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×

Nutrition

 

Content

Percentage of persons aged 18 and older who eat five or more servings of fruits and vegetables per day2

Content

Percentage of persons aged 14–17 (grades 9–12) who eat five or more servings of fruits and vegetables per day3

Total calories

Percentage of persons aged 18 and older who are 20 percent or more above optimal body mass index4

Exercise

 

Individual adult

Percentage of persons aged 18 and older who do not engage in physical activity or exercise

Individual youth

Percentage of persons aged 14–17 (grades 9–12) who do not engage in physical activity or exercise

Screenings and Tests

 

Hypertension

Percentage of persons aged 18 and older who had their blood pressure checked within past 2 years5

Cholesterol

Percentage of women aged 45 and older and men aged 35 and older who had their cholesterol checked within past 5 years6

Breast Cancer

Percentage of women aged 50 and older who received a mammogram within past 2 years7

Colon Cancer

Percentage of adults aged 50 and older who had a fecal occult blood test within past 12 months or a flexible sigmoidoscopy within past 5 years8

2 The numerical value in this measure is the level that is generally regarded as appropriate by the medical community; it does not represent a level that has been independently determined or endorsed by the panel.

3 See fn. 2.

4 See fn. 2.

5 See fn. 2.

6 See fn. 2.

7 See fn. 2.

8 See fn. 2.

Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×

Cervical Cancer

Percentage of women aged 18 and older who received a Pap smear within past 3 years9

Diabetes HbA1C

Percentage of persons with diabetes who had HbA1C checked within past 12 months10

Foot exam

Percentage of persons with diabetes who had a health professional examine their feet at least once within past 12 months11

Eye exam

Percentage of persons with diabetes who received a dilated eye exam within past 12 months12

Examples of Process Measures

Nutrition Program Strategy: Enable children to learn healthy dietary habits

Process Measure: Percentage of schools with menus that meet dietary guidelines for fat content and five or more servings of fruits and vegetables daily13

Physical Activity Program Strategy: Increase opportunities for sedentary working adults to exercise

Process Measure: Percentage of worksites with worksite wellness programs that include physical exercise

Smoking Program Strategy: See Table 3-1

Screening Program Strategy: Educating patients regarding need for and appropriate timing of screening tests

Process Measure: Percentage of persons with diabetes receiving diabetes health education

9 See fn. 2.

10 See fn. 2.

11 See fn. 2.

12 See fn. 2.

13 See fn. 2.

Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×

Screening Program Strategy: Improving access to screening services

Process Measure: Percentage of managed care organizations in which patients can schedule mammograms at convenient times for them

Screening Program Strategy: Implementing tracking and recall systems

Process Measure: Proportion of providers with chart-based or other real-time system for identifying women in need of mammography

Examples of Capacity Measures

Resources

Number of full-time health department employees for chronic disease prevention

Number of public service messages prepared by state agency shown annually for chronic disease prevention

Proficiencies

Number of key surveillance systems and data sets (i.e., death certificates, cancer registry data, birth certificates, Behavioral Risk Factor Surveillance System (BRFSS), Youth Risk Behavior Surveillance System (YRBSS), hospital discharge data, Medicaid and Medicare encounter information and other relevant local data sets) that are established and maintained

Percentage of local health departments receiving technical assistance and training

Percentage of labs that meet quality standards

Planning

Percentage of population served by systematic community planning process, with leadership provided by the official health agency and participation of all relevant groups (e.g., consumers, providers, advocators)

Percentage of population covered by written comprehensive chronic disease prevention plan(s) containing priorities and objectives based on needs, resources, and local demands

Community Involvement

Percentage of health care providers working under agreements established

Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×

with public health departments to provide population-based prevention programming to reduce major risk factors for premature morbidity and mortality

Proportion of health department programs that operate within the framework of a community coalition or have a community advisory group

STDS, HIV, and Tuberculosis

The long-term goal for prevention efforts directed against sexually transmitted diseases (STDs) are similar to those directed against human immunodeficiency virus (HIV) infections and tuberculosis, namely, the reduction of the suffering, complications, and loss of life that these infections cause. HIV infection, tuberculosis, and many of the STDs have a natural history that resembles noninfectious chronic diseases. For some of the STDs, and certainly for HIV infection and tuberculosis, the interval between the acquisition of infection and the development of serious consequences may be years (e.g., between cervical human papilloma virus infection and cervical cancer, between HIV infection and serious immune deficiency (AIDS), and between tuberculosis infection and cavitary lung disease). Monitoring the long-term consequences of these infections is important, but their tracking does not provide a useful short-term indication of the performance of prevention efforts.

However, not all of the manifestations of these infections are delayed in onset. Acute symptomatic diseases caused by some of the STDs, many of the bacterial forms of which are completely curable by antibiotics, occur shortly after the onset of infection, and reporting these acute syndromes can provide a valid indicator of the true incidence of new infections. For tuberculosis, a small proportion of new cases develop pulmonary disease early in the course of the infection. There does not appear to be a similarly easily identifiable acute condition that occurs early in the course of HIV infection. Also, even some of the serious complications of STDs may occur relatively soon after the onset of infection (e.g., pelvic inflammatory disease and epididymitis due to gonococcal or chlamydial infection). When HIV infection (and some STDs) occur during pregnancy, the vertical transmission of the infectious agent to the fetus or newborn may also result in serious consequences relatively early in the course of the maternal infection. Indicators that measure the prevention of this vertical transmission provide potentially valid measures of prevention efforts.

For these reasons, measuring progress in this public health area is more complex than it is for other areas (e.g., immunization). Similarly, selecting useful performance measures is difficult and complex, for several reasons related to the communicable nature and the typical courses of these diseases:

  1. The duration of the infectious state once the infection has occurred in an individual is often very long. (The typical duration is unique to each disease.)
Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×
  1. A substantial proportion of people with new STD, HIV, or tuberculosis infections remain free of symptoms for long periods of time.
  2. Even people with a newly acquired STD who do develop symptoms that prompt medical treatment will typically experience a presymptomatic interval (technically, the incubation period) during which they may be infectious to others.
  3. Effective treatment is available for many of these infections, which not only benefits the individual treated by curing the infection, but also prevents the spread of the infection to others in the population.
  4. Community spread of these infections appears to be maintained by a population of "core transmitters."
  5. The prevention value of early diagnosis and treatment of core transmitters is substantially higher than similar efforts for the general population.
  6. The predominant proportion of the spread of STDs and HIV infection in a community involves intimate sexual practices that are the object of considerable stigma in modern American society. Tuberculosis is associated with marginal and disenfranchised populations, thus bringing its own stigma. Stigma influences medical practice and reporting behaviors.
  7. A recent Institute of Medicine (1997a) report emphasizes three major strategies for preventing STDs: reducing the risk of exposure, reducing the probability that an exposed person becomes infected, and reducing the duration of the infectious state among persons who become infected. These three general strategies apply to not only STDs, but also to HIV infection and tuberculosis, although the emphasis on each approach varies by disease. The outcome indicator best suited to measuring the results of the first two strategies seeks to measure directly, or indirectly, the incidence of disease (i.e., the rate of new infections in a defined population in a defined period of time). Prevalence monitoring (i.e., the counting of existing infections in a defined population) best measures the third approach. Indicators of incidence and of prevalence are interrelated because, all other things being equal, prevalence depends on the incidence and the duration of infection.

    Potential health outcome indicators include those that attempt to measure incidence or monitor prevalence in a defined population. Indicators that attempt to measure important risk factors closely linked to disease incidence or prevalence, such as sexual behaviors, drug and alcohol use, or behaviors related to seeking medical treatment, are candidates for related outcome indicators. When these reductions can be measured in the core transmitter populations, they may be good candidates for risk status or intermediate outcome measures, as long as the data source(s) for such measures are based on sufficiently large samples to enable valid inferences to be drawn. Lastly, there are a group of indicators contingent on adequate and early treatment of cases, which can be closely linked to the prevention of further transmission, including the vertical transmission to fetuses or

Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×

newborns, either before or during childbirth. Some of these indicators may prove to be useful intermediate outcome indicators.

There are no reliable direct measures of the incidence of STDs, HIV infection, and tuberculosis in the general population. Rarely are patients or health providers able to determine the exact onset of an infection. The rate of reported cases of these infections as a part of routine communicable disease surveillance is influenced not only by the true incidence of the disease, but also by the likelihood that the infected individual seeks medical care, is tested or screened, receives the correct diagnosis, and finally, is reported in the surveillance system. Consequently, state communicable disease reporting systems, particularly when associated with laboratory reporting, can be used to monitor incidence rates for some diseases, but only with a full appreciation of the potential pitfalls of these systems. In the future, some states may be able to reliably measure the incidence of reported genital herpes. The panel has selected several illustrative examples of incidence measures that may be useful to assess how a particular state is performing in its prevention efforts for STDs and HIV infection. Unfortunately, the panel is unable to suggest any incidence measure for tuberculosis since the long latency period of the disease, combined with an absence of early or intermediate symptoms, makes any incidence measure of confirmed cases inappropriate for use in performance agreements that cover 3–5 years.

Monitoring of prevalence over an extended period of time in defined populations is a very attractive potential outcome measure. In reality, trends in empirically measured prevalence may be heavily influenced by factors other than the true prevalence—such as media campaigns aimed at encouraging groups to be tested, improved laboratory screening tests, and changes in medical practice. But, very focused prevalence monitoring may provide a useful outcome indicator for the effectiveness of prevention efforts, particularly when the monitoring occurs consistently over time at sites that serve the core populations.

One additional outcome-related indicator seems advisable for prevention programs for STDs, HIV infection, and tuberculosis because they are so inextricably linked to the quality of clinical care: measurement of client satisfaction with the services being provided. Measurement of client satisfaction is particularly relevant to core transmitters. Again, special surveys of client satisfaction will have to be conducted of these populations for states interested in using this outcome measure.

Just as reduction in the prevalence of tobacco use is a valuable risk-related outcome indicator of prevention progress against lung cancer and heart disease, changes in sexual behavior, alcohol use, and condom use—particularly among core transmitters—provide potentially valuable risk-related outcome measures for STDs and HIV infection. One key difference exists, however: for tobacco use, the value of reducing smoking is quite similar for most of the population; for STDs and HIV infection, however, the general population benefit from sexual behavior risk reduction will be much greater when it occurs in the core transmitter

Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×

population. Stated another way, considerable change in high-risk sexual behavior (or condom use) in a population at relatively low risk of infection might produce little demonstrable benefit in reducing incidence or prevalence in the population. Consequently, it is very important that measures of reduction of high risk sexual behavior, for example, be focused on the core groups. Similar arguments apply to reduction of injection drug use as an outcome indicator for HIV infection. A key challenge facing states that are interested in monitoring their progress in reducing the incidence of STDs and HIV infection among high-risk populations is that of small numbers: state population surveys such as BRFSS typically do not have sufficiently large samples for populations most at risk for STD and HIV transmission. In all likelihood, states that are interested in using such incidence measures will have to supplement or modify the sampling design of the BRFSS or conduct their own surveys.

Early, effective, and complete treatment of STDs and tuberculosis are essential hallmarks of preventing further spread in the community. As new therapies emerge for HIV, measurements of this outcome indicator may become important for HIV infection in the near future, as has already occurred for perinatal transmission of HIV. Intrapartum antiviral treatment, followed by treatment of the infant, is a risk status indicator of the prevention of HIV infection in newborns; similarly, standard treatment of pregnant women infected by syphilis prevents congenital syphilis.

In summary, there are three types of health outcome related measures available to states for performance agreements with DHHS in the areas of STDs, HIV infection, and tuberculosis: health status indicators (disease incidence and prevalence rates), consumer satisfaction, and risk status indicators (including completion of treatment).

Potential Health Status Outcome Measures

Incidence rates of selected STDs

Rate of reported gonococcal urethritis in men.

 

Rate of reported chlamydial urethritis in men14

 

Rate of reported cases of primary and secondary syphilis15

 

Rate of reported cases of congenital syphilis16

14 Can be used in states where chlamydial testing in men with urethritis is routinely performed and reported.

15 Because of the small number of reported syphilis cases, the incidence rate will be extremely unstable.

16 See fn. 15.

Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×

Incidence rates of HIV infection

Rate of reported newly diagnosed cases of HIV infection

 

Rate of perinatally acquired HIV infection of infants

Prevalence rates of selected STDs

Prevalence rate of gonococcal infection in women in defined populations

 

Prevalence rate of chlamydial infection in defined populations17

 

Prevalence rate of syphilis in defined risk groups, e.g., pregnant women18

 

Prevalence rate of rectal gonococcal infection in men

Prevalence rate of HIV infection

Seroprevalence of HIV infection in defined populations at high risk of the infection, e.g., pregnant women who abuse drugs

Potential Consumer Satisfaction Outcome Measure

Rates of consumer satisfaction with STD, HIV, and tuberculosis treatment programs

Potential Risk Status Measures

Risk status measures represent intermediate health outcomes (see fn. 1).

Rates of sexual activity among adolescents aged 14–17

Rates of sexual activity with multiple sex partners among people aged 18 and older

Rates of condom use during last episode of sexual intercourse among sexually active adolescents aged 14–17

Rates of condom use by persons aged 18 and older with multiple sex partners during last episode of sexual intercourse

Rates of condom use during last episode of sexual intercourse among men having sex with men

Rates of injection drug use among adolescents and adults

Completion rates of treatment for STDs, HIV infection, and tuberculosis

17 See fn. 14.

18 See fn. 15.

Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×

Examples of Process Measures

Program Strategy: Reduce barriers to receiving treatment from specific providers

Process Measure: Percentage of patients with insurance coverage for specific treatments

Process Measure: Percentage of patients reporting no transportation barriers to obtain necessary services

Process Measure: Percentage of physicians and other care providers receiving cultural competency training

Program Strategy: Improve quality of services provided

Process Measure: Percentage of cases followed up after most recent contact

Process Measure: Percentage of known intravenous drug users with access to needle exchange program

Examples of Capacity Measures

Resources

Percentage of high-risk communities with nearby testing and screening services from multiple types of health providers and public health organizations

Planning

Percentage of the state's population who reside in communities that are engaged in formal community processes for assessing and planning for HIV/STD/TB prevention and treatment services.

A primary challenge for the future is the development of accepted generalizable methodology to capture such information in high-risk target groups (e.g., injection drug users, men who have sex with men). Currently, these groups are poorly defined, disaffiliated, and difficult to reach, but there is broad consensus on the desired primary health outcome measures for these diseases (i.e., reductions of the incidence rates). The ability to accurately monitor these rates will require improved disease surveillance systems. Improvements will also have to be made in population survey methodologies used to assess risk behaviors. Greater standardization is also needed in order to improve comparability among states. Finally, there is a clear need for better integration and tracking of STD and HIV cases. Lack of integration affects treatment of individuals, identification of

Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×

their contacts, and the completeness of disease surveillance and tracking. It is complicated further when each disease is treated separately. Many providers are trained to screen or treat narrowly defined classes of diseases: e.g., a respiratory specialist might treat tuberculosis, but fail to diagnose STDs or HIV infection in some patients.

Mental Health

Many consumers, advocates, providers, and federal and state officials support the development of outcome measures for mental health in order to increase accountability and performance and to address the lack of public confidence in the effectiveness of public mental health services. The development of process measures in the public mental health field has been hampered by a lack of consensus on practice standards; although some states and private providers have developed standards, there is little agreement on them. Furthermore, there are limited research findings that establish a connection between individual process activities and mental health outcomes.

Although there is little agreement on linkages between specific process and mental health outcomes, there is some agreement on the dimensions that are important in evaluating mental health services, according to information provided by the Mental Health Statistics Improvement Program and the National Association of State Mental Health Program Directors:

  1. quality assurance—process activities that are thought to produce good outcomes;
  2. access to services and utilization of services;
  3. consumer satisfaction with services; and
  4. psychological and social outcomes.

Mental health PPG agreements should consider measures in all of these dimensions.

There are two distinct populations to be considered with respect to mental health outcomes: the consumers of mental health services and those in the general population who may or may not be consumers. Mental health providers have typically focused on outcomes for consumers, while mental health epidemiologists have focused on population outcomes. Because of limited resources, public mental health programs give priority to individuals with serious mental illnesses that require hospitalization or long-term outpatient treatment. However, population outcomes should be used when the measurement reflects an appropriate responsibility of the public mental health system, such as the number of homeless people who have a serious mental illness.

The measures presented below should be part of a future set of national mental health performance indicators; however, at this time, data to support these

Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×

measures are collected only in a limited number of states and, in most cases, they are not collected in a uniform fashion. Although data for any one of the measures selected by the panel may not be available for more than a few states, any of the measures could be used as part of performance agreement for a given state as long as it reflects a priority of that state. Given the lack of nationwide data for most of these measures, the inconsistencies in how states collect data on particular measures, and the variability in use of the federal mental health block grant funds, flexibility will be required in the final PPG measures negotiated with each state. As in the other areas covered in this report, the outcome and risk status measures listed below for mental health are not meant to serve as a mandatory set of measures for all states over the next 3–5 years; rather, it is expected that states will select those that reflect their program priorities and can be supported by available data resources. Over time, states should endeavor to collect this information in a standard manner in order to increase its utility and comparability. Although the measures listed below are a subset of those that might be developed in the future, they would be among the least expensive to collect and would provide some of the most useful information to evaluate the adequacy of mental health services. These measures are consistent with the values associated with recovery.

Potential Health Status Outcome Measure

Percentage of persons aged 18 and older receiving mental health services who experience reduced psychological distress

Potential Social Functioning Outcome Measures

Percentage of persons aged 18 and older receiving mental health services who experience increased level of functioning

Percentage of persons aged 18 and older receiving mental health services who report increased employment (including volunteer time)

Percentage of persons aged 18 and older with serious and persistent mental illness receiving mental health services who live in integrated, independent living situations or with family members

Percentage of children aged 17 and younger with serious emotional disorders receiving mental health services who live in noncustodial living situations

Percentage of persons aged 18 and older with serious mental illness who are in prisons and jails

Percentage of children aged 17 and younger with serious emotional disorders who are in juvenile justice facilities

Percentage of homeless persons aged 18 and older who have a serious mental illness

Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×

Potential Consumer Satisfaction Outcome Measures

Percentage of adolescents aged 14–17 or family members of children and adolescents or both who are satisfied with: (a) access to services, (b) appropriateness of services, and (c) perceptions of gain in personal outcomes

Percentage of persons (aged 18 and older) or their family members or both who are satisfied with: (a) access to mental health services, (b) appropriateness of services, and (c) perceptions of gain in personal outcomes

The measures involving homelessness and living in jail or juvenile justice facilities cannot be affected, in the short run, solely by the actions taken by the state agency for mental health services. Nevertheless, the panel believes that these population-based mental health outcome measures—when combined with related process and capacity measures that are under the direct control of these agencies—can provide useful insights regarding state progress in meeting important mental health goals. Over the long run (5–10 years), state agencies responsible for mental health services should be able to demonstrate their impact on improving these outcomes.

Examples of Process and Capacity Measures

Program Strategy: Improve access to services/utilization of services

Process Measure: Percentage of adults with serious and persistent mental illness who use health services

Process Measure: Percentage of youth with serious emotional disorders who use mental health services

Process Measure: Percentage of those who use services that are voluntary

Process Measure: Percentage of people requesting services who begin receiving services within 2 weeks of the initial request

Capacity Measure: Percentage of primary care providers who receive supplemental training in mental health services

Program Strategy: Improve quality assurance

Process Measure: Percentage of service plans that include input from consumers and family members

Process Measure: Percentage of children less than 5 years old who are screened and assessed for mental health intervention

Capacity Measure: Percentage of primary care providers who use standardized screening tools for assessing the mental health status of primary care clients

Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×

The development of performance indicators for mental health should be part of a national effort to develop a standardized framework for evaluating mental health services. As that framework is developed and the needed research is completed, PPG indicators for mental health can be refined. In order for such refinement to occur, public and private administrators of mental health services will need to agree on specific mental health outcome measures and the mechanisms for capturing information on those outcomes across states. Additional research will also be needed to identify practice standards that result in the desired outcomes.

Immunization

The biology and epidemiology of vaccine preventable diseases are well understood. Cases are, in general, easily identified, and health outcomes are well defined. A strong, causal relationship exists between immunization and disease prevention, and national standards and guidelines are well established and widely accepted. From a PPG perspective, clearly defined process (vaccination rate) and outcome (disease incidence rate) measures are the preferred means to assess vaccine preventable disease performance, and they are responsive to effective interventions within the 3–5 years proposed by DHHS. These measures provide policy-relevant information on population health and provide public health systems with the data required to assess program effectiveness and resource efficiency.

Many factors outside public health and health care delivery systems also have an impact on vaccination levels and disease incidence rates, such as immigration patterns and economic fluctuations. States also require measures to assess program effectiveness in order to facilitate adoption of effective and cost-effective interventions and to identify and terminate ineffective or inefficient programmatic initiatives. Thus, capacity and program variables also may be valuable adjuncts of immunization measures.

Although vaccination rates are risk reduction measures, the validity of their relationship to target health status outcomes is direct, proven, and universally accepted as a valid and reliable intermediate outcome measure. Indeed, given the episodic and unpredictable nature of outbreaks of vaccine-preventable diseases, a strong argument can be made that vaccination rates are a better measure of program performance than are direct measures of disease incidence. In addition to the fact that disease outbreaks are sporadic and somewhat random, the infrequency of cases is such that it may not be possible to develop statistically reliable state and metropolitan area estimates. Thus, while incident cases are relatively specific (their occurrence generally indicates a problem), sensitivity is relatively low (low vaccination rates may occur without a corresponding increase in incident cases).

Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×

Potential Health Status Outcome Measures

Reported incidence rate of representative vaccine-preventable diseases

Potential Risk Status Measures

Risk status measures represent intermediate health outcomes (see fn. 1).

Age-appropriate vaccination rates for target age groups for each major vaccine group:

children aged 2 years; children entering school at approximately 5 years of age

mumps, measles, and rubella

diphtheria-tetanus-pertussis

polio

hemophilus influenza B

hepatitis B

varicella

adults aged 65 years and older

diphtheria-tetanus

hepatitis B

influenza

pneumococcal pneumonia

Examples of Process Measures

Program Strategy: improve access to immunization services

Process Measure: Percentage of population who do not cite financial resources as a barrier to immunization

Program Strategy: Increase parent education and awareness

Process Measure: Percentage of parents with children under 18 who believe that the benefits of immunization outweigh the risks

Process Measure: Percentage of parents with children under 18 who report receiving immunization reminders from their immunization providers

For immunization, program capacity is, in effect, one of the core public health functions. A state's ability to monitor vaccine compliance, facilitate vaccine

Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×

access, and respond to disease outbreaks depends on both its ability to coordinate data collection with private practitioners and health delivery systems and federal data collection efforts. There are currently at least ten different sources of immunization data collected by various federal agencies. Unfortunately, many of these sources suffer from various methodological limitations that compromise their value and appropriateness for the purpose of state performance measurement (e.g., inadequate sampling frames and sample sizes that preclude reliable state-level estimates for all but the largest states and metropolitan areas; unclear data validity and reliability). For infant and childhood vaccination rate estimates, the newly developed National Immunization Survey (NIS) conducted by the National Program Immunication Office in conjunction with the National Center for Health Statistics appears to meet many of the needs of the PPG program.

Other potential sources of data include statewide registries, day care and Head Start programs, school reports, the Medicare statistical system, and for health maintenance organization's (HMOs). However, these sources are not yet developed or fully implemented and standardized across states; they require validity and reliability verification and possibly new data collection and reporting structures; and they are subject to selection bias. There are no current valid and reliable data collected on adolescent vaccination. Similarly, vaccination data from BRFSS on high-risk nonelderly adults are limited to influenza and pneumococcal pneumonia. Because most candidates for influenza and pneumonia vaccines are over 65 years old, the Medicare Statistical System can provide some estimate of vaccination, although the Medicare data on immunizations administered by hospitals and HMOs are not universally available.19 In addition, the quality of Medicare data for residents of skilled nursing homes is not clear.

State BRFSS data on adult immunizations may not be adjusted for risk, (e.g., age and presence of respiratory conditions), limiting the value of the data for benchmarking across states. The most efficient and cost-effective way to increase the available data on vaccine incidence may be to modify and expand the NIS, taking advantage of the large sampling frame to collect data on adolescents and adults. This use may require a somewhat expanded sampling frame to ensure adequate statistical power and will require the development of additional survey modules. Finally, whenever possible, coordination of data collection effort among other performance measurement efforts is highly desirable to maximize efficiency and minimize data burden and cost.

Many of the suggestions to the panel focused on programmatic process

19  

The current draft of Health Employer Data and Information Set (HEDIS 3.0) requires HMOs seeking accreditation from the National Committee for Quality Assurance to report on influenza immunizations for their Medicare members and for high-risk adults under age 65.

Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×

measures. Such measures may be appropriate to include as a portion of state performance measures under limited and specific conditions: e.g., to assess progress toward development and implementation of DHHS-state agreements on programmatic initiatives, such as vaccination surveillance or administration programs for women, infants, and children; statewide registry development and implementation; insurance coverage policies; and educational programs. However, the specification of such measures must await DHHS-state agreements and should be restricted to carefully specified circumstances until relevant outcome measures are available.

Substance Abuse

The substance abuse measures suggested to the panel had a number of common themes and fell under several distinct categories: treatment effectiveness; treatment completion; medical screening; use during pregnancy; HIV/STDs; overall use and consequences; youth use and consequences; other prevention activities; access and special needs; and general and infrastructure issues. A list of 120 suggested measures was distilled from these categories, from which the panel identified 11 health outcome (including risk reduction) measures that best met the selection criteria presented in Chapter 2, at least for some states.20

Although data for any one of the measures may not be available for more than a few states, such measures could be used as part of performance agreements for any state as long as the particular measure reflects a priority of that state and the specific data sources, populations, definitions, time frames, etc., are based on the data available in that state. Given the lack of nationwide data for most of these measures, as well as the variability in how states use their federal substance abuse block grant funds, some flexibility in the final PPG measures negotiated with each state will be needed. As in the other areas discussed in this report, the panel expects that states will select from among the suggested measures listed below, to the extent that the measures can be supported by their data resources and reflect program priorities. States also should be encouraged to propose other measures that meet the panel's guidelines.

It should be noted that several of the measures listed in other sections (e.g., chronic disease; prevention of disabilities; STDs, HIV infection, and tuberculosis) may also be appropriate measures in the substance abuse area. For example, the following measures could be negotiated as part of an individual state's PPG agreement if relevant to their substance abuse efforts:

20  

Tobacco is of increasing concern to substance abuse agencies; specific measures involving tobacco are discussed in the separate sections on chronic diseases and disabilities.

Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×

for chronic disease:

smoking during pregnancy

for STDs, HIV infection, and tuberculosis:

injection drug use during pregnancy

for disabilities:

alcohol, tobacco, or other drug use during pregnancy

States are encouraged to coordinate the measures they select in the various areas, as well as their data collection activities to measure them.

Many of the health status and risk reduction measures listed below are not affected, in the short run, solely by the actions taken by a state agency for alcohol and drug abuse. Nevertheless, these substance abuse outcome and risk reduction measures—when combined with related process and capacity measures that are under the direct control of the agencies—can provide useful insights regarding progress in reducing problems caused by alcohol and drug abuse. Over the long run (5–10 years), state agencies responsible for alcohol and other drug abuse should be able to demonstrate their agency's impact on reducing such abuse and on the resulting problems caused by these substances.

Potential Health Status Outcome Measures

Death rate of persons aged 15–65 attributed to (a) alcohol, (b) other drug use, and (c) combined agents

Percentage of emergency room encounters for alcohol or other drug-related causes21

Potential Social Functioning Outcome Measures

Prevalence rate of substance abuse clients who report experiencing diminished severity of problems after completing treatment as measured by the Addiction Severity Index (ASI) or a similar measure

Ratio of substance abuse clients involved with the criminal justice system before and after completing treatment

Potential Risk Status Measures

Risk status measures represent intermediate health outcomes (see fn. 1).

21  

Statewide estimates are not available from the available data system that supports this measure (DAWN); however, this measure should be included among those states that may choose from if they want to focus their efforts on a defined geographical area(s) as part of their performance agreements with DHHS.

Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×

Prevalence rate of adolescents aged 14–17 engaged in heavy drinking or other drug use22

Prevalence rate of persons aged 18 and older engaged in heavy drinking or other drug use23

Percentage of women who gave birth in the past year and reported using alcohol or other drugs during pregnancy

Mean age at first use of ''gateway" drugs (tobacco, marijuana, alcohol)

Percentage of adolescents aged 14–17 stating disapproval of marijuana use

Percentage of adolescents aged 14–17 who report parents or guardians who communicate non-use expectations

Percentage of drug abuse clients who engage in risk behaviors related to HIV/AIDS after completing treatment plan

Examples of Process Measures

Process Measure: Percentage of pregnant women screened for substance abuse

Process Measure: Percentage of drug abuse clients screened for STDs, HIV infection, and tuberculosis during treatment

Examples of Capacity Measures
Resources

Percentage of at-risk population(s) who have access to and receive specialized services

Planning

The presence or absence of statewide prevention and treatment needs assessment study completed within last 2 years

Percentage of providers that use uniform criteria to assess and match clients to appropriate services

There will need to be continued support for data definition and collection efforts at both the state and national level in order for states to report on the

22  

Although the estimated incidence rate would be a more appropriate measure of state agency performance, the most suitable data source for this measure is the YRBSS, which is a population survey and, therefore, only provides estimates of prevalence.

23  

Although the estimated incidence rate would be a more appropriate measure of state agency performance, the most suitable data source for this measure is the BRFSS, which is a population survey.

Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×

proposed substance abuse measures. In particular, support for state needs assessment studies that can support valid and reliable incidence measures of alcohol and other drug use, improved student surveys, expanded behavioral risk surveys, uniform client data sets, emergency room reporting, and client follow-up data will be critical to the ability of states to report in this important area.

Some of the measures recommended in the substance abuse area are based on self-report data, either by clients during and after treatment or by a sample of the total population. Although questions are often raised about the validity and reliability of self-report data in this area, studies that have used collateral sources to verify client self-reports have found a high degree of consistency between the clients' statements and statements from significant others (Hoffman and Harrison, 1991). One extensive review of a variety of research on the validity of substance abuse clients' self-reports (Sobell and Sobell, 1986) found that as long as clients' confidentiality was ensured and questions were objective and clear, client self-reports are sufficiently valid and reliable to be used in outcome studies.

An area of particular concern to the panel is what may happen to the ability of states to report on these measures as they move toward greater use of managed care and as previously separate funding sources are merged. As an example, many managed care firms rely heavily on consumer satisfaction surveys to measure quality of care. Such surveys are not meaningful for most substance abuse clients, most of whom dislike treatment even if their problems are reduced. The effects of major changes, such as the move to managed care and the merging of various funding streams, on the quality, cost, and effectiveness of client services will be impossible to measure if adequate attention and resources are not devoted to preserving or building data systems before such changes are implemented. It is important to include people at the state and local levels who are most knowledgeable about substance abuse in those decisions.

Sexual Assault, Disabilities, and Emergency Medical Services

The panel was charged by DHHS to propose candidate performance measures in three specific areas of prevention: sexual assault, disabilities, and emergency medical services. The panel has addressed this charge but recognizes that there are many other areas of prevention of concern to public health agencies such as injury prevention. The performance measures presented here may serve as useful models for measures that could be developed for other important areas of public health.

Sexual Assault

Candidate measures suggested to the panel focused on a broad range of issues related to sexual assault, from prevention to the provision of services to

Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×

those who become victims of these violent acts. A basic problem with developing meaningful indicators for sexual assault prevention programs is the difficult nature of data collection and the high degree of underreporting of assault and abuse. Another fundamental problem in measuring sexual assault is that it is currently viewed by many as a criminal justice issue rather than a public health issue. The panel identified only one measure that could be used with currently available data.

Potential Health Status Outcome Measure

Incidence rate of sexual assault reported by females

This outcome measure is very unlikely to be affected solely by actions taken by the state health agency. Nevertheless, this measure—when combined with related process and capacity measures that are under the direct control of state health agencies—can provide useful insights regarding progress in reducing the incidence of this behavior. Over the long run (5–10 years), effective programs of prevention should be able to accomplish a measurable reduction in the rate of sexual assault.

Examples of Process Measures

Process Measure: Percentage of sexual assault victims receiving acute medical and psychosocial services from specially trained personnel

Process Measure: Percentage of victims receiving postvictimization services

Examples of Capacity Measures

Resources

Percentage of at-risk population(s) who have access to and receive specialized services

Percentage of counties with rape crisis centers offering hot-line and other services

Proficiencies

Number of counties with a sexual assault surveillance system

Percentage of medical and criminal justice professionals involved with sexual assault cases who have had specialized training in these fields

Percentage of elementary and secondary schools providing educational instruction on the problem of sexual assault

Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×

Percentage of perpetrators of sexual assault who receive professional counseling designed to prevent reoccurrence

Disabilities

Performance indicators suggested to the panel reflected the focus of existing CDC programs—measures targeted toward preventing those disabilities that have their onset during childhood. The suggested indicators did not address disabilities that occur as a result of occupation or chronic illness; mental retardation resulting from congenital conditions was also not discussed. The panel believes that these are critical omissions from disability prevention and will address them during the second phase of its study.

The performance measures in this report focus on preventing initial impairment of function and preventing secondary disability due to complications from lack of or inadequate rehabilitation. Many disabilities are secondary to central nervous system illness and injury and although the panel does not offer specific measures, it believes such risk status indicators as rates of helmet use by operators of motorcycles, motorbikes, and bicycles could be useful to states in monitoring their progress toward meeting important health outcome goals. Seat belt use would also be an appropriate risk status indicator. However, data systems to support these measures are not available in most states.

Potential Health Status Outcome Measure

Percentage of newborns with neural tube defects

Potential Social Functioning Outcome Measure

Percentage of persons aged 18–65 with disabilities who are in the workforce

Potential Risk Status Measures

Risk status measures represent intermediate health outcomes (see fn. 1).

Percentage of children aged 6 or younger with blood lead greater than 10 micrograms per deciliter24

Percentage of women who gave birth in the past year and reported using alcohol, tobacco, or other drugs during pregnancy

The panel is aware that none of these measures is likely to be affected solely

24  

See fn. 2.

Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×

by actions taken by a state health agency. Nevertheless, the panel believes that these outcome measures—when combined with related process and capacity measures that are under the direct control of state health agencies—can provide useful insights regarding progress in, for example, increasing the rate of disability of persons in the workforce. The panel also believes that over the long run (5–10 years) state health agencies should be able to demonstrate their impact on increasing the percentage of people with disabilities who are working.

Examples of Process Measures

Program Strategy: Reduce the incidence of neural tube defects

Process Measure: Percentage of (high-risk) women screened for maternal serum alpha feto protein (MSAFP)

Process Measure: Percentage of high-risk women taking periconceptual folic acid supplementation

Program Strategy: Reduce the incidence of elevated blood lead

Process Measure: Number of counties with housing regulations designed to reduce lead hazards in low-income housing rentals

Process Measure: Percentage of parents living in homes built before 1950 who can cite lead from paint as a potential health hazard to their children

Examples of Capacity Measures

Resources

Percentage of disabled population(s) who have access to and receive specialized services

Proficiencies

Number of counties offering parents early childhood education programs focused on disabilities prevention

Planning

Number of counties actively monitoring the incidence of disabilities and the impact of disabilities

State and federal programs do not have a consistent definition of disability. This lack confounds efforts to accurately identify and measure indicators related to prevention and service delivery. The panel notes with interest the efforts by

Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×

the Social Security Administration through its Medical Evaluation Study to better quantify the number of people with disabilities in this country. This study will allow the development of better indicators to measure efforts to prevent and mitigate disabilities.

Emergency Medical Services

Because the panel's charge by DHHS in this area was to focus on the emergency medical services system, indicators that relate solely to medical care in the emergency departments of hospitals were not considered unless they directly affected the quality of care in the prehospital setting or the transport and bypass decisions.25 Among the measures suggested to the panel, the range of possible indicators covered aspects of emergency services from the initial call to treatment in specialized centers; the panel found, however, that data to support these measures were not available in most states. The panel selected measures that would be available in all jurisdictions and related to nationally accepted indicators of good emergency medical system performance. However, federal funding represents only a small portion of the funding for the emergency medical systems that exist and the cost of expanding them.

Potential Health Status Outcome Measure

Percentage of persons who suffer out-of-hospital cardiac arrest who survive

Examples of Process Measures

Process Measure: Percentage of trauma patients meeting state, or regional, triage guidelines that are transported to a trauma or burn center designated by the state or regional authority or meeting other nationally recognized criteria

Process Measure: Average time from initial call to arrival of the patient at the destination hospital

Process Measure: Percentage of emergency medical service systems with medical direction

Process Measure: Percentage of inappropriate calls to 911 or the emergency medical services system

Process Measure: Percentage of patients who receive appropriate early defibrillation

Process Measure: Percentage of the population served by poison control centers

25  

Indicators related to prevention of injuries were also not included as part of the charge to the panel.

Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×
Examples of Capacity Measures

Resources

Percentage of the state population with access to a trauma care system that includes:

legal authority to designate trauma centers

authority to establish triage procedures that allow prehospital personnel to bypass nearer facilities

trauma center identification and designation

field categorization and triage protocols

interhospital transfer agreements

linkage to the rehabilitation system

system evaluation activities

Number of counties with 911 or enhanced 911 systems

Number of counties with injury prevention programs

Proficiencies

Number of counties with 911 systems that have personnel who are able to communicate with users in their language and in a culturally competent way

Planning

Number of counties that maintain databases of prehospital care reports

Number of counties that support a statewide trauma registry

In the second phase of the panel's work, development of valid outcome measures for monitoring EMS system performance will be addressed. The panel recognizes that better systems of collecting prehospital care data with linkage to posthospital outcomes will be necessary. These measures will also need to be sensitive to the diversity of coverage areas from urban to rural, and from basic to advanced levels of care.

Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×
Page 18
Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×
Page 19
Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×
Page 20
Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×
Page 21
Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×
Page 22
Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×
Page 23
Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×
Page 24
Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×
Page 25
Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×
Page 26
Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×
Page 27
Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×
Page 28
Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×
Page 29
Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×
Page 30
Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×
Page 31
Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×
Page 32
Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×
Page 33
Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×
Page 34
Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×
Page 35
Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×
Page 36
Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×
Page 37
Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×
Page 38
Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×
Page 39
Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×
Page 40
Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×
Page 41
Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×
Page 42
Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×
Page 43
Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×
Page 44
Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×
Page 45
Suggested Citation:"3: POTENTIAL PPG MEASURES FOR 1997-2002." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×
Page 46
Next: 4: IMPLEMENTING PERFORMANCE-BASED AGREEMENTS »
Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health Get This Book
×
Buy Paperback | $44.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The U.S. Department of Health and Human Services is carrying out an ambitious new program to assure that funds for public health programs are spent as effectively as possible. Under the new program, every state will develop a set of performance objectives to measure its progress in terms of outcomes, processes, and capacity. In the first phase of the program, states are to propose such measures to be achieved over three to five years.

This book examines the technical issues involved in the development of performance measures in 10 areas: mental health, substance abuse, HIV infection, sexually transmitted diseases, tuberculosis, immunization, chronic diseases, disability, rape, and emergency medical services. From more than 3,200 candidate measures proposed by researchers, policymakers, and public health professionals, the panel proposes more than 50 potential outcome measures. The book details the advantages and limitations of potential measures as well as the data sources that can support them. This volume will be an invaluable resource to everyone involved in public health.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!