Using Performance Monitoring to Improve Community Health: Conceptual Framework and Community Experience

Using Performance Monitoring to Improve
Community Health: Conceptual Framework and
Community Experience

Workshop Summary

Committee on Using Performance Monitoring
to Improve Community Health

Ellen M. Weissman, Editor

Board on Health Promotion and Disease Prevention

INSTITUTE OF MEDICINE

NATIONAL ACADEMY PRESS
Washington, D.C. 1996

NATIONAL ACADEMY PRESS • 2101 Constitution Avenue, N.W. • Washington, D.C. 20418

NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance.

This report has been reviewed by a group other than the authors according to procedures approved by a Report Review Committee consisting of members of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine.

The Institute of Medicine was chartered in 1970 by the National Academy of Sciences to enlist distinguished members of the appropriate professions in the examination of policy matters pertaining to the health of the public. In this, the Institute acts under the Academy's 1863 congressional charter responsibility to be an adviser to the federal government and its own initiative in identifying issues of medical care, research, and education. Dr. Kenneth I. Shine is president of the Institute of Medicine.

Funding for this project was provided by the Office of the Assistant Secretary for Health, U.S. Department of Health and Human Services (contract no. 282-94-0032); The Robert Wood Johnson Foundation (grant no.024336); and the National Academy of Sciences. The views presented in this report are those of the Committee on Using Performance Monitoring to Improve Community Health and of the participants in the workshop and are not necessarily those of the funding organizations.

International Standard Book Number: 0-309-05594-6

Additional copies of this report are available for sale from the National Academy Press,2101 Constitution Avenue, N.W., Box 285, Washington, DC 20055. Call (800) 624-6242 or (202) 334-3313 (in the Washington metropolitan area), or visit the NAP on-line bookstore at http://www.nap.edu/bookstore.

Printed in the United States of America

COMMITTEE ON USING PERFORMANCE MONITORING
TO IMPROVE COMMUNITY HEALTH

BOBBIE A. BERKOWITZ* (Cochair), Deputy Secretary, Washington State Department of Health, Olympia, Washington
THOMAS S. INUI^† (Cochair), Professor and Chaff., Department of Ambulatory Care and Prevention, Harvard Medical School and Harvard Pilgrim Health Care, Boston, Massachusetts
ALAN W. CROSS (Vice Chair), Director, Center for Health Promotion and Disease Prevention, University of North Carolina, Chapel Hill, North Carolina
LARRY W. CHAMBERS, Epidemiology Consultant, Hamilton-Wentworth Regional Public Health Department, and Professor, Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ontario, Canada
THOMAS W. CHAPMAN,^‡ Chief Executive Officer, George Washington University Hospital, and Senior Vice President for Network Development, George Washington University Medical Center, Washington, D.C.
ELLIOTT S. FISHER, Associate Professor of Medicine and Community and Family Medicine, Dartmouth Medical School, Hanover, New Hampshire
JAMES L. GALE, Professor, Department of Epidemiology, School of Public Health, and Director, Northwest Center for Public Health Practice, University of Washington, Seattle; Health Officer, Kittitas County, Washington
KRISTINE GEBBIE^† (Liaison, Board on Health Promotion and Disease Prevention), Assistant Professor of Nursing, Columbia University School of Nursing, New York, New York
FERNANDO A. GUERRA, Director of Health, San Antonio Metropolitan Health District, San Antonio, Texas
GARLAND H. LAND, Director, Center for Health Information Management and Epidemiology, Missouri Department of Health, Jefferson City, Missouri
SHEILA LEATHERMAN, Executive Vice President, United HealthCare Corporation, Minneapolis, Minnesota
JOHN R. LUMPKIN, Director, Illinois Department of Public Health, Springfield, Illinois
WILLIAM J. MAYER, Vice President, Medical Affairs, Kellogg Company, Battle Creek, Michigan
ANA MARIA OSORIO, Chief, Occupational Health Branch, California Department of Health Services, Berkeley, California
SHOSHANNA SOFAER, Associate Professor and Associate Chair for Research, Department of Health Care Sciences, George Washington University Medical Center, Washington, D.C.
DEBORAH KLEIN WALKER, Assistant Commissioner, Bureau of Family and Community Health, Massachusetts Department of Public Health, Boston, Massachusetts
JOHN E. WARE, Jr.,^‡ Senior Scientist, The Health Institute, New England Medical Center, Boston, Massachusetts
RICHARD A. WRIGHT, Director, Community Health Services, Denver Department of Health and Hospitals, Denver, Colorado

Study Staff

Linda A. Bailey, Senior Program Officer (Co-Study Director)
Jane S. Durch, Program Officer (Co-Study Director)
Stephanie Y. Smith, Project Assistant
Michael A. Stoto, Director, Division of Health Promotion and Disease Prevention
Marissa Fuller, Research Associate
Sarah H. Reich, Project Assistant
Susan Thaul, Senior Program Officer

Workshop Consultant

Ellen M. Weissman, Johns Hopkins School of Hygiene and Public Health

*As of July 1, 1996, Deputy Director, Turning Point Program, University of Washington School of Public Health and Community Medicine, Seattle.
^†Member, Institute of Medicine.
^‡Served through December 1995.

Contents

INTRODUCTION

BACKGROUND

DETERMINANTS OF HEALTH

INTERVENTIONS TO IMPROVE HEALTH

COMMUNITY EXPERIENCE WITH PERFORMANCE MONITORING

DISCUSSION

SOCIAL CHANGE AND ACCOUNTABILITY

COMMENTARY

CONCLUDING OBSERVATIONS

REFERENCES

APPENDIXES
A Workshop Agenda
B Workshop Speakers and Guests

Using Performance Monitoring to Improve
Community Health: Conceptual Framework and
Community Experience

INTRODUCTION

In recent years, performance monitoring has gained increasing attention as a tool for evaluating the delivery of personal health care services and for examining population-based activities addressing the health of the public. This attention to performance monitoring is related to several factors, including concerns about ensuring the efficient and effective use of health care dollars in providing high-quality care and achieving the best possible health outcomes. Also contributing are a wider recognition that the health of the population depends on many factors beyond medical care and heightened concern about accountability for use of resources and whether desired results have been achieved.

An interest in understanding how monitoring the activities performed by health care and public health agencies and organizations might contribute to improving the health of entire communities is the basis of a study by the Institute of Medicine (IOM) Committee on Using Performance Monitoring to Improve Community Health. The study is being funded by the U.S. Department of Health and Human Services and The Robert Wood Johnson Foundation.

The Committee's Charge

The committee was asked to examine how performance monitoring can be used to promote improvements in community health. In particular, the committee was asked to consider the roles that public health and personal health care systems and other stakeholders play in influencing community-wide health, how their performance in connection with health improvement goals can be monitored, and how a performance monitoring system can be used to foster collaboration among these sectors and promote improvements in community health.

The committee brought together expertise in state and local health departments, epidemiology, public health indicators, health data, environmental health, adult and pediatric clinical medicine, managed care, community health and consumer interests, quality assessment, health services research, and employer concerns. The group met six times between February 1995 and April 1996. Workshops held in conjunction with two of these meetings gave the committee an opportunity to learn more about conceptual and applied work relevant to performance monitoring and to hear about a variety of community experiences.

The Workshop

The committee's second workshop, held on December 11, 1995, is summarized here. The purpose of this workshop was to discuss both conceptual models underlying performance monitoring and its use in specific communities. Workshop presentations on conceptual models addressed the determinants of health, social change, and accountability. Presentations and a panel discussion gave five professionals working in communities an opportunity to bring to the committee comments and observations based on practical experience in health improvement programs and performance monitoring.

This summary of the workshop presentations and discussions is based on notes from the presentations, a transcript of the taped proceedings, and comments from the speakers. It does not present opinions, conclusions, or recommendations of the committee. Conclusions and recommendations, which will reflect consideration of the workshop discussions, will be presented in the committee's final report.

BACKGROUND¹

As used by the committee, the term performance monitoring refers to a continuing community-based process of selecting indicators that can be used to measure the process and outcomes of an intervention strategy for health improvement, collecting and analyzing data on those indicators, and making the results available, to the community as a whole and specifically to those segments of the community engaged in health improvement activities, to inform assessments of the effectiveness of an intervention and the contributions of accountable entities. Performance monitoring should promote health improvement in a context of shared responsibility and accountability for achieving desired outcomes. Many parties within a community share responsibility for health (e.g., consumers, health care providers, businesses, government agencies, public service groups); those with responsibility for accomplishing specific tasks are accountable to the community for their performance.²

Several assumptions underlie the committee's approach to performance monitoring. First, it is increasingly necessary to use resources efficiently, that is, to accomplish tasks with a minimum of waste. Performance monitoring is expected to facilitate efficient approaches to improving the health of communities at a population level. Second, a performance monitoring system should have a broad enough perspective to monitor diverse factors that influence a community's health, including ones that may not appear obviously health related. Third, a wide range of actors share a stake in community health; thus, social action and changes that involve many sectors of the community are necessary. Finally, special attention to vulnerable populations is important, because equity is valued in community health.

The application of performance monitoring presents problems at the current level of knowledge and infrastructure. Although tools for indicator development exist, the conditions for creating operational monitoring systems at the community level may not. Furthermore, measurement strategies for an array of health issues are not universally available, and measures that are available may not always be applicable at the community level. In addition, most information systems are not yet able to support the identification and analysis of health problems and to track interventions.

A central task for the committee is the development of indicators suitable for community-based performance monitoring.³ Ideally, performance indicators measure states or critical processes that are potentially alterable and thought to have a demonstrable relationship to health outcomes. Those indicators may be measures of capacity, resources, processes, or actual health outcomes. Committee discussions suggest that indicator selection should be based on a community process that includes identification of stakeholders, adoption of a shared conceptual framework to analyze the community's health, selection of indicators appropriate to fundamental concerns, operational development of indicators, and field testing of indicators. Indicators should be descriptive; reliable, valid, and sensitive to changes in the community's health; important to stakeholders sensitive to declines in the health of vulnerable subpopulations; and useful in monitoring health initiatives.

DETERMINANTS OF HEALTH⁴

Health encompasses physical and psychosocial well-being, not simply the absence of disease. Because many factors influence health and well-being, understanding the nature and scope of these determining factors is an essential element in developing health improvement strategies and in determining what indicators may be appropriate elements of a performance monitoring effort. The workshop's opening presentation used the framework of the Health Field Model for examining the determinants of health.

Health Field Model

A model of the determinants of health proposed by Evans and Stoddart—the health field model—provides a broad conceptual framework for considering the factors that influence health in a community (Evans and Stoddart, 1990). Unlike a biomedical model that views health as the absence of disease, the field model includes functional capacity and well-being as health outcomes of interest (see Figure 1). The model also emphasizes general factors that affect many diseases or the health of large segments of the population, rather than specific factors that account for small changes in health at the individual level. It takes a multidisciplinary approach, uniting biomedical sciences, public health, psychology, statistics and epidemiology, economics, sociology, education, and other disciplines. Social, environmental, economic, and genetic factors are seen as contributing to differences in health status and, therefore, as presenting opportunities to intervene.

Although this type of model is not an entirely new paradigm, its implications for designing health improvement programs deserve attention. For example, the way in which (health) behavior is understood fundamentally changes. Rather than a voluntary act amenable to direct intervention, behavior can be seen as an intermediate factor that is itself shaped by multiple forces, particularly the social and physical environments and genetic endowment. At the same time, behavior remains a relevant target for intervention. The model also differentiates among disease, health and function, and well-being. They are affected by separate but overlapping factors, and therefore, indicators selected to monitor health improvement programs may need to differ depending on which outcome is of primary interest. The model also reinforces the interrelatedness of many factors. Outcomes are the product of complex interactions of factors rather than of individual factors operating in isolation. It was suggested that the interactions among factors may prove to be more important that the actions of any single factor.

Each of the factors included in the model is considered briefly in turn.

FIGURE 1 A model of the determinants of health. Reprinted from R.G. Evans and G.L. Stoddart, "Producing Health, Consuming Health Care," Social Science and Medicine," 31:1359, with permision from Elsevier Science Ltd, Kidlington, UK.

Social Environment

Among the elements of the social environment that have been linked to health are family structure, the educational system, social networks, social class, work setting, and level of prosperity.

Family structure, for example, is known to affect children's physical and mental health. On average, children in single-parent families do not do as well on measures of development, performance, and mental health as children in two-parent families. Children's relationships with their parents, social support, nurturance, and sense of self-efficacy have been shown to be related to their mental and physical health and even to their future economic productivity (Schor and Menaghan, 1995).

Education has an effect on health status separate from its influence on income. Years of formal education are strongly related to age-adjusted mortality in countries as disparate as Hungary, Norway, and England and Wales (Valkonen, 1989). Although most research is based on years of formal schooling, evidence suggests a broader relationship that includes the preschool period. An assessment at age 19 of participants in the Perry Preschool Study, which randomized children into a Head Start-like program, showed that participation in the preschool program was correlated with better school performance, attending college, and avoiding involvement with the criminal justice system (Weikart, 1989). Critical periods for education, particularly at young ages, may prove to be important in determining health. In addition, studies show that maternal educational attainment is a key determinant of child welfare and survival (Zill and Brim, 1983).

"Social networks" is a term that refers to an individual's integration into a self-defined community and the degree of connectedness to other individuals and to institutions. There is a strong inverse correlation between the number and frequency of close contacts and mortality from all causes, with odds ratios of 2:1 or higher and a clear "dose-response" relationship (Berkman and Syme, 1979). Although it is possible to see the impact of social networks on health, the pathways responsible for those effects are not yet known.

Social class is another well-described determinant of health, independent of income. Major studies have been done in Britain, where social class is defined more explicitly than in the United States. In the Whitehall study of British civil servants, Marmot et al. (1987) demonstrated a clear relationship between social class (based on job classification) and mortality. The relationship persists throughout the social hierarchy and is unchanged after adjusting for income and smoking. The effect of social class may raise uncomfortable issues in the United States but is important to consider in dealing with issues of health and equity.

The health effects of work-related factors are seen in studies of job decision latitude, autonomy, and cardiovascular mortality (Karasek and Theorell, 1990). Involuntary unemployment negatively affects both mental and physical health. Economic prosperity is also correlated with better health. Throughout history, the poor have, on average, died at younger ages than the rich. The relationship between prosperity and health holds across the economic spectrum. For every decile, quintile, or quartile of income, from lowest to highest, there is a decline in overall age-adjusted mortality. In international comparisons by the Organization for Economic Cooperation and Development, the difference in income between the highest and lowest deciles of income shows a stronger relationship with overall mortality rates than does median income (Wilkinson, 1992, 1994).

Genetic Endowment

Genetic factors are recognized as having a significant influence on health, and it will be important to gain a better understanding of these influences. For the most part, genetic factors are currently understood as contributing to a greater or lesser risk for health outcomes, rather than determining them with certainty. Briefly highlighted in the presentation was the link seen between genetics and behavior. Studies of twins separated at birth demonstrate a high concordance rate in alcoholism, schizophrenia, and affective disorders (Baird, 1994). Even so-called voluntary behaviors such as smoking and eating habits may be subject to genetic predispositions (e.g., Carmelli et al., 1992; de Castro, 1993; Falciglia and Norton, 1994). Health behaviors are complex, and the influences that determine them are likely to be extremely complex.

Genetic factors also interact with social and environmental factors to influence health and disease. It will be important to understand these interactions to learn why certain individuals with similar environmental exposures develop diseases whereas others do not (e.g., why most smokers do not develop lung cancer).

Physical Environment

The physical environment affects health and disease in diverse ways. Examples include exposures to toxic substances that produce lung disease or cancers; safety at home and work, which influences injury rates; poor housing conditions and overcrowding, which can increase the likelihood of violence, transmission of infectious diseases, and mental health problems; and urban-rural differences in cancer rates.

Behavior

In the field model framework, behavior is a response to the other determinants and can be seen as an "intermediate" determinant of health. It is shaped by many forces, particularly the social and physical environments and genetic endowment, as previously described. Behaviors related to health care, such as adherence to treatment regimens, are influenced by these forces as are behaviors that directly influence health, such as smoking.

Health Care

Health care has a limited but not negligible role as a determinant of health. Approximately 5 years of the 30-year increase in life expectancy achieved this century can be attributed to improved health services (Bunker et al., 1994). Of these 5 years, it has been estimated that curative services contribute about 3.5 and clinical preventive services about 1.5 years. The greatest share of this gain from health care can be attributed to diagnosis and treatment of coronary heart disease, which contributes 1 to 2 of these additional years of life.

Linking the Determinants

According to the presenter, the Evans and Stoddart field model helps in conceptualizing factors affecting health. Substantial evidence is available to support the relationship that many of these factors have with health. Currently incomplete, however, are descriptions of mechanisms underlying the linkages among the various determinants and full characterizations of the interactions among factors. Some evidence is available to demonstrate that these interactions exist. For example, high socioeconomic status is a buffer against the negative impact of perinatal stress on developmental outcomes in children at age 20 months (Werner, 1989). Similarly, high socioeconomic status reduces the negative impact of high umbilical lead levels on mental development (Bellinger et al., 1993). What is not yet available is an understanding of why the interactions occur.

INTERVENTIONS TO IMPROVE HEALTH

Many factors can influence the impact of interventions to improve health. It is possible to target various determinants of health to produce change at an individual level, a community level, or both. All aspects of each broad determinant of health are not equally amenable to intervention, however. For example, the social environment of isolated senior citizens can be improved by increasing contact with others, but their genetic endowment is not changeable.

Time frames for change following interventions can vary widely, from days to decades. Some successful interventions will produce observable results within a year or two, but others may be followed by long latency periods before significant changes can be observed in health status. The impact of an intervention may also be influenced by when it reaches an individual because there appear to be "critical periods" in human development. Certain interventions in childhood, for example, may have long-delayed yet long-lasting results. In addition, the population effects of interventions are important to consider. Small changes at the individual level may have important ramifications when applied to a whole community (Rose, 1992).

Community Interventions

The literature on community interventions is diffuse and difficult to summarize. A few observations based on that literature were shared with the committee. For example, the Healthy Cities-Healthy Communities activities demonstrate that a high level of interest in community interventions exists, but these activities have not yet generated a body of evidence that will allow them to be replicated in other settings. Study designs rarely meet high scientific standards. Although literature on advocacy and the process of community change abounds, validation through outcomes research is often lacking. Information linking process with outcome is inadequate, as are details describing implementation of interventions.

It was suggested that evidence that interventions have had a positive impact on the population is more likely to emerge in narrowly defined areas such as increasing immunization rates or decreasing workplace smoking. Similarly, one time accomplishments are easier to document than what is needed to sustain activities. Literature examining the difference between attaining goals and maintaining them is lacking, and this issue requires more attention.

Targets for Intervention

The traditional targets for intervention have been specific diseases or behaviors. The field model of the determinants of health suggests consideration of a wider array of targets. For example, if adolescents' sense of well-being can be improved by reducing their feelings of alienation and hopelessness, can unintended pregnancies, alcohol and other drug use, crime, and the school dropout rate all be reduced? A multidimensional approach would be required, focusing on education, social and community involvement, family preservation, and improved social networks for teens and their parents. Community-level interventions might include after-school programs, athletics (e.g., midnight basketball), and church-based programs.

The multidimensional approach may be unfamiliar to health professionals because it is new and relies on partnerships with people from fields beyond those traditionally encompassed by a medical model. It is, however, consistent with the field model and may provide expanded opportunities for performance monitoring and improving the community's health. The variety of ways in which community can be defined, such as geography, politics, or social networks, was also noted (Patrick and Wickizer, 1995). The committee was encouraged to consider all kinds of communities in seeking solutions to health needs.

Implications for Performance Monitoring

Performance monitoring should make use of measures of inputs, process and outcomes so that their interrelationships can be studied.⁵ It was suggested that key determinants of health should be monitored, regardless of whether they are amenable to change at the local level, so that communities can understand the range of important factors.

The value of both individual- and community-level data was emphasized. Subjective individual-level data may contribute important information about community needs. For example, information on social support, perceived barriers to service utilization, and attitudes toward the community and its resources is all relevant to health and to performance monitoring and can be obtained from community surveys.

The quality of cooperation among organizations is an often-neglected consideration for which community-level measures might be developed. The success of multiple organizations serving a particular community may depend on how well their services are coordinated. For example, senior citizens may be served by separate programs providing meals, transportation, outreach, and mental health services. Each program may be meeting its own goals, but if they are not working together, their overall impact may be diminished.

It was suggested to the committee that an initial step in performance monitoring is to determine which organizations and institutions in a community can affect health and disease. Those institutions can then be described with respect to goals and objectives, resources, and activities. What problems are being addressed? How effectively? What other activities might be added? Are these institutions educating the community about the problems and their responses?

Although organizations themselves can benefit from internal monitoring systems to determine their efficiency in resource utilization and whether desired outcomes are being achieved, they often lack the tools to adequately monitor their activities. If available, however, such tools may contribute to performance monitoring activities in the community. Important measures include units of services delivered, costs of services, proportion of need met, percentage of resources used to meet objectives, and impact. Community members can provide feedback, measuring how well individuals external to an organization rate the organization's efforts. In addition, an organization should consider how well its programs and services compare with "best practices." It was noted that efforts to identify best practice in developing and using community report cards are under way.

Performance monitoring provides an opportunity for a community to define and articulate expectations for organizations' contributions to the population's health. Although organizations might disagree with the appropriateness of the expectations, a useful dialogue may ensue. It was suggested that communities may want to focus special attention on expectations regarding managed care organizations (MCOs) and business. MCOs have improved provider education efforts and information tools such as clinical records, but "community" is often defined as their enrollees. Historically, MCOs have not considered the entire community or public health as their area of concern. A community expectation that part of their corporate and social responsibility is the health of the entire local population could encourage their broader involvement in public health activities.

Businesses, including MCOs, that have strong historical ties with a city or region may have greater interest in local health issues. However, as corporations expand to multiple regions, they may be less involved in the local communities where they have a presence. It was suggested that larger corporations operating in many locations should be encouraged to be involved in those communities. At the same time, smaller businesses with strong local bases should be educated and encouraged to become involved in community health efforts.

The presentation concluded with mention of another framework for assessing the community and health that translates the determinants of the field model into community terms (see Figure 2). Community social and physical environments are affected by cultural, political, policy, and economic systems and in turn, influence community response, activation, and social support, and ultimately community outcomes including social behaviors, community health, and quality of life (Patrick and Wickizer, 1995). Therefore, performance monitoring might also benefit from attention to the underlying cultural, political, and economic forces represented in this framework.

FIGURE 2 An organizing framework for studying community and health (illustrative factors for community social environment and physical environment). Source: Form D.L. Patrick and T.M. Wickizer, "Community and Health," p. 67, in Society and Health, B.C. Amick et al., eds. Copyright 1995 by B.C. Amick et al. Used by permission of Oxford University Press, Inc.

COMMUNITY EXPERIENCE WITH PERFORMANCE MONITORING

The committee heard presentations from five panelists describing health improvement programs in their communities. A moderator-led discussion followed the presentations.

McHenry County Project for Local Assessment of Need⁶

The McHenry County Project for Local Assessment of Need (McPLAN) is a framework for improving the community's health. McHenry County is the fastest growing county in Illinois and consists of 29 individual municipal areas. Although the county enjoys good health overall, certain subpopulations have less favorable health indicators.

The panelist noted that the State of Illinois was a pioneer in local community health assessment and planning through its network of local health departments. In 1993 and 1994, the Illinois Department of Public Health (IDHP), using a process patterned after the Assessment Protocol for Excellence in Public Health (APEXPH) (NACHO, 1991), coordinated a statewide project to assess the needs of local populations. McPLAN is part of this initiative. The process involved local health agencies and other health-related community organizations that promote health or work in related areas that contribute to health (e.g., education, jobs, housing). Institutionalization of the assessment process now requires that local health departments respond to local health priorities and maintain services in four basic areas (communicable disease, private sewage, private water, and food protection) rather than offer a standard set of 10 programs specified by the state. IDPH provides training and has developed a data system to better enable local health departments to conduct an effective assessment process.

In developing and implementing McPLAN, the McHenry County Department of Health has applied its experience over the past nine years in performance-based budgeting and community health needs assessment. These processes have been empowering for staff as well as community stakeholders In the budgeting process, problem statements are developed based on local needs assessment, and indicators are selected to serve as markers for appropriate public health interventions. Staffing and other resources needed to address these problems plus an annual review of the health department's mission and goals become the basis for developing a program budget. Each quarter, a review of indicator status and resource utilization allows further refinement of staffing and resource needs. Under McPLAN, both staff and community representatives are active participants in the process through advisory committees and the Community Health Committee. The health priorities selected for the county through the first application of the McPLAN process were environmental health, unintentional injuries, and cardiovascular disease.

The performance-based budgeting process has been used to incorporate health assessment findings into local public health programming and other stakeholder organization initiatives. The use of such a process over the past nine years in McHenry County has (1) aided in the training, focus, confidence, and perspective of staff; (2) led to a clearer understanding of the roles of the Board of Health, County Board, and related community organizations; (3) allowed movement toward allocation of a more appropriate level of resources to address identified issues (grants, appropriate fees, etc.); (4) led to greater involvement and understanding of other community providers; and (5) resulted in community-wide efforts to address childhood immunizations, improve access to health care for the medically indigent, develop joint grant applications, and begin discussion about a community-wide human services needs assessment in McHenry County.

City of Escondido Health Care and Community Services Project⁷

The goal of the Health Care and Community Services Project is to reduce the harmful effects of alcohol and other drug use in the community of Escondido, California (population, 120,000; county population, 2.6 million). The project coordinates a cross section of community services, including law enforcement, hospital emergency rooms, and community agencies. Integration of data systems, administrative coordination, financing, and training are other integral elements of the project's success. The municipal government functions as a facilitator for the community collaboration but does not provide services directly. Its interest is to reduce the cost of alcohol and drug use to the city and to improve the city's health.

Unlike most alcohol or drug control programs, which target individuals who are already dependent on alcohol or drugs, the Escondido project seeks to identify users who are at high risk of becoming dependent in the future. The objective is to influence drinking behavior before it reaches a critically destructive level, not to identify those already in need of specialized services (although such referrals are made when necessary). This approach is consistent with population data showing that the majority of alcohol and drug incidents involve users, not addicts. The program involves routine screening for alcohol or drug use in high-volume, high-risk situations. It includes a three- to five- minute screening interview and brief intervention. It is administered to all adults in hospital emergency rooms, health centers, and law enforcement settings. A new component of the program is the "Sobering Service," which provides services to individuals who would otherwise be sent to the police or to the emergency room for alcohol- or drug-related care.

Three important lessons were emphasized to the committee. First, the ability to cross sectors and create an integrated program made it possible to capture savings in one sector and make these resources available to the program. For example, the city is saving the money normally spent on booking people for alcohol-related offenses and investing it in the project. The project may become self-sustaining, because local private funds may soon be raised from managed care firms and combined with ongoing public funding for uninsured participants. (Initial funding for the project came from local city general funds, county government funds, and a matching grant from The Robert Wood Johnson Foundation.)

Second, the importance of data was emphasized. The availability of data helped to identify the stakeholders and to create a collaborative value system, based on community participation. Third, development of a data system will be important in monitoring and maintaining the integration of screening and brief intervention services within multiple collaborating agencies.

North Shore Community Health Network Area⁸

The Massachusetts state health department has designated 27 Community Health Network Areas (CHNAs) across the state with the goal of improving health at the community level. In each CHNA, representatives from provider groups, boards of health, community health centers, and neighborhoods work together to review community health needs, set priorities for health interventions, and help implement those interventions. The state is making available to the CHNAs data on community characteristics and health status that can be used in assessing health needs and setting priorities.

Experiences of the North Shore Community Health Network Area (NSCHNA), which encompasses eight towns north of Boston, were reviewed with the committee. The NSCHNA was one of three pilot efforts for the CHNA initiative, which began in 1992. It serves as an advocate for public health ideals, strengthens the public health team, focuses on the consumer, increases affiliations by integrating services with larger health systems, and de- emphasizes specific illnesses. It was noted that through the NSCHNA it has been possible to pool resources, which has stretched funds and may make more money available for efforts in areas such as prevention.

A variety of traditional health status indicators are monitored by the NSCHNA. These include economic and demographic statistics, cause-specific mortality. incidence of infectious diseases, maternal and child health indices, substance abuse rates, and hospital discharge data. High rates of lung cancer deaths and asthma hospitalization pointed to tobacco use as an area of particular concern. Thus, tobacco control emerged as the NSCHNA's initial priority. Activities that have achieved some success include promoting the removal of cigarette vending machines from areas accessible to children and adolescents and promoting expansion of smoke-free space in establishments such as malls and restaurants. Other CHNAs in the state have chosen to focus on issues such as immunization and a reduction in sexually transmitted diseases among 15- to 19-year olds

Arizona Partnership for Infant Immunization⁹

The goal of the Arizona Partnership for Infant Immunization is to improve the preschool immunization rate in Arizona by influencing provider behavior. In 1991, initial efforts were made to address immunization of preschool children. Available data showed that providers' perceptions that they were immunizing all their patients were inaccurate. Despite a historic lack of collaboration between the physician and HMO communities, representatives from each sector were convened as stakeholders Other identified stakeholders were advocacy groups, businesses, and foundations. Participation by pediatricians was high. Because influencing 2,000 providers directly would be very difficult attention was focused on the state's 25 health plans, hoping to influence providers via the payers.

To take further action, a formal partnership, the Arizona Partnership for Infant Immunization (TAPII), was formed. Collecting data on immunization rates was recognized as essential to the project, but reaching agreement on measurement methodology was difficult. Finally, CASA (Clinic Assessment Software Application), a tool of the Centers for Disease Control and Prevention (CDC, 1992), was chosen. Once a methodology was chosen, data collection and interventions were relatively easy to plan. Health plans are now using HEDIS (NCQA, 1993) and CASA methodologies to measure and collect data.

In 1993, only 46 percent of 2-year-olds were fully immunized. In 1994, the rates increased by 10 percent. These rates were determined from medical records but may have understated immunization levels if vaccine doses given by other providers were not recorded. In 1995, immunization rates measured by the National Immunization Survey were 77 percent.10 Provider education is a key component of the program. Once an immunization rate of 80-90 percent is reached, TAPII activities may be expanded to include parent education.

The panelist emphasized the importance of clearly defining a problem that motivated stakeholders to work together. Partners in the coalition have not yet agreed to work together on other problems but have been able to collaborate effectively on this clearly defined task.

Calhoun County Health Improvement Program¹¹

The Calhoun County Health Improvement Program (CCHIP) is a community-based program that began in 1993. Its mission is to improve community health in Calhoun County, Michigan. The county has a population of 136,000, with a minority population of 17,000. The county's health status indicators fall below state wide averages. CCHIP was developed with funding from the W.K. Kellogg Foundation. A participatory approach that includes providers, consumers, and payers is taken. It views "health care" more broadly than merely the provision of "medical care." Personal responsibility and primary prevention are central to its vision. Its organizational structure is conceived as a circle with improved health at the intersection of four quadrants: neighborhood groups, membership organizations, a governing board, and implementation teams.

Health assessment is conducted through collaboration between the Community Assessment Implementation Team (a CCHIP-based team) and the Calhoun County Health Department. Together, they have developed a health outcomes report and have shared responsibility for community response to the report.

The program has developed long-range goals based on a five-year plan with six focus areas: (1) community decision making, (2) community-wide health care coverage, (3) a comprehensive delivery system, (4) an integrated administrative structure, (5) a community-based health care information system, and (6) community assessment. The goals include decreasing the risks to health, increasing access to cost-effective health care through the establishment of a purchasing cooperative, improving decision making through a community health information network, and changing local and state policy to reflect community values and community decision-making processes.

DISCUSSION

In response to questions from the committee, workshop panelists discussed their experiences with performance monitoring. They specifically addressed identifying stakeholders, selecting health priorities and indicators, using indicators for accountability, gaining community support for performance monitoring, and implementing a performance monitoring system.

Identification of Stakeholders

As described by the panelists, identification of stakeholders proceeds in one of two ways, depending on whether the stakeholders are involved in defining or responding to the problem. When stakeholders are involved in problem identification, it is best to cast a wide net, leading particular groups to self-identify as stakeholders and become active collaborators. The stakeholder group may evolve as the process moves from problem identification to intervention to evaluation. It was noted that in the public sector different agencies or different personnel within agencies may become involved depending on the stage of the program.

When a problem is already defined and an intervention is suggested by existing data (as in TAPII), the participation of key stakeholders able to produce the desired results can be actively sought. In Arizona, the process was facilitated by participation of the governor, who convened a meeting of identified stakeholders Groups with divergent interests may be able to cooperate in implementing solutions to defined problems when data and interventions are available to focus their joint efforts.

Participation by "consumers," that is, members of the general public, was mentioned as an important stakeholder issue. Ideally, participants should reflect the various population groups in the community, based on factors such as age, race or ethnicity, and neighborhood. All members of coalitions are consumers in a sense, but most participants are invested in particular interests. Engaging those participants who are not affiliated with particular stakeholder groups can be difficult. Barriers include the difficulty in identifying interested individuals and the commitment in time and energy that is required. Other barriers to participation can include meetings scheduled for normal working hours or added costs such as transportation and child care. TAPII found that because consumers participated for only a short period of time focus groups and community surveys were helpful for bringing their perspectives into the process.

Selecting Health Issues and Performance Indicators

Epidemiologic data are often used to guide the selection of performance indicators, but community interest may argue for focusing attention on specific issues or indicators, even in the absence of supporting epidemiologic data. It was noted that in some cases, a "triggering event" may focus attention on a particular health issue. The trigger might be a severe adverse event such as a measles epidemic or a more positive stimulus such as the availability of funds and other resources earmarked for specific topics.

Social determinants of health (e.g., income, family structure) and epidemiology are sometimes viewed as separate issues because epidemiology traditionally is associated with the biomedical model. However, the scope of epidemiology has expanded to include measurement of social factors, and epidemiologic data can drive the development of interventions in the social realm. Some workshop participants believe that the distinction between the two is artificial.

A discussion of the utility of epidemiologic health status data based on small sample sizes generated dissenting views. Some participants commented that sampling error is too large to make detailed follow-up measurements a worthwhile use of resources. Changes on the order of a few percent per year are extremely difficult to measure at the community level. Even if changes are measurable, communities may lack the resources to collect such data accurately. Other participants suggested that although measurement error can be a problem, it is essential to quantify problems and the effects of interventions. Otherwise, efforts to solve problems could be completely misguided. They suggested that combining quantitative and qualitative information can provide a more meaningful picture of a community's health.

Should Performance Indicators Be Standardized?

Workshop participants noted that there is tension between the need for standardized performance indicators and the need for community flexibility in defining indicators. Standardized indicators are advantageous for making comparisons within and between communities, for simplifying the synthesis of data from different sources, and for developing data systems. However, in designing and monitoring interventions in individual communities, the development of more specific indices may be helpful and standardization may be less important.

It was also suggested that the dichotomy between standardization and individualization is artificial. Most programs would benefit from a combination of the two. A basic set of indicators could be developed, with modifications based on specific community needs. Alternatively, a broad spectrum of questions could be developed from which communities could choose appropriate subsets. The selection of indicators may be especially difficult in a diverse community. Participants pointed out that performance indicators could be coordinated with currently existing health indicators in the private and public sectors such as HEDIS 2.0 (NCQA, 1993), APEXPH (NACHO, 1991), Model Standards (APHA et al., 1991), and Healthy People 2000 (USDHHS, 1991).

Development of performance indicators requires stakeholder involvement. Some participants emphasized the importance of including private-sector stakeholders early in the planning stages in order to frame questions in a manner compatible with existing data bases. Distinguishing performance indicators from overall program evaluation was raised as an important distinction. The speaker suggested that performance monitoring should focus on the component parts of an intervention whereas program evaluation should examine the overall outcome.

Role of Performance Indicators in Stakeholder Accountability

Workshop participants agreed that defining accountability—which "actors" are responsible for what functions—is extremely important. Accountability for both process and outcome goals needs to be determined. The programs described at the workshop vary in how they hold groups accountable. For example, TAPII has led to ties between capitation and immunization rates, which fosters competition among health plans. Competition as a mechanism to gain accountability may be especially useful in the private sector. The Escondido program addresses accountability through formal written agreements detailing participation in the project, contract arrangements with providers, target rates for screening and performance, and management information systems for the integrated services. In other programs represented at the workshop, accountability is less explicitly defined. According to one panelist, it is hoped that "providers (will) come forward and increase their provision of services or education as it relates to the objectives."

Community Responses to Performance Indicators

The workshop participants reported that positive outcomes engender positive community response and that achievement of short-term objectives serves to cement community cooperation. They also commented on the importance of communicating realistic expectations to prevent discouragement with slower progress toward long-term objectives. Balancing short-term and long-term objectives helps maintain motivation.

Strong leadership is necessary to prevent coalitions from splintering into groups with self-serving agendas. On the other hand, outside leadership cannot substitute for communities' developing their own momentum to maintain programs. It was suggested that community groups need to be involved from the start in order for a community to be empowered and to continue projects regardless of changes in political personnel.

Tension between health problems articulated by the community and health problems identified by data analysis can potentially undermine community support for performance monitoring and health improvement activities. Participants suggested that the specific approach to selecting health issues should involve a larger community collaborative. However, the conceptual bases for selecting health issues should be founded on research that suggests that effective interventions will be possible.

Implementation: What Does It Take?

Issues common to the five programs discussed with the committee are availability of resources, leadership, training, and the development of organizational knowledge. The balance between data-driven and community-driven processes varies among the five programs, as does the degree of community involvement in defining problems and interventions. All use performance indicators in some form.

SOCIAL CHANGE AND ACCOUNTABILITY¹²

Recognizing that health improvement activities and performance monitoring imply the need for change in communities, the committee sought to explore some of the theories of social change and how they might relate specifically to health and health care. It was noted that change is ubiquitous today in health care systems, health care policy, and social policy and is occurring in multiple dimensions. Emphasis is shifting from individual health to population-based health; from tertiary to primary care; from preventive care to health promotion. Tension between controlling costs and improving health complicates change in all dimensions.

Change is not linear. It occurs in a specific context and is subject to complicated interactions. Change is a process of transition; therefore, it is fruitful to study both the change process and its outcome. To determine whether an outcome is causally related to a particular intervention, it is necessary to study the process of change linking the intervention and the outcome. The suggestion was made that natural experiments provide unique opportunities to study change and deserve more scrutiny than they currently receive.

The committee was reminded that people frequently resist change and that change can both arise from and contribute to conflict and tension. Although admittedly uncomfortable, conflict and tension may be necessary prerequisites for constructive change.

Models of Change

Three theoretical contructs that can be used in formulating models of change were noted. Structural functionalism is a positivist approach and is consistent with an epidemiologic orientation. Conflict theory views change as subjective and value laden. Its naturalistic approach parallels community development and participatory action orientations. Symbolic interactionism involves developing consensus to produce change (Thompson and Kinne, 1990).

Individual Change

Change at the individual level is described by several models. The "stages of change" model was developed to describe smoking cessation (Prochaska and DiClemente, 1986). Readiness for change progresses through stages of precontemplation, contemplation, action, and maintenance. For maximum impact, health interventions are chosen with attention to the individual's stage of readiness.

The Health Belief Model views behavioral change as the result of "triggers" (Rosenstock et al., 1988). Beliefs about susceptibility, severity, benefits of treatment, and barriers to treatment contribute to individuals' willingness to change their behavior. The committee was told that this concept, along with other models of behavioral change, may also be important in studying the change process at an organizational level.

Organizational Change

An organizational model of change described by Lewin (1976) is based on a three-stage process that includes "unfreezing" the old behavior, cognitive recognition of the need for a new behavior, and "refreezing" the new behavior. This description is accurate for many organizational change processes. In health care, however, change is currently so rapid that behavior is in a seemingly constant state of unfreezing and refreezing.

Other models also describe organizational change as a staged process (Beckhard and Harris, 1987; Bridges, 1980). Thompson and Kinne (1990) offer a community development model of change that considers change on a continuum from individual to community. The PRECEDE-PROCEED model developed by Green and Kreuter (1991) is also frequently used in health promotion. It approaches change through factors that are grouped as predisposing, enabling, or reinforcing.

Change in Health Care: Case Study of Alberta, Canada

Both initiating performance monitoring and responding to the problems identified by performance monitoring systems require changes in the community and on the part of various stakeholders Alberta, Canada, provides an example of the change process in health care.

Several tensions are influencing health care: individual versus population health; treatment of illness versus health promotion; meeting health needs versus managing health care costs; traditional versus new organizational models; current social conditions versus societal goals; and maintaining the status quo versus shifting paradigms (Casebeer and Hannah, 1995). Regionalization of the health care system in the province of Alberta constitutes a significant change in the arrangements for managing and providing health services. The change is largely an attempt to control increased health care spending, which grew from 20 to 32 percent of the provincial government's budget between 1980-1981 and 1993-1994, and to alter the orientation of health care provision (managing the system regionally and shifting to a population-based, community-based, health-promoting focus for care).

A study of change in health care and health policy identifies processes of change used by managers, as well as expected and actual health outcomes (Casebeer, 1996). Managers have suggested that successful change depends on the development of structures, processes, and outcomes that encourage the system to change in positive and sustainable ways.

With regard to structures, these managers are attempting to work with

• new and broader governance roles;

• leaner, flatter, more horizontal management of the system;

• new working arrangements for health care providers and managers; and new participatory roles for communities.

In relation to process issues, managers emphasized several critical aspects of change:

• the importance of sustaining political will;

• the pace of change;

• the capacity for shifting resources;

• the need for a renewed commitment to positive change;

• improved communication capabilities;

• better information;

• effective planning; and

• time for learning and adjusting.

Managers articulated a range of hopes and concerns in relation to short-term and long-term outcomes. For example, they expect that new management structures and savings would be short-term outcomes, new ways of developing services for better information would be medium-term outcomes, and improved services and health status would be long-term outcomes.

Gaining a better understanding of health care change such as that taking place in Alberta will require additional longitudinal and comparative experience as well as targeted research.

COMMENTARY¹³

The workshop discussions served as the basis for a commentary on community-based performance monitoring and issues to which the committee should give further attention. It was noted that the day's discussions focused broadly on community health improvement and community activation, rather than focusing more narrowly on performance indicators. This perspective is consistent with many community-based efforts to reduce health risks and prevent disease, such as the National Heart Lung and Blood Institute's cardiovascular risk reduction programs or the Kaiser Family Foundation's community health promotion grants program. Coalition building was central to these programs. They emphasized ensuring community involvement and participation of key stakeholders; needs assessment; project implementation based on the needs assessment; and program monitoring and evaluation.

It was suggested that although this approach, which is based on collaboration and community empowerment, is consistent with public health values, the evidence to date suggests that the model, as implemented in the past, may not work. Coalitions include varied interest groups and may be swayed by political concerns. The process may not select the most effective interventions at a population level. Efforts are being made, however, to bridge the gap that seems to exist between the community activation approach and the science of health improvement (Wandersman et al., 1995).

In contrast, the HEDIS approach relies on central planning and oversight. Although its top-down approach may conflict with the values and instincts of public health practitioners, it appears to be effective in promoting change. Its effectiveness was attributed to its visibility, its evidence-based approach, and its use of measures that lend themselves to managerial action.

The speaker proposed a new paradigm for community health improvement based on a synthesis of community partnerships with an evidence-based approach. First, cooperation with the private sector, particularly medical care, would be a key element. Second, the private sector requires a business reason such as competition to participate. Third, performance indicators should be used to focus attention on those health issues and interventions supported by scientific evidence, as well as to generate and sustain accountability. Finally, the partnership should generate specific implementation strategies. In sum, performance indicators should support a community participation model by helping community partnerships set priorities and design interventions based on evidence.

Also critical to consider is the issue of accountability. In the speaker's view, accountability should be clearly assigned within the community. It must also rest on all who have responsibility to act. For there to be true accountability, performance must be monitored.

The workshop discussions suggested that performance indicators are used for multiple purposes: to identify problems and generate hypotheses, as political tools for mobilization in the community, to suggest ideas for improvement, and in fact, to monitor the performance of specific sectors of the community. Among the characteristics of useful performance indicators is a focus on populations and rates, rather than on absolute numbers of contacts involved in the interventions. In other words, the denominator is as important as the numerator.

Indicators were described as most useful when they focus on areas where improvement is possible. Global health status indicators often have little practical use for guiding health improvement strategies. More useful are indicators that incorporate a "theory of improvement"—that is, they suggest a clear means of moving from measurement to action. Indicators that have been shown to change in intervention studies should be preferred over those that may be more conceptually elegant but may not be able to capture the impact of an intervention. "Responsive" indicators of this sort allow real change to be distinguished from random variation.

The value of standard epidemiologic health needs assessments was questioned. Often, needs assessments merely document problems that are already well known. On the other hand, needs assessments focused on factors in the community that influence program implementation—politics, resources, barriers, key players—may be very useful. The speaker also emphasized that although coalitions are an essential component of community-based health improvement projects, they can consume substantial resources. Participants may, for example, spend an average of 3-4 hours a month conducting coalition- related work. It has also proved difficult to document a relationship between the characteristics of coalition operations and health outcomes achieved. The contributions that coalitions make to health improvement activities need to be better understood.

The committee was urged to articulate a model (or models) of health improvement that specifies use of performance indicators and holds social and nonclinical improvement strategies to the same evidence base as clinical strategies. Such a model should

• help communities clarify accountability and consider ways in which to include the private and public sectors as accountable entities;

• identify performance indicators in the model of health improvement; indicators should not be expected to generate models of community improvement;

• illustrate its concepts with the selection of a limited number of "performance areas" that are characterized by (a) evidence that services affect health status, (b) a clear theory of improvement, and (c) some reasonable ideas about how to reach the entire population; and

• identify key input processes and intermediate outcomes within each performance area.

CONCLUDING OBSERVATIONS¹⁴

The workshop concluded with a review of lessons for the committee, beginning with comments on the field model. The field model appears especially useful from a public health perspective. The model facilitates focusing on population effects, and its broad inclusion of disease, well-being, health, and function provides a basis for expanding monitoring systems to include these areas.

The field model also provides a useful basis for addressing the committee's concern for equity in health and how to promote equity in health through a performance monitoring system. The model makes it possible to study equity as it relates to social class, family structure, education, and social networks. The model's treatment of genetics in interrelationship with other determinants and its view of behavior as an intermediate determinant are also helpful. Considering behavior as a product of various factors encourages users of the model to avoid blaming victims. Still needed, however, is better information regarding which determinants are actually amenable to intervention and whether community processes really lead to measurable community outcomes. Ways to measure cooperation are also lacking.

The panel's presentations and discussion illustrated differences among programs in the degree to which goals, performance measurement, and stakeholder roles have been articulated. The discussion also pointed out that community process can be catalyzed by a triggering event. This might be the availability of funding or public outcry when a situation is unacceptable. Different communities will require different approaches to the selection and use of performance indicators. It will be a challenge for the committee to propose a system that satisfies both "cookbook" and "menu" approaches.

Stakeholder identification appears to occur in two parallel tracks based on different responsibilities in some programs. One set of stakeholders is more involved in developing the information infrastructure, while the other set of stakeholders is involved in decision making or policy development. Potential trouble exists if the two groups do not communicate adequately. It was suggested that well-constructed coalitions of stakeholders can "keep the process honest." Ensuring meaningful consumer participation is another challenge shared by the programs, and is a topic that requires more attention. Panelists were sensitive to the need to listen both to stakeholders who are active participants and to those who are not before reaching conclusions about intervention strategies or performance indicators.

Concern about the potential for harmful use of data provided by performance monitoring was raised. There is a possibility that data could be misused in resource allocation if overly simplistic formulas are applied, and the committee must remain aware of these risks. Communities with multiple needs and few resources might lose funding for doing poorly, or communities that are achieving positive results might be at risk of losing funding if needs are assumed to be met. A community that is addressing a difficult problem may be doing a good job if it can maintain a given level of performance. For some health issues, prevalence of HIV/AIDS infection, for example, finding only a small increase might represent progress over higher increases in the past. It is also important to monitor "what is going right" rather than just looking for poor outcomes.

Final comments addressed social change issues. The models described to the committee contribute to the notion that the process of change is as important as the outcome. They also emphasize that the role of each stakeholder in the process is important for the committee to consider.

References

APHA (American Public Health Association), Association of Schools of Public Health,Association of State and Territorial Health Officials, National Association of County Health Officials, United States Conference of Local Health Officers,Department of Health and Human Services, Public Health Service, Centers for Disease Control. 1991. Healthy Communities 2000: Model Standards. 3rd ed.Washington, D.C.: APHA.

Baird, P.A. 1994. The Role of Genetics in Population Health. In Why Are Some People Healthy and Others Not?: The Determinants of Health of Populations. R.G. Evans, M.L. Barer, and T.R. Marmor, eds. New York: Aldine de Gruyter.

Beckhard, R., and Harris, R.T. 1987. Organizational Transitions: Managing Complex Change. 2nd ed. Don Mills, Ontario: Addison Wesley.

Bellinger, D., Leviton, A., Waternaux, C., Needleman, H., and Rabinowitz, M. 1993. Low-Level Lead Exposure, Social Class and Infant Development. Neurotoxicology and Teratology 10:497-504.

Berkman, L.F., and Syme, S.L. 1979. Social Networks, Host Resistance, and Mortality: A Nine Year Follow-up Study of Alameda County Residents. American Journal of Epidemiology 109:1 86-204.

Bridges, W. 1980. Transitions: Making Sense of Life's Changes. Reading, Mass.: Addison Wesley.

Bunker, J.P., Frazier, H.S., and Mosteller, F. 1994. Improving Health: Measuring Effects of Medical Care. Milbank Quarterly 72(2):225-255.

Carmelli, D., Swan, G.E., Robinette, D., and Fabsitz, R. 1992. Genetic Influence on Smoking: A Study of Male Twins. New England Journal of Medicine 327:829-833.

Casebeer, A.L. 1996. The Process of Change Related to Health Policy Shift. Unpublished Ph.D. dissertation. Department of Community Health Services, University of Calgary.

Casebeer, A.L., and Hannah, K.J. 1995. Evaluating the Process of Change Related to Health Policy Shift. Presented at Evaluation 1995-Evaluation for a New Century: A Global Perspective, Vancouver, British Columbia. November 1-5.

CDC (Centers for Disease Control and Prevention). 1992. Guidelines for Assessing Vaccination Levels of the 2-Year-Old Population in a Clinic Setting. Atlanta: U.S. Department of Health and Human Services, Public Health Service.

de Castro, J.M. 1993. Genetic Influences on Daily Intake and Meal Patterns of Humans. Physiology and Behavior 53:777-782.

Evans, R.G. and Stoddart, G.L. 1990. Producing Health, Consuming Health Care. Social Science and Medicine 31:1347-1363.

Falciglia, G.A., and Norton, P.A. 1994. Evidence for a Genetic Influence on Preference for Some Foods. Journal of the American Dietetic Association 94(2): 151-1 58.

Green, L.W., and Kreuter, M.W. 1991. Health Promotion Planning: An Educational and Environmental Approach. 2nd ed. Mountain View, Calif.: Mayfield.

IOM (Institute of Medicine). 1988. The Future of Public Health. Washington, D.C.:National Academy Press.

IOM 1990. Medicare: A Strategy for Quality Assurance. Vol I. K.N. Lohr, ed. Washington, D.C.: National Academy Press.

Karasek, R.A., and Theorell, T. 1990. Healthy Work Stress, Productivity and the Reconstruction of Working Life. New York: Basic Books.

Lewin, K. 1976. Field Theory as Human Science. Compiled by J. de River. New York: Gardner Press.

Marmot, M.G., Kogevinas, M., and Elston, M.A. 1987. Social/Economic Status and Disease. Annual Review of Public Health 8:111-135.

NACHO (National Association of County Health Officials). 1991. APEXPH: Assessment Protocol for Excellence in Public Health. Washington, D.C.: NACHO.

NCQA (National Committee for Quality Assurance). 1993. Health Plan Employer Data and Information Set and User's Manual, Version 2.0 (HEDIS 2.0). Washington, D.C.: NCQA.

Patrick, D.L., and Wickizer, T.M. 1995. Community and Health. In Society and Health. B.C. Amick., S. Levine, A.R. Tarlov, and D.C. Walsh, eds. New York: Oxford University Press.

Prochaska, J.O., and DiClemente, C.C. 1986. Toward a Comprehensive Model of Change. In Dealing with Addictive Behaviors. W. Meller, and N. Healhen, eds. New York: Plenum.

Rose, G. 1992. The Strategy of Preventive Medicine. New York: Oxford University Press.

Rosenstock, I.M., Strecher, V.J., and Becker, M.H. 1988. Social Learning Theory and the Health Belief Model. Health Education Quarterly 15(2): 175-183.

Schor, E.L., and Menaghan, E. 1995. Family Pathways to Child Health. In Society and Health. B.C. Amick, S. Levine, A.L. Tarlov, and D.C. Walsh, eds. New York: Oxford University Press.

Thompson, B., and Kinne, S. 1990. Social Change Theory: Applications to Community Health. In Health Promotion at the Community Level. N. Bracht, ed. Newbury Park, Calif.: Sage Publishing.

USDHHS (U.S. Department of Health and Human Services). 1991. Healthy People 2000: National Health Promotion and Disease Prevention Objectives. DHHS Pub. No. (PHS) 91-50212. Washington, D.C.: Office of the Assistant Secretary for Health.

Valkonen, T. 1989. Adult Mortality and Level of Education: A Comparison of Six Countries. In Health Inequalities in European Countries. J. Fox, ed. Aldershot, England: Gower.

Wandersman, A., Morrissey E., Crusto, C., et al. 1995. Bridging the Gap Between the State of the Science and the State of the Practice in Prevention: Program Improvement Through Program Evaluation. Presented at Community-Based Health Promotion: State of the Art and Recommendations for the Future, Seattle, Wash. October.

Weikart, D.P. 1989. Early Childhood Education and Primary Prevention. Prevention in Human Services 6(2):285-306.

Werner, E.E. 1989. Children of the Garden Island. Scientific American 260(April): 106-111.

Wilkinson, R.G. 1992. Income Distribution and Life Expectancy. British Medical Journal 304(6820): 165-168.

Wilkinson, R.G. 1994. The Epidemiological Transition: From Material Scarcity to Social Disadvantage? Daedalus 123(4):61-77.

Zill, N. II, and Brim, O.G., Jr. 1983. Development of Childhood Social Indicators. In Children, Families, and Government: Perspectives on American Social Policy. E.F. Zigler, S.L. Kagan, and E. Klugman, eds. New York: Cambridge University Press.

Appendix A

Workshop Agenda

December 11, 1995
Foundry Building—Room 2004
1055 Thomas Jefferson Street, N.W.
Washington, D.C.

8:30 a.m.

Welcome and Overview of the Committee's Approach to
Performance Monitoring
Thomas Inui, Harvard Medical School

9:00 a.m.

Presentation and Discussion on Determinants of Health
Jonathan Fielding, University of California at Los Angeles

10:00 a.m.

Community Experience with Performance Monitoring
Moderator: Alan Cross, University of North Carolina

Brief Program Descriptions

J. Maichle Bacon, McPlan, the McHenry County (Illinois) Project for Local Assessment of Need
Dennis Kelso, Health Care and Community Services Project (Escondido, California)
Tony Traino, North Shore Community Health Network (Massachusetts)
Laurie Carmody, Arizona Partnership for Infant Immunization
Bonnie Rencher, Calhoun County (Michigan) Health Improvement Program

11:00 a.m.

Panel Discussion:
Committee Questions on Performance Monitoring Experience and Perspectives

Continue Panel Discussion

3:30 p.m.

Presentation and Discussion on Issues of Social Change
and Accountability
Ann Casebeer, University of Calgary

4:15 p.m.

Commentary and Response
Edward Wagner, Group Health Cooperative of Puget Sound and University of Washington

4:45 p.m.

Concluding Discussion and Comments
Bobbie Berkowitz, Washington State Department of Health

5:15 p.m. Adjourn

Appendix B

Workshop Speakers and Guests

SPEAKERS

J. MAICHLE BACON, Public Health Administrator, McHenry County Department of Health, Woodstock, Illinois

LAURIE L. CARMODY, Public Health Consultant, Group Health Association of America, Washington, D.C.

ANN CASEBEER, Doctoral Candidate, University of Calgary, Department of Community Health Sciences, Calgary, Alberta

JONATHAN E. FIELDING, Professor of Health Services and Pediatrics, University of California at Los Angeles, Los Angeles, California

DENNIS J. KELSO, Director, Health Care and Community Services Project, Escondido, California

BONNIE RENCHER, Community Outreach Coordinator, Calhoun County Health Improvement Program, Battle Creek, Michigan

TONY TRAINO, Associate Director, Home Care Operations, Visiting Nurse Association of Greater Salem, Salem, Massachusetts

EDWARD H. WAGNER, Director, Center for Health Studies and W.A. (Sandy) MacColl Institute for Healthcare Innovation, Group Health Cooperative of Puget Sound, Seattle, Washington

INVITED GUESTS

DENNIS P. ANDRULIS, National Public Health and Hospital Institute, Washington, D.C.

MICHAEL BARRY, Public Health Foundation, Washington, D.C.

GEORGES C. BENJAMIN, Public Health Services, Department of Health and Mental Hygiene, Baltimore, Maryland

CHERYL BEVERSDORF, Association of State and Territorial Health Officials, Washington, D.C.

JACKIE BRYAN, Association of State and Territorial Health Officials, Washington, D.C.

LINDA K. DEMLO, Agency for Health Care Policy and Research, Rockville, Maryland

PATRICIA A. EBENER, Behavioral Scientist, RAND, Santa Monica, California

MARGO EDMUNDS, Institute of Medicine, Washington, D.C.

SARA GARSON, Center for the Advancement of Health, Washington, D.C.

CAREN GINSBERG, National Public Health and Hospital Institute, Washington, D.C.

SUSANNA GINSBURG, Lewin-VHI, Inc., Fairfax, Virginia

HOLLY GRASON, Child and Adolescent Health Policy Center, Johns Hopkins School of Hygiene and Public Health, Baltimore, Maryland

CLAUDE H. HALL, JR., Public Health Innovations Project, American Public Health Association, Washington, D.C.

RICHARD HEGNER, National Health Policy Forum, Washington, D.C.

PHYLLIS E. KAYE, American Health Planning Association, Washington, D.C.

ROZ LASKER, The New York Academy of Medicine, New York, New York

J. MICHAEL MCGINNIS, National Research Council, Washington, D.C.

CATHY MERCIL, National Committee for Quality Assurance, Washington, D.C.

MICHAEL MILLMAN, Health Resources and Services Administration, Rockville, Maryland

NANCY RAWDING, National Association of County and City Health Officials, Washington, D.C.

JORDAN RICHLAND, Partnership for Prevention, Washington, D.C.

JAMES SCANLON, U.S. Department of Health and Human Services, Washington, D.C.

JOE THOMPSON, U.S. Department of Health and Human Services, Washington, D.C.

KAREN TROCCOLI, National Association of County and City Health Officials, Washington, D.C.

JAMES WEED, National Center for Health Statistics, Hyattsville, Maryland

RONALD WILSON, National Center for Health Statistics, Hyattsville, Maryland

NOTES

¹This section is based on comments by Thomas Inui.

²The committee's approach to performance monitoring relies on both the public health "core functions" of assessment and assurance (IOM, 1988) and the health care activities related to quality assessment, assurance, and improvement (see IOM, 1990). From a public health perspective, assessment is the regular collection, analysis, and dissemination of information on the health of the community. Assurance refers to a governmental responsibility to ensure that services necessary to achieve agreed upon goals are provided. Quality assessment refers to measurement of processes and outcomes of health care and their comparison against a standard. Quality assurance employs such measurement within the context of a broader set of activities that includes steps to identify and correct problems. Quality improvement uses continuous measurement and analysis of processes and outcomes not only to address problems but also to maintain and enhance good performance. The health improvement activities envisioned by the committee combine a responsibility to the community for achieving health goals with techniques like those used in quality improvement.

³The indicators proposed by the committee will appear in the final report for this study.

⁴This section is based on a presentation by Jonathan Fielding.

⁵In the context of the committee's work, outcome measures describe a state of health or well-being (e.g., immunization rates) that is the product of factors that can be characterized on the basis of the field model. To monitor outcomes that change slowly, intermediate outcome measures may be used (e.g., monitoring changes in prevalence of smoking rather than changes in incidence of lung cancer). Process measures describe activities that are being performed in connection with efforts to achieve a desired outcome. Input measures (also referred to as measures of structure or capacity) describe the characteristics of resources (e.g., funds, personnel, equipment, time, policies) available or in use (e.g., number of doses of vaccine available).

⁶This section is based on a presentation by J. Maichle Bacon.

⁷This section is based on a presentation by Dennis Kelso.

⁸This section is based on a presentation by Tony Traino.

⁹This section is based on a presentation by Laurie Carmody.

¹⁰These results are based on parents' records, which typically show rates 15-20 percent higher than doctors' records.

¹¹This section is based on a presentation by Bonnie Rencher.

¹²This section is based on a presentation by Ann Casebeer.

¹³This section is based on comments by Edward Wagner.

¹⁴This section is based on comments by Bobbie Berkowitz.

Top of Document | NAP Home Page | Document Home Page