A Role for
Performance Monitoring
Committee on Using Performance Monitoring
to Improve Community Health
Jane S. Durch, Linda A. Bailey, and Michael A. Stoto, Editors
Division of Health Promotion and Disease Prevention
INSTITUTE OF MEDICINE
NATIONAL ACADEMY PRESS
Washington, D.C. 1997
NATIONAL ACADEMY PRESS • 2101 Constitution Avenue, N.W. • Washington, DC 20418
NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance.
This report has been reviewed by a group other than the authors according to procedures approved by a Report Review Committee consisting of members of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine.
The Institute of Medicine was chartered in 1970 by the National Academy of Sciences to enlist distinguished members of the appropriate professions in the examination of policy matters pertaining to the health of the public. In this, the Institute acts under the Academy's 1863 congressional charter responsibility to be an adviser to the federal government and its own initiative in identifying issues of medical care, research, and education. Dr. Kenneth I. Shine is president of the Institute of Medicine.
Funding for this project was provided by the Office of the Assistant Secretary for Health, U.S. Department of Health and Human Services (contract no. 282-94-0032); The Robert Wood Johnson Foundation (grant no.024336); and the Kellogg Endowment Fund of the National Academy of Sciences and the Institute of Medicine. The views presented in this report are those of the Committee on Using Performance Monitoring to Improve Community Health and are not necessarily those of the funding organizations.
Library of Congress Cataloging-in-Publication Data
Institute of Medicine (U.S.). Committee on Using Performance
97-6336
CIP
Copyright ©1997 by the National Academy of Sciences. All rights reserved. .
Printed in the United States of America
The serpent has been a symbol of long life, healing, and knowledge among almost all cultures and religions since the beginning of recorded history. The image adopted as a logotype by the Institute of Medicine is based on a relief carving from ancient Greece, now held by the Staatlichemuseen in Berlin.
BOBBIE A. BERKOWITZ* (Co-Chair), Deputy Secretary,
Washington State Department of Health, Olympia, Washington
THOMAS S. INUI† (Co-Chair), Professor and Chair, Department of Ambulatory Care and Prevention, Harvard Medical School and Harvard Pilgrim Health Care, Boston, Massachusetts
ALAN W. CROSS (Vice Chair), Professor of Social Medicine and Pediatrics and Director, Center for Health Promotion and Disease Prevention, University of North Carolina, Chapel Hill, North Carolina
LARRY W. CHAMBERS, Epidemiology Consultant, Hamilton-Wentworth Regional Public Health Department, and Professor, Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ontario, Canada
THOMAS W. CHAPMAN,‡ Chief Executive Officer, George Washington University Hospital, and Senior Vice President for Network Development, George Washington University Medical Center, Washington, D.C.
ELLIOTT S. FISHER, Co-Director, Veterans Affairs Outcomes Group, Veterans Affairs Medical Center, White River Junction, Vermont, and Associate Professor of Medicine and Community and Family Medicine, Dartmouth Medical School, Hanover, New Hampshire
JAMES L. GALE, Professor, Department of Epidemiology, School of Public Health and Community Medicine, and Director, Northwest Center for Public Health Practice, University of Washington, Seattle; Health Officer, Kittitas County, Washington
KRISTINE GEBBIE† (Liaison, Board on Health Promotion and Disease Prevention), Assistant Professor of Nursing, Columbia University School of Nursing, New York, New York
FERNANDO A. GUERRA, Director of Health, San Antonio Metropolitan Health District, San Antonio, Texas
GARLAND H. LAND, Director, Center for Health Information Management and Epidemiology, Missouri Department of Health, Jefferson City, Missouri
SHEILA LEATHERMAN, Executive Vice President, United HealthCare Corporation, Minneapolis, Minnesota
JOHN R. LUMPKIN, Director, Illinois Department of Public Health, Springfield, Illinois
WILLIAM J. MAYER, President and General Manager, Functional Foods Division, Kellogg Company, Battle Creek, Michigan
ANA MARIA OSORIO, Chief, Occupational Health Branch, California Department of Health Services, Berkeley, California
SHOSHANNA SOFAER, Associate Professor and Associate Chair for Research, Department of Health Care Sciences, George Washington University Medical Center, Washington, D.C.
DEBORAH KLEIN WALKER, Assistant Commissioner, Bureau of Family and Community Health, Massachusetts Department of Public Health, Boston, Massachusetts
JOHN E. WARE, Jr.,‡ Senior Scientist, The Health Institute, New England Medical Center, Boston, Massachusetts
RICHARD A. WRIGHT, Director, Community Health Services, Denver Department of Health and Hospitals, Denver, Colorado
Study Staff
Linda A. Bailey, Senior Program Officer (Co-Study Director)
Jane S. Durch, Program Officer (Co-Study Director)
Stephanie Y. Smith, Project Assistant
Michael A. Stoto, Director, Division of Health Promotion and Disease Prevention
Marissa W. Fuller, Research Associate
Sarah H. Reich, Project Assistant
Susan Thaul, Senior Program Officer
An interest in understanding how health care and public health activities might be coordinated and directed toward improving the health of entire communities was the basis for this study by the Institute of Medicine (IOM) Committee on Using Performance Monitoring to Improve Community Health, which we jointly chaired.
The IOM was asked by the U.S. Department of Health and Human Services and The Robert Wood Johnson Foundation to undertake a two-year study to examine the use of performance monitoring and develop sets of indicators that communities could use to promote the achievement of public health goals. The study was originally approved in mid-1994 when passage of federal health care reform legislation was anticipated. Part of the task outlined at that time was to identify public health indicators that could be measured through the national information network that was envisioned in the proposed Health Security Act.
By the committee's first meeting, comprehensive federal legislation was no longer expected and attention had shifted to opportunities for collaborative public-private activities at state and local levels. This change in the national policy environment resulted in further discussion with the study's sponsors to reframe the committee's task. After the committee's second meeting, a "vision statement" and work plan reflecting this modified context were developed in consultation with the sponsors. The vision statement appears in Appendix C of this report along with the summary of the committee's first workshop.
The revised task called for the committee to examine how a performance monitoring system could be used to improve the public's health by identifying the range of actors that can affect community health, monitoring the extent to which their actions make a constructive contribution to the health of the community, and promoting policy development and collaboration between public and private sector entities. The committee was also asked to develop prototypical sets of indicators for specific public health concerns that communities could use to monitor the performance of public health agencies, personal health care organizations, and other entities with a stake in community health.
The committee appointed to conduct the study brought together expertise in state and local health departments, epidemiology, public health indicators, health data, environmental health, adult and pediatric clinical medicine, managed care, community health and consumer interests, quality assessment, health services research, and employer concerns. The group met six times between February 1995 and April 1996. Workshops held in conjunction with our meetings in May and December 1995 gave us the opportunity to hear about a variety of community experiences and to learn more about work on performance monitoring being done by academic researchers and public and private organizations. Summaries of these workshops appear as Appendixes C and D of this report and also are posted on the World Wide Web (http://www. nap.edu/readingroom/).
The committee reviewed critical issues in using performance monitoring and the role it can play in community-based health improvement efforts. Our work pointed to the need for a broad view of the determinants of health and of the stakeholders that share responsibility for maintaining and enhancing health in a community. In this report, we propose an iterative and evolving community process for health improvement efforts in which performance monitoring is a critical tool for establishing meaningful stakeholder accountability. We also propose a set of indicators as the basis of a community profile that can provide background information needed to understand a community's health issues and can help communities identify specific issues that they might want to address. In addition, the committee developed prototypes of sets of performance indicators for some of those specific health issues (see Appendix A). The committee's work in developing these indicator sets illustrates how communities might apply the approach described in our report.
In the course of the committee's work a shared awareness evolved of the ways in which the public health and health care systems contribute to a community's well-being. Beyond the usual tasks of IOM committees-always complicated by subject complexity, relevance of multiple legitimate perspectives, and the need to forge multidisciplinary consensus-the committee's work required bridging what Kerr White has called the "schism" between the public health and personal care systems.1 Furthermore, we also needed to bring together three conceptual domains that have arisen separately-determinants of health, continuous improvement, and social activism. Finally, if these circumstances were not sufficiently daunting, a conceptual process that we entered into required major envisioning of systems not yet established, partnerships not yet forged, and the way in which individuals in organizations from different social sectors might choose to work together both for the common good and out of enlightened self-interest.
Our committee's principal "product" was a community health improvement process (CHIP), a method by which, on a community-wide basis, the health of the population might be improved. However complex this process of assessment, analysis, strategy formation, evaluation, and reassessment might be, we heard in our workshops individual presentations on programs and activities that seemed to us to represent the major features of our conceptual scheme at work in communities today. These current activities were never as holistically conceived, adequately resourced, thoroughly documented, and effective as our idealized vision of a possible future. They nevertheless represented steps toward a system of community-level effort that we believe will be necessary if the health of our community populations is ever to be truly maximized within available resources. Seeing and hearing about actual community cases in the present day encouraged us to think that the larger, more systemic achievement of a community health improvement process might yet be within our grasp.
For too long, the personal health care and public health systems have shouldered their respective roles and responsibilities for curing and preventing separately from each other, and often from the rest of the community as well. However, working alone and independently, our formal health systems cannot substantially improve population health at the level of fundamental determinants. The burden on these systems and the lost opportunities in our society from this fragmentation, segmentation, and isolation are evident in the resources consumed in repeatedly responding to the health consequences of persistent problems that can be traced to a variety of factors.
Instead, we need to invest in a process that mobilizes expertise and strategic action from a variety of community, state, and organizational entities if we are to substantially improve community and population health. The committee's experience over the course of this study suggests that developing a strategy for performance monitoring for health improvement at a community level constitutes a lens through which all potential contributors to community health become visible, their legitimate domain for action can be examined, and a virtually unlimited array of specifiable indicators of performance can be considered. In a complex, cross sectorial collaborative strategy, indicators for successful contributions to the overall strategy can help assure all parties that the effort each is making is having its intended effects. The challenge to communities will be to choose such measures wisely, using a method of choice-making that the committee hopes we have made explicit in this report.
No complete working model of the committee's vision will emerge quickly or easily. In particular, the emergence of partnerships to improve the health of communities, when that process entails the assumption of real accountability for measured performance, is likely to proceed slowly at first. However, the committee looks forward to seeing its proposed CHIP translated into practical applications, tested in a variety of community contexts, and improved. This will require a blend of imagination and creativity that will challenge, and we hope energize, all involved.
In closing, we note that this committee's work complements that of several other current or recently completed studies at the IOM and the National Research Council. A particularly closely related study, being conducted by the National Research Council Panel on Performance Measures and Data for Public Health Performance Partnership Grants, is examining technical issues involved in establishing state-level performance measures for federal grants in eight substantive areas. The panels first report, Assessment of Performance Measures in Public Health, which was released for comment in draft form in September 1996, is scheduled for completion in early 1997. A second report will address data and data system development needs.
Three related IOM reports were released in November 1996. Healthy Communities: New Partnerships for the Future of Public Health, from the Committee on Public Health, examines the evolving role of public health agencies, particularly in relation to community-focused activities and the growing prominence of managed care. The Hidden Epidemic: Confronting Sexually Transmitted Diseases, from the Committee on Prevention and Control of Sexually Transmitted Diseases, focuses on a specific health issue for which community-level efforts are recommended along with broader state and national strategies. Managing Managed Care: Quality Improvement in Behavioral Health, the report of the Committee on Quality Assurance and Accreditation Guidelines for Managed Behavioral Health Care, presents a framework for accreditation standards and quality improvements for managed behavioral health care and for developing, using, and evaluating performance indicators. We also note that our study is one of several that are part of the IOM Special Initiative on Health Care Quality, a three-year effort with goals that include evaluating and promoting appropriate use of tools for quality assessment and improvement.
We want to express our appreciation to the many people- listed by name in the Acknowledgments-who aided the committee in its work. As co-chairs of this difficult but rewarding study, we also want to commend the members of the committee for their thoughtful and insightful approach to the task put before them. Finally, on behalf of the entire committee, we want to thank the members of the IOM staff whose efforts successfully translated the committee's work into this report. Susan Thaul and Sarah Reich guided us through the initial meetings and workshop. Linda Bailey, Jane Durch, and Stephanie Smith, who joined the study staff in the midst of this process, saw us through additional meetings and another workshop as well as writing the report. Michael Stoto has been a valued contributor throughout the project.
Bobbie A. Berkowitz
Thomas S. Inui
Co-Chairs
The Committee on Using Performance Monitoring to Improve Community Health and the study staff are grateful for the generous assistance received from many individuals and organizations over the course of the study.
We particularly want to thank the speakers (listed here) and other participants (listed in Appendixes C and D) at the committee's two workshops. The speakers at the May 1995 workshop were Bill Beery, Group Health Cooperative of Puget Sound; Linda Demlo, Agency for Health Care Policy and Research; Richard Garfield, Columbia University School of Nursing; Randolph Gordon, Virginia Department of Health (Centers for Disease Control and Prevention at the time of the workshop); Claude Hall, Jr., American Public Health Association; James Krieger, Seattle-King County Department of Health; Roz Lasker, New York Academy of Medicine (Office of the Assistant Secretary for Health, Department of Health and Human Services, at the time of the workshop); Carl Osaki, Seattle-King County Department of Health; Nancy Rawding, National Association of County and City Health Officials; Cary Sennett, National Committee for Quality Assurance; Bernard Turnock, University of Illinois at Chicago; Margaret VanAmringe, Joint Commission on Accreditation of Healthcare Organizations; Elizabeth Ward, Washington State Department of Health; and Ronald Wilson, National Center for Health Statistics.
The speakers at the December 1995 workshop were J. Maichle Bacon, McHenry County (Illinois) Department of Health; Laurie L. Carmody, Group Health Association of America; Ann Casebeer, University of Calgary; Jonathan E. Fielding, University of California at Los Angeles School of Public Health; Dennis J. Kelso, Escondido (California) Health Care and Community Services Project; Bonnie Rencher, Calhoun County (Michigan) Health Improvement Program; Tony Traino, consultant, (Visiting Nurse Association of Greater Salem [Massachusetts] at the time of the workshop); and Edward H. Wagner, Group Health Cooperative of Puget Sound. The summary of this workshop was drafted by Ellen Weissman, Johns Hopkins School of Hygiene and Public Health.
The committee also wants to thank the individuals who reviewed and commented on initial drafts of the performance indicator sets that appear in Appendix A. These reviewers are Peter Briss, Centers for Disease Control and Prevention; Tim Byers, University of Colorado Health Sciences Center; Joseph Cassells; Gary Chase, Georgetown University Medical Center; Graham Colditz, Harvard Medical School; Margo Edmunds, Institute of Medicine; Steven Epstein, Georgetown University Medical Center; Amy Fine, Association of Maternal and Child Health Programs; Bernard Guyer, Johns Hopkins School of Hygiene and Public Health; Marie McCormick, Harvard School of Public Health; Paul Melinkovich, Denver Department of Community Health Services; Ricardo Muñoz, University of California at San Francisco; John Pinney, Pinney Associates; Lance Rodewald, Centers for Disease Control and Prevention; Harold Sox, Dartmouth-Hitchcock Medical Center; Robert Wallace, University of Iowa; and Kenneth Warner, University of Michigan School of Public Health. Reviewers from state health departments included Alan Weil, Colorado Department of Health Care Policy and Financing; Clinton C. Mudgett and Stephen E. Saunders, Illinois Department of Health; Bruce Cohen, Daniel Friedman, and Mary Ostrem, Massachusetts Department of Public Health; Sherri Homan, Bert Malone, and Marianne Ronan, Missouri Department of Health; Mimi Fields and Dan Rubin, Washington State Department of Health; and Richard Aronson and Katherine Kvale, Wisconsin Office of Maternal and Child Health.
Others whose assistance we would like to acknowledge are Richard Bogue, Hospital Research and Educational Trust, American Hospital Association; Erin Kenney, consultant (San Diego, California); Anne Klink, California Smoke-Free Cities; David Lansky, Foundation for Accountability; James McGee, Pennsylvania Health Care Cost Containment Council; Nancy Rigotti, Massachusetts General Hospital; Julie Trocchio, Catholic Health Association; Joan Twiss, California Healthy Cities Project; and Abraham Wandersman, University of South Carolina. The committee also expresses its appreciation to the National Research Council Panel on Performance Measures and Data for Public Health Performance Partnership Grants and to Jeffrey Koshel, the panels study director, for sharing materials and for allowing members of the committee staff to listen to some of their discussions.
The study was undertaken with funding from The Robert Wood Johnson Foundation and the U.S. Department of Health and Human Services. We appreciate the support of these organizations and the assistance provided by project officers Nancy Kaufman at The Robert Wood Johnson Foundation, Susanne Stoiber and James Scanlon in the Office of the Assistant Secretary for Planning and Evaluation at the U.S. Department of Health and Human Services, and Roz Lasker who served as project officer in the Office of the Assistant Secretary for Health until her move to the New York Academy of Medicine. In addition, we are grateful for additional funding received from the Kellogg Endowment Fund of the National Academy of Sciences and the Institute of Medicine.
Several members of the Institute of Medicine and National Academy of Sciences staff in addition to those listed with the committee made important contributions to the successful completion of this project: Mona Brinegar, Claudia Carl, Michael Edington, Sharon Galloway, Linda Kilroy, Dorothy Majewski, Amy O'Hara, Dan Quinn, Donna Thompson, and the staff of the National Academy Press. In addition, Florence Poillon provided copy editing for the report. We thank them for their assistance.
1 INTRODUCTION
4 A COMMUNITY HEALTH IMPROVEMENT PROCESS
APPENDIXES
A Prototype Performance Indicator Sets
ACRONYMS
INDEX
In communities, health is a product of many factors, and many segments of the community can contribute to and share responsibility for its protection and improvement. Changes in public policy, in public- and private-sector roles in health and health care, and in public expectations are presenting both opportunities and challenges for communities addressing health issues. Performance monitoring offers a tool to assess activities in the many sectors that can influence health and to promote both collaboration and accountability in working toward better health for the whole community, especially within the framework of a community-based health improvement process. This report from the Institute of Medicine (IOM) Committee on Using Performance Monitoring to Improve Community Health draws on lessons from a variety of current activities to outline the elements of a community health improvement process, discuss the role that performance monitoring can play in this process, and propose tools to help communities develop performance indicators.
The report reflects three important developments: (1) a broadening of our understanding of the nature of health and its determinants, (2) a greater appreciation of the importance of a community perspective, and (3) a growing interest in the use of performance measurement to improve the quality of health and other services in public and private settings
For both individuals and populations, health depends not only on medical care but also on other factors including individual behavior and genetic makeup and social and economic conditions for individuals and communities. The health field model, as described by Evans and Stoddart (1994) and discussed further in Chapter 2, presents these multiple determinants of health in a dynamic relationship (see Figure 1). The model's feedback loops link social environment, physical environment, genetic endowment, an individual's behavioral and biologic responses, disease, health care, health and function, well-being, and prosperity. This multidimensional perspective reinforces the value of public health's traditional emphasis on a population-based approach to community health issues.
As communities try to address their health issues in a comprehensive manner, all of the stakeholders will need to sort out their roles and responsibilities, which will vary from community to community. These interdependent sectors must address issues of shared responsibility for various aspects of community health and individual accountability for their actions. They also must participate in a process of community-wide social change that is necessary for health improvement efforts and related performance monitoring to succeed (Green and Kreuter, 1990). Most communities will have only limited experience with collaborative or coordinated efforts among these diverse groups. Effective collaboration will require a common language, an understanding of the multidimensional nature of the determinants of health, and a way to accommodate diversity in values and goals.
Performance monitoring has gained increasing attention as a tool for evaluating the delivery of personal health care services and for examining population-based activities addressing the health of the public (see Chapter 4 and Appendixes C and D). Although many performance monitoring activities are focused on specific health care organizations, only at the population level is it possible to examine the effectiveness of health promotion and disease prevention activities and to determine whether the needs of all segments of the community are being addressed.
As used by the committee, the term "performance monitoring" applies to a continuing community-based process of selecting indicators that can be used to measure the process and outcomes of an intervention strategy for health improvement, collecting and analyzing data on those indicators, and making the results available to the community to inform assessments of the effectiveness of an intervention and the contributions of accountable entities. Performance monitoring should promote health in a context of shared responsibility and individual accountability for achieving desired outcomes.
The monitoring process will depend on a limited number of indicators that can track critical processes and outcomes. A variety of tools are available for public health assessment. Some set, or provide a mechanism for setting, measurable health objectives and thus have some characteristics of performance measures (e.g., see APHA et al., 1991; NACHO, 1991; USDHHS, 1991). They are not, however, explicitly linked to the performance of specific entities in the community. To address this concern, the committee looked to evolving concepts of performance monitoring from the health services sector (e.g., NCQA, 1993); continuous quality improvement, particularly its application at the community level (e.g., Nolan and Knapp, 1996; Zablocki, 1996); and government reform (e.g., Osborne and Gaebler, 1992).
Based on its review of the determinants of health, the community-level forces that can influence them, and community experience with performance monitoring, the committee finds that a community health improvement process (CHIP) that includes performance monitoring, as outlined in this report, can be an effective tool for developing a shared vision and supporting a planned and integrated approach to improve community health. It offers a way for a community to address a collective responsibility and marshal resources of specific, accountable entities to improve the health of its members. The committee concluded, however, that individual communities will have to determine the specific allocation of responsibility and accountability. No universal approach can be prescribed. The committee's recommendations for operationalizing a CHIP are based on a variety of theoretical and practical models for community health improvement, continuous quality improvement, quality assurance, and performance monitoring in health care, public health, and other settings. However, the specifics of the committee's proposal have never been tested, in toto, in community settings. Therefore, attention is also given in this report to ways in which the proposed process can be evaluated.
The committee suggests that a CHIP should include two principal interacting cycles based on analysis, action, and measurement (see Figure 2). This process is described in more detail in Chapter 4. The problem identification and prioritization cycle focuses on identification and prioritization of health problems in the community, and the analysis and implementation cycle on a series of processes intended to devise, implement, and evaluate the impact of health improvement strategies to address the problems. The overall process differs from standard models primarily be cause of its emphasis on measurement to link performance and accountability on a community-wide basis.
The health assessment activities that are part of the problem identification and prioritization cycle should include production of a community health profile that can provide basic information to a community about its demographic and socioeconomic characteristics and its health status and health risks. This profile would provide background information that can help a community interpret other health data and identify issues that need more focused attention. The committee's proposed indicators for a community health profile are listed in Table 1.
Analyze the Health Issue
A community, through its health coalition or a designated agent such as the health department, must analyze the health issue to understand the contributing factors and how they operate in the community. A framework such as the field model should be used to ensure consideration not only of behavioral risks and health care issues but also of factors in the social and physical environments.
Inventory Health Resources
A community must assess the resources available for health improvement efforts. Relevant resources include those that can be applied to required tasks (e.g., organizations, influence, expertise, funding); protective factors within the community that can mitigate the impact of adverse conditions; and support available from public- and private-sector sources outside the community (e.g., funding, technical assistance).
Develop a Health Improvement Strategy
Health improvement strategies should seek to apply available resources as effectively as possible, given a community's specific features. Priority should be given to actions for which evidence of effectiveness is available and for which costs are considered appropriate in relation to expected health benefits. For many health issues, however, evidence for effective interventions will be limited. A community should not ignore those issues but will have to consider carefully what actions will make the best use of its resources. Communities should also consider the implications of not acting on a health issue.
Establish Accountability for Activities
Establishing accountability through a collaborative approach is a key to using performance monitoring in the health improvement process proposed by the committee. Specific entities must be willing to be accountable to the community for undertaking activities that are expected to contribute to achieving desired health outcomes. The committee sees a collective responsibility among all segments of a community to contribute to health improvements, but each entity must accept individual responsibility for performing those tasks that are consistent with its capabilities.Develop a Set of Performance Indicators
Performance indicators are needed to help community stake holders monitor whether the health improvement strategy is being implemented as intended and whether it is having the intended impact. These quantitative measures must apply to specific entities in the community that have accepted responsibility for some aspect of the health improvement effort. Because health issues have many dimensions and can be addressed by various sectors in the community, sets of indicators will be needed to assess performance.
Implement the Improvement Strategy
Implementation of health improvement strategies and interventions requires action by many segments of a community. The particular mix of activities and participants will depend on the health issue being addressed and on a community's organization and resources. In most instances, these activities will require the involvement of both public- and private-sector entities and often of entities that may not traditionally be seen as part of the health system.Monitor Process and Outcomes
Once a health improvement program is under way, performance monitoring becomes an essential guide. Information provided by the selected performance indicators should be reviewed regularly and used to inform further action. In assessing progress, a community coalition or other designated agent should consider whether accountable entities are taking appropriate actions and whether appropriate strategies and interventions have been adopted. The quantitative data provided by performance indicators should be interpreted in combination with qualitative information from the community. As current goals are achieved and new ones adopted, the analysis and implementation cycle of a CHIP should support initiation of new activities and selection of new indicators. Over time, a community, through its health coalition and the broader aspects of a CHIP, should reexamine its priorities and health improvement portfolio, adding new issues as progress is made on others.
In developing a health improvement program, every community must consider its particular circumstances (e.g., health concerns, resources, social and political perspectives). The committee cannot prescribe what actions individual communities should take to address their health concerns or who should be responsible for what, but it does believe that communities need to ad dress these issues and that an organized approach to health improvement that makes use of performance monitoring tools will help them achieve their goals.
Given the different perspectives and activities of personal health service, public health, and other organizations that can contribute to the health of communities and given differing views of the meaning of "health" in the community context, the committee recommends that
To operationalize the concept of shared responsibility and individual accountability for community health, stakeholders need to know, jointly and as clearly as possible, how the actions of each potentially accountable entity can contribute to the community's health. Thus, the committee recommends that
Selecting these indicators will require careful consideration of how to gain insight into progress achieved in the health improvement process. A set of indicators should balance population based measures of risk factors and health outcomes and health systems-based measures of services performed. To encourage full participation in the health improvement process, the selected performance measures should also be balanced across the interests and contributions of the various accountable entities in the community, including those whose primary mission is not health specific. Selection of performance indicators is discussed in Chapter 5, and prototype indicator sets for several health issues are presented in Appendix A. One example, for vaccine-preventable diseases, is shown in Table 2.
Because stakeholder-level performance measures will generally be unique to a particular community and to the circumstances of stakeholders in that community, the committee focused on developing community-level performance indicators. Such performance measures would permit communities and their health coalitions to ask, "How are we, as a community, performing in assuring the health of our citizenry?" The prototype indicators include measures for specific sectors in the community (e.g., managed care organizations, schools, employers, public health agencies), but a community may want measures for individual entities within those sectors.
Communities will need criteria to guide the selection of indicators. Criteria proposed by the committee include consistency with a conceptual framework (such as the field model) for understanding factors that contribute to the production of health, salience to community stakeholders, and support for the social change processes needed to achieve health improvements. Other proposed criteria are validity and reliability, availability of evidence linking performance and health improvement, sensitivity to changes in community health status, and availability of timely data at a reasonable cost. An operational definition should be developed for each measure to determine what data are needed and how (or if) they can be obtained. A review of existing indicator sets may suggest measures that could be adapted for community use and may be a source of tested operational definitions.
Many of the important underlying influences on health that the field model helps identify are often not amenable to change in the short run. For example, interventions aimed at critical developmental periods, such as educational programs in early childhood, may have long-term health benefits but produce little measurable effect in the near term. A desire to make observable progress could lead a CHIP to focus on other more immediately measurable problems or problems that may be high on the political agenda but of uncertain importance to the community's overall health (e.g., a new renal dialysis unit). A CHIP must also guard against becoming paralyzed by focusing on the undoable. To maintain momentum for community health coalitions, it may be reasonable to select some problems that are amenable to change and success in the short term. Thus, the committee recommends that
The proposed health improvement process and performance monitoring activities will require that communities have a sustainable system that provides for participation by major stakeholders and accountable entities. Thus, the committee recommends that
Participants should assume responsibility for contributing to the health of the community, not just furthering the goals of the organizations they represent.
As described in Chapters 3 and 4, a CHIP focuses on horizontal peer relationships in a community rather than vertical hierarchical relationships. Experience suggests that performance monitoring used as a basis for inspection and discipline of those not producing as expected is less effective in achieving improvements than is monitoring used as a tool for learning and process change (Berwick, 1989; Osborne and Gaebler, 1992). Rather, a CHIP should use performance monitoring to encourage productive action and collaboration from many sectors. Because the proposed community health improvement process is new, groups that carry it out should be "learning organizations" in the sense that the people, agencies, and community involved are organized to learn from their own experience and improve their operations.
All community initiatives require leadership, which may come from the public or the private sector. To institutionalize the health improvement process as a multiparty effort, the committee recommends that
Some communities will have appropriate coalitions in place, but others will have to expand existing groups or establish a workable forum for collective action for the first time. Strategies for improving the effectiveness of community coalitions for health improvement are discussed in Chapter 3.
For the CHIP to be effective, communities need data for community health profiles and performance measures. Since all parties share in the goal of improving community health, it is reasonable to combine public and private resources to support the data collection and analysis needed for communities to obtain health profile information, to conduct health status assessments and communicate results, and to sustain performance monitoring programs. Such resources could include funding, personnel, data, data processing, and analysis.
Both public and private sectors can contribute critical data for performance monitoring. Public health agencies, as part of the public health assessment function called for in The Future of Public Health, should promote, facilitate, and-where necessary and appropriate-perform community health assessments and monitor changes in key performance measures. Much of the necessary data and expertise exist at the state health department. Thus, the committee recommends that
Currently, most of these data are aggregated by standard geopolitical units such as counties and municipalities. The committee encourages making community health data available in a form that allows communities to prepare health profiles and performance measures according to their own definitions of "community" (e.g., geographic, socioeconomic, cultural). Geocoding of health-related data gathered for other purposes would be an important step toward improving the data for performance monitoring. For data available only at the community level, state health departments should provide models and technical assistance that communities can use in their own data collection activities.
Because data on and from all accountable entities are essential for effective performance monitoring, states and the federal government (in their policy development and regulatory roles) can assist communities by facilitating access to relevant data held by the private sector. In particular, the committee recommends that
Providing these data should be seen as part of the responsibility that these private-sector organizations have to the community (IOM, 1996; Showstack et al., 1996). Adequate safeguards for privacy and confidentiality must be provided for all CHIP data (IOM, 1994).
The relationship between the CHIP and public or private health service and other community organizations should be reciprocal. In addition to data that these organizations can provide to a CHIP, the organizations can use the other community data that are gathered, and this in turn should reinforce CHIP goals. For instance, state agencies designing publicly funded health services programs such as Medicaid managed care can specify the performance measures to be used in evaluating the contractors and the data that contractors must report. Alternatively, private health service organizations could use CHIP data to assess their contributions to the community's health under "community benefit" guidelines and regulations or in their own service planning and resource allocation decisions.
The assessment process should strive to include sites that vary both in the nature of the community and in the structures and processes used for performance monitoring. The assessment should also include estimates of the full range of public and private costs of carrying out the CHIP and should explore ways to achieve efficiencies in these efforts. These "natural experiments" should be studied to learn how local circumstances affect the way the CHIP is adapted by different communities; to identify the "necessary and desirable conditions" for implementation of the CHIP; and to assess whether or not the CHIP indeed results in a refocusing of attention on root causes of health problems and, ultimately, in important improvements in community health.
The current evaluations of a variety of community health interventions (e.g., Wagner et al., 1991; Elder et al., 1993; Wickizer et al., 1993; COMMIT, 1995a,b; Fortmann et al., 1995; Murray, 1995) can be expected to inform the development of specific interventions to address health problems, the community intervention process itself and analytic techniques to apply to community studies. The recently established Task Force on Community Preventive Services, organized by the Centers for Disease Control and Prevention, will compile evidence on a variety of community-level activities. The CHIP in its entirety can also be thought of as a "comprehensive community initiative," and ideas regarding the evaluation of such initiatives can be applied (see Connell et al., 1995).
For the community health improvement process to be effective, appropriate performance measurement tools must be developed further. Thus, the committee recommends that
These standard measures would be a resource available to communities, not a set of prescribed measures. The prototype indicator sets described in Appendix A of this report should be viewed as a starting point. Particular attention should be given to issues for which valid measures are not currently available, but the refinement of existing measures should also be addressed. The development of measures of "quality of life" and consumer satisfaction for use in community surveys is particularly important. Research to develop and improve techniques of measurement and analysis (e.g., small area analysis) that can be applied to community-level performance monitoring should be supported as well.
More generally, technical expertise based on experience with the community health improvement process must be developed and shared. Thus, the committee recommends that
These programs should introduce the concept of CHIP as a way of thinking about the application of a group of academic disciplines (epidemiology, biostatistics, environmental health, health behavior, and so on) to the practice of community health improvement. Among the other fields in which CHIP might be addressed are maternal and child health, behavioral sciences, and mental health and substance abuse counseling and program administration.
Berwick, D.M. 1989. Continuous Improvement as an Ideal in Health Care. New England Journal of Medicine 320:53-56.
COMMIT (Community Intervention Trial for Smoking Cessation). 1995a. I. Cohort Results from a Four-Year Community Intervention. American Journal of Public Health 85:183-192.
COMMIT. 1995b. II. Changes in Adult Cigarette Smoking Prevalence. American Journal of Public Health 85:193-200.
Connell, J.P., Kubisch, A.C., Schorr, L.B., and Weiss, C.H., eds. 1995. New Approaches to Evaluating Community Initiatives: Concepts, Methods, and Contexts. Washington, D.C.: Aspen Institute.
Elder, J.P., Schmid, T.L, Dower, P., and Hedlund, S. 1993. Community Heart Health Programs: Components, Rationale, and Strategies for Effective Interventions. Journal of Public Health Policy 14:463-479.
Evans, R.G., and Stoddart, G.L 1994. Producing Health, Consuming Health Care. In Why Are Some People Healthy and Others Not? The Determinants of Health of Populations. R.G. Evans, M.L. Barer, and T.R. Marmor, eds. New York: Aldine De Gruyter.
Fortmann, S.P., Flora, J.A., Winkleby, M.A., Schooler, C., Taylor, C.B., and Farquhar, J.W. 1995. Community Intervention Trials: Reflections on the Stanford Five-City Project Experience. American Journal of Epidemiology 142:576-586.
Green, L. W., and Kreuter, M. W. 1990. Health Promotion as a Public Health Strategy for the 1990s. Annual Review of Public Health 11:319-334.
IOM (Institute of Medicine). 1988. The Future of Public Health. Washington, D.C.: National Academy Press.
IOM. 1994. Health Data in the Information Age: Use, Disclosure, and Privacy. M.S. Donaldson and K.N. Lohr, eds. Washington, D.C.: National Academy Press.
IOM. 1996. Healthy Communities: New Partnerships for the Future of Public Health. M.A. Stoto, C. Abel, and A. Dievler, eds. Washington, D.C.: National Academy Press.
Murray, D. 1995. Design and Analysis of Community Trials: Lessons from the Minnesota Heart Health Program. American Journal of Epidemiology 142:569-575.
NACHO (National Association of County Health Officials). 1991. APEXPH: Assessment Protocol for Excellence in Public Health. Washington, D.C.: NACHO.
NCQA (National Committee for Quality Assurance). 1993. Health Plan Employer Data and Information Set and User's Manual, Version 2.0 (HEDIS 2.0). Washington, D.C.: NCQA.
Nolan, T.W., and Knapp, M. 1996. Community-wide Health Improvement: Lessons from the IHI-GOAL/QPC Learning Cooperative. The Quality Letter for Healthcare Leaders 8(1): 13-20.
Osborne, D., and Gaebler, T. 1992. Reinventing Government: How the Entrepreneurial Spirit Is Transforming the Public Sector. Reading, Mass.: Addison Wesley.
Showstack, J., Lurie, N., Leatherman, S., Fisher, E., and Inui, T. 1996. Health of the Public: The Private Sector Challenge. Journal of the American Medical Association 276:1071-1074.
USDHHS (U.S. Department of Health and Human Services). 1991. Healthy People 2000: National Health Promotion and Disease Prevention Objectives. DHHS Pub. No. (PHS) 91-50212. Washington, D.C.: Office of the Assistant Secretary for Health.
Wagner, E.H., Koepsell, T.D., Anderman, C., et al. 1991. The Evaluation of the Henry J. Kaiser Family Foundation's Community Health Promotion Program: Design. Journal of Clinical Epidemiology 44:685-699.
Wickizer, T.M., Von Korff, M., Cheadle, A., et al. 1993. Activating Communities for Health Promotion: A Process Evaluation Method. American Journal of Public Health 83:561-567.
Zablocki, E. 1996. Improving Community Health Status: Strategies for Success. The Quality Letter for Healthcare Leaders 8(1):2-12.