NATIONAL ACADEMY PRESS
2101 Constitution Avenue, N.W. Washington, D.C. 20418
NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance.
This report has been reviewed by a group other than the authors according to procedures approved by a Report Review Committee consisting of members of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine.
The Institute of Medicine was chartered in 1970 by the National Academy of Sciences to enlist distinguished members of the appropriate professions in the examination of policy matters pertaining to the health of the public. In this, the Institute acts under the Academy's 1863 congressional charter responsibility to be an adviser to the federal government and its own initiative in identifying issues of medical care, research, and education. Dr. Kenneth I. Shine is president of the Institute of Medicine.
Funding for this project was provided by the Office of the Assistant Secretary for Health, U.S. Department of Health and Human Services (contract no. 282-94-0032); The Robert Wood Johnson Foundation (grant no. 024336); and the Kellogg Endowment Fund of the National Academy of Sciences and the Institute of Medicine. The views presented in this report are those of the Committee on Using Performance Monitoring to Improve Community Health and are not necessarily those of the funding organizations.
Library of Congress Cataloging-in-Publication Data
Institute of Medicine (U.S.). Committee on Using Performance Monitoring to Improve Community Health.
Improving health in the community : a role for performance monitoring / Committee on Using Performance Monitoring to Improve Community Health ; Jane S. Durch, Linda A. Bailey, and Michael A. Stoto, editors.
Includes bibliographical references and index.
1. Community health services—United States—Evaluation. 2. Health status indicators—United States. 3. Health promotion— United States. I. Durch, Jane. II. Bailey, Linda A. III. Stoto, Michael A. IV. Title.
[DNLM: 1. Community Health Services—standards—United States. 2. Quality Assurance, Health Care—organization & administration— United States. 3. Community Health Planning—methods—United States. 4. Health Status Indicators—United States. WA 546 AA1 I59i 1997]
for Library of Congress 97-6336
First Printing, April 1997
Second Printing, January 1999
Additional copies of this report are available for sale from the
National Academy Press,
Box 285, 2101 Constitution Avenue, N.W., Washington, D.C. 20055. Call (800) 624-6242 or (202) 334-3313 (in the Washington metropolitan area), or visit the NAP online bookstore at http://www.nap.edu
Copyright 1997 by the National Academy of Sciences. All rights reserved.
Printed in the United States of America
The serpent has been a symbol of long life, healing, and knowledge among almost all cultures and religions since the beginning of recorded history. The image adopted as a logotype by the Institute of Medicine is based on a relief carving from ancient Greece, now held by the Staatlichemuseen in Berlin.
COMMITTEE ON USING PERFORMANCE MONITORING TO IMPROVE COMMUNITY HEALTH
BOBBIE A. BERKOWITZ* (Co-Chair), Deputy Secretary,
Washington State Department of Health, Olympia, Washington
THOMAS S. INUI† (Co-Chair), Professor and Chair,
Department of Ambulatory Care and Prevention, Harvard Medical School and Harvard Pilgrim Health Care, Boston, Massachusetts
ALAN W. CROSS (Vice Chair), Professor of Social Medicine and Pediatrics and Director,
Center for Health Promotion and Disease Prevention, University of North Carolina, Chapel Hill, North Carolina
LARRY W. CHAMBERS, Epidemiology Consultant,
Hamilton-Wentworth Regional Public Health Department, and
Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ontario, Canada
THOMAS W. CHAPMAN,‡ Chief Executive Officer,
George Washington University Hospital, and
Senior Vice President for Network Development,
George Washington University Medical Center, Washington, D.C.
ELLIOTT S. FISHER, Co-Director,
Veterans Affairs Outcomes Group, Veterans Affairs Medical Center, White River Junction, Vermont, and
Associate Professor of Medicine and Community and Family Medicine,
Dartmouth Medical School, Hanover, New Hampshire
JAMES L. GALE, Professor,
Department of Epidemiology, School of Public Health and Community Medicine, and
Northwest Center for Public Health Practice, University of Washington, Seattle; Health Officer, Kittitas County, Washington
KRISTINE GEBBIE† (Liaison, Board on Health Promotion and Disease Prevention), Assistant Professor of Nursing,
Columbia University School of Nursing, New York, New York
FERNANDO A. GUERRA, Director of Health,
San Antonio Metropolitan Health District, San Antonio, Texas
GARLAND H. LAND, Director,
Center for Health Information Management and Epidemiology, Missouri Department of Health, Jefferson City, Missouri
SHEILA LEATHERMAN, Executive Vice President,
United HealthCare Corporation, Minneapolis, Minnesota
JOHN R. LUMPKIN, Director,
Illinois Department of Public Health, Springfield, Illinois
WILLIAM J. MAYER, President and General Manager,
Functional Foods Division, Kellogg Company, Battle Creek, Michigan
ANA MARIA OSORIO, Chief,
Occupational Health Branch, California Department of Health Services, Berkeley, California
SHOSHANNA SOFAER, Associate Professor and Associate Chair for Research,
Department of Health Care Sciences, George Washington University Medical Center, Washington, D.C.
DEBORAH KLEIN WALKER, Assistant Commissioner,
Bureau of Family and Community Health, Massachusetts Department of Public Health, Boston, Massachusetts
JOHN E. WARE, Jr., ‡ Senior Scientist,
The Health Institute, New England Medical Center, Boston, Massachusetts
RICHARD A. WRIGHT, Director,
Community Health Services, Denver Department of Health and Hospitals, Denver, Colorado
Linda A. Bailey, Senior Program Officer (Co-Study Director)
Jane S. Durch, Program Officer (Co-Study Director)
Stephanie Y. Smith, Project Assistant
Michael A. Stoto, Director, Division of Health Promotion and Disease Prevention
Marissa W. Fuller, Research Associate
Sarah H. Reich, Project Assistant
Susan Thaul, Senior Program Officer
An interest in understanding how health care and public health activities might be coordinated and directed toward improving the health of entire communities was the basis for this study by the Institute of Medicine (IOM) Committee on Using Performance Monitoring to Improve Community Health, which we jointly chaired.
The IOM was asked by the U.S. Department of Health and Human Services and The Robert Wood Johnson Foundation to undertake a two-year study to examine the use of performance monitoring and develop sets of indicators that communities could use to promote the achievement of public health goals. The study was originally approved in mid-1994 when passage of federal health care reform legislation was anticipated. Part of the task outlined at that time was to identify public health indicators that could be measured through the national information network that was envisioned in the proposed Health Security Act.
By the committee's first meeting, comprehensive federal legislation was no longer expected and attention had shifted to opportunities for collaborative public-private activities at state and local levels. This change in the national policy environment resulted in further discussion with the study's sponsors to reframe the committee's task. After the committee's second meeting, a "vision statement" and work plan reflecting this modified context were developed in consultation with the sponsors. The vision state-
ment appears in Appendix C of this report along with the summary of the committee's first workshop.
The revised task called for the committee to examine how a performance monitoring system could be used to improve the public's health by identifying the range of actors that can affect community health, monitoring the extent to which their actions make a constructive contribution to the health of the community, and promoting policy development and collaboration between public and private sector entities. The committee was also asked to develop prototypical sets of indicators for specific public health concerns that communities could use to monitor the performance of public health agencies, personal health care organizations, and other entities with a stake in community health.
The committee appointed to conduct the study brought together expertise in state and local health departments, epidemiology, public health indicators, health data, environmental health, adult and pediatric clinical medicine, managed care, community health and consumer interests, quality assessment, health services research, and employer concerns. The group met six times between February 1995 and April 1996. Workshops held in conjunction with our meetings in May and December 1995 gave us the opportunity to hear about a variety of community experiences and to learn more about work on performance monitoring being done by academic researchers and public and private organizations. Summaries of these workshops appear as Appendixes C and D of this report and also are posted on the World Wide Web (http://www.nap.edu/readingroom/ ).
The committee reviewed critical issues in using performance monitoring and the role it can play in community-based health improvement efforts. Our work pointed to the need for a broad view of the determinants of health and of the stakeholders that share responsibility for maintaining and enhancing health in a community. In this report, we propose an iterative and evolving community process for health improvement efforts in which performance monitoring is a critical tool for establishing meaningful stakeholder accountability. We also propose a set of indicators as the basis of a community profile that can provide background information needed to understand a community's health issues and can help communities identify specific issues that they might want to address. In addition, the committee developed prototypes of sets of performance indicators for some of those specific health issues (see Appendix A). The committee's work in developing these
indicator sets illustrates how communities might apply the approach described in our report.
In the course of the committee's work a shared awareness evolved of the ways in which the public health and health care systems contribute to a community's well-being. Beyond the usual tasks of IOM committees—always complicated by subject complexity, relevance of multiple legitimate perspectives, and the need to forge multidisciplinary consensus—the committee's work required bridging what Kerr White has called the "schism" between the public health and personal care systems.1 Furthermore, we also needed to bring together three conceptual domains that have arisen separately—determinants of health, continuous improvement, and social activism. Finally, if these circumstances were not sufficiently daunting, a conceptual process that we entered into required major envisioning of systems not yet established, partnerships not yet forged, and the way in which individuals in organizations from different social sectors might choose to work together both for the common good and out of enlightened self-interest.
Our committee's principal "product" was a community health improvement process (CHIP), a method by which, on a community-wide basis, the health of the population might be improved. However complex this process of assessment, analysis, strategy formation, evaluation, and reassessment might be, we heard in our workshops individual presentations on programs and activities that seemed to us to represent the major features of our conceptual scheme at work in communities today. These current activities were never as holistically conceived, adequately resourced, thoroughly documented, and effective as our idealized vision of a possible future. They nevertheless represented steps toward a system of community-level effort that we believe will be necessary if the health of our community populations is ever to be truly maximized within available resources. Seeing and hearing about actual community cases in the present day encouraged us to think that the larger, more systemic achievement of a community health improvement process might yet be within our grasp.
For too long, the personal health care and public health systems have shouldered their respective roles and responsibilities for curing and preventing separately from each other, and often
from the rest of the community as well. However, working alone and independently, our formal health systems cannot substantially improve population health at the level of fundamental determinants. The burden on these systems and the lost opportunities in our society from this fragmentation, segmentation, and isolation are evident in the resources consumed in repeatedly responding to the health consequences of persistent problems that can be traced to a variety of factors.
Instead, we need to invest in a process that mobilizes expertise and strategic action from a variety of community, state, and organizational entities if we are to substantially improve community and population health. The committee's experience over the course of this study suggests that developing a strategy for performance monitoring for health improvement at a community level constitutes a lens through which all potential contributors to community health become visible, their legitimate domain for action can be examined, and a virtually unlimited array of specifiable indicators of performance can be considered. In a complex, cross-sectorial collaborative strategy, indicators for successful contributions to the overall strategy can help assure all parties that the effort each is making is having its intended effects. The challenge to communities will be to choose such measures wisely, using a method of choice-making that the committee hopes we have made explicit in this report.
No complete working model of the committee's vision will emerge quickly or easily. In particular, the emergence of partnerships to improve the health of communities, when that process entails the assumption of real accountability for measured performance, is likely to proceed slowly at first. However, the committee looks forward to seeing its proposed CHIP translated into practical applications, tested in a variety of community contexts, and improved. This will require a blend of imagination and creativity that will challenge, and we hope energize, all involved.
In closing, we note that this committee's work complements that of several other current or recently completed studies at the IOM and the National Research Council. A particularly closely related study, being conducted by the National Research Council's Panel on Performance Measures and Data for Public Health Performance Partnership Grants, is examining technical issues involved in establishing state-level performance measures for federal grants in eight substantive areas. The panel's first report, Assessment of Performance Measures in Public Health, which was released for comment in draft form in September 1996, is sched-
uled for completion in early 1997. A second report will address data and data system development needs.
Three related IOM reports were released in November 1996. Healthy Communities: New Partnerships for the Future of Public Health, from the Committee on Public Health, examines the evolving role of public health agencies, particularly in relation to community-focused activities and the growing prominence of managed care. The Hidden Epidemic: Confronting Sexually Transmitted Diseases, from the Committee on Prevention and Control of Sexually Transmitted Diseases, focuses on a specific health issue for which community-level efforts are recommended along with broader state and national strategies. Managing Managed Care: Quality Improvement in Behavioral Health, the report of the Committee on Quality Assurance and Accreditation Guidelines for Managed Behavioral Health Care, presents a framework for accreditation standards and quality improvements for managed behavioral health care and for developing, using, and evaluating performance indicators. We also note that our study is one of several that are part of the IOM Special Initiative on Health Care Quality, a three-year effort with goals that include evaluating and promoting appropriate use of tools for quality assessment and improvement.
We want to express our appreciation to the many people—listed by name in the Acknowledgments—who aided the committee in its work. As co-chairs of this difficult but rewarding study, we also want to commend the members of the committee for their thoughtful and insightful approach to the task put before them. Finally, on behalf of the entire committee, we want to thank the members of the IOM staff whose efforts successfully translated the committee's work into this report. Susan Thaul and Sarah Reich guided us through the initial meetings and workshop. Linda Bailey, Jane Durch, and Stephanie Smith, who joined the study staff in the midst of this process, saw us through additional meetings and another workshop as well as writing the report. Michael Stoto has been a valued contributor throughout the project.
Bobbie A. Berkowitz
Thomas S. Inui
The Committee on Using Performance Monitoring to Improve Community Health and the study staff are grateful for the generous assistance received from many individuals and organizations over the course of the study.
We particularly want to thank the speakers (listed here) and other participants (listed in Appendixes C and D) at the committee's two workshops. The speakers at the May 1995 workshop were Bill Beery, Group Health Cooperative of Puget Sound; Linda Demlo, Agency for Health Care Policy and Research; Richard Garfield, Columbia University School of Nursing; Randolph Gordon, Virginia Department of Health (Centers for Disease Control and Prevention at the time of the workshop); Claude Hall, Jr., American Public Health Association; James Krieger, Seattle-King County Department of Health; Roz Lasker, New York Academy of Medicine (Office of the Assistant Secretary for Health, Department of Health and Human Services, at the time of the workshop); Carl Osaki, Seattle-King County Department of Health; Nancy Rawding, National Association of County and City Health Officials; Cary Sennett, National Committee for Quality Assurance; Bernard Turnock, University of Illinois at Chicago; Margaret VanAmringe, Joint Commission on Accreditation of Healthcare Organizations; Elizabeth Ward, Washington State Department of Health; and Ronald Wilson, National Center for Health Statistics.
The speakers at the December 1995 workshop were J. Maichle Bacon, McHenry County (Illinois) Department of Health; Laurie L. Carmody, Group Health Association of America; Ann Casebeer, University of Calgary; Jonathan E. Fielding, University of California at Los Angeles School of Public Health; Dennis J. Kelso, Escondido (California) Health Care and Community Services Project; Bonnie Rencher, Calhoun County (Michigan) Health Improvement Program; Tony Traino, consultant, (Visiting Nurse Association of Greater Salem [Massachusetts] at the time of the workshop); and Edward H. Wagner, Group Health Cooperative of Puget Sound. The summary of this workshop was drafted by Ellen Weissman, Johns Hopkins School of Hygiene and Public Health.
The committee also wants to thank the individuals who reviewed and commented on initial drafts of the performance indicator sets that appear in Appendix A. These reviewers are Peter Briss, Centers for Disease Control and Prevention; Tim Byers, University of Colorado Health Sciences Center; Joseph Cassells; Gary Chase, Georgetown University Medical Center; Graham Colditz, Harvard Medical School; Margo Edmunds, Institute of Medicine; Steven Epstein, Georgetown University Medical Center; Amy Fine, Association of Maternal and Child Health Programs; Bernard Guyer, Johns Hopkins School of Hygiene and Public Health; Marie McCormick, Harward School of Public Health; Paul Melinkovich, Denver Department of Community Health Services; Ricardo Muñoz, University of California at San Francisco; John Pinney, Pinney Associates; Lance Rodewald, Centers for Disease Control and Prevention; Harold Sox, Dartmouth-Hitchcock Medical Center; Robert Wallace, University of Iowa; and Kenneth Warner, University of Michigan School of Public Health. Reviewers from state health departments included Alan Weil, Colorado Department of Health Care Policy and Financing; Clinton C. Mudgett and Stephen E. Saunders, Illinois Department of Health; Bruce Cohen, Daniel Friedman, and Mary Ostrem, Massachusetts Department of Public Health; Sherri Homan, Bert Malone, and Marianne Ronan, Missouri Department of Health; Mini Fields and Dan Rubin, Washington State Department of Health; and Richard Aronson and Katherine Kvale, Wisconsin Office of Maternal and Child Health.
Others whose assistance we would like to acknowledge are Richard Bogue, Hospital Research and Educational Trust, American Hospital Association; Erin Kenney, consultant (San Diego, California); Anne Klink, California Smoke-Free Cities; David Lansky, Foundation for Accountability; James McGee, Pennsylva-
nia Health Care Cost Containment Council; Nancy Rigotti, Massachusetts General Hospital; Julie Trocchio, Catholic Health Association; Joan Twiss, California Healthy Cities Project; and Abraham Wandersman, University of South Carolina. The committee also expresses its appreciation to the National Research Council Panel on Performance Measures and Data for Public Health Performance Partnership Grants and to Jeffrey Koshel, the panel's study director, for sharing materials and for allowing members of the committee staff to listen to some of their discussions.
The study was undertaken with funding from The Robert Wood Johnson Foundation and the U.S. Department of Health and Human Services. We appreciate the support of these organizations and the assistance provided by project officers Nancy Kaufman at The Robert Wood Johnson Foundation, Susanne Stoiber and James Scanlon in the Office of the Assistant Secretary for Planning and Evaluation at the U.S. Department of Health and Human Services, and Roz Lasker who served as project officer in the Office of the Assistant Secretary for Health until her move to the New York Academy of Medicine. In addition, we are grateful for additional funding received from the Kellogg Endowment Fund of the National Academy of Sciences and the Institute of Medicine.
Several members of the Institute of Medicine and National Academy of Sciences staff in addition to those listed with the committee made important contributions to the successful completion of this project: Mona Brinegar, Claudia Carl, Michael Edington, Sharon Galloway, Linda Kilroy, Dorothy Majewski, Amy O'Hara, Dan Quinn, Donna Thompson, and the staff of the National Academy Press. In addition, Florence Poillon provided copy editing for the report. We thank them for their assistance.