National Academies Press: OpenBook
Suggested Citation:"FRONT MATTER." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×

Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health

Edward B. Perrin and Jeffrey J. Koshel, Editors

Panel on Performance Measures and Data for Public Health Performance Partnership Grants

Committee on National Statistics

Commission on Behavioral and Social Sciences and Education

National Research Council

NATIONAL ACADEMY PRESS
Washington, D.C.
1997

Suggested Citation:"FRONT MATTER." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×

NATIONAL ACADEMY PRESS
2101 Constitution Avenue, N.W. Washington, D.C. 20418

NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance.

This report has been reviewed by a group other than the authors according to procedures approved by a Report Review Committee consisting of members of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine.

This study was supported by Contract No. 282-95-0034 between the National Academy of Sciences and the U.S. Department of Health and Human Services. Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the view of the organizations or agencies that provided support for this project.

Library of Congress Cataloging-in-Publication Data

Assessment of performance measures in public health, substance abuse, and mental health / Edward B. Perrin and Jeffrey J. Koshel, editors ; Panel on Performance Measures and Data for Public Health Performance Partnership Grants, Committee on National Statistics, Commission on Behavioral and Social Sciences and Education, National Research Council.

p. cm.

Includes bibliographical references (p.).

ISBN 0-309-05796-5 (pbk.)

1. Public health—Evaluation. 2. Public health—United States—Evaluation. I. Perrin, Edward. II. Koshel, Jeffrey. III. National Research Council (U.S.). Panel on Performance Measures and Data for Public Health Performance Partnership Grants.

RA427.A75 1997

362.1'0973—dc21

97-4915

CIP

Printed in the United States of America

Copyright 1997 by the National Academy of Sciences. All rights reserved.

Suggested Citation:"FRONT MATTER." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×

Panel on Performance Measures and Data for Public Health Performance Partnership Grants

EDWARD B. PERRIN (Chair),

Department of Health Services, School of Public Health and Community Medicine, University of Washington

JOHN C. BAILAR, III,

Department of Health Studies, University of Chicago

THOMAS J. BARRETT,

Colorado Mental Health Services, Denver, Colorado

DAVID W. FLEMING,

Center for Disease Prevention and Epidemiology, Oregon Health Division

V. JOSEPH HOTZ,

Irving B. Harris Graduate School of Public Policy Studies, University of Chicago

IRA R. KAUFMAN,

University of Medicine and Dentistry of New Jersey, Department of Environmental and Community Medicine, Robert Wood Johnson Medical School

JOHN LUMPKIN,

Illinois Department of Public Health

WILLIAM A. MORRILL,

Mathtech, Inc., Princeton, New Jersey

R. HEATHER PALMER,

Center for Quality of Care Research and Education, Harvard School of Public Health

J. SANFORD SCHWARTZ,

Leonard Davis Institute of Health Economics

MARY E. STUART,

Maryland Collaboratory for Business and Health, Department of Sociology and Anthropology, University of Maryland, Baltimore County

CYNTHIA P. TURNURE,

Chemical Dependency Program Division, Minnesota Department of Human Services

PAUL J. WIESNER,

DeKalb County Board of Health, Decatur, Georgia

GAIL R. WILENSKY,

Project Hope, Bethesda, Maryland

JEFFREY J. KOSHEL, Study Director

SUSAN M. SKILLMAN, Senior Research Associate

ASHLEY BOWERS, Research Assistant

ANU PEMMARAZU, Research Assistant

MICHELLE M. RUDDICK, Research Assistant

TELISSIA M. THOMPSON, Senior Project Assistant

THERESA RAPHAEL, Consultant

Suggested Citation:"FRONT MATTER." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×

Committee on National Statistics 1996–1997

NORMAN M. BRADBURN (Chair),

National Opinion Research Center, University of Chicago

JULIE DaVANZO,

RAND, Santa Monica, California

WILLIAM F. EDDY,

Department of Statistics, Carnegie Mellon University

JOHN F. GEWEKE,

Department of Economics, University of Minnesota, Minneapolis

JOEL B. GREENHOUSE,

Department of Statistics, Carnegie Mellon University

ERIC A. HANUSHEK,

W. Allen Wallis Institute of Political Economy and Department of Economics, University of Rochester

RODERICK J.A. LITTLE,

Department of Biostatistics, University of Michigan

CHARLES F. MANSKI,

Department of Economics, University of Wisconsin

WILLIAM NORDHAUS,

Department of Economics, Yale University

JANET L. NORWOOD,

Urban Institute, Washington, D.C.

EDWARD B. PERRIN,

Department of Health Services, School of Public Health and Community Medicine, University of Washington

PAUL R. ROSENBAUM,

Department of Statistics, Wharton School, University of Pennsylvania

KEITH F. RUST,

Westat, Inc., Rockville, Maryland

FRANCISCO J. SAMANIEGO,

Division of Statistics, University of California, Davis

MIRON L. STRAF, Director

KATHLEEN SASLAW, Administrative Associate

Suggested Citation:"FRONT MATTER." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×
Suggested Citation:"FRONT MATTER." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×
Suggested Citation:"FRONT MATTER." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×

Preface

In summer 1995, as part of its response to the need for assuring that public funding of health programs be related to documented program performance, the Office of the Assistant Secretary for Health of the U.S. Department of Health and Human Services (DHHS) requested that the National Research Council convene an expert panel to examine and report on the technical issues involved in establishing performance measures in ten substantive program areas. Such measures would be required as part of the proposed Performance Partnership Grants (PPG) Program, under which each state will negotiate with DHHS an action plan with performance objectives that are specific in terms of outcomes, processes, and capacity to be achieved over 3–5 years.

The panel divided its work into two phases. The objective of the first phase was a report to the Secretary of DHHS on performance measures in specified areas that would be useful to the PPG Program over the next 3–5 years. This report presents the panel's findings and recommendations of the first phase. In the second phase, the panel will consider and report on what needs to be done to improve performance measures, for example, by designing new data systems and surveys and increasing understanding of the relationship between programmatic interventions and health outcomes. This two-phase approach was adopted by the panel because of our conclusion that developmental work is needed in both the public and private sectors to adapt, refine, or add to existing data systems to make them more useful in performance measurement and to more clearly delineate the complex causal links between program processes and outcomes.

Because of the uncertainty about the structure of the substance abuse and mental health block grants legislation, as well as funding levels for various public

Page viii Cite
Suggested Citation:"FRONT MATTER." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×

health programs being considered for conversion to a PPG format, it is unlikely that PPG contracts between the federal government and state agencies will go into effect before fiscal 1998. Yet a growing number of states are moving to monitor and analyze outcomes on their own. Moreover, there appears to be a growing consensus within the public health, substance abuse, and mental health communities about the value of performance measurement. Indeed, many people believe that the case for increasing, or even maintaining, public funding will depend on documented program performance.

The development of performance measures is a continuation of earlier efforts to assess progress toward important public health goals. The broad acceptance of immunization rates and other such measures developed for Healthy People 2000, for example, has been instrumental in the creation of data bases and the mobilization of resources in many jurisdictions to assess progress toward those objectives.

The work of this panel has been furthered significantly by four regional meetings of state officials and consumers convened by DHHS—in Portland, Oregon; San Francisco, California; Chicago, Illinois; and Philadelphia, Pennsylvania—and by input from several national associations of state agency administrators (the National Association of State Alcohol and Drug Abuse Directors; the National Association of State Mental Health Program Directors; the Association of State and Territorial Health Officials; and the National Association of County and City Health Officials). DHHS stated that the purpose of the regional meetings was to "develop comprehensive lists of desired program results to initiate the process of developing performance objectives." Prior to the meetings, organizations representing potential PPG grant recipients were asked to consult on a state, regional, or national basis to solicit opinions from their membership about the PPG measures that are important for individual programs. Representatives of these institutional interests were invited to attend the regional meetings, and participants included representatives of state and local governments, public health groups, tribal governments, professional associations, providers, consumers, and advocacy groups. DHHS has actively involved these associations in soliciting information on potential measures and data sources for the PPG process.

Other efforts are under way to examine performance measures for areas of public health that are not included in the panel's charge. For example, two studies were recently completed at the Institute of Medicine (IOM), one focusing on community health performance measures and the other examining performance indicators, standards, and measures for accreditation and quality assurance for managed behavioral health care. The first study, conducted by the Committee on Using Performance Monitoring to Improve Community Health, developed prototypical sets of indicators for specific public health concerns that communities can use to monitor the performance of public health agencies, personal health care organizations, and other entities that can contribute to health improvement (Institute of Medicine, 1997b). The second study, conducted by

Suggested Citation:"FRONT MATTER." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×

the Committee on Quality Assurance and Accreditation Guidelines for Managed Behavioral Health Care, developed a framework for accreditation standards and quality improvements for managed behavioral health care and for developing, using, and evaluating performance indicators (Institute of Medicine, 1997c).

The panel is mindful of the great interest that surrounds the PPG concept: the eventual usefulness of our reports will depend on understanding and agreement by the federal and state officials and others who will be key players in implementing it. For that reason, this report was released in draft form for public comment. The panel was especially interested in receiving comments on its assessment of data availability and quality and the usefulness of the suggested measures in light of the limited empirical data that link program interventions to health outcomes. The panel received many comments, which contributed to this report; see Appendix D for an overview of the comments received and the panel's responses.

The panel appreciates the assistance of the staff of the Committee on National Statistics (CNSTAT) in preparing this report: Miron Straf, staff director of CNSTAT for developing the original project design; Telissia Thompson for organizing panel meetings; Anu Pemmarazu for preparing key technical materials; Theresa Raphael for preparing the literature review for the panel and preparing the annotated bibliography; and Michelle Ruddick and Ashley Bowers for coding and analyzing the responses from the field. Thanks also go to Sue Skillman, University of Washington, for her valued assistance to the committee chair. Finally, special thanks go to Jeff Koshel, study director for this panel, for his very capable management of the study process.

Edward Perrin, Chair

Panel on Performance Measures and Data for Public Health Performance Partnership Grants

The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Bruce M. Alberts is president of the National Academy of Sciences.

The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. William A. Wulf is president of the National Academy of Engineering.

The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Kenneth I. Shine is president of the Institute of Medicine.

The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy's purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Bruce M. Alberts and Dr. William A. Wulf are chairman and vice chairman, respectively, of the National Research Council.

Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health

Suggested Citation:"FRONT MATTER." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×

The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Bruce M. Alberts is president of the National Academy of Sciences.

The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. William A. Wulf is acting president of the National Academy of Engineering.

The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Kenneth I. Shine is president of the Institute of Medicine.

The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy's purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Bruce M. Alberts and Dr. William A. Wulf are chairman and interim vice-chairman, respectively, of the National Research Council.

Suggested Citation:"FRONT MATTER." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×

Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health

Suggested Citation:"FRONT MATTER." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×
This page in the original is blank.
Suggested Citation:"FRONT MATTER." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×
Page R1
Suggested Citation:"FRONT MATTER." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×
Page R2
Suggested Citation:"FRONT MATTER." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×
Page R3
Suggested Citation:"FRONT MATTER." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×
Page R4
Suggested Citation:"FRONT MATTER." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×
Page R5
Suggested Citation:"FRONT MATTER." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×
Page R6
Suggested Citation:"FRONT MATTER." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×
Page R7
Page viii Cite
Suggested Citation:"FRONT MATTER." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×
Page R8
Suggested Citation:"FRONT MATTER." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×
Page R9
Suggested Citation:"FRONT MATTER." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×
Page R10
Suggested Citation:"FRONT MATTER." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×
Page R11
Suggested Citation:"FRONT MATTER." National Research Council. 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, DC: The National Academies Press. doi: 10.17226/5806.
×
Page R12
Next: SUMMARY »
Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health Get This Book
×
Buy Paperback | $44.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The U.S. Department of Health and Human Services is carrying out an ambitious new program to assure that funds for public health programs are spent as effectively as possible. Under the new program, every state will develop a set of performance objectives to measure its progress in terms of outcomes, processes, and capacity. In the first phase of the program, states are to propose such measures to be achieved over three to five years.

This book examines the technical issues involved in the development of performance measures in 10 areas: mental health, substance abuse, HIV infection, sexually transmitted diseases, tuberculosis, immunization, chronic diseases, disability, rape, and emergency medical services. From more than 3,200 candidate measures proposed by researchers, policymakers, and public health professionals, the panel proposes more than 50 potential outcome measures. The book details the advantages and limitations of potential measures as well as the data sources that can support them. This volume will be an invaluable resource to everyone involved in public health.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!