Child and Adolescent
Health and Health Care Quality
MEASURING WHAT MATTERS
Committee on Pediatric Health and Health Care Quality Measures
Board on Children, Youth, and Families
Board on Health Care Services
INSTITUTE OF MEDICINE AND
NATIONAL RESEARCH COUNCIL
OF THE NATIONAL ACADEMIES
THE NATIONAL ACADEMIES PRESS
Washington, D.C.
THE NATIONAL ACADEMIES PRESS 500 Fifth Street, N.W. Washington, DC 20001
NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance.
This study was supported by Contract No. HHSP23320042-509X1 between the National Academy of Sciences and the Department of Health and Human Services. Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the view of the organizations or agencies that provided support for this project.
International Standard Book Number-13: 978-0-309-18623-0
International Standard Book Number-10: 0-309-18623-4
Additional copies of this report are available from the National Academies Press, 500 Fifth Street, N.W., Lockbox 285, Washington, DC 20055; (800) 624-6242 or (202) 334-3313 (in the Washington metropolitan area); Internet, http://www.nap.edu.
For more information about the Institute of Medicine, visit the IOM home page at: www.iom.edu.
Copyright 2011 by the National Academy of Sciences. All rights reserved.
Printed in the United States of America
The serpent has been a symbol of long life, healing, and knowledge among almost all cultures and religions since the beginning of recorded history. The serpent adopted as a logotype by the Institute of Medicine is a relief carving from ancient Greece, now held by the Staatliche Museen in Berlin.
Suggested citation: IOM (Institute of Medicine). 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press.
THE NATIONAL ACADEMIES
Advisers to the Nation on Science, Engineering, and Medicine
The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Ralph J. Cicerone is president of the National Academy of Sciences.
The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Charles M. Vest is president of the National Academy of Engineering.
The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Harvey V. Fineberg is president of the Institute of Medicine.
The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Ralph J. Cicerone and Dr. Charles M. Vest are chair and vice chair, respectively, of the National Research Council.
COMMITTEE ON PEDIATRIC HEALTH AND HEALTH CARE QUALITY MEASURES
GORDON H. DEFRIESE (Chair), Professor Emeritus of Social Medicine, Cecil G. Sheps Center for Health Service Research, University of North Carolina
PAULA A. BRAVEMAN, Professor of Family and Community Medicine, Director, Center on Social Disparities in Health, University of California, San Francisco
CLAIRE D. BRINDIS, Professor of Pediatrics and Director, Philip R. Lee Institute for Health Policy Studies, University of California, San Francisco
BARBARA J. BURNS, Professor of Medical Psychology and Director, Services Effectiveness Research Program, Department of Psychiatry and Behavioral Sciences, Duke University School of Medicine
GLENN FLORES, Judith and Charles Ginsburg Chair in Pediatrics, Department of Pediatrics, University of Texas Southwestern Medical Center
GARY L. FREED, Percy and Mary Murphy Professor of Pediatrics and Community Health, Department of Pediatrics, University of Michigan Health Systems
DEBORAH A. GROSS, Leonard and Helen Stulman Endowed Chair in Mental Health and Psychiatric Nursing, Department of Acute and Chronic Care, School of Nursing, The Johns Hopkins University
MAXINE HAYES, State Health Officer, State of Washington, Department of Health
CHARLES J. HOMER, President and Chief Executive Officer, National Initiative for Children’s Healthcare Quality
KEVIN B. JOHNSON, Professor and Vice Chair, Department of Biomedical Informatics, and Professor, Department of Pediatrics, Vanderbilt University School of Medicine
GENEVIEVE KENNEY, Principal Research Associate, The Urban Institute
MARIE C. MCCORMICK, Sumner and Esther Feldberg Professor of Maternal and Child Health, Department of Society, Human Development and Health, School of Public Health, Harvard University
KATHRYN M. MCDONALD, Executive Director, Center for Primary Care and Outcomes Research, Stanford University School of Medicine
MICHAEL J. O’GRADY, Senior Fellow, Health Policy and Evaluation Department, National Opinion Research Corporation at the University of Chicago
ALAN R. WEIL, Executive Director, National Academy for State Health Policy
ALAN M. ZASLAVSKY, Professor of Health Care Policy (Statistics), Department of Health Care Policy, Harvard Medical School
Study Staff
ROSEMARY CHALK, Study Director
PATTI SIMON, Program Officer
CHELSEA BODNAR, Fellow (January to April 2010)
YEONWOO LEBOVITZ, Research Associate (from November 2010)
WENDY KEENAN, Program Associate
JULIENNE PALBUSA, Research Assistant
PAMELLA ATAYI, Senior Program Assistant
Reviewers
This report has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise, in accordance with procedures approved by the National Research Council’s Report Review Committee. The purpose of this independent review is to provide candid and critical comments that will assist the institution in making its published report as sound as possible and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process. We wish to thank the following individuals for their review of this report:
Anne Beal, Aetna Foundation
Mary Byrne, Stone Foundation and Columbia University School of Nursing
Elena Fuentes-Afflick, Departments of Pediatrics and Epidemiology, University of California, San Francisco
Darcy Gruttadaro, National Alliance on Mental Illness Child & Adolescent Action Center
Kelly J. Kelleher, Office of Clinical Sciences, Columbus Children’s Research Institute, Ohio State University College of Medicine and Public Health
Jonathan Klein, American Academy of Pediatrics and Julius B. Richmond Center of Excellence
Milton Kotelchuck, MGH Center for Child and Adolescent Health Policy, MassGeneral Hospital for Children, and Harvard Medical School
Rita Mangione-Smith, University of Washington Center for Child Health, Behavior, and Development, Seattle Children’s Hospital Research Institute
Karen A. Matthews, Pittsburgh Mind-Body Center, Department of Psychiatry, University of Pittsburgh School of Medicine
Wilhelmine Miller, NORC at the University of Chicago
Michael Msall, Developmental and Behavioral Pediatrics, University of Chicago
Edward B. Perrin, Department of Health Services, University of Washington
Sandeep Wadhwa, 3M Health Information Systems
Deborah Klein Walker, Abt Associates Inc.
C. Jason Wang, Department of Pediatrics, Boston University School of Medicine, and Department of Community Health Sciences, Boston University School of Public Health, Boston Medical Center
Although the reviewers listed above have provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations nor did they see the final draft of the report before its release. The review of this report was overseen by Paul J. Wallace, The Permanente Federation and Kaiser Permanente, and Nancy E. Adler, Departments of Psychiatry and Pediatrics, and Center for Health and Community, University of California, San Francisco. Appointed by the National Research Council and Institute of Medicine, they were responsible for making certain that an independent examination of this report was carried out in accordance with institutional procedures and that all review comments were carefully considered. Responsibility for the final content of this report rests entirely with the authoring committee and the institution.
Preface
As the nation has invested in and made new commitments to programs offering health care services to children, child health advocates, policy makers, families, and the media have raised questions arise regarding the evidence to support claims that these efforts have led to improvements in the overall health status of children and adolescents or to a substantial increase in access to appropriate health care services for these populations. For those who use various categories of health care services, questions arise regarding the presumed impact on processes and outcomes of care. Even as recent legislation has enabled the expansion of child and adolescent health care services, concern persists as to whether significant gaps in access to these services exist and whether these gaps can be filled through a redirection of resources to meet the health care needs of particular populations.
All of these issues serve as the backdrop for the work of a committee convened by the Institute of Medicine (IOM) in fall 2009 to begin a year-long study of current national data systems pertaining to child and adolescent health status, health care access, and quality of care. The committee’s creation was one of the outgrowths of the reauthorization of the Children’s Health Insurance Program, enacted by the Congress in 2009. As it passed this important legislation, the Congress asked: “How can we know that our programmatic efforts are having the intended impact on the health of the nation’s children?”
Embedded in this question are concerns about health care outcomes and eventual summative evaluations of the overall health of the nation’s children and adolescents, but also concerns about the ability to monitor, evaluate, and manage an expanding array of programs and services for
these populations. The committee was asked to assess both the state and the science of child and adolescent health and health care quality measurement, as well as the capacity of existing data systems (particularly at the federal level) to track and evaluate programs and services intended to serve the health care needs of these populations, including the analytical capacity of federal and state agencies that use these data for these purposes.
The 16-member committee formed by the IOM to address these issues comprised an array of experts from the fields of clinical pediatrics, health services research, health program evaluation and policy analysis, and the statistical and epidemiological sciences. The committee’s work was augmented by the expert assistance of four consultants—Patricia MacTaggart, Gerry Fairbrother, Jessica McAuliffe, and Lisa Simpson—whose work greatly facilitated dealing with specific issues related to child health and health care quality measurement.
The committee is especially grateful for the assistance received from the staff of the IOM and the National Research Council, specifically from the Board on Children, Youth, and Families, whose director, Rosemary Chalk, served as principal study director for this study. Ms. Chalk is herself a widely known specialist on policy issues surrounding the health of children and is one of the most expert leaders of the processes through which a study of this kind takes place under the aegis of the National Academies. She was assisted by Patti Simon, program officer at the IOM, who was a mainstay in the committee’s communications and in the drafting of key chapters of the report. Other IOM staff who played key roles in assisting the committee were Pamella Atayi, senior program assistant; Wendy Keenan, program associate; and Julienne Marie Palbusa, research assistant. The committee is grateful for the work of each of these individuals.
In the course of this study, the committee concluded that the nation is fortunate to have a wide array of data sources and frequent analyses addressing the health and health care quality of children and adolescents, each providing a partial set of observations and benchmarks with which to answer some of the above questions of concern to the Congress and the American people. However, the patchwork of clinical information systems, periodic sample surveys, registries, and vital and health statistics reported by state and federal agencies does not facilitate the determination of reliable and valid indicators of either health status or health care access and quality for the nation’s youth as a whole. The committee’s survey of existing data sets and methods for their analysis revealed the need for a national core set of salient measures (some of which may require new data definitions and approaches to collection). These measures need to be collected in every jurisdiction; analyzed using a standard methodological approach; and made available to the nation in a form that will enable policy makers, health care administrators and providers, and the general public to assess
the health status and access to quality care of children and adolescents and to determine whether and to what extent programs funded to provide health care services for these populations are achieving their goals. It is the committee’s hope that its recommendations and the logic underlying them will resonate with those whose efforts will be critical to answering this call in the coming years. Surely having a national data set of this kind will do much to sharpen the nation’s focus and resolve to do what is necessary to ensure the health of its greatest resource—its youth.
Gordon H. DeFriese
Chair
Committee on Pediatric Health and
Health Care Quality Measures
Acknowledgments
Beyond the hard work of the committee and project staff, this report reflects contributions from various other individuals and groups that we want to acknowledge.
The committee greatly benefited from the opportunity for discussion with the individuals who made presentations at and attended the committee’s workshops and meetings (see Appendix B). The committee is thankful for the useful contributions of these many individuals.
This study was sponsored by the U.S. Department of Health and Human Services, the Centers for Medicare and Medicaid Services, and the Agency for Healthcare Research and Quality. We wish to thank Barbara Dailey, Denise Dougherty, Marsha Lillie-Blanton, and Cindy Mann for their support.
We would like to thank those who wrote papers that were invaluable to the committee’s discussions: Christina Bethell, Gerry Fairbrother (with colleagues Jessica McAuliffe and Rachel Sebastian), Patricia MacTaggart, and Paul Newacheck. Thank you also to Brett Brown, whom we commissioned to provide a technical review of the report, and Michael A. Stoto, who provided guidance on report drafts. Together, their insight and expertise added to the quality of the evidence presented. Additionally, Rona Briere and Alisa Decatur are to be credited for the superb editorial assistance they provided in preparing the final report.
4 EXISTING MEASURES OF CHILD AND ADOLESCENT HEALTH
Existing Measures of Child and Adolescent Health
Timeliness, Quality, Public Transparency, and Accessibility of Data on Child and Adolescent Health
Aggregating, Synthesizing, and Linking Multiple Data Sources
5 MEASURES OF QUALITY OF CHILD AND ADOLESCENT HEALTH CARE
Purpose of Quality Measurement
Critical Features of an Optimal Health Care Quality Measurement System for Children and Adolescents
Initial Efforts to Develop a Measurement System for Quality of Care for Children and Adolescents
Current Efforts to Improve the Measurement of Quality of Health Care for Children and Adolescents
Adequacy of These Efforts to Achieve an Optimal System
Promising State and Local Initiatives
6 CONCLUSIONS AND RECOMMENDATIONS
A Stepwise Approach to Measuring Health and Health Care Quality for Children and Adolescents
Step 2: Develop Annual Reports and Standardized Measures Based on Existing Data Sets
Step 3: Create New Measures and Data Sources in Priority Areas
Step 4: Improve Data Collection, Reporting, and Analysis
Step 5: Improve Public and Private Capacities to Use and Report Data
B Workshop Agenda and Participants
C Private-Sector Initiatives to Advance Health Care Quality and the Development of Quality Measures
D Overview of Data Sources for Measures of Health Care Quality for Children and Adolescents
E Biographical Sketches of Committee Members and Staff
____________
1 Only Appendixes A-E are printed in this volume. The other appendixes are included on the CD in the back of the report or online. Go to http://www.nap.edu/catalog.php?record_id=13084.