National Academies Press: OpenBook

Child and Adolescent Health and Health Care Quality: Measuring What Matters (2011)

Chapter: 5 Measures of Quality of Child and Adolescent Health Care

« Previous: 4 Existing Measures of Child and Adolescent Health
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×

5


Measures of Quality of Child and Adolescent Health Care

Summary of Key Findings

  • The prevalence and the aggregate cost of treatment of complex health conditions are lower for children and adolescents compared with the high cost of hospital care for adults and older adults.
  • Measurement of health care quality for children and adolescents serves the same purposes as that for adults—accountability, quality improvement, and population health. However, the development of quality measures for child and adolescent populations has been slow to emerge from the private sector because enhanced quality is unlikely to produce short-term reductions in health care costs (in contrast to the results associated with quality improvement efforts for adults and older adults).
  • The Centers for Medicare and Medicaid Services uses only seven standardized measures to assess care for children and adolescents enrolled in Medicaid programs, and these measures apply only to those who are enrolled in Medicaid managed care plans. These measures involve some of the most common chronic medical conditions, preventive services, and acute care. However, they generally measure specific elements of care rather than serving as comprehensive measures, and they miss
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
  • important areas of health (mental health, oral health, and inpatient care) and care processes (particularly care coordination across multiple settings).

  • The effectiveness of preventive services is particularly difficult to measure because the outcomes may not be known for many years, and the impact may vary according to the risk profile of the patient population. Current preventive measures are largely process focused, and do not consider the outcomes of screening efforts or whether recommended treatment services were actually provided in an effective, evidence-based, equitable, family-centered, or timely manner. This is the case largely because the measures are derived from the claims data generated by a single visit.
  • The number of children and adolescents who live in poverty or in low-income families (generally considered to be below 200 percent of the federal poverty level) is higher relative to adults, a fact that has a significant influence on their health outcomes. For example, high-quality asthma care may produce marginal outcomes for children and adolescents whose housing conditions create persistent risk factors for this condition.
  • The measurement of quality of care for children with special health care needs requires attention to their functional status and care transitions as well as their health conditions. Functional status measures for children and adolescents are not standardized, however, and frequently rely on parental reports rather than comparison of a child’s behavior or skills with those of others with similar health conditions.
  • Variations in the definitions of race, ethnicity, and primary language in state databases are major obstacles to the development and use of heath care quality measures for children and adolescents. A few states have made efforts to gather demographic data by linking their Medicaid or CHIP eligibility files to their claims data sets, but such strategies are not in common use.
  • Many states and some local districts have demonstrated interest in expanding the number and types of measures used to assess quality of care, as well as in applying the measures to children and adolescents enrolled in public and private health plans. Several states and local districts already collect data that can identify early antecedents of unhealthy behaviors that may have lifelong and communitywide consequences.
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×

In September 2010, the Secretary of Health and Human Services (HHS) released the first annual report on the quality of care for children in Medicaid and the Children’s Health Insurance Program (CHIP), as required by the Children’s Health Insurance Program Reauthorization Act (CHIPRA) of 2009 (HHS, 2010c). The report notes that nearly 40 million infants to young adults are now enrolled in these public health insurance plans, representing about half of this population in the United States.1 In addition, Medicaid pays for nearly half of the annual births nationwide.

Although the Secretary’s report provides an important benchmark for assessing the current status of quality measurement of child and adolescent health care services, it also highlights key barriers and challenges that have yet to be resolved. In summary, the Secretary’s report offers the following key findings:

Measurement and Reporting

  • A lack of uniformity and substantial variation in data reliability exist in state-based quality metrics, demonstrating a need for standardized data collection formats.
  • Medicaid managed care plans have developed an infrastructure and processes to support external quality reviews, but these efforts do not include information on children in fee-for-service payment arrangements, and they have not yielded statewide information.
  • Many state officials welcome the opportunity to create more meaningful and useful measures, and they recognize that current performance measures are inadequate.

Quality of and Access to Care

  • A report on the seven quality-of-care effectiveness measures in Medicaid managed care plans from 34 states showed mixed performance: three of the seven measures had relatively high 2008 performance rates, while four of the seven had relatively low performance rates for the same period (see Box 5-1 for a list of the seven measures).
  • Children enrolled in Medicaid/CHIP have much better access to primary care services than uninsured children and access com-

____________

1 Estimates of the rates of coverage of children in public health insurance plans may vary according to the source and method of data collection. The 2010 HHS Secretary’s report draws on administrative data from the Centers for Medicare and Medicaid Services (CMS). Other data sources, such as those collected by the Census Bureau and those based on parental reports, estimate that about 30–35 percent of children and adolescents are covered by public health insurance plans (HHS, 2010a).

Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×

BOX 5-1
Quality-of-Care Effectiveness Measures (Reported by the National Committee for Quality Assurance for Children in Medicaid Managed Care Plans)

  • Use of appropriate medication for asthma
  • Appropriate treatment for upper respiratory infection
  • Childhood immunizations
  • Lead screening
  • Chlamydia screening in 16- to 20-year-olds
  • Follow-up care for children prescribed medication for attention-deficit/hyperactivity disorder
  • Appropriate testing for children with pharyngitis

SOURCE: NCQA, 2009.

  • parable to that of privately insured children. However, access to specialty care services (e.g., dental, mental health) needs substantial improvement.

  • Once enrolled in Medicaid or CHIP, racial/ethnic minority children and children with special health care needs experience improved access to care, but disparities in access to and quality of care persist despite these gains.

This chapter reviews how efforts to improve the measurement of health care quality for children and adolescents have evolved in public and private health care settings. The analysis considers how these efforts compare against a vision of an optimal system of health care quality measurement for younger populations. The review highlights how the findings from the 2010 HHS Secretary’s report might be considered in light of persistent areas of need and emerging opportunities for improving the measurement of health care quality, particularly for children and adolescents enrolled in Medicaid and CHIP health plans. The focus is on five key questions:

  • What is the purpose of health care quality measurement for children and adolescents?
  • What are the critical features of an optimal health care quality measurement system for children and adolescents?
  • What steps have been taken to develop such a high-quality measurement system?
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
  • What are examples of current efforts to improve the system?
  • Are these efforts sufficient to achieve the vision of an optimal system?

The chapter also highlights promising state and local initiatives.

PURPOSE OF QUALITY MEASUREMENT

Measuring the quality of health care services overall fulfills three major purposes: accountability, quality improvement, and population health (IOM, 2006b):

  • Accountability—Quality measures provide a basis for assessing and comparing the performance of selected components of the health care system, including individual professionals, provider groups, hospitals, health plans, and states. Quality measures can identify differences in health care practices or processes, service delivery settings, health plans, and state programs and policies, as well as the outcomes associated with these differences. They provide a basis for determining whether the care patients receive from specific providers is consistent with evidence and professional standards. Such information can assist multiple stakeholders in making choices about providers, about plans, and about state policies and programs. These stakeholders include, among others, patients deciding on a clinician, hospital, or other provider from which to seek services and a health plan from which to obtain coverage; purchasers and health plans selecting providers to include in their health insurance networks; and quality oversight organizations making accreditation and certification decisions. Quality measures facilitate assessment and monitoring of the overall functioning of the health insurance and care delivery system over time, demonstrating who is being reached and who is not. Thus they provide the ability to hold health care state systems and health plans accountable for their performance.
  • Quality improvement—Quality measures can be useful for providers and others who are seeking to improve the quality of care. Such measures can identify gaps in performance that merit attention and can also be used to track progress as individuals and organizations undertake changes to improve care. Key users of such data for improvement include clinicians, quality improvement staff, and managers and members of health care organizations.
  • Population health—Quality measures must be able to identify gaps in population health, as well as in the clinical care of individuals,
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×

and to track progress in addressing these gaps. Stakeholders include those with specific responsibility for broad population health, as well as those involved in the delivery of health and public health services, such as communitywide programs and efforts to address racial and ethnic disparities and promote healthy behaviors.

The creation, selection, and certification of quality measures are driven by multiple public- and private-sector efforts aimed at accomplishing one or more of the above three objectives. Providers have tended to focus on opportunities for clinical improvement, while employers and other private payers have tended to place more emphasis on improving the effectiveness and efficiency of clinical services for prevalent, high-cost conditions that often require interactions with multiple health care providers and health care settings. As a result, many quality measures include an emphasis on procedures or settings associated with effective and timely clinical care, such as the use of specific treatments or the performance of specific tests and monitoring procedures. Public payers have placed more emphasis on broader population-level measurement, such as rates of hospitalization for preventable conditions or rates of rehospitalization.

As noted in earlier chapters, younger populations tend to be relatively healthy, and the frequency of high-cost conditions is much lower among children compared with adults. Given these characteristics and the increasing numbers of children insured through public payers, the public sector has demonstrated far greater interest to date, relative to the private sector, in the development and use of health care quality measures for children and adolescents.

Solid reasons exist for the lag in development and use of measures of health care quality for children and adolescents:

  • The relatively low cost of child health care and the rarity of many child health conditions have impeded clinical research into many of these conditions, limiting the quality of evidence underlying the effectiveness of treatment for younger populations. Without strong evidence linking care processes and outcomes, performance measurement (at least in the domain of effectiveness) is difficult.
  • Prevention is particularly important for children, but proving the effectiveness of many clinical preventive services in children is difficult. There is a long lag time between many of these preventive interventions and potential outcomes, and the determinants of these outcomes are complex—health care may be but one of many inputs. Consensus guidelines for preventive services in childhood are now available and widely used (e.g., Bright Futures or the Promoting Healthy Development Survey [PHDS]) but the effectiveness
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
  • of these services in improving health outcomes for children has not yet been demonstrated (Bethell et al., 2001).

  • The processes of care associated with many health services for children and adolescents involve detailed clinical information that may be included in medical records but is not easily accessible from claims-based or other administrative databases. The time and costs associated with abstracting clinical data (such as the results of routine clinical exams and screening tools associated with well-child care) impede the use of these data in large data collection efforts.
  • Because many important services and conditions other than health care, such as educational services and community nutrition programs, contribute to child health outcomes, a framework that explicitly acknowledges this shared accountability is especially important for child health. Measures that reflect this shared accountability, such as school readiness, can be constructed and used for quality improvement programs, but they are more difficult to apply within the narrow context of health care accountability. A more popular approach has been the use of a process measure, such as access to a medical home (or health home) for children and adolescents, especially those who have a complex or chronic health condition.

CRITICAL FEATURES OF AN OPTIMAL HEALTH CARE QUALITY MEASUREMENT SYSTEM FOR CHILDREN AND ADOLESCENTS

In considering the features of an optimal health care quality measurement system for children and adolescents, the committee took into account the charge for this study from the Congress; the principles underlying performance measurement as articulated in earlier Institute of Medicine (IOM) studies on performance measures (IOM, 2006b); the family/patient focus expressed in the IOM report Crossing the Quality Chasm: A New Health System for the 21st Century (IOM, 2001a); and the committee’s own deliberations, which were strongly influenced by another earlier IOM study, Children’s Health, the Nation’s Wealth (IOM and NRC, 2004). The optimal features thus identified are as follows:

  • Measures should address those topics that (1) are associated with the greatest burden of mortality and morbidity during childhood, (2) have the greatest potential impact over the life span, (3) address the drivers of high child health care costs, (4) are most sensitive to the quality and safety of services, and (5) are of greatest concern to patients/families.
  • Measures should be based on the best available evidence.
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
  • Measures should recognize the unique characteristics of the child and adolescent population, often termed “the four D’s” (differential epidemiology, developmental focus, dependent status, and different demography) (Forrest et al., 1997).
  • Measures should apply to the whole population of children and adolescents, not just those in specific health plans or states.
  • Measures should be capable of aggregation at multiple levels (provider, organization, community, state, nation).
  • Measures should take account of the social environment in which children live to allow assessment of how health care services interact with conditions of adversity to influence health outcomes.
  • Measures should be broader than the presence or absence of disease.
  • Measures should minimize the burden and cost of collection.
  • Measures should be able to generate data to drive action and quality improvement at the program (provider) and policy levels.
  • Measures should be readily available to all (transparent).
  • Measures should be available in a timely manner.
  • Measures should reflect patient, family, and community perspectives on quality.

The contrast between these features and the quality measures that are cited in the 2010 HHS Secretary’s report is remarkable. Only seven quality measures for children and adolescents are reviewed in that report, and those measures are used in only 30 states and only for those enrolled in Medicaid managed care programs. As a result, the nation is far from having a performance measurement system that can foster the incorporation of the above features into the development and use of quality measures for child and adolescent health care.

INITIAL EFFORTS TO DEVELOP A MEASUREMENT SYSTEM FOR QUALITY OF CARE FOR CHILDREN AND ADOLESCENTS

Initial efforts to improve the measurement of health care quality for the general population include reports by the IOM and initiatives by such groups as the National Committee for Quality Assurance (NCQA) and the National Quality Forum (NQF) (see Appendix C). More recently, the National Initiative for Children’s Healthcare Quality (NICHQ) and the Child and Adolescent Health Measurement Initiative (CAHMI) have sought to emphasize the need for consumer-driven measures of health care quality for children and adolescents, as well as to develop approaches for encouraging the use of available measures. These various efforts were encouraged and

Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×

supported through congressional guidance in the CHIP (1997) and CHIPRA (2009) legislation, which stimulated national leadership and research support by the Agency for Healthcare Research and Quality (AHRQ), as well as demonstration and state-based coordination efforts within the Centers for Medicaid and Medicare Services (CMS).

Initial Congressional Guidance and Federal Leadership

In 1997, Congress created the State Children’s Health Insurance Program (SCHIP), one success resulting from the earlier, intense, but ultimately unsuccessful battle over health care reform (Iglehart, 2007). The legislation proved to be a major advance for children and families in terms of expanding health plan coverage and access to services and reducing disparities in these two areas. In addition, the 1997 SCHIP legislation required that state annual reports include for the first time information on quality—a requirement that did not exist in Medicaid. HHS developed a standardized template for these state reports emphasizing the use of Healthcare Effectiveness Data and Information Set (HEDIS©) measures that had been developed as part of quality improvement efforts among private health plans—efforts focused largely on adult health care quality.

During the past two decades, CMS worked with multiple partners to develop quality measures that could be used to assess and improve health care services for children and adolescents enrolled in Medicaid and CHIP health plans. These efforts focused initially on use of the HEDIS measures and collaboration with NCQA. In 1996, for example, prior to passage of the CHIP legislation, Medicaid HEDIS was developed, incorporating standards specific to Medicaid populations (MacTaggart, 2010). Although the use of such measures was voluntary, more than 30 state Medicaid agencies were using at least portions of Medicaid HEDIS within a year. Further collaboration with NCQA, the American Public Human Services Association, and the Commonwealth Fund led to a 2002 report that covered 13 HEDIS measures, 5 of which were child-specific, from 176 managed care organizations in 33 states plus the District of Columbia and the Commonwealth of Puerto Rico (MacTaggart, 2010). One year later, HEDIS 2003 contained 23 measures designed for or applicable to children and adolescents, including child and adolescent immunization status, chlamydia screening for women, use of appropriate medications and follow-up care for people with asthma, and experience of care (based on responses to the Consumer Assessment of Healthcare Providers and Systems [CAHPS] 3.0H Child Survey, which included a screener for children with chronic conditions) (MacTaggart, 2010).

Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×

Early Attributes of Quality Measures for Children and Adolescents

In 2004, the Commonwealth Fund examined existing child health care quality measures within 19 different data sources or measurement sets (Beal et al., 2004; Leatherman and McCarthy, 2004). This analysis identified 396 quality measures for children within the 19 data sources, 10 of which relied on administrative or medical record data and 9 of which used survey data.

In categorizing these measures according to the six aspects of quality care identified in Crossing the Quality Chasm (IOM, 2001a)—safety, timeliness, effectiveness, equity, efficiency, and patient-centeredness—the authors found that more than half (59 percent) of the measures involve indicators of effectiveness, while about one-sixth (14 percent) are relevant to the safety domain (Beal et al., 2004). The safety measures involve primarily serious errors in health care delivery, especially medical and surgical errors during hospitalization. The authors also examined how existing health care quality measures are distributed across the different purposes of health care—acute care (getting better), preventive care (staying healthy), and chronic care management (living with illness). They found that a large proportion of the measures (40 percent) could be categorized under getting better, about 24 percent under staying healthy, and 17 percent under living with illness. There were no measures related to end-of-life care. Nearly one-fourth of all measures were not classifiable by the six aspects of quality care (Beal et al., 2004).

In addition, some efforts have focused on developing quality measures that would apply to children of all ages, as well as measures that would have specific application to selected age groups—infants, toddlers, children, and adolescents. Analyses of these efforts have shown that the majority of measures they produced can be applied to children across all age groups; however, there are no unique measures for school-aged children (aged 5−18) (Beal et al., 2004).

Despite the advances achieved through the above efforts, studies have revealed many areas in which quality measures fall short (Landon et al., 2007; Leatherman and McCarthy, 1999; Mangione-Smith et al., 2007; Thompson et al., 2003). In particular, measures fail to capture aspects of care that are relevant for low-income, Medicaid-insured children. Moreover, quality measures are lacking for many important areas of health care, even though major studies have demonstrated significant shortcomings in these areas both for the general population (such as inpatient care or mental health services) and with respect to disparities in access to and use of services (such as oral health care).

Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×

CURRENT EFFORTS TO IMPROVE THE MEASUREMENT OF QUALITY OF HEALTH CARE FOR CHILDREN AND ADOLESCENTS

Recognizing the shortcomings detailed above, recent health care legislation places new emphasis on the importance of measuring health care quality for younger populations. This legislation includes CHIPRA (especially Title IV), the Patient Protection and Affordable Care Act (ACA), the American Recovery and Reinvestment Act (ARRA), and the Health Information Technology for Economic and Clinical Health (HITECH) initiative included in the ARRA.

2009 Congressional Action and Title IV of CHIPRA

In reauthorizing the CHIP legislation in 2009, Congress renewed its initial efforts to foster the use of health care quality measures for Medicaid and CHIP populations, adding new dimensions that had emerged as major concerns. The new legislation required that quality measures specifically address three broad types of care (prevention, acute care, and chronic care) and specific elements of clinical quality (effectiveness, safety, patient and family experience, and equity [disparities]). Congress also directed that child health care quality measures specifically address mental as well as physical health care, care across the full spectrum of child development, care integration and access as reflected by accessibility of care in inpatient and outpatient settings, and the duration and stability of health insurance coverage.

Title IV of CHIPRA (P.L. 111-3) significantly expanded various child health care quality improvement initiatives and authorized several new efforts, including

  • development of an initial core set of health care quality measures for children enrolled in Medicaid or CHIP, to be supplemented by research grants to address incomplete or missing measures (known as the U18 awards);
  • a new program of 10 quality demonstration grants to states as the basis for a future national quality system for children’s health care;
  • creation of the Medicaid and CHIP Payment and Access Commission (MACPAC) to review Medicaid and CHIP access and payment policies and report recommendations to Congress;
  • creation of a Federal Quality Workgroup of the CHIPRA Steering Committee to ensure that the expertise of key HHS entities is brought to bear on improving quality measurement and the quality of health care for all children;
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
  • authorization of a $20 million program for the Census Bureau to improve state-specific estimates of the child population;
  • encouragement of collaboration between AHRQ and CMS to develop an electronic health record (EHR) format for children;
  • an AHRQ research program to develop, validate, and improve a final core set of pediatric quality measures, to be completed by January 1, 2013; and
  • technical support from AHRQ to CMS, including content for best practices related to the implementation of core measures, and an evaluation of outcomes of child health care quality demonstration projects.

Title IV also mandated a study by the National Academies “on the extent and quality of efforts to measure child health status and the quality of health care for children across the age span and in relation to preventive care, treatments for acute conditions, and treatments aimed at ameliorating or correcting physical, mental, and developmental conditions in children.” That study is the subject of this report.

The CHIPRA legislation represents a landmark in its emphasis on quality of care for the nation’s children, reflecting a drive toward achieving quality measures that can provide common data elements and facilitate consistent reporting by the states, with allowance for flexible use to address each state’s individual needs. Of importance, the legislation established that these provisions apply to both Medicaid and CHIP, moving toward a consistent strategy across both programs nationally and at the state level. It is important to note as well that CHIPRA included provisions calling for identification of existing quality measures in use by public and privately sponsored health care coverage arrangements, as well as provisions around guidance for reporting performance by the states and demonstration programs to showcase and test child health care quality measures. As noted above, CHIPRA also contained a provision and funding ($5 million) for development of an EHR format for children to support quality reporting. While the inclusion of this provision was important, the subsequent passage of the ARRA (discussed below) sharply enhanced the ability of the health care system (ambulatory care providers and hospitals) to implement EHRs through a $19 billion investment, although not as tightly focused on children.

The CHIPRA legislation recognized that stability of coverage is integrally linked to program quality overall. States are now required to report on eligibility criteria, enrollment, retention, use of simplification measures, access to care, and care coordination, among other aspects of coverage. Furthermore, reporting on these dimensions of coverage is required, whereas reporting on quality remains voluntary.

Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×

Other Recent Relevant Legislation

The CHIPRA legislation was soon followed by additional laws that lent further momentum to quality improvement efforts and the development of the next generation of health care quality measures. None of these more recent initiatives address the unique needs of children and adolescents, raising concern that the distinct demographic, developmental, epidemiological, and dependency characteristics of younger populations may once again be sidelined in the evolving national infrastructure for quality improvement in the health care system. The recent legislative initiatives include

  • the ACA (P.L. 111-148),
  • the ARRA (P.L. 111-5), and
  • the HITECH Act, part of ARRA.

These additional pieces of legislation placed further emphasis on issues of access (to both insurance coverage and health care services), quality, and cost in the health care system. Together, they have major implications for the future health of America’s children and adolescents. The expanded federal efforts also reflect the recognition that improving the quality and affordability of health care is an enterprise that requires strong collaboration between the private and public sectors at all levels of government. The ARRA legislation, for example, offers substantial incentives for physicians and hospitals to adopt EHRs to improve the quality and safety of care.

The structure of health care organizations and the financing of health care are currently in flux, with the pace of change accelerating as a result of numerous provisions in the ACA that encourage coordination within the health care delivery system and a payment structure that rewards quality and outcomes. These provisions include the creation of accountable care organizations (ACOs); the promotion of medical homes for Medicaid enrollees with chronic conditions; and various efforts to improve health care quality and reward better outcomes, such as lowering rates of hospital readmissions and hospital-acquired conditions.

With few exceptions, such as the Pediatric ACO Demonstration project, these payment reforms do not focus on children. Indeed, much of the push for payment reform is embedded in the Medicare program—the other end of the age spectrum. While few efforts to reform payment and delivery systems target children, other, broader changes that are slated for implementation under health care reform will certainly have an effect on children’s care.

The ACA includes a number of other provisions that are likely to have profound effects on children’s health and quality of care, including the development of a National Health Care Quality Strategy (National Quality

Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×

Strategy) and an associated strategic plan that will identify priorities for improving the “delivery of health care services, patient health outcomes, and population health,” as well as introduce new reporting requirements for health insurance plans that will lead to greater transparency and accountability (P.L. 111-148). The legislation also establishes the Center for Medicare and Medicaid Innovation, whose efforts could include the evaluation of new payment and delivery models for delivering care to children in Medicaid and CHIP.

The Obama Administration’s National Quality Strategy, announced by the Secretary of HHS, strives to align federal efforts with those of the states and the private sector and to foster collaborative partnerships wherever feasible. The ACA directs the Secretary of HHS to integrate these efforts into a cohesive strategic plan with priorities for improving the delivery of health care services, patient health outcomes, and population health. This strategic plan is due to Congress by January 1, 2011, and must include provisions for (1) agency-specific plans and benchmarks, (2) coordination among agencies, (3) strategies to align public and private payers, and (4) alignment with meaningful use of health information technology (HIT). The plan is to be updated and refined periodically through annual reports to Congress to accommodate emerging issues. Most notably, the National Quality Strategy, required by the ACA, strives to link improvements in health care quality and health outcomes and reduced costs. The draft strategy (not yet finalized) does not specifically use a life-course framework (as detailed in Chapter 2) in considering priorities; a life-course approach may be appropriate for future reports to Congress.

The ACA also calls for the development of a National Prevention and Health Promotion Strategy (National Prevention Strategy) that is scheduled to be released in March 2011. The National Prevention Strategy focuses attention on the need for community-based efforts emphasizing prevention to reduce the incidence of the leading causes of death and disability (Bilheimer, 2010).

Both the National Quality Strategy and the National Prevention Strategy are aimed at strengthening collaboration among public- and private-sector partners. The National Prevention Strategy will also be developed by consultation across the federal government. The two strategies will share common goals and priorities for healthy people and communities. Both will include an explicit focus on goals that require close collaboration between clinical and community partners (HHS, 2010d). These two initiatives will have a broad impact on the design of measures of health and health care quality for children and adolescents, highlighting the importance of improvements in the health care system and the social and economic environments of children and their families that can lead to better health outcomes for all.

Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×

While the legislation passed since the beginning of 2009 collectively advances opportunities to improve quality measurement dramatically for adults, the impact on children remains uncertain. The ACA extended the emphasis on quality and added a heavy emphasis on cost containment, value, and accountability, while the ARRA provided unprecedented funding for HITECH and a fresh emphasis on HIT for quality improvement. These legislative initiatives, especially those efforts authorized under CHIPRA, offer opportunities for significant improvement in the state of measurement of health care quality for children and adolescents. But realizing this potential depends not only on improving the inventory of quality measures and developing measures for priority health conditions, but also on creating data systems that can reduce the variability and inconsistency in the quality of state-level Medicaid databases, as well as on developing strategies that allow the states to use the data to improve system performance and health. Such strategies may require collaboration with other service settings so as to look beyond the health care system for environmental factors, such as poverty, stress, and inadequate nutrition that significantly influence the health status of children and adolescents.

Implementation Efforts of the Agency for Healthcare Research and Quality

The recent emphasis on health care quality for children and adolescents needs to be considered within the broader context of other quality improvement efforts at AHRQ. For example, Title IX of the Healthcare Research and Quality Act of 1999 (P.L. 106-129) required AHRQ to issue an annual public report on health care quality, beginning in 2003 (U.S. Congress (106th), 1999). In preparation, AHRQ funded the first in a series of IOM studies to produce recommendations for the selection of measures for this annual report, which has included a report on children’s health care quality even though such measures for children have not been as well developed as those for adults (Halfon et al., 1998; Homer et al., 1998; IOM, 2001b; McGlynn et al., 2000; Szilagyi and Schor, 1998).

In April 2009, AHRQ and CMS executed a Memorandum of Understanding (MOU) outlining which agency will take the lead role for various provisions in Title IV of CHIPRA. According to the MOU, AHRQ is leading the implementation of four provisions: the identification of the initial core measure set, the establishment of a quality measures program, the development of a model EHR, and the IOM study that is the basis for this report.

In responding to congressional guidance, AHRQ formed an expert advisory panel, the AHRQ National Advisory Council for Healthcare Research and Quality’s Subcommittee on Children’s Healthcare Quality Measures for

Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×

Medicaid and Child Health Insurance Programs (SNAC). The SNAC was charged with providing guidance on measure evaluation criteria to be used in identifying an initial core measure set for health care quality for children and adolescents and on a strategy for gathering additional measures and measure information from state programs and others. The SNAC’s recommendations were to be provided to CMS and the AHRQ National Advisory Council, which in turn would advise the director of AHRQ. The directors of AHRQ and CMS would then review and decide on the final core set to be presented to the Secretary of HHS for consideration.

The SNAC solicited proposals for candidate measures that were assessed by the subcommittee members according to three key evaluation criteria—validity, feasibility, and importance—using an adaptation of the RAND-University of California, Los Angeles (UCLA) modified Delphi process (a structured method of creating consensus through anonymous evaluations). The candidate list was reduced to a group of 25 core measures through a series of deliberations and panel discussions. The SNAC process gave immediate priority to the validity, reliability, and feasibility of the measures over the comprehensiveness of the set. All but three of the proposed core set of measures were recommended by the HHS Secretary. The deleted measures include child and adolescent suicide risk assessment, the CAHPS Clinician and Group Survey, and the NCQA HEDIS annual dental visit measure. Furthermore, the HHS Secretary chose to list the three separate well-child visit measures individually, resulting in a total of 24 measures.

The initial set of core measures proposed by the SNAC is well balanced across developmental stages. The measures are heavily weighted to address prevention and strongly oriented toward ambulatory settings in general and primary care in particular. Physical health is emphasized to a much greater extent than developmental, social, emotional, or mental health.

The core measures recommended by the SNAC, and ultimately revised and then adopted by the Secretary of HHS, set the stage for the development of a state-based measurement system that can be used to examine and compare the performance of different health plans in serving the needs of vulnerable children and adolescents (see Box 5-2). The ultimate goal is to support states in their efforts to adopt consistent, standardized statewide health and health care quality measures; encourage the use of existing data sources, including both population health surveys and administrative records; and provide a basis for comparing provider and health plan performance in contributing to the achievement of national and statewide health goals for children and adolescents.

AHRQ has begun the process of expanding the core measure set and has called for the development of new measures (AHRQ, 2010f). AHRQ also has funded 10 Centers of Excellence and demonstration projects, three of which include significant efforts to develop new measures. These grant

Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×

programs are designed to showcase best practices for improving care; some have developing measures, particularly for behavioral health, as part of their mandate (AHRQ, 2010d).

As noted in Chapter 2, different data sources are used for different objectives, and the nation lacks effective mechanisms that can link the health indicators generated by population health surveys to privately and publicly funded quality improvement efforts focused on measuring health care processes and outcomes in clinical care settings. Health care providers and organizational units such as hospitals, group practices, and health plans are especially concerned about being held accountable for the health of underserved populations when they cannot control all the factors that influence the health outcomes of these groups.

As federal, state, and local health and health care agencies move toward greater reliance on using data and indicators to drive improvements in the performance of the health care system, opportunities will emerge to align disparate interests and to develop standards of shared or partial accountability for the health status of selected populations. Inevitably, these efforts will require collaboration and discussion of shared goals, the creation of mechanisms to set benchmarks and timelines for achieving these goals, the designation of entities that can be held responsible for contributing to these efforts, and consideration of the extent to which public and private data sources created for specific aims can be used for other purposes.

Initiatives of the Centers for Medicare and Medicaid Services

Following the passage of the CHIPRA legislation in 2009, CMS accelerated its efforts to work with public- and private-sector organizations on improving quality-of-care measures that could be used within both Medicaid managed care organizations and primary care settings that serve Medicaid and CHIP populations. These efforts included the identification of a core set of measures for voluntary reporting by the states, as well as a comprehensive technical assistance plan that could be applied for states across all of the CHIPRA provisions. CMS also placed a new emphasis not only on supporting state efforts to improve Medicaid and CHIP programs with respect to the delivery of care, but also on encouraging the states to identify actual improvement in health outcomes as a major focus of these efforts. These initiatives have led to a close partnership with AHRQ and other federal agencies in support of the general vision of HHS of providing the right care for every person every time.

The 2010 HHS Secretary’s report also described emerging investments in building the infrastructure within federal and state agencies needed to assess the quality of care received by children and adolescents under Medicaid and CHIP (HHS, 2010c). These efforts include individualized support

Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×

BOX 5-2
Final Core Set of Measures for Children’s Health Care Quality Recommended by HHS Secretary*

Prevention and Health Promotion

  • Prenatal/Perinatal
  • — Frequency of ongoing prenatal care (National Committee for Quality Assurance [NCQA] measure)
  • — Timeliness of prenatal care (NCQA measure)
  • — Percent of live births weighing less than 2,500 grams
  • — Cesarean rate for low-risk first-birth women
  • Immunizations
  • —Childhood immunization status (NCQA measure)
  • —Adolescent immunization (NCQA measure revised for 2010)
  • Screening
  • —Body mass index (BMI) documentation ages 2−18
  • —Rates of screening using standardized screening tools for potential delays in social and emotional development (ABCD)
  • —Chlamydia screening for women
  • Well-Child Care
  • —Well-child visits in the first 15 months of life
  • —Well-child visits in the third through sixth years of life
  • —Well-child visits for ages 12−21 with primary care provider (PCP)
  • Dental
  • —Total eligibles receiving preventive dental care (EPSDT)

Management of Acute Conditions

  • Pharyngitis-appropriate testing (NCQA measure)

for and feedback to states with respect to the performance of managed care organizations that serve children and adolescents enrolled in Medicaid and CHIP plans and external quality reporting. CMS has piloted a pediatric measure program to develop and evaluate the core quality measures, as well as created a strategy for states’ voluntary collection and reporting of data on the performance measures. CMS also has developed a compendium of quality measures to give states options to consider in identifying quality measures that best support their specific quality strategies and address the needs of their populations.

A major challenge persists in the nature of collaboration between CMS initiatives and data collection and reporting practices within individual states: how to achieve and accelerate consistency across state quality reporting while allowing for states’ flexibility and innovative practices. In addressing this challenge, CMS has developed a national patient-centered

Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
  • Otitis media effusion—avoidance of inappropriate use
  • Total Early Periodic Screening, Diagnosis, and Treatment (EPSDT) eligibles who received dental treatment
  • Emergency department (ED) utilization—average of three ED visits per member
  • Pediatric catheter-associated blood stream infection rates (pediatric intensive care unit [PICU] and neonatal intensive care unit [NICU])

Management of Chronic Conditions

  • Annual number of asthma patients with one or more asthma-related ED visits
  • Follow-up care for children prescribed attention-deficit/hyperactivity disorder (ADHD) medication
  • Annual hemoglobin A1C testing

Family Experiences of Care

  • Consumer Assessment of Healthcare Providers and Systems (CAHPS) Health Plan Survey 4.0, Child Version
  • Survey for families of children with special health care needs

Availability of Services

  • Access of children and adolescents to a PCP

________________

* Based on recommendations by AHRQ National Advisory Council for Healthcare Research and Quality Subcommittee on Children’s Healthcare Quality Measures for Medicaid and CHIP Programs (SNAC).
SOURCE: HHS, 2009.

framework (focused on the “beneficiary”) that combines the efforts and successes of national initiatives with the multiple types of activities that are occurring at the state level (AHRQ, 2010e). Further opportunities to address this challenge include expanding and improving states’ access to encounter data (e.g., Medicaid Statistical Information System [MSIS] encounter data); resolving anomalies in state-level claims and enrollment records; and encouraging states to link to other databases, including the National Vital Statistics System (NVSS).

The annual report to Congress mandated by CHIPRA may eventually provide an opportunity for CMS to integrate the successes of both national and state efforts through the development of a menu of national and state-specific goals; public reports that describe progress toward meeting those goals; and reports on progress toward achieving consistent, standardized measures that can be used in both national and state-level data collection

Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×

and reporting efforts. In supporting these objectives, CMS is working with the states to develop an annual reporting template to facilitate the annual submission of publicly available information on the quality of pediatric care (AHRQ, 2010e). As authorized under CHIPRA, AHRQ and CMS also are collaborating to develop an EHR format for children and adolescents to standardize and facilitate reporting of quality indicators.

ADEQUACY OF THESE EFFORTS TO ACHIEVE AN OPTIMAL SYSTEM

The reporting specifications initiated by the CHIPRA legislation and states’ ability to respond to them are extremely important. It remains uncertain whether voluntary reporting will be an effective means of securing data from all states, and states have not yet demonstrated their capacity to report on all Medicaid and CHIP children and adolescents, not just those enrolled in managed care plans. Finally, it will be necessary to determine how states use health care quality measurement to improve outcomes for these children and adolescents, and how these outcomes compare with those of other populations of children and adolescents, such as those who are uninsured or are enrolled in private health plans.

The process initiated by AHRQ and CMS in identifying a small set of core measures for use by the states is an important beginning. Other key areas, including but not limited to the dimensions specified in the legislation, such as fostering greater consistency in the collection of racial and ethnic data or the collection of data on prevalent health conditions that involve mental health or substance abuse services, were not addressed in the initial core set either because no current measures existed or if they did, they did not pass the process established by the SNAC. Several additional key issues require attention in improving the usefulness of health care quality data now available in national and state-level data sets. Some of these issues involve improving the validity and reliability of data sources through consistent definitions and standardized criteria. Others pose greater challenges, requiring the collection of data across different care settings and time periods, or the collection of data in areas that involve difficult-to-measure or difficult-to-reach populations, especially with respect to preventive and mental and behavioral health services. Still others require new data sources and new data collection methods that can provide information about the social environments of populations enrolled in Medicaid and CHIP health plans, as well as the relationship between their health status and other measures of child well-being, such as educational achievement.

Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×

Need for Additional Work on Core Measures

The legislation required that the initial core set be drawn from existing measures, that these measures be improved, and that new measures be developed. The SNAC explicitly conceptualized and reached consensus on its approach by dividing possible measures into three categories: grounded, intermediate, and aspirational. Grounded measures were defined as currently feasible; many such measures were already in widespread use. Intermediate measures were defined as having good specifications with some isolated examples of use; however, they lacked broader established validity and existing widespread implementation. Aspirational measures were defined as those that were needed to fill an important gap but did not yet exist as valid or feasible measures. The consensus of the committee was to focus on identifying and choosing from the most grounded measures (AHRQ, 2010f).

Ultimately, the process used by the SNAC members to identify the initial core set of measures took into account validity, feasibility, and importance (AHRQ, 2010g). Validity was defined as being supported by scientific evidence or expert consensus. A measure considered to be valid supported a link between structure and outcomes, structure and processes, and processes and outcomes. In addition, the measure must have been judged to measure what it purported to measure. Finally, for a measure to meet the criteria for validity, it had to relate to an aspect of health that was thought to be impacted primarily by health care providers or the health care system. Although the strict accountability approach used by the SNAC is traditionally applied in quality measurement, this criterion has been challenged in a previous IOM report (IOM, 2006b) as well as the present report with respect to the need for shared accountability.

The feasibility criterion required that the data necessary to score a measure be available to organizations from administrative records, medical records, and/or surveys. The SNAC looked for existing detailed specifications that would allow for “reliable and unbiased” scoring of measures across government levels and health care organizations. To be ranked highly on feasibility, a measure also had to be in current use as mandated by legislation (AHRQ, 2010e).

In the refinement of the SNAC methodology for the second Delphi evaluation, a third criterion for evaluation, importance, was also applied. For a measure to be considered important, it needed to be deemed “actionable” in that there should be a clear intervention that could be undertaken to impact the measure. The definition and scoring of importance also included estimation that the cost of the condition measured was a significant burden on the American health care system, that there was evidence that the measure was indicative of a substantial quality problem, and that an assessment of accountability for the problem was possible. Finally, a con-

Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×

sideration in evaluating importance was that there should be documented variation in performance by socioeconomic factors, specifically race/ethnicity or insurance type (AHRQ, 2010h). The overall goal of the importance criterion was to identify “sentinel” measures for prevention and care that would signal the status of a substantial quality problem for which accountability could be assigned and action taken for improvement.

However, in accordance with the SNAC’s legislative mandate, the importance of a particular type of measure was not the driving force behind the identification of the core set. Thus the SNAC evaluated the current validity, feasibility, and importance of measures that could be improved upon directly by a change in health care services only. Now that the initial core set of measures has been developed, AHRQ and CMS are moving into the next phase of development for the core set of pediatric quality-of-care measures under a Pediatric Quality Measures Program. This program is charged with improving and strengthening the initial core set of measures by continuing to evaluate those measures, as well as increasing the portfolio of evidence-based measures that can be used by purchasers, providers, and consumers of health care for children (AHRQ, 2010c). This is being accomplished through awards totaling $55 million over a 4-year period for demonstration research and dissemination projects designed to implement and improve upon the core set of children’s health care measures (AHRQ, 2010c).

Gaps in Mental Health and Substance Abuse Measures

As noted in Chapter 4, nearly 20 percent of children and adolescents experience some type of mental health or substance abuse disorder (IOM, 2009c). The development of health care quality measures for these conditions for children and adolescents lags far behind that for adults. The measures used most commonly in Medicaid managed care plans involve two HEDIS indicators: one focused on attention-deficit/hyperactivity disorder (ADHD) medication and the other on psychiatric hospital follow-up.

A recent review by Bickman and colleagues (Bickman et al., in press) found little value for quality improvement from these two indicators, but there are reasons to pursue both. The low rate of follow-up by parents for ADHD medication has been well documented. For ADHD, the potential loss of benefit due to unfilled prescriptions or negative experiences with a given medication suggests the importance of further contact to reassess the reasons for nonadherence and consider alternative medications. With regard to posthospital follow-up for a psychiatric admission, the limited studies are equivocal. Attention might be given to a recent longitudinal study by James and colleagues (James et al., 2010) that documents significantly

Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×

lower rates of rehospitalization when community mental health resources are utilized.

The committee has identified opportunities to develop new priorities in considering the quality of mental health services for children and adolescents, as well as other types of care, within the framework recommended by the IOM for the AHRQ annual report on health care quality (Table 5-1) (IOM, 2001b). Under care coordination, for example, combining treatment plans for a parent’s mental health disorder with preventive services for the parent’s child(ren) is recommended in cases such as parental depression (IOM, 2009c). Second, a focus on disruptive behavior disorders (oppositional defiant and conduct disorders) is particularly important given their well-documented relationship to poor outcomes in childhood and adulthood and significant comorbidities (ADHD, trauma, and substance abuse). The prevalence of these disorders is high among adolescents in the general population and much higher among adolescents in juvenile justice and child welfare systems, and they are among the most frequent reasons for mental health specialty treatment. Evidence-based interventions to address these disorders have been developed for all age groups, although these interventions are not necessarily available in usual practice. Finally, this clinical group exhibits the highest risk of out-of-home placement—hence the focus on indicators related to residential treatment. According to parental reports, 628,000 adolescents had experienced care for emotional and behavioral problems under out-of-home placement in the past year: 510,000 in a hospital, 199,000 in a residential treatment center, and 112,000 in treatment foster care (SAMHSA, 2007). According to other surveys, 93,000 adolescents were in juvenile detention in 2006 (OJJDP, 2008), and 748,000 were in the foster care system in 2008 (ACF, 2010). This national estimate of nearly 1.5 million youth in such placements is astounding, and although not broken out by diagnosis here, suggests the need for attention to the quality of care for these high-risk adolescents (many of whom may already be enrolled in Medicaid or CHIP plans), as well as relevance to their extensive use of the health care system.

To the extent that the health care system routinely screens for mental health problems among children and adolescents, the focus is on cognitive and motor delays and autism. Screening for other mental health problems in children or their parents is much more limited, and the prevalence of selected disorders is not readily available in administrative data or national surveys. There is much evidence that the system is not detecting or dealing with mental health issues (Fairbrother et al., 2010). Furthermore, there is increasing concern regarding the treatment of certain highly prevalent mental health issues in children, including a lack of follow-up (Gardner et al., 2004) and an increase in off-label prescribing (Zito et al., 2008).

Routine screening for mental health and behavioral problems would

Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×

TABLE 5-1 Opportunities to Adapt the Priorities of the AHRQ Annual Report to Address the Quality of Services for Children and Adolescents

Component of Quality Type of Care
Preventive Care Acute Treatment Chronic Care Management End-of-Life Care
Effectiveness Prevention: immunizations, prenatal care, programs for improved nutrition and exercise Appropriate care for various common conditions, evidence-based practice (EBP) for behavioral disorders in young children (e.g., parent management training), parental treatment for psychiatric diagnoses Appropriate follow-up, early intervention and supportive services for disability (physical, learning, etc.) and early-stage mental disorders; in-home EBP treatment (e.g., multisystemic therapy [MST]) for serious behavioral disorders; monitoring of rate of treatment completion
Screening for physical and learning disabilities, mental health, substance abuse, and parental depression; nurse home visitation for high-risk infants
Safety Parent education about environmental hazards; assurance that contraindicated vaccines are not administered; developmentally appropriate screening for trauma, truancy, bullying, suicidality, and domestic violence; sex education Use of practices known to reduce hospital complications, including iatrogenic pneumothorax, decubitis ulcer, postoperative sepsis, inadvertent retention of a foreign body; monitoring of medications for evidence and off-label use; monitoring of iatrogenic treatment (e.g., group therapy for eating disorders) Medication management system to reduce adverse events due to known drug interactions; monitoring of psychotropic polypharmacy (3+ or 2 in one class); monitoring of seclusion/restraints in institutional and other placements Adequate education for patient and family about palliative care options to prevent imprudent use of aggressive treatments where risks of complications (including inadequate pain control) outweigh benefits at this stage of care
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
Timeliness Prevention and screening: immunizations and screening exams on schedule All-hours access for childhood diseases; timely referral    
Patient/Family-Centeredness Parent and adolescent assessments of communication of information about child’s developmental status, child’s needs to maintain healthy development; parent education at each developmental stage to foster cognitive, emotional, and social functioning Parent and adolescent assessments of communication, responsiveness to preferences, etc. in acute care; child/parent participation in decisions about treatment type Supports to children with special health care needs and their parents/caregivers; communication of treatment plans and prognosis; child/parent participation in treatment planning, implementation, and monitoring  
Access Regular source of care; primary care and schools as portals of entry to care; primary care providers trained to assess mental health/substance abuse Ability to obtain treatment for acute illness; insurance coverage for all youth; support for access to care (transportation and child care); access to a full continuum of care Availability of needed services for children with special health care needs; access to specialty care for major depression, including medication and psychosocial treatment; appropriate access for all age and racial/ethnic groups
 
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
Component of Quality Type of Care
Preventive Care Acute Treatment Chronic Care Management End-of-Life Care
Efficiency Ratio of positive screen for mental health and referral to appropriate care setting Reimbursement linked to evidence; appropriate assessment and follow-up for prescriptions for attention-deficit/hyperactivity disorder (ADHD) Link to community care after psychiatric hospitalization or other residential care  
Care Coordination “Medical home” measures; comprehensive developmental and treatment plans; integration of information and treatment among health care providers, schools, other institutions involved in child welfare
 
  Coordination of child and parent treatment in “medical home” (e.g., treatment of a parent’s depression should be coordinated with preventive services for the parent’s child(ren))
 
  Individualized team-developed care plans, tracking of modifications, and family input
 
  Integration of information and treatment among health care providers, schools, and other human services, especially child welfare and juvenile justice
Infrastructure Integration of data systems across all systems of care, comprehensive medical/developmental record; systems for referral
 
  Mental health content in electronic and medical records
 
  Feedback systems that report reduction in symptoms, improved functioning, and corrective actions
 
  Measurement of outcomes by child’s age and race/ethnicity, including family indicators and family experience with treatment
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×

require closer adherence to guidelines associated with well-child care and would need to include information collected by the child’s provider about the parent’s mental health as well. These efforts would require additional training of providers as well as appropriate screening instruments for primary care settings. Measurement of the quality of mental health care would also need to go beyond reporting a particular diagnosis and the use of medication. Some states (such as Massachusetts) mandate routine child screening for certain medications, and measures such as the Pediatric Symptom Checklist have been endorsed by NQF. But the use of these measures has not been adopted within Medicaid or CHIP health plans.

As one interviewee stated, “We have a measure of whether kids are being followed; but really, what we really want to know is whether the child needs to be on medication in the first place. What were the symptoms? Was the diagnosis correct? Is someone measuring the symptoms to ascertain that symptoms are improving?” (Fairbrother et al., 2010).

Emerging Signs That States and Clinicians Want to Do More

The goals of quality measurement are to create national benchmarks, highlight areas of performance that need improvement, and implement quality improvement strategies. For this to happen at a national level, states will need to report measures in a consistent way so the measures can be aggregated and compared across states and with national indicators. Achieving this goal will require capacity at the national and state levels to collect, warehouse, and analyze data; the use of standard definitions and selection criteria to guide reporting; and the creation of measures that are valid, reliable, feasible, and cost-effective for use by state agencies. While many states lack these capabilities, a few are taking steps toward building the technical resources and analytical skills that address these objectives.

The use of quality measures in state-level reporting may be enhanced by greater national benchmarking efforts and efforts to achieve more transparency in the state-based reports and other information presented to AHRQ and CMS. A recent IOM report, Future Directions for the National Healthcare Quality and Disparities Reports, recommends benchmarking as a strategy to make information more forward-looking and action-oriented, including the use of more creative data display mechanisms and general organization of the data sources (IOM, 2010a).

It is questionable whether the current collection of state-specific measures for assessment and quality improvement constitutes a federal data source. The potential to build a coordinated system through the state-based Medicaid data centers, under the guidance and oversight of CMS, does exist. But if the ultimate goal is for state programs to be able to compare their performance with each other and with national benchmarks, the measures

Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×

that are used in such assessments must be valid, reliable, and feasible for use. Furthermore, the question of whether CMS can adequately influence state data collection and reporting efforts through its own rule-making authorities or legislative action is required to support such activity must be addressed.

Despite the lack of uniformity across the states in their CHIP reporting, most states appear to be ready to collaborate in efforts to achieve greater consistency. A 2009 survey found that states want to be able to compare their own performance against that of others in a national data set using common metrics and methodologies (Smith et al., 2009). The survey also found that states want to enhance their quality improvement efforts by incorporating data on quality and performance into reimbursement methodologies for health plans and individual providers (Smith et al., 2009). It is important to note that this survey found that CHIP directors were more ready to move forward than were Medicaid directors, possibly reflecting the fact that most CHIP children are enrolled in managed care plans and that HEDIS specifications exist for this population, while children enrolled in Medicaid plans are more likely to receive care through fee-for-service and primary care case management arrangements.

At the clinical level, there is growing experience with examining quality and requirements for quality improvement as part of training. Clinicians are being asked to examine their practice against national or regional benchmarks. This focus on quality improvement as part of clinical practice offers fertile ground for the introduction and use of relevant quality measures.

Monitoring of Care Transitions and a Life-Course Perspective

Understanding of child and adolescent health has evolved to embody a life-course perspective, as discussed in Chapter 2, an approach that recognizes that children are in a constant state of development; that they have different needs from health care providers at different points in their development; that disease prognosis and treatment are affected by developmental factors; and that in this unique stage of life, children are perhaps even more susceptible to environmental influences on their health and well-being than are adults (IOM and NRC, 2004) (see also Chapter 4). Yet measures of child health care are not yet capable of monitoring services within a life-course perspective.

Linking Prenatal and Pediatric Care Data

One gap that results from failing to take a life-course perspective occurs during the transition from prenatal to pediatric health care. This gap stems from both the conceptualization of what is included in the measurement of

Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×

child health and the logistics of linking medical records across systems (for example, linking hospital records with primary care records, or obstetric records with pediatric records). Complications at birth, handled by the obstetrician and hospital physicians, are not always reported to the infant’s future pediatrician or family physician. The result is that pediatricians do not always know about pregnancy complications or birth issues that could have implications for the child’s current or future health and development.

Managing and Measuring Care Transitions for Chronic Disorders

The 2006 IOM report Performance Measurement: Accelerating Improvement (IOM, 2006b) identifies multiple difficulties associated with the management and measurement of care transitions, including “the misalignment of financial incentives, the unexplored accountability, the difficulty sorting out failed ‘hand-offs’ from worsening illness, the limited utility of administrative data, and the lack of training and support for clinicians in this area” (IOM, 2006b, p. 268). These issues apply equally to both adult care and care for younger populations of children and adolescents with chronic or complex disorders. Several efforts have evolved to address these problems, frequently focused on improving the capacity to measure the quality of care for an “episode” of illness as opposed to measuring specific procedures associated with a single office visit. But defects in the transmission of information and the absence of an evidence base regarding practices that contribute to effective care transitions impede the ability to assess high-quality care.

In addition to problems associated with the management of care transitions, there are challenges in measuring the quality of the transition. Most of the existing measures involve the quality of the transition from hospital to home for adults or an elderly population. Similar work is lacking and needs to be developed for children and adolescents, not only for the transition from hospital to home care, but also for the prenatal-to-early childhood transition and the transition from adolescence to young adulthood.

Measuring the Transition from Adolescent to Adult Health Care

Because of the current fragmented system in which coverage rules and health care providers are different for children and adults, the transition from adolescence to adulthood with respect to health care is often difficult. This gap may be especially acute for adolescents with special health care needs or with chronic health care problems, who have a critical need to find new doctors that serve adults, as well as find other forms of insurance coverage (Callahan et al., 2001; Scal et al., 1999). Population surveys shed some light on this issue, showing, for example, that at age 18 or 19, when

Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×

adolescents typically lose public coverage, they often fall out of the system and experience dramatically decreased care (Adams et al., 2007; White, 2002). However, population surveys are cross-sectional and do not follow the same adolescents through the transition. Further, there is a shift in the methodology used in cross-sectional surveys at this juncture: parents answer for children through age 17, whereas individuals answer for themselves thereafter. Questions are also different on child and adult surveys, hampering comparisons across age ranges.

Measuring Across Settings, Across Multiple Domains of Care, and Across Time

Measurement of quality for a specific hospital stay or given outpatient visit is more straightforward than measurement of the overall quality of care across an episode of illness. New measures of health care quality for children and adolescents will need to track individuals over time and across multiple encounters with different care settings within the health care system. For adults, these episode-of-care measures tend to focus on curative outcomes (such as reduced hospital stays or lower rates of readmission). But this approach may need to be modified when applied to children with chronic conditions, for whom the issues are rehabilitative and functional rather than curative. Emerging work within NCQA and the American Academy of Pediatrics is focused on such efforts, for example, through the use of medical or health home measures or the new NCQA health care supervision measures (AAP, 2002; NCQA, 2010).

Measuring child and adolescent health care requires having the ability to look across visits and services to determine whether all required components of care were delivered for a particular age (Scholle et al., 2009). It also requires the ability to determine whether the appropriate combination of drugs was prescribed for a given mental illness, for example, or whether appropriate care was provided after a hospitalization (not merely whether there was a visit) or whether the child was rehospitalized for the same condition.

Current measurement systems and metrics are not capturing these vital longitudinal dimensions of care. Much of the information on health care quality comes from administrative data that measure services delivered (for example, a lead screening test), but not the outcome of the test or followup treatment. While organizations understand the potential for medical record abstraction to create such quality measures, they face numerous barriers (financial, privacy, and infrastructure) to implementation. In addition, rehabilitative and other services are delivered in non−health care settings, such as special education services and social work support services.

Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×

Adequate assessment of the quality of care would require capturing these services as well.

Measuring Quality in Preventive Care Services

The conceptualization of “health” for children embodied in Children’s Health, the Nation’s Wealth and elsewhere calls for departure from thinking of health as physical health and absence of illness and extends the concept of prevention into health promotion and well-being (IOM and NRC, 2004). Measurement of this broader concept of “health” is now reflected in cross-sectional population surveys such as the National Survey of Children’s Health, but it is generally not present in the design, organization, and financing of the health care system. Significant efforts are now under way, however, to introduce broader concepts of prevention and health promotion into the routine services offered as part of well-child visits, especially through screening and practice changes to promote healthy development for children with or at risk for developmental delays. One such effort is the Assuring Better Child Health and Development (ABCD) initiative, sponsored by The Commonwealth Fund, in which state Medicaid agencies partner with others to increase the use of such screening and practice changes for low-income children (Kaye et al., 2006).

The Child and Adolescent Health Measurement Initiative (CAHMI) conducts two surveys aimed specifically at prevention measures: the Young Adult Health Care Survey (YAHCS) and the Promoting Healthy Development Survey (PHDS). The YAHCS seeks to assess whether young adults (aged 14–18) are receiving preventive services through a 54-question survey, which can be administered via telephone or mail (an online version is in development). The results are scored according to nine measures of care quality, emphasizing preventive screening and counseling on risky behaviors, sexual activity, and emotional health, as well as private and confidential care (CAHMI, 2010). The survey has been adopted by several states (California, Florida, New York, and Washington) for use in their quality improvement efforts (CAHMI, 2010), and its results can be used to create community-specific assessments of adolescent health.

The PHDS is a family-focused survey intended to capture both provider- and parent-based data that can be used during a well-child visit and can then become part of the medical record (Bethell et al., 2001). Parents may fill out the PHDS before the visit (either by mail, in the waiting room, or online), answering questions regarding concerns about the child, anticipatory guidance, and parental education needs and providing a brief assessment of the child’s development and family risk factors. If physicians have access to EHRs, they can use a link in the EHR to review the survey results for family risk screening, family risk assessment, and priority educational

Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×

needs prior to the visit. During the visit, physicians and other clinicians use the results to prioritize and individualize the content of the visit.

The PHDS-PLUS is an adaptation of the PHDS that provides a telephone/interviewer-administered survey for parents of young children (aged 3–48 months). One study indicates that data have been collected using the PHDS-PLUS from almost 14,000 children in Medicaid programs in seven states. These data provide a basis for comparing state-based performance in 11 designated topic areas, such as assessment of concerns about child development, family psychosocial assessment, and help with care coordination (Bethell et al., 2007).

Both the ABCD initiative and the PHDS-PLUS effort have developed quality measures that states can use as baseline information systems to improve their efforts to implement preventive and developmental services for children served through Medicaid managed care plans. These measures allow states to track the use of and experience with such preventive care services (Bethell et al., 2007; Scholle et al., 2009).

In the past, quality measures for preventive care have been largely process focused, examining whether a specific service had been delivered without examining the content of the visit or appropriate follow-up care. For example, there are HEDIS measures for whether a well-child visit has taken place or whether a chlamydia screening has occurred, but such measures do not consider the outcome of the screening procedure, or whether recommended treatment services were provided in a timely and effective manner (Scholle et al., 2009). This is largely because measures are derived from the claims data generated from a single visit. In the absence of EHRs and detailed clinical data on processes of care, efforts to abstract this information encounter major issues related to feasibility and cost.

Recently, important work has been done to expand the scope and flexibility of measurement approaches, particularly with regard to well-child visits (Scholle et al., 2009). New measures have been proposed that go beyond whether a visit has taken place to encompass the content and outcome of the visit. These new proposed measures assess whether all services that are required by age have been delivered across four domains: protection of health, healthy development, safe environment, and management and follow-up of health problems (Scholle et al., 2009). The age groups are infancy to 6 months, by age 2, by age 6, by age 13, and by age 18. Box 5-3 shows as an example the elements of care that need to have occurred by age 6. What is not yet known is the extent to which adherence to these services is associated with child health outcomes.

Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×

BOX 5-3
Quality Measures for Well-Child Visits (by age 6)

Protection of Health

  • Immunizations
  • Vision screening
  • Oral health exam
  • Blood pressure assessment
  • Hearing

Healthy Development

  • Developmental screening
  • Weight assessment and counseling for nutrition and physical activity
  • Counseling on screen time
  • Parental competencies

Safe Environment

  • Environmental tobacco assessment and counseling
  • Domestic violence
  • Firearm safety
  • Vehicle safety
  • Water safety
  • Sports safety

Management and Follow-Up of Health Problems

  • Individualized care plan

SOURCE: Scholle et al., 2009.

Opportunities to Link National Databases

Improved outcomes for populations of children and adolescents may be monitored through efforts to link to more national databases. The potential to link files across two or more national databases holds promise for providing further insight into contextual factors that constitute important health influences for children and adolescents, demographic variables that may be correlated with the use and quality of health care services, important outcomes for populations of children and/or adolescents, and improvements at the community and national levels.

A National Research Council (NRC) workshop summary reviews the strengths and limitations of key national databases that serve as sources for estimates of insurance coverage for children (NRC, 2010). These databases include data collected by the American Community Survey, the Current Population Survey, the National Health Interview Survey, the Medical Ex-

Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×

penditure Panel Survey (Household Component), the Survey of Income and Program Participation, and the National Health and Nutrition Examination Survey. The report notes that “the presenters emphasized conducting targeted methodological research, building bridges between the surveys so that they could benefit from the strengths of each other, and providing data users more information for analyzing and possibly further adjusting data” (NRC, 2010, pp. 3–12).

Jurisdictional Issues Among Federal Agencies

One persistent barrier to efforts to achieve an optimal national measurement system for child and adolescent health and health care quality involves jurisdictional issues among federal agencies. The surveys described in the preceding section, for example, are conducted by separate agencies within HHS as well as other federal departments. Although a coordinating mechanism exists in the form of the Federal Interagency Forum on Child and Family Statistics, no agency is charged with leadership in striving for greater consistency and standardization in such basic areas as definitions of race and ethnicity or the inclusion of common age breaks that could facilitate comparisons across multiple surveys. Furthermore, research on the design and use of innovative measures, especially in such areas as a life-course perspective, social and behavioral determinants of health, and family-focused measures, is limited. Although various coordinating and high-level workgroups have attempted to solve these problems, their efforts have met with little success. The absence of a central registry of all federally supported longitudinal studies of children and adolescents, for example, is a sign of the limited resources and support allocated for efforts to coordinate interagency data sets.

PROMISING STATE AND LOCAL INITIATIVES

In addition to national efforts to do more with existing databases, some states and localities are experimenting with strategies to enhance their use of state and local data sources. These efforts include integrating health care data sets, as well as linking health care information with other data sources. An alternative approach is “layering” data systems through geocode mapping to highlight areas of common interest where problem behaviors tend to cluster. The feasibility of taking these initiatives to scale involves numerous questions around agency lead, infrastructure development, and resources required, among others. The 2011 IOM report For the Public’s Health: The Role of Measurement in Action and Accountability addresses these linkages in greater detail (IOM, 2011a).

Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×

Massachusetts: Pregnancy to Early Life Longitudinal Data System

Massachusetts has developed a linked research database—the Pregnancy to Early Life Longitudinal (PELL) data system. This system links maternal and infant hospital discharge records with birth and fetal death records, and further links these records to additional public health and social services databases, including Early Intervention; the Supplemental Nutrition Program for Women, Infants, and Children (WIC); and birth defects and cancer registry data (BUSPH, 2010). The linkage in the Massachusetts PELL data system has generated numerous investigations of the quality of perinatal care (Clements et al., 2006, 2007; Declercq et al., 2007; Lazar et al., 2006; Shapiro-Mendoza et al., 2006; Tomashek et al., 2006).

Indiana: Child Health Improvement through Computer Automation System

The Child Health Improvement through Computer Automation (CHICA) system at Indiana Children’s Health Services aims to strengthen parental involvement by asking parents about risks and concerns as part of a pediatric visit. Parental responses help ensure that physicians know about issues that need to be addressed and can shape the visit to make it more efficient.

CHICA is a computer-based decision support and EHR system for pediatric preventive care and disease management. Parents fill out a prescreening form in the waiting room that includes questions about risks, concerns, and reasons for the visit. The handwritten responses are then scanned and uploaded into the computer system, which generates customized items on a form used by the physician when he or she sees the patient. For example, if the parent has indicated that the child lives with a smoker, CHICA will prompt the pediatrician to discuss smoking cessation as well as the dangers of secondhand smoke (Downs et al., 2008). The information is tracked from clinic to clinic and from visit to visit.

Colorado: County Health Profiles

The Colorado Department of Health Care Policy and Financing (CDHCPF) has adopted the position that its role is to hold payers accountable for the health of the populations they serve (Wadhwa, 2010). To this end, CDHCPF uses a county health ranking model developed by the University of Wisconsin to examine the impact of policies and programs and population health factors on health outcomes. This model proposes that the physical and the social and economic environments contribute 50 percent to health outcomes, while factors within the health care system (clinical

Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×

care and health behaviors) contribute the remaining 50 percent. The social and economic environment alone (rates of education, employment, income, family and social support, and community safety) is presumed to contribute 40 percent to population health outcomes. CDHCPF collects data from multiple state and federal sources and constructs health profiles for each county within the state. These county profiles organize health conditions and health care services into three categories: on the right track, needs improvement, and major challenges. This type of innovative practice requires expertise in working with multiple population health and administrative data sources, as well as statistical methods for comparing and analyzing data trends over time for selected populations.

Rhode Island: Asthma State Plan

One compelling example of a state-based partnership that uses data from multiple sources to address a chronic health problem is the Asthma State Plan adopted by the state of Rhode Island (RIACC, 2009). This plan is the result of a collaborative effort between the Rhode Island Department of Health and the Rhode Island Asthma Control Coalition, which consists of a variety of community health organizations. Recognizing that 11 percent of children in the state have asthma, the plan draws on an integrated chronic care health systems approach to effect change. This systems approach identifies 14 asthma-specific goals within five broad categories, which apply to both adults and children and include the following examples:

  • Ensure that policies, programs, and systemwide changes are based on and evaluated using timely, comprehensive, and accurate asthma data.
  • Decrease the disproportionate burden of asthma in racial and ethnic minority and low-income populations.
  • Reduce exposure within schools to environmental asthma triggers, irritants, and asthmagens.

In developing a data system to implement various objectives under each goal, the Asthma State Plan does not seek to integrate multiple data sets, but rather relies on a strategy that designates an agency as the principal actor and specifies the data set that should be used to monitor progress toward achieving each objective. To reduce exposure within schools, for example, the Rhode Island databases used by the Department of Education provide the basis for monitoring the following key objective: “By 2014, increase the number of ‘High Performance’ schools that adopt construction, maintenance, and cleaning practices from 0 in 2008 to 20” (RIACC, 2009, p. 36).

Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×

Multiple state-level data sources are used to monitor performance, including the 2005 Rhode Island Behavioral Risk Factor Surveillance System (BRFSS), the 2008+ Rhode Island BRFSS Call-back Survey, and the Rhode Island Chronic Care Collaborative (RICCC) Asthma Database. The Rhode Island Asthma State Plan demonstrates that federal and state-based data systems can be used to support collective action and quality improvement efforts designed to address child health problems. Such efforts require dedicated financial and human resources, however, both to support the initial organizing, planning, and goal-setting efforts and to sustain the activities associated with data monitoring, analyses, and progress reports.

Philadelphia: Kids Integrated Data Set

The city of Philadelphia has established linkages among multiple data sets maintained by the departments of education, human services, law enforcement, and others. The Kids Integrated Data Set (KIDS) provides basic guidance for public officials in determining where resources can be matched with “hot spots” of vulnerable populations and neighborhoods. At present, however, health information cannot be linked effectively into the KIDS program because of legal and administrative restrictions that prevent “memorializing the link” between an individual child’s health and educational records (Schwarz, 2010).

Austin, Texas: Children’s Optimal Health

Based in Austin, Texas, Children’s Optimal Health (COH) is aimed at improving children’s health in the central Texas area through the use of geographic information system (GIS) mapping. The group is a nonprofit association consisting of approximately 50 members, including hospital systems, universities, businesses, and local agencies involved in health, education, and housing. COH does not aim to integrate separate data sources, but draws on a wide range of proprietary and public data sets that would otherwise never be shared. These data sets undergo a layering process that integrates the data without compromising the confidentiality of individual patients or the institutional data holders or violating legal restrictions such as those associated with the Health Insurance Portability and Accountability Act and the Family Educational Rights and Privacy Act. The layered data are fed into the GIS to form a succinct and powerful visualization—such as color-coded maps—of the community’s health, which identifies social determinants (such as income or education levels) and highlights influential geographic factors (such as the location or clustering of specific businesses). These maps are then used by the participating groups within

Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×

COH to suggest steps for improvement and to formulate opportunities for collaborative policies (Sage et al., 2010).

In 2008, COH initiated a GIS mapping effort to address obesity trends among middle-school-aged children in Austin. First, the group sought data-sharing agreements from nearby hospital systems, federally qualified health centers, the city’s housing authority, the Austin independent school district, organizations that offer services to youth and families, and a national health information exchange. The data sets produced from these agreements included data on students’ body mass index, cardiovascular fitness, endurance, and flexibility, which could then be mapped to the locations of specific school districts. COH also collected police incident data that, once mapped, could account for less time spent outdoors because of a parent’s or child’s reduced sense of safety. A technical advisory committee ensured that all data were standardized and deidentified and offered initial interpretations of resulting maps. The final outcomes were presented at a community summit to engage the community in analyzing the findings and develop next steps (Sage et al., 2010).

SUMMARY

This chapter has provided an overview of child and adolescent health care quality measures, emerging opportunities to improve the development and use of measures, and unresolved difficulties that continue to challenge both the measurement of quality and the delivery of high-quality care for children and adolescents. A number of factors contribute to the current state of quality measurement. For example, the committee found that the motivations for creating and using quality measures for younger and older populations differ. One reason for this difference is the absence of private-sector incentives for the measurement and improvement of health care quality in younger populations. As a result, the state of health care quality measures for youth lags far behind that for adults. The absence of private-sector activity, coupled with the compelling need to improve health care quality and population health outcomes for children and adolescents, supports the need for a stronger public-sector presence in the design, collection, use, and reporting of such measures.

As described in this chapter, federal agencies have made some progress in addressing these shortcomings through the identification of an initial core set of standardized quality measures for children and adolescents. However, the exclusion of any measure for which validity, reliability, and feasibility have not been extensively documented has resulted in the neglect of measures for important areas of health for which evidence is limited (such as mental health, substance use, oral health, and relatively rare chronic conditions). Quality measures are especially important for the content of

Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×

and follow-up on preventive and early intervention services for children and adolescents.

In addition, studies of vulnerable and marginalized populations of children and adolescents require greater attention to social and economic factors in assessing health care quality and health outcomes for children and adolescents. Some measures are available in these areas, but their implementation is limited in the absence of state and national data systems that support such measurement.

Although a number of measures of child and adolescent functional status exist, none have been accepted as “standard” measures. As noted in the previous chapter, there is no agreement on the appropriate domains for these measures, and little is known about the sensitivity of most such measures to medical care interventions.

This chapter has highlighted a number of emerging opportunities to improve the development and use of child and adolescent health care quality measures. For example, using a life-course approach to measurement provides a more comprehensive view of child and adolescent health care. A life-course perspective can inform understanding of the outcomes of preventive and early interventions, as well as the health consequences associated with early social environments. Incorporating this perspective will require longitudinal data sets that can follow population groups across episodes of care, as well as the management and measurement of care transitions across multiple settings and across time. Such transitions are especially salient for children and adolescents with special health care needs or chronic health conditions, as well as for the general population across specific life transitions, such as those from the prenatal stage to early childhood and from adolescence to young adulthood.

This chapter has provided examples of state and local efforts that encourage collaboration; foster the use of population health and administrative data sets among health care providers and their institutions and other service settings; support quality improvement practices; and inform coordinated interventions to prevent and mitigate health risk behaviors, as well as address the social and environmental contexts in which behaviors develop. These strategies can improve the timeliness of data collection and the transparency of data sources, with the ultimate goal of improving child and adolescent health care quality. However, taking these efforts to scale will require a full examination of Health Insurance Portability and Accountability Act regulations and state and local capacity to analyze, interpret, and report on data, among other issues.

Finally, additional work is needed to expand the existing collection of measures of child and adolescent health care quality. It will be necessary to underscore the need for broader availability of outcome measures across sectors; the collection and reporting of measures of social influences on

Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×

health; and the creation of measures that can follow children and adolescents across different care settings, health plans, and multiple states over time. Clinical and comparative effectiveness research and the Centers of Excellence (U18) awards authorized by the CHIPRA legislation offer two important opportunities to build the evidence base for health care access and quality measures and to fill critical gaps, especially those gaps that address the specific characteristics and needs of younger populations. New initiatives associated with HIT and the creation of EHRs also offer substantial opportunities to foster the incorporation of children and adolescents into efforts to build the next generation of data sources and data collection methods. However, these efforts by themselves are unlikely to achieve this objective. Ultimately, greater alignment among federal agencies concerned with technology and quality measurement will be necessary.

Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
Page 135
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
Page 136
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
Page 137
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
Page 138
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
Page 139
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
Page 140
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
Page 141
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
Page 142
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
Page 143
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
Page 144
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
Page 145
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
Page 146
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
Page 147
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
Page 148
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
Page 149
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
Page 150
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
Page 151
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
Page 152
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
Page 153
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
Page 154
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
Page 155
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
Page 156
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
Page 157
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
Page 158
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
Page 159
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
Page 160
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
Page 161
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
Page 162
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
Page 163
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
Page 164
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
Page 165
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
Page 166
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
Page 167
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
Page 168
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
Page 169
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
Page 170
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
Page 171
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
Page 172
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
Page 173
Suggested Citation:"5 Measures of Quality of Child and Adolescent Health Care." Institute of Medicine and National Research Council. 2011. Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: The National Academies Press. doi: 10.17226/13084.
×
Page 174
Next: 6 Conclusions and Recommendations »
Child and Adolescent Health and Health Care Quality: Measuring What Matters Get This Book
×
Buy Paperback | $50.00 Buy Ebook | $39.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Increasing public investments in health care services for low-income and special needs children and adolescents in the United States have raised questions about whether these efforts improve their health outcomes. Yet it is difficult to assess the general health status and health care quality for younger populations, especially those at risk of poor health outcomes, because the United States has no national information system that can provide timely, comprehensive, and reliable indicators in these areas for children and adolescents. Without such a system in place, it is difficult to know whether and how selected health care initiatives and programs contribute to children's health status.

Child and Adolescent Health and Health Care Quality identifies key advances in the development of pediatric health and health care quality measures, examines the capacity of existing federal data sets to support these measures, and considers related research activities focused on the development of new measures to address current gaps. This book posits the need for a comprehensive strategy to make better use of existing data, to integrate different data sources, and to develop new data sources and collection methods for unique populations. Child and Adolescent Health and Health Care Quality looks closely at three areas: the nature, scope, and quality of existing data sources; gaps in measurement areas; and methodological areas that deserve attention.

Child and Adolescent Health and Health Care Quality makes recommendations for improving and strengthening the timeliness, quality, public transparency, and accessibility of information on child health and health care quality. This book will be a vital resource for health officials at the local, state, and national levels, as well as private and public health care organizations and researchers.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!