Innovations in Design and Utilization of Measurement Systems to Promote Children's Cognitive, Affective, and Behavioral Health: Workshop Summary (2015)

2

Maximizing the Value of National, State, and Local Measurement Systems

The workshop began with four presentations that featured examples of the successful use of data integration to inform research and policy pertinent to children’s cognitive, affective, and behavioral health. These examples extended from the local level to the state level to the national level, but all had several features in common. They relied on data gathered, combined, and disseminated in unusual ways to better understand the problems facing children and ways of overcoming those problems.

DATA ON CHILDREN’S HEALTH AND WELL-BEING IN THE STATE OF MARYLAND

Maryland is a state of about 5.8 million people, explained Anne Sheridan, executive director of the Maryland Governor’s Office for Children. Of the 857,000 children in the state’s K–12 public education system, 43 percent are eligible for free and reduced price meals. With two large urban areas—Baltimore and the suburbs of Washington, DC—the state is rich demographically, with about 55 percent whites, 29 percent African Americans, 8 percent Latinos, and 5 percent Asians. The state also has a strong county government system, with 23 counties plus the City of Baltimore, “so everything that you do, you need to automatically multiply by 24,” said Sheridan. The state’s economy is grounded in its academic and educational institutions, its proximity to Washington, DC, its medical institutions, and its thriving hospitality and tourism industry—which Sheridan referred to as “eds, beds, feds, and meds.”

The state now has more jobs than before the recession that began in 2008, and it has a relatively high median income. But it also has persistent pockets of poverty and challenges for many families, which has a great impact on children’s health and well-being.

Maryland has a Governor’s Office for Children and a Children’s Cabinet consisting of the heads of the departments responsible for child welfare, education, health, disabilities, juvenile services, and the budget. Moreover, unlike children’s cabinets in other states, the one in Maryland has a dedicated line item in the state budget devoted to early intervention and prevention programming in each of its jurisdictions. The cabinet functions as a grant maker. It asks jurisdictions to think about the highest needs in their areas, and funds are distributed to a wide variety of programs according to those needs. Historically, it has focused on children with the highest needs, who are also at risk of involvement in multiple state systems. “Not surprisingly,” said Sheridan, these are “the children and families who also cost the state the most money.”

The state uses two major data structures to measure improvements in children’s health and well-being. The first is a results and indicators framework that has been used for many years to guide Children’s Cabinet programming (see Figure 2-1). This approach focuses on planning, decision making, and budgeting around desired results or outcomes. It has 25 indicators that are associated with particular results, which enable data to be gathered that can be used to determine what is driving particular outcomes. The framework, which is publicly available, establishes shared goals, aligns grant making, and makes children a priority. The system was “a game changer when it was introduced 14 years ago,” said Sheridan. It also is fairly simple and accessible. “We could train you up on it in an afternoon and feel like you’d have the basic principles.” It has limitations in being produced only on an annual basis, not being a powerful tool for day-to-day management, and not being comprehensive. But “even with those limitations, it’s proved to be valuable over time.”

The second data structure is a statewide process called State Stat that was introduced by Governor Martin O’Malley after he was elected in 2006. Based on a similar process developed in New York City to track crime, State Stat sets clear expectations about statewide goals. Four areas fall under the purview of the Children’s Cabinet:

• End childhood hunger in Maryland by 2015.
• Improve student achievement and school, college, and career readiness by 25 percent by the end of 2015.
• Reduce violent crimes committed against women and children by 25 percent by the end of 2018.
• Reduce infant mortality in Maryland by 10 percent by 2017.

FIGURE 2-1 The state of Maryland’s indicators and results framework. The framework has indicators associated with a variety of positive outcomes.
SOURCE: Sheridan, 2014, adapted from Children’s Cabinet and Governor’s Office for Children, 2012.

The State Stat process is a rigorous, data-driven procedure in which the state officials responsible for these goals convene, review the information, and take a “deep dive” into the things that are contributing to or detracting from success, said Sheridan. The process is also designed to foster collaboration and transparency so problems can be identified and solved.

As an example of how information is used, Sheridan discussed the children’s hunger goal. Maryland has had a number of long-running child nutrition programs, but for a variety of reasons the programs remained underutilized. For example, every child who receives a free and reduced-price lunch should get a school breakfast, which contributes greatly to their ability to pay attention, to do well in school, and to remain healthy. To increase the use of the school breakfast program, the state convened a number of players at the state level, foundations, and corporate partners, and this group together mounted an outreach program that has encouraged schools to make breakfast part of the school day. From 2007 to 2008 and from to 2014 to 2015, the number of children eligible for a free or reduced-price lunch who ate a free or reduced-price breakfast at school every day rose from 44.6 percent to 60.9 percent. About 400 schools have introduced some sort of in-classroom grab-and-go breakfast, providing greater flexibility for students to eat breakfast, and about 75,000 more children are getting free and reduced-price breakfasts than was the case before.

A major obstacle in this effort, Sheridan said, was getting the county-wide and school-level data about breakfasts from the Maryland State

Department of Education. To some extent, the needed data were not being collected, and to some extent attitudes limited the release of data, noted Sheridan. Improvements on both fronts have resulted in school-level data mapped across the state. “You can go to the State Stat website, if you’re interested in how we’re doing on this goal, and pull your school down,” said Sheridan. “You can see how it’s doing, [which] is also a factor in communicating success, attracting more supporters for this enterprise, and hopefully showing what were the longer-term impacts of us doing a better job connecting kids to school breakfast.”

Analysis of the data showed that an increase in school breakfasts was linked to an improvement in test scores and attendance, with attendance linked to longer-term success. “We are able to make pretty definitive statements about the value proposition for making these investments,” said Sheridan. The result has been a “virtuous cycle of improvement, progress, and success . . . that’s really helped us.”

The ability to engage outside partners was “a big part of what drove the success of the school breakfast campaign,” said Sheridan. “Every big issue that the Governor’s Office for Children has worked on, its progress has been made possible by having [a] national partner. On our hunger goal, it’s been Share Our Strength. On our out of school time, it’s been largely the Charles Stewart Mott Foundation, which is building a statewide network.”

States will continue to be innovators and incubators of successful programs in the future, said Sheridan. “I don’t think the United States Congress is going to be offering a lot on that front for a number of years.” A major challenge at the state level is to make it safer and more attractive to invest in prevention. “It costs less to stay at the Four Seasons every night than it does for us to place you in a hardware-secured juvenile detention facility, but I am having a hard time getting folks to say that $2,000 a year for high-quality afterschool programming is a good investment. Need help here.” Policy makers are deluged with data, some of which is helpful and some of which is not. Data systems that are sustainable across administrations, so they do not change when a new administration is elected, can help align budgeting, messaging, and outcomes, Sheridan concluded. Also, she noted, “The difference between a dream and a goal is a deadline.”

“To improve outcomes for kids is an investment, and you have to look at it that way,” said Sheridan. “The more we can make data-driven arguments that X investment here leads to this outcome, . . . the better off we’re going to be.” At the local levels of government, people especially care about their own neighborhoods and communities, and this drive can be carried through to more broadly based levels of government as well.

HEALTH CARE EXPENDITURES FOR CHILDREN WITH DISRUPTIVE BEHAVIOR DISORDERS IN MEDICAID/CHIP

The Center for Medicare & Medicaid Innovation (CMMI) within the Centers for Medicare & Medicaid Services (CMS) conducts many of the innovative projects undertaken by CMS throughout the United States. Vetisha McClair, a research analyst at CMMI, described two intramural research projects conducted by members of this team as examples of the kinds of data on children’s health and well-being that are available and how they can be used.

Medicaid and the Children’s Health Insurance Program (CHIP) are together the largest source of health care for children in the country. Medicaid, which covers primarily low-income children, is a partnership between the federal government and states, and the benefit packages differ from state to state. CHIP is available in all states and it covers children from higher income families who are not eligible for care under their parent’s private insurance or whose parents lack insurance.

Few studies have examined the expenditures for children on Medicaid/ CHIP with disruptive behavior disorders (DBDs), said McClair. DBDs consists largely of conduct disorder, which is characterized by disregard for societal norms and the rights of others; oppositional defiant disorder, which is characterized by persistent patterns of anger and breaking rules; and are often comorbid with attention deficit hyperactivity disorder (ADHD). These conditions can become a large burden on both familial and health care resources and can significantly limit the educational experiences of children. Also, some children diagnosed with DBDs experience impairment that persists into adulthood, including the development of antisocial personality disorder.

A group of researchers within CMMI conducted a study, using health care claims data, to assess disease prevalence and expenditures for the treatment for DBDs among Medicaid/CHIP beneficiaries. The data came from Medicaid Analytic eXtract (MAX), which is a set of person-level and claim-level data files compiled annually that contain Medicaid eligibility, service utilization, and program payment information derived from state reporting. Eligibility files were combined with health care claims from in-patient, long-term care, prescription drug treatment, and other claims to develop an extensive dataset of information about the health care that children receive throughout the United States.

Medicaid claims data are very useful in examining national trends and intra-state health care policies. However, comparisons between individual states are not generally recommended, said McClair, due to differences in benefits and data quality. Individual states determine the benefit packages and services covered, therefore one cannot be ensured of uniformity in services between individual states.

The study was able to extract and examine claims data for 46 states and the District of Columbia, though the California and New York analyses were still under way at the time of the workshop because of the difficulties of analyzing such large samples. In addition, Maine and Alabama were excluded because of data inconsistencies. For children and teens under 20 with at least one diagnosis in a health service claim of conduct disorder, oppositional defiant disorder, and/or ADHD, outcomes were analyzed by sex, age, race and ethnicity, state, and delivery type (fee for service or managed care). Outcome measures included both prevalence in the study sample and expenditures.

The study found that the prevalence of DBDs among Medicaid beneficiaries increased slightly over the period studied, from 4.7 percent in 2006 to 5.3 percent in 2009. Within racial and ethnic groups, the prevalence of DBDs remained relatively stable, with whites and Hispanics/Latinos having a substantially higher prevalence than blacks (McClair et al., 2014). Over the 4-year period, the number of children eligible for Medicaid/ CHIP increased from 24.8 million to 27.7 million. As a result, the total Medicaid payment for beneficiaries with a DBD increased. However, the total Medicaid payment for DBD treatment remained more or less flat, and the proportion of expenditures devoted to mental health among all services addressing mental health issues decreased slightly over this period, McClair said.

She also described a related study on the use of prescription drugs among Medicaid beneficiaries. Individuals with a DBD have high rates of prescription drug use, and the number of these individuals who use prescription drugs went up 25.6 percent between 2006 and 2009. The total number of prescriptions also increased by 27.2 percent from 2006 to 2009, and the Medicaid payments increased by 28.8 percent, with the payments per beneficiary staying largely the same (Zhao et al., 2014). A large portion of drugs prescribed to children with DBDs were psychotherapeutic drugs, and this proportion has remained relatively stable for all beneficiaries and within racial subgroups.

McClair drew several conclusions from these data. First, these results could be explained by a number of exogenous factors, including delivery system reform and state-level changes to Medicaid/CHIP benefits.

Second, further research is needed to address not only the costs associated with the treatment of children with DBDs but how expenditures may vary by state and how those monies are allocated.

Finally, Medicaid data are a viable source of information about the state of health care for vulnerable populations, such as low-income children. Understanding the most current information available on medical expenditures for children with these conditions is crucial to instigating changes in the health care system that promote increased access to care

while containing costs, she said. “We need to continue research in this area and continue to use Medicaid data for both research and policy purposes.”

In response to a question, McClair observed that Medicare has always attracted much of the attention within CMS, resulting in less focus on children’s issues. But the Innovation Center has been tasked to look at specific populations and problems that drive costs, which has laid the groundwork for policy changes and new programs. For example, one potential area of focus in the future could include how CHIP may cover a child in a family but not cover a parent who has a substance abuse problem that is affecting the child. Innovative cross-agency programs could address problems within families in a more coordinated way, she said.

MULTIPLE SERVICE SYSTEMS USE AMONG ILLINOIS FAMILIES

Chapin Hall, which was an organization created to take care of orphans of the Civil War, was brought to the University of Chicago 30 years ago to have an impact on the well-being of children through better research and analysis. It has particularly sought to achieve that goal by building relationships with policy makers and by conducting research that meets the needs of the public sector. Known for its work with government social program administrative data, it employs a full range of research methods and addresses many issues relevant to children, families, and their neighborhoods.

Robert Goerge, senior research fellow at Chapin Hall, pointed out that subgroups of families are at highest risk of their children experiencing adverse childhood experiences. These families tend to be characterized by a combination of attributes:

• Chronically unemployed parents
• Very low socioeconomic status
• Long-term welfare program participants
• Single-parent families
• Mothers who had their first child as an adolescent
• Families without grandparents living with them
• Alcohol, drug use, mental illness
• Inadequate parenting skills, family breakdown, parental stress and mental illness, domestic violence, and parental history of maltreatment during childhood
• Community-level low socioeconomic status, living in an impoverished community, family size, and sibling spacing

To learn more about these families, Chapin Hall has created an Integrated Database on Child and Family Programs in Illinois that combines data from law enforcement, foster care and child maltreatment investiga-

FIGURE 2-2 The Integrated Database on Child and Family Programs in Illinois. The database combines data from a number of social service agencies.
NOTE: AABD = Aid to the Aged, Blind, and Disabled; CPD = Chicago Police Department; IDB = Integrated Database; SNAP = Supplemental Nutrition Assistance Program; SSI = Social Security Income; TANF = Temporary Assistance for Needy Families; UI = Unemployment Insurance; WIA = Workforce Investment Act.
SOURCE: Goerge, 2014.

tions, Medicaid programs, welfare programs, employment records, and education (see Figure 2-2). The data span the period from 1977 to 2014, but most are from 1990 forward.

The process of making administrative data suitable for research use includes three important steps¹:

__________________

¹ For discussion on privacy and use of personal data, see Chapter 6.

• De-duplication, or identifying records within a dataset that belong to the same individual;
• Record linkage, to identify and link individuals across datasets, which may require probabilistic record linkage; and
• Identifying relationships among individuals.

Administrative data have a variety of advantages and disadvantages compared with survey data, Goerge pointed out (see Table 2-1).

The data collected by Chapin Hall demonstrate that a small number of families use a large portion of the state’s highest-cost services, including expensive and intensive health, human service, and corrections programs. Families were defined as individuals who were “linked” through membership in the Supplemental Nutritional Assistance Program or Temporary

TABLE 2-1 Advantages and Disadvantages of Surveys and Administrative Data

SOURCE: Goerge, 2014, adapted from Wallgren and Wallgren, 2007.

Assistance for Needy Families cases, or were involved in a Department of Children and Family Services case; problems were defined as an instance of mental health service, substance abuse treatment, adult incarceration, juvenile incarceration, or foster care placement. Multisystem families whose members had at least two different types of problems, which represent 23 percent of families, account for 63 percent of system involvement and 86 percent of costs of foster care, Medicaid, and adult and juvenile incarceration (see Figure 2-3). If the state could develop a deeper understanding of the circumstances in which these families live, their histories, their geographic location, and their trajectories, along with what services they have used, it could provide more adequate and efficient services, Goerge observed.

Analysis of the data also revealed the percentage of multisystem families by county across the state, with the southern part of the state having the highest prevalence of multisystem families. Within the city of Chicago, some of the 77 Chicago area communities that sociologists have been studying for many years had up to 40 percent of the study population in multisystem families. At an even more detailed geographic level, some census tracts have more than 60 percent of children living in multisystem families, with some tracts above 90 percent. Furthermore, the percentage of children living in multisystem families is tightly correlated with education failure, Goerge noted.

The most common problem experienced by families was mental health service, accounting for about 221,000 problems among the total of 465,000

FIGURE 2-3 Costs of services for multi-system families (MSFs). Less than one-quarter of the families account for 86 percent of the costs of services.
SOURCE: Goerge, 2014.

problems experienced (see Figure 2-4). Among families with multiple problems, 94 percent had received mental health services. More than half of these families (54 percent) received both in-patient and out-patient services, which surprised the researchers, who expected more of the services to be out-patient. For substance abuse treatment, 42 percent of those who received any type of services were receiving in-patient services. “For about half of these families, it’s a very intensive and costly service,” noted Goerge.

The biggest surprise involved child abuse or neglect and intentional injury. Of the multisystem families, 81 percent had reported child abuse or neglect or a report of an intentional injury from the Medicaid claims data. “You wonder where the violence in Chicago comes from,” said Goerge. “Its source is these children experiencing tremendous amounts of violence in their homes. That’s why they’re on the streets.”

Future research directions for multisystem families include identifying unique characteristics of such families in specific places, discovering the problems that may have the greatest impact on the current state of the family, revealing whether single individuals account for multiple problems within a

FIGURE 2-4 Types of problems experienced by families. Mental health was the most common problem for families.
NOTE: The total number of problems (465,036) exceeds the total number of families with problems (285,722) because some families have multiple problems.
SOURCE: Goerge, 2014.

family or whether several family members encounter problems, delineating between families that have had few service spells versus families with multiple service spells, estimating the cost of providing state services to multiple-problem families, looking at asthma and other chronic conditions, examining assets such as employment and education, and conducting longitudinal analysis to determine when these families became multisystem families.

Turning from this example to broader issues, Goerge observed that states and cities are developing their administrative data sources faster than ever, and they are using the data for many different purposes. They also are making the data public, enabling data entrepreneurs to create apps that inform the public and policy makers. In addition, a number of federal initiatives are promoting the development of administrative data.

However, this is happening to different degrees in different cities, counties, and states, and there is a wide variation in who has access to the data and in the quality of the data. Best practices have not been disseminated to a sufficient extent, Goerge said, and state and cities are taking years to develop these efforts. States often rely on large corporate vendors, and these vendors typically go only so far—for example, they generally do not give states the ability to track individuals across systems. Meanwhile, government agencies do not have the skilled staff necessary to take full advantage of the efforts. In addition, state bureaucracies often want to hold on to their data, said Goerge. “They are more likely to share their data with us than with each other.”

Special interests promote the perspective that problems can be addressed one by one, Goerge observed, but everyone knows that early nutrition and good parenting is associated with learning, learning is associated with getting a job, a parent having a job is associated with child well-being, and lack of school success is associated with criminal behavior. This is why breaking down silos is necessary to make policy and development programs to improve the well-being of individuals and families, he said. Goerge reiterated a point made by McClair—that no one agency or group is responsible for all of the problems facing individual families. “Our policy in the United States is not family focused. Our policy in the United States is we do something when a child or adult is ready to hurt somebody else or hurt themselves.”

Thirty years ago, when there were less data, most public-sector agencies had handfuls of analysts, but now they are fortunate to have one. At the same time, the federal government is requiring evidence-based practice and evaluation in many areas of social programs, which is a major challenge, given the lack of research expertise in these agencies. Importantly, data-sharing agreements are becoming more complicated as identity theft becomes more prevalent, as federal regulations proliferate, and to the extent that sometimes leaders and legal teams can view information as power and perceive that data sharing could potentially lead to negative media. Goerge noted that much change happens incrementally through negotiations be-

tween government, not-for-profit agencies, and health care providers over the contracting and budgeting processes. The more that people have information about what is happening to children and families, the more this behind-the-scenes work yields progress. Contracts are the easiest way to work with government, even though universities are generally concerned that they limit academic freedom. Furthermore, it is hard to justify using state dollars to fund evaluations, so the few evaluations that are done tend to be federally funded.

The greatest need, Goerge concluded, is to better connect academics and government. Government currently does not have enough human capital to link to the researchers who can help. In addition, barriers keep data from flowing to those who need the information. The nature of these barriers varies from sector to sector and place to place, but there are common themes. Incentives have to be put into place for all jurisdictions to use their data to get smarter about what they are doing, Goerge emphasized, and all federal research projects should be reviewed so that they are effectively using administrative data.

USING DATA TO IMPROVE PATIENT-CENTERED OUTCOMES: A PERSPECTIVE FROM THE MEDICAID MEDICAL DIRECTORS

The Medicaid Medical Directors Network, which started as a knowledge transfer project of the Agency for Healthcare Research and Quality, has been meeting since 2005 and is now part of the National Association of Medicaid Directors. Situated at the intersection of practice, policy, and payment, it engages in collaborative work to achieve its vision of “improving the lives of Medicaid patients through excellent health stewardship.”

In the keynote address of the workshop, the director of the network, Jeff Schiff, who is medical director for the Minnesota Health Care Programs at the Minnesota Department of Health Services, described three projects of national significance that the network has undertaken. The first was on the use of psychotropic medicine among children. In 2007, pooled results of state analyses indicate that far more children in the foster care population received a psychotropic prescription than in the general population.² This finding has led to state policies and programs directed toward the use of such medications among this population, Schiff said.

The second project was on hospital readmissions. Trudnak et al. (2014) found that 9.4 percent of all hospital admissions among the Medicaid population in 19 states were 30-day readmissions. They also found a 10.9 percent mental and behavioral health readmission rate among children ages

________________

² More information is available at http://rci.rutgers.edu/~cseap/MMDLNAPKIDS.html (accessed March 25, 2015).

12 and younger, and a 11.7 percent rate among those ages 13 to 20. “In Medicaid, that’s obviously real dollars,” said Schiff.

The third project, which he described in greater detail, involved early elective delivery. Slightly less than half of all births in America are paid for by the Medicaid program, and early elective deliveries, which have been associated with various negative cognitive outcomes later in life, account for nearly 9 percent of births paid for by Medicaid (Fowler et al., 2014). Early elective deliveries have become a concern for the American College of Obstetrics and Gynecology, the Collaborative Improvement and Innovation Network of the Maternal and Child Health Bureau, and the National Center for Health Statistics of the Centers for Disease Control and Prevention (CDC). The Medicaid Medical Directors Network set about getting other organizations involved in the issue, first to improve birth outcomes and second, as a proof of concept that this kind of quality work can be done as a state-led national project.

The research process was to agree on research questions based on input from the state Medicaid medical directors and staff, to define data parameters with the state data staff, to set up data submissions, to analyze the resulting data, and to report the results and make the link to policy. One interesting thing about this process is that the states are motivated by comparing their results with other states, Schiff noted. Similar to quality improvement processes at the provide level, reporting confidentially to states offers opportunities for state level discussions and subsequent policy actions. Additionally, by using an existing data stream for improvement creates an atmosphere where states strive to improve the quality of their data, Schiff added. Though the results of the study were embargoed at the time of the workshop, Schiff noted that the research did find significant numbers of early elective deliveries.

Schiff drew several conclusions from the studies he described:

• Sharing information so that states can compare their results to the national aggregate tends to spur quality improvements at the state level.
• Involvement in the research at the state and local levels creates ownership of the data and quality improvement.
• Measurement and quality improvement occur best when the results are relevant to families and frontline providers.
• State-level quality improvement can be linked to state policies.
• Data and quality improve together. “The data will never be perfect,” said Schiff. “We have to make decisions based on the data we have.”
• Resources and a resourced infrastructure enable such projects to be undertaken.

Schiff also pointed to several criteria for the development of measures:

• Importance
• Scientific soundness, including clinical soundness (process to outcome relationship) and measurement soundness (reliability and validity)
• Usability
• Feasibility of data collection

In addition to these criteria, he emphasized the importance of involving the people who are getting measured. Such involvement can avoid “measurement fatigue” that is felt by many providers and can help measures be adopted. Schiff also noted that the involvement of data experts can bring energy, excitement, and expertise to a project.

Data linkage can be a challenge with Medicaid and vital records data. Medicaid records include claims, service data, and data that come in through managed care. They also include enrollment data on such factors as race and level of poverty. Timeliness can be an issue, because old information is not always actionable, and data are not always of high quality. Deciding on identifiers, matching records, establishing common definitions, and linking data involve a variety of processes and actors, and all of these issues need to be addressed, said Schiff.

Data linkage also provides an opportunity to improve data, Schiff pointed out. For example, linkages could make it possible to incorporate patient-reported outcomes into the data. Electronic health records, even knowing one small piece such as body mass index is available, would help with monitoring data.

In conclusion, Schiff listed several possible topics for the forum to consider:

• Foster care and educational outcomes
• Autism treatment and outcomes registry
• ADHD diagnosis and educational outcomes
• Psychotropic medications and race/ethnicity
• Behavioral health screening and mental health services
• Social complexity and improved well-being

All of these issues are vertically relevant at all levels of the system. But pursuing any such set of issues will require an infrastructure for project development and process steps, the time of state data and policy staff members, and integrated support, including network maintenance.

REFERENCES

Children’s Cabinet and Governor’s Office for Children. 2012. Maryland’s results for child well-being. Baltimore, MD: Maryland Children’s Cabinet, Governor’s Office for Children.

Fowler, T. T., J. Schiff, M. S. Applegate, K. Griffith, and G. L. Fairbrother. 2014. Early elective deliveries accounted for nearly 9 percent of births paid for by Medicaid. Health Affairs (Millwood) 33(12):2170-2178.

Goerge, R. 2014. Multiple service systems use among Illinois families. Presented at Institute of Medicine and National Research Council Workshop on Innovations in Design and Utilization of Measurement Systems to Promote Children’s Cognitive, Affective, and Behavioral Health, Washington, DC.

McClair, V.L., C. Cross-Barnet, S. Chan, W.D. Clark, and L. Zhao. August, 2014. Healthcare Expenditures for Children with Disruptive Behavior Disorders in Medicaid/CHIP. Poster presented at the Annual Convention of the American Psychological Association, Washington, DC.

Medicaid Medical Directors Learning Network (MMDLN) and Rutgers Center for Education and Research on Mental Health Therapeutics. 2010. Antipsychotic Medication Use in Medicaid Children and Adolescents: Report and Resource Guide from a 16-State Study. MMDLN/Rutgers CERTs. Publication #1. Distributed by Rutgers CERTs at http://rci.rutgers.edu/~cseap/MMDLNAPKIDS.html (accessed March 25, 2015).

Sheridan, A. 2014. Maximizing the value of national, state, and local measurement systems for research and policy. Presented at Institute of Medicine and National Research Council Workshop on Innovations in Design and Utilization of Measurement Systems to Promote Children’s Cognitive, Affective, and Behavioral Health, Washington, DC.

Trudnak, T., D. Kelley, J. Zerzan, K. Griffith, H. J. Jiang, and G. L. Fairbrother. 2014. Medicaid admissions and readmissions: Understanding the prevalence, payment, and most common diagnoses. Health Affairs (Millwood) 33(8):1337-1344.

Wallgren, A., and B. Wallgren. 2007. Register-based statistics: Administrative data for statistical purposes. Chichester, UK: Wiley.

Zhao, L., V.L. McClair, C. Cross-Barnet, S. Chan, and W. D. Clark. June, 2014. Disruptive Behavior Disorders in Medicaid-enrolled Children: Disease Prevalence and Medication Expenditures. Poster presented at AcademyHealth’s Annual Research Meeting, San Diego, CA.