Page 6
2
Overview of the Workshops
The workshops provided a forum for cross-fertilization of ideas and experience from differing vantage points, including those of experts in early childhood development and education; researchers; federal, state, and community officials engaged in making or implementing policy, including those in regulation and licensure; and early childhood care and education professionals. An overarching theme of the workshop was the need to move toward greater consensus on what should be measured, how it should be measured, and how the resulting performance data should be used in shaping policies and practices. Workshop participants emphasized the need for the voices and views of the early childhood field to be heard as performance measures are rapidly taking hold. As concerns about accountability for use of public funds in early childhood programs have increased, a variety of objectives and assessment strategies have emerged such as those for school readiness. And, under welfare reform, the Personal Responsibility and Work Opportunity Reconciliation Act (P.L. 104-193, 1996), other sets of performance indicators, including some for child care, have emerged.
This overview summarizes the key themes that emerged from the two workshops. First, it discusses the context in which performance measures are being defined and used and the challenges and tensions in the field about measurement and assessment. Next, it summarizes the general process for developing a performance measurement system, with examples of innovative state efforts highlighted in boxes throughout this section. Some
Page 7
of the specific concerns about assessing young children and the use of child outcomes as a measure for child care and education programs are then discussed. The overview also draws on experiences in assessing programs for children with special needs as well as the issue of cultural sensitivity in assessing child care programs. This is followed by examples of research on child care that relates to performance measures and issues of quality. The overview concludes with participants' ideas for next steps that could be taken to develop a useful system of performance measures.
CONTEXT AND CHALLENGES
As a first step, distinctions were drawn between performance measurement and program evaluation. While they are related, they are not identical processes, said Clara Pratt, Oregon State University. Performance measurement seeks to show the extent to which planned activities, outputs, and outcomes were achieved. Program evaluation and research, on the other hand, seek to establish causality and show that outcomes are the direct result of an intervention. Research is critical for identifying the probable causal relationships between teacher practices and children's learning and development, for example. Once research has established such linkages, then performance measures can track activities and outcomes achieved. Performance measurement needs to be informed by research (Pratt et al., 1999). In general, participants viewed performance measurement as part of the larger goal of improving the quality of care for young children. Standards for accreditation developed by professional groups contribute to the drive for quality as well, but they are purely voluntary. In addition, states use licensing and regulation to assure a level of quality, but participants noted that licensing generally represents minimum standards, although states vary in the stringency of their regulations, inspection, and enforcement (see Box 2-1).
Because there is no overall system of child care, the workshop necessarily focused on specific parts of the child care universe as a starting point, extrapolating lessons from experience with programs such as Head Start, center-based programs, and licensed family day care homes. Clearly these programs involve only a small proportion of children in care, with many more in informal child care by relatives, friends, and neighbors. (For a discussion of the District of Columbia's efforts to reach informal child care settings, see Box 2-2.) Some felt that focusing on measuring one component of the child care world, for example, publicly subsidized programs,
Page 8
BOX 2-1North Carolina and Quality Child Care
Workshop participant Stephanie Fanjul explained that North Carolina has an integrated and comprehensive system with both funds and regulatory functions for child care programs located within one office, the North Carolina Division of Child Development (NCDCD). In North Carolina almost all child care programs are licensed and regulated. Part of this success is due to the fact that NCDCD has created several initiatives for increasing quality without passing on additional expenses to parents. One such initiative is the NCDCD's Teach program. The program rewards early childhood educators if they obtain additional education. Scholarships are provided and once participants are enrolled in the continuing education program, they receive a bonus. The initiative has been extended further with the state's Wages program. In the Wages program child care providers, teachers, and directors are rewarded significantly for staying in their jobs with a wage stipend that can be as much as $3,000 per year. In addition, another indirect subsidy program, the North Carolina Cares Project, pays for health insurance for child care workers. These programs increase quality by providing security for the work force. These initiatives require significant planning. Programs have to give their employees the time to complete additional education and to promise and deliver certain bonuses as employees move up the ladder. The initiatives seem to be working very well in urban areas, she reported; when programs fully implement the initiatives, employees stay longer. And programs are reimbursed at a higher rate by the state because they qualify for a higher rating. In poor, rural communities the program has been less successful because of lack of access to education and a lower wage structure. State officials are implementing new ways to work with rural areas to help these efforts take hold. (For more information, please contact < www.dhhs.state.nc.us/dcd>.) |
can lay the groundwork for broader measures that can be used in other settings. Others noted that the lack of standards as a whole for child care posed obstacles not faced in programs such as Head Start, which has a clear set of standards as well as a clear funding stream. (See Chapter 4 for a discussion of Head Start and Early Head Start.)
Page 9
BOX 2-2Quality Initiatives in the District of Columbia
The District of Columbia initiated three approaches to improve quality in child care programs. One is aimed at reaching relatives who provide child care services, another focuses on family child care, and a third on early intervention programs. In 1998, the city started an outreach program to monitor child care providers, according to Barbara Kamara, of the District of Columbia Office of Early Childhood Development. For in-home and relative child care providers, the program for informal care providers consists of a monthly networking-group meeting of these providers, a bimonthly newsletter, and home visits every other month. The city monitors licensed family child care providers through a partnership with the public library system, in which a specially assigned librarian visits 20 licensed family child care homes on a monthly basis for a year. During the visits, the librarian focuses on how to organize the home environment, on use of educational and library resources, and on providing referral services for children and families. The librarian also supplies the child care providers with a newsletter developed by the library. The librarian also arranges visits to the neighborhood library for the providers and the children in their care, where activities are provided for the children while the care providers receive training in different teaching techniques. In addition to this training, a 24-hour training hotline is available for providers who would like some help but are not able to go to a library. For the early intervention program, a monitoring team makes yearly site visits. The team consists of early intervention program staff, members of the interagency coordinating council on early intervention services, and parents of children who are receiving the services. Providers are notified a month in advance of a site visit. During the visit, the review team monitors every aspect of the program—operations, development, and proper implementation of individual family service plans. They also check building procedures and fair or “last-resort” safeguards. (For more information please contact < www.washingtondc.gov/mayor/safe_passages/child_ready_learn.htm>.) |
Page 10
STEPS TOWARD DEVELOPING PERFORMANCE MEASURES
Determining the expectations for child care requires input from a broad range of groups, including parents, providers, researchers, early childhood experts, and community representatives. Participants noted that child care was inherently value-laden work and that goals for performance measurement must consider parental choice, family involvement, and cultural diversity. At the workshop, a number of presenters described experiences with developing performance measures at the national, state, and local levels. Each of these involves different stakeholders who seek different levels of information, and each has its own set of definitional and measurement issues. ( Chapter 4 summarizes examples presented at the workshop of initiatives at these different levels.) In brief, developing a performance measurement system could include the following steps:
-
Canvassing stakeholders on goals and baseline expectations for child care that meet parent and community expectations.
-
Selecting outcomes and outputs, both process and child-related.
-
Developing or selecting instruments that are valid and reliable.
-
Developing procedures that are efficient and realistic in cost.
-
Deciding on the level of aggregation of data.
-
Determining how programs and children will be sampled.
-
Analyzing and interpreting the data.
-
Providing reports to stakeholders and obtaining their feedback.
Because the policy environment can change, many participants felt that it is critical to design flexible and responsive systems of performance measurement. This heightens the likelihood that the results will be useful for several goals including accountability, program and system improvement, and continuing support and funding for enhancing quality in early childhood programs.
INVOLVING STAKEHOLDERS
Several states represented at the workshops, including Maine and Oregon, have involved stakeholders throughout the process, from goal development through the creation of performance measures. Workshop participant Bruce Clary, of the Muskie School of Public Policy at the University of
Page 11
Southern Maine, said that in making decisions about a system of standards and measures, his state found that it needed to consider a wide array of perspectives. In Maine, the group of stakeholders included legislators, consumers, and nonprofit providers. Clary said that the inclusion of stakeholders in the process was crucial to helping researchers define what needed to be measured and whether the standards being developed were valid. Also because so many stakeholders were included, measures from varied domains were included in its set of indicators (Clary et al., 1999).
In Oregon a similar approach was used, but most of the discussions and work toward developing standards were performed at the community and program levels and communicated to state level officials. Workshop participant Bobbie Weber, of Linn-Benton Community College, said this was unusual, since many states tend to focus on a top-down strategy, where state officials form the guidelines and communicate those standards to cities, towns, communities, and programs. Weber said Oregon's approach worked well because communities could determine which goals were meaningful to them and make decisions about which standards were most appropriate in their area. (For more information, contact the Oregon Child Care Research Partnership at < www.lbcc.cc.or.us/familyresources>.)
Other states have drawn upon public opinion surveys to help develop measures. In Nebraska, for example, the Gallup Organization conducted a survey for the Nebraska Children and Families Foundation to establish a baseline of Nebraskans' attitudes toward children and youth that could be measured repeatedly over time. The intention is to repeat the survey every few years to promote public engagement and discussion. This instrument helped Nebraska's public officials learn about the concerns of their constituents regarding how children are treated and cared for in the state as background for developing standards (Raikes et al., 1999).
The Massachusetts Department of Education chose to focus on the views of parents in examining its programs. As Jason Sachs of the Massachusetts Department of Education put it, programs are relatively flexible: changes in ratios, group sizes, and staff educational requirements, to name a few, are all possible.
The department surveyed parents on use patterns and preferences to help providers in the state develop programs that meet the needs of both parents and children. The department also asked parents about such issues as how many times their child had changed programs in a given year and whether, given unlimited resources, they would rather stay home with their children or send them to a child care center. The survey also asked if those
Page 12
who currently had children enrolled in a child care center would prefer to have them in the public schools or in a Head Start program (Sachs et al., 1999).
SELECTING OUTCOMES
The question of which outcomes should be measured generated lively debate. Measures can gauge structural features such as staff-child ratios, space, teacher education; outputs such as the number of licensed care centers, the number of child care slots, turnover rates; and outcomes such as child development, health status of children, and meeting family needs. On the one hand, many participants strongly felt that child outcomes should be paramount. Others voiced concerns about the risks of that approach and the difficulty of ascribing outcomes to the care provided. Given the powerful influences on children at home, in their neighborhoods, and in the varying settings in which they receive care, what is the role of child outcome measures? What should be measured in children, in what way, over what time period, and how can measures account for all the other influences on children? (See, for example, Kagan et al., 1997.) Participants differed in their views about whether and how to hold programs accountable for achieving desired results for children (see Box 2-3).
A Child Outcomes Perspective
Workshop presenter John Love, of Mathematica Policy Research Institute and a lead researcher on the national evaluation of the Early Head Start program, raised a number of questions to be addressed in considering a system that encompasses not only process measures but also child outcomes:
-
What outcomes do child care programs hope to achieve? What are the expectations?
-
Are these outcomes reasonable and realistic?
-
Can these outcomes be measured?
-
What procedures or systems are needed to make this happen?
-
How will the results be used?
-
What are the dangers?
Page 13
BOX 2-3Desired Results for Children in California
At the time the workshops were held, California was pilot testing an integrated system for implementing standards, monitoring, and technical assistance for early childhood care and education. All the state's subsidized center-based programs and family child care home networks are covered by the system, which also includes alternative payment programs that help families obtain subsidized care. Deborah Montgomery, American Institutes for Research, said that the state used Head Start standards supplemented with measures for turnover, continuity of care, and compensation. California's desired results for children include the following: they are personally and socially confident; they are effective learners; they show physical and motor confidence; and they are kept safe and healthy. California's program also established desired results for families, including support for their children's learning and development and achievement of their specific family goals. The state believes that these simple statements encourage broad “ownership” of the system being developed. At the program level, the state will be adopting the use of environmental rating scales, such as the Early Childhood Environment Rating Scale (ECERS). Programs are not being held responsible for the achievement of desired results by children, Montgomery said. But they are responsible for implementing new program standards, and that is where accountability will be measured. Programs will also be responsible for monitoring a child's progress toward desired results, and ensuring that a particular child's individual plan for learning has been based on data about that child. California plans to ask programs to conduct an annual self-review. That review would be coupled with the state's assessment of the program using ECERS and other environmental rating scales, and parent surveys that are administered by the programs. Over time, the state would be able to better target specific programs for technical assistance to improve quality. (For more information, please contact <www.airdc.org> or <“www.cde.ca.gov/cyfsbranch/child_development/> .) |
Page 14
Love said a review of the literature shows that the bulk of outcomes measures for child care are in the social/emotional domain. (This is in contrast to cognitive measures often associated with evaluations of preschool.) Some common measures include optimal peer interaction, positive social development, positive affect, advanced social skills, and more complex play behaviors. For example, in Hawaii, children should be well taken care of while they thrive physically and develop positive relationships, age-appropriate knowledge, an appreciation for diversity, and receptive language. In Florida, children should be able to comply with rules, cope with challenges, express needs, and develop problem solving and verbal communication skills (Love et al., 1996). (For more information, contact < www.mathematica-mpr.com>.)
In Vermont, efforts focused on looking at broad outcomes for children and families, according to Cheryl Mitchell, Deputy Secretary for the state's Human Services Agency. The state went through a process of involving its local communities in developing indicators. One longstanding indicator is the availability of quality child care. As is the case in many states, Vermont found great variability, with programs of very good quality focused on meeting the needs of families and children, and other communities where even minimal quality of care was lacking. So Vermont has set equity of access to quality as a major outcome. Mitchell also pointed out that the states efforts on performance measures became a way of helping to link resources to achieving quality. She acknowledged the role of the Head Start performance standards and the accreditation program of the National Association for the Education of Young Children (NAEYC) as examples that helped inform state legislators about the link between funding and quality. (For more information, contact < www.ahs.state.vt.us/>.)
Assessing Young Children
In its report, Principles and Recommendations for Early Childhood Assessments, the National Education Goals Panel (1998) emphasized that gathering accurate information about young children is difficult and potentially stressful to them. Formal assessments may also be costly and take resources that could otherwise be spent directly on programs and services for young children. Given these concerns, it is extremely important that there be a clear benefit to children and the quality of the care they receive.
The Panel set forth several principles for assessing young children. Effective assessments are those that are tailored to a specific purpose and are
Page 15
reliable, valid, and fair for that purpose. Assessments designed for one purpose are not necessarily valid if used for other purposes. For example, assessments designed for instructional planning may not have sufficient validity and technical accuracy to support decisions such as placing children in a special kindergarten designated for at-risk children. Misunderstanding of such issues has led to misuses of testing that concern many in the early childhood development field. Assessments of young children are also best when they address the full range of early learning and development, including physical well-being and motor development; social and emotional development; approaches toward learning, language development, and cognition; and general knowledge. Both the measures chosen and the data collection strategies employed should vary for children of different ages. Assessment policies that recognize that reliability and validity of assessments increase with children's age often yield the most realistic results (National Education Goals Panel, 1998).
CHILDREN WITH SPECIAL NEEDS
The workshops also discussed the needs of working families with young children who have disabilities. Research has shown that simply placing children with physical, mental, or emotional disabilities in programs with other children will not necessarily promote their developmental gains. Successful inclusion requires continued commitment, planning, and collaboration.
One key factor is provision of training, technical assistance, and support to staff. Unfortunately, ongoing training and consultative support for child care providers in approaches to meeting the diverse needs of children—particularly children with disabilities—is not always readily available. While curricula and training models for the inclusion of children with disabilities in child care have been available for a number of years, few states have systematically attempted to institute specialty training tied to a state system of career development or licensure.
The Map to Inclusive Child Care Project, funded by the Child Care Bureau, Administration for Children and Families, U.S. Department of Health and Human Services, aims to help states increase their capacity to serve children with disabilities within accessible, affordable child care programs. States chosen to participate in the Maps project are required to assemble a state team of at least 15 stakeholders that includes families of children with disabilities, child care providers, child care state administrators, Head Start representatives, and early intervention and preschool spe-
Page 16
cial education representatives. This team is assisted in developing a vision for inclusive child care specific to its state. The state teams also participate in a national institute to help increase the capacity of child care providers to accommodate children with disabilities by influencing funding, policy, and training.
In meeting the needs of children with disabilities, workshop participant Lynette Aytch, of the Frank Porter Graham Center at the University of North Carolina, said it is important to focus efforts not only on the child but also on his or her family. The major challenge in determining quality practices for children with special needs is that it includes a very broad range of settings and services. For example, a child with disabilities can be in a community child care setting. But in that setting, the child may receive some specialized services. The child's family may receive support services. Some of those services may be received at the center, some may take place in a home setting, and some may take place in a clinical setting.
Further complicating matters, each of these services may have individual goals that may not be integrated. Children will bring a variety of needs and strengths to a particular program, which can range from physical interventions and language instruction to social and emotional problems. Families will also have a multitude of issues that they may bring and for which they may need support services.
Aytch said that many features of high-quality practice are subjective. Many families of young children with disabilities defined quality as a caring provider who is willing to help the family meet its particular needs. So a key value in early intervention for children with disabilities is the centrality of families. That means that what happens in a particular interaction between a service provider and a child must be carried over to the family situation, because the parents have to play a pivotal role in sustaining any positive outcomes and effects of early intervention. Another important value is that parents are, to the extent that they desire, the primary decision makers. Programs should support mentoring that helps families take on more of a leadership role in services.
Aytch also found that families value the coordination of services so that they have easy access for children, and there is collaboration between all of the agencies that are involved. She also said that services should be individualized to address the specific needs of children and families; that models of intervention should be focused on needs and strengths; and that therapies and interventions should be integrated with typical routines and activities (Aytch et al., 1999).
Page 17
CULTURAL DIVERSITY
An important theme throughout the workshop was the need for sensitivity to cultural diversity in child care and in measures to assess programs. This includes diversity along a continuum—children's unique needs, their cultural and ethnic background, and the values their families seek in a child care setting. Asa Hilliard, of the Department of Education Policy Studies at Georgia State University, encouraged the child care community to broaden its attention to culture beyond simply tallying the numbers of minority children enrolled in programs. Hilliard called for greater flexibility in establishing performance measures for programs that serve children from a wide variety of cultural backgrounds and cautioned against the notion of “one size fits all.” He said that any assessments that are developed should have built-in sensitivities to language differences and recognize that measures of what is developmentally appropriate may differ according to a child's culture. Hilliard noted that it is going to take much more work and effort to develop assessment principles that adequately reflect cultural considerations. While specified benchmarks are useful for accountability, if viewed too narrowly, they may be interpreted as prerequisites to successful practice. Hilliard gave an example of a school with a minority enrollment that failed to gain accreditation because the teaching was determined to be developmentally inappropriate—the school was seen as hurrying the children. He noted that, despite good student performance and parent satisfaction, the accreditation team's rigid interpretation led to a cultural mismatch in applying criteria.
Hilliard also urged a focus on the issue of disproportionate numbers of minority students in special education, specifically in categories such as learning disabilities or language impairment. With respect to the latter, does the criteria for inclusion refer to articulation problems stemming from a physical or neurological disability, or diversity in articulation—that is, a possible mismatch in the language of assessment and the language of the child and his or her family? The need is to better understand the cultural nature of the assessment process to produce more equitable outcomes for children.
SCHOOL-AGED CHILDREN
Currently, very few early childhood centers take a long-term view of children once they finish the early childhood program and begin school, although a number of early childhood centers also run school-age child
Page 18
care programs. Michelle Seligson, of the Center for Research on Women at Wellesley College, noted that in the last three decades, the increase in the number of working mothers has raised interest in and demand for after-school programs. Teachers and school administrators are turning to after-school programs as another opportunity for teaching children and youth and engaging them in positive activities. Others see both early childhood and after-school programs as potential crime-prevention strategies. And social service providers view after-school programs as an opportunity to assist in welfare reform efforts by caring for children so that parents can work.
A diverse group of providers, many of whom are entrepreneurs, operate thousands of after-school programs that are funded largely by parents and a new funding source—the 21st Century Community Learning Center Grants (U.S. Department of Education, 2000). Researchers have begun to describe the quality of after-school programs, professional development of staff, and the kind of environment qualified and well-trained staff can create for children and youth. National standards have become the basis for a national accreditation program for after-school programs, run by the National School Age Care Alliance. These standards are based on an instrument developed at Wellesley College, called Assessing School Age Program Quality (Seligson, 1997).
Although there is still no agreement on what outcomes are expected for children who attend after-school programs, researchers are finding that these programs are having positive effects on children and youth. Reviews, such as the American Youth Policy Forum's 1998 report, Some Things Do Make A Difference for Youth, show that youngsters in good after-school programs do better in school, feel better about themselves, and have better social skills (James, 1997). Researchers at the University of Wisconsin have isolated positive interactions with adult staff as key to positive child outcomes. Flexible programs with varied activities make a difference as well, as does the number of times a child attends a program (Seligson, 1997).
RESEARCH ON CHILD CARE AND PERFORMANCE MEASURES
In the past decade, experts in early childhood development have gained a better understanding of the dimensions of quality of care and ways in which to measure it.
Longitudinal studies have found that children who have been in a
Page 19
higher quality program have been less distractible, more task-oriented, more considerate, happier, less shy, more socially competent, friendlier, and less aggressive when they start school than their peers who attended lower quality child care programs (Love et al., 1996). The National Institute of Child Health and Human Development's Study of Early Child Care and Youth Development has been key in providing research data on the relationships between children's experiences in child care and their development that is useful in the effort to measure programs (National Research Council and Institute of Medicine, 2000; National Institute of Child Health and Human Development, 2000).
Economics and Child Care
David Blau of the University of North Carolina, Chapel Hill, described his study of the economics of child care as it relates to policy. His research employed the Early Childhood Environment Rating Scale (ECERS), which measures the quality of child care centers on a scale of 1—inadequate level of care—to 7—excellent care (Harms and Clifford, 1980). According to Blau, good ECERS scores are associated with good developmental outcomes, and thus offer a more direct reflection of quality than standards that measure only group size or teacher training. To use ECERS scores effectively as a measure of quality, however, consumers must be willing to purchase child care that scores well on that measure. Producers must be willing to provide child care by following standards as measured by the instrument.
Blau further indicated that, in using this type of approach to determine quality, policy makers could decide to encourage consumers to demand and producers to supply high-quality care by issuing targeted subsidies for child care. A targeted subsidy is an allowance from the government that can be used only for child care that meets high-quality standards, such as those that surpass a target ECERS score. Under such an approach, government would help defray the cost of child care if quality child care is purchased. This tells providers that they will be eligible to serve subsidized consumers if they provide high-quality care as measured by ECERS.
Blau also noted that, although this is not an easy policy to implement, it could be more effective in accomplishing the goal of improving the quality of child care. Untargeted subsidies, for example, which can be used virtually for any arrangement, will work only if consumers have a strong willingness to pay for high-quality child care. However, according to Blau's
Page 20
research, parents do not appear to be willing to use general subsidies to pay for improved quality of care. Targeted subsidies, on the other hand, might provide the kind of financial incentive to use high-quality care.
Cost, Quality, and Outcomes Study
Richard Clifford of the Frank Porter Graham Child Development Center, University of North Carolina at Chapel Hill, described the Cost, Quality, and Outcomes Study, begun in 1993. The first phase involved more than 400 child care centers in Los Angeles County, California, Front Range, Colorado, Hartford-New Haven, Connecticut, and the Triad Area, North Carolina. In phase two, researchers selected 183 classes with pre-K children and 826 children going to kindergarten in 1994, and followed these children through the second grade. Researchers used ECERS to understand quality scores of child care centers in relation to a variety of environmental factors and child development outcomes.
Study results at the time of the workshop indicate that the quality of child care was an important element in having children prepared for school and in doing well as they began primary school. The beneficial developmental effect of child care quality appeared to continue well into school through the end of second grade. Moreover, researchers found that children whose mothers had low levels of education benefited more from high-quality child care than other children. In addition, children enrolled in programs with high ECERS ratings scored higher on measures of cognitive and social development even after taking into account differences in background factors known to be related to children's development. The study also found that children preferred programs that were rated higher in quality over those rated lower (Peisner-Feinberg et al., 1999).
Regulation and Licensing
The workshop also looked at the implication of regulations and licensing or the development of performance measures (see Box 2-4). Rebecca Kilburn of the RAND Corporation described a research study conducted for the National Institutes of Health that is collecting state-level child care regulations in a number of different areas, going back to 1985. Among the research questions are: How much variation exists across states in regulations for child care and early childhood programs? What characteristics of states predict how strictly they regulate care? How stringent are child care
Page 21
BOX 2-4Arkansas Better Chance Fund
In 1992, Arkansas established its Arkansas Better Chance Fund, which was responsible for distributing funds for early childhood programs. At that time, continued funding was tied not only to compliance with the state's rules, regulations, and licensing procedures, but also to scores on environmental rating scales appropriate for the early childhood age group. To help monitor the programs, the state enlisted the help of a training and assistance organization that had been working for several years with the Head Start program. The process brought together a group of 30 to 50 professionals including deans of education, school principals, and Head Start directors to help with monitoring. In 1995, the state legislature implemented a state accreditation procedure, which mandated that the State Board of Education approve early childhood programs as being developmentally or educationally appropriate for young children. At the same time, the state approved a tax credit for parents who chose to use higher quality care and legislation was introduced that gave employers tax credits for offering child care that met state accreditation standards. At the time of the workshops, only 10 percent of licensed and registered child care centers in Arkansas met state accreditation standards, said Kathy Stegall of the Arkansas Division of Child Care and Early Education. The state had set its benchmark at increasing the number of accredited child care centers by 10 percent to 15 percent per year over the next several years. The state had also begun training child care licensing staff in the use of environmental rating scales so that they can provide better assistance to programs. Although the state intends to maintain its cadre of professional evaluators, the new training effort gives the licensing staff an opportunity to recognize and identify high-quality programs, to encourage those programs to make an application for state accreditation, and to continue to raise the quality bar so that programs continue to improve. (For more information, please contact < www.state.ar.us/childcare/>.) |
regulations? Do regulations relate to some market features and outcomes such as price of care? Do regulations affect the availability of child care slots? Do families use more child care because they're more confident about safety and quality or less child care because regulations may have raised the price of care?
Page 22
Findings at the time of the workshop indicated that there are two broad categories for regulation of child care: health and safety, and instruction. As examples of variation, Kilburn pointed to regulations for child-to-staff ratios for two-year-olds. In 1997, the study found a median ratio of eight children to one staff person, but the range went from a low of four children to one staff person, to a maximum of twelve children to one staff person. Another example of variation is in use of criminal background checks. The study found that 42 states conduct background checks for early childhood center employees, and 40 states conduct background checks for family day-care homes.
Most regulations generally apply to licensed providers. But there is variation in exemptions from the regulations. In general, states exempt what are considered small family day care homes, but there is no consensus on the definition of “small.” In four states, small means fewer than six children. Five other states define small as fewer than five children. And in ten other states, small is defined as fewer than four children.
How do regulations affect the market? Not surprisingly, regulations that require, for example, fewer children per provider or education requirements for staff, tend to raise the price of child care. Kilburn noted that evidence indicates that the more expensive the care, the more likely people are to turn to unregulated care, undermining the goal of quality care for all children.
Kilburn reviewed some weaknesses in current regulatory practices. Exemption issues are particularly important. Care by relatives, for example, would be exempt as well as care provided by religious institutions in some states. The adequacy of resources for enforcement and inspection is another important issue. Finally, as noted throughout the workshop, the influences of parents on children are paramount, and regulations cannot impinge on parenting practices.
In the discussion following, participants noted that licensing generally represents minimum standards, even though states vary in the stringency of their regulations. But if the concern is measuring child outcomes, then the level must be above that. And if more stringent regulations move some families out of the regulated system, shouldn't the emphasis be on support for providers so they can stay in the regulated system? The question was asked whether research had been done looking at simultaneous regulatory or subsidy changes. Kilburn said that she knew of none but agreed funding relief could help minimize unintended consequences of regulations.
Page 23
NEXT STEPS
Participants were asked to provide their feedback on next steps. Many favored another series of meetings each of which would focus on some specific issues, using definitions of terms such as quality, problems of assessment, the relationship between state and federal funding requirements, and the inclusion of families in policy decisions regarding early childhood education.
Many participants voiced the need for early childhood experts to define more precisely what is meant by such terms as quality and outcomes. After those definitions are established, many suggested that goals and desired outcomes should be identified at the national, state, and program levels. In addition, many felt that a common set of outcome indicators should be developed at the national level that could be used by all states. Research must also continue after the performance measures are in place to determine whether the standards and measures are appropriate to various community needs and to suggest improvements when policies and programs are not working well. (See Chapter 4 for a discussion of a variety of efforts at the national, state, and local levels.)
Workshop participants agreed that important studies have been completed, but they urged a sustained focus on those problems and questions of greatest interest to practitioners and policy makers. Overall, participants expressed a need for researchers to work in closer partnership with the program staff and to disseminate information in a manner that can be easily understood by practitioners. Many said that funding was needed for every state to conduct research about quality in early childhood education. In the area of specific research needs, participants said that more research is needed on assessment tools for school readiness. They also suggested that more research is needed to understand why some parents are not accessing subsidies for which they appear eligible. Further research is also needed on the cost of high-quality care.
Some participants also felt that there were inconsistencies in the ways in which funding is allocated and suggested that policy makers at the federal level help those at the state level to better understand the available funding sources. Similarly, a lack of coordination among the measurement requirements of different funding sources was seen as a hurdle to understanding and implementing performance measures. There was also concern that policy makers make sure that laws intended to aid low-income families
Page 24
are consistent with those aimed at reforming welfare in relation to early childhood programs.
At the program level, several participants requested that federal and state governments provide more funding for training of providers of care to infants and toddlers. In fact, many participants felt that significant support was needed to build a workforce in early childhood education who could translate research findings to the program level, act as technical assistance advisors, and design and implement research initiatives that inform the field, programs, parents, and policy makers.
CONCLUDING THOUGHTS
In the words of Martha Zaslow, of Child Trends, Inc., several areas of differences—or dialectics—about performance measures emerged at the workshops. Concerns about the purpose of performance measures arose throughout the workshops—the tension between information for accountability and information to improve services. The former implies sanctioning—funds are withheld if performance measures are not met; the latter, some participants felt, reflect the drive to make programs better.
Which level is most appropriate for developing performance measures also was an issue for debate. Many participants stressed that the vision and goals must emerge at the local or program level. On the other hand, there is a need for measures and indicators that have broad geographic applicability.
Differences also emerged on who should supply information—parents, program providers, or the children themselves? Most notably, participants differed about whether to emphasize careful delineation of child outcomes and expectations in child care or to focus on delivery of programs and services. Throughout, participants were keenly aware of the difficulties of attributing child outcomes to child care given the diversity of settings children are in and all the other influences on their lives.
In sum, developing performance measures entails what many saw as an arduous process of consensus-building on all these issues and on the challenges of identifying and aggregating reliable data. These workshops provided a step in this necessarily long-term process, and much more remains to be done.
