National Academies Press: OpenBook

Early Childhood Assessment: Why, What, and How (2008)

Chapter: 6 Measuring Quality in Early Childhood Environments

« Previous: 5 Assessing Learning and Development
Suggested Citation:"6 Measuring Quality in Early Childhood Environments." National Research Council. 2008. Early Childhood Assessment: Why, What, and How. Washington, DC: The National Academies Press. doi: 10.17226/12446.
×
Page 145
Suggested Citation:"6 Measuring Quality in Early Childhood Environments." National Research Council. 2008. Early Childhood Assessment: Why, What, and How. Washington, DC: The National Academies Press. doi: 10.17226/12446.
×
Page 146
Suggested Citation:"6 Measuring Quality in Early Childhood Environments." National Research Council. 2008. Early Childhood Assessment: Why, What, and How. Washington, DC: The National Academies Press. doi: 10.17226/12446.
×
Page 147
Suggested Citation:"6 Measuring Quality in Early Childhood Environments." National Research Council. 2008. Early Childhood Assessment: Why, What, and How. Washington, DC: The National Academies Press. doi: 10.17226/12446.
×
Page 148
Suggested Citation:"6 Measuring Quality in Early Childhood Environments." National Research Council. 2008. Early Childhood Assessment: Why, What, and How. Washington, DC: The National Academies Press. doi: 10.17226/12446.
×
Page 149
Suggested Citation:"6 Measuring Quality in Early Childhood Environments." National Research Council. 2008. Early Childhood Assessment: Why, What, and How. Washington, DC: The National Academies Press. doi: 10.17226/12446.
×
Page 150
Suggested Citation:"6 Measuring Quality in Early Childhood Environments." National Research Council. 2008. Early Childhood Assessment: Why, What, and How. Washington, DC: The National Academies Press. doi: 10.17226/12446.
×
Page 151
Suggested Citation:"6 Measuring Quality in Early Childhood Environments." National Research Council. 2008. Early Childhood Assessment: Why, What, and How. Washington, DC: The National Academies Press. doi: 10.17226/12446.
×
Page 152
Suggested Citation:"6 Measuring Quality in Early Childhood Environments." National Research Council. 2008. Early Childhood Assessment: Why, What, and How. Washington, DC: The National Academies Press. doi: 10.17226/12446.
×
Page 153
Suggested Citation:"6 Measuring Quality in Early Childhood Environments." National Research Council. 2008. Early Childhood Assessment: Why, What, and How. Washington, DC: The National Academies Press. doi: 10.17226/12446.
×
Page 154
Suggested Citation:"6 Measuring Quality in Early Childhood Environments." National Research Council. 2008. Early Childhood Assessment: Why, What, and How. Washington, DC: The National Academies Press. doi: 10.17226/12446.
×
Page 155
Suggested Citation:"6 Measuring Quality in Early Childhood Environments." National Research Council. 2008. Early Childhood Assessment: Why, What, and How. Washington, DC: The National Academies Press. doi: 10.17226/12446.
×
Page 156
Suggested Citation:"6 Measuring Quality in Early Childhood Environments." National Research Council. 2008. Early Childhood Assessment: Why, What, and How. Washington, DC: The National Academies Press. doi: 10.17226/12446.
×
Page 157
Suggested Citation:"6 Measuring Quality in Early Childhood Environments." National Research Council. 2008. Early Childhood Assessment: Why, What, and How. Washington, DC: The National Academies Press. doi: 10.17226/12446.
×
Page 158
Suggested Citation:"6 Measuring Quality in Early Childhood Environments." National Research Council. 2008. Early Childhood Assessment: Why, What, and How. Washington, DC: The National Academies Press. doi: 10.17226/12446.
×
Page 159
Suggested Citation:"6 Measuring Quality in Early Childhood Environments." National Research Council. 2008. Early Childhood Assessment: Why, What, and How. Washington, DC: The National Academies Press. doi: 10.17226/12446.
×
Page 160
Suggested Citation:"6 Measuring Quality in Early Childhood Environments." National Research Council. 2008. Early Childhood Assessment: Why, What, and How. Washington, DC: The National Academies Press. doi: 10.17226/12446.
×
Page 161
Suggested Citation:"6 Measuring Quality in Early Childhood Environments." National Research Council. 2008. Early Childhood Assessment: Why, What, and How. Washington, DC: The National Academies Press. doi: 10.17226/12446.
×
Page 162
Suggested Citation:"6 Measuring Quality in Early Childhood Environments." National Research Council. 2008. Early Childhood Assessment: Why, What, and How. Washington, DC: The National Academies Press. doi: 10.17226/12446.
×
Page 163
Suggested Citation:"6 Measuring Quality in Early Childhood Environments." National Research Council. 2008. Early Childhood Assessment: Why, What, and How. Washington, DC: The National Academies Press. doi: 10.17226/12446.
×
Page 164
Suggested Citation:"6 Measuring Quality in Early Childhood Environments." National Research Council. 2008. Early Childhood Assessment: Why, What, and How. Washington, DC: The National Academies Press. doi: 10.17226/12446.
×
Page 165
Suggested Citation:"6 Measuring Quality in Early Childhood Environments." National Research Council. 2008. Early Childhood Assessment: Why, What, and How. Washington, DC: The National Academies Press. doi: 10.17226/12446.
×
Page 166
Suggested Citation:"6 Measuring Quality in Early Childhood Environments." National Research Council. 2008. Early Childhood Assessment: Why, What, and How. Washington, DC: The National Academies Press. doi: 10.17226/12446.
×
Page 167
Suggested Citation:"6 Measuring Quality in Early Childhood Environments." National Research Council. 2008. Early Childhood Assessment: Why, What, and How. Washington, DC: The National Academies Press. doi: 10.17226/12446.
×
Page 168
Suggested Citation:"6 Measuring Quality in Early Childhood Environments." National Research Council. 2008. Early Childhood Assessment: Why, What, and How. Washington, DC: The National Academies Press. doi: 10.17226/12446.
×
Page 169
Suggested Citation:"6 Measuring Quality in Early Childhood Environments." National Research Council. 2008. Early Childhood Assessment: Why, What, and How. Washington, DC: The National Academies Press. doi: 10.17226/12446.
×
Page 170
Suggested Citation:"6 Measuring Quality in Early Childhood Environments." National Research Council. 2008. Early Childhood Assessment: Why, What, and How. Washington, DC: The National Academies Press. doi: 10.17226/12446.
×
Page 171
Suggested Citation:"6 Measuring Quality in Early Childhood Environments." National Research Council. 2008. Early Childhood Assessment: Why, What, and How. Washington, DC: The National Academies Press. doi: 10.17226/12446.
×
Page 172
Suggested Citation:"6 Measuring Quality in Early Childhood Environments." National Research Council. 2008. Early Childhood Assessment: Why, What, and How. Washington, DC: The National Academies Press. doi: 10.17226/12446.
×
Page 173
Suggested Citation:"6 Measuring Quality in Early Childhood Environments." National Research Council. 2008. Early Childhood Assessment: Why, What, and How. Washington, DC: The National Academies Press. doi: 10.17226/12446.
×
Page 174
Suggested Citation:"6 Measuring Quality in Early Childhood Environments." National Research Council. 2008. Early Childhood Assessment: Why, What, and How. Washington, DC: The National Academies Press. doi: 10.17226/12446.
×
Page 175
Suggested Citation:"6 Measuring Quality in Early Childhood Environments." National Research Council. 2008. Early Childhood Assessment: Why, What, and How. Washington, DC: The National Academies Press. doi: 10.17226/12446.
×
Page 176
Suggested Citation:"6 Measuring Quality in Early Childhood Environments." National Research Council. 2008. Early Childhood Assessment: Why, What, and How. Washington, DC: The National Academies Press. doi: 10.17226/12446.
×
Page 177
Suggested Citation:"6 Measuring Quality in Early Childhood Environments." National Research Council. 2008. Early Childhood Assessment: Why, What, and How. Washington, DC: The National Academies Press. doi: 10.17226/12446.
×
Page 178

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

6 Measuring Quality in Early Childhood Environments T he domains of importance in early childhood all show mutability as a result of aspects of the environment. In this chapter, we review measures of quality in family and in early care and educational environments. Sometimes the family or the quality of the early care and educational setting is an out- come in its own right—the target of an intervention, for example. In other cases, it is a mediator of the effects of an intervention (e.g., improving family financial resources, introducing a new preschool curriculum, providing professional development) on child-level outcomes. In both these cases, it is crucial to have reliable and usable instruments from which one can draw valid inferences about the quality of the environment. Infants, toddlers, and young children need supportive, responsive, and stimulating relationships with caregivers and stimulating and safe environments to thrive (McCartney and P ­ hillips, 2006; National Research Council and Institute of Medi- cine, 2000). The National Academies synthesis of research on early development From Neurons to Neighborhoods concluded that “early environments matter and nurturing relationships are essential” (National Research Council and Institute of Medicine, 2000, p. 4). Families provide the primary care for children and are often the focus of early intervention programs. Home visiting programs are designed to promote positive, supportive parenting and to 145

146 EARLY CHILDHOOD ASSESSMENT reduce harsh negative parenting of infants, thereby indirectly enhancing their cognitive and social development (Wasik and Bryant, 2001). State or federally funded child care and educational programs are designed to promote children’s cognitive, academic, and social skills directly (National Institute for Early Education Research, 2006). Parents and policy makers want to know about the ­ quality of programs or family environments to ensure that they are enhancing, or at least not harming, children’s develop- ment. Accordingly, assessing children’s home and center-based environments, as well as child outcomes, has become an impor- tant part of assessment systems for young children (Adams, Tout, and Zaslow, 2007; Mitchell, 2005). OBSERVATIONAL MEASURES FOR MULTIPLE PURPOSES Many observational measures have been developed to assess the quality of home or early childhood care and education pro- grams. Selection of a measure requires consideration of the child population, the purpose of the observations, and the domains of most interest. For a program serving English language learners, for example, opportunities for children to develop language and vocabulary in their native language as well as English would be particularly important. Observational measures serve a number of purposes. First, they can be used for caregiver and teacher professional develop- ment. They can call administrators’ and caregivers’ or teachers’ attention to their own behaviors and practices that might pro- mote positive child outcomes. Having caregivers and teachers evaluate their own or each other’s classrooms and home-based care settings, as well as having two people (either an admin- istrator and a caregiver/teacher or two caregivers/teachers) evaluate the same setting, can be instructive and can provide good m ­ aterial for discussion. Administrators of formal early care and education ­programs—such as child care centers, preschools, pre­ kindergartens, and Head Start programs—can also use classroom observation measures as part of their teacher/caregiver evalua- tion strategy, as a more objective, sharable set of criteria for obser- vation. Several promising professional development programs use observational measures as the basis for improving quality

MEASURING QUALITY IN EARLY CHILDHOOD ENVIRONMENTS 147 of child care. For example, Pianta and colleagues use their tool, the CLASS (Pianta, La Paro, and Hamre, 2007), to promote more intentional instruction, classroom management, and emotional support in the classroom through their professional program, My Teaching Partner (Kinzie et al., 2006). The Quality Interven- tions for Early Care and Education (QUINCE) intervention and evaluation, which uses on-site technical assistance to improve the quality of home-based as well as center-based child care, uses the environmental ratings scales, the Family Day Care Environment Rating Scale, or FDCERS (Harms and Clifford, 1989), and the Early Childhood Environment Rating Scale-Revised, or ECERS-R (Harms, Clifford, and Cryer, 1998), to promote the use of age- appropriate activities and enhance teacher-child interactions in their program, which follows the Partners for Inclusion model (Bryant, 2007; Wesley, 1994). Second, observational measures can be used in formative assessment of programs that are striving to improve their ­quality. Periodic observations and examination of scores on different dimensions can help identify weaknesses that require further attention. Fourteen states now have quality ratings systems avail- able to the public, with summary ratings of the quality of early care and education, and many more states are developing such systems, with the aim of improving information to consumers and providing supports to improve quality (Tout, Zaslow, and Martinez-Beck, forthcoming). Local communities as well are developing such systems. In most fully developed state quality ratings systems, an observational measure of the quality of the early care and education environment—usually the ECERS-R, FDCERS, or the infant and toddler version of this measure, the Infant/Toddler Environment Rating Scale (Harms, Cryer, and Clifford, 1990)—is used as one component of the overall rating of the environment, which usually includes multiple components, selected and weighted differently in each state or community. The rating of the environment is used not only as a contributor to the summary rating of quality, but also as a source of detailed infor- mation about the facets of quality that need improvement and in which changes will help progress to the next quality rating. Third, classroom observations can be used for accountability purposes, instead of or as a supplement to child outcome mea-

148 EARLY CHILDHOOD ASSESSMENT sures. Child care quality has been a consistent modest to moder- ate positive predictor of children’s cognitive and language skills in large, multisite studies and smaller local studies (Howes et al., 2008; Lamb, 1998; NICHD Early Child Care Research Network, 2006; Peisner-Feinberg et al., 2001; Vandell, 2004) and a some- what consistent predictor of social skill (NICHD Early Child Care Research Network, 2006; Peisner-Feinberg et al., 2001; Vandell, 2004). Using early childhood assessments as part of an aligned system requires the capacity to juxtapose information about q ­ uality in the early care and education setting with change scores on children’s development (along with other key components). Thus, a complete system will require both ratings of the envi- ronment and assessments of children at multiple points in time, although this is expensive. In some federal and state efforts, observations of early care and education settings serve both a monitoring and accountabil- ity function and a formative function, providing information to improve quality. Thus, for example, as part of ­monitoring and accountability, the Head Start Impact Study collected observa- tions of the quality of Head Start programs as well as of formal early care and education programs serving children in the con- trol group (U.S. Department of Health and Human Services, Administration for Children and Families, 2005). Similarly, the Head Start Family and Child Experiences Survey (FACES) regu- larly collects observational data on a nationally representative sample of Head Start programs. The observational data are used in combination with child outcome data as part of ongoing pro- gram monitoring. However, the observational ­ratings and child outcomes together are also used to inform ongoing program improvement (see discussion in Zaslow, 2008). As one example, information from Head Start FACES was instrumental in shap- ing an increased focus in Head Start programs on early literacy development. Information from the Head Start Impact Study has also been instrumental in increasing professional development for Head Start teachers, focusing on early mathematics develop- ment in young children and how best to foster it. Fourth, classroom observations are useful for research. Indeed, most measures were originally developed as part of a research initiative. An extensive body of work looks at the rela-

MEASURING QUALITY IN EARLY CHILDHOOD ENVIRONMENTS 149 tionship of observational measures to child outcomes, especially in classroom-type early care and educational settings, and to a lesser extent in home-based care (Bryant, forthcoming; Burchinal, forthcoming; Burchinal et al., 2008). In addition, observational measures are used in evaluation studies to assess whether an intervention to improve practice in home-based or center-based early care and educational settings has affected caregiver/teacher practice or overall quality (for example, Bryant, 2007, and Pianta, 2007). An observational measure designed to assess parenting skills as a tool in caregiver or teacher professional development or for formative assessment should be detailed and descriptive so that it can help to direct improvement. In contrast, a measure used for research, summative assessment, or for accountability purposes, even if detailed, should be easily summarized in quan- tifiable ratings, so that scores can be compared over time and across settings. Purposes, in turn, have implications for who conducts the observation. If the goal is professional development or formative assessment, observations might be done by individuals directly involved. For example, observations of parenting skills might be done by a home visitor; a child care program teacher or admin- istrator could do observations of early care and education. If summative assessment or accountability is a goal, it is preferable that observation measures be administered by someone who is not directly connected to the program being evaluated, although program staff may sometimes perform this role if sufficient safe- guards are in place to ensure the reliability and validity of the observations. Most existing measures assess the social environment well and the learning environment at a very general level, but only a few adequately assess practices designed to teach academic or social skills specifically. Development of observational mea- sures is just beginning to catch up with the increased political emphasis on academic preparation in programs for young chil- dren (National Institute for Early Education Research, 2006). We summarize below some existing observational measures of the home and center-based environments, without attempting to be exhaustive. For all of these measures, there is some evidence for

150 EARLY CHILDHOOD ASSESSMENT their reliability and validity, and many include demonstrated associations with child outcomes. Note that although home or classroom observations may not be as labor-intensive or expensive as assessing individual child outcomes, all of these measures require a fair amount of assessor/ observer training for the results to be valid and reliable. Develop- ers of observational systems should provide clear and sufficiently intensive training criteria. Publishers of some instruments, like the CLASS (Pianta, La Paro, and Hamre, 2007), require training to be conducted by a publisher-trained and certified trainer, with different training for different user purposes. As specified by the developers, many natural observations of center-based or home settings require a minimum of 3 hours to ensure that sufficient sampling of the environment has occurred. Semistructured observations or interviews can require less time because they draw on specific kinds of interactions across all participants. Recommended times for the measures in Appendix Table 6-1 range from 1.5 hours to 2 half-days. Details on these measures can be found in the literature cited and, for many, in a compendium profiling observational measures for early childhood care and education environments prepared by Child Trends (Child Trends, 2007). Appendix Table 6-1 is a summary of some important characteristics of each measure discussed. The stars indicate that the dimension is represented somewhat (one star) or substantially (two stars). Assessing Home Environments Parents “structure the experiences and shape the environ- ments within which early development unfolds” (National Research Council and Institute of Medicine, 2000, p. 226). Chil- dren’s early cognitive, social, and physical development are all clearly linked to their relationships with their primary caregivers and the kinds of experiences available in their home environments (McCartney and Phillips, 2006). Theories of development focus on two overlapping dimensions related to assessing families in early childhood. The quality of relationships between the child and his or her primary caregivers is viewed as central for all forms of development, especially socioemotional skills (Bornstein and

MEASURING QUALITY IN EARLY CHILDHOOD ENVIRONMENTS 151 Sawyer, in McCartney and Phillips, 2006, pp. 381-398). And the quality of cognitive stimulation clearly plays a critical role for cognitive, language, and social development (Bradley et al., 2001; Fuligni, Han, and Brooks-Gunn, 2004). There have been many theoretical and empirical systems developed for describing how families affect children’s cogni- tive, language, and social development. These systems almost always include at least three dimensions: (1) the quality of the parent-child relationships and more distally of mother-father and whole-family relationships, (2) the quality of stimulation provided directly by caregivers in interactions with the child and by the objects that are available in the family environment, and (3) provision of basic needs and safety monitoring. These dimen- sions are listed below with a set of characteristics that are believed to be important for each dimension: 1. Relationships (mother-child, father-child, other primary caregiver-child, and more distally mother-father), emo- tional climate, social interactions, support for social skills development, and discipline strategies: A. Degree to which adults are affectionate, supportive, attentive, and respectful toward children. B. Explicit support for social skills (e.g., encouraging children to “use their words,” modeling and engaging children in conversations about social problem-solving skills, encouraging the use of learned strategies to solve real social conflicts). C. Degree to which primary caregivers use consistent behavior management techniques that are not harsh or demeaning. 2. Cognitive stimulation: A. Extent to which primary caregivers use the home envi- ronment to provide and scaffold learning activities for the child. B. Degree to which primary caregivers provide stimulating activities in the community. C. Degree to which primary caregivers talk to the child, engage the child in conversation, and use elaborated language in those verbal interactions.

152 EARLY CHILDHOOD ASSESSMENT D. Frequency with which children are exposed to books and have books read to them. E. Literacy resources (e.g., books, magazines, writing mate- rials, computers) in the home. 3. Provision of basic needs and safety monitoring: A. Degree to which the home environment is free of haz- ards, clean, and organized. B. Degree to which toys, books, and other child-friendly materials are available to the child without adult mediation. C. Presence of or access to outdoor play areas or areas in which gross motor play can occur. Primary Caregiver-Child Interactions Primary caregiver-child interactions typically either are coded from videotapes of semistructured 10- to 20-minute observations in which the primary caregiver is asked to engage the child in age-appropriate activities or are rated live during longer observa- tions in the home. An example is a measure used in the National Institute of Child Health and Human Development (NICHD) Study of Early Child Care and Youth Development (SECCYD) (NICHD Early Child Care Research Network, 1999, 2003) and the Early Head Start study (U.S. Department of Health and Human Services, Administration for Children and Families, 2004); differ- ent procedures are used for the youngest infants, toddlers, and preschoolers. Mothers of 6-month-olds were asked to play with their infants for 7 minutes and were told that they could use any toy or object available in the home or none at all. For the remain- ing 8 minutes, mothers were given a standard set of toys they could use in play. At 15, 24, and 36 months, the observation protocol followed a three-boxes procedure in which mothers were asked to show their children age-appropriate toys in three containers in a set order. The mother was asked to have her child play with the toys in each of the three containers and to do so in the order specified, but she was told she could spend as long or as little time on each activity as she wished. Videotapes were coded by research assis- tants who had attended centralized training sessions. At 6, 15, and

MEASURING QUALITY IN EARLY CHILDHOOD ENVIRONMENTS 153 24 months, mothers were rated on a 4-7 point scale (ranging from “not at all characteristic” to “highly characteristic”) to describe maternal sensitivity to child nondistress, cognitive stimulation, intrusiveness, positive regard, and negative regard. At 36 and 54 months, the mothers were rated on 7-point ratings of support- ive presence, hostility (reversed), and respect for autonomy. The composite scores were the strongest predictor of chil- dren’s cognitive, language, academic, and social outcomes when considered with demographic, parental attitude, and schooling characteristics in the NICHD SECCYD (NICHD Early Child Care Research Network, 2006). Similar observational systems have been developed and used for attachment (e.g., Egeland and Deinard, 1975), special education (e.g., Yoder and Warren, 2001), and effects of differing welfare reform policies on children (Weinfield, Egeland, and Ogawa, 1998). These observation proce- dures can also be used in assessing the quality of out-of-home care for infants and toddlers. Cognitive Stimulation Cognitive stimulation is reflected in a dimension called q ­ uality of instruction, which is often measured using a videotaped laboratory procedure in which mother and child pairs participate in a series of developmentally appropriate problem-solving situ- ations (Englund et al., 2004). The mother’s instructional behavior is rated on a 7-point scale that reflects how well she structured the situation and coordinated her behavior to the child’s activity and needs for assistance. The scale ranges from 1, indicating poor quality of instruction (uninvolved or unstructured), to 7, indicat- ing effective instruction throughout the session. The rating from this measure correlated with subsequent scores on standardized achievement tests in several studies (see, e.g., Connell and Prinz, 2002; Englund et al., 2004; Pianta and Egeland, 1994; Pianta and Harbers, 1996; Pianta, ­Egeland, and Sroufe, 1990; Pianta, Nimetz, and Bennett, 1997). Other observational rating systems focus on the quality of cognitive (DeTemple and Snow, 1998) or affective (Frosch, Cox, and Goldman, 2001) interaction specifically sur- rounding book reading. Recently the Child/Home Early Language and Literacy

154 EARLY CHILDHOOD ASSESSMENT Observation (CHELLO) observation measure was developed to assess the quality of early childhood language and literacy practices in mixed-age home-based child care settings (Neuman, Dwyer, and Koh, 2007). The measure complements a classroom observation measure (ELLCO) described below. A checklist is used to assess the literacy environment (books, writing materials, and cognitively stimulating toys) and a 1-5 rating scale includes items that assess the physical environment for learning, support for learning, and teaching strategies (e.g., vocabulary building, use of print, storytelling). The CHELLO total score has been shown to be correlated with growth in children’s language skills (PPVT), phonological skills, and ability to do language-oriented mathematical problems. Home Observation for Measurement of the Environment We single out the Home Observation for Measurement of the Environment (HOME; Caldwell and Bradley, 1984) for discussion because it is such a widely used assessment of the home environ- ment. The focus is on the child in the environment, experiencing objects, events, and transactions occurring in connection with the family surroundings. There are separate forms for assessing infants and toddlers and older children. The infant/toddler ver- sion of the inventory (IT-HOME) focuses on infancy (birth to age 3 years). It is composed of 45 items clustered into 6 subscales: (1) parental responsivity, (2) acceptance of child, (3) organization of the environment, (4) learning materials, (5) parental involvement, and (6) variety in experience. Each item is scored in binary fashion (yes/no). Information used to score the items is obtained during the course of the home visit by means of observation and semis- tructured interview. The early childhood version of the inventory (EC-HOME) is used during early childhood (ages 3 to 6). It is composed of 55 items clustered into 8 subscales: (1) learning materials, (2) lan- guage stimulation, (3) physical environment, (4) responsivity, (5) academic stimulation, (6) modeling, (7) variety, and (8) accep- tance. Each item is scored in binary fashion (yes/no). Information on items is obtained either through observation or through asking the mother. Typically the total score is used, although a recent

MEASURING QUALITY IN EARLY CHILDHOOD ENVIRONMENTS 155 factor analysis (Fuligni et al., 2004) produced scales that appear to differentiate between stimulation in the home environment for language and literacy and for social development. As with parental sensitivity, the quality of the home envi- ronment has been shown to be a moderate to strong predictor of academic and social outcomes for young children regardless of income or ethnicity (Bradley et al., 2001). Zaslow et al. (2006) found that the HOME, direct observations of mother-child inter- action, and maternal self-report measures collected during the preschool years all predicted child outcomes during middle childhood in a low-income sample with family background char- acteristics controlled. However, direct observations showed the strongest pattern of prediction. Center-Based Early Childhood Environments Early childhood care and education programs are increas- ingly being held accountable for their effects on children’s development, and thus assessments of quality are needed. As noted earlier, measures of quality are also being used to inform efforts to improve quality at the community and state levels, and in research evaluating specific quality improvement efforts. The specific dimensions measured will vary as a function of program goals, as discussed throughout this report. It is important for measures of the environment to be used in conjunction with mea- sures of child outcomes to provide a context for understanding the extent to which children show positive development dur- ing the time they are participating in early care and education. This ­ section describes strategies for assessing program quality directly. Many indicators that have been connected to child outcomes are fairly easy to quantify. Examples are staff-child ratios, number of children in a classroom, amount spent per child, the training and experience of teachers, and teacher turnover. Other quality variables are less easily quantifiable but are nonetheless impor- tant, such as opportunities for professional development for staff and the nature of the curriculum. Information on these variables is best obtained by interviews with program directors, surveys, or inspection of records. Some of these indicators, such as teacher/

156 EARLY CHILDHOOD ASSESSMENT caregiver education, staff-child ratio, and group size are related to better child outcomes across a number of studies (Howes, 1997; NICHD Early Child Care Research Network, 2002; Phillipsen et al., 1997; Vandell, 2004), although recent evidence has raised questions about whether teacher/caregiver education is related to child outcomes in publicly funded center-based child care set- tings, such as prekindergarten and Head Start programs (Early et al., 2006). Recent work suggests that teacher/caregiver education may play a different role in early care and educational settings with more versus less supports, requirements, and monitoring. A study in California by Vu, Jeon, and Howes (in press) found that teacher education contributed to quality in the less supported early care and education settings (such as private child care) but not in the more supported settings (such as state-­sponsored prekindergarten). ClassroomS Earlier we presented a list of dimensions on which parents and caregivers influence the development of young children. A similar set of dimensions of quality that are observable in the classroom are believed to contribute to children’s physical, socio- emotional, and cognitive development. For some of them, there is good empirical evidence linking quality on the dimension to children’s development (see Box 6-1). Observation Measures Most existing measures assess the social environment well and the learning environment at a very general level, but only a few adequately assess practices related to cognition or academic skill domains. Development of observation measures is just beginning to catch up with the increased political emphasis on academic preparation. Early measures included only a few very general items related to practices designed to promote language and cognitive development. Thus, for example, many measures include items assessing the degree to which children choose activities, but few provide very much information on the degree

MEASURING QUALITY IN EARLY CHILDHOOD ENVIRONMENTS 157 to which children are given specific kinds of opportunities to develop literacy, mathematical, or science skills. We summarize below selected observational measures that have been developed and used to assess early childhood pro- grams. For each of these measures, there is some evidence for their reliability and validity. (Evidence on the reliability and validity of these and other observational measures is summarized by Child Trends, 2007, in a compendium providing profiles of measures of quality in early childhood care and educational settings.) Few measures have demonstrated effects on child outcomes, although most assess practices that have been associated with positive child outcomes. Note that although classroom observations may not be as labor-intensive or expensive as assessing individual child outcomes, a fair amount of training is necessary to use all of these measures reliably. Observations generally should be done for a minimum of 3 hours before a classroom is rated. For full-day pro- grams, a full-day observation is preferable, and observations on two separate days are always desirable. The developers of some measures require their own training and certification. Assessment Profile for Early Childhood Programs The Assessment Profile for Early Childhood Programs (APECP; Abbot-Shinn and Sibley, 1992) is an observational check- list with dichotomous items that provides a global assessment of overall preschool classroom environment; it includes subscales that address specific aspects of the dimensions thought to define global quality. These scales include (1) learning environment (provisions for and accessibility of materials, space conducive to child independence), (2) scheduling (written plans assessed for balance and variety of activities), (3) curriculum (degree to which alternative techniques are used to facilitate learning, based on assessment of children in class; degree to which children are encouraged to be active in guiding their own learning; the role of the teacher in facilitating learning), (4) interacting (teachers’ posi- tive interactions, responsiveness, and management of children), and (5) individualizing (support for individualized learning expe- riences through assessment, parent communication, and referrals; plans for children with special needs).

158 EARLY CHILDHOOD ASSESSMENT BOX 6-1 Dimensions of Quality Observable in the Classroom 1. Emotional climate, social interactions, support for social skills development, and discipline strategies: A. Degree to which adults are affectionate, supportive, attentive, and respectful toward children. B. Explicit support for social skills (e.g., encouraging children to “use their words,” modeling and engaging children in conver- sations about social problem solving skills, encouraging use of learned strategies to solve real social conflicts). C. Conversations about feelings. D. Collaboration and cooperation opportunities. E. Clarity and developmental appropriateness of rules. F. Teachers’ use of redirection, positive reinforcement, encour- agement, and explanations to minimize negative behavior. 2. Instructional activities—an explicit curriculum with specified learning goals for children. 3. General—individualized (adjusted to children’s skills and in- terests); purposeful, planned instruction; integration of content areas; children actively interacting with materials. 4. Language—adults engage in conversations with children; activi- ties that encourage conversation among children; explicit efforts to develop vocabulary and language skills in the context of meaningful activities. 5. Literacy—children read to and given opportunities to read; rhyming words, initial sounds, letter–sound links, and spellings of common words pointed out and practiced; functions and fea- tures of print pointed out; opportunities to dictate and write using invented spelling made available. 6. Mathematics—activities that involve counting objects, measur- ing, identifying shapes, creating patterns, telling time, classifying and seriating objects; instruction on concepts (e.g., big, bigger, equal, one-to-one correspondence, spatial relationships). 7. Science—active manipulation of materials (e.g., sink and float) with adult engaging children in prediction, systematic observa- tion and analysis; instruction on scientific concepts linked to active exploration (e.g., care and observations of live animals).

MEASURING QUALITY IN EARLY CHILDHOOD ENVIRONMENTS 159 8. Interactions with parents—activities and opportunities for par- ents to be informed about the program and their child. 9. Cultural responsiveness: A. Evidence of supports for linguistic and cultural diversity (e.g., pictures, books, language). B. Activities that expose children to diverse languages and cultural practices. C. Support for native language development. D. Support for learning English. 10. Safety: A. Adult-child ratio. B. Absence of broken furniture, any objects that could cause physical harm. C. Sufficient space; open pathways. D. Place for personal hygiene (e.g., teeth brushing, hand washing). 11. Materials: A. Technology (e.g., computers). B. Music (e.g., CD player). C. Creativity (e.g., art supplies, easels, play dough). D. Dramatic play (e.g., store, post office, kitchen, clothes). E. Science (e.g., sand, water, plants, live animals). F. Literacy (e.g., books, writing materials). G. Math (e.g., counting objects, blocks, measuring instru- ments). H. Fine motor (e.g., materials for drawing, scissors). 12. Physical arrangement: A. Space and equipment for gross motor activities (e.g., climbing equipment, swings, balls). B. Place for quiet and rest (e.g., rugs and pillows out of the center of activity). C. Children’s access to materials. 13. Adaptations for children with disabilities.

160 EARLY CHILDHOOD ASSESSMENT The 75 items are scored on a yes/no basis according to whether or not they characterize a program during each time interval observed. Typically, programs are observed in 15- to 20-minute time periods over a course of 3 hours (e.g., one time period per hour), thus yielding up to 3 yes/no scores for each item. Although the measure includes items related to academic instruction, the yes/no format is a major limitation. Thus, for example, the caregiver/teacher asking only one open-ended question or writing down one word dictated by a child during an observation period gets a “yes” score for that observation period. The measure also does not differentiate among kinds of instruc- tional approaches. For example, scores go up whether children are asked questions that require remembering specific facts (such as who, what, when questions), or questions that are open-ended or problem-solving (such as why and how questions). Scores on the learning environment are also substantially affected by the number of materials of a particular kind rather than the quality of their use. Also, some of the items require inspection of records (e.g., lesson plans, daily schedule). The APECP scores have been related to child outcomes in both program improvement and observational studies (Lambert, Abbott-Shinn, and Sibley, 2006). Caregiver Interaction Scale The Caregiver Interaction Scale (CIS; Arnett, 1989) provides a global rating of caregiver/teacher sensitivity and responsive- ness to all children in the setting. It has been used in both center and home-based care and for infants, toddlers, and preschool- ers. It focuses on caregiver/teacher interactions with children, especially on responsiveness and emotional tone. The measure consists of 26 items measuring teachers’: (1) sensitivity (e.g., “seems enthusiastic about the children’s activities and efforts”), (2) harshness (e.g., “seems unnecessarily harsh when scolding or prohibiting children”), (3) detachment (e.g., “spends considerable time in activity not involving interventions with the children”), and (4) permissiveness (e.g., “expects the children to exercise self- control”). Each item is rated on a 4-point Likert scale with 1 being “not at all” to 4 being “very much.” The focus on teacher-child

MEASURING QUALITY IN EARLY CHILDHOOD ENVIRONMENTS 161 interaction is a strength if that is the primary goal. The measure must be supplemented with another measure if other dimensions of the classroom context need to be assessed. Classroom Assessment Scoring System The Classroom Assessment Scoring System (CLASS; Pianta, La Paro, and Hamre, 2007) provides an assessment of the overall preschool classroom in terms of the teacher’s sensitivity, quality of instruction across all academic areas, and classroom manage- ment. It assesses 10 domains of teacher-child interaction that form three subscales: (1) emotional support: (a) positive climate, (b) negative climate, (c) teacher sensitivity, (d) regard for children’s perspectives; (2) classroom organization: (a) behavior manage- ment (proactive, nondisruptive, reinforcing positive behavior), (b) productivity (efficient use of time), (c) instructional learning for- mats (teacher enabling of children’s experience, exploration and manipulation of materials); and (3) instructional support: (a) con- cept development, (b) quality of feedback, (c) language modeling. Each dimension is rated on a 7-point Likert-type scale. Observers rate all dimensions after at least four 20-minute intervals. The measure assesses instruction, but only at a very general level. There are no specific items related to literacy or mathematical instruction. A limitation is that there are only nine items focused on classroom practice, which include many different practices. The CLASS, developed relatively recently, was used in an 11-state evaluation of prekindergarten programs. The instruc- tional climate score provided the best, albeit modest, prediction of gains in children’s language and literacy skills relative to scores from other widely used instruments (Howes et al., 2008). Classroom Practices Inventory The Classroom Practices Inventory (CPI; Hyson, Hirsh-Pasek, and Rescorla, 1990) was developed to differentiate between devel- opmentally appropriate practices, according to 1987 guidelines of the National Association for the Education of Young Children (NAEYC), and highly didactic practices. It focuses on the teaching practices the teacher uses with the entire preschool classroom. The

162 EARLY CHILDHOOD ASSESSMENT measure contains 26 items divided into two subscales. The emo- tional climate subscale assesses the teacher’s warmth, encourage- ment, and positive guidance. In the program focus subscale, half of the 20 items refer to didactic, teacher-directed practices (e.g., large-group instruction; workbooks, ditto sheets, and flashcards; memorization and drill; art projects that involve copying; focus on getting the right answer), which were considered develop- mentally inappropriate by NAEYC. Of the 10 items that describe positive activities, most concern child choice and initiative and diversity of activities and materials that children can manipulate. Three of the items refer to positive instructional approaches (e.g., “teachers ask questions that encourage children to give more than one right answer”). The CPI described center-based child care preschool programs in the 10-site NICHD Study of Early Child Care and Youth Devel- opment. The program focus score predicted children’s language and academic outcomes at 4.5 years in analyses that adjusted for family characteristics in unpublished analyses (available from the authors on request). A Developmentally Appropriate Practices Template A Developmentally Appropriate Practices Template (ADAPT; Van Horn and Ramey, 2004) has 19 items based on the 1987 NAEYC guidelines. It also focuses on the teaching practices the teacher uses with the entire preschool classroom. Items are anchored on a 1 (developmentally inappropriate) to 5 (developmentally appropriate) scale, with descriptions for each anchor. The items form three scales: (1) integrated curriculum (e.g., “teacher adapts instruction to children’s interests, needs, and prior knowledge”; “literacy integrated across content areas with literacy materials of social relevance”), (2) social-emotional emphasis (e.g., “children’s social and emotional development consistently supported by peers and teachers”; “children and teacher collaborate, class- room exemplifies community of learners with shared goals”), and (3) child-centered approaches (e.g., “children encouraged to choose and interact with materials to create and problem-solve”; “children work interdependently to complete task or project and make joint decisions”). Instructional practices are described

MEASURING QUALITY IN EARLY CHILDHOOD ENVIRONMENTS 163 at a fairly general level and focus primarily on integration and child-centeredness. Early Childhood Environment Rating Scale-Revised Edition The Early Childhood Environment Rating Scale-Revised E ­ dition (ECERS-R; Harms, Clifford, and Cryer, 1998) is the most widely used measure of early childhood environments for both evaluation and research purposes. Its goal is to describe the overall quality of the preschool classroom based on the quality of teacher-child interactions and types of activities available in the classroom. The original scale was published in 1980 (Harms and ­ Clifford, 1980) and was revised in 1998 (Harms, Clifford, and Cryer, 1998). The two measures have been compared (Sakai et al., 2003), with the scores on the revised version being highly correlated with those on the original scale but also being on aver- age about .5 points lower. Developers of other scales frequently use correlations with the ECERS as a check on the validity of the new scale. The ECERS primarily focuses on the structural quality of early childhood programs as defined by 43 items that make up 7 subscales: (1) space and furnishings, (2) personal care routines, (3) language-reasoning, (4) activities, (5) interaction, (6) program structure, and (7) parents and staff. Each item is rated on a 1-7 scale with descriptions anchored at odd numbers, such that 1 represents an “inadequate situation,” 3 is “minimal,” 5 is “good,” and 7 is an “excellent situation.” The ECERS assesses the quality and quantity of books and mathematical materials in the classroom and assesses very global practices in the language- reasoning subscale (e.g., “a wide selection of books are acces- sible for a substantial portion of the day”; “children are asked questions to encourage them to give longer and more complex answers”). It does not measure instructional practices. Factor analyses of the instrument have consistently yielded two dimen- sions (National Association for the Education of Young Children, 2005). The first dimension describes the quality and quantity of teacher-child interactions across multiple types of activities, and the second dimension describes the extent to which a variety of age-­appropriate activities are provided. The ECERS or ECERS-R child-related total scores have been

164 EARLY CHILDHOOD ASSESSMENT modestly to moderately related to children’s language and social skills across a large number of studies (Bryant et al., 1994; B ­ urchinal, Peisner-­Feinberg, et al., 2000; Burchinal, Roberts, et al., 2000; Howes, ­ Phillips, and Whitebrook, 1992; McCartney, 1984; Peisner-­Feinberg and ­Burchinal, 1997; Phillips, McCartney, and Scarr, 1987). The magnitude of these associations tends to be modest, with partial correlations of 0.06 < r < 0.17 across stud- ies. While most of these studies focused on total scores, a recent pre­kindergarten evaluation study reported that summary scores describing caregiver-child interactions were stronger predictors of child outcomes than summary scores describing the types and quality of activities available in the setting (Howes et al., 2008). Early Childhood Environment Rating Scale-Extension The Early Childhood Environment Rating Scale-Extension (ECERS-E; Sylva, Siraj-Blatchford, and Taggart, 2003; Sylva et al., 2006) was developed to supplement the ECERS-R, which was deemed by the authors to be insufficiently sensitive to important teaching practices that support children’s intellectual develop- ment. Focusing on the quality of instruction for the preschool classroom, it has 4 separate subscales consisting of 18 items: (1) literacy (e.g., adult reading with child, attention paid to sounds in words, books, and print available and discussed), (2) mathe­matics (e.g., counting encouraged, number games, reading and writing numbers, shapes, matching and comparing), (3) science/environment (e.g., science resources, exploration of natural materials, scientific concepts introduced), and (4) diver- sity (e.g., planning for individual needs, race and gender equality addressed). The measure was tailored to tap the dimensions of quality defined by a new curriculum in England. Following the format of the ECERS-R, detailed descriptions are provided for each item; items are scored 1 (inadequate) through 7 (excellent). The measure is conservative in the sense that there are stringent rules for getting a relatively high score; a lower score could be given if one very specific practice was not seen. The measure would also favor ­better resourced programs because many items require the presence of specific learning materials. Reports of studies by the ECERS-E developers (Sylva et al., 2006) claim that

MEASURING QUALITY IN EARLY CHILDHOOD ENVIRONMENTS 165 the instrument has predictive validity for pre-reading scores, early number concepts, and nonverbal reasoning. Early Childhood Classroom Observation Measure The Early Childhood Classroom Observation Measure (ECCOM; Stipek and Byler, 2004) also focuses on the quality of the preschool classroom in terms of both teacher sensitivity and class- room management style. It contains 32 highly descriptive items with 3 subscales: (1) management (teachers provide children with choices both in the context of teacher-planned activities and dur- ing free time; rules and routines are clear but flexible; children are given developmentally appropriate responsibilities; discipline is brief and nondisruptive, often involving explanations or assisting children in their own social problem solving); (2) social climate (teachers are warm, responsive, attentive, and respectful of chil- dren; tasks and activities are flexible and adapted to children’s individual skill levels, interests, and experiences outside the class- room; social and communication skills are taught directly and in the context of naturally occurring social conflicts); and (3) learning climate and instruction (individualized but clearly articulated standards; coherent lessons; focus on understanding; children are active participants in instructional conversations; broad array of literacy experiences; mathematical instruction emphasizes pro- cesses and problem solving). Each of the items is rated at the end of the observations using a scale of 1 (practices are rarely seen) to 5 (practices predominate). A “classroom resources” checklist is also included to document materials in views that are related to technology, literacy, mathematics, dramatic play, art, gross motor equipment, and real-life objects. One limitation of the measure is that each item includes a number of different practices. As a con- sequence, the item score does not provide information on exactly which of the practices were observed. Early Language and Literacy Classroom Observation The Early Language and Literacy Classroom Observation (ELLCO; Smith and Dickinson, 2002) focuses on the quality of the language and literacy experiences in a preschool classroom. It is

166 EARLY CHILDHOOD ASSESSMENT one of the few measures that provides detailed information on literacy instruction and could be combined with a measure that includes items on other dimensions of classroom practice. It can be administered in as little as 1.5 hours. The Literacy Environ- ment Checklist assesses the visibility and accessibility of such literacy-related materials as books, an alphabet, word cards, teacher dictation, alphabet puzzles, and writing tools. There are also 14 ratings that are made at the end of a classroom observa- tion, using a rubric on a 1 (deficient) to 5 (exemplary) scale. The scale includes a few items on classroom management and climate, but most items focus on language-learning opportunities (e.g., oral language facilitation; book reading and discussion; instruc- tion in and opportunities to write meaningful text; frequent and various approaches to assessment). Accompanying the observa- tion measure is a teacher interview designed to clarify aspects of the observation. Finally, the Literacy Activities Rating Scale asks observers to record the amount of time spent on nine literacy behaviors related to book reading and writing. Studies have shown that the ELLCO explained a significant amount of the between-classroom variation in children’s receptive vocabulary (Peabody Picture Vocabulary Test) and early literacy skills (Dickinson et al., 2000; Smith and Dickinson, 2002) as well as social skills (Dickinson et al., 2001). Emerging Academics Snapshot The Emerging Academics Snapshot (EAS; Ritchie et al., 2001) focuses on social and academic experiences of individual children in the preschool classroom. The experiences of selected children are often tallied to form a classroom composite, although the indi- vidual experiences are also used as predictors of that child’s out- comes. It is a time-sampling observational instrument designed to describe children’s exposure to instruction and engagement in academic activities, as well as to describe activities and adult responsive involvement. It includes 27 items that are coded as present or absent in 20-second periods, followed by a 40-second coding period. The instrument can be used in either a traditional time-sampled procedure—one child at a time—or as a snapshot.

MEASURING QUALITY IN EARLY CHILDHOOD ENVIRONMENTS 167 When one child at a time is sampled, at least three 5-minute p ­ eriods divided into 20-second intervals should be collected across a 1- to 2-hour period. When used in snapshot fashion, up to four children can be sampled in succession. To assess a program, a subset of randomly identified children could be observed and their data averaged. Subcategories include (1) children’s activity setting, for example (a) routines (standing in line, cleanup, wait- ing for materials, etc.); (b) whole group; (c) small group time; (d) centers/free choice; (2) engagement with adults (didactic, scaffolds, uses home language of child); (3) engagement with activities (being read to, copying, engaged in mathematics, sci- ence, fantasy play, on the computer); and (4) peer interaction (e.g., solitary, parallel, cooperative pretend). The measure is descriptive and does not yield quality scores. It would not be appropriate for accountability purposes, but it can be instructive in teacher professional development and as a formative assessment tool to provide descriptive information on how children are spending their time. At least some evidence suggests that the EAS measures aspects of the child care environment related to children’s out- comes. It was used in the 11-state evaluation of prekindergarten programs. Gains in literacy outcomes were predicted by time spent in literacy-related activities (Howes et al., 2008). Family Day Care Rating Scale The Family Day Care Rating Scale (FDCRS), designed for family day care programs, uses the same format as the ECERS-R. The 38 items form seven subscales: (1) space and furnishings, (2) personal care routines, (3) listening and talking, (4) ­activities, (5) interaction, (6) program structure, and (7) parents and provider. A growing focus on the quality of home-based child care has resulted in greater use of the FDCRS, but few studies have mea- sured both the quality of care and child outcomes. In perhaps the largest study, FDCRS scores predicted children’s social and language skills (Kontos, Howes, and Galinsky, 1996).

168 EARLY CHILDHOOD ASSESSMENT Infant and Toddler Environmental Rating Scale The Infant and Toddler Environmental Rating Scale-Revised (ITERS-R; Harms et al., 1990) uses the same format as the ECERS-R, but it is designed to assess center-based child care programs for infants and toddlers. The 43 items are organized to cover seven categories: (1) space and furnishings, (2) personal care routines, (3) listening and talking, (4) activities, (5) interaction, (6) program structure, and (7) parents and staff. While many studies have used the ITERS to document the quality of infant/toddler center care (e.g., Helburn, 1995), relatively few studies have also measured the infants or toddlers themselves. In one study that measured both the classroom and home environments and infant outcomes, the ITERS total score predicted both the level and rate of change in infant and toddler’s language and IQ scores in a study of black children attending center-based care (Burchinal, Roberts, et al., 2000). Observation Measure of Language and Literacy Instruction The Observation Measure of Language and Literacy Instruc- tion (OMLIT; Abt Associates Inc., 2006) focuses on measuring the quality of the literacy practices in preschool classrooms. It is a battery of observation instruments that assess instructional practices and qualities of the environment in early childhood education classrooms that have been shown to support the development of oral language and emergent literacy skills. A classroom description is also included that provides contextual information, such as the number of children, their ages, and the languages they speak and that are used in instruction. There are five instruments: 1. The Classroom Literacy Opportunities Checklist is an inventory of 54 classroom literacy resources in 7 categories (e.g., text material and reading/listening areas; writing materials and writing area; diversity in the literacy materi- als). Items are coded either on a 3-point scale (minimally supplied, adequately supplied, well-supplied) or a 2-point scale (minimally supplied or well-supplied).

MEASURING QUALITY IN EARLY CHILDHOOD ENVIRONMENTS 169 2. The Snapshot of Classroom Activities identifies literacy activities and integration of literacy materials in other activities, languages spoken, and count of adults and chil- dren present. 3. The Read Aloud Profile assesses dialogic reading practices on seven dimensions (e.g., pre-reading “set up,” strategies used while reading, language(s) used). 4. The Classroom Literacy Instruction Profile describes the literacy activities and the instructional methods used by staff. Staff in the classroom are followed for 10 minutes at 15-minute intervals over the observation period, coding literacy “events.” Codes are both descriptive (the strate- gies used) and evaluative (e.g., the cognitive challenge presented by the dialogue/discussion between the staff member and the children). 5. The Quality of Instruction in Language and Literacy mea- sure rates the frequency and quality of literacy instruction and support for language and literacy development. Each of the 11 items is rated on a 5-point scale. The OMLIT is extraordinarily detailed and comprehensive, and accompanying materials provide an extensive rationale for the choice of items. It is unlikely that all of the scales would be used, but specific selections could be made. Observation Record of the Caregiving Environment The Observation Record of the Caregiving Environment (ORCE; National Institute of Child Health and Human Develop- ment, NICHD Early Child Care Research Network, 2000) focuses on the sensitivity and responsiveness of the caregiver to an indi- vidual child. It can be used in home- or center-based settings for infants, toddlers, and preschoolers. It is collected in one or two 3-hour visits to the child’s home- or center-based care. The observer collects time-sampling observations of behaviors and completes ratings of the child’s caregiver. The behavior scales provide a record of the occurrence or quantity of specific acts, and the qualitative scales take into account the quality (and nuances) of the caregiver’s behavior in relation to the child’s behavior. The most frequently

170 EARLY CHILDHOOD ASSESSMENT used quality measure, the positive caregiving composite, is cal- culated slightly differently for each age level. At 6, 15, and 24 months, positive care-giving composite scores are the mean of five 4-point qualitative ratings (sensitivity to child’s nondistress signals, stimulation of cognitive development, positive regard for child, emotional detachment [reflected], flatness of affect [reflected]). At 36 months, these five scales plus two additional subscales, “fosters child’s exploration” and “intrusive” [reflected], are included in the composite. At 54 months, the ­positive caregiving composite is the mean of 4-point ratings of caregivers’ sensitivity/responsivity, stimulation of cognitive development, intrusiveness (reflected), and detachment (reflected). The behaviors observed include language stimulation, positive talk (e.g., praise, encouragement), positive physical contact and other behaviors (e.g., positive affect, stimula- tion of social development, restricting activity, speaking negatively to child, etc.) as well as the amount of time the child positively or negatively interacted with the caregiver and other children. The ORCE composite quality ratings predicted concurrent and later child outcomes in the 10-site NICHD Study of Early Child Care and Youth Development in analyses that adjusted for family demographic and parenting characteristics. Children who experienced more responsive and stimulating care according to the ORCE consistently had high language and cognitive scores and tended to have better social skills while in child care (NICHD Early Child Care Research Network, 2006) and to demonstrate better language skills through fifth grade (Belsky et al., 2007) and better academic skills through third grade (NICHD Early Child Care Research Network, 2005). Preschool Classroom Mathematics Inventory The Preschool Classroom Mathematics Inventory (PCMI; National Institute for Early Education Research, 2007) was created to assess the quality of mathematics instruction for the preschool classroom and is modeled after Supports for the Early Literacy Assessment (see below). The 17 items assess instruction and learning opportunities related to (1) number (e.g., materials for counting, comparing number, and estimating; teachers encourage children to recombine and count); (2) mathematical concept (e.g.,

MEASURING QUALITY IN EARLY CHILDHOOD ENVIRONMENTS 171 measuring and comparing amounts, time, classifying, seriation); and (3) parents (efforts to involve parents in supporting children’s mathematical development). A 5-point scale is used, with a score of 5 representing strong evidence of a developmentally appropri- ate mathematics program. The one item on parents could not be given a score without a conversation with a teacher or director. This is the only measure that focuses entirely on mathematical learning opportunities. A limitation is that scores may not reflect the instructional program accurately because on any given day an observer is not likely to see the full range of mathematical activities that a program provides. To accurately reflect children’s opportunity to learn, it would be necessary to visit the program more than once or to rely on teacher or administrator reports. Preschool Program Quality Assessment The Preschool Program Quality Assessment (PQA; High/ Scope, 2003) provides an overall quality rating of the preschool classroom as well as descriptions of dimensions thought to define overall quality. It includes 63 5-point scales describing a broad array of program characteristics, with the endpoints (1 and 5) and the midpoint (3) defined and illustrated with examples. There are seven sections: (1) learning environment (e.g., defined interest areas, varied and open-ended materials, diversity-related materials); (2) daily routine (e.g., consistent, time for child- i ­ nitiated activities, small-group time); (3) adult-child interaction (e.g., warm and caring atmosphere, adults as partners in play); (4) curriculum planning and assessment (e.g., team teaching, comprehensive child records); (5) parent involvement and f ­ amily services (e.g., opportunities for involvement, staff-parent informal interactions); (6) staff qualifications and development (e.g., ongoing professional development, instructional staff back- ground); and (7) program management (e.g., program licensed, operating policies and procedures). Some of the items are rated following observations. Others require information provided by administrators. The observation items tend to emphasize efforts to promote children’s personal initiative, problem solving, and explorations. The PQA manual (High/Scope, 2003) states that scores for

172 EARLY CHILDHOOD ASSESSMENT preschool classrooms have been shown to predict concurrent measures of children’s language, and change in scores on the High/Scope child observation record, but gives little information on the studies that underlie these assertions. Supports for Early Literacy Assessment The Supports for Early Literacy Assessment (SELA; Smith and colleagues, in development) focuses on literacy learning opportunities in the preschool classroom. It consists of 20 items concerning: (1) the literacy environment (print used for a purpose, such as labeling; inviting places to look at books; array of books; writing materials available; literacy items and props in pretend area); (2) language development (encouragement to use and extend oral language; introduction of new words, concepts, and linguistic structures; activities to promote oral language; books shared); (3) print/books concepts (calling attention to functions and features of print); (4) phonological awareness; (5) letters and words (promoting letter recognition and interest in writing); (6) parent involvement (home-based supports for literacy; regular communication with parents); and (7) sites with English language learners, promoting maintenance and development of children’s native language. Scores range from 1 to 5, with 1 considered very low quality and 5 ideal quality. The measure is one of the few that provides substantial information on the literacy environment. One limitation is that some items require an interview with the teacher to complete. Supports for English Language Learners Classroom Assessment The Supports for English Language Learners Classroom Assessment (SELLCA; National Institute for Early Education Research, 2005) consists of 8 items, with scores ranging from 1 (minimal evidence) to 5 (strong evidence). It assesses the degree to which the teacher incorporates the cultural backgrounds of the children in the classroom and encourages parent participation; provides literacy materials and encourages children to use their native language; and supports English language development.

MEASURING QUALITY IN EARLY CHILDHOOD ENVIRONMENTS 173 Observations need to be supplemented with an interview of the director or a teacher to complete the scale. STRATEGY FOR ASSESSING PROGRAM QUALITY We have described direct observation as a strategy for a ­ ssessing program quality, focusing particularly on systematic assessments of practices that are believed or known to be associ- ated with child outcomes and that yield numerical scores, allow- ing comparisons over time and across classrooms. Such measures can serve several related purposes. Many classroom observation measures exist that can be used or adapted to meet the specific needs of a program. Prior to select- ing a measure, it is necessary to be clear about the goals of the pro- gram and the criteria for quality. Available measures vary along several dimensions. First, they vary in whether they focus on the child care or educational experiences of the individual child or the entire classroom. Second, some measures provide a global assessment of the child care experiences, whereas other measures are designed to focus more closely on a specific aspect of those experiences. Third, they vary in how much they focus on various program qualities—the socioemotional context versus opportuni- ties for children to develop academic skills, for example. Finally, many measures were designed for preschool classrooms, but some were designed to measure home-based child care or child care for infants and toddlers. We note here that there is research underway examining current quality rating systems. One recent study by the Rand Corporation (2008) addressed aspects of the validity of the “Quali- star” rating system, implemented in child care centers and family care sites serving over 1,300 children. Centers showed improve- ment in measured program quality during the course of the study, but the authors found little evidence that quality ratings predicted child outcomes, and problems were found with the data used for some of the component measures in the system. The study had significant technical problems, including high child attrition, which limited the conclusions that could be drawn. More work examining existing quality rating systems could provide welcome information for those charged with assessing program quality.

APPENDIX TABLE 6-1  Environmental Observation Instruments 174 Used for Features of Environment Observed Physical Social/ Learning Language Age Type of Environment, Emotional Environment/ and Descriptive Instrument Group Setting Materialsa Climateb Opportunities Literacy Math Detailc Ratings of Parent- 6 months- Home or ** ** Child Interactions 11 years lab Quality of Instruction 2-5 years Home or ** lab Home Observation for 6 months- Home ** ** ** ** Measurement of the 5 years Environment Assessment Profile Infant, Center ** ** ** * * for Early Childhood toddler, Programs (APECP) preschool Caregiver Interaction Infant, All child * Scale (cis) toddler, care preschool Child/Home Early Preschool Home- ** * ** ** Language and based Literacy Observation child care (CHELLO) Classroom Preschool- Center/ ** ** ** Assessment Scoring 3rd grade school System (CLASS)

Classroom Practices 4-5 years Center ** ** * * Inventory (CPI) A Developmentally 1st-3rd School ** * * * Appropriate Practices grades Template (ADAPT) Early Childhood 2.5-5 years Center ** * * * ** Environment Rating Scale-Revised (ECERS-R) Early Childhood 2.5-5 years Center * ** ** ** ** Environment Rating Scale-Extension (ECERS-E) Early Childhood 4-7 years Center/ * ** ** * * ** Classroom school Observation Measure (ECCOM) Early Language and Pre-K-3rd Center/ * * ** * Literacy Classroom grade school Observation (ellco) Emerging Academics 10 Center/ * * * * Snapshot (EAS) months- school 8 years 175 continued

APPENDIX TABLE 6-1  Continued 176 Used for Features of Environment Observed Physical Social/ Learning Language Age Type of Environment, Emotional Environment/ and Descriptive Instrument Group Setting Materialsa Climateb Opportunities Literacy Math Detailc Family Day Care Infant- Home- * Rating Scale 12 years based (FDCERS) child care Infant and Toddler Birth- Center ** * * * ** Environmental 30 months Rating Scale-Revised (ITERS-R) Observation Measure Preschool Center * ** ** of Language and Literacy Instruction (OMLIT) Observation Record Available All child ** of the Caregiving for 6-54 care Environment (ORCE) months Preschool Classroom Preschool Center ** ** Mathematics Inventory (PCMI)

Preschool Program Preschool Center ** * * * ** Quality Assessment, 2nd ed. (PQA) Supports for Early 3-5 years Center ** ** Literacy Assessment (SELA) Supports for Preschool Center ** ** English Language Learners Classroom Assessment (SELLCA) NOTES: Single asterisk = Instrument provides some representation of this feature. Two asterisks = Instrument provides substantial representation of this feature. aSafety, physical arrangement, materials. bEmotional climate, social interactions with adults, support for social skill development. cLevel of detail in descriptions. 177

Next: Part III: How to Assess »
Early Childhood Assessment: Why, What, and How Get This Book
×

The assessment of young children's development and learning has recently taken on new importance. Private and government organizations are developing programs to enhance the school readiness of all young children, especially children from economically disadvantaged homes and communities and children with special needs.

Well-planned and effective assessment can inform teaching and program improvement, and contribute to better outcomes for children. This book affirms that assessments can make crucial contributions to the improvement of children's well-being, but only if they are well designed, implemented effectively, developed in the context of systematic planning, and are interpreted and used appropriately. Otherwise, assessment of children and programs can have negative consequences for both. The value of assessments therefore requires fundamental attention to their purpose and the design of the larger systems in which they are used.

Early Childhood Assessment addresses these issues by identifying the important outcomes for children from birth to age 5 and the quality and purposes of different techniques and instruments for developmental assessments.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!