6
Measuring Quality in Early Childhood Environments

The domains of importance in early childhood all show mutability as a result of aspects of the environment. In this chapter, we review measures of quality in family and in early care and educational environments. Sometimes the family or the quality of the early care and educational setting is an outcome in its own right—the target of an intervention, for example. In other cases, it is a mediator of the effects of an intervention (e.g., improving family financial resources, introducing a new preschool curriculum, providing professional development) on child-level outcomes. In both these cases, it is crucial to have reliable and usable instruments from which one can draw valid inferences about the quality of the environment.

Infants, toddlers, and young children need supportive, responsive, and stimulating relationships with caregivers and stimulating and safe environments to thrive (McCartney and Phillips, 2006; National Research Council and Institute of Medicine, 2000). The National Academies synthesis of research on early development From Neurons to Neighborhoods concluded that “early environments matter and nurturing relationships are essential” (National Research Council and Institute of Medicine, 2000, p. 4). Families provide the primary care for children and are often the focus of early intervention programs. Home visiting programs are designed to promote positive, supportive parenting and to



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 145
6 Measuring Quality in Early Childhood Environments T he domains of importance in early childhood all show mutability as a result of aspects of the environment. In this chapter, we review measures of quality in family and in early care and educational environments. Sometimes the family or the quality of the early care and educational setting is an out- come in its own right—the target of an intervention, for example. In other cases, it is a mediator of the effects of an intervention (e.g., improving family financial resources, introducing a new preschool curriculum, providing professional development) on child-level outcomes. In both these cases, it is crucial to have reliable and usable instruments from which one can draw valid inferences about the quality of the environment. I nfants, toddlers, and young children need supportive, responsive, and stimulating relationships with caregivers and stimulating and safe environments to thrive (McCartney and Phillips, 2006; National Research Council and Institute of Medi- cine, 2000). The National Academies synthesis of research on early development From Neurons to Neighborhoods concluded that “early environments matter and nurturing relationships are essential” (National Research Council and Institute of Medicine, 2000, p. 4). Families provide the primary care for children and are often the focus of early intervention programs. Home visiting programs are designed to promote positive, supportive parenting and to 

OCR for page 145
 EARLY CHILDHOOD ASSESSMENT reduce harsh negative parenting of infants, thereby indirectly enhancing their cognitive and social development (Wasik and Bryant, 2001). State or federally funded child care and educational programs are designed to promote children’s cognitive, academic, and social skills directly (National Institute for Early Education Research, 2006). Parents and policy makers want to know about the quality of programs or family environments to ensure that they are enhancing, or at least not harming, children’s develop- ment. Accordingly, assessing children’s home and center-based environments, as well as child outcomes, has become an impor- tant part of assessment systems for young children (Adams, Tout, and Zaslow, 2007; Mitchell, 2005). OBSERVATIONAL MEASURES FOR MULTIPLE PURPOSES Many observational measures have been developed to assess the quality of home or early childhood care and education pro- grams. Selection of a measure requires consideration of the child population, the purpose of the observations, and the domains of most interest. For a program serving English language learners, for example, opportunities for children to develop language and vocabulary in their native language as well as English would be particularly important. Observational measures serve a number of purposes. First, they can be used for caregiver and teacher professional develop- ment. They can call administrators’ and caregivers’ or teachers’ attention to their own behaviors and practices that might pro- mote positive child outcomes. Having caregivers and teachers evaluate their own or each other’s classrooms and home-based care settings, as well as having two people (either an admin- istrator and a caregiver/teacher or two caregivers/teachers) evaluate the same setting, can be instructive and can provide good material for discussion. Administrators of formal early care and education programs—such as child care centers, preschools, pre- kindergartens, and Head Start programs—can also use classroom observation measures as part of their teacher/caregiver evalua- tion strategy, as a more objective, sharable set of criteria for obser- vation. Several promising professional development programs use observational measures as the basis for improving quality

OCR for page 145
 MEASURING QUALITY IN EARLY CHILDHOOD ENVIRONMENTS of child care. For example, Pianta and colleagues use their tool, the CLASS (Pianta, La Paro, and Hamre, 2007), to promote more intentional instruction, classroom management, and emotional support in the classroom through their professional program, My Teaching Partner (Kinzie et al., 2006). The Quality Interven- tions for Early Care and Education (QUINCE) intervention and evaluation, which uses on-site technical assistance to improve the quality of home-based as well as center-based child care, uses the environmental ratings scales, the Family Day Care Environment Rating Scale, or FDCERS (Harms and Clifford, 1989), and the Early Childhood Environment Rating Scale-Revised, or ECERS-R (Harms, Clifford, and Cryer, 1998), to promote the use of age- appropriate activities and enhance teacher-child interactions in their program, which follows the Partners for Inclusion model (Bryant, 2007; Wesley, 1994). Second, observational measures can be used in formative assessment of programs that are striving to improve their quality. Periodic observations and examination of scores on different dimensions can help identify weaknesses that require further attention. Fourteen states now have quality ratings systems avail- able to the public, with summary ratings of the quality of early care and education, and many more states are developing such systems, with the aim of improving information to consumers and providing supports to improve quality (Tout, Zaslow, and Martinez-Beck, forthcoming). Local communities as well are developing such systems. In most fully developed state quality ratings systems, an observational measure of the quality of the early care and education environment—usually the ECERS-R, FDCERS, or the infant and toddler version of this measure, the Infant/Toddler Environment Rating Scale (Harms, Cryer, and Clifford, 1990)—is used as one component of the overall rating of the environment, which usually includes multiple components, selected and weighted differently in each state or community. The rating of the environment is used not only as a contributor to the summary rating of quality, but also as a source of detailed infor- mation about the facets of quality that need improvement and in which changes will help progress to the next quality rating. Third, classroom observations can be used for accountability purposes, instead of or as a supplement to child outcome mea-

OCR for page 145
 EARLY CHILDHOOD ASSESSMENT sures. Child care quality has been a consistent modest to moder- ate positive predictor of children’s cognitive and language skills in large, multisite studies and smaller local studies (Howes et al., 2008; Lamb, 1998; NICHD Early Child Care Research Network, 2006; Peisner-Feinberg et al., 2001; Vandell, 2004) and a some- what consistent predictor of social skill (NICHD Early Child Care Research Network, 2006; Peisner-Feinberg et al., 2001; Vandell, 2004). Using early childhood assessments as part of an aligned system requires the capacity to juxtapose information about quality in the early care and education setting with change scores on children’s development (along with other key components). Thus, a complete system will require both ratings of the envi- ronment and assessments of children at multiple points in time, although this is expensive. In some federal and state efforts, observations of early care and education settings serve both a monitoring and accountabil- ity function and a formative function, providing information to improve quality. Thus, for example, as part of monitoring and accountability, the Head Start Impact Study collected observa- tions of the quality of Head Start programs as well as of formal early care and education programs serving children in the con- trol group (U.S. Department of Health and Human Services, Administration for Children and Families, 2005). Similarly, the Head Start Family and Child Experiences Survey (FACES) regu- larly collects observational data on a nationally representative sample of Head Start programs. The observational data are used in combination with child outcome data as part of ongoing pro- gram monitoring. However, the observational ratings and child outcomes together are also used to inform ongoing program improvement (see discussion in Zaslow, 2008). As one example, information from Head Start FACES was instrumental in shap- ing an increased focus in Head Start programs on early literacy development. Information from the Head Start Impact Study has also been instrumental in increasing professional development for Head Start teachers, focusing on early mathematics develop- ment in young children and how best to foster it. F ourth, classroom observations are useful for research. Indeed, most measures were originally developed as part of a research initiative. An extensive body of work looks at the rela-

OCR for page 145
 MEASURING QUALITY IN EARLY CHILDHOOD ENVIRONMENTS tionship of observational measures to child outcomes, especially in classroom-type early care and educational settings, and to a lesser extent in home-based care (Bryant, forthcoming; Burchinal, forthcoming; Burchinal et al., 2008). In addition, observational measures are used in evaluation studies to assess whether an intervention to improve practice in home-based or center-based early care and educational settings has affected caregiver/teacher practice or overall quality (for example, Bryant, 2007, and Pianta, 2007). An observational measure designed to assess parenting skills as a tool in caregiver or teacher professional development or for formative assessment should be detailed and descriptive so that it can help to direct improvement. In contrast, a measure used for research, summative assessment, or for accountability purposes, even if detailed, should be easily summarized in quan- tifiable ratings, so that scores can be compared over time and across settings. Purposes, in turn, have implications for who conducts the observation. If the goal is professional development or formative assessment, observations might be done by individuals directly involved. For example, observations of parenting skills might be done by a home visitor; a child care program teacher or admin- istrator could do observations of early care and education. If summative assessment or accountability is a goal, it is preferable that observation measures be administered by someone who is not directly connected to the program being evaluated, although program staff may sometimes perform this role if sufficient safe- guards are in place to ensure the reliability and validity of the observations. Most existing measures assess the social environment well and the learning environment at a very general level, but only a few adequately assess practices designed to teach academic or social skills specifically. Development of observational mea- sures is just beginning to catch up with the increased political emphasis on academic preparation in programs for young chil- dren (National Institute for Early Education Research, 2006). We summarize below some existing observational measures of the home and center-based environments, without attempting to be exhaustive. For all of these measures, there is some evidence for

OCR for page 145
0 EARLY CHILDHOOD ASSESSMENT their reliability and validity, and many include demonstrated associations with child outcomes. Note that although home or classroom observations may not be as labor-intensive or expensive as assessing individual child outcomes, all of these measures require a fair amount of assessor/ observer training for the results to be valid and reliable. Develop- ers of observational systems should provide clear and sufficiently intensive training criteria. Publishers of some instruments, like the CLASS (Pianta, La Paro, and Hamre, 2007), require training to be conducted by a publisher-trained and certified trainer, with different training for different user purposes. As specified by the developers, many natural observations of center-based or home settings require a minimum of 3 hours to ensure that sufficient sampling of the environment has occurred. Semistructured observations or interviews can require less time because they draw on specific kinds of interactions across all participants. Recommended times for the measures in Appendix Table 6-1 range from 1.5 hours to 2 half-days. Details on these measures can be found in the literature cited and, for many, in a compendium profiling observational measures for early childhood care and education environments prepared by Child Trends (Child Trends, 2007). Appendix Table 6-1 is a summary of some important characteristics of each measure discussed. The stars indicate that the dimension is represented somewhat (one star) or substantially (two stars). ASSESSING HOME ENVIRONMENTS Parents “structure the experiences and shape the environ- ments within which early development unfolds” (National Research Council and Institute of Medicine, 2000, p. 226). Chil- dren’s early cognitive, social, and physical development are all clearly linked to their relationships with their primary caregivers and the kinds of experiences available in their home environments (McCartney and Phillips, 2006). Theories of development focus on two overlapping dimensions related to assessing families in early childhood. The quality of relationships between the child and his or her primary caregivers is viewed as central for all forms of development, especially socioemotional skills (Bornstein and

OCR for page 145
 MEASURING QUALITY IN EARLY CHILDHOOD ENVIRONMENTS Sawyer, in McCartney and Phillips, 2006, pp. 381-398). And the quality of cognitive stimulation clearly plays a critical role for cognitive, language, and social development (Bradley et al., 2001; Fuligni, Han, and Brooks-Gunn, 2004). There have been many theoretical and empirical systems developed for describing how families affect children’s cogni- tive, language, and social development. These systems almost always include at least three dimensions: (1) the quality of the parent-child relationships and more distally of mother-father and whole-family relationships, (2) the quality of stimulation provided directly by caregivers in interactions with the child and by the objects that are available in the family environment, and (3) provision of basic needs and safety monitoring. These dimen- sions are listed below with a set of characteristics that are believed to be important for each dimension: 1. Relationships (mother-child, father-child, other primary caregiver-child, and more distally mother-father), emo- tional climate, social interactions, support for social skills development, and discipline strategies: A. Degree to which adults are affectionate, supportive, attentive, and respectful toward children. B. Explicit support for social skills (e.g., encouraging children to “use their words,” modeling and engaging children in conversations about social problem-solving skills, encouraging the use of learned strategies to solve real social conflicts). C. Degree to which primary caregivers use consistent behavior management techniques that are not harsh or demeaning. 2. Cognitive stimulation: A. Extent to which primary caregivers use the home envi- ronment to provide and scaffold learning activities for the child. B. Degree to which primary caregivers provide stimulating activities in the community. C. Degree to which primary caregivers talk to the child, engage the child in conversation, and use elaborated language in those verbal interactions.

OCR for page 145
 EARLY CHILDHOOD ASSESSMENT D. Frequency with which children are exposed to books and have books read to them. E. Literacy resources (e.g., books, magazines, writing mate- rials, computers) in the home. 3. Provision of basic needs and safety monitoring: A. Degree to which the home environment is free of haz- ards, clean, and organized. B. Degree to which toys, books, and other child-friendly m aterials are available to the child without adult mediation. C. Presence of or access to outdoor play areas or areas in which gross motor play can occur. Primary Caregiver-Child Interactions Primary caregiver-child interactions typically either are coded from videotapes of semistructured 10- to 20-minute observations in which the primary caregiver is asked to engage the child in age-appropriate activities or are rated live during longer observa- tions in the home. An example is a measure used in the National Institute of Child Health and Human Development (NICHD) Study of Early Child Care and Youth Development (SECCYD) (NICHD Early Child Care Research Network, 1999, 2003) and the Early Head Start study (U.S. Department of Health and Human Services, Administration for Children and Families, 2004); differ- ent procedures are used for the youngest infants, toddlers, and preschoolers. Mothers of 6-month-olds were asked to play with their infants for 7 minutes and were told that they could use any toy or object available in the home or none at all. For the remain- ing 8 minutes, mothers were given a standard set of toys they could use in play. At 15, 24, and 36 months, the observation protocol followed a three-boxes procedure in which mothers were asked to show their children age-appropriate toys in three containers in a set order. The mother was asked to have her child play with the toys in each of the three containers and to do so in the order specified, but she was told she could spend as long or as little time on each activity as she wished. Videotapes were coded by research assis- tants who had attended centralized training sessions. At 6, 15, and

OCR for page 145
 MEASURING QUALITY IN EARLY CHILDHOOD ENVIRONMENTS 24 months, mothers were rated on a 4-7 point scale (ranging from “not at all characteristic” to “highly characteristic”) to describe maternal sensitivity to child nondistress, cognitive stimulation, intrusiveness, positive regard, and negative regard. At 36 and 54 months, the mothers were rated on 7-point ratings of support- ive presence, hostility (reversed), and respect for autonomy. The composite scores were the strongest predictor of chil- dren’s cognitive, language, academic, and social outcomes when considered with demographic, parental attitude, and schooling characteristics in the NICHD SECCYD (NICHD Early Child Care Research Network, 2006). Similar observational systems have been developed and used for attachment (e.g., Egeland and Deinard, 1975), special education (e.g., Yoder and Warren, 2001), and effects of differing welfare reform policies on children (Weinfield, Egeland, and Ogawa, 1998). These observation proce- dures can also be used in assessing the quality of out-of-home care for infants and toddlers. Cognitive Stimulation Cognitive stimulation is reflected in a dimension called quality of instruction, which is often measured using a videotaped laboratory procedure in which mother and child pairs participate in a series of developmentally appropriate problem-solving situ- ations (Englund et al., 2004). The mother’s instructional behavior is rated on a 7-point scale that reflects how well she structured the situation and coordinated her behavior to the child’s activity and needs for assistance. The scale ranges from 1, indicating poor quality of instruction (uninvolved or unstructured), to 7, indicat- ing effective instruction throughout the session. The rating from this measure correlated with subsequent scores on standardized achievement tests in several studies (see, e.g., Connell and Prinz, 2002; Englund et al., 2004; Pianta and Egeland, 1994; Pianta and Harbers, 1996; Pianta, Egeland, and Sroufe, 1990; Pianta, Nimetz, and Bennett, 1997). Other observational rating systems focus on the quality of cognitive (DeTemple and Snow, 1998) or affective (Frosch, Cox, and Goldman, 2001) interaction specifically sur- rounding book reading. Recently the Child/Home Early Language and Literacy

OCR for page 145
 EARLY CHILDHOOD ASSESSMENT Observation (CHELLO) observation measure was developed to assess the quality of early childhood language and literacy practices in mixed-age home-based child care settings (Neuman, Dwyer, and Koh, 2007). The measure complements a classroom observation measure (ELLCO) described below. A checklist is used to assess the literacy environment (books, writing materials, and cognitively stimulating toys) and a 1-5 rating scale includes items that assess the physical environment for learning, support for learning, and teaching strategies (e.g., vocabulary building, use of print, storytelling). The CHELLO total score has been shown to be correlated with growth in children’s language skills (PPVT), phonological skills, and ability to do language-oriented mathematical problems. Home Observation for Measurement of the Environment We single out the Home Observation for Measurement of the Environment (HOME; Caldwell and Bradley, 1984) for discussion because it is such a widely used assessment of the home environ- ment. The focus is on the child in the environment, experiencing objects, events, and transactions occurring in connection with the family surroundings. There are separate forms for assessing infants and toddlers and older children. The infant/toddler ver- sion of the inventory (IT-HOME) focuses on infancy (birth to age 3 years). It is composed of 45 items clustered into 6 subscales: (1) parental responsivity, (2) acceptance of child, (3) organization of the environment, (4) learning materials, (5) parental involvement, and (6) variety in experience. Each item is scored in binary fashion (yes/no). Information used to score the items is obtained during the course of the home visit by means of observation and semis- tructured interview. The early childhood version of the inventory (EC-HOME) is used during early childhood (ages 3 to 6). It is composed of 55 items clustered into 8 subscales: (1) learning materials, (2) lan- guage stimulation, (3) physical environment, (4) responsivity, (5) academic stimulation, (6) modeling, (7) variety, and (8) accep- tance. Each item is scored in binary fashion (yes/no). Information on items is obtained either through observation or through asking the mother. Typically the total score is used, although a recent

OCR for page 145
 MEASURING QUALITY IN EARLY CHILDHOOD ENVIRONMENTS factor analysis (Fuligni et al., 2004) produced scales that appear to differentiate between stimulation in the home environment for language and literacy and for social development. As with parental sensitivity, the quality of the home envi- ronment has been shown to be a moderate to strong predictor of academic and social outcomes for young children regardless of income or ethnicity (Bradley et al., 2001). Zaslow et al. (2006) found that the HOME, direct observations of mother-child inter- action, and maternal self-report measures collected during the preschool years all predicted child outcomes during middle childhood in a low-income sample with family background char- acteristics controlled. However, direct observations showed the strongest pattern of prediction. CENTER-BASED EARLY CHILDHOOD ENVIRONMENTS Early childhood care and education programs are increas- ingly being held accountable for their effects on children’s development, and thus assessments of quality are needed. As noted earlier, measures of quality are also being used to inform efforts to improve quality at the community and state levels, and in research evaluating specific quality improvement efforts. The specific dimensions measured will vary as a function of program goals, as discussed throughout this report. It is important for measures of the environment to be used in conjunction with mea- sures of child outcomes to provide a context for understanding the extent to which children show positive development dur- ing the time they are participating in early care and education. This section describes strategies for assessing program quality directly. Many indicators that have been connected to child outcomes are fairly easy to quantify. Examples are staff-child ratios, number of children in a classroom, amount spent per child, the training and experience of teachers, and teacher turnover. Other quality variables are less easily quantifiable but are nonetheless impor- tant, such as opportunities for professional development for staff and the nature of the curriculum. Information on these variables is best obtained by interviews with program directors, surveys, or inspection of records. Some of these indicators, such as teacher/

OCR for page 145
 EARLY CHILDHOOD ASSESSMENT Infant and Toddler Environmental Rating Scale The Infant and Toddler Environmental Rating Scale-Revised (ITERS-R; Harms et al., 1990) uses the same format as the ECERS-R, but it is designed to assess center-based child care programs for infants and toddlers. The 43 items are organized to cover seven categories: (1) space and furnishings, (2) personal care routines, (3) listening and talking, (4) activities, (5) interaction, (6) program structure, and (7) parents and staff. W hile many studies have used the ITERS to document the quality of infant/toddler center care (e.g., Helburn, 1995), relatively few studies have also measured the infants or toddlers themselves. In one study that measured both the classroom and home environments and infant outcomes, the ITERS total score predicted both the level and rate of change in infant and toddler’s language and IQ scores in a study of black children attending center-based care (Burchinal, Roberts, et al., 2000). Observation Measure of Language and Literacy Instruction The Observation Measure of Language and Literacy Instruc - tion (OMLIT; Abt Associates Inc., 2006) focuses on measuring the quality of the literacy practices in preschool classrooms. It is a battery of observation instruments that assess instructional practices and qualities of the environment in early childhood education classrooms that have been shown to support the development of oral language and emergent literacy skills. A classroom description is also included that provides contextual information, such as the number of children, their ages, and the languages they speak and that are used in instruction. There are five instruments: 1. The Classroom Literacy Opportunities Checklist is an inventory of 54 classroom literacy resources in 7 categories (e.g., text material and reading/listening areas; writing materials and writing area; diversity in the literacy materi- als). Items are coded either on a 3-point scale (minimally supplied, adequately supplied, well-supplied) or a 2-point scale (minimally supplied or well-supplied).

OCR for page 145
 MEASURING QUALITY IN EARLY CHILDHOOD ENVIRONMENTS 2. The Snapshot of Classroom Activities identifies literacy activities and integration of literacy materials in other activities, languages spoken, and count of adults and chil- dren present. 3. The Read Aloud Profile assesses dialogic reading practices on seven dimensions (e.g., pre-reading “set up,” strategies used while reading, language(s) used). 4. The Classroom Literacy Instruction Profile describes the literacy activities and the instructional methods used by staff. Staff in the classroom are followed for 10 minutes at 15-minute intervals over the observation period, coding literacy “events.” Codes are both descriptive (the strate- gies used) and evaluative (e.g., the cognitive challenge presented by the dialogue/discussion between the staff member and the children). 5. The Quality of Instruction in Language and Literacy mea- sure rates the frequency and quality of literacy instruction and support for language and literacy development. Each of the 11 items is rated on a 5-point scale. The OMLIT is extraordinarily detailed and comprehensive, and accompanying materials provide an extensive rationale for the choice of items. It is unlikely that all of the scales would be used, but specific selections could be made. Observation Record of the Caregiving Environment The Observation Record of the Caregiving Environment (ORCE; National Institute of Child Health and Human Develop- ment, NICHD Early Child Care Research Network, 2000) focuses on the sensitivity and responsiveness of the caregiver to an indi- vidual child. It can be used in home- or center-based settings for infants, toddlers, and preschoolers. It is collected in one or two 3-hour visits to the child’s home- or center-based care. The observer collects time-sampling observations of behaviors and completes ratings of the child’s caregiver. The behavior scales provide a record of the occurrence or quantity of specific acts, and the qualitative scales take into account the quality (and nuances) of the caregiver’s behavior in relation to the child’s behavior. The most frequently

OCR for page 145
0 EARLY CHILDHOOD ASSESSMENT used quality measure, the positive caregiving composite, is cal- culated slightly differently for each age level. At 6, 15, and 24 months, positive care-giving composite scores are the mean of five 4-point qualitative ratings (sensitivity to child’s nondistress signals, stimulation of cognitive development, positive regard for child, emotional detachment [reflected], flatness of affect [reflected]). At 36 months, these five scales plus two additional subscales, “fosters child’s exploration” and “intrusive” [reflected], are included in the composite. At 54 months, the positive caregiving composite is the mean of 4-point ratings of caregivers’ sensitivity/responsivity, stimulation of cognitive development, intrusiveness (reflected), and detachment (reflected). The behaviors observed include language stimulation, positive talk (e.g., praise, encouragement), positive physical contact and other behaviors (e.g., positive affect, stimula- tion of social development, restricting activity, speaking negatively to child, etc.) as well as the amount of time the child positively or negatively interacted with the caregiver and other children. The ORCE composite quality ratings predicted concurrent and later child outcomes in the 10-site NICHD Study of Early Child Care and Youth Development in analyses that adjusted for family demographic and parenting characteristics. Children who experienced more responsive and stimulating care according to the ORCE consistently had high language and cognitive scores and tended to have better social skills while in child care (NICHD Early Child Care Research Network, 2006) and to demonstrate better language skills through fifth grade (Belsky et al., 2007) and better academic skills through third grade (NICHD Early Child Care Research Network, 2005). Preschool Classroom Mathematics Inventory The Preschool Classroom Mathematics Inventory (PCMI; National Institute for Early Education Research, 2007) was created to assess the quality of mathematics instruction for the preschool classroom and is modeled after Supports for the Early Literacy Assessment (see below). The 17 items assess instruction and learning opportunities related to (1) number (e.g., materials for counting, comparing number, and estimating; teachers encourage children to recombine and count); (2) mathematical concept (e.g.,

OCR for page 145
 MEASURING QUALITY IN EARLY CHILDHOOD ENVIRONMENTS measuring and comparing amounts, time, classifying, seriation); and (3) parents (efforts to involve parents in supporting children’s mathematical development). A 5-point scale is used, with a score of 5 representing strong evidence of a developmentally appropri- ate mathematics program. The one item on parents could not be given a score without a conversation with a teacher or director. This is the only measure that focuses entirely on mathematical learning opportunities. A limitation is that scores may not reflect the instructional program accurately because on any given day an observer is not likely to see the full range of mathematical activities that a program provides. To accurately reflect children’s opportunity to learn, it would be necessary to visit the program more than once or to rely on teacher or administrator reports. Preschool Program Quality Assessment The Preschool Program Quality Assessment (PQA; High/ Scope, 2003) provides an overall quality rating of the preschool classroom as well as descriptions of dimensions thought to define overall quality. It includes 63 5-point scales describing a broad array of program characteristics, with the endpoints (1 and 5) and the midpoint (3) defined and illustrated with examples. There are seven sections: (1) learning environment (e.g., defined interest areas, varied and open-ended materials, diversity-related materials); (2) daily routine (e.g., consistent, time for child- initiated activities, small-group time); (3) adult-child interaction (e.g., warm and caring atmosphere, adults as partners in play); (4) curriculum planning and assessment (e.g., team teaching, c omprehensive child records); (5) parent involvement and family services (e.g., opportunities for involvement, staff-parent informal interactions); (6) staff qualifications and development (e.g., ongoing professional development, instructional staff back- ground); and (7) program management (e.g., program licensed, operating policies and procedures). Some of the items are rated following observations. Others require information provided by administrators. The observation items tend to emphasize efforts to promote children’s personal initiative, problem solving, and explorations. The PQA manual (High/Scope, 2003) states that scores for

OCR for page 145
 EARLY CHILDHOOD ASSESSMENT preschool classrooms have been shown to predict concurrent measures of children’s language, and change in scores on the High/Scope child observation record, but gives little information on the studies that underlie these assertions. Supports for Early Literacy Assessment The Supports for Early Literacy Assessment (SELA; Smith and colleagues, in development) focuses on literacy learning opportunities in the preschool classroom. It consists of 20 items concerning: (1) the literacy environment (print used for a purpose, such as labeling; inviting places to look at books; array of books; writing materials available; literacy items and props in pretend area); (2) language development (encouragement to use and extend oral language; introduction of new words, concepts, and linguistic structures; activities to promote oral language; books shared); (3) print/books concepts (calling attention to functions and features of print); (4) phonological awareness; (5) letters and words (promoting letter recognition and interest in writing); (6) parent involvement (home-based supports for literacy; regular communication with parents); and (7) sites with English language learners, promoting maintenance and development of children’s native language. Scores range from 1 to 5, with 1 considered very low quality and 5 ideal quality. The measure is one of the few that provides substantial information on the literacy environment. One limitation is that some items require an interview with the teacher to complete. Supports for English Language Learners Classroom Assessment The Supports for English Language Learners Classroom Assessment (SELLCA; National Institute for Early Education Research, 2005) consists of 8 items, with scores ranging from 1 (minimal evidence) to 5 (strong evidence). It assesses the degree to which the teacher incorporates the cultural backgrounds of the children in the classroom and encourages parent participation; provides literacy materials and encourages children to use their native language; and supports English language development.

OCR for page 145
 MEASURING QUALITY IN EARLY CHILDHOOD ENVIRONMENTS Observations need to be supplemented with an interview of the director or a teacher to complete the scale. STRATEGY FOR ASSESSING PROGRAM QUALITY We have described direct observation as a strategy for assessing program quality, focusing particularly on systematic assessments of practices that are believed or known to be associ- ated with child outcomes and that yield numerical scores, allow- ing comparisons over time and across classrooms. Such measures can serve several related purposes. Many classroom observation measures exist that can be used or adapted to meet the specific needs of a program. Prior to select- ing a measure, it is necessary to be clear about the goals of the pro- gram and the criteria for quality. Available measures vary along several dimensions. First, they vary in whether they focus on the child care or educational experiences of the individual child or the entire classroom. Second, some measures provide a global assessment of the child care experiences, whereas other measures are designed to focus more closely on a specific aspect of those experiences. Third, they vary in how much they focus on various program qualities—the socioemotional context versus opportuni- ties for children to develop academic skills, for example. Finally, many measures were designed for preschool classrooms, but some were designed to measure home-based child care or child care for infants and toddlers. We note here that there is research underway examining current quality rating systems. One recent study by the RAND Corporation (2008) addressed aspects of the validity of the “Quali- star” rating system, implemented in child care centers and family care sites serving over 1,300 children. Centers showed improve- ment in measured program quality during the course of the study, but the authors found little evidence that quality ratings predicted child outcomes, and problems were found with the data used for some of the component measures in the system. The study had significant technical problems, including high child attrition, which limited the conclusions that could be drawn. More work examining existing quality rating systems could provide welcome information for those charged with assessing program quality.

OCR for page 145
APPENDIx TABLE 6-1 Environmental Observation Instruments  Used for Features of Environment Observed Physical Social/ Learning Language Age Type of Environment, Emotional Environment/ and Descriptive Instrument Group Setting Materialsa Climateb Opportunities Literacy Math Detailc Ratings of Parent- 6 months- Home or ** ** Child Interactions 11 years lab Quality of Instruction 2-5 years Home or ** lab Home Observation for ** ** ** ** 6 months- Home Measurement of the 5 years Environment Assessment Profile ** ** ** * Infant, Center * for Early Childhood toddler, Programs (APECP) preschool Caregiver Interaction Infant, All child * Scale (CIS) toddler, care preschool Child/Home Early ** * ** ** Preschool Home- Language and based Literacy Observation child care (CHELLO) Classroom ** ** Preschool- Center/ ** Assessment Scoring 3rd grade school System (CLASS)

OCR for page 145
Classroom Practices ** ** * 4-5 years Center * Inventory (CPI) A Developmentally ** * * 1st-3rd School * Appropriate Practices grades Template (ADAPT) Early Childhood ** * * * 2.5-5 years Center ** Environment Rating Scale-Revised (ECERS-R) Early Childhood * 2.5-5 years Center ** ** ** ** Environment Rating Scale-Extension (ECERS-E) Early Childhood * ** 4-7 years Center/ ** * * ** Classroom school Observation Measure (ECCOM) Early Language and * * Pre-K-3rd Center/ ** * Literacy Classroom grade school Observation (ELLCO) Emerging Academics 10 Center/ * * * * Snapshot (EAS) months- school 8 years  continued

OCR for page 145
APPENDIx TABLE 6-1 Continued  Used for Features of Environment Observed Physical Social/ Learning Language Age Type of Environment, Emotional Environment/ and Descriptive Instrument Group Setting Materialsa Climateb Opportunities Literacy Math Detailc Family Day Care * Infant- Home- Rating Scale 12 years based (FDCERS) child care Infant and Toddler ** * * * Birth- Center ** Environmental 30 months Rating Scale-Revised (ITERS-R) Observation Measure ** Preschool Center * ** of Language and Literacy Instruction (OMLIT) Observation Record Available All child ** of the Caregiving for 6-54 care Environment (ORCE) months Preschool Classroom Preschool Center ** ** Mathematics Inventory (PCMI)

OCR for page 145
Preschool Program ** * * * Preschool Center ** Quality Assessment, 2nd ed. (PQA) Supports for Early 3-5 years Center ** ** Literacy Assessment (SELA) Supports for Preschool Center ** ** English Language Learners Classroom Assessment (SELLCA) NOTES: Single asterisk = Instrument provides some representation of this feature. Two asterisks = Instrument provides substantial representation of this feature. aSafety, physical arrangement, materials. bEmotional climate, social interactions with adults, support for social skill development. cLevel of detail in descriptions. 

OCR for page 145