Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 301
Early Childhood Assessment: Why, What, and How 10 Thinking Systematically In this volume we have discussed the dimensions of assessment, including its purposes, the domains to be assessed, and guidelines for selecting, implementing, and using information from assessments. Beyond this, however, one cannot make use of assessments optimally without thinking of them as part of a larger system. Assessments are used in the service of higher level goals—ensuring the well-being of children and their families, ensuring that societal resources are deployed productively, distributing scarce educational or medical resources equitably, facilitating the relevance of educational outcomes to economic challenges, making informed decisions about contexts for the growth and development of children, and so on. Assessments by themselves cannot achieve these higher goals, although they are a crucial part of a larger system designed to address them. Only when the entire system is considered can reasonable decisions about assessment be made. This chapter argues that early childhood assessment needs to be viewed not as an isolated process, but as integrated in a system that includes a clearly articulated higher level goal, such as optimal growth, development, and learning for all children; that defines strategies for achieving the goal, such as adequate funding, excellent teaching practices, and well-designed educational environments; that recognizes the other elements of infrastructure
OCR for page 302
Early Childhood Assessment: Why, What, and How instrumental to achieving the goal, such as professional development and mechanisms for monitoring quality in the educational environment; and that selects assessment instruments and procedures that fit with the other elements in service of the goal. We begin by noting the multiple state and federal structures in which early childhood assessments are being implemented. These structures have emerged from different sources with different funding streams (e.g., federally funded Head Start, state-funded prekindergarten, foundation-funded intervention programs) and rarely display complete convergence of performance standards, criteria, goals, or program monitoring procedures. Thus, referring to “a larger system of early care and education” is slightly deceptive, or perhaps aspirational. Furthermore, even the well-established programs in the “system” may lack key components—for example, they may assess child outcomes but not relate those outcomes to measures of the environment, or they may not have a mechanism in place for sharing child outcome data in helpful ways with caregivers and teachers. We use recent National Research Council reports, state experiences with the No Child Left Behind Act, and the recent work of the Pew Foundation–sponsored National Early Childhood Accountability Task Force—a national effort focused on accountability in early childhood—as a basis for articulating the components needed in order for early childhood assessment to be part of a fully integrated system. We also provide some examples of progress toward this goal at the state level. Although we did not find any examples of fully integrated systems, in which services are provided by a single source and the assessment infrastructure is fully aligned and developed, the three states we describe are moving toward integrating early childhood assessment in a well-articulated system. WHAT DO WE MEAN BY A SYSTEM? The idea of a system comes up often in education discussions and analyses—there are education systems, instructional systems, assessment systems, professional development systems—but it is not always clear what the word actually means. Systems have a number of important features, which are enumerated in Systems
OCR for page 303
Early Childhood Assessment: Why, What, and How for State Science Assessment (National Research Council, 2006).1 In particular, they are organized around specific goals; they are made up of subsystems, each of which serves its own purposes; the subsystems must work well both autonomously and in harmony with one another for the larger system to work well; and a missing or poorly operating subsystem may cause a system to function poorly or not at all. In our use of the term with reference to early childhood assessment, the committee intends that assessments be seen as a part or subsystem2 of a larger system of early childhood care and education, which addresses the multiple aspects of child development and influences discussed in this volume; that selection of assessments be intimately linked to goals defined by that larger system; that procedures for sharing information about and using information from assessments be considered as part of the process of selecting and administering assessments; and that different parts of the assessment system itself (standards, constructs, measures, indicators) work together. Systems need to have well-developed feedback loops to prevent over- or undercompensation for changes in a single part. Feedback loops occur whenever an output of some subsystem connects back to one of its inputs. For example, a fundamental feedback loop occurs in the classroom when a teacher identifies problems that children are having with an idea or skill and adjusts his or her instructional techniques and the learning environment in response. When this causes the children to learn the idea or skill successfully, one would say that the feedback loop has worked effectively. Implementation of a similar feedback loop at the level of the program takes child performance as the input for identifying classrooms in which teachers need additional 1 This section and the following one on infrastructure draw heavily on the content of the National Research Council’s 2006 report, Systems for State Science Assessment. 2 Although assessment is here defined as a subsystem of a larger system, throughout this chapter we refer to the “assessment system” for the sake of simplicity, except when the distinction is important.
OCR for page 304
Early Childhood Assessment: Why, What, and How assistance in implementing instructional activities. These two subsystems—the individual- and the program-level feedback from child performance to teacher supports—function well as part of a larger system if the same or consistent information is used in both loops. However, if, for example, the teacher is responding to child performance so as to enhance creative problem solving, whereas the institution is encouraging teachers to focus on children’s rote memorization capacity, then the subsystems conflict and do not constitute a well-functioning system. In a well-designed program, the assessment subsystem is part of a larger system of early childhood care and education comprised of multiple interacting subsystems. These other systems include the early learning standards, which describe what young children should know and be able to do at the end of the program; the curriculum, which describes the experiences and activities that children will have; and the teaching practices, which describe the conditions under which learning should take place, including interactions among the teachers and children as well as the provisioning and organization of the physical environment (National Research Council, 2006). The relationships among these four subsystems are illustrated in Figure 10-1, adapted from the “curriculum, instruction, assessment (CIA) triangle” commonly cited in the educational assessment community. Each of these subsystems is also affected by other forces, for example, laws intended to influence what children are expected to learn, professional development practices, and teacher preparation policies influenced by professional organizations and accrediting agencies. We argue in this chapter that all these components must be thought of as part of a larger system, and that they must be designed so as to be coherent with one another, as well as with the policy and education system they are a part of, and with the goals for child development that the entire system is meant to be promoting. We reframe these arguments as a conclusion to this chapter. INFRASTRUCTURE FOR AN ASSESSMENT SYSTEM An early childhood assessment subsystem should be part of a larger system with a strong infrastructure that is designed to
OCR for page 305
Early Childhood Assessment: Why, What, and How FIGURE 10-1 A schematic relationship (the “CIA Triangle”) among early learning standards, curriculum, instructional practices, and assessment. provide high-quality early care and education. The infrastructure is the foundation on which the assessment subsystem rests and is critical to its smooth and effective functioning (National Early Childhood Accountability Task Force, 2007). The infrastructure encompasses several components that together form the system: Standards: A comprehensive, well-articulated set of standards for both program quality and children’s learning that are aligned to one another and that define the constructs focused on in assessment as well as the performance levels identified as acceptable. Assessments: Multiple approaches to documenting child performance and reviewing program quality that are of high quality and connect to one another in well-defined ways, from which strategic selection can be made depending on specific purposes. Reporting: A procedure, defined on the basis of the standards and the assessments, designed to maintain
OCR for page 306
Early Childhood Assessment: Why, What, and How an integrated, user-accessible database of assessment results, provide for quality assurance and integrity of data, and generate reports for the varied user audiences and purposes. Professional development: Ongoing opportunities provided to those at all levels (practitioners, assessment administrators, program directors, policy makers) to understand the standards and the assessments and to learn to use the data and data reports with integrity for their own purposes. Opportunity to learn: Procedures for ensuring that the environments in which children are spending time offer high-quality support for development and learning, as well as safety, enjoyment, and affectively positive relationships. This is crucial when decisions about children or programs are based on assessment outcomes. Inclusion: Methods and procedures for ensuring that all children served by the program will be assessed fairly, regardless of their language, culture, or disabilities, and with tools that provide the most useful information for fostering their development and learning. Resources: Assurance that the resources needed to ensure the development and implementation of the system components are available or will be recruited. Monitoring and evaluation: Procedures for continuously monitoring the system itself to ensure that it is operating effectively and that all elements are working together to serve the interests of the children. This infrastructure must be in place to create and sustain an assessment subsystem within a larger system of early childhood care and education. Ensuring the adequacy of each of these components raises some critical challenges. A challenge to the adoption of systems-level thinking about early childhood care and education, and thus about early childhood assessment, is the absence, under current U.S. policies, of a unified structure for early care and education. The current variety of separate programs segregated by setting, by agency, and by funding streams, with their numerous challenges to delivering uniformly high-quality early
OCR for page 307
Early Childhood Assessment: Why, What, and How care and education services, also serves as a barrier to developing a unified system of assessment. While the suggestion that these many barriers to an integrated system must be vaulted may seem unrealistic, we argue that a vision of a well-integrated, coherent system is needed to guide the development of policy for young children. We expand on the importance of each component of a well-organized system below. Standards The most fundamental aspect of the assessment system is the set of explicit goals for children’s development and learning around which the larger system is organized, thus providing the basis for coherence among the various elements. In most educational settings, these are referred to as “standards,” but in early childhood education sometimes other terms, such as “guidelines” or “foundations,” have been used. Whatever they are named, these standards direct the design of curriculum, the choice of teaching practices, and the priorities of teachers in setting instructional goals, planning activities and experiences, and organizing the environment. They are the starting point for developing assessments, judging performance levels, and rating children’s and the program’s growth and performance. Standards are also the framework for reporting children’s performance to educators and the public and for focusing program improvement efforts. Note that, although these standards are to be applied to children’s performance, they can be used as one input in establishing accountability for teachers, centers, and states (National Research Council, 2006). Thus, while some may see holding teachers, early care and education settings, and states to these standards for children’s performance as potentially punitive, others argue that they constitute a defense of the right of children to a high-quality and fair early childhood environment. Note that when applying the same logic to the programs in which children are to be educated, an equivalent set of statements can be made regarding program standards. For example, consider the No Child Left Behind Act (NCLB), which requires states to have reading, mathematics, and science standards for K-12 education that must be of “high quality,”
OCR for page 308
Early Childhood Assessment: Why, What, and How although the act says relatively little about what characterizes standards of high quality. While we are emphatically not recommending that the NCLB regime be extended to early childhood education, it is important to understand the NCLB framework, as it is the most common reference point on standards in the United States, and states are being asked by the federal government to align their preschool standards with their K-12 standards. Under the act, the word “standards” refers both to content standards and to achievement standards. The law requires states to develop challenging academic standards of both types, and a federal guidance document describes them as follows (U.S. Department of Education, 2004): Academic content standards must specify what all children are expected to know and be able to do; contain coherent and rigorous content; and encourage the teaching of advanced skills. Academic achievement standards must be aligned with the state’s academic content standards. For each content area, a state’s academic achievement standards must include at least two levels of achievement (proficient and advanced) that reflect mastery of the material in the state’s academic content standards, and a third level of achievement (basic) to provide information about the progress of lower-achieving children toward mastering the proficient and advanced levels. Note that achievement standards are often also referred to as performance standards. The NCLB-driven standards apply to children in grades 3-12 and link directly to the explicitly defined academic content areas that are also assessed in determining adequate yearly progress for schools. It would be inappropriate to borrow this model unchanged and apply it to early childhood settings, in which explicit instruction in well-defined academic content areas is not characteristic of excellent care and education. The Council of Chief State School Officers defines common standards and assessment-related terms in language relevant to the early childhood community (http://www.ccsso.org/projects/SCASS/projects/early_childhood_education_assessment_consortium/publications_and_products/2838.cfm).
OCR for page 309
Early Childhood Assessment: Why, What, and How It defines standards as “widely accepted statements of expectations for children’s learning or the quality of schools and other programs.” Of critical importance in this definition is the inclusion of program standards on equal footing with expectations for children’s learning. The report Systems for State Science Assessment (National Research Council, 2006) examines the role of standards in certain educational assessments and recommends that they be designed with a list of specific qualities in mind: standards should be clear, detailed, and complete; be reasonable in scope; be correct in their academic and scientific foundations; have a clear conceptual framework; be based on sound models of learning; and describe performance expectations and proficiency levels. State standards that have been developed for K-12 education do not meet these requirements as a whole, although some come closer than others. Recent analyses of states’ early childhood standards also suggest some misunderstanding of the difference between content and performance (Neuman and Roskos, 2005; Scott-Little, Kagan, and Frelow, 2003a). Appendix C presents a brief description of the current status of state standards for early childhood education, and includes some discussion of the efforts to align early childhood with K-12 standards. Standards should be arranged and detailed in ways that clearly identify what children need to know and be able to do and how their ideas and skills will develop over time. Learning progressions (also called “learning trajectories”) and learning performances are two useful approaches to arranging and detailing standards so as to guide curriculum, teaching practices, and assessment. Learning progressions are descriptions of successively more sophisticated ways of thinking and behaving that tend to follow one another as children mature and learn: they lay out in text and through examples what it means to move toward more mature understanding and performance. A useful example of the ideas of learning progressions and learning performances in the preschool years is California’s Desired Results Developmental Profiles-Revised (DRDP-R) and
OCR for page 310
Early Childhood Assessment: Why, What, and How its learning progression for interpersonal skills. This learning progression has been viewed as being composed of six areas, for each of which a measure (or observational guide) has been constructed: expressions of empathy, building cooperative relationships with adults, developing friendships, building cooperative play with other children, conflict negotiation, and awareness of diversity in self and others. The learning progression itself is summarized in the DRDP-R Preschool instrument (California Department of Education, 2005). Taking the interpersonal skills example further, we can examine one of the measures to see what the learning progression looks like. For example, consider the measure “building cooperative play with other children.” For the chosen measure, the progression, expressed as four successive levels, is as follows (starting from the lowest): interacts with other children side-by-side as they play with similar materials, engages with another child or children in play involving a common idea or purpose, shows preference for particular playmates but plays cooperatively with a variety of children, and leads or participates in planning cooperative play with other children. This measure in the learning progression is brought to life by examples of learning performances that could illustrate the different levels. Examples for the lowest level (a in the list above) are: plays blocks side-by-side with other children, hands another child a toy that he or she is looking for, and hands a bucket to a child sitting next to him or her in the sandbox.
OCR for page 311
Early Childhood Assessment: Why, What, and How Note that the teachers are encouraged to develop their own examples, so that these three do not become canonical. To illustrate changes to the second level in this measure, examples for the next level (b in the list) are as follows: plays with blocks with another child, plays in sand to build a castle with several other children, and joins another child to help look for a lost toy. More examples of learning performances are shown in Figure 10-2, which is a copy of the scoring guide for the measure “building cooperative play with other children.” Learning progressions should be developed around the organizing principles of child development, such as self-regulation. Such organizing principles—which are sometimes referred to as the “big ideas” of a curriculum—are the coherent foundation for the concepts, theories, principles, and explanatory schemes for child development (National Research Council, 2006). Organizing standards around these big ideas represents a fundamental shift from the more traditional organizational structure used in K-12 standards, in which standards are grouped under discrete topic headings. For example, instead of listing “knowledge of 10 letters” as a desirable outcome for a 4-year-old, one might list letter recognition and phonological awareness as examples of performances under a heading such as “emergent understanding of literacy forms.” A likely positive outcome of reorganizing standards from many discrete topics to a few big ideas is a shift from breadth to depth of coverage, from long lists of goals to a relatively small set of foundational values, principles, and concepts. If those values, principles, and concepts are the target of instruction, they can develop naturally and be extended over time. Specifying learning performances is a technique for elaborating on content standards by describing what children should be able to do if they have achieved a standard. Some examples of learning performances: children should be able to interact
OCR for page 330
Early Childhood Assessment: Why, What, and How Hold programs accountable to program standards that support the achievement of desired results and are used to measure program quality. Provide a data collection mechanism for evaluating the quality of individual child development programs. Create a base of information on the relationships between processes and results that can be used to target technical assistance to improve practice in all child development programs. At the state level, educators use the desired results system to identify successes and areas for improvement so that CDE can provide support and technical assistance to increase program quality. At the program level, practitioners use the desired results system to determine the extent to which children and families are achieving the desired results, so that quality improvement activities may be effectively targeted to directly benefit program participants. The desired results system encourages differences in the structure and objectives of individual child development programs. It is culturally sensitive and linguistically responsive to the diverse populations of children and families served. Including Children with Disabilities The desired results system is also being coordinated with a concurrent project, Desired Results: Access for Children with Disabilities Project (DR Access, http://www.draccess.org/index.html). The DR Access project is funded by the CDE Special Education Division and coordinates with the DCRF system in two ways. First, DR Access staff members worked with CDE staff members and CDE’s contractors during the development of the desired results system to make the Desired Results Developmental Profile as inclusive and appropriate as possible for assessing the progress of young children with disabilities. Second, DR Access staff members have also developed a system of adaptations and guidelines for the Desired Results Developmental Profile that allows practitioners to assess children with disabilities in an appropriate manner within the structure of the desired results system. Through these two approaches, DR Access staff members
OCR for page 331
Early Childhood Assessment: Why, What, and How ensured that the desired results system was responsive to the needs of young children with disabilities and was applicable to all settings in which they and their families were served. The vision held by the contributors to desired results and DR Access was that, through collaboration, a continuity of outcomes would be achieved for all children in CDE programs. Components of the System The desired results system has six basic components: desired results, indicators, themes, measures, criteria for success, and measurement tools. Desired results: The six desired results, to which all CDE-funded child care and development programs are expected to contribute, are that children are personally and socially competent, are effective learners, show physical and motor competence, are safe and healthy, and have families that support their learning and development, and achieve their goals. These desired results encompass the four developmental domains—cognitive, socioemotional, language, and physical development. Indicators: An indicator defines a desired result more specifically so that it can be measured. For example, an indicator of the desired result “children are personally and socially competent” is that “children show self-awareness and a positive self-concept.” Desired results are generally better measured by using multiple indicators; no single indicator gives full information on all aspects of achievement. Themes: A theme describes the aspect of development that is being measured for each indicator (e.g., self-awareness: dependence and interdependence, understanding that one’s self is a separate being with an identity of its own and with connectedness to others). Measures: A measure quantifies achievement of a particular indicator and developmental theme (e.g., a preschooler can communicate easily with familiar adults).
OCR for page 332
Early Childhood Assessment: Why, What, and How Criteria: The criteria for success define the acceptable level of achievement for each indicator (e.g., English language learners who entered the program with no comprehension of English now participate in read-alouds by repeating key words). Measurement tools: A measurement tool is the actual instrument or procedure used to capture or track information on indicators and standards of achievement (e.g., the Desired Results Developmental Profile). Professional Development The training and implementation phase of desired results for center-based programs and family child care home networks is being carried out in a series of regional training sessions for local program administrators. Assisted by the California Institute on Human Services, CDE is providing comprehensive training designed to facilitate implementation of the desired results system in programs at the local level and to build the capacity of local programs to train staff members who work directly with children. Participation in the training is by invitation only, and sites are selected one year before they are due for a Coordinated Compliance Review or Contract Monitoring Review. Nebraska Results Matter (http://ectc.nde.ne.gov/special_projects/results_matter/results_matter.htm) is designed to improve programs and child and family outcomes for all children in Nebraska from birth to age 5, whether they are served through school districts, the Early Development Network (Part C of the Individuals with Disabilities Education Act), newly implemented infant and toddler programs funded through the Early Childhood Endowment, or community partners. The system grew out of earlier efforts to monitor and evaluate state-funded preschool programs. Its broader application came as a result of recent federal requirements for reporting outcomes for children with disabilities. The system employs both program quality assessment and child outcome assessment to accomplish several purposes: improve experiences, learning, development, and lives of young children
OCR for page 333
Early Childhood Assessment: Why, What, and How (birth to age 5) and their families; inform program practices; demonstrate program effectiveness; guide the development of local and state policies and procedures; and provide data to demonstrate results. The system is administered through the Nebraska Department of Education. Major partners include the state’s Early Childhood Training Center, Health and Human Services, the Munroe-Meyer Institute at the University of Nebraska Medical Center, and multi-county educational service units. The system operates with the advice of the Results Matter Child Measurement Task Force. Child Assessment Child assessment tools were selected based on whether they employ ongoing observation of children engaged in real activities, with people they know, in natural settings; reflect evidence-based practices; engage families and primary care providers as active participants; integrate information gathered across settings; are individualized to address each child’s unique ways of learning; inform decisions about day-to-day learning opportunities for children; and reflect the belief that development and learning are rooted in culture supported by the family. The selected tools also reflect optimal congruence with Nebraska’s Early Learning Guidelines (Birth to Three and Three to Five; http://ectc.nde.ne.gov/ELG/elg.htm) and are congruent with the program standards found in Rule 11, Regulations for Early Childhood Programs (http://www.nde.state.ne.us/LEGAL/RULE11.html). These tools are the High/Scope Child Observation Record (COR), the Creative Curriculum Developmental Continuum, and the Assessment, Evaluation and Programming System (AEPS). The state has purchased licenses for the use of these tools; programs complete the assessment online. Some districts have chosen to use more than one assessment and thus more than one online system. Districts began entering data in 2006, and the first data were reported to the Office of Special Education Programs in the U.S. Department of Education in February 2008. The use of these tools supported through the online data system provides the state with unprecedented opportunities to compile needed data,
OCR for page 334
Early Childhood Assessment: Why, What, and How not only for the required state and local reporting functions, but also for ongoing program improvement and curriculum planning. Nebraska’s system is responsive to the federal mandate of the IDEA Part C (birth to age 3) and Part B, 619 (ages 3 to 5), as well as the state requirements of Nebraska Department of Education Rule 11, Regulations for Early Childhood Programs (http://www.nde.state.ne.us/LEGAL/RULE11.html), which apply to all pre-K programs operated through public schools. Program Quality Assessment The system also includes regular evaluation of programs to ensure that they achieve and maintain overall high quality, employ qualified staff, and operate in compliance with federal and state guidelines. Programs receiving state funding are required to conduct an annual evaluation using one of the environment rating scales, such as the Infant/Toddler Environment Rating Scale-Revised, ITERS-R (Harms, Clifford, and Cryer, 1998); Early Childhood Environment Rating Scale-Revised, ECERS-R (Harms, Cryer, and Clifford, 1990); or the Early Language and Literacy Classroom Observation, ELLCO (Smith and Dickinson, 2002), and complete Nebraska’s Rule 11 reporting and approval processes. Data obtained from these tools are used to develop improvement plans. In addition, programs are strongly encouraged to participate in the accreditation process of the National Association for the Education of Young Children and receive technical and financial assistance to do so. Professional Development Programs receive continuous support to ensure that their participation in Results Matter does generate the highest quality data and knowledge about how to use it to improve program quality and child and family outcomes. The state’s Early Childhood Training Center, in cooperation with the organizations that provide the program and child assessment tools, regularly offers training in their use. The state maintains a cadre of professionals who have achieved reliability in the use of the environment rating scales. In addition, each program provider is required to submit
OCR for page 335
Early Childhood Assessment: Why, What, and How a Fidelity Process Plan to address how the reliability and validity of the child observational data will be monitored and recorded. These plans describe initial training and subsequent activities to strengthen the validity of the data. New Jersey New Jersey’s Abbott Preschool Program is designed to provide high-quality preschool education to children ages 3 and 4 in 31 of the state’s poorest districts. The program has a mixed delivery system and is conducted in school districts and community-based centers, including Head Start programs. Curriculum and Instruction The New Jersey State Department of Education (NJDOE) has developed a set of early learning standards—the Preschool Expectations: Standards of Quality (2004)—which outline what children should know and be able to do at the end of their preschool program across a comprehensive set of domains. Five curriculum models have been approved: Creative Curriculum, High/Scope, Tools of the Mind, Curiosity Corner, and Bank Street. Each is aligned to the Preschool Expectations. Each district is required to select one of these approved curriculum models and to provide early childhood educators with professional development related to appropriate curriculum implementation. Assessments The NJDOE designed two performance-based assessments in the areas of literacy and mathematics that were linked directly to the Preschool Expectations: the Early Learning Assessment System-Literacy (ELAS-L; New Jersey Office of Early Childhood Education, 2004) and the Early Learning Assessment System for Mathematics (ELAS-M; New Jersey Office of Early Childhood Education, 2006). In the initial years of the preschool program, the state provided professional development for teachers in the observation and documentation of young children’s learning and in administering and scoring the ELAS assessments. While these
OCR for page 336
Early Childhood Assessment: Why, What, and How measures were originally intended to be used both for program evaluation and to inform instructional practice, state officials decided that they would be used only for instructional planning. In the ninth year of Abbott preschool implementation, the districts must select a commercially produced performance-based assessment that covers the entire range of domains in the Preschool Expectations. The ELAS instruments may still be used in the areas of literacy and mathematics. Assessment at Various Levels At the classroom level, teachers administer a performance-based assessment that covers the range of domains outlined in the Preschool Expectations. These formative assessments are intended to inform instructional practice and to give teachers direct information on the learning and development of individual children. Up one level, a sample of the community-based and school district classrooms is assessed for quality on the ECERS-R. The results of these measures are used for teacher professional development. ECERS-R scores are also reported at the district level and used to monitor classroom quality across the 31 districts. Statewide, a longitudinal study is tracking the progress of a sample of children who have participated in the Abbott preschool program on nationally normed measures of language, literacy, and mathematics. In addition, a regression discontinuity design is being used to estimate the impact of preschool on the performance of children who received it in comparison to those who did not. THINKING ABOUT ASSESSMENT AS A SYSTEM Despite the clear advantages of a systems approach to early childhood care and education, there is no doubt that the move toward systematicity will encounter many obstacles. The states and the federal government often effect change in the early childhood system by introducing new programs, local or limited innovations, and underfunded mandates. These might well constitute good models or useful efforts, but they undermine efforts to build coherence across programs and funding sources at the same time.
OCR for page 337
Early Childhood Assessment: Why, What, and How Similarly, laudable efforts to increase accountability can lead to consequences that undermine coherence. For practical reasons, accountability efforts typically involve selection of a small number of assessment instruments that carry high stakes for the program. Concentrating attention on a specific test rather than on building a system can lead to unintended consequences. When the results of that test have significant repercussions, one consequence is often that the prevailing instruction and curriculum will come to be significantly affected by the particulars of that test—specifically, by the details of material tested or the formats used. In this situation, gains observed in test results may not represent true gains in learning or progress toward meeting standards. Instead, they may primarily reflect children’s improved ability to respond to items on a particular kind of test. A typical pattern is that test scores in the first years after a new test is introduced will show significant—and publicly celebrated—increases, particularly if high stakes are involved, but these improvements tend to level off after that initial uplift (see Linn, 2003, for a general survey; see also Herman and Perry, 2002, for an example from California). Further evidence of this phenomenon comes from cases in which alternate indicators of the tested skill fail to match the gains shown by the high-stakes test. If children have indeed improved in mathematics, for example, gains should be evident on other indicators of mathematical skill; if not, the gains are suspect. The disjunction between the high-stakes and alternate tests of the same skill has been observed with older children for mathematics (e.g., see Koretz and Baron, 1998, for an example from Kentucky) and is the typical pattern seen when comparing results on state tests to those on National Assessment of Educational Progress (Linn, 2003). Some observers believe that such patterns as these illustrate the limits of what can be achieved primarily through test preparation, and that continuing improvement over the long term will require more meaningful changes in the teaching, learning process, and assessment. These findings suggest the need for a systematic approach in which it is possible to validate gains and the meaning of test scores continuously over time. Assessment by itself cannot improve children’s learning—it is the correct use of assessment information that can bring about that aim. If they are to improve learning, assessments must be based
OCR for page 338
Early Childhood Assessment: Why, What, and How on the early learning objectives and be set in contexts that relate to curriculum and teaching practices that are common in early childhood education. Assessment should appraise what children are being taught, and what is taught should embody the aims of learning described in the standards. Thus, all of the elements in the early childhood education system have to be built on a shared vision of what is important for children to know and understand, how teaching practices affect that knowledge and understanding over time, and what can be taken as evidence that learning and development have occurred (National Research Council, 2001). The following criteria, developed by the committee, operationalize these somewhat abstract principles in important characteristics that child outcome measures should have if they are to provide useful evidence for the improvement of early care and education systems. A clearly articulated purpose for the testing. Identification of why particular assessments were selected in relation to the purpose. A clear theory connecting the assessment results and quality of care. Observation of quality of instruction and specification of what would need to be focused on for improvement. A clear plan for following up to improve program quality. Strategizing to collect the required information with a minimum of testing. Appropriate preparation of testers to minimize disruptive effects on child responses. Assessment systems must operate at multiple levels—individual child, classroom, center, school district, state, and national levels. An assessment system is thus sensitive to a variety of influences—some that originate from the top and spread down, and others that work from the bottom up (National Research Council, 2001). Assessments of children must be based on an appreciation of the development and learning of typically developing children and of the typical range of variation for children of any age. This knowledge must be based on the best scientific evidence available,
OCR for page 339
Early Childhood Assessment: Why, What, and How must be sensitive to the values inherent in choosing to concentrate on specific areas rather than others, and must be completed by sound professional expertise (National Research Council, 2001). An example of an instrument designed according to these principles is the Desired Results Developmental Profile-Revised, a part of which is illustrated in Figure 10-2. Thus, a successful system of assessments must be coherent in a variety of ways (National Research Council, 2001, 2006).3 It will be horizontally coherent when the curriculum, instruction, and assessment are all aligned with the early learning standards, target the same goals for learning, and work together to support children’s developing knowledge and skill across all domains. It will be vertically coherent when there is a shared understanding at all levels of the system (classroom, center, school or program, and state) of the goals for children’s learning and development that underlie the standards, as well as consensus about the purposes and uses of assessment. And it will be developmentally coherent when it takes into account what is known about how children’s understanding develops over time and the content knowledge, abilities, and understanding that are needed for learning to progress at each stage of the process. Developmental coherence should extend across the boundaries between preschool and K-12 schooling, to ensure that the goals for young children’s learning and development are formulated by taking into account later goals and expectations and with an understanding of how early accomplishments do and do not predict later achievement. These coherences are necessary in the interrelationship of all the subsystems. For example, the development of early learning standards, curriculum, and the design of teaching practices and assessments should be guided by the same framework for understanding what is being attempted in the classroom that informs the training of beginning teachers and the continuing professional development of experienced teachers. The reporting of assessment results to parents, teachers, and other stakeholders should also be based on this same framework, as should the evaluations of effectiveness built into all systems. Each child should have an 3 This section on coherence draws heavily upon the content of the National Research Council’s 2006 report, Systems for State Science Assessment.
OCR for page 340
Early Childhood Assessment: Why, What, and How equivalent opportunity to achieve the defined goals, and the allocation of resources should reflect those goals. We emphasize that a system of assessment is only as good as the effectiveness—and coherence—of all of its components.