1
Introduction

Every society nurtures a set of goals for its children, although the balance among those goals may be contested within societies and may vary across them. People want their children to be safe and healthy, to be happy and well-adjusted, to be competent in some array of domains and accomplished in one or two of those, to be trustworthy, to have good friends and to establish loving relationships, to be guided by ethical commitments, and to be prepared cognitively and morally to contribute to society in small or large ways. Each of those goals encompasses wide variation: some parents value accomplishment in athletics highly, while others value music, and yet others value academics above all. Ethical commitments for some parents imply the adherence to a particular creed, and for others mean wrestling to develop one’s own moral imperatives. Happiness for some means ongoing membership in family or clan, and for others means increasing individualization and independence. Nonetheless, at least at the general level sketched here, these societal goals for childhood are widely shared.

THE POLICY CONTEXT

Policies focused on child development connect to a subset of these goals rather well and have largely ignored others. Policies



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 15
1 Introduction E very society nurtures a set of goals for its children, although the balance among those goals may be contested within societies and may vary across them. People want their chil- dren to be safe and healthy, to be happy and well-adjusted, to be competent in some array of domains and accomplished in one or two of those, to be trustworthy, to have good friends and to estab- lish loving relationships, to be guided by ethical commitments, and to be prepared cognitively and morally to contribute to soci- ety in small or large ways. Each of those goals encompasses wide variation: some parents value accomplishment in athletics highly, while others value music, and yet others value academics above all. Ethical commitments for some parents imply the adherence to a particular creed, and for others mean wrestling to develop one’s own moral imperatives. Happiness for some means ongoing membership in family or clan, and for others means increasing individualization and independence. Nonetheless, at least at the general level sketched here, these societal goals for childhood are widely shared. THE POLICY CONTExT Policies focused on child development connect to a subset of these goals rather well and have largely ignored others. Policies 

OCR for page 15
 EARLY CHILDHOOD ASSESSMENT promote infant and child safety and physical health, but societal attention to children’s mental health is much less universal. Edu- cation policies, starting with the common school and continuing through the No Child Left Behind Act of 2001, have been designed to ensure adequate accomplishments in particular domains; read- ing and mathematics are almost always included, but science, history, literature, art, music, and athletics receive more intermit- tent and contested support. American society has largely avoided making policies related to “positive ethics”—how one should act—consistent with the separation of church and state. The criminal code can be seen as a set of ethical guidelines focusing on the negative side—what one should not do—but here as well the policies relevant to children typically exempt them from full responsibility even for wrongful actions. The largest body of child-oriented federal, state, and local policies focuses on a subset of goals for child development: It is fairly uncontroversial that society should legislate and appropri- ate funding to ensure safety and health and to promote academic achievement. Much less attention has traditionally been devoted to happiness; trustworthiness; friendship and social relationships; membership in family, society, or nation; moral development; or leading a productive life. One might conceptualize the policies as a map that provides a distorted representation of the underlying landscape, much as the Mercator projection of the earth greatly overestimates the areas of land masses at the poles. The “policy projection” of child development has often shrunk the size of social, emotional, and relational domains to focus on health and academics. This perspective directly reflects (and may indeed result from) the “researcher’s projection” and the associated “measurement pro- jection.” Somewhat more attention has been given by the field of child development to language, literacy, and cognition than to happiness, emotional health, friendship, or morality (although some of these goals are beginning to attract research attention and to be represented in states’ early childhood standards), and the tools available to measure development in that first set of domains are more numerous and more precise. Assessment strategies also traditionally have focused on rather discrete aspects of a child’s functioning, such as vocabulary

OCR for page 15
 INTRODUCTION or fine motor skills, because these lend themselves more readily to measurement. Discrete skills are valuable and valued because they allow children to carry out meaningful and important func- tions in day-to-day life, such as having conversations and forming friendships, understanding family stories and stories in books, and taking care of their own feeding and dressing needs. How children put discrete skills together to be able to carry out impor- tant day-to-day life functions is important from an outcomes per- spective, but measurement strategies have not typically focused on more global functioning. It goes far beyond the charge of this committee to analyze the history of this situation or to investigate the direction of causal- ity; perhaps the ease of measurement in some domains has led to greater interest in them, or perhaps interest in them has led to better measurement. Nonetheless, we wish to emphasize that we are acutely aware of the danger of writing a report about “devel- opmental outcomes and assessments” that takes for granted the outcomes and assessments available, without at least inquiring what the impact might be of a different or expanded set. We also wish to emphasize our view, consistent with that of most developmental theorists, that understanding children’s development of any outcome requires having information not just about a child’s performance on the assessment but also about the conditions that have led to that performance and the conditions under which the performance is assessed. Many early childhood educators prefer indirect forms of assessment, such as observa- tion of the child in a natural environment or parent or caregiver reports, to direct assessment. Nonetheless, direct assessments are widely used and offer rich information about individual chil- dren and groups of children. When they are used, however, the scores obtained should be richly contextualized. A child’s score on a vocabulary test reflects not just the child’s capacity to learn words, but also the language environment in which the child has lived since birth, the child’s ease with the testing procedure, and the child’s relationship with the tester. The younger the child, the more important are these considerations. Policy makers recognize the importance of the environment in determining child outcomes; many of the initiatives they propose and support are designed to change that environment in order

OCR for page 15
 EARLY CHILDHOOD ASSESSMENT to influence the outcomes, for example by preventing malnutri- tion in pregnant women and infants, or increasing resources for early childhood education, or promoting time for recess and active play to reduce obesity. Social policy makers are committed environmentalists when designing programs, but they too often forget their environmentalist convictions when dictating ways of assessing the outcomes of those programs. Assessment of young children is crucial in meeting a variety of purposes. It provides information with which caregivers and teachers can better understand individual children’s develop- mental progress and status and how well they are learning, and it can inform caregiving, instruction, and provision of needed services. It helps early childhood program staff determine how well they are meeting their objectives for the children they serve, and it informs program design and implementation. It provides some of the information needed for program accountability and contributes to advancing knowledge of child development. Furthermore, the tools available for assessing young children and their environments have increased vastly in number and variety in recent years. Advances in child development research and demands from educators, evaluation researchers, and policy makers have converged to provide a dizzying array of assessment options—thus enhancing the urgency of providing some guide- lines for deciding when and what to assess, choosing and using assessment tools, and interpreting assessment data. The assessment of young children’s development and learning has taken on new importance as investment in early childhood edu- cation rises. Private and government organizations are increasingly implementing programs for young children, many of them targeted toward those from disadvantaged homes and communities. These programs attempt to improve children’s chances for optimal devel- opment by compensating in various ways for perceived deficien- cies. Some of the more intensive interventions include teaching parenting skills through home visits, providing child care services that nurture development, and offering such preschool programs as Head Start and state prekindergarten (pre-K) programs. At the same time, the last decade or so has seen societal and government initiatives promoting accountability for such programs, especially those that are publicly funded. In this

OCR for page 15
 INTRODUCTION atmosphere, laws like the Government Performance and Results Act and the No Child Left Behind Act have been passed. School systems and government agencies are being asked to set goals, track progress, analyze strengths and weaknesses, and report on their achievements, with consequences when goals are not met. It is therefore not surprising that there is now considerable demand for early childhood intervention programs to prove their worth. This desire for accountability in early childhood programs may lead quite directly to the proposition that it is possible (and reasonable) to measure program quality and hold programs accountable by measuring the “outputs” or “products” of the programs—that is, assessing the children. After all, that is what is being done for school-age children to satisfy the requirements of the No Child Left Behind Act, and pressures for standards, assessment, and accountability have multiplied for young chil- dren as well. We argue in this volume, though, that thinking about accountability for early childhood programs requires an understanding of much more than just how well children score on tests. Interpreting outcome scores collected from children in an early childhood program requires the presence of a larger system, in the context of which particular assessments are selected, imple- mented, and interpreted. Using child outcome scores properly requires that a number of conditions be met: 1. a clearly articulated purpose for the testing, 2. identification of why particular assessments were selected in relation to the purpose, 3. a clear theory connecting the assessment results and quality of care, 4. observation of the quality of instruction and specification of what would be needed for improvement, 5. a clear plan for following up to improve program quality, 6. strategizing to collect the required information with a mini- mum of testing, and 7. appropriate preparation of testers to minimize disruptive effects on children’s responses. On one hand, we recognize that having all these conditions in place is challenging. Doing assessment well is difficult, and

OCR for page 15
0 EARLY CHILDHOOD ASSESSMENT designing assessment systems that serve the purpose of ensuring optimal outcomes for young children requires the investment of time, money, and considerable expertise. Failing to make those investments risks negative effects on children, on those respon- sible for care and education of young children, and ultimately on society. On the other hand, implementing assessment as a crucial, though neither simple nor inexpensive, part of a well-articulated early childhood care and education system offers the possibility of improved programs, better informed parents and care and edu- cation providers, happier and more accomplished children, and more solid evidence concerning program effectiveness. THE COMMITTEE’S CHARGE In the context described above, the U.S. Department of Health and Human Services Office of Head Start implemented the Head Start National Reporting System (NRS) in 2003. (This assessment and its origins are discussed more fully in Chapter 3.) The NRS met with a great deal of well-publicized critical reaction from early childhood researchers and advocates, some of it based on the belief that such an assessment was inappropriate, and some criticizing the NRS design, development, and implementation process. Partly in response to this criticism, Congress included a requirement for an independent study by the National Research Council (NRC) of developmental outcomes and their assessment in funding legislation for the Administration for Children and Families (ACF) in fiscal year (FY) 2006. In September 2006, the NRC, an operating arm of the National Academies, entered into a contract with the Office of Head Start of the ACF in the U.S. Department of Health and Human Services, at the request of the House Subcommittee on Education, to perform this study. The study was overseen jointly by the Board on Children, Youth, and Families (a joint activity of the NRC and the Institute of Medicine) and the NRC’s Board on Testing and Assessment. The Committee on Developmental Outcomes and Assessments for Young Children was appointed following the procedures mandated for all NRC committee appointments. Those procedures are designed to ensure that committee members are chosen for their expertise, indepen-

OCR for page 15
 INTRODUCTION dence, and diversity and that the committee’s membership is balanced and without conflicts of interest. Brief biographies of the committee members appear in Appendix E. The committee’s charge as described in the Academies pro- posal, incorporated by reference in the contract with the ACF reads: The committee will respond to a congressional mandate for a National Research Council panel to “review and provide guidance on appropriate outcomes and assessments for young children.” The committee will focus on two key topics: (1) the identification of key outcomes associated with early stages of child development for children ages 0-5, and (2) the quality and purpose of different state-of-the art techniques and instruments for developmental assessments. In the first area, the committee will review the research base associated with developmental outcomes for children ages 0-5 in different domains, including physical, cognitive, social, psycho- biological, and emotional. This review will include consideration of the range of variation associated with developmental outcomes in different child populations according to gender, SES status, race/ethnicity, and age. Special attention will be given to outcomes that are specified as the focus of early childhood programming, such as Head Start, as well as outcomes that allow states to moni- tor the developmental capacities of young children and to support programs that make positive contributions to these outcomes. In the second area, the committee will examine the available range of techniques and instruments for assessing these outcomes, paying particular attention to the empirical evidence available about the reliability, validity, fairness and other considerations related to the quality and use of the developmental assessments. The review will consider issues related to the use of assessments in screening the developmental status of special populations of children (such as children with developmental disabilities, children from minority cultures, and children whose home language is not English). The committee will also examine the criteria that should guide the selection of assessment techniques for different purposes, such as guiding curriculum and instructional decisions for individual children, or program evaluation and program accountability, and the ability to link early childhood interventions such as Head Start with wider community goals for young children. Special consider- ation will be given to the training requirements that are necessary for the use of assessments in different program settings and with different child populations. The committee will, to the extent possi-

OCR for page 15
 EARLY CHILDHOOD ASSESSMENT ble, identify opportunities to link measurement improvement strat- egies within diverse settings (such as educational, developmental, and pediatric programs for young children) to avoid duplication and to maximize collaboration and efficiencies. The committee will provide recommendations to practitioners and policy makers about criteria for the selection of appropriate assessment tools for different purposes, as well as how to collect and use contextual information to interpret assessment results appropriately for young children. The committee will also develop a research agenda to improve the quality and suitability of devel- opmental assessment tools that can be used in a variety of early childhood program and service environments. THE COMMITTEE’S APPROACH At the first meeting, the committee identified information needs in several domains and developed plans for obtaining and analyzing the needed information and for organizing the report. After reviewing the charge and the time available to complete the work, the committee discussed the scope of the tasks and determined what would and would not be attempted. We did not think it appropriate to perform in-depth technical reviews of exist- ing instruments, nor to attempt to develop a list of “approved” assessment instruments. We chose instead to develop principles and criteria for the selection of appropriate instruments for vari- ous assessment purposes. The committee gathered information from a broad range of sources on a number of issues: • Appropriate purposes for assessing young children and uses for assessment results — Defining appropriate uses and identifying user groups — Identifying potential misuses of assessment results — Using children’s assessment results to make decisions about programs • Decisions to be made in assessing young children — Choosing domains that should be assessed — Selecting direct versus observational, in-context, or “authentic” assessment — Deciding when to sample children or items (or both) versus administering all items to all children

OCR for page 15
 INTRODUCTION • Reviewing psychometric criteria — Defining reliability and validity in assessments for young children — Reviewing a sample of available assessments for their psychometric adequacy — Seeking information about validity in less frequently studied populations • Information and opinions about the NRS • Special challenges of assessing language-minority children and children with disabilities in a fair and useful manner We used several methods to gather the information needed, including literature review, briefings by the ACF and congressional staff and others, and a public forum for stakeholders. The committee and staff searched for and reviewed a large number of ACF documents and online information relevant to Head Start and Early Head Start programs and to the NRS, the assessment effort instituted by Head Start in 2003 that was a major impetus for the commissioning of this report. Committee members drew on their expertise and professional experience in child development, early childhood care and education, and assessment in reviewing and evaluating these materials. The ACF materials reviewed include • documents describing Head Start and Early Head Start programs, standards, frameworks, and research projects; • documents describing the NRS, as well as its development and implementation; and • web pages maintained by ACF organizations, including Head Start, the Office of Planning, Research and Evalua- tion, the Early Childhood Learning and Knowledge Center, the National Head Start Association, and others. The committee also reviewed reports of the U.S. Govern- ment Accountability Office, the U.S. Department of Education, and other agencies relevant to early childhood assessment. In addition to all of these materials, some of the stakeholders and other sources provided documents for our review. Some of these

OCR for page 15
 EARLY CHILDHOOD ASSESSMENT were clearly opinion pieces advocating specific points of view or courses of action and were evaluated as such. The committee reviewed scientific and professional literature in early childhood development and assessment, as well as infor- mation on early learning guidelines, standards, and frameworks developed by states and by organizations active in early child- hood education. We were especially interested in materials on developmental outcomes, assessment methods, and instruments, including existing reviews of early childhood assessment instru- ments and material on children in special populations and with special needs. Previous NRC reports including From Neurons to Neighborhoods: The Science of Early Childhood Deelopment (National Research Council and Institute of Medicine, 2000) and Eager to Learn: Educating Our Preschoolers (National Research Council, 2001), also provided much useful information. We read with spe- cial interest the report of the National Early Childhood Account- ability Task Force, released about halfway through our work, and received a briefing on that report from the task force chair.1 We invited ACF personnel and staff members of the House and Senate education subcommittees to brief the committee at our first meeting. Some ACF personnel also attended the stake- holder forum, described below. The committee also asked for and received briefings from some individuals representing organiza- tions involved with the NRS, to better understand the issues sur- rounding that assessment. Nicholas Zill of Westat, the contractor with major responsibility for its development and implementa- tion, briefed the committee at the first meeting, as did Samuel Meisels, a prominent child development researcher and critic of the NRS. In order to better understand the issues in the child develop- ment and early education community concerning assessments, the committee decided it would be useful to hear from various stakeholders involved in or affected by early childhood assess- ments. It was also important to ensure that the relevant groups had the opportunity to tell the committee about their views on the issues important to them and about their specific concerns. 1Two members of this committee, Eugene Garcia and Jacqueline Jones, were also members of the task force.

OCR for page 15
 INTRODUCTION After consultation with ACF staff and general discussion in the committee, a number of stakeholders were identified. Repre- sentatives from these organizations were invited to speak briefly at an open meeting of the committee structured as a public forum and to submit written responses to questions posed by the com- mittee. We invited a total of 55 organizations to participate in a public forum on July 6, 2007. Appendix B includes the agenda for the meeting, a list of participants, and the list of questions the stakeholder groups were asked to consider. The committee made a good-faith effort to reach a broad s ampling of stakeholders, although several interest groups whose inputs we solicited chose not to participate. We under- stand that we may not have heard all relevant points of view but worked with the information obtained from those who agreed to participate. STRUCTURE OF THE REPORT This report is organized into four parts. Part I includes this introduction, Chapter 2, on purposes of assessment, and Chap- ter 3, a brief history of early childhood standards. Part II concentrates on what should be assessed and why. Chapter 4 discusses screening assessments, particularly for infants and young toddlers; Chapter 5 focuses on the domains typically assessed in young children and approaches to assessing them; and Chapter 6 discusses methods for measuring the quality of early childhood environments. Part III focuses on assessment methods. Chapter 7 addresses psychometric issues in assessment, and Chapter 8 deals with issues in assessing ethnic/racial minority and language-minority children and children with disabilities. Chapter 9 discusses the implementation of assessments. Part IV, on assessing systematically, has two chapters. Chap- ter 10 is a discussion of the need for systems of assessment and how that need might be satisfied, and Chapter 11 provides the committee’s guidance on assessments, including a proposed research agenda. The report has five appendixes. Appendix A is a glossary of some important terms used in our discussions. Appendix B has

OCR for page 15
 EARLY CHILDHOOD ASSESSMENT information on the stakeholder forum held as part of the commit- tee’s information-gathering efforts. Appendix C has information on the domains included in state pre-K learning standards, as well as a description of recent state standards development. Appendix D provides sources for detailed information on assessment instru- ments. Appendix E contains brief biographical sketches of the committee members and staff.