Cover Image

PAPERBACK
$34.75



View/Hide Left Panel

2

Workshop Summary

A wealth of information was presented at the workshop on which much of this report was based. Researchers, legal experts, federal officials, state and local policy makers and administrators, and representatives from test publishers and advocacy groups all gathered to present and discuss the issues at hand. The material the committee has collected, from the presentations and materials supplied, the workshop discussions, and supplemental reading, is organized here in the form of discussions of some basic questions one might ask about the testing of English-language learners in U.S. schools.

WHO ARE THE ENGLISH-LANGUAGE LEARNERS IN OUR SCHOOLS?

The population of students for whom language is an issue is a particularly difficult one to characterize because it is extremely diverse and changing rapidly. Nationwide, more than six million school-age children, or 14 percent, live in homes where languages other then English are used. Of these children, about 45 percent are identified as Limited English Proficient (LEP), and need special assistance in school. Of the 14 percent, about three-fourths are from Spanish-speaking families and about three-fourths are poor and attend high-poverty schools. Five states, California, Texas, New York, Florida, and Illinois, are home to the majority of these students; indeed 40 percent of LEP students can be found in just 6 percent



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 12
Testing English-Language Learners in U.S. Schools: Report and Workshop Summary 2 Workshop Summary A wealth of information was presented at the workshop on which much of this report was based. Researchers, legal experts, federal officials, state and local policy makers and administrators, and representatives from test publishers and advocacy groups all gathered to present and discuss the issues at hand. The material the committee has collected, from the presentations and materials supplied, the workshop discussions, and supplemental reading, is organized here in the form of discussions of some basic questions one might ask about the testing of English-language learners in U.S. schools. WHO ARE THE ENGLISH-LANGUAGE LEARNERS IN OUR SCHOOLS? The population of students for whom language is an issue is a particularly difficult one to characterize because it is extremely diverse and changing rapidly. Nationwide, more than six million school-age children, or 14 percent, live in homes where languages other then English are used. Of these children, about 45 percent are identified as Limited English Proficient (LEP), and need special assistance in school. Of the 14 percent, about three-fourths are from Spanish-speaking families and about three-fourths are poor and attend high-poverty schools. Five states, California, Texas, New York, Florida, and Illinois, are home to the majority of these students; indeed 40 percent of LEP students can be found in just 6 percent

OCR for page 12
Testing English-Language Learners in U.S. Schools: Report and Workshop Summary of school districts (National Center for Education Statistics (NCES),1997:37-38). Even within these high-concentration areas, many schools must adapt to rapid change. Committee member James Kadamus described the situation in New York State, which has identified 178,000 students in its system as English-language learners, 6 percent of its total student population. These students speak approximately 135 different languages and are heavily concentrated in the state's urban areas. New York's current policy is to use census and other data to calculate the five most populous language groups, and to offer bilingual programs in those languages. The tough part is that they must recalculate every year and quickly adapt their resources for the revised combination of languages. The traditional destinations for immigrants—coastal cities, for example—are not the only jurisdictions that have been adjusting to rapid demographic shifts. Wausau, Wisconsin, a city that in 1980 was found by the U.S. Census to be “the most ethnically homogeneous city in the nation,” is an example of a place that has experienced an unexpected but very rapid shift. A trickle of refugees who arrived in the late 1970s has increased to a population of about 4,200 (in a city of some 37,500), and the city's elementary school population is now 20 percent Asian (virtually all from the Hmong population). This rapid growth has strained the school system financially and required it to work quickly to develop bilingual programs (Stephenson, 1998:3, 4, and 8, Beck, 1994). Communities in many regions of the country have experienced similar changes, and, while population projections are far from foolproof, current predictions are that the percentage of the population that is of Hispanic origin will grow, immigration rates will remain constant, and the populations of elementary and secondary school-age children will fluctuate but increase overall by 2050 (U.S. Census Bureau, 1996:1-9, 29). Such projections, and the experience of communities such as Wausau, suggest that a closer look at policies for educating language minority children may move higher on the agenda in many communities. While it is clear that a great deal of flexibility is required of U.S. schools, it is important to remember that at present one of the most pressing challenges educators face in this context—in terms of numbers—is that of educating Spanish-background elementary students concentrated in high-poverty schools. Native Spanish speakers make up approximately three-fourths of the population that has been identified as limited-English proficient (National Center for Education Statistics, 1997:39). As was noted

OCR for page 12
Testing English-Language Learners in U.S. Schools: Report and Workshop Summary above, the population of native Spanish speakers in schools is likely to increase. Demographers forecast that “If immigration and birth rates remain at current levels, the total Hispanic population will grow at least three times faster than the population as a whole for several decades” (Suro, 1998:6). Moreover, Hispanic students are the most highly segregated group in America's public schools—that is, most likely to attend schools with non-diverse populations—and their segregation is increasing (Orfield and Eaton, 1996:59). The Hispanic population itself is quite diverse, representing many countries of origin and cultural traditions and a range in socio-economic status. Moreover, as a group, both Hispanic and other students who enter U.S. schools not proficient in English also bring with them a wide range of previous academic experience. Some are fully literate in their native languages and ready to plunge into content appropriate to their age and grade level. Others are not, and bring very different needs to the classroom (LaCelle-Peterson and Rivera, 1994:59). Over half of Hispanic students attend schools in which the majority of students are classified as low socio-economic status (McUsic, 1999). Nevertheless, the challenge of bringing native Spanish speakers, frequently from low-income families and with other sources of academic disadvantage, to proficiency in English, while maintaining their academic progress in all subjects, is one that schools and districts around the country face. WHAT ARE THE LEGAL REQUIREMENTS? One session of the workshop provided a review for workshop participants of the case law that has applied to the testing of English-language learners. BOTA member William Taylor began with a focus on Title I of the 1994 Elementary and Secondary Education Act (ESEA), which he described as “a bill of rights for English-language learners and other poor kids.”5 He observed that it explicitly requires states to hold LEP students 5   The ESEA is legislation that extends the authorization of appropriations for educational programs of many kinds originally made under the ESEA of 1965. Title I of the ESEA (known during the 1980s as Chapter 1) is the section of the legislation that specifically addresses the needs of disadvantaged children. The 1994 legislation explicitly requires, for the first time, that—for Title I purposes—educational jurisdictions hold all students to the same performance standards and include them in the same assessment program regardless of their need for Title I services or other supports.

OCR for page 12
Testing English-Language Learners in U.S. Schools: Report and Workshop Summary to the same high standards that apply to others, at least for Title I assessments, and that it calls for appropriate assessments and proper accommodations to be used to achieve this accountability. The legislation specifically requires ...the inclusion of limited English proficient students who shall be assessed, to the extent practicable, in the language and form most likely to yield accurate and reliable information on what such students know and can do, to determine such students' mastery of skills in subjects other than English (Office of Elementary and Secondary Education, 2000). (It is important to note that the requirements of Title I are legally binding only on assessments conducted for purposes of Title I program accountability. Other testing conducted by states and districts, including many graduation and promotion exams, are not governed by these requirements.) The basic requirement is for all students to be assessed, and for reasonable accommodations to be made for students with disabilities or limited English proficiency. Judith Johnson of the U.S. Department of Education described the specific implications for assessing LEP students, noting that states are required to identify the languages spoken by students within their systems and “make every effort to develop” assessments that can be used with these students (Office of Elementary and Secondary Education, 2000). They are required to consider either accommodations or native language testing to obtain test scores that are valid for the students for whom they are responsible, depending on the students' needs and the instruction they have received. 6 These requirements are qualified by the phrase “to the extent practicable” to allow states some leeway in addressing changing populations of non-native English speakers and other practical concerns. While some flexibility in strategies is allowed, states are required to apply their policies regarding accommodations consistently across districts and schools. States are required to include all LEP students in assessments (again this applies only to assessment programs used for Title I purposes), and must make a determination for each student of what form of testing, accommodations, or alternate language testing would yield the most valid and reliable results for that student. The content and performance standards against which LEP students are tested may not be less rigorous than those for other students, and LEP students must be tested at all of the grades included on the statewide testing system. 6   See the footnote on page 25 for discussion of different kinds of native-language testing.

OCR for page 12
Testing English-Language Learners in U.S. Schools: Report and Workshop Summary The Title I legislation builds on the development over several decades of legal standards that have affected the schooling of English-language learners, and the discussion of those developments properly begins with Title VI of the 1964 Civil Rights Act. According to Title VI, discrimination based on race, color, or national origin is prohibited. Title VI requires schools and districts to provide “equal educational opportunity” for students whose limited English proficiency “excludes them from effective participation in the educational program offered by the district.” A Supreme Court case, Lau v. Nichols, 414 U.S. 563 (1974), later followed up on this point, holding that “there is no equality of treatment merely by providing students with the same facilities, textbooks, teachers, and curriculum; for students who do not understand English are effectively foreclosed from any meaningful education.” Also in 1974, Congress enacted the Equal Educational Opportunities Act (EEOA), judicial interpretations of which made more explicit what states and school districts must do to enable LEP students to participate “meaningfully” in the educational programs they offer. These standards were used in a 1981 appeals court ruling, Casteneda v. Pickard, 648 F. 2d 989 (5th Cir. 1981), which articulates three basic requirements for LEP programs. A 1991 policy statement from the Office of Civil Rights explains how the Casteneda case applies to possible violations of Title VI of the Civil Rights Act. 7 The policy statement notes that “Title VI does not mandate any particular program for instruction for LEP students,” but goes on to say that such programs must be “recognized as sound by some experts in the field or [be] considered a legitimate experimental strategy,” “[be] reasonably calculated to implement effectively the educational theory adopted by the school; ” and “succeed[s], after a legitimate trial, in producing results indicating that students' language barriers are actually being overcome” (Assistant Secretary for Civil Rights, 1991:1). These legal requirements illustrate an important aspect of civil rights law regarding education, that “. . . educational principles are, importantly, part and parcel of the legal inquiry, and they guide legal judgments about 7   The Office of Civil Rights of the U.S. Department of Education is the federal agency charged with enforcement of civil rights law as it pertains to education. Its policy statements describe the legal standards and court precedents that are relevant to particular issues and are designed to assist policy makers and others in adhering to the law.

OCR for page 12
Testing English-Language Learners in U.S. Schools: Report and Workshop Summary high-stakes tests.” (Coleman, 1998). Arthur Coleman and Judith Johnson of the U.S. Department of Education both spoke about the federal perspective on the requirements regarding English-language learners. Both observed that the details of policy implementation and the manner in which general principles are applied to specific local circumstances are the critical determinants of how successful a policy will be and the extent to which it will be in compliance with federal regulations. Workshop participants concurred, noting that the legal standards, while crucial for holding educators responsible for adhering to important goals, provide only a framework in which to make the complex decisions policy makers face. A number of participants reminded the group that educators face decisions every day that either are not straightforward under the laws as framed, or must be made in a context that makes achieving the goals framed by the laws, and by researchers and other experts, extremely difficult. WHAT ARE THE ACADEMIC NEEDS OF ENGLISH-LANGUAGE LEARNERS? Committee member Kenji Hakuta made a presentation at the workshop on the current state of knowledge about the academic needs of English-language learners. A primary distinction to make in characterizing this student population, he pointed out, is that between oral language mastery and academic language mastery. Researchers who have studied language acquisition use a variety of methods to measure proficiency with oral and written language. Using these measures as well as case studies of children and adults learning a second language, researchers have noted that while people can often learn basic conversational skills quite quickly, close study of their speech reveals that it typically takes three to five years for them to develop true oral proficiency, to use “the elaborate, syntactically and lexically complex code of the proficient language user” (quoted in Hakuta, Butler, and Witt, 1999:3). Developing what researchers call academic English proficiency takes even longer, four to seven years, on average (Hakuta, Butler, and Witt, 1999). By academic English proficiency the experts mean the capacity to use spoken and written English with sufficient complexity that one's performance in an academic setting is not impaired. By this definition, however, academic English proficiency is not fixed but is understood in relation to the academic expectations that confront a particular learner at a particu-

OCR for page 12
Testing English-Language Learners in U.S. Schools: Report and Workshop Summary lar age.8 Defining academic English proficiency is also complicated by other factors, most notably socio-economic ones. Indeed, disentangling the effects of limited English proficiency on a student's learning and performance from the effects of other factors is not easily done. Family environment, poverty, and uneven teacher quality are just a few of the factors that can influence performance and these factors are particularly likely to affect the language-minority population because, as was noted above, their poverty rate is high relative to that of the population as a whole. Researchers and others have recognized that isolating language as a factor is both important and difficult, and that detailed understanding of students' language skills as they progress toward proficiency is very important. NRC committees and others have recommended that students be evaluated regularly so that their teachers can modify their educational programs as they progress (National Research Council, 1999a, 1999b). Tests of language proficiency exist for this purpose, but many focus on grammatical structure and technical mastery, and recent research on bilingual competence has lead to the development of other means of evaluating students' language, including oral interviews, teacher checklists, and story retelling. Many educators have also adopted portfolio assessments as a way of monitoring language growth, though most of these newer methods are more difficult to standardize and administer than traditional ones, for obvious reasons (National Research Council, 1997:117). The key here may not be the particular means by which students' progress is monitored, but the extent to which educators perceive learning spoken and written English as a variable and gradual process rather than one with defined stages and timelines that apply equally to all students. A particular point that emerged from the discussion was that educators and administrators need to recognize and plan for the time needed for students to achieve both oral and written fluency and the ability to operate in the school system without supports. The complexity of defining these students may seem very confusing for officials who must find ways to classify large numbers of students for academic placement and testing, but understanding the stages of students' 8   Indeed, as the linguistic demands of schooling increase, many English-language learners, even those who have been identified as “proficient” in elementary school, may need continued support and instruction to develop the advanced proficiency required for secondary- and college-level study.

OCR for page 12
Testing English-Language Learners in U.S. Schools: Report and Workshop Summary progress toward fluency is important. The duration of bilingual programs is often based on the length of time a student has lived in the United States, attended a U.S. school, or been offered particular programs. In California, for example, the law limits the time for special language-related services to “a temporary transition not normally intended to exceed one year” (Proposition 227, Article 2, Sec. 305). After that time is expired, students are asked, with some exceptions, to function in an English-only setting regardless of their level of English proficiency. Arbitrary timelines, it was noted, have little relationship to the learning trajectory students must travel; such cutoffs generally allow little adjustment for individual circumstances and are unlikely to serve all, or even most, students' needs. WHAT ABOUT TESTING? Decisions about what kinds of testing are suitable for particular students or groups of students can seem equally arbitrary but are equally important. Rules and policies governing the inclusion of language-minority students are far from uniform from state to state, and across different kinds of testing programs. They also change frequently as states adjust to population changes, political shifts, and emerging evidence from both research and practice. Moreover, as Charlene Rivera noted at the workshop, many state policies are those that have been developed for children with disabilities and do not reflect a clear focus on the needs of English-language learners. Some states use proficiency measures to determine which students should be included, while others use a combination of criteria, such as number of years in the state's system, test scores, school performance, and teacher judgment (National Research Council, 1997:119). Currently, of the 49 states that use statewide assessments, one has no policy on inclusion in or exemption from the test and two allow no exemptions (that is, they require all students to participate regardless of their linguistic status). The remaining 46 states include English-language learners in testing after a certain amount of time in the system. Eleven states allow a two-year delay in testing, twenty-one states allow three years, two states allow more than three, and one state has no time limit (Rivera et al., 2000). One basic tension around the question of identifying appropriate policies is between the benefits that may come from having test results for particular students and the harm that may come from having results that are inaccurate. Like almost any issue involving educational testing, this one is best understood in the context of specific circumstances. Educators

OCR for page 12
Testing English-Language Learners in U.S. Schools: Report and Workshop Summary know that many different tests exist for many different purposes, and the cost-benefit analysis may look somewhat different depending on the purpose. The principal purposes for educational tests are well known: Accountability—providing evidence of the performance of teachers, administrators, schools, districts, or states, relative to established standards or benchmarks, or in comparison to others, or both. Decisions about students—providing data that is used in making important decisions about individual students, such as placement in academic programs, grade promotion, or graduation. Program evaluation—providing evidence of the outcome of a particular educational program in terms of student performance. Tracking of long-term trends—providing evidence of changes in the performance of groups of students, such as those enrolled in a particular grade, school, or school district, or those belonging to population sub-groups, etc. Diagnosis—providing information about students' strengths and weaknesses with regard to specific material or skills (such as proficiency in English, for example), for use in improving teaching and learning. Each of these purposes can be achieved only if the test is valid for the particular purpose for which it is being used, and it is important to note that tests valid for one purpose may be invalid for others (National Research Council, 1999a). Measurement experts are in clear agreement that the validity of test results—and the accuracy of judgments based on those results—are seriously impaired if test-takers' performance is affected by factors other than their knowledge of the material being tested. Thus, for example, if a test is needed to determine how much students know about mathematics, its results will be invalid if the test-takers' limited English proficiency prevents them from understanding all of the questions, presenting their answers, or completing the work in the allotted time. In other words, a test cannot provide valid information about a student's knowledge or skills if a language barrier prevents the students from demonstrating what they know and can do. For an individual student, being tested in a language in which he or she is not proficient will mean incorrect assessment of his or her knowledge or skills unless appropriate accommodations are used. Such incorrect assessments, especially when used to support high-stakes decisions about a

OCR for page 12
Testing English-Language Learners in U.S. Schools: Report and Workshop Summary student's education, can have lasting effects on his or her academic progress. For example, students who are incorrectly placed in an academic track or retained in grade because of a misused assessment may be increasingly susceptible to school failure or dropping out as a result (National Research Council, 1999a). Language issues can also make it difficult to assess the achievement of particular groups of students. The aggregate performance of language subgroups that are inappropriately tested can be seriously misunderstood, and decisions influenced by invalid test results can have significant impact on their lives. The changing policies on including English-language learners in national tests and surveys illustrate this point. The National Assessment of Educational Progress (NAEP), for example, for many years excluded many English-language learners from testing, as did the National Education Longitudinal Study (NELS).9 (Since 1995 NAEP has followed new guidelines and included more of these students.) The difficulty with such policies was that under them the useful data collected about English-language learners were limited. Since those students selected for testing were generally those most proficient in English, only a modest proportion of all English-language learners were tested, and the data were skewed, reflecting the performance of the most proficient subset of these students. In general, English-language learners have been excluded from testing because of the difficulty of obtaining valid measures of their performance. However, in response to the call for information about how well all students are progressing toward standards, progress has been made in recent years in improving on existing assessment instruments and procedures for English-language learners. Measurement experts may have improved their understanding about the testing practices needed to achieve valid results, but a variety of factors place pressure on the educators who decide which students to include on various tests. As was made clear at the workshop, policy makers and administrators know that following best practice guidelines can be complicated in the real world for a variety of reasons. 9   NAEP is a large-scale assessment administered to randomly selected samples of 4th, 8th, and 12th graders to determine their knowledge and skills in selected subjects, including reading, mathematics, and science. NELS is a longitudinal survey through which data about U.S. students' progress through school are collected.

OCR for page 12
Testing English-Language Learners in U.S. Schools: Report and Workshop Summary Educators do not have sufficient effective tools for classifying students at various levels of English proficiency. Without these it can be very difficult to identify the point at which students are ready to participate in a particular test. The methods that are used to identify English-language learners include reviewing registration and enrollment records; conducting home language surveys, interviews, and observations; and using referrals, classroom grades and performance, and test results (National Center for Education Statistics, 1997:36). Researchers have found that test results are the most common means (LaCelle-Peterson and Rivera, 1994:65). Nevertheless, there is currently not a detailed, universally accepted definition of English-language learners. Two key points about the necessary tools emerged at the workshop. First, two very different kinds of information about English-language learners' progress are needed. Educators need to know not only which students lack proficiency in English and who may need academic supports, but also which students need to be accommodated in or even excluded from testing. They also need ways of monitoring these students' progress after they are identified as English-language learners. As noted above, it typically takes an English-language learner three to seven years to develop full academic proficiency and to be ready to be reclassified (as no longer needing language supports). Teachers and administrators need ways of making sure that these students' language skills are improving throughout the time they are classified as English-language learners so they can ensure the students are receiving the support they need. A separate but related need is for data about the progress of these students as a group. Schools and districts need to be held accountable for how well they are helping these students progress, not just because it's a good idea but also because doing so is required by law. To monitor their own progress and the effectiveness of their programs, these jurisdictions need more information than tests that identify students as English-language learners or reclassify them as mainstream students can provide. Virtually all educational tests rely to some extent on language skills, regardless of what they were designed to measure, and some rely very heavily on them. While educators may appreciate the importance of measuring academic skills on their own, irrespective of language skills, doing so can be difficult. Research by Jamal Abedi, who presented at the workshop, and others has shown that proficiency in English is strongly related to performance on a test of mathematics, and that reducing the linguistic complexity of test

OCR for page 12
Testing English-Language Learners in U.S. Schools: Report and Workshop Summary questions and instructions can yield higher performance in all groups (not only those for whom English is a second language), and particularly among lower ability students (Abedi et al., 1995). Educators who work with English-language learners need ways of distinguishing their academic progress from their developing language skills so they can identify any problems with academic skills and content early. Aggregated data on these students' academic achievement, as distinct from their language status, is also needed, both for accountability purposes and to help educators evaluate their own programs. Political pressures can be difficult to ignore. When educators or schools are held accountable for students' achievement, for example, there may be pressures to exclude English-language learners from large-scale assessments in order to boost pass rates. Excluding English-language learners from assessments aimed at system accountability, however, can mean that those students' needs will be unrecognized, and even that resources they need will not be allocated to them. Choosing not to include students who are less likely to perform well on an accountability test—and, as noted, language-minority students are frequently in that category for reasons other than language —can also be a way of shaping the results of tests that can have significant implications for schools, districts, and, increasingly, individual teachers and administrators. Decisions about appropriate testing are closely tied to questions about the instruction students have received and the opportunities they have had to learn particular material and academic skills. While a full discussion of the many kinds of academic programs that are offered to English-language learners is beyond the scope of this report, it is clear that they vary widely. To the extent that English-language learners have not been taught the same material as mainstream students, or have been held to different standards, those circumstances need to be factored into decisions about what testing makes sense for those students. In practice, researchers have found that language minority students frequently either do not have access to all of the courses other students do, are placed in less demanding academic tracks, are not taught by teachers trained to work with English-language learners, or are taught by less experienced or able teachers (LaCelle-Peterson and Rivera, 1994, National Research Council, 1999a). Committee member Jay Heubert reminded the group of professional standards in this area. The American Psychological Association's (APA) Joint Standards assert that pro-

OCR for page 12
Testing English-Language Learners in U.S. Schools: Report and Workshop Summary motion and graduation tests should cover only the “content and skills that students have had an opportunity to learn ” (AERA, APA, and NCME, 1999:146, Standard 13.5). The NRC's High Stakes reached a similar conclusion: “Tests should be used for high-stakes decisions...only after schools have implemented changes in teaching and curriculum that ensure that students have been taught the knowledge and skills on which they will be tested” (National Research Council, 1999a). The American Educational Research Association also, in a July 2000 Policy Statement Concerning High Stakes Testing, recommends that “[w]hen content standards and associated tests are introduced as a reform to . . . improve current practice, opportunities to access appropriate materials . . . should be provided before . . . students are sanctioned for failing to meet the new standards.” (AERA, 2000:2) Rebecca Zwick of the University of California at Santa Barbara made several important points at the workshop in this context. Noting that policy makers' dilemmas about testing are often intensified by the fact that different advocacy groups may be pushing in different directions, she also reminded the group that the testing purpose should be the key to decisions about inclusion. She reinforced the critical distinction between tests that yield individual scores and those designed to measure the performance of groups. Zwick also noted that performance assessments have been proposed as alternatives to traditional tests that seem to be less susceptible to group differences. She maintained that group differences are often actually larger with such tests because they impose heavy language burdens on test takers. Her conclusion was that while testing should be done thoughtfully, many of the problems associated with it are actually symptoms of wider problems. Schools in high-poverty areas, as many with large concentrations of English-language learners are, for example, frequently lack important educational resources by comparison with other schools in wealthier areas (Levin, 1996:229).10 When such schools are staffed by less experienced teachers, have deteriorating or inferior physical plants, and have other significant disadvantages, the problems associated with testing their English-language learners are compounded. Conversely, if resource alloca- 10   Levin notes in this chapter from an earlier NRC report that although there are clearly schools that are “seriously underfunded,” “Adequate resources are a necessary condition for meeting the educational needs of at-risk populations but not a sufficient condition.”

OCR for page 12
Testing English-Language Learners in U.S. Schools: Report and Workshop Summary tion and other factors were in place to produce better learning for all students, some of the differences associated with testing could be ameliorated. WHAT ABOUT ACCOMMODATIONS? Educators have devised a variety of test accommodations—means by which the disadvantages students who are not proficient in English face in testing can be at least partially compensated for. Accommodations have become more frequently used as educators have focused on the need for information about how English-language learners are faring. A successful accommodation is a way of improving the accuracy of the information collected by the test and an important way of addressing the tension between the goals of inclusion and accuracy. However, there are a number of pitfalls in their use, and when one considers the variety that characterizes the population of English-language learners, this is not surprising. As is the case with inclusion policies, states' policies on accommodation vary widely (Rivera et al., 2000). Of the 49 states with tests, 40 have a policy and 37 allow accommodations. Among those that allow accommodations, there is considerable variation. The most commonly used methods of accommodating language minority students' needs include allowing extra time, extra breaks, or other flexibility in scheduling; administering the test in small groups; simplification or translation of directions; use of dictionaries or glossaries; reading of questions aloud or allowing students to dictate answers or use a scribe; assessing in students' native language or allowing students to respond in their native language;11 allowing students to choose either English or native-language versions of test questions; 11   It is important to note that “assessing in students' native language” can mean either administering a “parallel” version of an English language test or administering a different test, in the native language, that targets the same or closely related constructs as the original English version of the test. The latter case is considered not an accommodation but a different assessment. The parallel test might be developed along with the English version or translated from it, although observers have noted the many difficulties inherent in translating tests (Kopriva, 2000).

OCR for page 12
Testing English-Language Learners in U.S. Schools: Report and Workshop Summary administration of the test by a person familiar with test-takers ' primary language and culture (National Research Council, 1997:119). Each of these accommodations was devised in response to specific notions of English-language learners' needs, but particular ones may be suitable in some settings but not others. Indeed, as Charlene Rivera noted, many of these accommodations are prohibited in some states and allowed in others, or prohibited or allowed only on certain components of assessments. The probable reason for the variety—and the source of potential problems—may lie in the challenge of targeting the accommodation to the specific needs of particular students and the knowledge or skills being assessed. Some of the important factors to be considered in matching accommodations to particular testing situations that were discussed at the workshop include the following: It is important to ensure that accommodations offered for a particular test do not affect its validity—that is, that they allow the student to be tested on the intended content—and are appropriate for the students tested. As noted above, it can be difficult to disentangle English language skills from other academic skills. Some accommodations have the potential to give the students who are offered them an advantage over native English speakers, though the goal is only to provide an equal chance for all students to demonstrate their knowledge of the subject matter being assessed. It is important to ensure that the accommodations used actually address the students' needs—and do not introduce other kinds of problems. Providing a glossary for use during testing, for example, might be ideal for students who have experience with them and have a clear sense of how to use them in a testing situation. For others, the glossary may be a confusing, time-consuming distraction that may depress their performance. The appropriateness of an accommodation needs to be carefully evaluated in the context of the testing purpose. Different accommodations may be appropriate for the same students depending on what particular information is being sought. For example, the heavy language demands in testing a student's knowledge of history may require a different approach than would the relatively lesser linguistic demands in testing mathematics skills. Different approaches might well also be called for depending on whether the test was designed for use in academic placement or in a system accountability test.

OCR for page 12
Testing English-Language Learners in U.S. Schools: Report and Workshop Summary Accommodations need to be considered in the context of instruction. For example, if students have not been instructed in Spanish, testing them in Spanish is unlikely to yield valid information about their performance. These goals reflect well established principles of measurement (as articulated in the APA Joint Standards and elsewhere, for example), but it is important to note that actually accomplishing them is easier said than done. Two earlier NRC reports have taken note of the fact that “Research that can inform policy and guidelines for making decisions about exemptions, modifications, and accommodations in assessment procedures is urgently needed” (National Research Council, 1997, 1999a). The committee is well aware of the practical difficulties that face those who need to make decisions about appropriate use of accommodations and hopes that these observations will be useful in the absence of more conclusive research findings. DIFFERENT APPROACHES States and districts around the country are currently addressing their English-language learners' needs in very different ways, and little formal research has been done to develop a clear national picture of how these students are tested, or how the policies relate to actual practice. Representatives from several jurisdictions described their policies and programs at the workshop. Philadelphia. Because Pennsylvania has very limited legislation regarding English-language learners, the city of Philadelphia has developed comprehensive English for Speakers of Other Languages (ESOL) and bilingual programs organized under a central Office of Language Equity Issues. Mary Ramirez, the director of that office, described for the workshop the city's assessment policies for English-language learners. Philadelphia uses the Stanford Achievement Test-9th Edition (SAT-9), a commercially available achievement test based on national standards developed by professional societies and other national groups, and Apprenda 2, a Spanish-language version of the SAT-9. The city's policy is to test virtually all students but they have identified three ESOL levels, and those identified through an evaluation process as beginners are excluded from testing. Certain accommodations are permitted for all English-language learners tested, depending on their proficiency

OCR for page 12
Testing English-Language Learners in U.S. Schools: Report and Workshop Summary level and the subject being tested, and these are described in public documents. The principles guiding the city's accommodation policies are described in the supporting documents as follows: The testing situation should not be the first time the student encounters an accommodated strategy. For students with disabilities, the testing accommodation should parallel the instructional accommodation as described in the IEP [Individualized Education Plan]. For English-language learners, accommodation should result in enhanced comprehension of the test directions and questions (School District of Philadelphia, 1999:10). Philadelphia has also taken seriously the responsibility of tracking their results and have test results that can be disaggregated by gender and ethnicity and also by ESOL proficiency level and by native language. California. Sonia Hernandez of the California Department of Education provided a brief history of the numerous changes that have occurred in California 's policies regarding English-language learners recently. (The information Hernandez presented has been supplemented by information available on the state's website, http://star.cde.ca.gov/.) Having dropped the California Learning Assessment System (CLAS), California has a new testing program, the Standardized Testing and Reporting Program, or STAR. This program incorporates the SAT-9, a commercially produced Spanish language test, the Spanish Assessment of Basic Education (SABE), which will provide information about English-language learners' reading, language, and mathematics skills. The approximately 2,000,000 English-language learners in the system all take the SAT-9 and are also required to take the SABE if they have been enrolled in California public schools for less than 12 months (that test is used at the discretion of the district after 12 months). The state is also developing a test of English Language Development (ELD), based on existing standards, which will assess students' English listening comprehension, speaking, reading, and writing. A high school exit examination in language arts and mathematics is currently being developed, as are mandatory tests linked to the optional state standards. Hernandez noted that the passage of Proposition 227 has greatly complicated the testing situation for English-language learners since it has meant a significant drop in native-language instruction for these students as well

OCR for page 12
Testing English-Language Learners in U.S. Schools: Report and Workshop Summary as a sharp increase in recommendations that they be placed in SpecialEducation programs. Ms. Hernandez noted that many in the state have been advocating that changes be made to improve the coherence of the state's various testing initiatives. She noted also that data collection efforts need to be improved so that the state can better track its English-language learners, and be held accountable for their performance.