Educating One and All Students with Disabilities and Standards-Based Reform (1997) / Chapter Skim
Currently Skimming:
5 ACCOUNTABILITY AND ASSESSMENT
5 ACCOUNTABILITY AND ASSESSMENT
Pages 151-194
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 151... ...
In contrast, for special education, accountability is centered on the individualized education program (IEP) , an essentially private document that structures the educational goals and curriculum of an individual student 151
|
From page 152... ...
Many students with disabilities, however, are exempted from taking common assessments for a variety of reasons, including confusion about the kinds of testing accommodations that are available or allowable, local concerns about the
|
From page 153... ...
Although they recognize that student test scores alone cannot be used to judge the quality of a particular school' s program, reform advocates assume that schoolwide trends in assessment scores and the distribution of those scores across student groups, such as those with disabilities, can inform parents and the public generally about how well a school is educating its students. Ideally, an assessment program should achieve both goals.
|
From page 154... ...
and other issues related to assessment for program eligibility purposes and preparation of the IEP. OVERVIEW OF ACCOUNTABILITY SYSTEMS Accountability systems are intended to provide information to families, elected officials, and the public on the educational performance of students, teachers, schools, and school districts, to assure them that public funds are being used legitimately and productively.
|
From page 155... ...
According to a recent survey of state assessment programs, nearly every state and many school districts and schools now have some kind of assessment-based accountability framework in place (Bond et al., 1996~. In 1994-95, 45 states had active statewide assessment programs.
|
From page 156... ...
Infrequently Administered in many standards-based systems, the external assessments used for accountability are administered infrequently. For example, Maryland's School Performance Assessment Program (MSPAP)
|
From page 157... ...
For example, the Kentucky Education Reform Act required that the results of the proposed assessment that has become KIRIS be linked to the National Assessment of Educational Progress to provide a national standard of comparison, and the most recent version of KIRIS will include some use of commercial tests, the results of which are reported in terms of national norms. Performance Assessment the standards-based reform movement has been accompanied by changes in the character of assessments to reflect the changing goals of instruction, as discussed in Chapter 4.
|
From page 158... ...
Sometimes states or school districts simply decide to exclude from aggregated test scores any students who are receiving special education services (see Thurlow et al., 1995b)
|
From page 159... ...
Within the past five years, increasing numbers of states have written guidelines outlining their policies on the use of accommodations. In 1992, 21 states indicated they had written guidelines on the use of accommodations by students with disabilities in their statewide assessments; by early 1995,39 states had such written guidelines (Thurlow et al., 1995a)
|
From page 160... ...
The large-scale assessments that typify standards-based reform are in many ways unlike those typically used in special education. Although including students with disabilities in these assessments may benefit them, the assessments themselves are not designed to manage the instruction delivered to individual students with disabilities.
|
From page 161... ...
Assessment Design Assessment programs associated with standards-based reform should satisfy basic principles of measurement, regardless of whether the assessment is traditional, performance-based, or a combination. Performance assessments, which comprise the bulk of standards-based assessments, are relatively new, and empirical evidence on their quality, although growing, is limited.
|
From page 162... ...
The ability to make generalizations from a limited number of performance tasks about students' competence in performing purportedly similar tasks is problematic (Breland et al., 1987; Dunbar et al., 1991; Gao et al., 1994~. By design, many performance assessments associated with standards-based reform require students to integrate a variety of knowledge and skills to produce a product or performance.
|
From page 163... ...
Accordingly, results for small schools (e.g., elementary schools and small rural secondary schools) or for small groups, such as students with disabilities, are likely to be less reliable and more subject to fluctuations due to random characteristics of individual members of the group.
|
From page 164... ...
An additional constraint is at work for students who score particularly high or low. Although traditional test theory assumes that error is constant along the full range of test scores (Green et al., 1984)
|
From page 165... ...
The reliability of test scores for individual students is critical when scores are used to make decisions about instructional placements or receipt of a diploma, but it is often relatively unimportant when scores are aggregated to characterize the performance of large groups. Conversely, the sampling of students is irrelevant when a score is used only to draw inferences about the individual who has taken the test, but it can be a major source of unreliability when scores are aggregated to describe the performance of small groups.
|
From page 166... ...
Research tailored more directly to elementary and secondary school students participating in the kinds of assessments used in standards-based reform is urgently needed to evaluate these concerns. Link Between Use and Validity Although people often speak of "valid" or "invalid" tests, validity is not an attribute of a test per se.
|
From page 167... ...
In a system that imposed high stakes for individual test scores, the students with disabilities in School B would be unfairly disadvantaged, perhaps to the extent that the assessment would fail to meet ethical and legal scrutiny. However, if scores are used to reward and penalize staff, not students, the low scores of students with disabilities in School B would be fair, in that they would accurately reflect bad practice and would lead to negative consequences for the staff.
|
From page 168... ...
Performance Comparability Because the assessments of standards-based reforms are generally used to support the same inferences for all students that is, whether students have reached performance standards the comparability of test results for students with and without disabilities is a critical aspect of validity. Comparability of test scores has many meanings, and the movement toward standards-based reform and performance assessment has made the issue of comparability even more complex.
|
From page 169... ...
It is possible that the performance of some students with disabilities may be particularly affected by contextual differences, which could interfere with the validity of inferences. For example, some students have disabilities that cause them to work somewhat slowly on certain tasks, thereby making their perfor ~Kentucky's education reform program, widely considered pathbreaking in both general and special education, is an exception.
|
From page 170... ...
Most students with disabilities have cognitive impairments that presumably are related to at least some of the constructs tested. Relationships between disabilities and assessed constructs have important implications for the validity of inferences based on test scores.
|
From page 171... ...
First, as already noted, many performance assessments deliberately mix constructs and modes of response, making it more difficult to segregate the specific skills involved, especially those pertinent to a given disability. Second, the inconsistent classification of students with cognitive and learning disabilities does not provide clear criteria for describing the characteristics of various categories of disability, thus making guidelines for valid accommodations problematic (see Chapter 3)
|
From page 172... ...
These different tests may or may not be related conceptually to the regular assessments, but they are constructed as distinct assessments. Examples include Kentucky's alternative portfolio assessments and Maryland's Independence Mastery Assessment Program (IMAP)
|
From page 173... ...
This risk is explicitly recognized in the guidelines provided by some state education agencies for avoiding these errors, although their guidance is sometimes very general and limited. For example, Maryland's Requirements and Guidelines for Exemptions, Excuses, and Accommodations for Maryland Statewide Assessment Programs (Maryland State Department of Edu
|
From page 176... ...
3~: Addressing the issue of validity involves an examination of the purpose of the test and the specific skills to be measured. For example, if an objective of the writing test is to measure handwriting ability, that objective would be substantially altered by allowing a student to dictate his/her response.
|
From page 177... ...
Much of the case law and research pertaining to accommodations has focused on disabilities that are fairly unambiguous in terms of both diagnosis and functional implications, such as visual, hearing, and physical disabilities. In contrast, many of the students currently identified for special education have disabilities in particular, learning disabilities that do not have clear or consistently used diagnostic criteria or characteristics, as explained in Chapter 3.
|
From page 178... ...
Efforts to identify the links between disability categories and distortions in test scores are likely to be complicated by the widespread trend in special education policies away from the use of formal taxonomies of disabilities to make decisions about individual children. For example, Maryland's guidelines for accommodations expressly mandate that "accommodations must be based upon individual needs and not upon a category of disability" (Maryland State Department of Education, 1995:2~.
|
From page 179... ...
In addition, until recently, students whose disabilities are directly related to tested constructs constituted a relatively small percentage of those taking college admissions tests and postsecondary exams; students with mental retardation generally do not take them, and until recently, relatively few of the students who took them were reported to have learning disabilities. Thus, most of these studies include relatively few students from the groups for whom the validity of scores is likely to be particularly problematic or especially difficult to ascertain, yet these students constitute well over half of all elementary and secondary school students with disabilities.
|
From page 180... ...
Some mathematics items were differentially difficult in braille format for visually disabled students. Test content appeared similar for students without disabilities and those with physical and learning disabilities (despite the reports of some of the latter that verbal items were particularly difficult)
|
From page 181... ...
Nonetheless, these findings suggest that a need for additional time should not be assumed. Clearly, the effects on test scores of providing additional time warrant empirical investigation.
|
From page 182... ...
Consequently, standards-based assessments are most likely to generate results for a multiplicity of disabilities and accommodations, with few of the specific combinations frequent enough to support separate scaling of assessment results. Clearly, more research on the validity of scores from accommodated testing is needed in particular, research tailored directly to the particular assessments and inferences central to standards-based reform.
|
From page 183... ...
Given the wide range of differences in performance levels across all students, including students with disabilities, it is unlikely that the same set of items will be appropriate for everyone. Second, item response theory makes it possible to assess changes in the reliability (precision)
|
From page 184... ...
Consequently, ensuring fair comparisons becomes a major issue. The public's right to know and to have accountable schools must be balanced against individual student rights and the disparate resources and learning opportunities available to different schools and students.
|
From page 185... ...
The new federal Title I legislation requires the results of Title I standards-based assessment to be disaggregated at the state, district, and school levels by race and ethnicity, gender, English proficiency, migrant status, and economic disadvantage and by comparisons of students with and without disabilities. There are several arguments in favor of disaggregating the scores of students with and without disabilities.
|
From page 186... ...
Although detailed disaggregation may improve the meaningfulness of results for larger groups and larger aggregates, it will not provide useful aggregate comparisons for smaller disability groups or smaller aggregates. Detailed disaggregation also would run counter to the current movement within special education to avoid formal classifications and to focus instead on individual students' functional capabilities and needs.
|
From page 187... ...
One set of federal courts has already addressed the constitutional issues arising when a state links performance on a statewide test to the award of a high school diploma. A federal appellate court held unconstitutional a Florida law requiring students to pass a statewide minimum competency test in order to receive a high school diploma.
|
From page 188... ...
2d 179) , specifically assessed the impact on students with disabilities who had been in special education of using a minimum competency test to determine the award of high school diplomas.
|
From page 189... ...
Estimated costs of performance-based assessment programs range from less than $2 to over $100 per student tested. This variation reflects differences in the subjects tested, how many students are tested, how they are assessed (e.g., mix of multiple-choice, open-ended questions, performance tasks, portfolios)
|
From page 190... ...
Research is currently under way in a few states that have taken the lead with policies to increase participation, but it will be some time before those efforts can provide substantial information. Greater participation of students with disabilities in large-scale assessments could have both positive and negative effects on aggregated test scores.
|
From page 191... ...
Reporting participation rates of students with disabilities in a consistent and systematic manner is important if comparisons are to be made fairly. Increased participation rates could also contribute to a more accurate description of student performance.
|
From page 192... ...
Existing data are inadequate to determine participation rates for students with disabilities in extant assessments associated with standards-based reform or to track the assessment accommodations they have received. What few data do exist suggest considerable variability in participation rates among states and among local educational agencies within states.
|
From page 193... ...
The recent development of assessments associated with standards-based reform, combined with the existence of legal rights governing the education of students with disabilities, has required that state education
|
From page 194... ...
194 EDUCATING ONE AND ALL agencies, local education agencies, and local school personnel design and implement assessment procedures that in some cases are beyond the realm of existing, expert knowledge. The sooner the research base can match the demands of policy, the more likely that students with disabilities can participate meaningfully in standards-based assessments.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.