National Academies Press: OpenBook

Learning From Experience: Evaluating Early Childhood Demonstration Programs (1982)

Chapter: Informing Policy Makers About Programs for Handicapped Children

« Previous: Measuring the Outcomes of Day Care
Suggested Citation:"Informing Policy Makers About Programs for Handicapped Children." National Research Council. 1982. Learning From Experience: Evaluating Early Childhood Demonstration Programs. Washington, DC: The National Academies Press.
×
Page 163
Suggested Citation:"Informing Policy Makers About Programs for Handicapped Children." National Research Council. 1982. Learning From Experience: Evaluating Early Childhood Demonstration Programs. Washington, DC: The National Academies Press.
×
Page 164
Suggested Citation:"Informing Policy Makers About Programs for Handicapped Children." National Research Council. 1982. Learning From Experience: Evaluating Early Childhood Demonstration Programs. Washington, DC: The National Academies Press.
×
Page 165
Suggested Citation:"Informing Policy Makers About Programs for Handicapped Children." National Research Council. 1982. Learning From Experience: Evaluating Early Childhood Demonstration Programs. Washington, DC: The National Academies Press.
×
Page 166
Suggested Citation:"Informing Policy Makers About Programs for Handicapped Children." National Research Council. 1982. Learning From Experience: Evaluating Early Childhood Demonstration Programs. Washington, DC: The National Academies Press.
×
Page 167
Suggested Citation:"Informing Policy Makers About Programs for Handicapped Children." National Research Council. 1982. Learning From Experience: Evaluating Early Childhood Demonstration Programs. Washington, DC: The National Academies Press.
×
Page 168
Suggested Citation:"Informing Policy Makers About Programs for Handicapped Children." National Research Council. 1982. Learning From Experience: Evaluating Early Childhood Demonstration Programs. Washington, DC: The National Academies Press.
×
Page 169
Suggested Citation:"Informing Policy Makers About Programs for Handicapped Children." National Research Council. 1982. Learning From Experience: Evaluating Early Childhood Demonstration Programs. Washington, DC: The National Academies Press.
×
Page 170
Suggested Citation:"Informing Policy Makers About Programs for Handicapped Children." National Research Council. 1982. Learning From Experience: Evaluating Early Childhood Demonstration Programs. Washington, DC: The National Academies Press.
×
Page 171
Suggested Citation:"Informing Policy Makers About Programs for Handicapped Children." National Research Council. 1982. Learning From Experience: Evaluating Early Childhood Demonstration Programs. Washington, DC: The National Academies Press.
×
Page 172
Suggested Citation:"Informing Policy Makers About Programs for Handicapped Children." National Research Council. 1982. Learning From Experience: Evaluating Early Childhood Demonstration Programs. Washington, DC: The National Academies Press.
×
Page 173
Suggested Citation:"Informing Policy Makers About Programs for Handicapped Children." National Research Council. 1982. Learning From Experience: Evaluating Early Childhood Demonstration Programs. Washington, DC: The National Academies Press.
×
Page 174
Suggested Citation:"Informing Policy Makers About Programs for Handicapped Children." National Research Council. 1982. Learning From Experience: Evaluating Early Childhood Demonstration Programs. Washington, DC: The National Academies Press.
×
Page 175
Suggested Citation:"Informing Policy Makers About Programs for Handicapped Children." National Research Council. 1982. Learning From Experience: Evaluating Early Childhood Demonstration Programs. Washington, DC: The National Academies Press.
×
Page 176
Suggested Citation:"Informing Policy Makers About Programs for Handicapped Children." National Research Council. 1982. Learning From Experience: Evaluating Early Childhood Demonstration Programs. Washington, DC: The National Academies Press.
×
Page 177
Suggested Citation:"Informing Policy Makers About Programs for Handicapped Children." National Research Council. 1982. Learning From Experience: Evaluating Early Childhood Demonstration Programs. Washington, DC: The National Academies Press.
×
Page 178
Suggested Citation:"Informing Policy Makers About Programs for Handicapped Children." National Research Council. 1982. Learning From Experience: Evaluating Early Childhood Demonstration Programs. Washington, DC: The National Academies Press.
×
Page 179
Suggested Citation:"Informing Policy Makers About Programs for Handicapped Children." National Research Council. 1982. Learning From Experience: Evaluating Early Childhood Demonstration Programs. Washington, DC: The National Academies Press.
×
Page 180
Suggested Citation:"Informing Policy Makers About Programs for Handicapped Children." National Research Council. 1982. Learning From Experience: Evaluating Early Childhood Demonstration Programs. Washington, DC: The National Academies Press.
×
Page 181
Suggested Citation:"Informing Policy Makers About Programs for Handicapped Children." National Research Council. 1982. Learning From Experience: Evaluating Early Childhood Demonstration Programs. Washington, DC: The National Academies Press.
×
Page 182
Suggested Citation:"Informing Policy Makers About Programs for Handicapped Children." National Research Council. 1982. Learning From Experience: Evaluating Early Childhood Demonstration Programs. Washington, DC: The National Academies Press.
×
Page 183
Suggested Citation:"Informing Policy Makers About Programs for Handicapped Children." National Research Council. 1982. Learning From Experience: Evaluating Early Childhood Demonstration Programs. Washington, DC: The National Academies Press.
×
Page 184
Suggested Citation:"Informing Policy Makers About Programs for Handicapped Children." National Research Council. 1982. Learning From Experience: Evaluating Early Childhood Demonstration Programs. Washington, DC: The National Academies Press.
×
Page 185
Suggested Citation:"Informing Policy Makers About Programs for Handicapped Children." National Research Council. 1982. Learning From Experience: Evaluating Early Childhood Demonstration Programs. Washington, DC: The National Academies Press.
×
Page 186

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Informing Policy Makers About Programs for Handicapped Children Mary M. Kennedy and Carry L. McDaniels When it comes to federal education legislation, the saying "last but not least" applies well to handicapped children. They have been last to receive the attention of Congress, receiving federal educational assistance only after services have been authorized for disadvantaged and bilingual children. The services that have been authorized, however, have by no means been trivial. Over the past 15 years Congress has sponsored a number of different programs for handicapped children, including research and demonstration, teacher training, and the production of media and materials for handicapped learners. In 1975 congressional assistance for handi- capped children culminated in a major piece of civil rights legislation: the Education for All Handicapped Children Act of 1975. This chapter describes two particular federal programs established for handicapped children. Through these examples we hope to illustrate the kinds of information federal policy makers need to improve their policies regarding handicapped children as well as the kinds of evaluation studies that would provide such information. The two programs differ in funding mechanisms, require- ments, and intent, thus offering a valuable contrast for exploring questions regarding the types of evaluations and outcome measures that are useful to federal policy makers. The Handicapped Children's Early Education Program is a discretionary grant program, offering federal funds to public or private agencies that plan to develop new strategies for serving young handicapped children. The Education for All Handicapped Children Act of 1975 is a formula grant program, providing funds to all school districts serving handicapped children and stipulating a variety of standards that must be met. The 1981 163

164 appropriations for these two programs were $20 million and $940 million, respectively. Congress has rarely questioned the effectiveness of either of these programs during its authorization or appropriations hearings. Still missing from the history of federal legislation for the handicapped is the large national impact study and the ensuing debate over the interpretation of the data. These twin events, the yin and yang of educational evaluation, have been a tradition in other areas of federal education legislation, but have not been involved in programs for handicapped children. This paper aims to discover why that is and to determine what kinds of outcome measures are of interest to federal policy makers. This paper has five sections. The first two describe the two programs. The third discusses the federal role that has apparently been adopted for the education of handicapped children. The fourth section describes outcomes of interest to federal policy makers, and the fifth section describes contributions evaluators can make. THE HANDICAPPED CHILDREN' S EARLY EDUCATION PROGRAM: DISCRETIONARY GRANTS FOR EARLY EDUCATION The Handicapped Children's Early Education Program was initiated during a period of high expectations for early intervention. The rationale for the program was put forth by the chairman of the Select Committee on Education when he opened the 1968 House hearings on the proposed program (U.S. Congress, House, 1968:2): Studies of child development have shown that early education can accelerate the development of handi- capped children; yet most parents find that, while their children may be diagnosed as handicapped at birth or shortly thereafter, they must keep those children at home from school until they are at least five or six years of age. This is a waste of the critical years of a child's life. The chairman's conviction was reiterated by the first witness, who said, "There is no sounder proposition in education than that the earlier the child is educated, the greater the return for the energy spent." The program began with $1 million in 1968 and continues today with $20 million annually. It followed on the heels

165 of such federal programs as Head Start and Follow Through. The projects supported by the program were labeled as both "experimental" and "demonstrations"; that is, the program offered a situation in which theoretical concepts could he transformed into real activities and tested under real circumstances. By supporting demonstration projects Congress showed its faith in the general concept of early childhood intervention and provided a forum in which specific ideas might be tested, elaborated, and refined. The Handicapped Children's Early Education Program aimed not only to serve young handicapped children but also to stimulate a new pattern of interactions among profes- sionals, one in which ideas could readily pass from one researcher to another and from researchers to practitioners. The implication is that the goal of Congress was not to provide a particular type of early education to young handicapped children but to stimulate interest in and explorations into the possibilities of early education for handicapped children. From the point of view of Congress, then, the effectiveness of particular program variations was an issue for the research community to address, whereas the "outcome" of interest to Congress was whether these explorations were occurring. How one measures the success of a program in expediting exploration or in stimulating the production of new ideas is not clear. The program does emphasize the dissemina- tion of new program strategies, and funds are used to support a number of mechanisms for communicating ideas. Roughly a fourth of the projects are funded specifically for outreach activities, and many of the remaining demonstration projects are associated with colleges and universities, whose faculty disseminate research findings through professional journals. In addition, two centers for technical assistance foster cross-fertilization of ideas among projects holding conferences by brokering assistance and by (U.S. Office of Education, 1979). These activities are often designed not only to stimulate the research community and agencies that provide services iThis is not to say that they do not care whether the projects would benefit young handicapped children; they seemed to assume that benefits would accrue and therefore to concentrate their efforts on stimulating program development.

166 to handicapped children but also to stimulate the general public as well. Concomitant with the activities supported by the program has been a growing public concern about the needs and rights of handicapped children. The Education for All Handicapped Children Act: Financial Assistance to Schools Since 1968 Congress's interest in education has evolved in such a way that civil rights and equitable educational treatment have superseded targeted compensa- tion as a national priority. The Education for All Handicapped Children Act of 1975 is a good example of this newly evolving federal concern. It came into being in the company of such federal initiatives as the Family Education Rights and Privacy Act and the Emergency School Aid Act. The need for the act was justified by such statements as, "More than half of the handicapped children in the United States do not receive appropriate educa- tional services which would enable them to have full equality of opportunity," and "[It is] in the national interest that the federal government . . . provide programs that meet the educational needs of handicapped children in order to assure equal protection of the law" (P.L. 94-142, Section 3). The act was designed to ensure each handicapped child a free, appropriate public educa- tion commensurate with his or her unique needs. It contains the following four requirements: 1. The program and placement for any given handi- capped child must be determined on the basis of his or her individual needs rather than on the basis of any category or label that might be assigned to the child. 2. The child's individual needs are to be determined by a group of individuals that includes the child's teacher and parents (and, where appropriate, the child). 3. If the parents and school personnel disagree as to what constitutes an appropriate educational program, the dispute should be resolved by a neutral third party. Either the parents or the school can request a court ruling or an impartial hearing. 4. To the extent possible, the child's needs should be met in a setting in which the child will be able to interact with nonhandicapped children.

167 The first three requirements are built on a different set of assumptions from those held by most members of the research and development community. Researchers tend to assume that the ultimate criterion for the appropriateness of educational programs is an empirical one; this assump- tion leads to the inference that one of the "outcomes" that should be measured for the Education for All Handi- capped Children Act is the empirical validity of the individual educational programs. Yet these three require- ments imply not only that what is appropriate for a child must be determined individually, but also that the child (or, more often, the parents) has a right to contribute to the final decision, even to take it to federal court if necessary. This means that the people deemed best qualified by Congress to judge the appropriateness of individual educational programs are not necessarily those who apply empirical criteria. The first requirement disallows reliance on knowledge of a child's labeled handicapping condition to prescribe an educational program. By contrast, most research related to the education of handicapped children has been conducted using categories of handicapping conditions as independent variables. Such data allow researchers to define trends and establish predictable patterns regarding the relative benefits of different programs to different kinds of children. Since this requirement restricts the use of such labels for prescribing educational programs, in principle it curtails the use of a significant body of educational research. While the first requirement involves the type of infor- mation that can be used to make educational decisions, the second states who is to be involved in developing programs for children. The child's educational program needs are to be determined by both parents and the school staff-- people who know the child well. The empirical information that can be used is not restricted by this requirement, but encouraging the contributions of personal and profes- sional judgments, in principle at least, effectively reduces the relative contribution of empirical information. 2 2We are emphasizing here the apparent intent of the legislation rather than its actual effects. There is evidence, for example, that school personnel may actually use more data when interacting with parents in order to justify their proposed programs.

168 The third requirement suggests that negotiators who disagree should turn to impartial third parties to resolve their disputes regarding the appropriate educational program for a child. At these impartial hearings, testi- mony from a number of witnesses, some of whom may present empirical evidence and some of whom may not, is con- sidered. Once again, the contribution of empirical information to decision making is tempered by an emphasis on the personal values of individuals who may use whatever criteria they choose. The fourth requirement appears to differ from the first three. Whereas the first three requirements establish a decision-making process and indicate no preference about the outcome, the fourth requirement actually lays down a criterion for what kind of educational programs are appro- priate. Congress has taken the position that for handi- capped children integrated education is, in general, more appropriate than segregated education. The main reason integration has not been left completely to the discretion of parents and teachers is that Congress anticipated resistance from both these groups. Teachers of non- handicapped children often feel burdened by the presence of handicapped children in their classrooms, and parents of handicapped children may worry about the effects of exposing their children to the mainstream, preferring for them the more protective environment of separate classes or schools. In addition, many handicapped adults--the deaf population, for example--enjoy the security and familiarity of a separate culture in many communities. Parents of deaf children often argue that the most appro- priate education for their sons and daughters is one in which they are exposed primarily to other deaf children. The requirement for placing children in the least restrictive environment is particularly interesting from a researcher's perspective--one cannot help but wonder whether Congress was privy to any empirical evidence to the effect that such placements are more educationally effective. In fact, some testimony to the contrary was introduced to Congress in 1963 (U.S. Congress, House, 1963:20): A recurring question is that of special classes for educable mentally retarded children. Are such classes really helpful or do they tend to keep these youngsters out of contact with other children? The results of a four-year research project concerned with the efficacy of special

169 classes for educable mentally retarded children, completed at the University of Illinois, show that with a well-designed curriculum and trained teachers certain clear differences are emerging between those groups of identified educable mentally handicapped youngsters who have had special classes and those who have not. A group of children attending special classes since school entrance appeared to be more advanced academically and socially than those who entered and remained with a group attending regular classes. Moreover, intelligence quotients as measured by standard tests showed improvement, in many instances to the degree where an individual who at 6 years of age was judged to be mentally retarded was now considered to have advanced to the slow learner level and possibly even within the slow average range. Implications based on the final results of this research emphasize the need for early identification and placement into special classes. If this is the case, then why should education in a normal classroom be part of the entitlement? The answer is closely related to the reason for establishing a right to education in the first place. Those who value access to education often defend their position by pointing to long-range results, stating, for instance, that education provides children with the "basic minimum skills necessary for the employment of the rights of speech and the full participation in the political process" (San Antonio Independent School District v. Rodriguez, 36 L.Ed. 2d 45, 1973); or, in the case of the Education for All Handicapped Children Act, by stating that education can "enable them to have full equality of opportunity" (P.L. 94-142, Section 3, emphasis added). Those who hold that education is a means to other opportunities also tend to believe that integration involves more than simply not denying opportunities, that it is an important mechanism for social change because it exposes the minority and majority groups (handicapped and nonhandicapped, in this case) to one another, presumably increasing each group's tolerance and understanding of the other. By forcing interactions between these groups, integration tends to blend them so that the "different" children lose many of their distinguishing characteristics. This effect, known as "normalization," is often used to justify integrated education for handicapped children. The effects of

170 normalization are evident in this mother's statement (Brightman and Sullivan, 1979:14): I used to hide behind a tree outside the playground and just watch. It was painful to see him with regular kids, this retarded kid of mine. Then one day, I went up to get him after school and for a few moments I couldn't find him. He looked like all the other kids. His posture, the way he walked, everything. . . . I think separate school where every kid has the same disability is the worst thing you can do for a kid. It just serves to reinforce the disability. s The children affected by this law are those who have traditionally received special education in separate classrooms, often in the school basement or in Quonset huts in the school yard. In small districts, special education children of all ages and handicapping conditions were often pooled in a single classroom; in larger districts, a variety of classrooms were usually used--one for the mentally retarded, one for the learning disabled, one for the emotionally disturbed, and so on. Once children were labeled with a particular handicapping condition, the program assignment followed automatically. These practices persist in part because teachers are certified to teach categories of children, and they persist despite evidence that neither tests nor profes- sional judgments can discriminate among relatively mild forms of retardation, learning disabilities, and emotional disturbances (Craig and McEachron, 1975). Many of the classroom activities designed for children with mild handicapping conditions look much like classes for disadvantaged youngsters (Goldstein et al., 1976). Rather than raising questions about the appropriateness of services relative to issues of child development and curriculum, these practices led to questions regarding the administration of special education and the extent to which children's rights to an appropriate education were being met. Federal Role in the Education of the Handicapped In neither the Handicapped Children's Early Education Program nor the Education for All Handicapped Act has Congress actually made any educational stipulations.

171 Rather than defining the quantity, location, or type of preschool education to be offered to young handicapped children, the Handicapped Children's Early Education Program is designed to stimulate the development and dissemination of preschool program innovations and to heighten awareness among a variety of different groups regarding the nature of handicapping conditions and the benefits of early childhood services to handicapped children. To ensure that ideas extend beyond researchers, the legislation also requires that individual projects acquaint the community to be served with the problems and potentialities of such children. And, just as the Education for All Handicapped Children Act requires parent involvement in the development of individualized educational programs, the Handicapped Children's Early Education Program requires parent involvement in the development and operation of early childhood projects. For both enactments, then, the primary influence on practice has not been to define it, but to define the decision-making processes that affect practice and to expand the number and types of participants involved in these decisions. There are several reasons why Congress chose this particular strategy. First, education has traditionally and constitu- tionally been viewed as a state and local enterprise, and as such it has been considered an activity for which local control should be paramount. Because of the pervasive influence of education on society and on the lives of individuals, however, both the quality of education and the distribution of educational services are of concern to federal policy makers, who see education as an attractive means for promoting social changes. Thus, although Congress wants to improve both the quality of education and the distribution of educational services, it does not want to take major decision-making powers away from state or local education agencies. Hence, Congress encourages educational improvements without mandating what the improvements ought to be. Second, even when Congress is interested in providing an entitlement for educational services (as with the Education for All Handicapped Children Act), a definition of that entitlement is elusive. Education is not a tangible product that can be exchanged between individ- uals, nor can one determine what quantity of it is sufficient. Moreover, the diversity of the handicapped population further complicates the problem of defining a blanket entitlement or a "best" program. The population

172 of handicapped children varies from children with mild speech impairments, to blind children who have adapted to their blindness and can function in regular classrooms, to children who are so severely retarded or physically disabled that they continue to need training in feeding themselves when they are teenagers. All of these children are expected to benefit from programs for the handicapped. The sheer diversity of the population of handicapped children precludes a narrowly focused mandate; it leads instead to the sort of mandates written into the Handicapped Children's Early Education Program and the Education for All Handicapped Children Act, in which the kind of education given to handicapped children is under the scrutiny of a variety of groups who have a stake in the matter. Third, the Congress of the United States is an elected body that makes its own decisions by group processes. New policies are formed through a complex sequence of committee meetings, hearings, negotiations, and votes. It is a group that is accustomed to participatory decision making, and one that assumes that these processes lead to reasonable conclusions. The Handicapped Children's Early Education Program and the Education for All Handicapped Children Act reflect a belief that if the right people are involved in a decision, it will, for the most part, be a reasonable decision. No empirical tests of correctness are necessary. Fourth, Congress is constrained in a very practical way by the extent to which its policies or laws can be enforced. Some laws are easily enforceable; others are not. Those that regulate material quantities or mechani- cal performances (such as automobile mileage) are relatively easy to monitor. Other laws, such as those defining public broadcasting, are enforceable in large part simply because the activities are public. Citizens who are aware of and in agreement with these policies serve as volunteer overseers of compliance. Without their aid the government would not be able to monitor compliance. Many educational policies, which fall into yet another category, relate to behaviors that are not necessarily public: the behavior of individual teachers toward children in their classrooms or the decisions school staffs make about their students. In the case of the Education for All Handicapped Children Act, Congress, fearing that some school staffs might make unfair or uninformed decisions about services for handicapped children, created the opportunity for parents to partici

173 pate in the process. This essentially converted the decision-making process from a private one to a public one, thereby enlisting the aid of sympathetic citizens to help oversee compliance. Such a requirement does more than simply provide parents with a right to participate: It also opens the process to the scrutiny of local advocacy groups. These groups, being more familiar with educational jargon and often more knowledgeable about parents' rights than are parents themselves, may be invited by the parents to accompany them to the school when decisions are being made concerning their handicapped children. Outcomes of Interest to Congress Although education per se is essentially a state and local issue, many educational problems are widely distributed across the country--so much so that they come to be recognized as national issues. If Congress perceives a particular educational issue as nationally distributed, it may respond by providing financial assistance to state and local agencies to help them address the issue. Such a response creates issues that are uniquely federal, having to do with federal funding mechanisms; federal, state, and local agency relation- ships; eligibility rules; and so forth. These federal issues involve areas that are controllable by the federal administration--management and funding, for example--and are separate from the educational issues that may have stimulated the original legislation. Many researchers are unaware of the difference between the federal goals for such programs (which involve the interrelationships among agencies and other groups) and educational problems (which are more likely to involve the cognitive or social development of handicapped children). These researchers assume that since local programs are designed to affect children's development, the outcome of most interest to Congress is the program's national educational impact; that is, an aggregation of local effects or a kind of "gross national product" of cognitive changes in children. And while it is true that Congress hopes to influence these outcomes, it is also true that its primary concerns relate to those issues that it can control. In the case of the Handicapped Children's Early Education Program and the Education for All Handicapped Children Act, the most salient federal

174 issues involve federal-state relationships; school-parent relationships; and relationships among researchers, local program developers, parents, and the federal government. The initial hearings held when H.R. 17829, a bill proposing the Handicapped Children's Early Education Program, was being considered illustrate this difference. Although the congressional members referred to the new projects as "experimental," they demonstrated little interest in evaluations of whether the Programs were beneficial to handicapped children. In fact, in 89 pages of testimony, only one series of questions touched on evaluation (U.S. Congress, House, 1968:11-12): Congressman John Erlenborn: I think it is understood in any demonstration project or experimental project that some projects will be worthwhile and some will be failures. Will you be prepared, after the first year of operations, to make this judgment and say that those programs that have not proven themselves will no longer be funded? Or will they continue because they have gotten in on the ground floor? James Gallagher, representative of the administration, responded to what appeared to be a request for assurance of effectiveness by describing the uses of both formative and summative evaluations of programs. Erlenborn was not interested in evaluation, however, in the way Gallagher thought he was, as Erlenborn's next questions demonstrate; Gallagher's replies are omitted (U.S. Congress, House, 1968:11-12): With an appropriation of $10 million for fiscal year 1970, with 50 states no doubt wanting to participate, and with 435 members of Congress and a hundred Senators as proponents of their States, how are you going to determine where you are going to establish these demonstration projects? Maybe I could get at the question another way. How many programs do you believe will be funded with the $10 million, 50, or more or less? You may have two or more programs in a single state? Do you think you might have any state that would not have a program?

175 Erlenborn's questions suggest two things. First, he was more interested in the possibility of political influences in the placement of grants than in the effectiveness of projects per se. For him, the importance of evidence of program effectiveness lay in its potential to mitigate these political influences. Second, although Erlenborn expressed an oblique interest in the effective- ness of individual projects, he demanded no assurances that the overall program would be effective. In fact, the kinds of assurances that these policy makers wanted were primarily related to the distribution of funds. Congressman Augustus Hawkins (Calif.) said (U.S. Congress, House, 1968:13), "I assume there are already some models . . . to be built upon. I was wondering whether or not the approach would be to go to those areas which have pioneered, such as California, for example, and concentrate on the experience and background of the experimentation that has already gone on?" Congressman John Dent (Pa.), describing the experiences of his home state, said, (U.S. Congress, House, 1968:10): You can measure the neglect of our handicapped children by the miles that the handicapped child lives away from a metropolitan center. I have the kind of State that has city and rural areas. . I am hoping . . . you will not worry too much about large classes but, more important, to get classes out into the rural areas so that rural school systems can get some kind of impetus to their programs. · · - Congressman William Scherle introduced yet another consideration into the funding decision (U.S. Congress, House, 1968:11): "I think top priority should be given to these institutions already established for physically disabled preschool children." These people were clearly worried about how the small appropriation for the program would be distributed among several, and in their view worthwhile, priorities. They asked whether priorities would be set according to political pressures, the experience or expertise of the requesting agencies, the distribution of children between urban and rural areas, or the type of handicapping condition served by an agency. How could they develop funding criteria that would be both valid and fair? This problem was obviously a difficult one, one that Congress must frequently

176 grapple with. It is also uniquely federal--that is, although issues of whether and how to serve young handicapped children are raised throughout the nation, questions about how to distribute federal funds are raised only in Washington. Contributions Evaluators Can Make Enactments like the Handicapped Children's Early Education Program and the Education for All Handicapped Children Act affect more than just handicapped children. They affect parents, teachers, researchers, and local and state educational administrative agencies. Both enact- ments initiate far-reaching changes in the patterns of relationships among individuals and in so doing they alter the demand for, as well as the supply of, educational services. The nature of Congress's efforts suggests that the authors of these bills may have had a somewhat complex (if not always explicit) idea of what a better society might be like, a vision that includes notions about the relationship between the individual and govern- ment, the relationship between education and government, and the relationship between education and other aspects of social life. Included in this vision are a number of assumptions about who should be involved In eaucat~ona' decision making and how the several parts of society-- parents, educational agencies and researchers--should interact to affect educational Practices. It these 1nceracclons represent the "outcomes" of interest, then traditional investigative mecuoas using two-variable models, i.e., models with one cause and one effect, will be inappropriate for measuring them. And it may not even be possible to test two-variable subsystems within the total system. A variable identified as independent may not be amenable to manipulation or control by a researcher because it is continually being influenced by other components of the social-educational system. And investigations of naturally occurring events may also be inadequate if they are limited to discrete pieces of the system. If investigations are to be of interest to Congress, then the outcomes investigated must be based on models of the social-educational system that approximate the complexity of the interactions these programs might produce--models that could take into account mutual influences, chain reactions, and other tangled networks of causes and effects. For example, the

177 Handicapped Children's Early Education Program was based on the explicit premise that early childhood programs provide an important contribution to handicapped children's development, and its design implies a number of congressional assumptions regarding the relationship between early childhood educational programs and other early childhood services; the role of federal, state, and local agencies in offering these services; and the contributions research and development can make toward improving services. Similarly, the Education for All Handicapped Children Act was based on the explicit idea that an individual has the right to participate in decisions that will affect him or her. That premise is associated by Congress with a number of implicit assumptions regarding parent-school relationships, the role of education in enabling children to enjoy equal opportunities, the social and personal effects of integration, and so on. It is these implicit assumptions that researchers must discover, test, and incorporate into their research models if they are to be useful to policy makers. Suppose, for example, one wished to evaluate the effectiveness of the due process system involved in the Education for All Handicapped Children Act. Since there are several stages in the process of dispute resolution, he or she might want to describe its effects at each step. Keeping in mind that while the dispute is being settled, the child is still being educated somewhere, the researcher must ask how different the current educational program is from the program eventually determined to be appropriate and what the consequences of temporary place- ment are for the child pending resolution of the dispute. If the researcher were to expand the study in order to measure the program characteristics well enough to describe the difference between the original program and the one eventually determined to be appropriate, there would still be no way of determining whether the original program's "degree of appropriateness" was acceptable relative to the appropriateness of the parents' right to due process. Measures of these two effects of the act could not be equated, nor could they be summed to provide a net outcome of the policy. To add yet another layer of complexity to the problem, it is possible for either party to terminate the dispute at any time on the grounds that the proceedings themselves are having an adverse effect on the child, for which the child would not be compensated if that party won the dispute. An appropriate model for

178 investigating dispute resolution, then, is one that can enlighten policy makers about how all these facets of the system interact or fail to interact. For purposes of informing policy makers, however, the model would not have the elegance and precision that two- variable models tend to have. Although the enactments may imply that their creators had a clearly focused picture of the social processes involved, the picture is no doubt impressionistic. The first contours were probably sketched in by political scientists, and over the years the testimony of many witnesses has added more brush- strokes, so that what researchers are given to study blurs in some Places, has overlays of paint in others, and multiple images in still others. Even if researchers were to refine a model of the social-educational system so that it was focused more precisely, it might not be more useful to Congress, which, as it is, can affect the system in only general ways. A reasonable model for describing the special educa- tional system, then, is one that is developed by using the language and the degree of precision appropriate to the policy-making process. Such models, however, should not be considered inferior or less challenging than those developed for other purposes. Many valued social and political models are not defined precisely. The model of the relationships among the executive, legislative, and judicial branches of government, for instance, is often described as a system of checks and balances. Most informed citizens could generate examples of how that system works and would probably know if it was not working properly. Yet few (perhaps none) would be able to define this system precisely. The fact that it involves three, rather than two, entities increases its complexity enormously. The special education system may involve even more elements. In addition to parents, children, school systems, and researchers, it includes prevailing theories regarding effective strategies for services, community attitudes toward the handicapped, state laws regarding services to handicapped children, and the immediate histories of individual school districts, such as their traditional strategies for serving handicapped children, their experiences with due process, and their experiences with the research and development community. Nor are social systems like mechanical systems. Models of social systems cannot be expected to have the clarity or lasting quality of models of simpler mechanical

179 systems. Social systems are not closed; they do not start and stop as mechanical systems do. And because they are in perpetual motion, models must account for several kinds of mutual or circular causal relationships. A new federal policy influences individuals the way a wave influences sand. Each grain moves in a slightly different direction, so that the total effect might be better presented by a general description of the beach than by summing up the movements of all the grains of sand. The construction of models of Congress's picture of the social and educational systems that affect handi- capped children will allow researchers to determine in a general way which aspects of a system influence the quality of programs that young handicapped children receive and which appear to influence parents' abilities to exercise their rights. Already, investigators are finding evidence that special education mandates are not being readily implemented in schools (Kirp et al., 1976; Stearns et al., 1980; Weatherly and Lipsky, 1978). Their evidence suggests the need for understanding the entire system in order to determine how mandates can influence it. To provide such information to policy makers, the researcher should engage in close, continuous study of the system--study that will yield not quantified measures of outcomes but narrative descriptions of the inter- actions of all parties involved in the social-educational process, descriptions that would provide policy makers with an understanding of how the system is functioning overall, how the various parts interact, and what aspects would need to be changed to make it function differently. To learn these kinds of things investigators would have to observe the naturally occurring dynamic operations of special education systems. Many investigators are reluctant to conduct case studies because they feel that the small number of cases involved in such investigations do not permit general- izations to the full population. The concerns expressed by these researchers reflect a number of scientific assumptions that are almost as vague as many of the congressional assumptions underlying these two enactments involving handicapped children. Larger samples are presumed to allow generalizations partly because the statistics describing the sample can be used to estimate the parameters of the population. Statistical inferences about the population are based on these estimates, and analytic tools have been developed that permit researchers not only to estimate a population value but also to

180 estimate their confidence in that value. The argument that case studies involve too few cases from which to generalize is based on the assumption that the method of generalizing will be a statistical one, that the general- ized statement must be precise, and that the "fact" that is to be generalized is a quantitative fact. But the kind of statements that are needed to describe the effects of such programs as the Handicapped Children's Early Education Program and the Education for All Handicapped Children Act do not require such precision; and the facts to be generalized about their effects are not quantities, but dynamic interactions among individuals and institu- tions. What is estimated for the population is how, not how often, these components can influence one another. And if researchers use case studies to develop reasonable models of how the system functions, their models will specify the sources of variation among cases that are relevant, so that sound, nonstatistical, general state- ments can be made (Kennedy, 1979). Although these investigations may provide an under . . ~ standing of how the system works, continuous, intensive Study of any particular system, aside from being expen- sive, is impractical. Automobiles are driven on the assumption that they are functioning properly. Motorists do not stop every mile or so and tear down the engine to see whether all the parts are synchronized. Instead, they wait for signs of trouble, a clank or rattle, perhaps. Where federal policies are concerned, Congress hears these clanks in testimony, in letters, and from lobbyists, so there is no apparent need for researchers to report them. But reseachers can, like mechanics, interpret these noises. They can inform Congress when the noises are merely part of the normal functioning of the system (and do not necessarily imply a dysfunction) and when they are indicative of needed repairs. Researchers can do more than wait for the clanks. They can independently measure certain aspects of the system. The quality of an automobile's functioning can be estimated by such indicators as miles traveled per gallon, the amount of oil being burned, or the quantity of its noxious emissions. Measures such as these can provide useful estimates of the extent to which the automobile is functioning properly. In addition to creating a model of the social-educational system, researchers could devise a second, complementary form of outcome measure, one that indicates the status of the system at regular intervals. Since the special education . . .

181 system is a part of a larger social-educational system, it will influence and be influenced by the larger system, so that regular spot checks of its operation are required over time. Already, studies initiated during the first two years of implementation of the Education for All Handicapped Children Act, such as Newkirk et al.'s survey (1978) of state definitions of handicapping conditions and Kotin's survey (1977) of state due process proce- dures, are outdated. Given that it is impractical for researchers to provide continuous descriptions of dynamic effects of the policy, these status indicators offer a cheap, effective alternative for monitoring global changes in the social-educational system. There are many measures already at hand or that could be developed easily to indicate the status of the special educational system. Some of the following measures might be pertinent to the Handicapped Children's Early Education Program: The distribution of projects reflecting different theoretical orientations. The number of contributions to the professional literature that derive from the projects. · The number of graduate students or teachers who receive in-service training in these projects. · The distribution of projects serving different ages of children or children with different handicapping conditions. The distribution of projects across geographic regions. · The number of projects housed in public schools versus university laboratories or experimental units in hospitals. · The number of projects that continue to operate after federal funds are removed. The measures given below might be relevant to the Education for All Handicapped Children Act: · The proportion of children served as handicapped. · The proportion of those children served who are being served with nonhandicapped children. · The proportion of minority children served as handicapped. · The proportion of educational decisions that are appealed. · The proportion of appeals that are re-appealed the following year.

182 When the Education for All Handicapped Children Act was passed in 1975, several of the findings used to justify the need for it were based on such indicators, and a recent report criticizing the administration's enforce- ment of the act was also based to a large extent on these kinds of indicators (Education Advocates Coalition, 1980). In many cases, the issues raised on these two occasions were similar. For example, in 1975 Congress found (P.L. 94-142, Section 3) that "more than half of the handicapped children in the United States do not receive appropriate educational services which would enable them to have full equality of opportunity," and in 1980 the critics charged (Education Advocates Coalition, 1980:4) that "children are frequently denied related services, such as physical therapy, occupational therapy, school health services, and transportation, essential to enable them to benefit from special education." In 1975 Congress found (P.L. 94-142, Section 3) that "one million of the handicapped children in the United States are excluded entirely from the public school system and will not go through the educational process with their peers" while in 1980 critics claimed that "children remain unnecessarily segregated in special schools and classes for the handicapped." These measures are often called "atheoretical" by the research community for two reasons. First, they are value-free, in the sense that they merely describe certain aspects of special education systems. Second, they are hard to interpret in the scientific sense of attributing their causes to particular events. But they are not hard to interpret in the social-political sense; tremendously important values are embodied in these data. While the indicators themselves may be value-free, their interpreta tions are not. The importance of these indicators does not lie in their status as measures of "outcomes" in the traditional meaning of the term. Indeed, no one knows what the right proportions for many of these measures should be. The outcomes of interest are in the system itself, and the indicators are important because they provide some clues as to how the system is working. That policy makers interpret these indicators according to presumed relation- ships between the system and the measure means that the value of the indicators depends less on their technical precision than on the extent to which their relationship to the rest of the system is understood. The debate and discussion following from the 1980 critique of the

183 administration of the Education for All Handicapped Children Act included much of what might normally be considered the job of the research community, i.e., to infer causal relationships responsible for these numbers. Rather than drawing these inferences from controlled studies, a forensic process was used to generate rival hypotheses; these in turn were tested either by reference to findings from case studies, if these are available, or by alternate analyses and displays of the available numerical indicators of different aspects of the system. In fact, the forensic process is similar to the critical process often used by researchers following the release of findings from a large-scale impact study--with two important differences. First, the debate following an impact study usually centers on such issues as the relative merits of different statistical treatments of the data or the validity of certain outcome measures, whereas the debate following the recent critique of the administration of the Education for All Handicapped Children Act centered on the relationships among the federal, state, and local education agencies. Second, the debate following an impact study is often restricted to members of the research and development community, whereas this recent debate included members of interest groups, lawmakers, and federal administrators. Many researchers will find this forensic process disturbing in part because it deprives them of an aspect of their trade in which they take great pride--the process of drawing inferences from data. The relationship between data and decision making that has been described here brings into question the appropriate role of researchers and their unique skills in social problem-solving: To what extent can or should the researcher interpret, either in the scientific sense or in the social-political sense, the data that he or she may be gathering for policy makers? In the example given here, the most appropriate contribution for that group would be to provide two kinds of data: intensive descriptions of the processes by which the social and educational system operates and quantita- tive atheoretical indicators of the special education system's functioning. Such a contribution would be similar to that provided by economists; they regularly produce a variety of economic indicators for policy makers, who in turn find these indicators useful because the economists have also developed a reasonable,

184 if imprecise, model of how the economic system operates. Models of the special education system must be developed if special education indicators are to be useful. Yet even the combination of these two kinds of data is not sufficient without a forensic process to provide the causal interpretations necessary for policy modification. The information ultimately used by policy makers results from a three-way interaction among descriptions of processes, indicators of processes, and the forensic process itself. This process need not exclude researchers from causal interpretations. Indeed, researchers could be in the center of the forensic process, conducting ad hoc analyses and searching through extant data as alternate hypotheses are raised. Such a role would not preclude them from raising and testing their own hypotheses, nor from maintaining a role of objectivity or neutrality. The particular posture that researchers take with respect to the forensic process is as much at their discretion as it is when they debate the meaning of large-scale,impact studies. The mention of large-scale impact studies brings up a second objection that researchers may have to the kind of interaction between data and debate that has been suggested here. The data involved in this debate do not define the particular kinds of program effects that most researchers are accustomed to measuring and that most researchers feel are of paramount importance: the cognitive and social development of handicapped children. Shouldn't these outcomes be a part of the debate about the effectiveness of Congress's programs for handicapped children? Our analysis of the Education for All Handicapped Children Act and the Handicapped Children's Early Education Program has not suggested that Congress lacks interest in these outcomes. Rather, we suggest that Congress has indeed recognized those outcomes as a national issue. We also suggest that Congress itself has limited its participation to one of creating a forum in which other concerned parties can exchange ideas and evidence about these outcomes. Federal policy makers probably assume that studies of cognitive and social outcomes will be carried out and that the results will be discussed and debated among service delivery agencies, parents, advocacy groups, and researchers. The quality of educational programs, while it is the concern of all, is not a federal policy issue; what is an issue is who is involved in determining these programs.

185 The situation is analogous to a division of labor. If other groups will concern themselves with methods of improving programs, Congress can then address itself to the best ways of distributing funds. Those researchers who think such a division of labor is inappropriate may try to persuade Congress to attend more to questions of treatment efficacy. But in so doing they will be entering the forensic process, not as impartial researchers but as advocates. REFERENCES Brightman, A., and Sullivan, B. (1979) Disabled children and their families, progress report 1-9. Cambridge, Mass.: American Institute for Research in the Behavioral Sciences. Craig, P. A., and McEachron, N. B. (1975) Whom do teachers identify as handicapped? Studies of Handicapped Students (Vol. 1). Menlo Park, Calif.: SRI International. Education Advocates Coalition (1980) Report on Federal Compliance Activities to Implement the Education for All Handicapped Children Act. Washington, D.C.: Children's Defense Fund. Goldstein, H., Arkall, C., Asheroff, S. C., Hurley, O. L., and Lilly, M. S. (1976) Schools. In N. Hobbs, ea., Issues in the Classification of Children. San Francisco, Calif.: Jossey-Bass, Inc., Publishers. Kennedy, M. (1979) Generalizing from single cases. Evaluation Ouarterly 3:661-678. Kirp, D. L., Kuriloff, P. J., and Buss, W. G. (1976) Legal mandates and organizational change. In N. Hobbs ea., Issues in the Classification of Children. San Francisco, Calif.: Jossey-Bass, Inc., Publishers. Kotin, L. (1977) Due Process in Special Education: Legal Analysis. Cambridge, Mass.: Research Institute for Educational Problems, Inc. Newkirk., D., Block D., and Shrybman, J. (1978) An Analysis of Categorical Definitions, Diagnostic Methods, Diagnostic Criteria and

186 Personnel Utilization in the Classification of Handicapped Children. Reston, Va.: Council for Exceptional Children. Stearns, M., Green, D., and David, J. (19801 Local Implementation of P.L. 94-142: First Year Report of a Longitudinal Study. Menlo Park, Calif.: SRI International. U.S. Congress, House (1968) Hearings Before the Select Committee on Education, Committee on Education and Labor, on H.R. 17829. Washington, D.C.: U.S. Government Printing Office. U.S. Congress, House (1963) Hearings Before the Special Subcommittee on Labor, Committee on Education and Labor, on H.R. 6013 and H.R. 6025. Statement of Lindley J. Stiles. Washington, D.C.: U.S. Government Printing Office. U.S. Congress, House (1968) H.R. 17829. 90th Congress, 2nd Session, Section 2(a). Washington, D.C.: U.S. Government Printing Office. U.S. Office of Education (1979) Handicapped Children's Early Education Program. Bureau of Education for the Handicapped. Washington, D.C.: U.S. Department of Health, Education, and Welfare. Weatherly and Lipeky (1977) Street-level bureaucrats and institutional innovation: implementing special education reform. Harvard Educational Review 47:171-197.

Next: Preschool Education for Disadvantaged Children »
Learning From Experience: Evaluating Early Childhood Demonstration Programs Get This Book
×
MyNAP members save 10% online.
Login or Register to save!
  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!