A Valedictory: Reflections on 60 Years in Educational Testing (1995)

program in its 1972 reincarnation, and friends of mine had been involved recently. (My not entirely positive comment was the only quote printed when the Los Angeles Times reporter broke the original story.) The next 10 weeks proved to be the most extended stress experience of my life. I finished the report on the day I came down here; so now it really is valedictory time.

In order to disentangle my remarks from California politics, I identify few comments with CLAS specifically. I shall blur references to incidents to leave you unsure whether the large-scale testing activity I mention here and there was operating in 1939, or 1968, or 1992, or some other year. Most of my points could be illustrated by several of the projects; the same things keep happening. Blurring incidents is probably wise in a talk like this anyway, because incidents are hard to document and my perceptions may be unfair. Singling out offender A, for the sake of concreteness, would miss the essential point: that these difficulties recur because of the nature of testing and the organizations engaged with it. Plus ça change, plus c'est la meme chose.

Harold Lasswell said that the central theme of politics is who gets what.¹ Testing is engaged inevitably in determining who gets what. One thinks immediately of employment tests and college admissions, but it doesn't stop there. What CLAS reports about the several schools in Monterey has direct consequences for the selling prices of houses: up in some neighborhoods, down in others. The movement toward high stakes embodied in the Goals 2000 legislation of 1994 is dedicated to the propositions that he that hath shall be given, and that testing shall determine who hath. This new meritocracy is not the product of militant hereditarians. On the contrary—we have here egalitarianism carried to its illogical extreme. If men and women are created equal, then differences in competence within their ranks must reflect that some take better advantage of opportunity than others. The remedy for inequality then is to pressure laggards to work harder.

The current developments in educational testing have three elements. I am enthusiastic about the first principle, to evaluate educational programs by seeking evidence of

transferable learning rather than by asking what is recalled from specific lessons. Teaching for transfer and testing for transfer are advertised as an application of modern cognitive psychology. But it is the principle I was weaned on in premodern times. When I did my doctoral work at Chicago, three professors in the department, Judd, Breslich, and Tyler, had just produced the book Education as Cultivation of the Higher Mental Processes. ² To the extent that the standards of the National Council of Teachers of Mathematics embody that ideal, and the extent is considerable, I am all for such movements. It is an idea whose time should have come long ago.

There is truth in the allegation that low-grade multiple-choice and fill-in-the-blank questions have led teachers to emphasize the most routine of minimum competences. Lessons that can be programmed, in the sense of the tiny-steps-for-tiny-feet methodology of the 1960s, constitute training, not education. Don't misunderstand; I have high regard for the use of computers in instruction and in measurement when the task presents the variation required for ecological validity.

The second source of energy behind the new assessment is the politician 's desire to be seen as doing something about education, and to do it without spending significant money. Testing is just the thing. It is wonderfully cheap, compared with other possible expenditures. The leaders' show of demanding results goes over big with the public. But that brings us to the third element, the call for accountability. This is a slogan so popular that it is not open to debate.

The call springs from the premise that people are no damn good; teachers don't really work at their jobs, and pupils work just enough to stay out of trouble. James March once said, “The demand for accountability is a sign of pathology in the social system.” The distrust was never more blatant than in a speech Nicholas Murray Butler made to businessmen in 1906, in speaking about the appointment of a school superintendent. “With a responsible and conspicuous officer, we can hold him, if we choose, severely responsible and commend him for the good and punish him for the bad. The advantage is that we then have somebody whom we can get at. The great difficulty is that we don't get at them often enough.”³

Goals 2000 promotes assessments and prohibits, for five years, their use as criteria for promotion or graduation. But that amounts to a positive declaration that Congress

expects that five years hence the assessments will be used in those ways—to get at youngsters who do not perform better than their parents did in school.

The Board on Testing and Assessment needs no reminder from me that opportunity to learn is a serious problem. We have seen beginnings of scholarly inquiry into the matter, and there is a legislative gesture in Goals 2000. But we know that there will be unequal opportunities when tests are used to certify students. The point I want to make here is simply that I have been listening to a lot of talk about this, and no one says how the people who certify are going to avoid penalizing the school leaver who has had substandard opportunities. I have heard it said, for instance, that you can certify a school that is providing good opportunity, and then you can on the basis of individual performance certify or not certify the members of its student body. But if those certificates are going to help youngsters get jobs, then the one from the school that is not certified is just as badly off as the one who failed the test in the certified school. The link between talk about opportunity to learn and decision making is totally missing at this point.

The accountability demands on teachers are also unjust, to the extent that they assume that teaching for transfer is mostly a matter of trying harder. I can admire the intellectual puzzles presented in the new math assessments. Many of them do require pupils to exercise mental discipline in organizing givens of a kind they never saw before and to draw inferences out of the structure they have created. But there is precious little that I, as an educational psychologist, could tell teachers about how to organize lessons so that, at the end of the year, transfer will occur to new sets of givens with another flavor.

I put in my Educational Psychology text for teachers the maxims on this, but they are mostly about what not to do. The ability to think appears to be primarily a by-product of engaging in thinking, rather than a skill that one can give lessons in. An educational program can be structured to develop higher mental processes; I certainly believe that. I have watched with great interest the programs under development by psychologists like Reuven Feuerstein and Ann Brown. The program development is a large task, and installation requires intensive retraining of teachers. It is not just a matter of working harder.

When stakes for schools become significant, gamesmanship begins to take over. As I understand the Kentucky Educational Reform Act, schools are required to make a straight-line improvement in assessment results, year after year, at whatever pitch would be required to reach the ideal in the twentieth year of the assessment. Failure to keep on this track, in any year, is to bring intensive supervision, or financial penalties,