generate some 4th graders reading at the average level for 2nd graders, while some 4th graders will read at the average level for 6th graders. If what President Clinton means is literally that all 4th graders should read at least as well as the average 4th grader reads today, this probably means, given the inevitable distribution of outcomes, that the average 4th grader will have to read as well as today's average 6th or 7th grader (and even this standard will leave some 4th grade students, at the far left tail of the distribution, not meeting the minimum standard of today's average 4th grade level.)

Norm-referenced standards, therefore, are ultimately disconcerting for calculation of adequate resources. And the efforts of the education policy community to develop criterion-referenced standards, which in principle are necessary if resource levels are to be attached, are still relatively undeveloped. Great attention has been paid to the analyses of NAEP scores by the National Assessment Governing Board (NAGB). According to NAGB, for example, in reading, only 30 percent of 4th and 8th graders, and only 36 percent of 12th graders are "proficient." In math, only 21 percent of 4th graders, 24 percent of 8th graders, and only 16 percent of 12th graders are "proficient." Yet authoritative analyses of the NAGB performance standards have found them technically flawed and misleading. This makes them unsuitable for links to resource levels. The General Accounting Office found that considerably more students were probably "proficient" than the NAGB standards implied (U.S. General Accounting Office, 1993:31-32). A National Academy of Education panel concluded that the procedures by which these achievement levels had been established were "fundamentally flawed" and were "subject to large biases'' and the achievement levels by which few American students had been judged proficient were set "unreasonably high" (National Academy of Education, 1993:xxii, 148). The panel recommended to the Department of Education that NAEP Achievement Levels should not be used. In fact, the panel stated, continued use of these standards could set back the cause of education reform because it would harm the credibility of the NAEP test itself.13

The logical problems inherent in attempting to develop an outcome standard from norm-referenced rather than criterion-referenced assessments are also implicit in Richard Murnane and Frank Levy's widely cited proposal that our standard be the ability to do math and to read at a 9th-grade level, among other competencies (Murnane and Levy, 1996). By definition, however, the average 9th-grade student now does math and reads at a 9th-grade level. What Murnane and Levy must mean is that schools must be reformed so that no high school graduate or dropout reads below the level that is now average for 9th graders. They cannot mean this in a literal sense, but unspecified is precisely what the left tail of the distribution must look like to be acceptable.

In the absence of an inventory of occupational projections and related academic requirements, can outcome standards be inferred statistically? We think not. Card and Krueger have shown that investment in education is positively



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement