really want our population to have, and highlight the conditions or procedures that are likely to [produce] that result,” she concluded.

Marshall Smith noted that the United States has a complex and loosely structured education system and such systems are difficult to change through policy. The diffuse policy authority results in reduced coherence and predictability, he suggested. It is possible for new ideas to permeate such a system, but the knowledge base in education has often suffered from weak credibility. A respected indicator system, in his view, could provide useful leverage for innovation and improvement. He identified several specific goals for the indicator system:

•   Each indicator should be made up of multiple measures or statistics, each of which is an important component. For example, academic growth for students from 6 to 18 could be one indicator and might include results from NAEP at 4th grade, PISA at age 15, and college admissions tests between ages 15 and 18.

•   The set of indicators should reflect a vision of the future, should tell a story, and should be very easy to understand. The indicators should be flexible and able to change subtly over time as conditions change (in schools, the workplace, and with technology of all sorts, for example). Statistical procedures could be used to maintain trends)

•   Whenever possible, the indicators should be aligned with international indicators so that comparisons can be made.

INDICATOR SYSTEM STRUCTURE

The way in which the indicator system is ultimately structured will reflect the goals it is designed to serve and, as Elliott observed, those goals have not yet been set. The indicators that were used to monitor progress toward the National Education Goals established in 1990, in Elliott’s view, illustrate the importance of considering the goals carefully. Among those goals was that U.S. students would be first in the world in mathematics and science achievement by 2000, he noted, but “the effort lost steam as it became clear that action was not being taken to reach the goals.”

Elliott noted that the workshop demonstrated that the presenters and participants each brought their own values to the question of what is most important to measure and also that rapidly changing contexts will need to be taken into account. There are at least four models to consider, Elliott added: the committee’s framework: the Lifelong Learning indicators (ELLI) (see Chapter 6); the broad goals defined by the European Union (Economic Security, Social Cohesion, and Sustainability (Bertelsman Stiftung, 2008); and the four strands suggested by Marshall Smith, which were:

1.  Human Outcomes—indicators of learning and doing;

2.  Byproduct outcomes of learning, both for individuals and for society—research, innovation, the arts (music, dance, painting etc.), literature, sports, tolerance for others, etc.;

3.  Formal infrastructure for learning—quality and availability of, for example, preschools, schools, certification systems, and formal networks; and

4.  informal infrastructure for learning—quality and availability of supports within



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement