International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies (1995)

Purpose The New Standards Project is a voluntary association committed to the joint development of new standards and new examination systems for U.S. schools. The goal is to improve student performance radically by using standards and assessments as instruments for comprehensive and systemic reform of American education. The 25 state and district Partners in the New Standards Project are developers and users of a system of assessments based on portfolios, projects, and performance tasks, employing the content and performance standards defined by groups such as the National Council of Teachers of Mathematics. The Partners seek mutual association in order to link their own assessments to a shared national standard. It is intended that the standards used in the development of assessments will be internationally competitive.

A process of collaborative benchmarking of education standards has been defined by which several countries can compare their educational standards and expectations. This process is designed primarily to serve the New Standards Governing Board in its standards-setting process. The Board needs information from high achieving nations about their content and performance standards in mathematics, language arts, and science for key transition points in a child's schooling career. With this information, the Board will be prepared to assure the New Standards Partners that they are linking to a system of internationally competitive content and performance standards.

Organization and Management The New Standards Project co-directors are located at the Learning Research and Development Center at the University of Pittsburgh and the National Center on Education and the Economy. A coordinator for international benchmarking is located at the Learning Research and Development Center. The project is also served by a consultant.

Design New Standards International Benchmarking is a case study. Data collection includes interviews with teachers and other education specialists, examination and translation of documents related to education, and collection of graded student work samples.

Participants

Australia, Great Britain, France, Germany, Japan, the Netherlands, Sweden, and the United States. New Standards is benchmarking U.S. mathematics standards/tasks/portfolios with mathematics education in Australia, France, Japan, the Netherlands, and Sweden; it hopes to add Germany and Great Britain. Other countries will be used in language arts and science.

Sample

Case studies.

Procedures and Summary of Content The benchmarking model begins with data collection aimed at describing the standards and procedures used by appropriate counterparts in a field of endeavor. Rather than trying to convert all standards to a common metric, benchmarking describes what others

are doing in their own terms and within their own system of values. This is different from other comparative efforts that focus on how students perform on a test of common items.

A carefully chosen set of questions about issues serves as the framework for collaborative international benchmarking. The New Standards process for collecting benchmark information engages a set of collaborators from several countries in generating information that can be used by educators and policy makers in each country. The common descriptors do not force an artificial structure on the systems investigated. Rather, the common set of questions yields a description of each country that sets various elements of its goals and standards practice in a systemic context.

The collaborative benchmarking model employs a set of common questions in order to obtain descriptive and contextualized comparisons of goals and standards. The result is a set of rich descriptions of the educational standards of each participating country. Once the standards are understood through the collaborative benchmarking research process, presentation formats can be customized to present decision makers in each country with the information they need to inform their own decision making. Decision makers include teachers and other education professionals, parents, legislators, other policy makers, and the general public. Benchmarking does not imply that goals and standards found desirable in other countries will necessarily be adopted; it holds only that the experience of other countries can be useful in setting goals and standards at home.

Data Collection and Analyses The six organizing questions for the New Standards benchmarking process are:

• What is the structure of the education systems in comparison countries? What are the key transition points and decision points for students?

• What are students in other countries expected to know and be able to do at key transition points in their schooling careers?

• What kinds of performances are used to demonstrate competence?

• What counts as “good enough” in these performances?

• What portion of the cohort is meeting the standard?

• What reform efforts are underway or on the horizon?

These questions express the concerns that have motivated New Standards Partners to come together in an effort to link their state- and district-level assessments to a shared national standard: What is and is not working in other states? How much of what kind of work should we be requiring of our young people? What must the students in our state know and be able to do in order to take their places as citizens and workers? What does it mean to be educated at the turn of this second millennium?

To determine the content standards other nations use in mathematics, language arts, science, and other subject areas, the benchmarking process examines standards clearly expressed in national or regional curricula as well as standards that must be inferred from texts, syllabi, and class work. Materials collected and analyzed for content standards include national, regional, or school curricula; legislative or ministerial directives related to content; commonly used textbooks; exams; and examples of classwork. Teachers and other education professionals in each partner country review the research analysis at each step of data collection and interpretation to provide assistance in understanding official documents and to fill the gap between those documents and actual classroom practice.

Internationally benchmarking the New Standards assessment system includes comparing it with other assessment systems, where they exist, and otherwise with testing or examining practices. Materials collected and analyzed to understand the kinds of performances students around the world engage in to demonstrate competence are examples of external national exams (including information on when

they are given and for what purpose ) and examples of internal school or classroom exams and other evaluation activities. A professional review panel discusses and reviews these materials for understanding of the assessment system.

To determine benchmarks for “best in class” performances in schools around the world, New Standards collects scored student work on exams and classwork, scoring rubrics or guides, and teacher commentary on what the scores mean. Although this information is difficult to collect, New Standards believes that collections of work samples will be the cornerstone of genuine internationally competitive standards. New Standards will encourage research partners to collect and compile student work when possible. In some countries, New Standards may have to do this work, although compilation from outside the country is less desirable.

New Standards is committed to the ideal that all children in the United States can learn and that they can be held to higher standards of achievement. Benchmarks need to include information about what this ideal might mean in other countries; what portion of students are meeting the standards they have set? This information is gathered through data on pass/fail rates for national exams; where such data are not collected, the data result from professional estimates of percentages of students meeting the standards and percentages of students proceeding through various tracks, with clear descriptions of those tracks.

To benchmark not only to current practice but to education reforms underway or being planned, the following information is needed from each of the countries chosen for benchmarking: plans currently mandated by the ministry of education, either in the development or implementation stage; “best guesses” from university and research personnel; “best guesses” from university and research personnel regarding the direction of change; and research and development work conducted by companies developing texts and other curricular materials.

As American states are working with their districts to articulate both content and performance standards and to benchmark against internationally competitive mathematics achievement, the leading states are raising expectations for their own students. In an international environment as open to observation and communication as that of developed economies, we may expect that successful American efforts to improve student performance will draw as much attention from other nations as their efforts have hitherto drawn from us. This mutual interest sustains the collaborative mode of New Standards research and promises that benchmarking, undertaken to meet the needs of New Standards Partners in the United States, will remain international in scope.

Reports on mathematics, language arts, and science will be published.

Setting standards across borders, OECD Network A summary volume. Lauren Resnick and Katherine Nolan.

Possibilities for making New Standards Project benchmark activities materials available for public use are to be determined.

Funding Funding is provided by The Pew Charitable Trusts and the John D. and Catherine T. MacArthur Foundation, and by Partner contributions.

Lauren B. Resnick, Co-director

Learning Research Development Center

University of Pittsburgh

3939 O'Hara St., Rm. 824

Pittsburgh, Pennsylvania 15260

telephone: 412/ 624-7485

facsimile: 412/ 624-3051

Marc Tucker, Co-director

National Center on Education and the Economy

39 State Street, Suite 500

Rochester, New York 14612

telephone: 716/ 546-7620

facsimile: 716/ 546-3145

Katherine Nolan, Coordinator for International Benchmarking

Learning Research and Development Center

3939 O'Hara Street

Pittsburgh, Pennsylvania 15260

telephone: 412/ 624-8319

facsimile: 412/ 624-1470

Resnick, Lauren B., and Katherlne J. Nolan

1994 Setting standards across borders. Organisation for Economic Co-operation and Development, Network A summary volume. (submitted) Organisation for Economic Co-operation and Development, Paris, France.

Resnick, Lauren B., Katherine J. Nolan, and Daniel P. Resnick

1994 Benchmarking Education Standards. Prepared for the Conference on Uses of International Education Data, convened by the National Research Council Board on International Comparative Studies in Education. February.

******

NOTE: This study summary was reviewed and edited by the office of Lauren Resnick at the Learning Research and Development Center in Pittsburgh in May 1994; Katherine Nolan provided database information on December 6, 1994.