National Academies Press: OpenBook
« Previous: International Adult Literacy Study
Suggested Citation:"International Assessment of Educational Progress." National Research Council. 1995. International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies. Washington, DC: The National Academies Press. doi: 10.17226/9174.
×

INTERNATIONAL ASSESSMENT OF EDUCATIONAL PROGRESS

International Organization

Educational Testing Service

Years of Data Collection

International Assessment of Educational Progress - I: 1988

International Assessment of Educational Progress - II: 1991

Purpose

Since 1983 Educational Testing Service has administered the National Assessment of Educational Progress and related projects. NAEP is an ongoing, congressionally mandated project established to conduct national surveys of the educational attainments of students in the United States. Its primary goal is to determine and report the status of and trends over time in educational achievement. NAEP was initiated in 1969 to obtain comprehensive and dependable national educational-achievement data in a uniform, scientific manner.

International Assessment of Educational Progress - I: The purpose of IAEP-I was to collect and report data on what students know and can do, on the educational and cultural factors associated with achievement, and on students' attitudes. IAEP-I assessed mathematics and science achievement of 13-year-old students in the United States and five other countries. The United States joined with the five other countries to explore the feasibility of reducing the time and money requirements for international comparative studies by capitalizing on the National Assessment of Educational Progress materials and procedures. IAEP-I permitted interested countries to experiment with NAEP technologies to determine their appropriateness for local evaluation projects.

International Evaluation of Educational Progress - II: The purpose of IAEP-II was to collect and report data on what students know and can do, on the educational and cultural factors associated with achievement, and on students' attitudes. IAEP-II assessed mathematics and science skills of samples of 9- and 13-year-old students from the United States and 19 other countries, using technology developed for the National Assessment of Educational Progress. By utilizing existing NAEP technology and procedures, time and money required to conduct these international comparative studies was reduced and many interested countries were able to experiment with the innovative psychometric techniques.

Countries that participated are large and small, rich and poor, and have varied ethnic, religious, language, and cultural traditions. Educational goals, expectations, and the meaning of achievement vary among each of these countries. Each of the countries that participated did so for its own reasons: to compare its results with those of neighbors or competitors; to learn about the educational policies and practices of countries whose students seem to regularly achieve success in mathematics and science; to establish a baseline of data within its own country against which progress could be measured in the future.

All participants shared a common interest in identifying what is possible for 9- and 13-year-olds to know and to be able to do in mathematics and science. Knowledge of what is possible produces new enthusiasm, raises sights, establishes new challenges, and ultimately can improve personal and societal performance. The assessment information can be used by each

Suggested Citation:"International Assessment of Educational Progress." National Research Council. 1995. International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies. Washington, DC: The National Academies Press. doi: 10.17226/9174.
×

country to set goals and develop curriculums and practices in harmony with its own values and culture.

An optional geography component was included in this project, as a first step toward acquiring international measures of geography skills, knowledge, and education. Through this optional geography component, participants hoped to identify behaviors and practices that contribute to high levels of achievement, so that education policy makers would have information that might enable them to improve domestic performance in concert with national educational goals.

Organization and Management

International Assessment of Educational Progress - I: A project director, located at the Educational Testing Service, worked in collaboration with the National Foundation for Educational Research (UK); Ministry of Education and Science (Spain); New Brunswick, Ontario, and Quebec Ministries of Education (Canada); Laval University data analysts; and Educational Testing Service consultants.

International Assessment of Educational Progress - II: International coordination was done by staff of the Canadian Data Analysis Group, Educational Testing Service, and Westat. In all countries except Brazil and Mozambique an independent, trained observer interviewed the country project manager about all aspects of the project and visited one or more test administration sites. In most cases, the observer was fluent in the language of the assessment.

Decisions concerning the design and implementation of the project were made collaboratively by the representatives of the provinces and countries involved in the survey. The National Academy of Sciences/National Research Council's Board on International Comparative Studies in Education reviewed plans for the project at several stages of its development and made suggestions to improve the technical quality of the study.

Design

Participants

International Assessment of Educational Progress - I: Five countries and four Canadian provinces with extensive experience in large-scale assessment participated. (Canada does not have a federal system of education.) Twelve student populations were included from: British Columbia, Ireland, Korea, New Brunswick (English), New Brunswick (French), Ontario (English), Ontario (French), Quebec (English), Quebec (French), Spain, United Kingdom (sample drawn from students in England, Scotland, and Wales), and United States. Students were 13 years old (born January 1, 1974-December 31, 1974), and were selected from public and private elementary, middle, and secondary schools.

International Assessment of Educational Progress - II: Twenty countries assessed the achievement of 13-year-old students and 14 assessed 9-year-old students in mathematics and science. In some cases, participants assessed virtually all age-eligible children in their countries; in other cases they confined samples to certain geographic regions, language groups, or grade levels. In some countries significant proportions of age-eligible children were not represented because they did not attend school. Low rates of school or student participation in some countries mean results may be biased.

Suggested Citation:"International Assessment of Educational Progress." National Research Council. 1995. International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies. Washington, DC: The National Academies Press. doi: 10.17226/9174.
×

Country participation is as follows:

Brazil

cities of São Paulo and Fortaleza, restricted grades, in-school population

Canada

four provinces at age 9 and nine out of 10 provinces at age 13

China

20 out of 29 provinces and independent cities, restricted grades, in-school population

England

all students, low participation at ages 9 and 13

France

all students

Hungary

all students

Ireland

all students

Israel

Hebrew-speaking schools

Italy

province of Emilia-Romagna, low participation at age 9

Jordan

all students

Korea

all students

Mozambique

cities of Maputo and Beira, in-school population, low participation (Mozambique did not assess Science.)

Portugal

restricted grades, in-school population at age 13

Scotland

all students, low participation at age 9

Slovenia

all students

Soviet Union

14 out of 15 republics, Russian-speaking schools

Spain

all regions except Cataluña, Spanish-speaking schools

Switzerland

15 out of 26 cantons

Taiwan

all students

United States

all students

The geography portion of the IAEP was a special probe into the geographic knowledge and skills of 13-year-olds. Twenty countries assessed the mathematics and science achievement of 13-year-olds; of these 20 countries, nine countries also administered the optional geography component:

Canada

eight out of 10 provinces

Hungary

all students

Ireland

all students

Korea

all students

Scotland

all students

Slovenia

all students

Soviet Union

14 out of 15 republics, Russian-language schools

Spain

all regions except Cataluña, Spanish-language schools

United States

all students

Sample

International Assessment of Educational Progress - I: A random sample of about 2,000 students from 100 different schools was selected from each population. In the United States, the sample size was about 1,000 students in 200 schools. A total of 24,000 students was assessed.

International Assessment of Educational Progress - II: Typically, a random sample of 3,300 students from about 110 different schools was selected from each population at each age level; half were assessed in mathematics and half in science. A total of about 175,000 9- and 13-year-olds (born in calendar years 1981 and 1977) were tested in 13 different languages in March 1991. Some countries drew samples from virtually all children in the appropriate age group; others assessed only children in specific geographic areas, language groups, or grade

Suggested Citation:"International Assessment of Educational Progress." National Research Council. 1995. International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies. Washington, DC: The National Academies Press. doi: 10.17226/9174.
×

levels. The definition of populations often followed the structure of school systems, political divisions, and cultural distinctions.

Procedures and Summary of Content

International Assessment of Educational Progress - I: All countries and provinces followed standardized administration procedures and administered the assessments during February 1988.

International Assessment of Educational Progress - II: Participants recognized fundamental differences from country to country, but assembled tests that focus on common curriculum elements. In order to form contexts for interpreting student achievement data, they added questions about students' home background and classroom experiences and the characteristics of the schools they attend. As a survey research project, this assessment could not establish cause-and-effect relationships, but it could provide clues that might help to explain high and low performance.

The IAEP-II assessment used a four-part survey: a main assessment of 13-year-olds' performance in mathematics and science; an assessment of 9-year-olds' performance in mathematics and science; an experimental, performance-based assessment of 13-year-olds' ability to use equipment and materials to solve mathematics and science problems; and a short probe of the geography skills and knowledge of 13-year-olds. All countries participated in the main assessment of 13-year-olds; participation in the other assessment components was optional.

Each participating country was responsible for carrying out all aspects of the project, including sampling, survey administration, quality control, and data entry using standardized procedures that were developed for the project. Several training manuals were developed for the project. Several international training sessions were held, during which each step of the assessment process was explained.

The assessment was developed through a consensus-building process that involved curriculum and measurement experts from each of the participating countries and provinces. Several existing NAEP frameworks and the IAEP-I framework were reviewed by participants and evaluated as to their appropriateness for their own countries' curriculums. Together, the participants then adapted the NAEP frameworks to reflect an international consensus of subject-specific topics and cognitive processes that they believed reasonably reflected curriculums being implemented in their own school systems.

The nine countries that participated in the geography component participated in the development of the geography framework that guided the design of the instruments. Curricular experts in each country reviewed the appropriateness of all potential questions for their own students.

Once the participants had agreed upon common frameworks and the relative emphases that would be placed on each topic and cognitive process category of the assessment, more than one-half submitted test items from their countries' own assessment programs that they felt were appropriate and met the requirements of the IAEP assessment. Many questions from the United States' NAEP assessments were included as well. The items with the highest ratings across all countries were placed into a pool of acceptable questions from which a subset was selected and pilot-tested in all of the participating provinces and countries.

Suggested Citation:"International Assessment of Educational Progress." National Research Council. 1995. International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies. Washington, DC: The National Academies Press. doi: 10.17226/9174.
×

While acceptable to all, the content areas measured in the tests were not equally representative of each country's curriculum and the multiple-choice question testing format was not equally familiar to students from all countries. Countries differ in the age at which students start school and in policies for promotion, students at age 13 were further along in their schooling in some countries than in others.

Data Collection and Analyses

International Assessment of Educational Progress - I: Students were administered a 45-minute mathematics assessment (63 questions) and a 45-minute science assessment (60 questions), selected from the total pool of 281 mathematics and 188 science questions used in the 1986 Unites States' National Assessment of Educational Progress. In addition, students answered questions about their school experiences and attitudes, and teachers rated students' exposure to the concepts tested by the items. The assessment was administered in February 1988, except in the United States, where the data were collected during the January-through-middle-March NAEP assessment.

Each country and province was responsible for developing a data file following a standard format, for checking ranges of responses, and resolving inconsistencies in the data. Quebec Ministry of Education staff also checked the files. ETS staff calculated weights for the United States and Canadian participants and verified weights for other participants. A Laval University (Quebec) research team conducted data analysis in consultation with ETS researchers and data analysts.

The first stage of analysis involved calculation of the percentage of correct answers and standard errors for individual questions and groups of questions. The second stage of analysis involved scaling of mathematics and science results using item response theory technology. The assessment collected opportunity-to-learn information to determine to what extent students in the participating populations had been exposed to various mathematics and science content areas covered in the IAEP questions.

Educational Testing Service measurement specialists worked with colleagues from five other countries to translate and adapt techniques used in the United States for the National Assessment of Educational Progress. Achievement results that permit comparisons and present valid and reliable findings were presented in A World of Differences: An International Assessment of Mathematics and Science. Each participating country had the opportunity to experiment with new measurement practices. Cost-effectiveness of sampling techniques, the power and limitations of Item Response Theory, and the usefulness of new reporting techniques were demonstrated. Experts in each participating country had hands-on experiences with problems and potential of new assessment techniques.

International Assessment of Educational Progress - II: The achievement tests lasted one hour. The tests given to 9-year-olds included 62 questions in mathematics and 60 questions in science. The tests given to 13-year-olds included 76 questions in mathematics and 72 questions in science. In addition, students at each age spent about 10 minutes responding to questions about their backgrounds and home and school experiences. Finally, students in countries assessing geography spent an additional 10 minutes responding to questions on geography. School administrators completed a school questionnaire.

National Center for Education Statistics commissioner Emerson Elliott emphasized factors to be kept in mind in interpreting the results of this assessment: 1) the importance of cultural

Suggested Citation:"International Assessment of Educational Progress." National Research Council. 1995. International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies. Washington, DC: The National Academies Press. doi: 10.17226/9174.
×

differences and of each country's educational traditions and practices and 2) the consensus process that produced the mathematics and science frameworks used in the development of IAEP II reflects only the curricular elements common to all participating countries.

Data analysis carried out between September and December 1991 involved checking dimensionality of the mathematics and science items at each age level through factor analysis techniques and item calibration on each of the dimensions established; computation of item and average percents correct; ability scoring using the plausible-values technology developed for the NAEP study; scale anchoring; and linkage of the two age groups.

Assessment results were obtained for 52 populations:

Population

Language

9-year-olds mathematics

9-year-olds science

13-year-olds mathematics

13-year-olds science

Brazil

Portuguese

--

--

X

X

Canada

         

Alberta

English

--

--

X

X

Br.Col.

English

X

X

X

X

Manitoba

English

--

--

X

X

Manitoba

French

--

--

X

X

NewBr.

English

X

X

X

X

New Br.

French

--

--

X

X

Newfound.

English

--

--

X

X

Nova Sc.

English

--

--

X

X

Ontario

English

X

X

X

X

Ontario

French

X

X

X

X

Quebec

English

X

X

X

X

Quebec

French

X

X

X

X

Saskatch.

English

--

--

X

X

Saskatch.

French

--

--

X

X

China

Mandarin

--

--

X

X

England

English

X

X

X

X

France

French

--

--

X

X

Hungary

Hungarian

X

X

X

X

Ireland

English

X

X

X

X

Israel

Hebrew

X

X

X

X

Italy

Italian

X

X

X

X

Jordan

Arabic

--

--

X

X

Korea

Korean

X

X

X

X

Mozambique

Portuguese

--

--

--

X

Portugal

Portuguese

X

X

X

X

Scotland

English

X

X

X

X

Slovenia

Slovene

X

X

X

X

Soviet Un.

Russian

X

X

X

X

Spain

Spanish

X

X

X

X

Switzerland

French/German/Italian

--

--

X

X

Taiwan

Mandarin

X

X

X

X

United St.

English

X

X

X

X

Suggested Citation:"International Assessment of Educational Progress." National Research Council. 1995. International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies. Washington, DC: The National Academies Press. doi: 10.17226/9174.
×

Population

13-year-olds geography option

13-year-olds performance option

Canada

   

Alberta

X

X

British Columbia

X

X

Manitoba

X

--

New Brunswick

X

--

Newfoundland

X

--

Nova Scotia

--

X

Ontario

X

X

Quebec

X

--

Saskatchewan

X

X

England

X

X

Hungary

X

--

Ireland

X

--

Korea

X

--

Scotland

X

X

Slovenia

X

X

Soviet Union

X

X

Spain

X

--

Taiwan

--

X

United States

X

--

Analysis of the second International Assessment of Educational Progress using item response theory technology was completed and almanacs of results were produced in 1992. A second volume of the IAEP Technical Report, which describes the steps in applying item response theory methodology, was also produced in 1992. This technical report describes the creation of a reference population that was used for all subsequent analyses. After ensuring that the reference population adequately represented each of the participating populations and met assumptions of unidimensionality, these data were used to calibrate item perimeters. Proficiency scores were then estimated, making use of plausible values technology. Each age group was scaled separately and then linked using the Stocking-Lord procedure. The resulting scale was assigned a mean of 500 and a standard deviation of 100. Average proficiency scores were then calculated for participating populations, following the same procedures used for the percent correct analyses.

Secondary analysis of IAEP results. The Educational Testing Service Center for the Assessment of Educational Progress prepared two short reports that focus on implications of the IAEP results for U.S. efforts at standard setting and curriculum reform. These reports, written in collaboration with U.S. mathematics and science educators, make use of item-by-item results and comment on what students in the United States know and can do well and where U.S. students fare behind student peers in other countries. They will be published by the National Science Foundation.

Timetable

International Assessment of Educational Progress - I:

Suggested Citation:"International Assessment of Educational Progress." National Research Council. 1995. International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies. Washington, DC: The National Academies Press. doi: 10.17226/9174.
×

1987-1989

Project implementation was carried out through a series of meetings to select assessment items, review pilot-test results, and review and interpret final results. Decisions were made collaboratively; follow-up coordination was provided by ETS staff.

International Assessment of Educational Progress - II:

1989

Pilot testing was conducted November 1989-February 1990.

1990

The pilot review meeting was held in Montreal. (May)

Data was collected in the southern hemisphere. (September)

1991

Data was collected in the northern hemisphere. (March)

1992

Reports were published on results of IAEP geography, mathematics and science.

An almanac of results of IAEP using item response theory published.

Two volumes of the IAEP Technical Report were published.

Publications

A World of Differences: An International Assessment of Mathematics and Science. Report of the International Assessment of Educational Progress. Educational Testing Service. (1989)

A World of Differences: Technical Report. Report of the International Assessment of Educational Progress. Educational Testing Service. (1989)

Learning Mathematics. Report of The International Assessment of Educational Progress. Educational Testing Service. (1992)

Learning Science. Report of The International Assessment of Educational Progress. Educational Testing Service. (1992)

Learning Geography. Report of the International Assessment of Educational Progress. Educational Testing Service. (1992)

Performance Assessment: An International Experiment. Report of the International Assessment of Educational Progress. Educational Testing Service. (1992)

IAEP Technical Report. Report of the International Assessment of Educational Progress. Educational Testing Service. (1992)

IAEP Technical Report: Volume Two. Report of the International Assessment of Educational Progress. Educational Testing Service. (1992)

Two short reports on implications of IAEP results for U.S. efforts at standard setting and curriculum reform will be published by the National Science Foundation.

Funding

International Assessment of Educational Progress - I: Funding was provided by the U.S. Department of Education and the National Science Foundation for overall coordination,

Suggested Citation:"International Assessment of Educational Progress." National Research Council. 1995. International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies. Washington, DC: The National Academies Press. doi: 10.17226/9174.
×

sampling, U.S. data analysis, and reporting. Participating countries and provinces acquired support for local data collection and coordination.

International Assessment of Educational Progress - II: Funding was provided by The National Science Foundation and the U.S. Department of Education's National Center for Education Statistics provided a grant for the expenses of overall coordination, sampling, data analysis, and reporting. The Carnegie Corporation provided additional funds to cover the travel expenses of some of the participants who could not meet the financial burdens of traveling to the project 's coordination and training meetings, held in Canada, England, France, Hong Kong, and the United States. Participating countries acquired support for local data collection and coordination.

Information Sources

Nancy A. Mead, International Coordinator/Project Director

Center for the Assessment of Educational Progress

Educational Testing Service

P.O. Box 6710

Princeton, New Jersey 08541-6710

telephone: 609/ 734-1526

e-mail: nmead@rosedale.org

Educan Inc.

1991 IAEP Notes. 4:September. Educan Inc., Longueuil, Quebec, Canada.

Educational Testing Service

1991 The 1991 IAEP Assessment: Objectives for Mathematics, Science, and Geography. Educational Testing Service, Princeton, New Jersey.

1992 IAEP Technical Report. Report of the International Assessment of Educational Progress. ETS, Princeton, New Jersey.

1992 IAEP Technical Report: Volume Two. Report of the International Assessment of Educational Progress. ETS, Princeton, New Jersey.

Lapointe, Archie E.

1988 IAEP preliminary findings; comparison with IEA findings. Presentation to the Board on International Comparative Studies in Education. November 4.

1991 IAEP analysis and report plans. Presentation to the Board on International Comparative Studies in Education. January 28.

1991 IAEP report draft. Presentation to the Board on International Comparative Studies in Education. September 23.

Lapointe, Archie E., Nancy A. Mead, and Eugene Johnson

1990 IAEP status. Presentation to the Board on International Comparative Studies in Education. February 2.

Lapointe, Archie E., Nancy A. Mead, and Janice M. Askew

1992 Learning Mathematics. Report of the International Assessment of Educational Progress. Report No. 22-CAEP-01. February. Educational Testing Service, Princeton, New Jersey.

Lapointe, Archie E., Janice M. Askew, and Nancy A. Mead

1992 Learning Science. Report of the International Assessment of Educational Progress. Report No. 22-CAEP-02. February. Educational Testing Service, Princeton, New Jersey.

Suggested Citation:"International Assessment of Educational Progress." National Research Council. 1995. International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies. Washington, DC: The National Academies Press. doi: 10.17226/9174.
×

Lapointe, Archie E., Nancy A. Mead, and Gary W. Phillips

1989 A World of Differences: An International Assessment of Mathematics and Science. Report of the First International Assessment of Educational Progress. Report No. 19-CAEP-01. January. Educational Testing Service, Princeton, New Jersey.

Lazer, Stephen

1992 Learning About the World. Report of The International Assessment of Educational Progress. Report No. 22-CAEP-05. June. Educational Testing Service, Princeton, New Jersey.

Mead, Nancy A.

1990 Second International Assessment of Educational Progress. Status report prepared for the Board on International Comparative Studies in Education. April 25.

1992 Second International Assessment of Educational Progress. Status report prepared for the Board on International Comparative Studies in Education. September.

1993 Secondary Analysis of IAEP Results. Status report prepared for the Board on International Comparative Studies in Education. January.

Semple, Brian McLean

1992 Performance Assessment: An International Experiment. Report of the International Assessment of Educational Achievement. Report No. 22-CAEP-06. July. Educational Testing Service, Princeton, New Jersey.

******

NOTE: This study summary was reviewed and edited by Nancy Mead at the Educational Testing Service on June 10, 1994.

Suggested Citation:"International Assessment of Educational Progress." National Research Council. 1995. International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies. Washington, DC: The National Academies Press. doi: 10.17226/9174.
×
Page 48
Suggested Citation:"International Assessment of Educational Progress." National Research Council. 1995. International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies. Washington, DC: The National Academies Press. doi: 10.17226/9174.
×
Page 49
Suggested Citation:"International Assessment of Educational Progress." National Research Council. 1995. International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies. Washington, DC: The National Academies Press. doi: 10.17226/9174.
×
Page 50
Suggested Citation:"International Assessment of Educational Progress." National Research Council. 1995. International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies. Washington, DC: The National Academies Press. doi: 10.17226/9174.
×
Page 51
Suggested Citation:"International Assessment of Educational Progress." National Research Council. 1995. International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies. Washington, DC: The National Academies Press. doi: 10.17226/9174.
×
Page 52
Suggested Citation:"International Assessment of Educational Progress." National Research Council. 1995. International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies. Washington, DC: The National Academies Press. doi: 10.17226/9174.
×
Page 53
Suggested Citation:"International Assessment of Educational Progress." National Research Council. 1995. International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies. Washington, DC: The National Academies Press. doi: 10.17226/9174.
×
Page 54
Suggested Citation:"International Assessment of Educational Progress." National Research Council. 1995. International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies. Washington, DC: The National Academies Press. doi: 10.17226/9174.
×
Page 55
Suggested Citation:"International Assessment of Educational Progress." National Research Council. 1995. International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies. Washington, DC: The National Academies Press. doi: 10.17226/9174.
×
Page 56
Suggested Citation:"International Assessment of Educational Progress." National Research Council. 1995. International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies. Washington, DC: The National Academies Press. doi: 10.17226/9174.
×
Page 57
Next: International Comparative Study of Mathematics and Science Tests »
International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies Get This Book
×
MyNAP members save 10% online.
Login or Register to save!
  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!