Executive Summary

In his 1997 State of the Union address, President Clinton announced a federal initiative to develop tests of 4th-grade reading and 8th-grade mathematics that could be administered on a voluntary basis by states and school districts beginning in spring 1999. The principal purpose of the Voluntary National Tests (VNT) is to provide parents and teachers with systematic and reliable information about the verbal and quantitative skills that students have achieved at two key points in their educational careers. The U.S. Department of Education anticipated that this information would serve as a catalyst for continued school improvement, by focusing parental and community attention on achievement and by providing an additional tool to hold school systems accountable for their students' performance in relation to nationwide standards.

Shortly after initial development work on the VNT, Congress transferred responsibility for VNT policies, direction, and guidelines from the department to the National Assessment Governing Board (NAGB, the governing body for the National Assessment of Educational Progress). Test development activities were to continue, but Congress prohibited pilot and field testing and operational use of the VNT pending further consideration. At the same time, Congress called on the National Research Council (NRC) to assess the VNT development activities. Specifically, the NRC was charged to evaluate:

  1. the technical quality of any test items for 4th-grade reading and 8th-grade mathematics;

  2. the validity, reliability, and adequacy of developed test items;

  3. the validity of any developed design which links test results to student performance levels;

  4. the degree to which any developed test items provide valid and useful information to the public;

  5. whether the test items are free from racial, cultural, or gender bias;

  6. whether the test items address the needs of disadvantaged, limited English proficient, and disabled students; and

  7. whether the test items can be used for tracking, graduation, or promotion of students.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 1
Evaluation of the Voluntary National Tests, Year 2: Final Report Executive Summary In his 1997 State of the Union address, President Clinton announced a federal initiative to develop tests of 4th-grade reading and 8th-grade mathematics that could be administered on a voluntary basis by states and school districts beginning in spring 1999. The principal purpose of the Voluntary National Tests (VNT) is to provide parents and teachers with systematic and reliable information about the verbal and quantitative skills that students have achieved at two key points in their educational careers. The U.S. Department of Education anticipated that this information would serve as a catalyst for continued school improvement, by focusing parental and community attention on achievement and by providing an additional tool to hold school systems accountable for their students' performance in relation to nationwide standards. Shortly after initial development work on the VNT, Congress transferred responsibility for VNT policies, direction, and guidelines from the department to the National Assessment Governing Board (NAGB, the governing body for the National Assessment of Educational Progress). Test development activities were to continue, but Congress prohibited pilot and field testing and operational use of the VNT pending further consideration. At the same time, Congress called on the National Research Council (NRC) to assess the VNT development activities. Specifically, the NRC was charged to evaluate: the technical quality of any test items for 4th-grade reading and 8th-grade mathematics; the validity, reliability, and adequacy of developed test items; the validity of any developed design which links test results to student performance levels; the degree to which any developed test items provide valid and useful information to the public; whether the test items are free from racial, cultural, or gender bias; whether the test items address the needs of disadvantaged, limited English proficient, and disabled students; and whether the test items can be used for tracking, graduation, or promotion of students.

OCR for page 1
Evaluation of the Voluntary National Tests, Year 2: Final Report Since the evaluation began, the NRC has issued three reports on VNT development: an interim and final report on the first year's work and an interim report earlier on this second year's work. This final report includes the findings and recommendations from the interim report, modified by new information and analysis, and presents our overall conclusions and recommendations regarding the VNT. Congress must answer the overarching policy question about the VNT: whether development should continue or be terminated. The committee does not take a position either for or against continued development. However, the committee has reached two general conclusions about the development of the VNT. Our first conclusion deals with our overall assessment. CONCLUSION VNT development is generally on course. A large number of items have been written, and the quality of the items that came through the contractor's development and review process is comparable to the quality of items from NAEP. Plans for pilot testing these items are generally sound. The committee does have a number of specific recommendations to further improve item and test quality, to improve future cycles of the test development process, to enhance the meaningful inclusion of all students, to provide effective reports of results, and to prevent misuse of test results. However, there is no evidence that the current process should be halted on technical grounds. Our second general conclusion addresses the potential value of a pilot test of VNT items. CONCLUSION The planned pilot test of VNT test items presents opportunities for research on a number of important test development topics that will be useful to NAEP and state and local assessment programs even if the VNT is eventually terminated. These research opportunities include: (1) assessing the quality and effectiveness of the VNT's item development, review, and revisions processes; (2) collecting empirical data on the effect of different threats to ''linkability;" and (3) assessing the feasibility, effects, and validity of alternative testing accommodations for students with disabilities or limited English proficiency. In addition, the items themselves are likely to be useful for other testing programs. Finally, the committee offers an overall recommendation to Congress in considering its decision about the VNT. RECOMMENDATION TO CONGRESS The decision to continue or terminate the VNT should be based on a carefully articulated statement of the expected value and costs of the program, including a detailed examination of underlying assumptions and a delineation of possible unintended outcomes. To the maximum extent possible, research on results from other educational reform efforts should be laid out to support or contradict assumptions in this value-and-cost statement. Information on the likelihood of use by states, districts, and individuals should also be considered in making a decision about the VNT.