National Academies Press: OpenBook

Evaluation of the Voluntary National Tests: Phase 1 (1999)

Chapter: Appendix F: Revised Item Development and Review Schedule for VNT

« Previous: Appendix E: Descriptions of Achievement Levels for Basic, Proficient, and Advanced
Suggested Citation:"Appendix F: Revised Item Development and Review Schedule for VNT." National Research Council. 1999. Evaluation of the Voluntary National Tests: Phase 1. Washington, DC: The National Academies Press. doi: 10.17226/6324.
×

Appendix F Revised Item Development and Review Schedule for VNT.

Below is a summary of the key dates in 1998 for delivery of items, NAGB review, and panel meetings:

Date

Activity

8/18–25

Achievement level review panels

8/18–22

Content expert panels

9/1–4 (tent.)

Content coverage panels

9/28

1st batch of math items to NAGB

10/5

1st batch of reading items to NAGB

10/16

Reviews back from NAGB

10/30

Final batches of items to NAGB

11/13

Final reviews back from NAGB

11/19–21

NAGB final action on items

Suggested Citation:"Appendix F: Revised Item Development and Review Schedule for VNT." National Research Council. 1999. Evaluation of the Voluntary National Tests: Phase 1. Washington, DC: The National Academies Press. doi: 10.17226/6324.
×

The following chart summarizes the review panels, tentative meeting dates, purpose, and composition:

Review Panel

Tentative Dates

Purpose

Composition

Ongoing Reading Expert Panel (ORE)

8/18-8/20

To review items, using information from cognitive labs, with the goal of providing final recommendations to the publishers

Content experts, including teachers and curriculum specialists with experience in elementary school reading instruction

Ongoing Math Expert Panel (OME)

8/20-8/22

 

Teachers and curriculum specialists with experience in middle school math instruction

Independent Reading Achievement Levels Panel (IRAL) and Independent Math Achievement Levels Panel (IMAL)

Early September

To ensure that the NAEP achievement levels are well covered by the VNT items

Teachers, curriculum experts and teacher educators familiar with NAEP achievement levels and content areas of math and reading

Independent Reading Content Coverage Panel (IRCC) and Independent Math Content Coverage Panel (IMCC)

First week of September

To ensure that the NAEP content frameworks are well covered by the VNT items

Teachers, curriculum experts, and teacher educators familiar with NAEP content frameworks

The following table summarizes the revised plans for VNT item development:

Month

Date

Activity

July

7/27–28

• Readability study held at AIR

 

7/27–31

• Identify ORE and ON-IE Panel members

 

 

• Plan expanded item review activities, including budget

 

 

• Identify ORE and OME Panel members

 

 

• Arrange for meeting space

 

 

• Determine review sequence for items (e.g., ECRs first in math, intertextual sets first in reading)

 

Late July

• SAC #1 reviews reading items via teleconference

August

8/3–7

• Receive all reading and math items in their most current state from publishers

 

 

• Contact IRAL, IMAL, IRCC, and IMCC Panel members

Suggested Citation:"Appendix F: Revised Item Development and Review Schedule for VNT." National Research Council. 1999. Evaluation of the Voluntary National Tests: Phase 1. Washington, DC: The National Academies Press. doi: 10.17226/6324.
×

Month

Date

Activity

 

 

• Prepare items and related cognitive lab data in preparation for OME and ORE Panel meetings

 

 

• Develop OME and ORE Panel meeting agendas

 

 

• Design format/process for providing recommendations to publishers from OME and ORE Panel meetings

 

 

• Begin reviewing cognitive lab tapes for rubric development and to inform content panels

 

8/5–6

• SAC # 2 reviews math items in person

 

8/6–8

• NAGB meeting

 

8/10–14

• Finish preparing meeting materials for OME and ORE Panels

 

 

• Begin preparing materials for IRAL and IMAL Panel meetings

 

 

• Begin preparing student work for rubric refinement

 

8/10

• Rubric refinement work begins at Harcourt and Riverside

 

8/18–21 (tentative)

• IRAL Panel meeting

 

8/22–25 (tentative)

• IMAL Panel meeting

 

8/18–20 (tentative)

• OME Panel meeting

 

8/20–22 (tentative)

• ORE Panel meeting

 

8/24–9/4

• Process reviews from OME and ORE Panels

 

 

• Send revision requests to publishers as completed

 

 

• Send final recommendations to Harcourt and Riverside by 9/4

September

First week of September

• IRAL and IMAL Panel meetings

 

Mid-September

• Additional OME and ORE Panel meetings, as needed

 

 

• Revisions to math and reading items processed at AIR

 

 

• Requests for revision sent to Harcourt and Riverside

 

9/21

• First batch of revised items due from Harcourt and Riverside

 

9/21–10/2

• Check revised items to prepare for NAGB

 

 

• Prepare items for submission to NAGB

 

9/28

• Math items submitted to NAGB

 

September – October

• Additional item development, if needed, based on IRAL, IMAL, IRCC, and IMCC Panel meetings

October

10/5

• Reading items to NAGB

 

October

• Continue process of reviewing OME and ORE experts' comments on reading and math items

 

 

• Submit requests for revision to publishers

 

 

• Process items as they are returned from publishers

 

10/16

• All reviews to items in batch #1 returned from NAGB.

 

10/30

• Final batch of math and reading items to NAGB for review (assuming no new item development)

Suggested Citation:"Appendix F: Revised Item Development and Review Schedule for VNT." National Research Council. 1999. Evaluation of the Voluntary National Tests: Phase 1. Washington, DC: The National Academies Press. doi: 10.17226/6324.
×

Month

Date

Activity

November

11/2

• AIR completes review of newly developed items, if necessary

 

11/3–10

• Publishers revise newly developed items, if necessary

 

11/16

• Newly developed items to NAGB for review, if necessary

 

11/13

• Final NAGB review of items (excluding any newly developed items)

 

11/19–21

• NAGB Board meeting and final action on items

 

 

• Final NAGB review of newly developed items

 

11/30

• NAGB Reading and Math Committees final action on newly developed items, if necessary

December

December

• Pilot test forms assembled

January

1/1/1999

• Camera-ready copy to printers

Suggested Citation:"Appendix F: Revised Item Development and Review Schedule for VNT." National Research Council. 1999. Evaluation of the Voluntary National Tests: Phase 1. Washington, DC: The National Academies Press. doi: 10.17226/6324.
×
Page 66
Suggested Citation:"Appendix F: Revised Item Development and Review Schedule for VNT." National Research Council. 1999. Evaluation of the Voluntary National Tests: Phase 1. Washington, DC: The National Academies Press. doi: 10.17226/6324.
×
Page 67
Suggested Citation:"Appendix F: Revised Item Development and Review Schedule for VNT." National Research Council. 1999. Evaluation of the Voluntary National Tests: Phase 1. Washington, DC: The National Academies Press. doi: 10.17226/6324.
×
Page 68
Suggested Citation:"Appendix F: Revised Item Development and Review Schedule for VNT." National Research Council. 1999. Evaluation of the Voluntary National Tests: Phase 1. Washington, DC: The National Academies Press. doi: 10.17226/6324.
×
Page 69
Next: Appendix G: Observations of Cognitive Labs and Bias Reviews »
Evaluation of the Voluntary National Tests: Phase 1 Get This Book
×
Buy Paperback | $47.00 Buy Ebook | $37.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

In his 1997 State of the Union address, President Clinton announced a federal initiative to develop tests of 4th-grade reading and 8th-grade mathematics that would provide reliable information about student performance at two key points in their educational careers. According to the U.S. Department of Education, the Voluntary National Tests (VNT) would create a catalyst for continued school improvement by focusing parental and community-wide attention on achievement and would become new tools to hold school systems accountable for their students' performance. The National Assessment Governing Board (NAGB) has responsibility for development of the VNT. Congress recognized that a testing program of the scale and magnitude of the VNT initiative raises many important technical questions and requires quality control throughout development and implementation. In P.L. 105-78, Congress called on the National Research Council (NRC) to evaluate a series of technical issues pertaining to the validity of test items, the validity of proposed links between the VNT and the National Assessment of Educational Progress (NAEP), plans for the accommodation and inclusion of students with disabilities and English-language learners, plans for reporting test information to parents and the public, and potential uses of the tests. This report covers phase 1 of the evaluation (November 1997-July 1998) and focuses on three principal issues: test specifications and frameworks; preliminary evidence of the quality of test items; and plans for the pilot and field test studies, for inclusion and accommodation, and for reporting VNT results.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!