Appendix F Revised Item Development and Review Schedule for VNT.

Below is a summary of the key dates in 1998 for delivery of items, NAGB review, and panel meetings:

Date

Activity

8/18–25

Achievement level review panels

8/18–22

Content expert panels

9/1–4 (tent.)

Content coverage panels

9/28

1st batch of math items to NAGB

10/5

1st batch of reading items to NAGB

10/16

Reviews back from NAGB

10/30

Final batches of items to NAGB

11/13

Final reviews back from NAGB

11/19–21

NAGB final action on items



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 66
--> Appendix F Revised Item Development and Review Schedule for VNT. Below is a summary of the key dates in 1998 for delivery of items, NAGB review, and panel meetings: Date Activity 8/18–25 Achievement level review panels 8/18–22 Content expert panels 9/1–4 (tent.) Content coverage panels 9/28 1st batch of math items to NAGB 10/5 1st batch of reading items to NAGB 10/16 Reviews back from NAGB 10/30 Final batches of items to NAGB 11/13 Final reviews back from NAGB 11/19–21 NAGB final action on items

OCR for page 66
--> The following chart summarizes the review panels, tentative meeting dates, purpose, and composition: Review Panel Tentative Dates Purpose Composition Ongoing Reading Expert Panel (ORE) 8/18-8/20 To review items, using information from cognitive labs, with the goal of providing final recommendations to the publishers Content experts, including teachers and curriculum specialists with experience in elementary school reading instruction Ongoing Math Expert Panel (OME) 8/20-8/22   Teachers and curriculum specialists with experience in middle school math instruction Independent Reading Achievement Levels Panel (IRAL) and Independent Math Achievement Levels Panel (IMAL) Early September To ensure that the NAEP achievement levels are well covered by the VNT items Teachers, curriculum experts and teacher educators familiar with NAEP achievement levels and content areas of math and reading Independent Reading Content Coverage Panel (IRCC) and Independent Math Content Coverage Panel (IMCC) First week of September To ensure that the NAEP content frameworks are well covered by the VNT items Teachers, curriculum experts, and teacher educators familiar with NAEP content frameworks The following table summarizes the revised plans for VNT item development: Month Date Activity July 7/27–28 • Readability study held at AIR   7/27–31 • Identify ORE and ON-IE Panel members     • Plan expanded item review activities, including budget     • Identify ORE and OME Panel members     • Arrange for meeting space     • Determine review sequence for items (e.g., ECRs first in math, intertextual sets first in reading)   Late July • SAC #1 reviews reading items via teleconference August 8/3–7 • Receive all reading and math items in their most current state from publishers     • Contact IRAL, IMAL, IRCC, and IMCC Panel members

OCR for page 66
--> Month Date Activity     • Prepare items and related cognitive lab data in preparation for OME and ORE Panel meetings     • Develop OME and ORE Panel meeting agendas     • Design format/process for providing recommendations to publishers from OME and ORE Panel meetings     • Begin reviewing cognitive lab tapes for rubric development and to inform content panels   8/5–6 • SAC # 2 reviews math items in person   8/6–8 • NAGB meeting   8/10–14 • Finish preparing meeting materials for OME and ORE Panels     • Begin preparing materials for IRAL and IMAL Panel meetings     • Begin preparing student work for rubric refinement   8/10 • Rubric refinement work begins at Harcourt and Riverside   8/18–21 (tentative) • IRAL Panel meeting   8/22–25 (tentative) • IMAL Panel meeting   8/18–20 (tentative) • OME Panel meeting   8/20–22 (tentative) • ORE Panel meeting   8/24–9/4 • Process reviews from OME and ORE Panels     • Send revision requests to publishers as completed     • Send final recommendations to Harcourt and Riverside by 9/4 September First week of September • IRAL and IMAL Panel meetings   Mid-September • Additional OME and ORE Panel meetings, as needed     • Revisions to math and reading items processed at AIR     • Requests for revision sent to Harcourt and Riverside   9/21 • First batch of revised items due from Harcourt and Riverside   9/21–10/2 • Check revised items to prepare for NAGB     • Prepare items for submission to NAGB   9/28 • Math items submitted to NAGB   September – October • Additional item development, if needed, based on IRAL, IMAL, IRCC, and IMCC Panel meetings October 10/5 • Reading items to NAGB   October • Continue process of reviewing OME and ORE experts' comments on reading and math items     • Submit requests for revision to publishers     • Process items as they are returned from publishers   10/16 • All reviews to items in batch #1 returned from NAGB.   10/30 • Final batch of math and reading items to NAGB for review (assuming no new item development)

OCR for page 66
--> Month Date Activity November 11/2 • AIR completes review of newly developed items, if necessary   11/3–10 • Publishers revise newly developed items, if necessary   11/16 • Newly developed items to NAGB for review, if necessary   11/13 • Final NAGB review of items (excluding any newly developed items)   11/19–21 • NAGB Board meeting and final action on items     • Final NAGB review of newly developed items   11/30 • NAGB Reading and Math Committees final action on newly developed items, if necessary December December • Pilot test forms assembled January 1/1/1999 • Camera-ready copy to printers