Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 267
GRADING THE NATION'S REPORT CARD: Evaluating NAEP and Transforming the Assessment of Educational Progress
Index
A
Accommodations/adaptations, 51, 88-89, 91-112 (passim), 191
Americans with Disabilities Act, 88
Goals 2000: Educate America Act, 10, 88, 96
individualized education plans, 88, 94, 99-100
Individuals with Disabilities Education Act, 88
items read orally to poor readers, 13, 95, 99, 105
test administration procedures, 91, 92, 99-100
Accountability, 2, 10, 19, 23, 27, students with disabilities/English-language learners, 88, 91, 94, 107, 109, 110-111, 112
Achievement-level descriptions, 114, 128, 134-139, 141, 152-154, 163-173 (passim)
preliminary, 119, 134-135, 165, 170, 176-179, 182, 183, 256, 257, 259, 260
Achievement levels, 2, 7, 9, 10, 17, 20, 25, 72, 73, 162-184, cost factors, 8
definitional issues, 54, 124, 137-138, 139, 169-170, 186
frameworks and assessment development process, 115, 116, 117, 121 , 124-125, 128, 134, 135-140, 143-147 (passim), 152-154, 158, 159, 160, 192, 194, 195, 196
reporting, 2, 6, 7, 25-31, 32, 33, 72, 73, 135, 162, 163-165
special studies, 16, 57, 110
use of NAEP results, 2, 31, 171, 176, 180, 182, 184, 246
see also Mathematics achievement;
Performance standards;
Reading achievement;
Science achievement;
Writing achievement
Achievement-level setting, 2, 7, 31, 48, 116, 134-135, 152, 162, 163-168, 171, 176, 179-184, 186, 246, 256-261
anchoring, 2, 170-171, 173, 177, 257-261
Angoff procedure, 165, 166, 168
NCES role, 167, 175-176, 187, 196
see also Cutscores
Administration procedures, testing, 12-13, 65
costs, 78
grade-vs age-level testing, 13, 14
individual testing, 12-13, 99
item response time, 13, 62, 99, 128, 130-131
items read orally due to poor readers, 13, 95, 99, 105
professional administrators, 13, 82, 86
state vs national NAEP, 82, 85
OCR for page 268
GRADING THE NATION'S REPORT CARD: Evaluating NAEP and Transforming the Assessment of Educational Progress
students with disabilities/English-language learners, 91, 92, 99-100
see also Group testing
Administration schedules, testing, 18, 121, 191, 192-193, 195
Administrative factors, see Governance factors;
School-level measures
Advanced achievement levels, 2, 7, 64, 147, 152, 163, 164-171, 173 , 174, 175, 180, 183, 186, 256, 259, 260
Advanced placement, 165, 167, 169, 174
Advisory Council on Education Statistics, 53
African Americans, 44, 62
Age factors, 12, 13, 79
grade-vs age-level testing, 13, 14, 192-193
see also Grade-level factors
Alexander-James Panel, 2, 15
American College Testing, 19, 168-169, 190
American Institutes for Research, 190
Americans with Disabilities Act, 88
Analysis methods, 20, 189, 193, 196
coordinated system of education indicators, 45, 49-51 (passim), 55-56
costs, 78
current NAEP, 35, 58, 62, 63-64, 65, 67, 68-70, 71, 78
frameworks and assessment development process, 116, 117
reporting metrics, 72
secondary data, 42
streamlining NAEP design, 4, 14, 62, 63-64, 65, 67, 68-70, 71, 78 , 79
students with disabilities/English-language learners, 101, 104, 108, 110
text analysis, 43, 49, 219-220
video analyses, 43, 46, 48, 194
see also Computer-based testing and analysis;
Content coverage and analysis;
Item analysis and development
Anchoring, 2, 170-171, 173, 177, 257-261
Angoff procedure, 165, 166, 168
Applied Measurement in Education, 11
Arts achievement, 193-194
Assessment development process, 2, 4, 5-6, 33, 67, 92, 114-161, 190-191, 194, 195, 220-236
achievement levels, 115, 116, 117, 121, 124-125, 128, 134, 135-140 , 145-147, 152-154, 158, 159, 160, 192, 194, 195, 196;
see also ''mathematics achievement'' and "science achievement" infra
constructed-response items, 115, 122, 123, 128-129, 132, 146, 149 , 150, 153-154
contractors, 115, 116, 118, 119, 136, 148-150, 190, 192-193, 195
defined, 115
funding, 149-150, 151, 155, 191
governance, 117-118, 134-135
interpretive information, 114, 116, 137, 153, 154-157, 159, 160-161
item analysis and development, 117, 118-119, 122-123, 125, 128-137 , 142, 149-155, 160, 219-236, 256-257
knowledge and skills measures, 115, 121, 125, 126, 127, 128, 131-133, 138, 139, 140, 156, 159, 233, 256-257
large-scale assessments, 114, 125, 128, 133, 136, 147-149, 156, 158, 159, 161, 194
main NAEP, 114, 118-121, 126-128, 136, 194, 195, 259
mathematics achievement, 118, 121, 122-124, 126-127, 132, 136, 142
NCES role, 115, 116, 118, 122, 134, 148-149, 154
performance standards, 114, 115, 118, 123, 125, 133, 142-143, 146 , 149, 155, 159, 220, 257, 259-260
reading achievement, 118, 119, 120, 122, 123, 127-128, 132, 133, 136, 141-142, 219-230
reporting procedures, 114, 117, 121, 128, 135, 145-146, 151, 154-160 (passim), 221
science achievement, 116, 118, 126, 129-130, 132-133, 134, 136, 30, 132-133, 134, 136, 142, 143-144, 231-236
scoring, 114, 117, 118, 119, 125, 132, 133-134, 135, 136, 146, 149 , 150, 153-154, 156, 160, 161, 219-220, 226-227, 234-235, 257
summary scores, 116, 117, 121, 128, 139, 152, 154
task measures, 119, 128, 132, 136, 139, 142-144, 146, 149, 153-154 , 160, 219-220
see also Field tests;
Item analysis and development
Assessment in Transition: Monitoring the Nation's Educational Progress, 25, 124
Association of State Assessment Programs, 97
OCR for page 269
GRADING THE NATION'S REPORT CARD: Evaluating NAEP and Transforming the Assessment of Educational Progress
B
Background variables, 34
instructional, 65
students, 35, 40, 42, 64, 95, 119, 122, 149, 188, 190, 220
teachers, 40, 41, 43
Balanced incomplete blocks, 62, 63, 66
Barometer information, see Descriptive information
Bias, see Error of measurement
Bilingual education, 91, 98-101 (passim), 104, 105
translated versions of NAEP, 95, 97, 100, 101, 111
Black persons, see African AmericansC
Center for Equity and Excellence, 98
Civics, 18, 61, 136
Class size, 26, 44
Cognitive factors and theory, 6, 7, 19-20, 128, 133, 137-141, 143-146, 166
see also Knowledge and skills measures;
Learning theory;
Problem solving
Common Core of Data, 36-37, 51
Computer-based testing and analysis, 32, 63, 148, 194
The Condition of Education, 35
Constructed-response items, 80, 136, 166, 174
frameworks and assessment development process, 115, 122, 123, 128-129, 132, 146, 149, 150, 153-154
main vs trend NAEP, 79
national NAEP, 80
nonresponse rates, 123
Content coverage and analysis, 12, 27, 48-49, 70, 79, 118
text analysis, 43, 49, 219-220
Contextual data, see Interpretive information
Contracts and contractors, 17, 31, 68, 82, 116, 118, 136, 150, 166 , 168, 177, 191, 195, 196
administration of testing, state vs national, 82
Educational Testing Service, 14, 17, 31, 93, 118, 119, 148-149, 150, 190
frameworks and assessment development process, 115, 116, 118, 119 , 136, 148-150, 190, 192-193, 195
KPMG Peat Marwick LLP, 25, 66, 69, 70, 78
NAEP, redesign/transitional, 148, 175, 187, 189-196 (passim)
National Computer Systems, 119
RAND, 40, 46
scheduled contracts and administration by framework, 192-193, 195
WESTAT, 17, 119
Coordinated system of education indicators, 4, 6, 8, 20, 22-55, 73, 158, 186, 187, 188
analysis methods, 45, 49-51 (passim), 55-56
costs of, 51-52, 53
curriculum, 23, 40, 41, 43, 54
data collection, 4, 6, 8, 20, 22-55, 106
Department of Education role, 23-24, 34, 50, 53-54
frameworks, 24, 33, 38, 40, 47, 51-55 (passim)
funding, 23, 41, 42, 44, 53
governance, 45, 51-54
NAGB, 22-25, 31, 187
international components, 42, 43, 46, 158, 188
interpretive information, 4, 23, 24, 25, 27-55, 45, 115
knowledge and skills measures, 23-24, 47
large-scale assessments, 24, 34, 47, 51, 54, 188
NCES, 31, 34-38, 40-41, 45-46, 49-54, 187
new paradigm NAEP, 46-47, 48, 55, 84, 85, 109, 110, 146-147, 151, 158, 187-197
pedagogy, 23, 40-43 (passim), 46, 47-49, 54
reporting procedures, 25-26, 73
sampling, 45, 46, 47, 49, 50-51, 107
skills measures, 24, 47
standards, general, 23, 45
state-level factors, 45-46
students with disabilities/English-language learners, 106, 107
task measures, 23, 24, 33, 47, 54
validity issues, 40, 46
Core NAEP, 7, 47, 57, 78, 80, 82, 84, 85, 147-149, 153, 155, 158, 181-182, 187, 188, 194
students with disabilities/English-language learners, 109, 110, 112
see also Mathematics achievement;
Reading achievement;
Science achievement;
Writing achievement
Cost and cost-benefit factors, 8, 10, 78
elementary/secondary school expenditures, data on, 35
OCR for page 270
GRADING THE NATION'S REPORT CARD: Evaluating NAEP and Transforming the Assessment of Educational Progress
indicator systems, 38, 51-52, 53
NAEP design, general, 3, 8, 62, 78, 84, 149-150, 151
national/state systems separate, 66, 78, 82, 85
sampling designs, 62, 66
state-level indicators, 62, 66, 78, 82, 85
students with disabilities/English-language learners, 8, 107-108, 111
trend NAEP, 78
see also Accountability;
Funding
Council of Chief State School Officers, 17, 40, 46, 53, 96, 98
Council of the Great City Schools, 53
CSEI, see Coordinated system of education indicators
Cultural bias, 63
Curriculum, 6
cognitive theory and, 19-20
coordinated system of education indicators, 23, 40, 41, 43, 54
high school, 80-81
NAEP design and, 12, 16, 20, 58, 79, 80-81
special studies, 16, 57
standards, 10, 179, 153
students with disabilities/English-language learners, 111
Curriculum and Evaluation Standards for School Mathematics, 10, 122
Cutscores, 64, 166, 167-171, 173-177 (passim), 180, 182, 183, 256 , 260D
Databases, 50-52, 196
Data collection, general, 4, 11, 14, 20, 23, 196, 259
coordinated system of education indicators, 4, 6, 8, 20, 22-55, 106
frameworks, 115, 150
mixed methods of, 43, 45, 47-49, 54
NAGB objectives, 25
national vs state, 65
streamlining NAEP design, 4, 7, 65, 68, 73, 76, 79-82, 84, 190, 196
students with disabilities/English-language learners, 87, 96, 104 , 106, 107, 108, 112, 113
see also Large-scale assessments;
Sampling
Data presentation, see Reporting procedures
Demographic factors, 1, 152, 19
performance measures, 27, 163
reporting, 1-2, 27, 31, 34, 163
socioeconomic status, 14, 50
poverty, 14, 44
social indicator systems, 3, 24, 38-39, 152, 153
summary scores, 14
students with disabilities/English-language learners, 89-90
see also Age factors;
Gender factors;
Race/ethnicity;
Urban/rural
factors
Department of Education, 1, 3, 187
coordinated system of education indicators, 23-24, 34, 50, 53-54
English-language learners, 105, 106
students with disabilities, 106, 109
see also Contracts and contractors;
National Center for Education
Statistics;
terms beginning "Office of…"
Department of Education Organization Act, 88
Descriptive information, 3, 8, 27-32, 33, 57, 72, 114, 128, 164, 197
cost factors, 8
students with disabilities, 101
Design/Feasibility Team, 66, 72, 78-79, 190
Difficulty level, see Item difficulty
Dimensionality of data, 63, 70
Disabilities, see Learning disabilities;
Students with disabilities
Dropouts, 57, 81, 86E
Early Childhood Longitudinal Study, 36-37, 46, 110, 188
Economic factors, see Cost and cost-benefit factors;
Funding;
Poverty;
Socioeconomic status
Educating One and All: Students with Disabilities and Standards-Based Reform, 91, 94
Educational indicators, 4, 9, 10, 19, 154
summary scores, 128
see also Coordinated system of educational indicators
Educational Longitudinal Study, 36-37
Educational Testing Service, 14, 17, 31, 93, 118, 119, 148-149, 150, 190
Education Commission of the States, 14
OCR for page 271
GRADING THE NATION'S REPORT CARD: Evaluating NAEP and Transforming the Assessment of Educational Progress
Education Counts, 38
Education reform, 2, 9-10, 19, 20, 25-27 (passim), 118, 153
coordinated system of education indicators, 41
see also Accountability
Education Week, 172
Elementary and Secondary Education Act, 14
English-language learners, 7-8, 14, 19, 20, 26, 87-113, 134, 186, 191
accommodations/adaptations, 51, 88-89, 91-112 (passim), 191
analysis methods, 101, 104, 108, 110
bilingual education, 91, 98-101 (passim), 104, 105
translated versions of NAEP, 95, 97, 100, 101, 111
coordinated system of education indicators, 106, 107
core NAEP, 109, 110, 112
cost factors, 8, 107-108, 111
data collection, 87, 96, 104, 106, 107, 108, 112, 113
defined, 87(n. 1)
demography of, 89-90
educational indicators, 88, 106, 114
field tests, 98-101
funding, 108, 109
Goals 2000: Educate America Act, 10, 88, 96
grade-level factors, 90
historical perspectives, 5, 92-98
identification/classification, 102, 104-105, 106, 113
inclusion, 45, 92, 93, 95, 97-104, 106, 109, 111-113, 191
interpretive information, 5, 87, 99, 106, 112
large-scale assessments, 88, 89, 91-93, 96, 97, 98, 104, 105, 106-109, 110, 112-113
legislation, 10, 88, 96, 106, 108
mathematics achievement, 98-101, 103, 110
NCES role, 91, 93, 94, 97, 105, 106, 109, 191-192
numbers by grade level, 90
participation, 5, 19, 57
performance standards, 87, 89, 94
reading achievement, 94, 110
regional factors, 89-90
reporting procedures, 108, 111
sampling, 101, 102, 103, 106, 107
school-level measures, 89-90
skills measures, 104-105
standards, general, 5, 87, 89, 91, 92, 94, 98-102, 109, 111-113
state role, 88, 93, 94, 97, 98, 104, 106, 109, 110
summary scores, 106
test administration procedures, 91, 92, 99-100
TIMSS, 44
translated versions of NAEP, 95, 97, 100, 101, 111
urban/rural factors, 89-90
validity issues, 88, 91, 95, 106, 109-111
see also Hispanic students;
Participation, students with disabilities/English-language
learners
Equality of Opportunity Surveys, 50
Error of measurement, 63, 69, 82-83, 86, 118, 175
probability sampling, 62, 64, 66, 68-69, 175
Estimation procedures, 58, 76, 80, 86, 175
Evaluative information, 3, 4, 7, 8, 11, 27-33 (passim), 57
teacher evaluation of NAEP, 77-78, 129-130
see also Achievement levels;
Performance standards
Extended-response items, 128, 132, 136, 140, 149, 150, 155, 190-191 , 194, 220, 225, 226, 229, 230F
Family factors, 27, 34, 41, 42, 44
Federal government
national curriculum, 13-14
standards, general, 10
see also Department of Education;
Funding;
Legislation
Field tests, 51, 117, 118, 119, 133-134, 136, 150, 194
reporting procedures, tables/displays, 71
Fit of items, 63, 69, 70, 101, 118, 122-123, 125, 128-132, 220, 221, 229
Focused Reports, 65
Focus on NAEP, 65
Foreign countries, see International perspectives;
specific countries
Foreign-language speakers, see English-language learners
OCR for page 272
GRADING THE NATION'S REPORT CARD: Evaluating NAEP and Transforming the Assessment of Educational Progress
Frameworks, 2, 4, 5-6, 17, 114-161, 165, 194, 257
achievement levels, 115, 116, 117, 121, 124-125, 128, 134, 135-140 , 145-147, 152-154, 158, 159, 160, 192, 194, 195, 196;
see also other "mathematics achievement" and "science achievement" infra
analysis methods, 116, 117
constructed-response items, 115, 122, 123, 128-129, 132, 146, 149 , 150, 153-154
contractors, 115, 116, 118, 119, 136, 148-150, 190, 192-193, 195
coordinated system of education indicators, 24, 33, 38, 40, 47, 51-55 (passim)
data collection, 115, 150
design streamlining, 57, 58-61, 62, 65-66, 77
funding, 149-150, 151, 155, 191
governance, 117-118, 134-135 NAGB, 116, 118, 127, 194, 196
interpretive information, 114, 116, 137, 153, 154-157, 159, 160-161
item analysis and development, 117, 118-119, 122-123, 125, 128-137 , 142, 149-155, 160, 219-236, 256-257
knowledge and skills measures, 115, 121, 125, 126, 127, 128, 131-133, 138, 139, 140, 156, 159, 233, 256-257
large-scale assessments, 114, 125, 128, 133, 136, 147-149, 156, 158, 159, 161, 194
main NAEP, 114, 118-121, 126-128, 136, 194, 195, 259
mathematics achievement, 118, 121, 122-124, 126-127, 132, 136, 142
national NAEP, 59-60
NCES role, 115, 116, 118, 122, 134, 148-149, 154
performance standards, 114, 115, 118, 123, 125, 133, 142-143, 146 , 149, 155, 159, 220, 257, 259-260
reading achievement, 118, 119, 120, 122, 123, 127-128, 132, 133, 136, 141-142, 219-230
reporting procedures, 114, 117, 121, 128, 135, 145-146, 151, 154-160 (passim), 221
scheduled contracts and administration by framework, 192-193, 195
science achievement, 116, 118, 126, 129-130, 132-133, 134, 136, 142, 143-144, 231-236
scoring, 114, 117, 118, 119, 125, 132, 133-134, 135, 136, 146, 149 ,150, 153-154, 156, 160, 161, 219-220, 226-227, 234-235, 257
summary scores, 116, 117, 121, 128, 139, 152, 154
task measures, 119, 128, 132, 136, 139, 142-144, 146, 149, 153-154 ,160, 219-220
trend NAEP, 77-78, 192, 194
From Data to Information, 40, 41, 49-50
Funding, 19
coordinated system of education indicators, 23, 41, 42, 44, 53
framework and assessment development process, 149-150, 151, 155, 191
students with disabilities/English-language learners, 108, 109
see also Accountability;
Contracts and contractors;
Cost and cost-benefit factorsG
Gender factors, 2, 68, 152, 169
Geography achievement, 18, 61, 136, 166
Goals 2000: Educate America Act, 10, 88, 96
Governance factors, 17-19
coordinated system of education indicators, 45, 51-54
frameworks and assessment development process, 117-118, 134-135
reporting, 19, 31, 72, 164, 172, 190
see also National Assessment Governing Board
Government role, see Federal government;
Local government;
State government
Grade-level factors, 13, 14, 15, 31, 172-173, 192-193
see also Age factors
Graduate Record Exam, 93
Group testing, 12-13
group tasks, 47, 138H
Hawkins-Stafford Elementary and Secondary School Improvement Amendment , 35
High School and Beyond, 31, 46
High school students, 44, 80-81, 85, 86, 186
dropouts, 57, 81, 86
OCR for page 273
GRADING THE NATION'S REPORT CARD: Evaluating NAEP and Transforming the Assessment of Educational Progress
Hispanic students, 44, 62, 95, 99
bilingual education, 91, 98-101 (passim), 105, 110
translated versions of NAEP, 95, 97, 100, 101, 111
Historical perspectives, 1-2, 9-15, 19, 163
achievement measures, 4
contextual data, 24
indicator systems, 38
sociopolitical context, 9-11
students with disabilities/English language learners, 5, 92-98
History, as academic subject, 18, 32, 52, 61, 136, 166I
Improving America's Schools Act, 2, 10, 11, 17, 24, 88, 122, 162, 175, 196
Improving Schooling for Language-Minority Children: A Research Agenda , 91
Inclusion, 45, 92, 93, 95, 97-104, 106, 109, 111-113, 191
The Inclusion of Students with Disabilities and Limited Proficient Students in Large-Scale Assessments: A Summary of Recent Progress , 92
Indicator systems
costs of, 38, 51-52, 53
general, 38-39
purposes saved by 1, 3-4, 9, 10, 27-32, 34, 35-39, 42-45, 186;
see also Accountability
social, 3, 24, 38-39, 152, 153
see also Coordinated system of educational indicators
Individualized education plans, 88, 94, 99-100
Individual-level measures, see Student-level measures
Individuals with Disabilities Education Act, 88
Individual testing, 12-13
students with disabilities/English language learners, 99
Institute on Educational Governance, Finance, Policy Making, and Management, 53
Institute on Student Achievement, Curriculum, and Assessment, 52
Instruction, see Pedagogy
Instructional Reports, 65
Integrating Federal Statistics on Children, 40
International perspectives, 2, 15, 85, 110
coordinated system of education indicators, 42, 43, 46, 158, 188
sociopolitical trends, 9-10
TIMSS, 36-37, 40, 42-44, 46, 48, 110, 169, 174, 188, 259
Interpretive information, 6, 9, 19
cost factors, 8
coordinated system of education indicators, 4, 23, 24, 25, 27-55, 45, 115
frameworks and assessment development processes, 114, 116, 137, 153, 154-157, 159, 160-161
mass media, 9-10, 12, 25-30, 68, 71, 171
streamlining NAEP design, 4, 5, 57, 64, 65, 67-68, 71, 72, 73, 81 , 86, 190
students with disabilities/English-language learners, 5, 87, 99, 106, 112
see also Descriptive information;
Evaluative information
IRT, see Item response theory
Item analysis and development, 4, 7, 14, 62-64, 69, 178-179, 192
anchoring, 2, 170-171, 173, 177, 257-261
fit of items, 63, 69, 70, 101, 118, 122-123, 125, 128-132, 220, 221, 229
frameworks and assessment development process, 117, 118-119, 122-123, 125, 128-137, 142, 149-155, 160, 219-236, 256-257
matrix sampling, 13, 62, 68
balanced incomplete blocks, 62, 63, 66
see also Field tests
Item authoring, 256-257
Item difficulty, 13, 14, 100-101, 110, 119, 139, 166, 174-175, 179 , 238, 243, 245, 257
Item families, 146, 156-157, 160, 193, 237-255
Item mapping, 51, 169, 233, 257-259, 260-261
Item response theory, 62-64, 69, 72, 95, 101J
Jackknife methods, 64, 69K
Knowledge and skills measures, 5, 6, 12, 13, 19-20, 27, 77
coordinated system of education indicators, 23-24, 47
English-language learners, 104-105
OCR for page 274
GRADING THE NATION'S REPORT CARD: Evaluating NAEP and Transforming the Assessment of Educational Progress
frameworks and assessment development process, 115, 121, 125, 126 , 127, 128, 131-133, 138, 139, 140, 159, 233, 256-257
main NAEP, 4-5, 15-16, 77, 78-80
main vs trend NAEP, 77
see also Task measures
KPMG Peat Marwick LLP, 25, 66, 69, 70, 78L
Language factors, see Bilingual education;
English-language learners
Large-scale assessments, 3-10 (passim), 21
achievement-level setting, 182
coordinated system of education indicators, 24, 34, 47, 51, 54, 188
frameworks and assessment development processes, 114, 125, 128, 133, 136, 147-149, 156, 158, 159, 161, 194
streamlining NAEP design, 56, 57, 70, 80, 84
students with disabilities/English-language learners, 88, 89, 91-93, 96, 97, 98, 104, 105, 106-109, 110, 112-113
Learning disabilities, 88, 95, 104, 108, 110
Office of Special Education and Rehabilitative Services, 91
Learning theory, 5-6, 19-20, 128, 137, 138-141, 151, 160
LEAs, see School districts
Legislation, 10
Americans with Disabilities Act, 88
Department of Education Organization Act, 88
Elementary and Secondary Education Act, 14
Goals 2000: Educate America Act, 10, 88, 96
Hawkins-Stafford Elementary and Secondary School Improvement Amendment , 35
Improving America's Schools Act, 2, 10, 11, 17, 24, 88, 122, 162, 175, 196
Individuals with Disabilities Education Act, 88
National Education Statistics Act, 52
Perkins Act, 88
students with disabilities/English-language learners, 10, 88, 96, 106, 108
Literacy, see English-language learners;
Reading achievement;
Writing achievement
Local Education Authorities, see School districts
Local factors and trends, 4
age-vs grade-level testing, 14
streamlining testing, 81-82
see also School-level measures
Local government
national curriculum, 13
standards, 56
Longitudinal studies, 31, 34, 36-37, 39, 42, 43, 46, 85, 110, 158
see also specific studiesM
Main NAEP
frameworks and assessment development process, 114, 118-121, 126-128, 136, 194, 195, 259
knowledge measures, 4-5, 15-16, 77, 78-80
mathematics achievement, 15, 17, 28-30, 65, 77-78, 80, 84, 118, 119, 125, 126-127, 132, 156, 157, 171, 192-193
national trends, 15-16, 56, 57, 58, 65
performance standards, 4-5, 15-16, 77, 78-80
reading achievement, 15, 17, 65, 80, 84, 118, 120, 125, 127-128, 133, 171, 192-193, 219
reporting, 163, 171
science achievement, 15, 17, 28-30, 65, 77-78, 80, 84, 116, 118, 121, 125, 126, 157, 168, 192-193, 259
state, 15, 16, 56, 57, 58, 65-66
students with disabilities/English-language learners, 103
trend vs, 4-5, 15-16, 58, 63, 64, 65, 73-80, 84, 85
writing achievement, 15, 17, 65, 80, 84, 192-193
Market-basket reporting, 72, 189
Mass media, 9-10, 12, 25-30, 68, 71, 171
Mathematics achievement, 7, 10, 27, 52, 58, 60, 65
English-language learners, 98-101, 103, 110
frameworks and assessment development process, 118, 121, 122-124, 126-127, 132, 136, 142
OCR for page 275
GRADING THE NATION'S REPORT CARD: Evaluating NAEP and Transforming the Assessment of Educational Progress
item families, 237-255
main NAEP, 15, 17, 28-30, 65, 77-78, 80, 84, 118, 119, 125, 126-127, 132, 156, 157, 171, 192-193
NAEP administration schedule, 18
National Council of Teachers of Mathematics, 122, 155, 193
national NAEP, 18, 60, 63, 73, 75, 77
new paradigm NAEP, 84
reporting, 28, 58, 59, 73, 171, 172
standards, 10, 166, 171, 172, 176
state NAEP, 82
students with disabilities, 103
TIMSS, 36-37, 40, 42-44, 46, 48, 110, 169, 174, 188, 259
trend NAEP, 15, 17, 18, 60, 63, 73, 75, 77, 80, 84, 102
trend vs main NAEP, 73, 75-80 (passim)
see also Achievement levels;
Core NAEP
Matrix sampling, 13, 62, 68
balanced incomplete blocks, 62, 63, 66
Minority groups, see Race/ethnicity
Mixed methods of data collection, 43, 45, 47-49, 54
Multiple assessment strategies, 4, 6, 7, 47, 57, 66, 73-80
Multiple-choice items, 12, 118, 128-129, 136, 146, 150, 174, 229, 230
guessing/nonresponse, 13, 64
item response theory, 63
main vs trend NAEP, 79
Multiple-methods NAEP, 6, 7, 8, 10, 46, 47, 66, 80, 81, 84, 85, 109, 110, 124, 147-156 (passim), 158, 181-182, 186, 188, 190, 193, 194N
NAEP Facts, 65
NAEP Redesigned, 148-149
NAEP Report Cards, see Report Cards
National Academy of Education, 7, 25-26, 66, 67, 122, 137
performance standards, 167, 168, 174
students with disabilities/English-language learners, 92, 94, 96, 98
National Assessment Governing Board, 1, 2, 3, 11, 17, 65-66, 185, 189
achievement-level setting, 165-166, 168-171, 173-174, 175, 180, 182, 183, 187, 191, 196, 260
coordinated system of education indicators, 22-25, 31, 187
Design/Feasibility Team, 66, 72, 78-79, 190
frameworks and assessment development process, 116, 118, 127, 194, 196
NAEP objectives, general, 24-25, 78-79, 162, 189-190
reporting, 31, 72, 164, 172, 190
state government role, 25, 65-66
National Assessment of Educational Progress, see Core NAEP;
Main NAEP;
Multiple-methods NAEP;
National NAEP;
New paradigm NAEP;
State NAEP;
Trend NAEP
National Association of State Test Directors, 53
National Center for Education Statistics, 1-3, 17, 185, 189
achievement-level setting, 167, 175-176, 187, 196
coordinated system of education indicators, 31, 34-38, 40-41, 45-46, 49-54, 187
data collection, general, 36-37, 196
frameworks and assessment development process, 115, 116, 118, 122, 134, 148-149, 154
reporting procedures, 31, 154, 196
students with disabilities/English-language learners, 91, 93, 94, 97, 105, 106, 109, 191-192
see also Contracts and contractors
National Center for History in Schools, 52
National Center for Improving Student Learning and Achievement in Mathematics and Science, 52
National Center for Research on Evaluation, Standards, and Student Testing, 52, 97, 98
National Center for Research on the Organization and Restructuring of Schools, 53
National Center for the Improvement of Early Reading Achievement, 52
National Center for the Study of Teaching and Policy, 53
National Center on Education Outcomes, 92
National Center on Increasing the Effectiveness of State and Local Education Reform Efforts, 53
National Computer Systems, 119
National Council of Teachers of Mathematics, 122, 155, 193
OCR for page 276
GRADING THE NATION'S REPORT CARD: Evaluating NAEP and Transforming the Assessment of Educational Progress
National Education Longitudinal Study, 36-37, 44, 46, 50, 51, 110 , 188
National Education Statistics Act, 52
National Governors' Association, 15, 172
National Household Education Survey, 35-36
National Longitudinal Survey of Youth, 44
National NAEP, 15-16, 56, 57, 58-60, 63, 65, 73
administration of testing, 82
constructed-response items, 80
costs, 66, 78, 82, 85
frameworks, 59-60
main NAEP, 15-16, 56, 57, 58, 65
mathematics achievement, 18, 60, 63, 73, 75, 77
multiple assessments, 73-77
pedagogy, 16, 58
reading achievement, 16, 18, 59, 73-75, 77
reporting, 1-2, 14, 71
sampling, 59-60, 62, 66
science achievement, 18, 60, 63
state vs, 65, 81-84, 85
writing achievement, 16, 18, 59
National Research and Development Center on English Learning and Achievement, 52
National Science Education Standards, 10
National trends, 5, 19, 46-46
coordinated system of education indicators, 45-46
main NAEP, 15-16, 56, 57, 58, 65
NAEP results reporting, 1-2, 14
NAGB objectives, 24
sampling, 59-63, 68, 194
standards and goals, 10
streamlining, 81-84, 189
students with disabilities/English-language learners, 110
see also National NAEP
A Nation at Risk, 1, 9, 14
The Nation's Report Card, 15, 154, 172
NCES, see National Center for Education Statistics
New paradigm NAEP, 46-47, 48, 55, 84, 85, 109, 110, 146-147, 151, 158, 187, 189, 194-196
Newspapers, see Mass media
Non-native English speakers, see English-language learners
Nonresponse rates, 64, 80O
Office of Bilingual Education and Minority Languages, 91
Office of Civil Rights, 91
Office of Education Research and Improvement, 46, 91, 94
Office of Special Education and Rehabilitative Services, 91
Office of the General Counsel, 91
Open-ended items, 22, 146
see also Extended-response itemsP
Participation, students with disabilities/English-language learners , 5, 19, 57, 87-89, 91-113 (passim)
accommodations/adaptations, 51, 88-89, 91-112 (passim), 191
Goals 2000: Educate America Act, 10, 88, 96
Pedagogy
coordinated system of education indicators, 23, 40-43 (passim), 46, 47-49, 54
national NAEP, 16, 58
teacher background, 40, 41, 43
trend NAEP, 77
video techniques, instructional methods analysis, 43, 46, 48, 194
Performance standards, 3, 17, 19, 20, 23, 44, 48, 162-184, 196
advanced level, 2, 7, 64, 147, 152, 163, 164-171, 173, 174, 175, 180, 183, 186, 256, 259, 260
defined, 162-164
demographic factors, 27, 163
frameworks and assessment development process, 114, 115, 118, 123, 125, 133, 142-143, 146, 149, 155, 159, 220, 257, 259-260
history of, 15
main NAEP, 4-5, 15-16, 77, 78-80
reporting of, 2, 6, 7, 25-31, 32, 33, 72, 73, 162, 163-165
students with disabilities/English-language learners, 87, 89, 94
see also Achievement levels;
Constructed-response items;
Cutscores;
Extended-response items;
Task measures
OCR for page 277
GRADING THE NATION'S REPORT CARD: Evaluating NAEP and Transforming the Assessment of Educational Progress
Perkins Act, 88
Policy Statement on Redesigning the National Assessment of Educational Progress, 11
Political factors, 9-11, 13-15, 19, 20, 23, 25-26, 31, 38-39
stakeholders, 3, 25, 80
see also Accountability;
Education reform
Poverty, 14, 44
Preliminary achievement-level descriptions, 119, 134-135, 165, 170, 176-179, 182, 183, 256, 257, 259, 260;
see also Achievement-level descriptions
Press, see Mass media
Private School Universe Study, 36-37
Problem solving, 47, 57, 121, 127, 136-137, 138, 140, 146Q
Quality Counts, 172R
Race/ethnicity, 14, 50, 79, 169
African Americans, 44, 62
bilingual education, 91, 98-101 (passim), 105, 110
cultural bias, 63
Hispanics, 44, 62, 95, 99
translated versions of NAEP, 95, 97, 100, 101, 111
trend NAEP, 44, 79
RAND, 40, 46
Reading achievement, 7, 65, 166
frameworks and assessment development process, 118, 119, 120, 122, 123, 127-128, 132, 133, 136, 141-142, 219-230
items read orally due to poor readers, 13, 95, 99, 105
main NAEP, 15, 17, 65, 80, 84, 118, 120, 125, 127-128, 133, 171, 192-193, 219
NAEP administration schedule, 18
NAEP mandate, 24
national NAEP, 16, 18, 59, 73-75, 77
new paradigm NAEP, 84
oral, 16
reporting, 31-32
state-level trends, 26, 59
students with disabilities/English-language learners, 94, 110
text analysis, 43, 49, 219-220
time factors, reading passages, 122
trend NAEP, 15, 17, 18, 59, 63, 73-75, 77, 80, 84
see also Achievement levels;
Core NAEP
Reform of education, see Education reform
Reform Up Close, 40
Regional factors and trends
English-language learners, 89-90
NAEP results reporting, 1, 14, 19, 152
Report Cards, 8, 65, 71, 72, 73, 107, 121, 152, 154, 163, 170, 172
Reporting metrics, 72, 84, 86, 159, 162, 164, 171, 189
Reporting procedures, 1-2, 8, 20, 25-26, 64-65, 66, 67-68, 70-73, 86, 196
achievement levels, 2, 6, 7, 25-31, 32, 33, 72, 73, 162, 163-165
coordinated system of education indicators, 25-26, 73
costs, 78
demographic factors, 1-2, 27, 31, 34, 163
field tests, tables/displays, 71
frameworks and assessment development process, 114, 117, 121, 128, 135, 145-146, 151, 154-160 (passim), 221
governance, 19, 31, 72, 164, 172, 190
market-basket reporting, 72, 189
mathematics achievement, 28, 58, 59, 73, 171, 172
national NAEP, 1-2, 14, 71
NCES role, 31, 154, 196
performance standards, 2, 6, 7, 25-31, 32, 33, 72, 73, 162, 163-165
reading achievement, 31-32
regional trends, 1, 14, 19, 152
scoring and scoring criteria, 72, see also Summary scores
standards-based, 2, 7, 25-26, 33, 162, 163-165, 167-168, 171-173, 182, 184
students with disabilities/English-language learners, 108, 111
state NAEP, 2, 10, 14, 15, 26, 31, 68, 71
timeliness, 67, 71, 152, 154
trend NAEP, 73, 163
see also Descriptive information;
Evaluative information;
Interpretive information
OCR for page 278
GRADING THE NATION'S REPORT CARD: Evaluating NAEP and Transforming the Assessment of Educational Progress
Rural areas, see Urban/rural factorsS
Sampling, 2, 4, 11, 13, 14, 17, 20, 45, 46, 47, 49, 50-51, 56, 57, 58-63, 64, 66, 68, 86, 161
coordinated system of education indicators, 45, 46, 47, 49, 50-51, 107
cost factors, 62, 66
dropouts, 81, 86
errors, 82-83
probability sampling, 62, 64, 66, 68-69, 175
jackknife methods, 64, 69
matrix sampling, 13, 62, 68
balanced incomplete blocks, 62, 63, 66
national NAEP, 59-60, 62, 66
national trends, other, 59-63, 68, 194
state-level, 59-62, 65-66, 68, 83, 85, 194
state vs national NAEP, 83, 85
students with disabilities/English-language learners, 101, 102, 103, 106, 107
time factors, 69
see also Demographic factors;
Error of measurement;
Race/ethnicity
Scaling, 4, 56, 66, 67, 72, 80, 101, 111, 117, 154, 163, 261
Scholastic Aptitude Test, 50, 93
School districts, 10, 82, 104
School-level measures, 27, 34, 39, 40, 41, 42, 45, 46, 47, 51
English-language learners, 89-90
Schools and Staffing Survey, 36-37, 41, 51
Science achievement, 7, 10, 27, 60, 65, 78, 80, 259
database, 50
frameworks and assessment development process, 116, 118, 126, 129-130, 132-133, 134, 136, 142, 143-144, 231-236
main NAEP, 15, 17, 28-30, 65, 77-78, 80, 84, 116, 118, 121, 125, 126, 157, 168, 192-193, 259
NAEP administration schedule, 18
national NAEP, 18, 60, 63
new paradigm NAEP, 84
standards, 10, 166, 168-171, 178-179
TIMSS, 36-37, 40, 42-44, 46, 48, 110, 169, 174, 188, 259
trend NAEP, 15, 17, 18, 60, 63, 80, 84, 259
see also Achievement levels;
Core NAEP
Scoring and scoring criteria, 67, 192
frameworks and assessment development process, 114, 117, 118, 119, 125, 132, 133-134, 135, 136, 146, 149, 150, 153-154, 156, 160, 161, 219-220, 226-227, 234-235, 257
levels and types of knowledge, 6
multicategory, 67
nonresponse rates, 64, 80
reporting, 72
state vs national NAEP, 82-83
see also Cutscores;
Summary scores
Second-language speakers, see English-language learners
Sex differences, see Gender factors
Social indicator systems, 3, 24, 38-39, 152, 153
Socioeconomic status, 50
poverty, 14, 44
Sociopolitical factors, see Demographic factors;
Political factors
Spanish-speaking students, see Hispanic students
Special studies, 16, 57, 110
Special Study Panel on Education Indicators, 35, 37, 38, 40-41
Standards, general, 19, 20, 27, 126
coordinated system of education indicators, 23, 45
curriculum, 10, 179, 153
mathematics achievement, 58, 59, 73
NAEP history, 12, 15
reporting, standards-based, 2, 7, 25-26, 33, 162, 163-165, 167-168, 171-173, 182, 184
science achievement, 10, 166, 168-171, 178-179
students with disabilities/English-language learners, 5, 87, 89, 91, 92, 94, 98-102, 109, 111-113
see also Achievement-level descriptions;
Achievement levels;
Achievement-level
setting;
Performance standards;
Preliminary achievement-level descriptions;
Scoring and scoring criteria
State Collaboratives on Assessment and Standards Project, 40, 46
State government, 19
costs, 78
curriculum reform, 13, 153
evaluation of programs, 11
NAGB and, 25, 65-66
OCR for page 279
GRADING THE NATION'S REPORT CARD: Evaluating NAEP and Transforming the Assessment of Educational Progress
national curriculum, 13
National Governors' Association, 15, 172
standards, 10, 56
students with disabilities/English-language learners, 88, 93, 94, 97, 98, 104, 106, 109, 110
State-level factors and trends, 4, 5, 23, 26, 189
age-vs grade-level testing, 14
coordinated system of education indicators, 45-46
main NAEP, 15, 16, 56, 57, 58, 65-66
reading achievement, 26, 59
sampling, 59-62, 65-66, 68, 83, 85, 194
sociopolitical factors, 10
State NAEP, 14, 15, 16, 45-46, 56, 57, 58, 65-66, 192-193
cost factors, 62, 66, 78, 82, 85
national NAEP vs, 65, 81-84, 85
objectives, 24
reporting, 2, 10, 14, 15, 26, 31, 68, 71
sampling, 59-62, 65-66, 68, 83, 85, 194
streamlining, 81-84
Student-level measures, 40, 42, 43, 45, 46
Students with disabilities, 5, 7-8, 14, 19, 20, 87-113, 134, 191
accommodations/adaptations, 51, 88-89, 91-112 (passim), 191
accountability, 91, 94
achievement levels, 88, 107, 109, 110-111, 112
administration procedures, testing, 91, 92, 99-100
analysis methods, 101, 104, 108, 110
coordinated system of education indicators, 106, 107
core NAEP, 109, 110, 112
cost factors, 8, 107-108, 111
curriculum, 111
data collection, 87, 96, 104, 106, 107, 108, 112, 113
demography of, 89-90
Department of Education role, 106, 109
NCES, 91, 93, 94, 97, 105, 106, 109, 191-192
educational indicators, general, 88, 106, 114
funding, 108, 109
Goals 2000: Educate America Act, 10, 88, 96
historical perspectives, 5, 92-98
identification/classification, 102, 104-105, 106
inclusion, 45, 92, 93, 95, 97-104, 106, 109, 111-113, 191
individualized education plans, 88
interpretive information, 5, 87, 99, 106, 112
large-scale assessments, 88, 89, 91-93, 96, 97, 98, 104, 106-109, 110, 112-113
legislation, 10, 88, 106, 108
NCES role, 91, 93, 94, 97, 105, 106, 109, 191-192
participation, 5, 19, 57, 87-88
performance standards, 87, 89, 94
reading achievement, 94, 110
reporting procedures, 108, 111
sampling, 101, 102, 103, 106, 107
standards, 5, 87, 89, 91, 92, 94, 98-102, 109, 111-113
validity issues, 88, 91, 95, 106, 109-111
see also Learning disabilities
Students with limited English proficiency, see English-language learners
Student/teacher ratios, 44
Summary scores, 1-2, 6
demographic trends, 14
error of measurement, 63
frameworks and assessment development process, 116, 117, 121, 128, 139, 152, 154T
Task measures, 4, 12, 192
coordinated system of education indicators, 23, 24, 33, 47, 54
framework and assessment development process, 119, 128, 132, 136, 139, 142-144, 146, 149, 153-154, 160, 219-220
group, 47, 138
streamlining NAEP design, 67, 68, 71, 81, 86
see also Extended-response items
Teacher background variables, 40, 41, 43
Teacher evaluation of NAEP, 77-78, 129-130
Teaching methods, see Pedagogy
Technical Review Panel, 11, 166
Test administration procedures, see Administration procedures, testing
Text analysis, 219-220
textbooks, 43, 49
Third International Mathematics and Science Study, 36-37, 40, 42-44, 46, 48, 110, 169, 174, 188, 259
OCR for page 280
GRADING THE NATION'S REPORT CARD: Evaluating NAEP and Transforming the Assessment of Educational Progress
Time factors
administration schedules, 18, 121, 191, 192-193, 195
assessment development processes, 124, 136, 159, 160
item response time, 13, 62, 99, 128, 130-131
reading passages, 122
reporting of results, 67, 71, 152, 154
sampling, 69
Trend NAEP, 4, 27, 56, 58, 65, 84, 85
achievement levels, 4-5, 15-16, 56, 58, 77, 78-80
cost factors, 78
frameworks, 77-78, 192, 194
main vs, 4-5, 15-16, 57, 58, 63, 64, 65, 73, 77, 78-80, 84, 85
mathematics achievement, 15, 17, 18, 60, 63, 73, 75, 77, 80, 84, 102
new paradigm, 47, 194
race, 44, 79
reading achievement, 15, 17, 18, 59, 63, 73-75, 77, 80, 84
reporting, 73, 163
sampling, 66
science achievement, 15, 17, 18, 60, 63, 80, 84, 259
writing achievement, 15, 17, 18, 59, 63, 80, 84
Trends in Academic Progress, 65
Trial State Assessment, 25, 94, 96-98 (passim), 174U
Urban/rural factors
English-language learners, 89-90
Use of NAEP results, 11, 20, 25-31, 39, 114, 181, 256
achievement levels, 2, 31, 171, 176, 180, 182, 184, 246
mass media, 9-10, 12, 25-30, 68, 71, 171
school districts, 104
stakeholders, 3, 25, 80V
Validating National Curriculum Indicators, 40, 46
Validity issues, 11, 122, 153, 167-168, 173, 176-177, 182, 189, 221, 226
analysis and scaling models, 67
content, 12
coordinated system of education indicators, 40, 46
students with disabilities/English-language learners, 88, 91, 95, 106, 109-111
Validity Studies Panel, 11
Video techniques, instructional methods analysis, 43, 46, 48, 194W
WESTAT, 17, 119
Writing achievement, 7, 59, 65
main NAEP, 15, 17, 65, 80, 84, 192-193
NAEP administration schedule, 18
NAEP mandate, 24
national NAEP, 16, 18, 59
new paradigm NAEP, 84 extended performance, 16
trend NAEP, 15, 17, 18, 59, 63, 80, 84
trend vs main NAEP, 79, 80
see also Core NAEP
Representative terms from entire chapter:
assessment development
