| ||||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||
| Copyright © 2009. National Academy of Sciences. All rights reserved. Terms of Use and Privacy Statement |
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 203
Measuring Literacy: Performance Levels for Adults
Appendix A
The Committee’s Public Forums on Performance Levels for NAAL
Public Forum—February 27, 2004
Panelists
Cynthia Baur, U.S. Department of Health and Human Services
Beth Beuhlmann, U.S. Chamber of Commerce, Workforce Preparation
Richard Colvin, Hechinger Institute
Leslie Farr, Ohio State University
Milton Goldberg, Education Commission of the States (ECS)
Richard Long, International Reading Association
Christopher Mazzeo, National Governors Association
Gemma Santos, Miami Dade Public Schools
Tony Sarmiento, Senior Service America, Inc.
Linda Taylor, Comprehensive Adult Student Assessment System
Robert Wedgeworth, Proliteracy Worldwide
Participants
Joan Auchter, GED Testing Service/American Council on Education
Justin Baer, American Institutes for Research (AIR)
Amy Baide, Department of Homeland Security
Sandra Baxter, The National Institute for Literacy (NIFL)
Jaleh Behroozi, The National Institute for Literacy (NIFL)
Martha Berlin, Westat
OCR for page 204
Measuring Literacy: Performance Levels for Adults
Peggy Carr, Department of Education, National Center for Education Statistics (NCES)
June Crawford, The National Institute for Literacy (NIFL)
Elizabeth Greenberg, American Institutes for Research (AIR)
Ricardo Hernandez, Office of Vocational and Adult Education (OVAE)
Shannon Holmes, U.S. Conference of Mayors
Eugene Johnson, American Institutes for Research (AIR)
Linda Johnston Lloyd, Health Resources & Services Administration (HRSA)
Michael Jones, Office of Vocational and Adult Education (OVAE)
Cheryl Keenan, Office of Vocational and Adult Education (OVAE)
Irwin Kirsch, Educational Testing Service (ETS)
Andy Kolstad, National Center for Education Statistics (NCES)
Mark Kutner, American Institutes for Research (AIR)
Mariann Lemke, National Center for Education Statistics (NCES)
Anne Lewis, freelance journalist
Lennox McLendon, National Adult Education Professional Development Consortium
Wendy Mettger, Mettger Communications
Leyla Mohadjer, Westat
Gerri Ratliff, Department of Homeland Security
Lyn Schaefer, GED Testing Service
Peggy Seufert, American Institutes for Research (AIR)
Sondra Stein, The National Institute for Literacy (NIFL)
Lynn Thai, Department of Homeland Security
Peter Waite, Proliteracy America
Dan Wagner, National Center on Adult Literacy
Maria White, Department of Health and Human Services
Sheida White, National Center for Education Statistics (NCES)
Kentaro Yamamoto, Educational Testing Service (ETS)
Representatives from State Departments of Adult Education—April 22-23, 2004
Bob Bickerton, Massachusetts
Steve Coffman, Missouri
Donna Cornelius, Massachusetts
Cheryl King, Kentucky
Tom Orvino, New York
Ann Serino, Massachusetts
Reecie Stagnolia, Kentucky
Linda Young, Oklahoma
OCR for page 205
Measuring Literacy: Performance Levels for Adults
Stakeholder Questions and Alternative Versions of Performance-Level Descriptions for the National Assessment of Adult Literacy
In what ways did you use the results from the 1992 NALS? What were the strengths and weaknesses of these performance levels? To what extent did these performance levels provide you with the information that you needed?
NAAL measures skills in the areas of prose, document, and quantitative literacy. To what extent is it useful and informative to have different performance level descriptions for each area? Are results from the three areas of literacy used differently? If so, how?
The attachment presents three alternative versions of performance-level descriptions for the prose literacy scale. Sample 1 is simply a reformatted version of the existing performance-level descriptions with 5 levels. Sample 2 is a 4-level model, and Sample 3 is a 3-level model. Please comment on how many levels are needed. What types of decisions are made at the various levels? What are the critical distinctions that need to be made?
Level Labels: The three samples present different labels for the levels. Sample 1 uses numbers (Col. 2). Samples 2 and 3 use phrases as labels (Col. 2). In addition, Sample 3 presents a narrative description of the label (Col. 3). Please comment on these alternative labels. What types of labels are useful and informative? Feel free to make suggestions for alternative labels.
Level Descriptions: The three samples present different ways of describing the skills represented by the performance level. Sample 1 describes the tasks associated with the level (Col. 3). Sample 2 describes what an average respondent who scores at this level should be able to do (Col. 3). Sample 3 (Col. 4) describes what the average respondent who scores at this level is able to do and not able to do in probabilistic terms (i.e., likely, not likely). Please comment on these alternative ways of describing the skills associated with the levels. What types of descriptions are useful and informative? Feel free to make suggestions for alternative descriptions.
Sample Tasks: The three samples present different ways of exemplifying the tasks respondents who score at the level should be able to do. Samples 1 and 2 (Col. 4) are similar and provide examples drawn from actual assessment. Sample 3 (Col. 5) attempts to generalize from assessment tasks to real world tasks. Please comment on the extent to which these exemplifications are useful and informative.
Relationships Between Prose Scores and Background Data: Samples 2 and 3 present the relationships between NAAL scores and key real-world factors as measured on the background questionnaire. Sample 2 (Col. 5) uses societal factors (income, education, voting) and Sample 3 (Col. 6) uses reading related factors. (Please be aware that the percentages in
OCR for page 206
Measuring Literacy: Performance Levels for Adults
cluded in these examples are purely hypothetical. If we were to recommend this format, the percentages would be based on analyses with NAAL data.) Please comment on the utility of this type of information.
OCR for page 207
Measuring Literacy: Performance Levels for Adults
SAMPLES ARE ON THE FOLLOWING PAGES
OCR for page 208
Measuring Literacy: Performance Levels for Adults
Sample 1: Five-Level Model, Based on Current PLDs for Prose Literacy
(Col. 1)
Level
(Col. 2)
Label
(Col. 3)
Description of tasks
(Col. 4)
Sample NAAL tasks associated with the level
I
Level 1
Most of the tasks in this level require the reader to read relatively short text to locate a single piece of information which is identical to or synonymous with the information given in the question or directive. If plausible but incorrect information is present in the text, it tends not to be located near the correct information.
Locate one piece of information in a sports article.
Identify the country in a reading passage.
Underline sentence explaining action stated in short article.
II
Level 2
Some tasks in this level require readers to locate a single piece of information in the text; however, several distractors or plausible but incorrect pieces of information may be present, or low-level inference may be required. Other tasks require the reader to integrate two or more pieces of information or to compare and contrast easily identifiable information based on a criterion provided in the question or directive.
Underline the meaning of a term given in government brochure on supplemental security income.
Locate two features of information in sports article.
Interpret instructions from an appliance warranty.
III
Level 3
Tasks in this level tend to require readers to make literal or synonymous matches between the text and information given in the task, or to make matches that require low-level inferences. Other tasks ask readers to integrate information from dense or lengthy text that contains no organizational aids
Write a brief letter to complain about an error on a bill.
Read a news article and identify a sentence that provides interpretation of a situation.
Read lengthy article to identify two behaviors that meet a stated condition.
OCR for page 209
Measuring Literacy: Performance Levels for Adults
such as headings. Readers may also be asked to generate a response based on information that can be easily identified in the text. Distracting information is present, but is not located near the correct information.
IV
Level 4
These tasks require readers to perform multiple-feature matches and to integrate or synthesize information from complex or lengthy passages. More complex inferences are needed to perform successfully. Conditional information is frequently present in tasks in this level and must be taken into consideration by the reader.
State in writing an argument made in a newspaper article.
Explain differences between two types of employee benefits.
Contrast views expressed in two editorials on technologies available to make fuel-efficient cars.
V
Level 5
Some tasks in this level require the reader to search for information in dense text which contains a number of plausible distractors. Others ask readers to make high-level inferences or use specialized background knowledge. Some tasks ask readers to contrast complex information.
Compare approaches stated in a narrative on growing up.
Summarize two ways lawyers may challenge prospective jurors.
Interpret a phrase from a lengthy news article.
OCR for page 210
Measuring Literacy: Performance Levels for Adults
Sample 2: Four-Level Model for Prose Literacy
(Col. 1)
Level
(Col. 2)
Label
(Col. 3)
The average respondent who scores at this level should be able to:
(Col. 4)
Sample of NAAL tasks the average respondent should be able to:
(Col. 5)
Relationships with societal factors
I
Below Basic
Identify letters and numbers.
Point to orally presented words.
Read aloud words, phrases. or short sentences.
Locate specific information on a food label.
Identify a specific word in an advertisement.
Locate an address in an advertisement.
>50% of being poor or near poor
>50% chance of not having a U.S. high school diploma
<50% chance of voting in most recent election
II
Basic
Locate a single piece of information in a brief written text that uses organizational aids such as headings.
Make simple inferences.
Provide appropriate answers to questions that require integrating two or more pieces of information.
Locate one piece of information in a sports article.
Identify the country in a reading passage.
Interpret brief instructions from warranty information.
25-50% of being poor or near poor
50% chance of having at least a high school diploma
50-60% chance of voting in most recent election
OCR for page 211
Measuring Literacy: Performance Levels for Adults
III
Intermediate
Generate a response based on information that is easily identifiable in dense or lengthy text.
Integrate information from dense or lengthy text.
Write a brief letter to complain about an error on a bill.
Read a magazine article and identify the facts that support a specific inference.
10-25% of being poor or near poor
50% chance of having at least some college
60-70% chance of voting in recent election
IV
Advanced
Search for information that is in dense texts.
Integrate or synthesize information from complex texts.
Perform complex inferences.
State in writing an argument made in a newspaper article.
Explain the differences between two types of employee benefits.
Contrast views expressed in two editorials.
<10% of being poor or near poor
50% chance of having at least an associates college degree
70-80% chance of voting in recent election
NOTES: NAAL background data will be analyzed to provide information about the likely relationships between literacy levels and the three societal factors included in Column 5; the current figures are for demonstration purposes only. Many of these descriptions may not generalize to English language learners (ELL). ELLs may be literate in languages other than English, and the relationships with societal factors (Col. 5) may be different for ELLs than for native English speakers. The response mode may also affect ELLs’ performance, since writing skills in English may develop at a slower pace than reading or speaking skills.
OCR for page 212
Measuring Literacy: Performance Levels for Adults
Sample 3: Three-Level Model for Prose Literacy
(Col. 1)
Level
(Col. 2)
Label
(Col. 3)
Description of label for individuals who score at this level:
(Col. 4)
The average respondent who scores at this level:
(Col. 5)
Sample real-world tasks the average respondent at this level should be able to do:
(Col. 6)
Relationship to reading-oriented factors
I
Minimally Literate in English
Are not able to independently handle most of the tasks of daily living that require literacy skills in English.
Is likely to be able to identify letters and numbers; point to words when they are presented orally; orally read individual words, phrases, or short sentences.
Is not likely to able to read connected text.
Place signature in proper place on a form.
Follow a simple recipe.
Identify an address in an advertisement.
>80% chance of never reading the newspaper
>80% chance of needing a lot of help with printed information
II
Somewhat Literate in English
Should be able to independently handle some of the tasks of daily living that require literacy skills in English.
Is likely to be able to locate a single piece of information in a brief written text with organizational aids, such as headings; to integrate two or more pieces of information; and to compare and contrast easily identifiable information.
Follow directions on a medicine bottle.
Read aloud to a child in preschool or elementary grade.
40-80% chance of never reading the newspaper
40-80% chance of needing a lot of help with printed information
OCR for page 213
Measuring Literacy: Performance Levels for Adults
Is not likely to be able to integrate or synthesize information from dense text with no organizational aids.
III
Fully Literate in English
Should be able to independently handle most of the tasks of daily living that require literacy skills in English.
Is likely to be able to integrate or synthesize information from dense or lengthy text that contains no organizational aids; and to generate a response based on information that can be identified in the text.
Read and understand New York Times.
Understand a proposition on a ballot.
<10% chance of never reading the newspaper
<10% chance of needing a lot of help with printed information
NOTES: NAAL background data will be analyzed to provide information about the likely relationships between literacy levels and the reading oriented factors in Column 6; the current figures are for demonstration purposes only. These descriptions may not generalize to English language learners (ELL). ELLs may be literate in languages other than English, and their performance on real world tasks (Col. 5 and 6) may be different than for native English speakers. The response mode may also affect ELLs’ performance, since writing skills in English may deve lop at a slower pace than reading or speaking skills.
Representative terms from entire chapter:
performance levels
Items in cart [0]
