Appendix A


Agenda and Participants for
the January Workshop

 

Assessment of 21st Century Skills
Workshop
January 12–13, 2011

University of California, Irvine
Beckman Conference Center
Huntington Room

AGENDA

Wednesday, January 12

 

9:30-9:40

Opening Remarks Welcome

 

Stuart Elliott (Director, Board on Testing and Assessment)

 

Bruce Fuchs (National Institutes of Health, cosponsor of the project)

   
  Overview of Workshop
  Joan Herman (CRESST, Chair of Workshop Steering Committee)
9:40-12:15 Session 1: Background Information
  Moderators: Joan Herman and Pat Kyllonen (ETS and
  Workshop Steering Committee)
   
  (9:40-10:00) Why Are 21st Century Skills Important?
  Richard Murnane (Harvard University)


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 127
Appendix A Agenda and Participants for the January Workshop Assessment of 21st Century Skills Workshop January 12–13, 2011 University of California, Irvine Beckman Conference Center Huntington Room AGENDA Wednesday, January 12 9:30-9:40 Opening Remarks Welcome Stuart Elliott (Director, Board on Testing and Assessment) Bruce Fuchs (National Institutes of Health, cosponsor of the project) Overview of Workshop Joan Herman (CRESST, Chair of Workshop Steering Committee) Session 1: Background Information 9:40-12:15 Moderators: Joan Herman and Pat Kyllonen (ETS and Workshop Steering Committee) (9:40-10:00) Why Are 21st Century Skills Important? Richard Murnane (Harvard University) 127

OCR for page 127
128 ASSESSING 21ST CENTURY SKILLS This presentation will address the following issues: • hat is unique in the 21st century that makes these W skills especially valuable in the labor market and/ or in other life domains (learning, family life, civic engagement)? • ow does the growing use of computers and H technology affect the labor market and the demand for 21st century skills? • hat does more recent research suggest about the skills W needed to be successful in the 21st century? (10:00-10:15) How Will You Know If Your Students Are 21st Century Ready? Deborah Boisvert (Boston Area Advanced Technical Education) The presenter will respond to the opening presentation, reflecting her work with employers to define, teach, and assess 21st century skills of computer technicians. (10:15-10:45) The Teaching and Learning of 21st Century Skills Eric Anderman (Ohio State University) This presentation will address the following issues: • hat is known about the extent to which the three skill W clusters and/or the skills within them can be taught and learned? • o what extent are learning, teaching, and assessment T of the three skill clusters domain specific or domain general? (10:45-11:00) Discussion Moderators will lead a question-and-answer session with the presenters and audience members. 11:00-11:15 Break

OCR for page 127
129 APPENDIX A (11:15-11:35) Approaches to Developing Assessments of 21st Century Skills Deirdre Knapp (HumRRO, Workshop Steering Committee) This presentation will address the following issues: • hat are the different approaches to assessment of W these skills and what steps are involved in carrying out these approaches? • hat processes are used for identifying the skills to be W measured, operationalizing the skills through the test blueprint, and creating assessment tasks and scoring procedures? • ow should the intended uses of the assessment results H guide the test development process? • hat steps should be taken to ensure that the W assessments are reliable and valid? (11:35-11:55) Unique Challenges and Opportunities in the Assessment of 21st Century Skills Steven Wise (Northwest Evaluation Association, Workshop Steering Committee) This presentation will address the following issues: • hat are the unique challenges and opportunities W for defining and measuring these constructs, when compared to more traditional academic skills and knowledge? • ow might the results of these assessments be used? H Should they be used for high-stakes purposes? • hat issues may arise in relation to the validity, W reliability, and fairness of assessments of these skills? (11:55-12:15) Questions and Discussion Moderators will lead a question-and-answer session with the presenters and audience members. 12:15-1:15 Lunch in the Beckman Center Cafeteria Continued discussion of ideas presented during the morning sessions

OCR for page 127
130 ASSESSING 21ST CENTURY SKILLS 1:15-3:45 Session 2: Assessing Cognitive Skills Moderators: Greg Duncan (University of California, Irvine, Workshop Steering Committee) and Paul Sackett (University of Minnesota, Workshop Steering Committee) (1:15-1:45) Defining and Measuring Cognitive Skills Nathan Kuncel (University of Minnesota) This presentation will address the following issues: • hat are 21st century cognitive skills? To what extent W do they differ from each other and from general cognitive ability? What are the conceptual differences that are proposed to exist between these constructs? • hat are the existing measures of these constructs, and W to what extent do these existing measures match their conceptual specifications? • hat are the relationships between the existing W measures of these constructs? (1:45-2:00) Questions and Discussion Moderators will lead a question-and-answer session with the presenters and audience members. (2:00-3:15) Panel Discussion: Examples of Assessments of Cognitive Skills For each example, the panelists will address the following issues: • hat skill or skills are measured? Why are these skills W important? • hat is the purpose of the assessment? W • hat strategies were used to develop the assessment W and why were these selected? • hat assessment methods are used and why were these W selected? • ow is the assessment scored? What data are available H on the technical quality of the assessment, including validity, reliability, fairness, and comparability across administrations? • hat data are available on the cost and practical W feasibility of the assessment?

OCR for page 127
131 APPENDIX A (2:00-2:20) Interactive Problem Solving for PISA 2012 Joachim Funke (University of Heidelberg), by video conference (2:20-2:40) Operation ARIES!: Learning Critical Thinking about Science with Intelligent Conversational Agents in a Game Environment Art Graesser (University of Memphis) and Heather Butler (Claremont McKenna College) (2:40-3:00) Intrusive and Unobtrusive Assessment of Entrepreneurial and Technical Skills through Simulation and Gaming John Behrens (Cisco Systems) (3:00-3:20) Assessment of Critical Thinking and Problem Solving on the Multistate Bar Exam Susan Case (National Conference of Bar Examiners) 3:20-3:30 Break (3:30-4:00) Moderated Discussion Moderators will explore the following issues with panelists and audience members: • hat are the implications of the presentations (and W examples) for the design of 21st century assessments for K-12 and higher education? • o common themes or approaches emerge from the D examples? How might the noneducation examples generalize to education? • ow might 21st century assessments be incorporated H into current research efforts, such as the development of assessment systems by the two-state consortia? What functions can/should the assessments serve? How might the results be used? • hat equity and accessibility challenges do these W assessments raise? • hat barriers might slow development and/or use of W assessments of 21st century skills? How might they be overcome?

OCR for page 127
132 ASSESSING 21ST CENTURY SKILLS 4:00-5:00 Synthesis of Key Ideas Moderator: Joan Herman (4:00-4:20) Discussion: Eva Baker (CRESST) (4:20-4:40) Discussion: Richard Murnane (Harvard University) Discussants will reflect on the day’s discussions and offer their synthesis of the ideas presented. Audience members will be invited to ask questions and share their ideas as well. 5:00 Conclude Formal Agenda for Day 1 Joan Herman 5:30 Working Group Dinner (at Beckman Center) Plan for the second day of the workshop Thursday, January 13 9:00-11:45 Session 3: Assessing Interpersonal Skills Moderators: Deirdre Knapp and Juan Sanchez (Florida International University, Workshop Steering Committee) (9:00-9:30) Defining and Measuring Interpersonal Skills Steve Fiore (University of Central Florida) This presentation will address the following issues: • hat are 21st century interpersonal skills and why are W they important? • ow are these skills typically assessed? What are the H challenges in assessing them? • hat types of assessments are currently available to W evaluate these skills? (9:30-9:40) Questions and Discussion Moderators will lead a question-and-answer session with the presenters and audience members.

OCR for page 127
133 APPENDIX A (9:40-11:00) Panel Discussion: Examples of Assessments of Interpersonal Skills For each example, the panelists will address the following issues: • hat skill or skills are measured? Why are these skills W important? • hat is the purpose of the assessment? W • hat strategies were used to develop the assessment W and why were they selected? • hat assessment methods are used and why were these W selected? • ow is the assessment scored? What data are available H on the technical quality of the assessment, including validity, reliability, fairness, and comparability across administrations? • hat data are available on the cost and practical W feasibility of the assessment? (9:40-10:00) Online Portfolio Assessments of the 4 Cs Bob Lenz (Envision Schools) (10:00-10:20) 21st Century Skills in STEM Workforce Training Assessments Louise Yarnall (SRI) (10:20-10:40) Using Situational Judgment Tests for Medical School Admissions Filip Lievens (Ghent University, Belgium), by video conference (10:40-11:00) Assessment Centers 2011: Fifty Years of Best Practice and Today’s Innovations Lynn Gracin Collins (SH&A/Fenestra) 11:00-11:10 Break

OCR for page 127
134 ASSESSING 21ST CENTURY SKILLS (11:10-11:45) Moderated Discussion Moderators will explore the following issues with panelists and audience members: • hat are the implications of the presentations (and W examples) for the design of 21st century assessments for K-12 and higher education? • o common themes or approaches emerge from the D examples? How might the noneducation examples generalize to education? • ow might 21st century assessments be incorporated H into current research efforts, such as the development of assessment systems by the two-state consortia)? What functions can/should the assessments serve? How might the results be used? • hat equity and accessibility challenges do these W assessments raise? • hat barriers might slow development and/or use of W assessments of 21st century skills? How might they be overcome? 11:45-12:45 Working Lunch in the Beckman Center Cafeteria Continued discussion of ideas presented during the morning sessions 12:45-3:30 Session 4: Assessing Intrapersonal Skills Moderators: Pat Kyllonen and Steven Wise (12:45-1:15) Assessment of Self-Regulation and Related Constructs: Prospects and Challenges Rick Hoyle (Duke University) This presentation will address the following issues: • hat are 21st century intrapersonal skills and why are W they important? • ow are these skills typically assessed? What are the H challenges in assessing them? • hat types of assessments are currently available to W evaluate these skills? (1:15-1:30) Discussion

OCR for page 127
135 APPENDIX A (1:30-3:30) Panel Discussion: Examples of Assessments of Intrapersonal Skills For each example, the panelists will address the following issues: • hat skill or skills are measured? Why are these skills W important? • hat is the purpose of the assessment? W • hat strategies were used to develop the assessment W and why were these selected? • hat assessment methods are used and why were these W selected? • ow is the assessment scored? What data are available H on the technical quality of the assessment, including validity, reliability, fairness, and comparability across administrations? • hat data are available on the cost and practical W feasibility of the assessment? (1:30-1:50) Integrity Testing for Employee Selection Paul Sackett (University of Minnesota, Workshop Steering Committee) (1:50-2:10) Targeting Context-Specific Self-Regulated Learning (SRL) Processes: An Overview and Illustration of SRL Microanalysis Tim Cleary (University of Wisconsin–Milwaukee) (2:10-2:30) Assessing Behavioral Problems That Predict Poor Educational and Life Outcomes Candice Odgers (University of California, Irvine) (2:30-2:50) Out of the Maze? In Search of Skills for Emotional Intelligence Gerald Matthews (University of Cincinnati) 2:50-3:00 Break

OCR for page 127
136 ASSESSING 21ST CENTURY SKILLS (3:00-3:30) Moderated Discussion Moderators will explore the following issues with panelists and audience members: • hat are the implications of the presentations (and W examples) for the design of 21st century assessments for K-12 and higher education? • o common themes or approaches emerge from the D examples? How might the noneducation examples generalize to education? • ow might 21st century assessments be incorporated H into current research efforts, such as the development of assessment systems by the two-state consortia? What functions can/should the assessments serve? How might the results be used? • hat equity and accessibility challenges do these W assessments raise? • hat barriers might slow development and/or use of W assessments of 21st century skills? How might they be overcome? 3:30-4:00 Session 5: Reflection and Synthesis Moderated discussion led by workshop steering committee 4:00 Closing Remarks, Adjourn Joan Herman PARTICIPANTS Eric Anderman, Ohio State University John Behrens, Cisco Systems Lola Berber-Jimenez, California Polytechnic Science Project Paul Bloomberg, Transformative Inquiry Design for Effective Schools and Systems Deborah Boisvert, University of Massachusetts, Boston Liane Brouillette, University of California, Irvine Christopher Brown, Pearson Foundation Peggy Burke, Transformative Inquiry Design for Effective Schools and Systems Heather Butler, Claremont McKenna College

OCR for page 127
137 APPENDIX A Susan Case, National Conference of Bar Examiners Tim Cleary, University of Wisconsin–Milwaukee Sara Clough, ACT, Inc. Lynn Gracin Collins, Sandra Hartog & Associates/Fenestra, Inc. Emily Dalton Smith, Gates Foundation Tran Dang, University of California, Irvine Greg J. Duncan, University of California, Irvine Steve Fiore, University of Central Florida Dennis Frezzo, Cisco Systems Bruce Fuchs, National Institutes of Health Joachim Funke, University of Heidelberg Tracy Gardner, General Educational Development Testing Service Nicole Gerardi, University of California, Los Angeles Art Graesser, University of Memphis Valerie Greenhill, e-luminate Erika Hall, Pearson Foundation Joan Herman, National Center for Research on Evaluation, Standards, and Student Testing, University of California, Los Angeles (committee chair) Rick Hoyle, Duke University John Jackson, National Science Foundation Stuart Kahl, Measured Progress Deirdre J. Knapp, HumRRO (committee member) Art Kramer, University of Illinois Brandi Kujala, Educational Policy Improvement Center Nathan Kuncel, University of Minnesota Patrick Kyllonen, Educational Testing Service (committee member) Robert Lenz, Envision Schools Filip Lievens, University of Ghent María Alicia López Freeman, California Science Project Tim Magner, Partnership for 21st Century Skills Michael Martinez, University of California, Irvine Gerald Matthews, University of Cincinnati Mick McManus, University of Queensland Beth Miller, Nellie Mae Education Foundation Julia Rankin Morandi, Los Angeles Education Partnership Richard Murnane, Harvard University (committee member) Suzanne Nakashima, California Science Project Paul Nichols, National Center for the Improvement of Educational Assessment Candice Odgers, University of California, Irvine Cornelia Orr, National Assessment Governing Board

OCR for page 127
138 ASSESSING 21ST CENTURY SKILLS Pamela Paek, National Center for the Improvement of Educational Assessment Jason Ravitz, Buck Institute for Education Michael Russell, University of California, Irvine Paul Sackett, University of Minnesota (committee member) Andrea Saenz, U.S. Department of Education Juan I. Sanchez, Florida International University (committee member) Mary Seburn, Educational Policy Improvement Center Brian Stecher, RAND Christine Tell, Achieve Cathy Tran, University of California, Irvine Bernie Trilling, Oracle Education Foundation Jerry Valadez, California State University, Fresno Marjorie Wine, General Educational Development Testing Service Steven Wise, Northwest Evaluation Association (committee member) Louise Yarnall, SRI Raymond Yeagley, Northwest Evaluation Association Linda Zimmerman, Pearson Doron Zinger, Olive Crest Academy