Index

A

AAAS. See American Association for the Advancement of Science

Abilities for a technological world, standards related to, 239243

Abilities of technological design, 207209

benchmarks for grades 5–8, 208209

benchmarks for grades 9–12, 209

benchmarks for grades K–4, 207208

Accommodating disabilities, 50

ACT. See American College Testing

Adaptive assessments, computer-based, 162–164

Adult Literacy and Lifeskills (ALL) Survey, 10, 185186

Agricultural and related biotechnologies, 31, 244245

benchmarks for grades 3–5, 244

benchmarks for grades 6–8, 244245

benchmarks for grades 9–12, 245

benchmarks for grades K–2, 244

Agriculture, 218220

benchmarks for grades 3–5, 218219

benchmarks for grades 6–8, 219

benchmarks for grades 9–12, 220

benchmarks for grades K–2, 218

ALL. See Adult Literacy and Lifeskills Survey

American Association for the Advancement of Science (AAAS), 1, 20, 29, 66, 131

American College Testing (ACT), 106, 340

American Federation of Teachers, 24

American Society for Engineering Education (ASEE), 113, 284

Applied Technology Assessment, 341

Applying the design process, 239241

benchmarks for grades 3–5, 240

benchmarks for grades 6–8, 240

benchmarks for grades 9–12, 240241

benchmarks for grades K–2, 239240

APU. See Assessment Performance Unit

Armed Services Vocational Aptitude Battery (ASVAB), 97, 103, 163, 267268

background, 267

committee observations, 268

sample items, 268

scope, 267268

ASEE. See American Society for Engineering Education

Assessing technological literacy, 27, 38–39

benefits, 34

developing a conceptual framework, 5–6

existing instruments, 45

general principles, 6, 175176

matrix for, 5

recommendations by category and target population, 7

Assessing the impact of products and systems, 242243

benchmarks for grades 3–5, 242

benchmarks for grades 6–8, 242

benchmarks for grades 9–12, 242243

benchmarks for grades K–2, 242

Assessment

defined, 64

focus-group-style, 153

opportunities for, 176186

possible purposes for, 49



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 347
Tech Tally: Approaches to Assessing Technological Literacy Index A AAAS. See American Association for the Advancement of Science Abilities for a technological world, standards related to, 239–243 Abilities of technological design, 207–209 benchmarks for grades 5–8, 208–209 benchmarks for grades 9–12, 209 benchmarks for grades K–4, 207–208 Accommodating disabilities, 50 ACT. See American College Testing Adaptive assessments, computer-based, 162–164 Adult Literacy and Lifeskills (ALL) Survey, 10, 185–186 Agricultural and related biotechnologies, 31, 244–245 benchmarks for grades 3–5, 244 benchmarks for grades 6–8, 244–245 benchmarks for grades 9–12, 245 benchmarks for grades K–2, 244 Agriculture, 218–220 benchmarks for grades 3–5, 218–219 benchmarks for grades 6–8, 219 benchmarks for grades 9–12, 220 benchmarks for grades K–2, 218 ALL. See Adult Literacy and Lifeskills Survey American Association for the Advancement of Science (AAAS), 1, 20, 29, 66, 131 American College Testing (ACT), 106, 340 American Federation of Teachers, 24 American Society for Engineering Education (ASEE), 113, 284 Applied Technology Assessment, 341 Applying the design process, 239–241 benchmarks for grades 3–5, 240 benchmarks for grades 6–8, 240 benchmarks for grades 9–12, 240–241 benchmarks for grades K–2, 239–240 APU. See Assessment Performance Unit Armed Services Vocational Aptitude Battery (ASVAB), 97, 103, 163, 267–268 background, 267 committee observations, 268 sample items, 268 scope, 267–268 ASEE. See American Society for Engineering Education Assessing technological literacy, 2–7, 38–39 benefits, 3–4 developing a conceptual framework, 5–6 existing instruments, 4–5 general principles, 6, 175–176 matrix for, 5 recommendations by category and target population, 7 Assessing the impact of products and systems, 242–243 benchmarks for grades 3–5, 242 benchmarks for grades 6–8, 242 benchmarks for grades 9–12, 242–243 benchmarks for grades K–2, 242 Assessment defined, 64 focus-group-style, 153 opportunities for, 176–186 possible purposes for, 49

OCR for page 347
Tech Tally: Approaches to Assessing Technological Literacy Assessment as a design challenge, 41–61 the design process, 41–59 imperfect design, 59 inherent uncertainties, 60 Assessment for broad populations, 146–152 administration and logistics, 149–150 content, 148–149 description and rationale, 146 obstacles to implementation, 150 performance levels, 149 purpose, 147–148 sample assessment items, 150–152 Assessment matrix filling, 115–123 for technological literacy, 5, 53, 118–123 Assessment methods, computer-based, 161–174 Assessment of Performance in Design and Technology, 95, 107, 268–272 background, 268–269 committee observations, 271–272 sample items, 270–271 scope, 269–270 Assessment Performance Unit (APU), 269 Assessment primer, 63–92 cognition, 72–80 research on technological learning, 80–86 testing and measurement, 64–72 Assessments for visitors to museums and other informal learning institutions, 153–157 administration and logistics, 155–156 content, 155 description and rationale, 153 obstacles to implementation, 156–157 performance levels, 155 purpose, 153–155 sample test items, 157 ASVAB. See Armed Services Vocational Aptitude Battery Attentive publics, 147 Attitudes, and the assessment of technological literacy, 36 Attitudes toward technology, 36–37, 113–115 Pupils Attitudes Toward Technology, 115 survey of teachers’ attitudes toward engineering, 114 views on science, technology, and society, 115 Awareness Survey on Genetically Modified Foods, 97, 272–274 background, 272 committee observations, 274 sample items, 273 scope, 272–273 B Being Fluent with Information Technology, 35–36 Benchmarks for science literacy, 212–228 standards related to “the designed world,” 218–228 standards related to “the nature of technology,” 212–218 Benchmarks for Science Literacy, 29, 131, 137, 142 Benefits, of assessing technological literacy, 22–24 Bias, avoiding, 50 Bloch, Erich, 19 Bookmark standards-setting, 72 Boston Museum of Science, 114 C Capabilities, of a technologically literate person, 34 Capability dimension, 105–111 Challenges and opportunities for assessing technological literacy in the United States (workshop agenda), 251–254 Children, learning by, 81–82 Cognition, 72–80 conceptual change, 78–80 knowledge transfer, 75–77 metacognition, 77–78 nature of expertise, 73–75 Cognitive dimensions of assessment frameworks, 55 Cognitive research, 75 Cognitive science, insights about how people learn, 49–50 Committee on Assessing Technological Literacy, 2, 25 charge to, 25–26 Communication, 224–225 benchmarks for grades 3–5, 224 benchmarks for grades 6–8, 224 benchmarks for grades 9–12, 225 benchmarks for grades K–2, 224

OCR for page 347
Tech Tally: Approaches to Assessing Technological Literacy Computer-based assessment methods, 161–174 computer-based adaptive assessments, 162–164 computer-based and web-based games, 168–169 electronic portfolios, 170–171 electronic questionnaires, 171–172 simulations, 164–168 Computer-based games, 168–169 Computer literacy, 169 Concept inventories, 85 Conceptual change, 78–80 Conceptual ecology, 79 Conceptual framework for identifying design criteria, 51–56 assessment matrix for technological literacy, 53 cognitive dimensions of assessment frameworks, 55 Conceptual frameworks, 51–56 developing, 5–6 Conceptual understanding, 84 Consensus, 272–273 Constraints, identifying, 45–48 Constructed-response formats, 70 Construction technologies, 31, 250 benchmarks for grades 3–5, 250 benchmarks for grades 6–8, 250 benchmarks for grades 9–12, 250 benchmarks for grades K–2, 250 Content knowledge, 100 Content standards, reflecting appropriate, 50 Core concepts of technology, 230–231 benchmarks for grades 3–5, 230 benchmarks for grades 6–8, 230–231 benchmarks for grades 9–12, 231 benchmarks for grades K–2, 230 Criterion-referenced interpretations, 72 Critical thinking, 112 and the decision-making dimension, 111–112 by a technologically literate person, 34 Cultural, social, economic, and political effects of technology, 232–233 benchmarks for grades 3–5, 232 benchmarks for grades 6–8, 233 benchmarks for grades 9–12, 233 benchmarks for grades K–2, 232 Currency, reviewing items for, 50 D Data from the NAEP long-term science assessment, 101 DBS. See Design-Based Science Decision making, 112 by a technologically literate person, 34 Defining technological literacy, 29–40 assessing technological literacy, 38–39 attitudes and the assessment of technological literacy, 36 attitudes toward technology, 36–37 the designed world, 30–31 dimensions of technological literacy, 37–38 Design and systems, 214–215 benchmarks for grades 3–5, 214 benchmarks for grades 6–8, 214–215 benchmarks for grades 9–12, 215 benchmarks for grades K–2, 214 Design attributes, 236–237 benchmarks for grades 3–5, 237 benchmarks for grades 6–8, 237 benchmarks for grades 9–12, 237 benchmarks for grades K–2, 236 Design-Based Science (DBS), 95, 274–277 background, 274 committee observations, 276–277 sample items, 275–276 scope, 275 Design criteria, 48–56 avoiding bias and accommodating disabilities, 50 conceptual framework, 51–56 encouraging higher-order thinking, 50, 280, 335 gathering data useful to the purpose, 49 general criteria, 48–50 identifying, 48–56 meeting a specific purpose, 48–49 possible purposes for assessments, 49 reflecting appropriate content standards, 50 reviewing items for currency, 50 using insights from cognitive science about how people learn, 49–50 Design process, 41–59 applying, 239–241 interative, 43 linear steps in, 43–59 standards related to, 236–239 and technology, 271

OCR for page 347
Tech Tally: Approaches to Assessing Technological Literacy Design Team Assessments for Engineering Students, 95, 277–280 background, 277 committee observations, 280 sample items, 278–280 scope, 277–278 Design Team Knowledge Assessment, 104 Design Technology, 95, 111, 280–284 background, 280–281 committee observations, 283–284 sample items, 282–283 scope, 281–282 “The designed world,” 30–31 agriculture and related biotechnologies, 31 construction technologies, 31 energy and power technologies, 31 information and communication technologies, 31 manufacturing technologies, 31 medical technologies, 31 standards related to, 218–228 taxonomy for, 31 technology through the lens of Hurricane Katrina, 32 transportation technologies, 31 Dewey, John, 33n DIAG tools, 166 Digital “immigrants,” 169 Disabilities, accommodating, 50 E ECD. See Evidence-centered design Educational Testing Service (ETS), 104, 163, 179, 182, 185, 296, 320 Effective formats, 70 Electronic portfolios, 170–171 Electronic questionnaires, 171–172 Energy and power technologies, 31, 245–246 benchmarks for grades 3–5, 245 benchmarks for grades 6–8, 245–246 benchmarks for grades 9–12, 246 benchmarks for grades K–2, 245 Energy sources and use, 222–223 benchmarks for grades 3–5, 222 benchmarks for grades 6–8, 222–223 benchmarks for grades 9–12, 223 benchmarks for grades K–2, 222 Engineering design, 237–238 benchmarks for grades 3–5, 238 benchmarks for grades 6–8, 238 benchmarks for grades 9–12, 238 benchmarks for grades K–2, 237 Engineering design process, 277–278 Engineering K–12 Center Teacher Survey, 94, 97, 284–286 background, 284 committee observations, 286 sample items, 285 scope, 284–285 Engineering Our Digital Future, 304 Engineers Week Committee, 291 Environmental effects of technology, 233–234 benchmarks for grades 3–5, 233 benchmarks for grades 6–8, 234 benchmarks for grades 9–12, 234 benchmarks for grades K–2, 233 Errors of measurement, defined, 65 ETS. See Educational Testing Service Eurobarometer—Europeans, science, and technology, 97, 113, 286–288 background, 286 committee observations, 288 sample items, 287–288 scope, 287 European Commission Candidate Countries Eurobarometer—science and technology, 97, 288–290 background, 288–289 committee observations, 290 sample items, 289–290 scope, 289 European Union, 113, 286, 289 Evidence-centered design (ECD), 167 Existing instruments, 4–5 mapping to the dimensions of technological literacy, 98–112 Expertise, 73–75 F Feedback, 59 Filling the assessment matrix, 115–123 Findings and recommendations, 8–15, 175–196 by category and target population, 7 conclusion, 193–195 definition of technology, 192–193 framework development, 190–191 innovative measurement techniques, 188–190

OCR for page 347
Tech Tally: Approaches to Assessing Technological Literacy for K–12 students, 8–9, 95–96, 178–181 for K–12 teachers, 9, 97, 181–184 opportunities for assessment, 176–186 for out-of-school adults, 10, 97, 184–186 research on learning, 186–188 summary table, 194 Force Concept Inventory, 85 Fortus, David, 274–275 Framework development, 13–14, 190–191 Future City Competition—Judges Manual, 95, 111, 291–293 background, 291 committee observations, 292–293 sample items, 292 scope, 291–292 G Gallup Poll on What Americans Think About Technology, 14, 29, 97, 105, 148, 184, 288, 293–295 background, 293 committee observations, 295 sample items, 294–295 scope, 294 General public, 148, 152 General Social Survey (GSS), 10, 185–186 Goldsmiths College, 171 GSS. See General Social Survey H Health technology, 227–228 benchmarks for grades 3–5, 227 benchmarks for grades 6–8, 227 benchmarks for grades 9–12, 227–228 benchmarks for grades K–2, 227 Higher-order thinking, 50, 280, 335 complex, 50 involving nuanced judgment, 50 involving self-regulation, 50 involving the imposition of meaning, 50 involving uncertainty, 50 nonalgorithmic, 50 requiring effort, 50 requiring multiple criteria, 50 yielding multiple solutions, 50 History, influence of technology on, 235–236 How People Learn: Brain, Mind, Experience, and School, 27 Hurricane Katrina, 31–32, 44 technology through the lens of, 32 I IALS. See International Adult Literacy Surveys IBO. See International Baccalaureate Organization ICT. See Information and Communication Technology Literacy Assessment Illinois Science Assessment Framework, 51 Illinois Standards Achievement Test—Science, 95, 299–301 background, 299 committee observations, 301 sample items, 300–301 scope, 299 Illinois State Board of Education, 100 Impact of products and systems, assessing, 242–243 Impairments, 134 Industrial Technology Literacy Test, 95, 301–303 background, 301–302 committee observations, 303 sample items, 302–303 scope, 302 Infinity Project Pretest and Final Test, 95, 304–307 background, 304 committee observations, 307 sample items, 305–307 scope, 304–305 Information and communication technologies, 31, 246–247 benchmarks for grades 3–5, 246 benchmarks for grades 6–8, 246–247 benchmarks for grades 9–12, 247 benchmarks for grades K–2, 246 Information and Communication Technology (ICT) Literacy Assessment, 95, 179, 296–298 background, 296 committee observations, 298 sample items, 297–298 scope, 297 Information in a Global Society, 112 Information processing, 225–226 benchmarks for grades 3–5, 225 benchmarks for grades 6–8, 226

OCR for page 347
Tech Tally: Approaches to Assessing Technological Literacy benchmarks for grades 9–12, 226 benchmarks for grades K–2, 225 Information Technology in a Global Society, 95, 307–310 background, 307–308 committee observations, 310 sample items, 309–310 scope, 308–309 Innovative measurement techniques, 188–190 exploiting, 12–13 Instrument review, 93–126 assessment matrix for technological literacy, 118–123 attitudes toward technology, 113–115 filling the assessment matrix, 115–123 mapping existing instruments to the dimensions of technological literacy, 98–112 technology-literacy-related assessment instruments, 95–97 Instrument summaries, 265–345 Armed Services Vocational Aptitude Battery, 97, 103, 163, 267–268 Assessment of Performance in Design and Technology, 95, 107, 268–272 Awareness Survey on Genetically Modified Foods, 97, 272–274 Design-Based Science, 95, 274–277 Design Team Assessments for Engineering Students, 95, 277–280 Design Technology, 95, 111, 280–284 Engineering K–12 Center Teacher Survey, 94, 97, 284–286 Eurobarometer—Europeans, Science, and Technology, 97, 113, 286–288 European Commission Candidate Countries Eurobarometer—Science and Technology, 97, 288–290 Future City Competition—Judges Manual, 95, 111, 291–293 Gallup Poll on What Americans Think About Technology, 97, 293–295 ICT Literacy Assessment, 95, 179, 296–298 Illinois Standards Achievement Test—Science, 95, 299–301 Industrial Technology Literacy Test, 95, 301–303 Infinity Project Pretest and Final Test, 95, 304–307 Information Technology in a Global Society, 95, 307–310 Massachusetts Comprehensive Assessment System—Science and Technology/Engineering, 95–96, 98–99, 311–313 Multiple-Choice Instrument for Monitoring Views on Science-Technology-Society Topics, 96, 313–317 New York State Intermediate Assessment in Technology, 96, 317–319 Praxis Specialty Area Test for Technology Education, 94, 97, 104, 182, 320–322 Provincial Learning Assessment in Technology Literacy, 96, 101–102, 110, 112, 322–325 Pupils’ Attitudes Toward Technology, 96, 103, 115, 325–327 Science and Technology—Public Attitudes and Public Understanding, 97, 327–330 Student Individualized Performance Inventory, 96, 106, 330–332 Survey of Technological Literacy of Elementary and Junior High School Students, 96, 333–335 Test of Technological Literacy, 96, 336–337 TL50—Technological Literacy Instrument, 96, 337–339 WorkKeys—Applied Technology, 96, 106, 111, 340–345 Interative processes, design as, 43 International Adult Literacy Surveys (IALS), 185 International Association for the Evaluation of Educational Achievement, 8, 180 International Baccalaureate Organization (IBO), 111–112, 281, 307 International Society for Technology in Education (ISTE), 192 International Technology Education Association (ITEA), 1, 10, 14–15, 20, 22, 36, 42, 51, 54–56, 66, 131, 155, 186, 192–193, 293–295 Standards for Technological Literacy, 30, 176, 295, 320, 322, 332, 345

OCR for page 347
Tech Tally: Approaches to Assessing Technological Literacy Issues in technology, 215–218 benchmarks for grades 3–5, 216 benchmarks for grades 6–8, 216–217 benchmarks for grades 9–12, 217–218 benchmarks for grades K–2, 215 ISTE. See International Society for Technology in Education ITEA. See International Technology Education Association K Knowing What Students Know: The Science and Design of Educational Assessment, 27, 38, 72 Knowledge ability to transfer, 76 benchmarks, 131 domain of, 70–71 prior, 77 students’ use of, 110 of a technologically literate person, 34 Knowledge dimension, 98–105 content knowledge, 100 data from the NAEP long-term science assessment, 101 process knowledge, 100 productive thinking scale, 100 Knowledge transfer, 75–77, 276 L Learning “free-choice,” 153–154 lifelong, 154 related to engineering, 83–86 related to technology, 80–83 research on, 186–188 Level of technological literacy, 20–22 M Maintaining technological products and systems, 241–242 benchmarks for grades 3–5, 241 benchmarks for grades 6–8, 241 benchmarks for grades 9–12, 241–242 benchmarks for grades K–2, 241 Manufacturing technologies, 31, 248–249 benchmarks for grades 3–5, 249 benchmarks for grades 6–8, 249 benchmarks for grades 9–12, 249 benchmarks for grades K–2, 248 Mapping existing instruments capability dimension, 105–111 critical-thinking and decision-making dimension, 111–112 dimensions of technological literacy, 98–112 knowledge dimension, 98–105 Maryland State Department of Education, 134 Massachusetts Comprehensive Assessment System (MCAS)—Science and Technology/Engineering, 95–96, 98–99, 311–313 background, 311 committee observations, 313 sample items, 312–313 scope, 311–312 Massive, multiplayer, on-line games (MMOGs), 168 Materials and manufacturing, 220–222 benchmarks for grades 3–5, 220–221 benchmarks for grades 6–8, 221 benchmarks for grades 9–12, 221–222 benchmarks for grades K–2, 220 Mathematical literacy, 33 Matrix-sample assessment of 7th graders, 136–140 administration and logistics, 138 content specifications, 137–138 description and rationale, 136–137 obstacles to implementation, 138–139 performance levels, 138 purpose, 137 sample assessment item, 139–140 Matrix sampling, 71, 139, 183 MCAS. See Massachusetts Comprehensive Assessment System MCREL. See Mid-Continent Research for Education and Learning Measurement, defined, 64 Measurement issues, 69–71 constructed-response formats, 70 domain of knowledge, 70–71 effective, practical formats, 70 performance-assessment formats, 70 selected-response formats, 69 Medical technologies, 31, 243–244 benchmarks for grades 3–5, 243 benchmarks for grades 6–8, 243 benchmarks for grades 9–12, 244 benchmarks for grades K–2, 243

OCR for page 347
Tech Tally: Approaches to Assessing Technological Literacy Mental models, assessing, 84 Metacognition, 77–78 Mid-Continent Research for Education and Learning (MCREL), 142 MMOGs. See Massive, multiplayer, on-line games Modified Angoff standards-setting, 72 “Monkey Wrench Conspiracy,” 168 Multiple-Choice Instrument for Monitoring Views on Science-Technology-Society Topics, 96, 313–317 background, 313–314 committee observations, 316–317 sample items, 314–316 scope, 314 N NAEP. See National Assessment of Educational Progress NAGB. See National Assessment Governing Board National Academies Press (NAP), 17 National Academy of Engineering, 15, 25, 192–193 National Adult Literacy Survey, 185 National Assessment Governing Board (NAGB), 8, 13–14, 51, 180–181, 183, 191 National Assessment of Educational Progress (NAEP), 5, 8, 20, 51–54, 71, 101, 136–138, 178–181 National Center for Education Statistics, 146n, 185 National Center for Technological Literacy, 114 National Council for Excellence in Critical Thinking, 112 National Education Association, 24 National Household Education Survey (NHES), 10, 185–186 National Institute of Standards and Technology, 13, 189 National Institutes of Health, 10, 186 National Opinion Research Center, 185 National Research Council (NRC), 15, 25, 38, 66, 131, 192–193 National-sample assessment of teachers, 140–145 administration and logistics, 143 content, 142–143 description and rationale, 140–141 obstacles to implementation, 144 performance levels, 143 performance rubric for sample task, 141 purpose, 142 sample assessment items, 144–145 National Science Board (NSB), 105, 148, 327–328 National Science Education Standards, 30, 131, 137, 142, 207–212 abilities to distinguish between natural objects and objects made by humans, 212 standards related to “science and technology,” 207–209 understanding about science and technology, 210–211 National Science Foundation (NSF), 1, 8–14, 19, 25, 41, 180–181, 184–188, 191 “The nature of technology,” standards related to, 212–218 NCLB. See No Child Left Behind Act New York Hall of Science, 156 New York State Intermediate Assessment in Technology, 96, 317–319 background, 317 committee observations, 319 sample items, 318–319 scope, 317–318 NHES. See National Household Education Survey No Child Left Behind Act (NCLB), 9, 14, 24, 45, 60, 72, 129, 133–134, 143, 179, 183 Norm-referenced interpretations, 71 North Carolina State University, 105, 273 NRC. See National Research Council NSB. See National Science Board NSF. See National Science Foundation Numeracy, 33 O Obstacles to assessing technological literacy, 24–25 Online surveys, 171 Opportunities for assessment, 7–10, 176–186 findings and recommendations for K–12 students, 8–9, 95–96, 178–181

OCR for page 347
Tech Tally: Approaches to Assessing Technological Literacy findings and recommendations for K–12 teachers, 9, 97, 181–184 findings and recommendations for out-of-school adults, 10, 97, 184–186 general findings, 176–178 Organization for Economic Cooperation and Development, 8, 180 P PATT-USA. See Pupils’ Attitudes Toward Technology Performance-assessment formats, 70 Pilot testing, 68 PISA. See Programme for International Student Assessment Policy-attentive citizens, 147, 151–152 Potential solutions, refining, 58–59 Practical formats, 70 Praxis Specialty Area Test for Technology Education, 94, 97, 104, 182, 320–322 background, 320 committee observations, 322 sample items, 321–322 scope, 320–321 Problem solvers, unskilled, 74 Problem solving—troubleshooting, research and development, invention and innovation, and experimentation, 238–239 benchmarks for grades 3–5, 239 benchmarks for grades 6–8, 239 benchmarks for grades 9–12, 239 benchmarks for grades K–2, 238–239 Problems, defined, 44–45 Process knowledge, 100 Productive Thinking Scale (PTS), 100 Programme for International Student Assessment (PISA), 8, 127, 180 Provincial Learning Assessment in Technology Literacy, 96, 101–102, 110, 112, 322–325 background, 322–323 committee observations, 325 sample items, 324 scope, 323 PTS. See Productive Thinking Scale Pupils’ Attitudes Toward Technology (PATT-USA), 96, 103, 115, 325–327 background, 325 committee observations, 327 sample items, 326–327 scope, 326 Purpose gathering data useful to, 49 meeting a specific, 48–49 Purposes for assessments, 49 accountability, 49 improving curriculum and instruction, 49 informing informal-education programs, 49 informing legislation and public policy, 49 informing product design and marketing, 49 meeting graduation/matriculation requirements, 49 meeting professional licensing requirements, 49 program evaluation, 49 as a research tool, 49 screening for hiring/employment, 49 Q Qualitative methodologies, 83 Questionnaires electronic, 171–172 open-ended, 83 very short, 171 R Reading literacy, 32 Recommendations, 8–15, 95–97, 175–196 Relationships among technologies and the connections with other fields of study, 231–232 benchmarks for grades 3–5, 232 benchmarks for grades 6–8, 232 benchmarks for grades 9–12, 232 benchmarks for grades K–2, 231 Reliability, defined, 64–65 Renaissance Learning Star Reading Test, 164 Reporting results, 71–72 criterion-referenced interpretations, 72

OCR for page 347
Tech Tally: Approaches to Assessing Technological Literacy norm-referenced interpretations, 71 standards-based interpretation, 72 Research conducting relevant, 56 on learning, 11–12 Research on technological learning, 80–86 concept inventories, 85 learning related to engineering, 83–86 learning related to technology, 80–83 Resource constraints, 144 Review, of instruments, 93–126 S Sample cases, from theory to practice, 127–160 Science and Engineering Indicators, 20, 105, 113, 184 “Science and technology” standards related to, 207–209 understanding about, 210–211 Science and Technology—Public Attitudes and Public Understanding, 97, 327–330 background, 327–328 committee observations, 329–330 sample items, 328–329 scope, 328 Science for All Americans, 35, 190 Science/technology/society (STS) topics, 314 Scientific literacy, 33 Scope of technology, 229 benchmark for grades 3–5, 229 benchmark for grades 6–8, 229 benchmark for grades 9–12, 229 benchmark for grades K–2, 229 Selected-response formats, 69 Shen, Benjamin, 147 Sim City™ software, 111, 292 Simulations, 164–168 SIPI. See Student Individualized Performance Inventory Society, technology-dependent, 195 Society’s role in the development and use of technology, 234–235 benchmarks for grades 3–5, 234 benchmarks for grades 6–8, 235 benchmarks for grades 9–12, 235 benchmarks for grades K–2, 234 Software piracy, 83 Solutions, identifying potential, 57–58 Stakeholders, input from, 47 Stand-alone testing, 60 Standards-based interpretation, 72 Standards for Educational and Psychological Testing, 162 Standards for technological literacy, 229–250 Standards for Technological Literacy: Content for the Study of Technology, 22, 26, 29, 36, 42, 51, 54, 65, 93, 104, 106, 131, 137, 142, 185, 193 Standards related to abilities for a technological world, 239–243 applying the design process, 239–241 assessing the impact of products and systems, 242–243 using and maintaining technological products and systems, 241–242 Standards related to design, 236–239 the attributes of design, 236–237 engineering design, 237–238 role of troubleshooting, research and development, invention and innovation, and experimentation in problem solving, 238–239 Standards related to “science and technology,” 207–209 abilities of technological design, 207–209 Standards related to technology and society, 232–236 the cultural, social, economic, and political effects of technology, 232–233 the effects of technology on the environment, 233–234 influence of technology on history, 235–236 the role of society in the development and use of technology, 234–235 Standards related to the design world, 243–250 agricultural and related biotechnologies, 244–245 construction technologies, 250 energy and power technologies, 245–246 information and communication technologies, 246–247 manufacturing technologies, 248–249 medical technologies, 243–244 transportation technologies, 247–248

OCR for page 347
Tech Tally: Approaches to Assessing Technological Literacy Standards related to “the designed world,” 218–228 agriculture, 218–220 communication, 224–225 energy sources and use, 222–223 health technology, 227–228 information processing, 225–226 materials and manufacturing, 220–222 Standards related to “the nature of technology,” 212–218, 229–232 characteristics and scope of technology, 229 core concepts of technology, 230–231 design and systems, 214–215 issues in technology, 215–218 the relationships among technologies and the connections with other fields of study, 231–232 technology and science, 212–213 Statewide grade-level assessment, 129–136 administration and logistics, 133 content, 131 description and rationale, 129–130 obstacles to implementation, 133–135 performance levels, 131–132 purpose, 130 sample assessment items, 135–136 weighting for items assessing knowledge, capability, and critical thinking and decision making, 132 Stereotyping, 135 Strategy selection, 74 STS. See Science/technology/society topics Student Individualized Performance Inventory (SIPI), 96, 106, 330–332 background, 330 committee observations, 332 sample items, 331–332 scope, 331 Survey of Teachers’ Attitudes About Engineering, 113–114 Survey of Technological Literacy of Elementary and Junior High School Students, 96, 333–335 background, 333 committee observations, 335 sample items, 334–335 scope, 333–334 Survey Research Center, 185–186 Surveys of consumers, 10, 186 T Technical competence, 35 Technically Speaking: Why All Americans Need to Know More About Technology, 1, 4–5, 15, 20, 22, 26, 30, 33, 36–37, 42, 46, 54, 57, 93, 98, 105, 111, 128, 155, 192 Technological design process conducting relevant research, 56 defining the problem, 44–45 identifying constraints, 45–48 identifying design criteria, 48–56 identifying potential solutions, 57–58 input from stakeholders, 47 linear steps in, 43–59 refining potential solutions, 58–59 Technological literacy, 32–36 characteristics of a technologically literate person, 34 defined, 29–40 distinguishing from technical competence, 35 Technological products and systems, using and maintaining, 241–242 Technologically literate people, 34 capabilities, 34 critical thinking and decision making, 34 knowledge, 34 Technology attitudes toward, 113–115 defined, 192–193 design and, 271 standards related to the nature of, 229–232 through the lens of Hurricane Katrina, 32 Technology and science, 212–213 benchmarks for grades 3–5, 212–213 benchmarks for grades 6–8, 213 benchmarks for grades 9–12, 213 benchmarks for grades K–2, 212 Technology and society, standards related to, 232–236 Technology consumers, 147, 150–151 Technology for All Americans, 35 Technology-literacy-related assessment instruments, 95–97 Technology-related standards and benchmarks, 207–250 benchmarks for science literacy, 212–228

OCR for page 347
Tech Tally: Approaches to Assessing Technological Literacy National Science Education Standards, 207–212 standards for technological literacy, 229–250 Test of Technological Literacy, 96, 336–337 background, 336 committee observations, 337 sample items, 336–337 scope, 336 Test-taking skills, 69 Testing, defined, 64 Testing and measurement, 64–72 avoiding bias, 68 basic vocabulary, 64–65 central themes, 66–69 ensuring fairness, 68–69 measurement issues, 69–71 purpose defined, 66–67 reporting of results, 71–72 selecting content, 67 Thinking Through Technology: The Path Between Engineering and Philosophy, 54 TIDEE. See Transferable Integrated Design Engineering Education Consortium TIMSS. See Trends in Mathematics and Science Study TL50—Technological Literacy Instrument, 96, 337–339 background, 337–338 committee observations, 339 sample items, 338–339 scope, 338 “Trading off,” 57–58 Training, in administering tests, 135 Transferable Integrated Design Engineering Education (TIDEE) Consortium, 103, 277 Transportation technologies, 31, 247–248 benchmarks for grades 3–5, 247–248 benchmarks for grades 6–8, 248 benchmarks for grades 9–12, 248 benchmarks for grades K–2, 247 Trends in Mathematics and Science Study (TIMSS), 8, 71, 179–180 U Understanding about science and technology, 210–211 benchmarks for grades 5–8, 210–211 benchmarks for grades 9–12, 211 benchmarks for grades K–4, 210 University of Michigan, 185–186 University of Southern California, 166 U.S. Census Bureau, 146n U.S. Department of Defense, 10, 163, 186 U.S. Department of Education, 8–15, 72, 180, 183–188, 191–193 V Validity, defined, 65 Verbal protocol analysis, 83 Views on Science, Technology, and Society (VOSTS), 114–115 Visualization, development of, 82 Vocabulary of testing and measurement, 64–65 assessment defined, 64 errors of measurement defined, 65 measurement defined, 64 reliability defined, 64–65 testing defined, 64 validity defined, 65 VOSTS. See Views on Science, Technology, and Society W Web-based games, 168–169 WestEd, 179, 185 WorkKeys—Applied Technology, 96, 106, 111, 340–345 background, 340 committee observations, 345 sample items, 341–345 scope, 341 World of design, standards related to, 243–250