Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
3 The Case for Kâ12 Engineering Education P roponents have put forth a number of reasons for adding Kâ12 engineering education to the school curriculum (Box 3-1). Their arguments are similar to arguments for improving STEM education. Both cases are based on changes in the worldâincreasing complexity, inter- connectivity, competitiveness, and technology dependenceâthat pose new challenges for individuals and for nations that cannot be met by continuing education as usual. We will need a steady supply of well-trained engineers, scientists, and other technical workers, as well as a technologically and sci- entiï¬cally literate general public, to succeed and prosper in the twenty-ï¬rst century (Augustine, 2007; BSCS, 2007). In this chapter, we present a detailed discussion of the case for Kâ12 engineering education, focusing on various aspects of the argument and on supporting data. THE BENEFITS OF Kâ12 ENGINEERING EDUCATION The potential beneï¬ts to students of including engineering education in Kâ12 schools can be grouped into ï¬ve areas: improved learning and achievement in science and mathematics; increased awareness of engineering and the work of engineers; understanding of and the ability to engage in engineering design; 49
50 ENGINEERING IN Kâ12 EDUCATION BOX 3-1 Statements from Selected Kâ12 Engineering Education Programs The âEngineering by Designââ¢ Program is a model used by schools developing themes in the STEM and IT Clusters that are seeking to increase all studentsâ achievement in technology, science, mathematics, and English through authentic learning. ITEA http://www.iteaconnect.org/EbD/ebd.htm âThe Inï¬nity Projectâ is helping close the gap between the number of engineering graduates we currently produce in the United States and the large need for high-quality engineering graduates in the near future. For our next generation of college graduates to be competitive in the global world of technology, we need to take steps now to encourage more young students to pursue engineering. Southern Methodist University http://www.inï¬nity-project.org/inï¬nity/inï¬nity_hist.html The âEngineering is Elementaryâ project aims to foster engineering and technological literacy among children. Boston Museum of Science http://www.mos.org/eie/index.php interest in pursuing engineering as a career; and increased technological literacy. Although only a small percentage of students has had an opportunity to study engineering in elementary and secondary schools in the United States, a number of curricula for teaching engineering have been developedâmany of which are described in Chapter 4. Curriculum developers, cognitive sci- entists, and others have studied the effects of these curricula and other Kâ12 engineering initiatives on student learning, interests, and attitudes. Based on their research, it is possible to assess the evidence for these beneï¬ts. The remainder of this chapter provides the highlights and key ï¬ndings of a commissioned review of the relevant research literature, which includes articles published in peer-reviewed journals, conference papers, program
THE CASE FOR Kâ12 ENGINEERING EDUCATION 51 evaluations, and unpublished documents such as dissertations (Svihla et al., unpublished). Overall, the review turned up limited evidence for many of the beneï¬ts predicted or claimed for Kâ12 engineering education. This does not mean that the beneï¬ts do not exist, but it does conï¬rm that relatively few well- designed, carefully executed studies have been conducted on this subject. This issue is discussed in greater detail at the end of this chapter and in Chapter 6. Improved Learning and Achievement in Science and Mathematics One of the claims most often made about Kâ12 engineering education is that it improves learning and achievement in science and mathematics. This is a particularly compelling claim because, for the past two decades, many concerted efforts have been made to improve Kâ12 science and math- ematics education in the United States. By most accounts those efforts have had relatively unimpressive results (Box 3-2). How might engineering education improve learning in science and mathematics? In theory, if students are taught science and mathematics concepts and skills while solving engineering or engineering-like problems, they will be able to grasp these concepts and learn these skills more easily and retain them better, because the engineering design approach can provide real-world context to what are otherwise very abstract concepts. Preliminary evidence supports this theory. For example, students who took courses developed by âProject Lead the Wayâ (PLTW) scored signiï¬- cantly higher on science and mathematics in the NAEP than students in a random, stratiï¬ed comparison group (Bottoms and Anthony, 2005; Bottoms and Uhn, 2007). Research using a state achievement test as the basis of comparison has found more mixed results. PLTW students from schools serving a high proportion of low-income families showed less improvement in mathematics scores from grade 8 to 10 and no statistical difference in sci- ence achievement scores over that period, compared with a control group (Tran and Nathan, In press). And PLTW students attending schools serv- ing predominantly afï¬uent families exhibited small gains in mathematics achievement but no improvement in science achievement, compared with a control sample (Tran and Nathan, In press). Students who had taken the âEngineering Our Future New Jerseyâ course, which is offered in 32 elementary, middle, and high schools in the state, demonstrated signiï¬cant improvements in scores on both science and
52 ENGINEERING IN Kâ12 EDUCATION BOX 3-2 The Push to Improve Kâ12 Science and Mathematics Education In 1990, the Department of Education National Education Goals Panel released a report detailing necessary improvements in U.S. education. In that report, science and mathematics were the only subjects addressed speciï¬cally. Goal 5 was, âBy the year 2000, United States students will be ï¬rst in the world in mathematics and science achievementâ (DOEd, 1989). Eleven years later, when the department published a deï¬nitive study of science and mathematics teaching in the United States, the conclusion was that little progress had been made toward reaching that goal (DOEd, 2000). In the past few years, many studies, such as Rising Above the Gather- ing Storm: Energizing and Employing America for a Brighter Economic Future, have argued that improving science and mathematics education will require substantial reform (NAS et al., 2007). Many of these reports include data from the National Assessment of Educational Progress (NAEP) and two ongoing international comparative assessments, the Trends in International Mathematics and Science Study (TIMSS) and the Programme for International Student Assessment (PISA), to support the contention that U.S. Kâ12 students, particularly secondary students, simply do not measure up. Although TIMSS and PISA data are often used as indicators, some have argued that most interpretations of these data overstate the U.S. achievement problem, in part because they do not account for differences in the educational systems of the participating countries (Lowell and Salzman, 2007). In 2007, the Department of Education published a review of all feder- ally funded programs with a math or science education focus, looking at their effectiveness and at ways to integrate and coordinate them. The report focused on 115 programs that it considered to have the best evalu- ations and concluded that there was very little hard evidence as to which programs were effective and which were not (DOEd, 2007). mathematics achievement tests1 (Hotaling et al., 2007). Statistically signiï¬- cant gains in science and mathematics scores have also been reported by the 1In this study, the results were not disaggregated, and no measure of variance was pro- vided. Thus we cannot know if the gains were uniform or if some subgroups were more or less impacted.
THE CASE FOR Kâ12 ENGINEERING EDUCATION 53 Center for Innovation in Engineering and Science Education at Stevens Institute of Technology, which has created a variety of online, problem-based Kâ12 engineering curricula (McKay and McGrath, 2007). Students who had participated in âEngineering is Elementary,â a program developed by the Boston Museum of Science that integrates engineering with science content for elementary students, showed improvement in a post-test measuring science and engineering knowledge (Lachapelle and Cunningham, 2007). Unfortunately, there was no control group for comparison in this study. Engineering design has been shown to encourage mathematical think- ing. Akins and Burghardt (2006) studied teams of middle and high school students who applied mathematical reasoning to solve problems in a design challenge. Pre-test results were used to disaggregate students into quartiles, and all quartiles showed improvement on math and science tests. (No tests of signiï¬cance were conducted, but post-test scores were 21 percent to 125 percent higher than pre-test scores.) The authors noted that the lowest scoring teams had the highest score gains, which suggests that engineering design has the potential to narrow achievement gaps; this possibility was not noted by the researchers, however. In some cases, standardized test scores were not impacted by student participation in engineering activities, but other measures, such as the ability to explain, analyze, predict, or reason about science, mathematics, or tech- nology, demonstrate that the students had learned a great deal. For example, a program at one inner-city school involved designing remote-control vehicles. Although the scores of students who participated in the program did not show improvements on district-wide physics achievement tests, pre- post measures showed that the students had a better understanding of the physics related to their vehicles (Barnett, 2005). In the âIntegrated Mathematics, Science, and Technologyâ (IMaST) curriculum project, participating students and non-IMaST students had similar gains on state mathematics and science achievement tests, but IMaST students scored higher on TIMSS math items than students in a control group (Satchwell and Loepp, 2002). Notably, IMaST students scored higher on measures related to âprocessâ (i.e., mathematical problem solving and science inquiry) whereas the control students scored higher on measures related to âknowingâ (i.e., understanding routine mathematics operations and scientiï¬c information). A few studies have been done on the potential of Kâ12 engineering to differentially affect math and science achievement among girls and under- represented minorities. In a middle-school study of modules in which engi-
54 ENGINEERING IN Kâ12 EDUCATION neering design and science were integrated, pre-post data showed that the achievement gap for African American and Latino/a students was narrowed, but the achievement gap for girls was increased (Cantrel et al., 2006). It is not clear in this case whether the students engaged in a truly iterative design process, which has been shown to encourage science learning for girls and students from families of low socio-economic status (SES) (Kolodner et al., 2003). Barnett (2005) reported on a study of inner-city, low SES, pre- dominantly ethnic-minority high school students that included a signiï¬cant population of English language learners and many students with disabilities. All of these students had participated in a project that involved designing remotely operated vehicles. Pre-post data revealed that, overall, the studentsâ understanding of physics had improved. However, the improvement did not translate to higher scores on a district-wide ï¬nal exam in physics. So-called challenge-based environments can mimic design or motivate students to solve problems in order to learn engineering, science, and math- ematics content. In a three-year study of this approach, âlegacy cycles,â Klein and Sherwood (2005) found that students in the experimental group had statistically larger gains in measures of relevant science knowledge and con- cepts. Although most of the modules did not involve design, they did require problem solving in the context of engineering and had many design elements. The researchers argue that design challenges embedded in science activities increase the likelihood that students will explore variables rather than stop- ping their inquiries as soon as the design criteria have been satisï¬ed. The âMath out of the Boxâ program uses a modiï¬ed legacy cycle in which engi- neering provides a context for learning applied mathematics (Diaz and King, 2007). This program has been implemented in several schools; the ones that have continued to use it have found that achievement scores in mathematics have risen, particularly for low-SES and African American students. The schools that discontinued the program found that mathematics scores fell. Qualitative research in the learning sciences provides some insights into how and why science and mathematics learning may be impacted by participation in engineering activities, particularly design activities. Fortus et al. (2004) recorded signiï¬cant increases in science knowledge among ninth graders engaged in the âDesigned-Based Scienceâ curriculum. The researchers suggest that this effect can be explained in part by studentsâ personal ownership of science content as compared with consensus-driven ownership in other forms of inquiry. Students using this curriculum were also able to transfer their understanding of a concept from the original context to a different context (Fortus et al., 2005), which the researchers
THE CASE FOR Kâ12 ENGINEERING EDUCATION 55 attribute to the way the design activity is structured to support learning for understanding in the context of solving a problem. Roth (2001) suggests that design activities, which present distributed representations of ideas, can stimulate discussions about science concepts. Ideas represented through design can then be inspected and tested. Penner et al. (1998) explored how the design by elementary students of a physical model of an elbow can support science and mathematics learning related to the mechanics of motion. The success of the project depended on students having multiple opportunities to engage in and discuss their design experiences, teachersâ use of analogies, and sense-making based on data col- lection and interpretation. Redesign gives students a chance to explore con- nections between science and design, to test their ideas, and to decide how to correct their designs and then adjust the corresponding understanding of the relevant scientiï¬c principle or concept (Sadler et al., 2000). In summary, the available evidence suggests that under certain cir- cumstances, engineering education can boost learning and achievement in science and mathematics. These effects may be more signiï¬cant for certain populations, particularly underrepresented minority students. However, the positive effects are not universal and research has not clearly established the causal mechanism(s) to explain such beneï¬ts when they occur. Increased Awareness of Engineering and the Work of Engineers This goal, improving studentsâ awareness of engineering and the work of engineers, can be of great beneï¬t to a society, because engineering is central to technology development, and technology inï¬uences the well-being of everyone. Conversely, a lack of awareness of engineering and misconceptions or ignorance about what engineers do can be detrimental to a society. On a practical level, young people who believe engineers drive trains or repair car engines or who have negative stereotypes of the profession are unlikely ever to consider studying engineering or pursuing it as a career. If enough young- sters feel this way, it may become increasingly difï¬cult to attract and retain a technically proï¬cient workforce. Generally, individuals who do not have a basic idea of what engineers do are unlikely to appreciate how engineering and science contribute to economic development, quality of life, national security, and health care; such awareness is one aspect of technological literacy (NAE and NRC, 2006). The engineering community, including engineering professional soci- eties, schools of engineering, and ï¬rms that depend heavily on engineering
56 ENGINEERING IN Kâ12 EDUCATION talent, have spent hundreds of millions of dollars annually on initiatives to raise the level of the public understanding of engineering (NAE, 2002), for the most part unsuccessfully. For example, researchers have found that Kâ12 teachers and students generally have a poor understanding of what engineers do (Cunningham and Knight, 2004; Cunningham et al., 2005; Oware et al., 2007). Survey data suggest that many adults in the United States believe that engineers, as compared with scientists, are not as responsive to societal and community concerns and are not as important in saving lives (Harris Interactive, 2004). This widespread misconception reveals a lack of awareness of the many ways engineering has dramatically improved the human condition (e.g., www.greatachievements.org). Teens and adults strongly associate engineering with skills in mathematics and science, according to recent online polling, but much more rarely with creativity, rewarding work, or a positive effect on the world (NAE, 2008). Findings like these have prompted advocates of Kâ12 engineering edu- cation to argue for the importance of young people having opportunities to learn about engineers, engineering, and technology. Research has shown that participation in engineering education activities can provide those oppor- tunities. For example, assessments showed that students who participated in the âEngineering Our Future New Jerseyâ program were able to name signiï¬- cantly more types of engineers and to describe types of engineering activities (Hirsch et al., 2005). These students were also able to recognize technology and the work of engineers (Hotaling et al., 2007). Teachers, too, may be more aware of engineering career options after leading engineering design activities with students (McGrath et al., 2008). Pre-post tests found that young children who took part in the âEngineer- ing is Elementaryâ program had a signiï¬cantly broader conception of what technology is and were able to identify activities undertaken by engineers (Lachapelle and Cunningham, 2007). According to a study by graduate teaching fellows in Kâ12 education funded by the National Science Founda- tion, such changes in studentsâ awareness of engineers and engineering can be sustained over time (Lyons and Thompson, 2006). Understanding of and the Ability to Engage in Engineering Design The iterative, open-ended, problem-solving method known as engi- neering design is the central activity of engineers. For this reason, a good deal of Kâ12, as well as post-secondary, engineering education is spent on
THE CASE FOR Kâ12 ENGINEERING EDUCATION 57 developing studentsâ understanding and capabilities in this area. In addition, as was mentioned above, design activities provide a real-world focus for abstract concepts, which may have a positive impact on learning not only in engineering, but also in other subjects, such as mathematics and science. In this section, we consider the evidence related to how well students learn to understand and engage in engineering design. Data from a number of studies suggest that engineering design as prac- ticed by engineers is neither quickly learned by students nor easily taught by teachers. Issues common to novice design, such as using trial-and-error methods (rather than a systematic approach) and spending too much time on deï¬ning the problem, have been well documented (e.g., Hill and Smith, 1998; Ressler and Ressler, 2004). Unless the teacher explicitly encourages a systematic approach, the design process can be overwhelmed by student excitement about hands-on activities (Seiler et al., 2001). Speciï¬c concepts integral to engineering design also pose challenges to students. For example, in a project in which undergraduate engineer- ing and education students developed design activities for students in the seventh through twelfth grades, Bergin et al. (2007) found that the Kâ12 students had difï¬culty understanding the idea of constraints. Penner et al. (1997) found that elementary students struggled to use modeling in a way that reï¬ects engineering practice. In this study, student pairs were asked to design a functional model of an elbow. At ï¬rst, the children tended to see models as small versions of the thing itself, and their ï¬rst design iterations copied the form of an elbow but could not perform the functions of an elbow. After some discussion, it was clear that students had not isolated the motion of the elbow but had inferred a great range of motion based on the pivot of the shoulder. After experimenting with real elbow movements, they began a second iteration of modeling. This time the models incorporated constraints but also included nonfunctional but physically similar details, such as a representation of veins. Interest in Pursuing Engineering as a Career As many reports and commentators have noted, the economic competi- tiveness of the United States depends in large part on our ability to attract, train, and retain a large corps of highly qualiï¬ed, creative engineers in a variety of ï¬elds (e.g., NAS et al., 2007). Unfortunately, many students who are capable of becoming engineers never even enter the educational pipeline leading to an engineering career because they either do not understand what
58 ENGINEERING IN Kâ12 EDUCATION engineers do or they believe that they do not have the necessary aptitude or interests to become engineers. This is particularly common for females and students from certain minorities, who are greatly underrepresented in engi- neering schools and in engineering practice (see Table 2-1) (Chang, 2002). Up to now, the primary strategy for ensuring that the engineering pipeline is ï¬lled has been to insist that high school graduates have a good grounding in science and mathematics. Thus students are not exposed to engineering until they enter college, frequently not until their junior year. Kâ12 engineering programs offer a different strategy. By introducing stu- dents to engineering in Kâ12 programsâin theory, at leastâmore of them, from a wider variety of backgrounds, will be attracted to the ï¬eld. Although keeping the engineering pipeline ï¬owing is an explicit goal of only a handful of the curricula we examined, the idea that exposure to engi- neering thinking, particularly design experiences, will attract more students to the pursuit of engineering or technology-related studies and careers seems intuitively sound. In this section, we examine the evidence for how Kâ12 engineering education affects student interest in engineering and related factors, such as school attendance, retention, and persistence. Research has shown that students who choose to participate in engineer- ing-related activities and coursework may become more interested in pursu- ing careers in engineering. For instance, both girls and boys who attended the âDiscover Engineeringâ summer camp at Ryerson University in Canada reported an increased interest in engineering as a career (Anderson and Northwood, 2002). A follow-up study showed that approximately one-third of camp participants actually went on to pursue engineering degrees (Ander- son et al., 2005). However, without a comparison group we cannot know if this group of students was representative of the general population. Not all students respond the same way to educational interventions. Thus it is important to determine how speciï¬c groups tend to respond. For instance, in an engineering enrichment program for gifted students, par- ticipants completed small design projects as part of reaching a larger design goal (Bayles et al., 2007). Following the experience, 11 percent of students indicated that they felt less conï¬dent about their ability to become engineers, and 41 percent said they felt more conï¬dent. In a survey of students enter- ing the âDiscover Engineeringâ outreach program, Anderson and Gilbride (2003a) found that boys were signiï¬cantly more interested than girls in pursuing engineering careers. Boys who claimed to have more knowledge of engineering were more interested than less-knowledgeable boys, but girls who claimed to be more knowledgeable were not more interested than their
THE CASE FOR Kâ12 ENGINEERING EDUCATION 59 less knowledgeable peers. An assessment of student interest in engineering following participation in the program showed an increase in interest among both boys and girls, but girlsâ interest did not rise to parity (Anderson and Gilbride, 2003b). In a study of a different program, both boys and girls reported gains in conï¬dence about engineering as a career after participating in engineering design activities, and girls and boys had equal scores (Zarske et al., 2007). An investigation of why the two studies produced different results could be potentially informative. Some evidence suggests that engineering activities have coincided with higher school attendance, perhaps a reï¬ection of increased interest. Barnett (2005) reported that attendance increased for a group of inner-city high school science students (largely from low-SES ethnic minorities) who were randomly assigned to classes in which the major focus was on engineering design projects, compared to their peers who were taught the standard sci- ence curriculum. Studies have also been done on retention levels and persistence in engi- neering, primarily for students already interested in engineering. Most high school students who took an introductory engineering-design course based on a course for ï¬rst-year college students, for example, went on to pursue engineering degrees in college (Bayles, 2005). Students who take courses from PLTW, a four-year college preparatory program, tend to take more advanced science and math courses and to consider them important to their future (Bottoms and Anthony, 2005; Bottoms and Uhn, 2007). Most PLTW students say they plan to attend college (Walcerz, 2007), although this cannot deï¬nitively be attributed to participation in PLTW because this is a self-selected group. The same students reported feeling conï¬dent about their career choices (mostly engineering and technology) because of the courses they took in high school. In addition, participation in PLTW has been shown to reduce attrition rates in college engineering programs and to increase the percentage of degrees attained (Taylor et al., 2006). These ï¬ndings are positive, but the students who choose to take such courses cannot be considered a general population. In a study of the long-term impact of a two-week engineering camp for middle schoolers, participating students were likelier than a control group to take STEM courses (Hubelbank et al., 2007). This ï¬nding is signiï¬cant because both groups had applied to attend the camp, and the participants were selected by lottery. The camp experience did not affect studentsâ interest in college-level engineering, however. Students in the control and experi- mental groups were equally likely to pursue engineering degrees.
60 ENGINEERING IN Kâ12 EDUCATION Participation in Kâ12 engineering education programs may correlate with an increase in applications to engineering colleges. Zarske et al. (2007) found this to be true for a Kâ12 program in Colorado. However, although the number of applications increased, many applicants had not completed the coursework necessary for acceptance into the college program. One way of supporting these students is to provide a bridge program. Anderson- Rowland et al. (1999) demonstrated a signiï¬cantly higher level of retention for students who attended the Summer Bridging Program (SBP) at Arizona State University, a program for entering minority freshmen. However, the effects of SBP were difï¬cult to determine because participants were also required to enroll in an Academic Success Seminar during their freshman year. Increased Technological Literacy Many have argued that Kâ12 engineering classes improve studentsâ technological literacy. Although this argument might not have been compel- ling 20 years ago, there is a growing appreciation today of the importance of technological literacy to individuals and to society as a whole. As deï¬ned in Technically Speaking: Why All Americans Need to Know More About Tech- nology, âtechnological literacy combines basic knowledge about the various technologies in our world with the ability to think critically about technology and to make well-informed decisions about technological issuesâ (NAE and NRC, 2006). A technologically literate person understands the essential characteris- tics of technology and how it inï¬uences society and the factors that shape technology, including engineering. Concepts central to engineering, such as systems, trade-offs, and intended and unintended consequences, provide a foundation for making informed decisions in a technologically dependent society like ours. In Technically Speaking, the case for technological literacy is spelled out in detail. A technologically literate person can make informed decisions about his or her use of personal technologies, for example. Technologically literate citizens can be effective participants in decision-making processes involving technologyâfor instance, whether a city should support the build- ing of a coal-ï¬red power plant. In a society with a growing number of jobs that require technological skills and savvy, employers are more likely to ï¬nd technologically competent workers if the general population is technologi- cally literate.
THE CASE FOR Kâ12 ENGINEERING EDUCATION 61 In Kâ12 schools, technological literacy is largely the purview of technol- ogy education teachers. In the United States, 25,000 to 35,000 such teachers work in Kâ12 schools, mostly middle schools and high schools (Dugger, 2007). In 2000, the International Technology Education Association (ITEA) published Standards for Technological Literacy: Content for the Study of Technology, which accelerated an ongoing shift in the ï¬eld of technology education away from its beginnings in industrial arts toward an emphasis on a broad understanding of the concept of technology. The standards in the ITEA document, developed with input from the National Academy of Engineering and National Research Council, include benchmarks related to engineering design (Box 3-3). ITEA and others have also produced curricular materials (e.g., âEngineering by Design,â âEngineering is Elementaryâ) that attempt to meet the learning goals spelled out in the standards. Research shows that many Americansâchildren and adultsâhave a narrow, sometimes incorrect, view of technology. In one study, students BOX 3-3 Selected Engineering-Design-Related Benchmarks, by Grade Band To comprehend engineering design, students should learn that: The engineering design process includes identifying a problem, look- ing for ideas, developing solutions, and sharing solutions with others. (Grades Kâ2) Models are used to communicate and test design ideas and processes. (Grades 3â5) Design involves a series of steps, which can be performed in different sequences and repeated as necessary. (Grades 6â8) Engineering design is inï¬uenced by personal characteristics, such as creativity, resourcefulness, and the ability to visualize and think ab- stractly. (Grades 9â12) SOURCE: ITEA, 2000.
62 ENGINEERING IN Kâ12 EDUCATION in lower elementary grades associated technology mostly with things that require electricity (they conï¬ated technology with lightning) (Cunningham, et al., 2005). Only a few children recognized bridges and bandages, for exam- ple, as technologies. First graders identiï¬ed parrots as a technology nearly as often as they did cups. Surveys of adults have shown that the vast majority associate technology primarily with computers (ITEA, 2004). Several studies have shown that students who have been exposed to engineering educa- tion have a broader conception of technology and have corrected some misconceptions (Hotaling et al., 2007; Lachapelle and Cunningham, 2007). Being able to recognize technology is a basic prerequisite for technological literacy. The committee did not ï¬nd any published research that explicitly ties Kâ12 engineering education to improvements in other aspects of tech- nological literacy. One reason may be that technological literacy, unlike science literacy, is a relatively new idea in education. In addition, there are signiï¬cant challenges associated with the development of assessments of technological literacy. In an extended discussion of the latter problem, Tech Tally: Approaches to Assessing Technological Literacy (NAE and NRC, 2006) pointed out that the âcapabilitiesâ dimension of technological literacy may be especially difï¬cult to measure. In that report the study committee reviewed 28 existing assessment instruments for measuring some aspect of technological literacy, even if they were not designed for that purpose. The committee found that none of these instruments was completely adequate for measuring technological literacy and that only two explicitly targeted engineering learning; one was developed for students in âThe Inï¬nity Project,â and the other was an achievement test for ï¬fth, eighth, and tenth graders in Massachusetts. Interest on the national level in the technological literacy of Kâ12 stu- dents and improvements in measuring instruments, such as assessments, may increase in coming years. For example, when a revised version of the science portion of NAEP is administered for the ï¬rst time in 2009, 10 percent of test items will focus on technological design (NAGB, 2008a). In addi- tion, the National Assessment Governing Board, which oversees NAEP, has recently funded a feasibility study for an assessment of technological literacy (NAGB, 2008b). If the study, which runs until 2012, ï¬nds that technological literacy can be validly and reliably measured, NAGB may add an assessment of technological literacy to its portfolio of tests.
THE CASE FOR Kâ12 ENGINEERING EDUCATION 63 LIMITATIONS OF THE DATA Besides the relatively small number of studies on the impacts of teaching engineering concepts and skills to Kâ12 students, our review of the literature revealed a number of weaknesses in the methodologies used in some studies. Several of these are highlighted below in hopes that they will be addressed in future research on the impacts of this emerging area of education. An overarching concern with the data is that assessments of whether these well-intentioned initiatives achieve their desired goals frequently appear to be an afterthought. Assessments require advanced planning and viable pre-tests. Although pre- and post-tests cannot replace longitudinal data, they do indicate changes over time. Follow-up surveys can be used to determine the persistence of these changes. Another problem is that the data are not âgeneralizable.â For example, students who participate in engineering camps, clubs, and courses have chosen to do so. Thus the ï¬ndings about the effectiveness of these activities cannot be generalized to students who do not choose to participate in these programs. This issue involves not only methodology. Because the ï¬ndings do not provide information about the speciï¬c impacts on women and under- represented minorities or on students who are not initially interested in learning about engineering, these assessments tell us little or nothing about the effectiveness of engineering education on general student populations. This can be a serious problem, because a goal of many of these programs is to increase the number of women and underrepresented minorities in engi- neering classes and ultimately in engineering practice. When data on Kâ12 engineering education initiatives are collected, they often indicate only if participants enjoyed the program and include self- reported changes. It is known that participants in studies sometimes report positive results simply because they are in a study, the so-called Hawthorne effect (Landsberger, 1958). This methodological weakness could be addressed by measuring learning on pre-and post-tests. Most of the studies we reviewed did not assess the impact of engineering education on student subgroups. The problem arises because in presenting data, it is critical to provide measures of central tendency and distribution. For example, the same average may be found for tightly clustered data, indi- cating that most respondents have similar scores, or for widely distributed data, indicating that approximately equal numbers of people had scores above and below the mean. The critical factor is the meaning of the spread. For instance, did minority students or students who most need to learn fall below the mean? The simple solution is to disaggregate data. This is only
64 ENGINEERING IN Kâ12 EDUCATION viable, of course, if the number of subgroup members in the study is suf- ï¬cient to permit statistically valid comparisons. These problems are not limited to studies of engineering education. In fact, deï¬nitive data about the impacts of educational interventions in most subjects are hard to come by. Even for the best-studied areas, such as reading and mathematics, little convincing evidence is available about the effective- ness of teaching approaches. In 2007, for example, the U.S. Department of Education published a review of all federally funded programs with a math or science education focus with the intent of determining their effectiveness as a basis for integrating and coordinating them. The report focused on 115 programs, 24 of them Kâ12 programs for which the âbestâ evaluations were done (DOEd, 2007). [D]espite decades of signiï¬cant federal investment in science and math educa- tion, there is a general dearth of evidence of effective practices and activities in STEM education. Even the 10 well-designed studies [that the review identiï¬ed] would require replication and validation to be used as the basis for decisions about education policy or classroom practice. In short, the lack of a strong evidence base for the beneï¬ts of Kâ12 engineering education is consistent with the situation for much educational research in the STEM arena. This is another reason, if any were needed, for those who promote Kâ12 engineering education to pursue empirical, meth- odologically sound impact studies. REFERENCES Akins, L., and D. Burghardt. 2006. Work in Progress: Improving Kâ12 Mathematics Under- standing with Engineering Design Projects. Pp. 13â14 in Frontiers in Education Con- ference, 36th Annual. New York: Institute of Electrical and Electronics Engineers. Anderson, L.S., and K.A. Gilbride. 2003a. Bringing Engineering to Kâ12 Classrooms: Initia- tives and Results. Proceedings of the 2003 American Society for Engineering Education Annual Conference, Nashville, Tennessee, June 22â25, 2003. Available online at http:// soa.asee.org/paper/conference/paper-view.cfm?id=18386 (accessed January 16, 2009). Anderson, L.S., and K.A. Gilbride. 2003b. Pre-university Outreach: Encouraging Students to Consider Engineering Careers. Paper presented at 3rd Global Congress on Engi- neering Education, Glasgow, Scotland, June 30âJuly 5, 2002. Available online at http:// www.discoverengineering.ryerson.ca/publications/2002UICEE.pdf (accessed January 16, 2009). Anderson, L.S., and D.O. Northwood. 2002. Recruitment and Retention Strategies to Increase Diversity in Engineering. Paper presented at the International Conference on Engineering Education ICEE, Manchester, U.K., August 18â21, 2002. Available online at http://www.ineer.org/Events/ICEE2002/Proceedings/Papers/Index/O065-O070/O069. pdf (accessed January 14, 2009).
THE CASE FOR Kâ12 ENGINEERING EDUCATION 65 Anderson, L.S., K.A. Gilbride, and N. Bajaj. 2005. Discover Engineering Follow-up Surveys: Assessment/Evaluation of Recruitment Programs. Paper presented at the WEPAN/ NAMEPA Joint Conference, Las Vegas, Nevada, April 11, 2005. Available online at http://www.x-cd.com/wepan05/24.pdf (accessed January 13, 2009). Anderson-Rowland, M.R., S.L. Blaisdell, S.L. Fletcher, P.A. Fussell, M.A. McCartney, M.A. Reyes, et al. 1999. A collaborative effort to recruit and retain underrepresented engi- neering students. Journal of Women and Minorities in Science and Engineering 5(4): 323â349. Augustine, N.R. 2007. Is America Falling off the Flat Earth? Washington, D.C.: The National Academies Press. Also available online at http://www.nap.edu/catalog.php?record_ id=12021 (accessed May 2, 2008). Barnett, M. 2005. Engaging inner city students in learning through designing remote oper- ated vehicles. Journal of Science Education and Technology 14(1): 87â100. Bayles, T.M. 2005. Project-Based Learning Design Projects for Introduction to Engineering Design Courses. Paper presented at the American Society for Engineering Education Annual Conference and Exposition Portland, Oregon, June 12â15, 2005. Available online at http://soa.asee.org/paper/conference/paper-view.cfm?id=21005 (accessed Janu- ary 14, 2009). Bayles, T.M., J. Rice, G. Russ, and T. Monterastelli. 2007. High School Outreach: A Look at Renewable Energy. Paper presented at the American Society for Engineering Educa- tion, Honolulu, Hawaii, June 27, 2007. Available online at http://soa.asee.org/paper/ conference/paper-view.cfm?id=4795 (accessed January 14, 2009). Bergin, D., S.K. Khanna, and J. Lynch. 2007. Infusing design into the G7â12 curriculum: two example cases. International Journal of Engineering Education 23(1): 43â49. Bottoms, G., and K. Anthony. 2005. Project Lead the Way: A Pre-Engineering Curriculum That Works. Southern Regional Education Board. Available online at http://www.sreb. org/programs/hstw/publications/briefs/05V08_Research_PLTW.pdf (accessed May 9, 2008). Bottoms, G., and J. Uhn. 2007. Project Lead the Way Works: A New Type of Career and Technical Program. Southern Educational Review Board. Available online at http:// www.sreb.org/programs/hstw/publications/2007pubs/07V29_Research_Brief_PLTW.pdf (accessed January 15, 2009). BSCS (Biological Sciences Curriculum Study). 2007. A Decade of Action: Sustaining Global Competitiveness. Colorado Springs, Colo.: BSCS. Cantrell, P., G. Pekcan, A. Itani, and N. Velasquez-Bryant. 2006. The effects of engineering modules on student learning in middle school science classrooms. Journal of Engi- neering Education 95(4): 301â310. Chang, J.C. 2002. Women and Minorities in the Science, Mathematics, and Engineering Pipeline. ERIC Digest. Available online at http://eric.ed.gov/ERICDocs/data/ericdocs2sql/ content_storage_01/0000019b/80/1a/52/ec.pdf (accessed May 13, 2008). Cunningham, C., and M. Knight. 2004. Draw an Engineer Test: Development of a Tool to Investigate Studentsâ Ideas about Engineers and Engineering. Paper presented at the American Society for Engineering Education Annual Conference and Exposition. Salt Lake City, Utah, June 20â23, 2004. Available online at http://soa.asee.org/paper/ conference/paper-view.cfm?id=19444 (accessed January 16, 2009).
66 ENGINEERING IN Kâ12 EDUCATION Cunningham, C., C. Lachapelle, and A. Lindgren-Streicher. 2005. Assessing Elementary School Studentsâ Conceptions of Engineering and Technology. Paper presented at the American Society for Engineering Education Annual Conference and Exposition, Portland, Oregon, June 12â15, 2005. Available online at http://soa.asee.org/paper/ conference/paper-view.cfm?id=21451 (accessed January 16, 2009). Diaz, D. and P. King. 2007. Adapting a Post-Secondary STEM Instructional Model to Kâ5 Mathematics Instruction. Paper presented at the American Society of Engineering Education Annual Conference and Exposition, Honolulu, Hawaii, June 24â27, 2007. Available online at http://www.mathoutofthebox.org/aseepaperdiaz.pdf (accessed Janu- ary 15, 2009). DOEd (U.S. Department of Education). 1989. Report of the National Education Goals Panel. Washington, D.C.: DOEd. DOEd. 2000. Before Itâs Too Late: A Report to the Nation from the National Commission on Mathematics and Science Teaching for the 21st Century. Washington, D.C.: DOEd. Also available online at http://www.ed.gov/inits/Math/glenn/report.pdf (accessed May 2, 2008). DOEd. 2007. Report of the Academic Competitiveness Council. Washington, D.C.: DOEd. Also available at http://www.ed.gov/about/inits/ed/competitiveness/acc-mathscience/ index.html (accessed April 15, 2008). Dugger, W.E., Jr. 2007. The status of technology education in the United States: a triennial ï¬nding of the ï¬ndings from the states. The Technology Teacher 67(1): 14â21. Fortus, D., R.C. Dershimer, J.S. Krajcik, R.W. Marx, and R. Mamlok-Naaman. 2004. Design- based science and student learning. Journal of Research in Science Teaching 41(10): 1081â1110. Fortus, D., J.S. Krajcik, R.C. Dershimer, R.W. Marx, and R. Mamlok-Naaman. 2005. Design- based science and real-world problem-solving. International Journal of Science Edu- cation, 27(7): 855â879. Harris Interactive. 2004. American Perspectives on Engineers and Engineering. Poll con- ducted for the American Association of Engineering Societies. Final report, Feb- ruary 13, 2004. Available online at http://www.aaes.org/harris_2004_ï¬les/frame.htm (accessed July 6, 2007). Hill, A.M., and H. Smith. 1998. Practice meets theory in technology education: a case of authentic learning in the high school setting. Journal of Technology Education 9(2): 29â45. Hirsch, L.S., H. Kimmel, R. Rockland, and J. Bloom. 2005. Implementing Pre-engineering Curricula in High School Science and Mathematics. Paper presented at the 35th Annual Frontiers in Education Conference, Indianapolis, Indiana, October 19â22, 2005. Available online at http://ï¬e-conference.org/ï¬e2005/index.htm (accessed Janu- ary 15, 2009). Hotaling, L., B. McGrath, M. McKay, C. Shields, S. Lowes, C.M. Cunningham, et al. 2007. Engineering Our Future New Jersey. Paper presented at the American Society for Engineering Education Annual Conference and Exposition Proceedings, Honolulu, Hawaii, June 24â27, 2007. Available online at http://soa.asee.org/paper/conference/ paper-view.cfm?id=4611 (accessed January 15, 2009). Hubelbank, J., C. Demetry, S.E. Nicholson, S. Blaisdell, P. Quinn, E. Rosenthal, et al. 2007. Long-Term Effects of a Middle School Engineering Outreach Program for Girls: A
THE CASE FOR Kâ12 ENGINEERING EDUCATION 67 Controlled Study. Paper presented at the American Society for Engineering Education, Honolulu, Hawaii, June 24â27, 2007. Available online at http://soa.asee.org/paper/ conference/paper-view.cfm?id=4368 (accessed January 15, 2009). ITEA (International Technology Education Association). 2000. Standards for Technological Literacy: Content for the Study of Technology. Reston, Va.: ITEA. ITEA. 2004. The Second Installment of the ITEA/Gallup Poll and What It Reveals as to How Americans Think about Technology: A Report of the Second Survey Conducted by the Gallup Organization for the International Technology Education Association. Available online at http://www.iteaconnect.org/TAA/PDFs/GallupPoll2004.pdf (accessed December 22, 2008). Klein, S., and R.D. Sherwood. 2005. Biomedical engineering and cognitive science as the basis for secondary science curriculum development: a three year study. School Sci- ence and Mathematics 105(8): 384â401. Kolodner, J.L., P.J. Camp, D. Crismond, B. Fasse, J. Gray, and J. Holbrook. 2003. Problem- based learning meets case-based reasoning in the science classroom: putting learning- by-design into practice. Journal of the Learning Sciences 12(4): 495â547. Lachapelle, C.P., and C.M. Cunningham. 2007. Engineering is Elementary: Childrenâs Changing Understandings of Science and Engineering. Paper presented at the Ameri- can Society for Engineering Education Annual Conference and Exposition, Honolulu, Hawaii, June 24â27, 2007. Available online at http://soa.asee.org/paper/conference/ paper-view.cfm?id=3247 (accessed January 15, 2009). Landsberger, H.A. 1958. Hawthorne Revisited, Management and the Worker: Its Critics, and Developments in Human Relations in Industry. Ithaca, N.Y.: Cornell University Press. Lowell, L.B., and H. Salzman. 2007. Into the Eye of the Storm: Assessing the Evidence on Science and Engineering Education, Quality, and Workforce Demand. October 2007. Washington, D.C.: The Urban Institute. Lyons, J., and S. Thompson. 2006. Investigating the Long-Term Impact of an Engineering- Based GKâ12 Program on Studentsâ Perceptions of Engineering. 2006 ASEE Annual Conference and Exposition: Excellence in Education, Chicago, Illinois, June 18â21, 2006. Available online at http://soa.asee.org/paper/conference/paper-view.cfm?id=2184 (accessed January 15, 2009). McGrath, B., J. Sayres, S. Lowes, and P. Lin. 2008. Underwater LEGO Robotics as the Vehicle to Engage Students in STEM: The BUILD IT Projectâs First Year of Classroom Imple- mentation. Paper presented at the fall 2008 American Society for Engineering Edu- cation Mid-Atlantic Conference, October 18, 2008, Stevens Institute of Technology, Hoboken, N.J. Available online at http://www.stevens.edu/asee/ï¬leadmin/asee/pdf/Mc- Grath_-_ï¬nal.pdf (accessed January 15, 2009). McKay, M., and B. McGrath. 2007. Real-world problem-solving using real-time data. Inter- national Journal of Engineering Education 23(1): 36â42. NAE (National Academy of Engineering). 2002. Technically Speaking: Why All Americans Need to Know More About Technology. Washington, D.C.: National Academy Press. Also available online at http://www.nap.edu/openbook.php?record_id=10250 (accessed May 7, 2008). NAE. 2008. Changing the Conversation: Messages for Improving Public Understanding of Engineering. Washington, D.C.: The National Academies Press.
68 ENGINEERING IN Kâ12 EDUCATION NAE and NRC (National Research Council). 2006. Tech Tally: Approaches to Assessing Technological Literacy. Washington, D.C.: The National Academies Press. NAGB. (National Assessment Governing Board). 2008a. Science Framework for the 2009 National Assessment of Educational Progress. September 2008, Exhibit 18. Distribu- tion of items by science practice and grade, p. 113. Available online at http://www. nagb.org/frameworks/science-09.pdf (accessed September 19, 2008). NAGB. 2008b. Conducting a Special Study in Technological Literacy for the 2012 National Assessment of Educational Progress. Solicitation ED-08-R-0032. Available online at https://www.fbo.gov/download/a0d/a0dd95e5669a97e35fd90eedcc609428/ TECH_LIT_RFP.pdf (accessed September 19, 2008). NAS, NAE, and IOM (National Academy of Sciences, National Academy of Engineering, and Institute of Medicine). 2007. Rising Above the Gathering Storm: Energizing and Employing America for a Brighter Economic Future. Washington, D.C.: The National Academies Press. Oware, E., B. Capobianco, and H. Diefes-Dux. 2007. Gifted Studentsâ Perceptions of Engi- neers? A Study of Students in a Summer Outreach Program. Paper presented at the American Society of Engineering Education Annual Conference and Exposition, Honolulu, Hawaii, June 24â27, 2007. Available online at http://soa.asee.org/paper/ conference/paper-view.cfm?id=5401 (accessed January 15, 2009). Penner, D.E., N.D. Giles, R. Lehrer, and L. Schauble. 1997. Building functional models: designing an elbow. Journal of Research in Science Teaching 34(2): 125â143. Penner, D.E., R. Lehrer, and L. Schauble. 1998. From physical models to biomechanics: a design-based modeling approach. Journal of the Learning Sciences 7(3-4): 429â449. Ressler, S.J., and E.K. Ressler. 2004. Using a nationwide Internet-based bridge design con- test as a vehicle for engineering outreach. Journal of Engineering Education 93(2): 117-128. Roth, W.M. 2001. Learning science through technological design. Journal of Research in Science Teaching 38(7): 768â790. Sadler, P.M., H.P. Coyle, and M. Schwartz. 2000. Engineering competitions in the middle school classroom: key elements in developing effective design challenges. Journal of the Learning Sciences 9(3): 299â327. Satchwell, R.E., and F.L. Loepp. 2002. Designing and implementing an integrated math- ematics, science, and technology curriculum for the middle school. Journal of Indus- trial Teacher Education 39(3): 41â66. Seiler, G., K. Tobin, and J. Sokolic. 2001. Design, technology, and science: sites for learning, resistance, and social reproduction in urban schools. Journal of Research in Science Teaching 38(7): 746â767. Svihla, V., J. Marshall, and A.J. Petrosino. Unpublished. Kâ12 Engineering Education Impacts. Paper prepared for the National Academy of Engineering and National Research Council Committee on Kâ12 Engineering Education. Taylor, K.D., G.N. Foster, and M.M. Ratcliff. 2006. Project Lead the Way: Raising the Tech- nical Literacy of High School Graduates. Paper presented at the 9th International Conference on Engineering Education, San Juan, Puerto Rico, June 27âJuly 1, 2006. Available online at http://www.icee.usm.edu/icee/conferences/icee2006/papers/3603.pdf (accessed January 16, 2009).
THE CASE FOR Kâ12 ENGINEERING EDUCATION 69 Tran, N., and M. Nathan. In press. An investigation of the relationship between pre-college engineering studies and student achievement in science and mathematics. Journal of Engineering Education. Tran, N., and M. Nathan. In press. Effects of pre-college engineering studies on math- ematics and science achievements for high school students. International Journal of Engineering Education (Special Issue on Applications of Engineering Education Research). Walcerz, D. 2007. Report on the Third Year of Implementation of the TrueOutcomes Assessment System for Project Lead the Way. Available online at http://www.pltw. org/pdfs/AnnualReport-2007-Public-Release.pdf (accessed January 16, 2009). Zarske, M., J.L. Yowell, J.F. Sullivan, D. Knight, and D. Wiant. 2007. The TEAMS Program: A Study of a Grades 3â12 Engineering Continuum. Paper presented at the American Society of Engineering Education Annual Conference and Exposition, Honolulu, Hawaii, June 24â27, 2007. Available online at http://www.pltw.org/pdfs/AnnualReport- 2007-Public-Release.pdf (accessed January 16, 2009).