Click for next page ( 128


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 127
5 A Role for the Internet in American Education? Lessons from Cisco Networking Academies Richard Murnane, Nancy Sharkey, and Frank Levy As of July 2001, Cisco Networking Academies located in all 50 states and 130 countries were teaching more than 160,000 students how to design, build, and maintain computer networks. The students learn from a common curriculum offered that is in nine languages and delivered over the Internet. Since the majority of Cisco Net- working Academies in the United States reside in public high schools and community colleges institutions that educate the vast majority of American youth the team developing the Academy program con- fronted many of the same problems that beset American education. This paper describes how the Cisco Academies team dealt with these problems and particularly the use it made of information technology in crafting solutions. We begin by providing a brief description of changes in the American economy that pose new educational challenges. The next section explains why we chose to study the Cisco Networking Academy Pro- gram and the methodology of our case study. We follow this with a brief history of the Cisco Networking Academies. The fourth section provides the core of the paper and describes how the Cisco Academies Richard Murnane is the Thompson Professor of Education and Society at Harvard University's Graduate School of Education. Nancy Sharkey is a doctoral candidate at Harvard University Graduate School of Education. Frank Levy is the Daniel Rose Professor of Urban Economics in the Department of Urban Studies and Planning at M.I.T. Financial support for the research and writing of this paper was provided by the National Research Council and the Russell Sage Foundation. We would like to thank the many people connected to the Cisco Networking Academy Program who provided information and allowed us to observe their work. We appreciate the helpful comments on an earlier draft of this paper provided by Clifford Adelman, Lisa Lynch, and Linda Roberts. 127

OCR for page 127
team dealt with seven challenges that face American education. The last section provides a summary of key lessons. CHANGES IN THE AMERICAN ECONOMY In recent decades, changes in the American economy have pro- duced striking changes in the labor market earnings of American workers. Since 1979 the real earnings of workers with no postsecondary educa- tion, especially males, have declined markedly, both in absolute terms and relative to the earnings of workers with college degrees. Since 1970 the variation in earnings among workers with the same educa- tional credentials has increased. changes in the distribution of earnings reflect to a significant extent an increase in the demand for skills. The logic is that the college/ high school earnings differential has increased because college graduates are more likely to possess the skills employers increasingly demand, and the variation in earnings among workers with the same educa- tional credentials has increased because the premiums employers pay to specially skilled workers has increased (see Murnane, Willett, and Levy, 1995~. This "skills" explanation for the changes in the distri- bution of labor market earnings raises the obvious question: Which skills are increasingly valued by employers? A number of blue ribbon commissions (Secretary's Commission on Achieving Necessary Skills, 1991; National Institute for Literacy, 2000) and several research groups (Marshall and Tucker, 1991; Murnane and Levy, 1996), have produced lists. While varying in details, all lists include mathematics, reading, writing skills, problem-solving skills, computer skills, and the ability Most economists believe that the to work prociuct~ve~y with people from different backgrounds. Among the driving forces underlying current educational reform efforts is evidence from the National Assessment of Educational Progress (NAEP) that a great many American students leave high school with- out mastery of these critical skills. Of particular concern are patterns in NAEP test score data showing that children of color, the most rapidly growing groups in the U.S. student population and in the next generation's labor force, are especially likely to leave school lacking critical skills. While many factors including low family income con- tribute to the low average academic achievement of children of color, low-quality education plays a key role. In response to concern about skills, almost every state in the country is engaged in standards-based educational reforms aimed at improving the skills of American students. It is too early to know how much of a difference standards-based reforms will make to the skills of American students (see Murnane and Levy, 2001~. How- ever, it is clear that the reform efforts are encountering a number of significant problems. These include: Designing and implementing professional development efforts that increase teachers' effectiveness. 128 A ROLE FOR THE INTERNET IN AMERICAN EDUCATION?

OCR for page 127
Recruiting and retaining skilled teachers, especially in subject areas that pay well outside of teaching. 3. Retaining a focus on developing critical skills for all students and avoiding the plethora of disjointed programs that compete for resources and student attention and do not add up to a coherent educational experience. 4. Improving curricula and keeping them up-to-date. 5. Designing and administering assessments that not only measure students' mastery of critical skills but also provide incentives for teachers to improve instruction. 6. Maintaining high-quality instruction in all classrooms. 7. Providing students with credentials that employers and colleges value, thereby providing incentives for students to do the hard work that skill mastery requires. These problems are not new to American education. Yet their salience is particularly great today as schools struggle to provide all students with the critical skills they will need to earn enough to support families and to participate effectively in a changing demo- cratic society. Can new technology contribute to solving these difficult problems? Over the last 80 years, many innovators have thought so. In 1922 Thomas Edison stated: "I believe that the motion picture is destined to revolutionize our educational system and that in a few years it will supplant largely, if not entirely, the use of textbooks" (cited in Cuban, 1986, p. 9~. In the 1930s, enthusiasts touted radio as "textbooks of the air" (p. 19~. In the first decades after World War II, advocates saw instructional television as bringing the world to the classroom. More recently computers were seen by some as revolutionizing edu- cation. In 1984 the computer pioneer Seymour Papert wrote: "There won't be schools in the future.... I think the computer will blow up the school. That is. the school defined as something where there are . O classes, teachers running exams, people structured in groups bv age. following a curriculum all of that." A, ~ , A, Contrary to these prophecies, the educational historian Larry Cuban concludes that none of these new technologies have had a major impact on how education takes place in American schools. None have replaced the teacher and his or her daily work with 15 to 40 students in a physical classroom as the core technology of American education (Cuban, 1986~. The most recent attempt to improve schooling by making use of technology focuses on the Internet. Will the Internet be different from past technological innovations that did find their way into schools but did not revolutionize education? Only time will tell for sure. However, we can gain insights by examining how one group, the team that developed the Cisco Networking Academies, used the Internet to develop and implement a curriculum currently studied by more than 160,000 students around the world. RICHARD MURNANE, NANCY SHARKEY, AND FRANK LEVY 129

OCR for page 127
THE CISCO NETWORKING ACADEMY PROGRAM Why Study This Program? There were five reasons why we chose to study the Cisco Net- working Academies. First, the program has grown extraordinarily rapidly, passing the market test of whether a great many high schools, community colleges, and not-for-profit organizations find it valuable. Second, the program is aimed primarily at high school students and other people who do not have a four-year college degree. As such, it is an exception to the general pattern in the United States that the most in-depth training goes to workers who have the most formal education. Third, in the United States the program is delivered primarily in public high schools and community colleges, institutions central to the effort to prepare the next generation of Americans for life in a rapidly changing society. Understanding how the Academies program achieved such rapid growth within existing institutions may provide insights about ways to improve the performance of these institutions. Fourth, materials describing the program state that it is aligned with national skills standards. This is intriguing because it suggests the possibility that the Academies program may not only prepare students to build and maintain computer networks but also might teach more generic skills useful in other occupations. Finally, students who com- plete the program and pass examinations administered by an independent organization receive credentials that may improve access to good jobs. The opportunity to earn a credential is interesting because an inability to signal skill mastery to potential employers may be an important reason why many American high school students do not do the hard work that skill mastery requires (see Bishop, Mane, Bishop, and Moriarty, 2001 ). Until quite recently the Academy program consisted of a four- semester curriculum that prepares students to take the examination for the Cisco Certified Network Associate (CCNA) credential. Recently, new offerings have been added. First, a second four-semester sequence of courses now leads to a more advanced credential (the Cisco Certified Network Professional, or CCNP). The second new offering features partner-sponsored courses on the fundamentals of UNIX and Web design (developed by Sun Microsystems and Adobe Systems respectively, but delivered through the Cisco Networking Academy Program). We focus this paper solely on the initial four-semester sequence of courses. We want to be clear on the limits of this case study. We do not seek to evaluate the effectiveness of the Cisco Networking Academies in providing marketable skills to students. .. . . . . . . . ~ . . . . . . . . ~ In fact, Cisco does not collect data on students, demographic information and post-program outcomes that would be needed for a systematic evaluation of the program. We do not attempt to generalize from this case study about how education and training programs using the Internet work. We lack the resources to investigate systematically how different the Cisco Academies are from the many other information technology-related 130 A ROLE FOR THE INTERNET IN AMERICAN EDUCATION?

OCR for page 127
education and training programs (see Adelman, 2000, for a discussion of the dramatic growth in IT-related training programs). Despite these limitations we believe that a description of the develop- ment of the Cisco Academies program is valuable for two reasons. First, its extraordinary rate of growth prompts questions about how the program works and why it is appealing to high schools and com- munity colleges. Second, learning how the Academies team dealt with generic problems that hinder improvement in American educa- tion may provide ideas about how to deal with obstacles to progress. Methodology We began work on this case by learning as much as we could about the history and organization of the Academies program from materials available on the Internet. This made us realize the impor- tance of interviewing instructors in Local and Regional Academies, participating students, and Cisco personnel who played key roles in the development and administration of the program. We requested and received permission from Cisco Systems to conduct such inter- views. We then developed semi-structured interview protocols to guide the interviews. We interviewed instructors and students in six Local Academies and four Regional Academies (all terms defined below) in the New England area. Approximately half of the Academies were located in high schools, including two in central city school districts. One was located in a community-based organization. The rest were located in publicly supported community colleges. We also spent several hours observing classes in each of two Academies located in urban high schools and two located in community colleges. We spent two days at the Cisco Academies curriculum and assessment development center in Phoenix, interviewing the Cisco personnel who started the program and those currently responsible for curriculum, student assessment, and instructor training. We also conducted tele- phone interviews with several other Cisco employees and consultants working on the Academies program. We sent an early draft of this paper to the Cisco employees and Academy instructors whom we interviewed, requesting comments and corrections of factual errors. We used their feedback in revising the paper. GENESIS AND DEVELOPMENT Cisco Systems sells routers, switches, software, and advice on building and maintaining computer networks to organizations around the world, including educational institutions. In 1993 John Morgridge, then CEO of Cisco Systems, hired George Ward to help build Cisco's market in educational institutions. Ward, a consulting engineer who had been head of World Wide Networks at Motorola, was well suited RICHARD MURNANE, NANCY SHARKEY, AND FRANK LEVY 131

OCR for page 127
to the task. Not only did he understand networks; but he also liked talking to educators and helping them to design networks that met their schools' needs. Cisco's sales to schools increased. With the sales growth came an increasingly serious business problem. Many schools and school districts lacked the expertise to maintain the networks. Nor did they have the funds to contract out network main- tenance. In Ward's words, "I'd go in and design Enetworks] and build them and leave, and they would crash. There was limited support staff, and even more limited training" (Zehr, 1998~. Ward's response waLs to develop a 4()-hour training nro~ram aimed at teaching networking . .. . . . . .. . . . .. . . of. SKlllS to school personnel so they COU1d maintain their networks. -l-ne program included a PowerPoint presentation and a variety of hands- on activities. Ward spent much of the next year presenting the train- ing program at schools around the country. He quickly learned that his training program was popular; however, he also learned that he could not meet the burgeoning demand for training. Cisco needed another approach to supporting school districts' needs for networking expertise. An experience during Ward's year of travel provided an idea for a new approach. At some sites, schools allowed high school students to sit in on the training. Ward found that the students often grasped the critical skills more quickly than the adults. This led him to wonder whether Cisco should train students to maintain their schools' net- works. To test this idea, Ward asked the principal of Thurgood Marshall High School, an inner-city public school in San Francisco, if he could offer a one-week training program to students in August 1996. The principal, always looking for new ways to engage students, asked one of his teachers, Dennis Frezzo, to make the arrangements. Frezzo was an interesting choice. An optical and electrical engl- neer by background, Frezzo had decided in the early 1990s that he wanted to try teaching. He earned a master's degree in education from Stanford and started teaching at Thurgood Marshall in Septem- ber 1994. Over the next two years, he developed several hands-on courses aimed at developing students' interest in science. Because Frezzo knew which Thurgood Marshall students were interested in science, the school principal asked him to recruit students for the summer program. However, with only two weeks to recruit students, Frezzo found that most of the highly motivated students he had in mind were not available. This left him to dig quite deep into the pool of motivation and academic talent to recruit a group of students for Ward's training program. O - - - o 1 0 ~ . . _ _= _ _ _ -- ---- ------- --- -- -or - _ _ _ _ _ the training program, which was a mixture of Ward's PowerPoint presentation and many hands-on activities, worked well. Most sur- prisingly, Ward learned that some students at risk of dropping out due to a lack of interest in their academic programs were intrigued by his hands-on training program. This led him to return to Thurgood Marshall in December and propose that Frezzo and a colleague, Jai Gosine, try teaching a one-semester course for students. Frezzo and Gosine piloted 132 A ROLE FOR THE INTERNET IN AMERICAN EDUCATION?

OCR for page 127
the course in the spring of 1997, starting out with no curriculum and no equipment but lots of enthusiasm. The results were sufficiently positive to lead Ward to take the next steps. Ward took the idea of training students to John Morgridge, who was deeply interested in education and believed that technology could play a key role in improving it. While the CEO had some doubts about whether high school students could master the relatively complex skills needed to trouble-shoot network problems, he was intrigued by Ward's idea and agreed to provide corporate funding for the next steps. Now that Ward had support to develop his idea, he needed help. Recognizing that he knew little about how to develop a good educa- tional program, he recruited Alex Belous, the director of technology education for the state of Arizona, to work with him. Belous's career history was very different from Ward's. After graduating from college in 1973, Belous started teaching first grade in Cave Creek, a rural community 40 miles from Phoenix. While teaching elementary school children in Cave Creek over the next 12 years, Belous came to appre- ciate the difficulty rural school districts with very limited resources faced in providing students with a high-quality education. This became even more apparent to Belous when he became the guidance counselor at Cave Creek's first high school, built in 1985. A great many of Cave Creek's high school students were not going to college and their high school education did not prepare them to prosper in a changing economy. An entrepreneur at heart, Belous went to the Arizona State Department of Education and asked how he could get resources for his school to better serve these students. The advice was to develop a vocational education program. So Belous developed a program to teach computer-based office skills to Cave Creek high school students. This experience led him to believe that technology could play a role in improving education for rural children. From Cave Creek, Belous moved to the Arizona State Department of Education. Through this job, he met George Ward. Alex Belous was a great find for Ward. He understood curriculum development and the types of challenges high school teachers would face in teaching technological skills to students. He also understood state education bureaucracies. Perhaps most importantly, he shared Ward's interest in using technology to improve the quality of educa- tion provided to underserved children. Working initially as a Cisco consultant, Belous led the effort to turn Ward's 40-hour training program for adults into a curriculum for high school students. They agreed on four principles that would guide their effort: 1. The curriculum would be delivered online; there would be no printed version. 2. Well-trained instructors would teach the curriculum. 3. The curriculum would be updated frequently to keep it abreast of RICHARD MURNANE, NANCY SHARKEY, AND FRANK LEVY 133

OCR for page 127
include both content and pedagogy. changing technology and to improve it as students and instructors pointed out problems. 4. Assessment of student skills would be done online. Beyond these general principles, Ward and Belous had no master plan. When they began to work together, neither imagined that their ideas would develop over the next five years into an eight-semester program serving more than 160,000 students. Encouraged by the interest Thurgood Marshall students showed in the pilot program, Ward and Belous invited interested educators (many of whom had participated in Ward's training courses) to a meeting at Cisco headquarters in San Jose in April 1997. Ward laid out to the group the idea of an educational program for high school students. To bring the idea to life, Frezzo described what he and his colleague had learned while trying out the one-semester program at Thurgood Marshall High School. Ward then sketched out the possibility of a four-semester program with an online curriculum and online assessment of student skills. He emphasized that this would be a partnership. Cisco would not just provide the curriculum. It would also provide training on how to teach the curriculum and ongoing support. In return, it would hold educators accountable for student learning. He asked the partici- pants whether they would be interested in this partnership. While some participants were wary of the accountability concept, many expressed enthusiasm about the potential relationship. Ward and Belous concluded from the meeting that a significant number of high schools were interested in teaching courses on design- ing, building, and maintaining computer networks. Thus, they went ahead in developing the curriculum and recruiting school partners. When John Morgridge officially announced the Cisco Networking Academy Program in October 1997, 64 high schools in seven states were teaching the first semester of the Academies curriculum. Following the public announcement, the number of inquiries grew rapidly. While delighted by the interest, Ward and Belous quickly realized that they faced new questions. Who was going to train the instructors? What would be the financial arrangements of the partnerships? Over the next months, Ward and Belous and their collaborators worked out answers to these questions, while at the same time pushing ahead with the development of Semesters 2, 3, and 4 of the curriculum. The answer to the training question was to develop a highly leveraged "train the trainer" model, as illustrated in Figure 5-1. Training would The first instructors Ward had trained became the heads of Cisco Academy Training Centers (CATCs). Cisco would pay them to provide training to instructors at Regional Academies, who in turn would provide training to instructors at Local Academies. Initially Ward and Belous envisioned that the CATCs and Regional Academies, as well as the Local Academies, would all be in high schools. Soon, however, they found that community colleges were 134 A ROLE FOR THE INTERNET IN AMERICAN EDUCATION?

OCR for page 127
| Cisco Academy Training Center ~ | Regional Academy Regional Academy Local Academy Local Academy Local Academy Local Academy Local Academy Local Academy FIGURE 5-1 Cisco Networking Academies organizational structure. interested in becoming CATCs and Regional Academies and that state governments were interested in having community colleges play this role. By 2001, half of the Regional Academies in the United States were located in community colleges. Table 5-1 provides a summary of the types of institutions in the United States in which Local Academies and Regional Academies are located. Table 5-2 provides information on the worldwide growth in the number of Cisco Academies. TABLE 5-1 and Location Number of Cisco Acaclemies by Type of Acaclemy, Type of Institution, Within the United States Location Local Academies Regional and Local Regional Only High schools 2,424 67 20 Vocational programs [a] 337 23 1 Junior high schools (grades 7-9) 11 4 2 Two-year community colleges 486 211 20 Four-year colleges 101 35 7 Postsecondary technical schools 72 12 1 Nonprofit organizations 61 7 3 Military service institutions 50 0 1 Other types of organizations 82 19 22 U.S. Total 3,624 378 77 Outside United States Local Academies Regional and Local Regional only Non-U.S. Total 2,197 442 174 Worldwide Total 5,821 820 251 SOURCE: Compiled in April 2001 from information available on the Academy locator portion of the Cisco Academy Web site: http://cisco.netacad.net/cnacs/pub-doc/locator.shtml NOTE: From the available information, it is not possible to distinguish clearly between Cisco Academies in vocational high schools serving teenagers and Academies in institutions with the label of vocational education that serve adults. RICHARD MURNANE, NANCY SHARKEY, AND FRANK LEVY 135

OCR for page 127
TABLE 5-2 indicators of the Growth of Cisco Networking Academies WorIc3wicle Academy Statistics Past (FY '98) Current (FY '01 Future (Forecast) Local only 395 6,419 Regional / local 61 859 Subtotal local 396 7,278 Regional 184 271 CATCs 39 Total locations 580 7,559 10,140 Countries 8 130 SOURCE: Cisco Networking Academy Program (2000~. Over time the answer to the financial arrangements question emerged. Local Academies receive the online curriculum at no cost. Regional Academies may charge Local Academies for instructor training, on a cost-recovery basis. Local Academies need to buy from Cisco a rack of hardware containing five routers and two switches at a highly sub- sidized price of less than $10,000. Additional costs to the Local Academies may include several thousand dollars for testing equip- ment and disposable materials (such as cables). They are also re- quired to purchase from Cisco the "SmartNet" service contract for approximately $1,200 per year after the first year. (The cost of SmartNet is covered in the Lab Bundle cost in the first year.) This entitles the Academies to "24/7" technical support for rapid service on their rout- ers and switches. Exceptions to these financial arrangements are for Academies in economic empowerment zones, which are rural and urban areas desig- nated by the federal government as high unemployment areas, and for Academies in Native American communities. Cisco provides the package of routers and switches at no cost to the 100 Academies that are located in empowerment zones and approximately 13 Academies lo- cated in Native American communities. Regional Academies receive one set of routers and switches from Cisco at no cost. The Cisco Learning Institute, a public charity funded and formed by Cisco, pays for CATCs to provide training to two Regional Academy instructors. In return, Regional Academies as- sume three responsibilities. First, they must make a "good faith ef- fort" to recruit 10 Local Academies. Second, they provide instructor training and ongoing support to the Local Academies they recruit. Third, they monitor the quality of the Local Academy programs. Since the training, supporting, and monitoring functions use significant resources, Cisco allows Regional Academies to charge Local Academies annual support fees and fees for training sufficient to recover their costs. These fees are set by Regional Academies and range from zero to several thousand dollars per year. 136 A ROLE FOR THE INTERNET IN AMERICAN EDUCATION?

OCR for page 127
THE ACADEMY PROGRAM'S EXPERIENCE WITH AMERICAN EDUCATIONAL PROBLEMS Since most Cisco Networking Academies in the United States are located in public high schools and community colleges, the program confronts many of the challenges that face all attempts to improve the quality of education provided by these institutions. In this section we describe how the Cisco Academies program has dealt with some of these problems. Providing High-Quality Instructor Training Providing teachers with training (usually called professional develop- ment in education circles) that improves their effectiveness in help- ing students to master critical skills has been an ongoing challenge for American public education. While school districts typically de- vote significant resources to professional development, most of the money goes to fund one-day workshops that have little effect on how teachers teach (see Murnane and Levy, 1996, Ch. 7~. The Cisco Networking Academy Program faced a particularly difficult training challenge. Local Academies chose to teach the Cisco networking curriculum. Many instructors chosen by the Local Academies were high school teachers or community college instructors with considerable teaching experience but no knowledge of computer networking. Others were adjunct faculty drawn from industry who understood computer net- working but had no teaching experience. The challenge was to pro- vide both the technical knowledge needed to teach the curriculum and the teaching skills needed to motivate students and explain Quite difficult technical concepts. It had no control over the instructors The training program that the Academy team designed with the help of professional educators provided intensive hands-on instruc- tion, tests of skill mastery, and ongoing support. Newly designated Local Academy instructors receive eight days of full-time instruction on the first semester's curriculum at a Regional Academy. Participants study the same curriculum that they will subsequently teach. They are expected to have read the online curriculum before coming to the training and to review the relevant chapters each night during the training. Participants do the same hands-on labs their students will do and they take the same Internet-based end-of-chapter tests aimed at pro- viding feedback on their skill mastery. The training guidelines specify that each participant must do a "teach-back" in each semester. The teach-back consists of preparing a lesson plan for a particular chapter and teaching the chapter to other participants and the Regional Academy instructor. In order to receive authorization to teach the first semester curriculum, participants must pass the online final examination with a score of at least 80, as well as successfully complete the teach- RICHARD MURNANE, NANCY SHARKEY, AND FRANK LEVY 137

OCR for page 127
answer the same questions at the end of each chapter and at the end of each semester, software randomly assigns the order of the questions for each test taker. Instructors value the random ordering because it increases the likelihood that student's responses reflect their own knowl- edge. Student's responses to the multiple-choice questions go directly to the Cisco Academy server in Arizona and students receive back their exam grade in a minute or two. The exam grades for each student are automatically entered into a spreadsheet that the instructor can access to check on student progress and to assign final course grades. Student responses on individual test items are recorded on the Academy server and are analyzed by the assessment team to identify problematic ques- tions. Since the exams are provided to students online from the Acad- emy server, the assessment team can easily test new questions and replace inadequate questions with better ones. The assessment team uses two strategies to align the assessments with curriculum goals. First, the person who prepares each exam question documents the particular curriculum goal that the question addresses. Then each item goes through a multistage quality review. Second, the end-of-course multiple-choice exam for each semester is complemented by a hands-on practical examination. While individual instructors have discretion in designing the hands-on examinations, the instructor's guide outlines the types of tasks that should be assessed. For example, the task for the second semester is to build a network that uses five routers. The hands-on exam for the fourth semester is comprehensive in that it assesses mastery of skills taught in all four semesters. For example, students are asked to use IF addresses (Semester 1), apply a routing protocol (Semester 2), make an access list (Semester 3), and construct a wide area network (Semester 4~. Students must pass the hands-on examination as well as achieve a satisfactory score (as defined by the instructor) on the end-of-course multiple-choice exam in order to receive credit for the course. Behrens has several initiatives under way to improve the Academy assessment program and to incorporate technological advances devel- oped by Educational Testing Service and other organizations on the frontier of test development. One initiative will overcome the current limitation that students only learn their score on each examination, not which items they answered incorrectly. In the future, feedback will not only include identification of questions answered incorrectly, but also links to the relevant sections of the curriculum and to addi- tional resources. A second initiative is the development of multiple forms of each end-of-semester and chapter test, with comparable scor- ing across the different forms. This will allow instructors to use different forms of the tests for different purposes. A third initiative is the development of new online assessment formats, including micro- simulations that require students to program emulated routers. A fourth is the development of adaptive testing that tailors the difficulty of the 148 A ROLE FOR THE INTERNET IN AMERICAN EDUCATION?

OCR for page 127
test items to the skill of the test taker. This permits more accurate assessment of the skills of individual students. Maintaining High-Quality Instruction A premise underlying standards-based educational reforms is that all children will receive consistently good instruction. Indeed, fulfilling this premise is necessary to justify withholding grade promotions and high school diplomas from students who do not score well on state- mandated standardized assessments. Yet a great many studies over the last 30 years have documented that the quality of teaching in American schools varies enormously.8 Assuring that all students enrolled in Cisco Networking Academies receive consistently good instruction has been a goal of the program since its inception. In fact, the one uneasy response to George Ward's initial presentation of the Academy concept to a group of educators in spring 1997 dealt with accountability. Educators wanted to know what Ward meant by the term and what responsibilities they would incur if they signed up to host a Local Academy. The structural design of the Academy program makes quality assurance a particularly difficult challenge. Administration is very decentralized. CATCs are responsible for monitoring the quality of Regional Academies; in turn, the Regional Academies are responsible for monitoring the quality of Local Academies. Cisco does not select the instructors in Local and Regional Academies, nor does it pay them. The only sanction Cisco can impose on a Local Academy that does not provide consistently high-quality instruction is to work with the Regional Academy to deny program affiliation. As a result, the primary strat- egy for assuring instructional quality is to identify problems as quickly as possible and then to offer a variety of opportunities for improving instruction. The Academy program makes extensive use of information col- lected online in identifying problems. Student's end-of-semester exam scores provide one source of information. The Academy Quality Assurance Team monitors the distribution of scores in each class, looking for classes in which a large percentage of students earn low scores. Student surveys provide a second source of information. Every student completes an online survey at the end of every semester that asks for ratings of the instructor, the curriculum, and the course assignments. Instructors can learn about their performance by accessing online the average rating their students gave on each of the 15 ques- ~Researchers from different disciplines using different methods have documented the wide variance in teaching quality in American schools. See Hanushek (1994) for a discussion of the evidence by economists. See Grossman (1990) for evidence on differences in effectiveness among English teachers and Mayer (1998) for a discus- sion of differences among math teachers. RICHARD MURNANE, NANCY SHARKEY, AND FRANK LEVY 149

OCR for page 127
tions in the survey. (See Table 5-3 for an example of feedback to an instructor from student surveys.) The single number describing the average rating students gave on the 15 survey questions is sent to the Regional Academy overseeing the particular Local Academy and to the Academy Quality Assurance team. In addition, participants in instructor training complete online evalu- ations of the quality of their training. The survey asks participants to rate the class content ("The order of course topics aided my learn- ing"), as well as the instructor's pedagogical style ("Class participation was enhanced through effective use of questions") and content knowl- edge ("Analogies and real-life experiences of the instructor added value to the course". These ratings are available to the trainer, and a summary of these ratings is also transmitted to the Quality Assurance Team. The Academy team also uses the Internet to evaluate com- plaints about training quality. When the director of training receives an e-mail message complaining about the quality of a training pro- gram, he sends an online survey to all participants in that particular training class. The responses help him to judge whether the initial e- mail reflects a significant training problem or an isolated personality conflict. A primary mechanism for quality assurance is the annual audit. A three-person team (including one consultant with expertise in peda- gogy, a second Know~eagean~e anout tecnno~ogy, and one member of the Cisco Academy team) audits each CATC. It inquires whether the CATC fulfilled its contractual obligation to conduct annual audits of the Regional Academies under it. Part of this process involves learn- ing whether the Regional Academies conducted audits of the Local Academies under them. The auditors of CATCs also examine the student test score distributions and average student survey grades for all Regional and Local Academies under the particular CATC. The auditing team asks whether the CATC had identified patterns of low student scores or poor class ratings. It also explores whether the CATC brought the potential problem to the attention of the relevant Regional Academy and whether it followed up to see whether the Regional Academy worked with the relevant instructor to diagnose the source of the problem and develop a remediation plan.9 Another part of each audit is to check that instructors of all Academy courses have kept their CCNA credential up-to-date, a process requiring them to pass the CCNA exam every three years. This requirement that instructors periodically demonstrate mastery of subject matter content is a bone of contention with many instructors. To date, however, the program has stuck by this requirement. _ _ . . . . 90f course, there are several reasons students could have low scores on the end-of- course examinations, including low reading skills, inadequate facilities, and poor teaching. The Regional Academy is responsible for exploring the reason for low scores among students in a Local Academy in its jurisdiction and for suggesting improvement strategies. 150 A ROLE FOR THE INTERNET IN AMERICAN EDUCATION?

OCR for page 127
TABLE 5-3 Sample of Course Feedback Results Min Max Mean Question 5 5 5 5 5 5 5 4.85 4.77 4.54 4.54 4.23 4.69 4.69 4.69 4.92 4.15 3.85 4.15 4.38 4.31 4.62 The instructor was adequately prepared to teach this course. Analogies and real-life experiences of the instructor added value to this course. Presentations were clear and easy to understand. Answers to questions were provided in a timely manner. Class participation was enhanced through effective use of questions. The class was interesting and enjoyable. "Best Practices" and good teaching strategies were modeled during the training. Grouping strategies were utilized effectively. Class members felt comfortable approaching the instructor with questions/ ideas. The order of course topics aided my learning. The course schedule allowed me to complete the stated course objectives. The activities and labs helped me achieve the stated course objectives. The lesson assessment tools helped me evaluate my knowledge of the lesson. Group work aided my learning. Overall, the course materials were of high quality. SOURCE: Cisco Academy instructor. Quality assurance in the Cisco Academy program depends criti- cally on the efforts of Regional Academies, the majority of which in the United States are in community colleges. While Cisco allows Regional Academies to recover the costs they incur in supporting Local Academies and training their instructors, and a few charge an annual support fee as high as $15,000, many charge little or nothing. The rationale, several Regional Academy instructors told us, is that state legislatures expect community colleges to serve as community assets. Supporting Cisco Local Academies in area high schools is one way of demonstrating that commitment. Of course, this arrange- ment means that the quality of support Local Academies receive and the integrity of the quality assurance plan depend critically on the good will of the community colleges and high schools that serve as Regional Academies. Signaling Skills Employers in the United States find it very difficult to obtain reliable information on the skills of high school graduates who apply for jobs. The high school diploma indicates only that students com- pleted four years of high school, not that they mastered particular skills. High schools typically do not respond quickly to requests for student transcripts. A consequence of the lack of information on the skills of high school graduates is that employers typically offer the same wages and conditions of employment to high school graduates with strong skills that they offer to graduates with much weaker skills. RICHARD MURNANE, NANCY SHARKEY, AND FRANK LEVY 151

OCR for page 127
Bishop et al. (2001) have argued that this hiring pattern reduces high school students' incentives to do the hard work that skill mastery requires. Graduates of the four-semester Cisco Networking Academy may obtain the Cisco Certified Networking Associate (CCNA) credential by passing the same examination that graduates of commercial train- ing institutes take. The CCNA exam is a multiple-choice exam administered by an independent testing organization, Prometric.~ Academy graduates may also obtain the vender-neutral Network+ credential by passing a different examination offered by an industry trade organization, CompTIA. We have no way of assessing whether the opportunity to acquire industry- recognized credentials has motivated students to enroll in Academy classes and to work hard to acquire critical skills. However, many Academy students in community colleges told us that their reason for enrolling in the Academy courses was to prepare for the CCNA exam. Credentials that can be earned by passing examinations predate the Internet by many years. However, the online format of the CCNA exams mean that scoring is done almost instantaneously and the cre- dential can be awarded much more quickly than if paper answer sheets were sent to a central scoring office. This may increase the attractive- ness of the credential and the Academy program to potential students. Economists have two questions about the economic value of credentials like the CCNA. The first is whether it permits the recipient to earn more than he or she otherwise would have earned. The second is whether any wage premium that is earned stems from skills the individual acquired while studying for the credential or whether the premium stems from skills that the individual already had before obtaining the credential but could not signal to employers in the absence of the credential. It has proven extremely difficult to answer these questions for most education and training credentials, and the CCNA credential is no exception in this regard. One survey reports that recipients of the CCNA credential earn approximately 10 percent more than IT workers without this credential (Gabelhouse, 2000~. However, since individuals voluntarily decide whether to try to obtain the credential, it is possible that those who do obtain it differ from those workers who do not in dimensions such as motivation that affect earnings. Consequently the earnings differences could stem from differences between workers who obtain the CCNA and those who do not that are not observed by the researcher but are observed by employers. Thus, all that can be said is that the survey evidence is consistent with community college students' perception that the CCNA credential brings significant economic benefits. Was explained by Adelman (2000), the CCNA is one of a great many IT creden- tials that can be obtained by passing examinations, typically administered by one of three large organizations (including Prometric). The examinations for some creden- tials include hands-on performance assessments. 152 A ROLE FOR THE INTERNET IN AMERICAN EDUCATION?

OCR for page 127
To learn more about employers' treatment of the CCNA creden- tial, we conducted a computer search for job postings that listed it as required or desired. We found many such jobs, with annual salaries ranging from $35,000 to $70,000. However, as illustrated in Box 5-1, almost all of the jobs, even those paying the lowest salaries, required RICHARD MURNANE, NANCY SHARKEY, AND FRANK LEVY 153

OCR for page 127
a minimum of one year of experience in computer networking. Adelman (2000) has documented that this is the common pattern for IT jobs. To help solve the experience problem, Cisco has begun a process of matching Academy students and graduates desiring work experience with employers who want help with particular projects. Students in- terested in jobs or internships post resumes on the Academy Web site. (If a student is under 18, he or she is identified only by ID number, not by name.) Employers who want student help post internship and job listings on a Cisco Web site (http://wpl.netacad.net). They imme- diately receive a list of all Academies within 50 miles. The employer then requests that the help-wanted listing be sent to all nearby Acad- emies or to a selected subset. Academy instructors then make avail- able to employers the resumes of those students whose skills seem to fit the job requirements. Employers may contact students at least 18 years of age directly. Academy instructors arrange interviews for students under the age of 18. This example illustrates two points. The Internet can be a valuable resource in improving matchmaking between employers and students (Autor, 2001~. By increasing students' opportunities to gain work experience that is a necessary qualification for most jobs, the Internet may improve incentives for students to do the hard work that skill mastery requires. . . lo. The second point is quite different. Considerable attention is needed to assure that the enormous search potential of the Internet does not result in violations of federal and state laws regarding the privacy of information pertaining to minors. Concern with these laws explains why Cisco collects no information on the characteristics of Academy students, not even names. LESSONS The Academy program differs from state standards-based reform efforts in two notable respects. First, it has a common national cur- riculum. While the merits of this approach can be debated, one ad- vantage we observed is that mobile students find no difficulty in ob- taining credit in one state for Cisco Academy courses completed in another state. Another difference is that aspiring instructors must demonstrate mastery of the curriculum they will teach by passing the same end-of- semester exams that the students take and by passing the national CCNA exam every three years. While this does not guarantee that teachers know how to teach the curriculum, it does provide some assurance of subject matter mastery. The brief history of the Cisco Networking Academies demonstrates that the Internet has the potential to be a valuable resource for improving education. For the Academies, the Internet has been important in developing and improving curriculum, in distributing an up-to-date curriculum to underserved populations, in assessing student skills, in 154 A ROLE FOR THE INTERNET IN AMERICAN EDUCATION?

OCR for page 127
monitoring the quality of instruction, and in providing teachers with advice on technical and pedagogical issues and with a resource for keeping track of student progress and exam grades. At the same time, the Academies' experience shows that technology is not a substitute for highly motivated teachers with command of their subject matter and with the pedagogical skills to help students master critical skills. To the credit of George Ward and Alex Belous, they never intended that online curriculum and assessment would substitute for first-rate instructors. From the beginning they envi- sioned the Cisco Academies program as blending online resources with high-quality classroom instruction. The brief history of the Academies program given here shows that technology cannot insulate high schools and community colleges from the challenge of attracting and retaining talented teachers who typically are offered salaries that are considerably lower than those they could earn in other jobs. Another lesson is that it takes time to figure out how best to make use of technology. For example, the Academies team has learned that it would be better to have students fill out the course satisfaction surveys before they receive their end-of-course exam grades rather than after. The reason is that analysis of survey responses suggests that students who do not do well on the final exam are particularly critical of the course and the instructor. The team has also learned that retaining for auditing purposes only the single number that repre- sents the average rating students in a class give on all 15 rating questions is a mistake. While a low average grade may identify a quality problem, average responses on individual questions are needed to diagnose the source of the problem. The Academy team also learned that there is a trade-off between keeping the curriculum up- to-date and keeping it stable so that instructors can make lesson plans ahead of time. Too frequent curriculum revisions create more prob- lems than they solve. These examples illustrate that even with a team that included experienced professional educators and experts on information tech- nology, figuring out how to make the most effective use of the Internet to improve education required much trial and error. The important gains do not come from simply substituting online resources for books. They come from developing new teaching and learning roles. For example, in the Academy program the most effective instructors are not lecturers who spend most class time talking while students take notes. They are coaches who engage students in learning from each other and in developing the confidence to assume responsibility for their learning and for commenting constructively on the quality of instruction. For many teachers and students, these are new roles that are not easy to assume. Figuring out how to bring about these cul- tural changes and how to use the Internet to facilitate these changes is a slow process. A final question to consider is whether the Internet is different from the many technological innovations radio, instructional films, RICHARD MURNANE, NANCY SHARKEY, AND FRANK LEVY 155

OCR for page 127
TV formerly seen as transforming education. In perhaps the most critical respect, the Internet is not different in that it is not a substitute for effective interaction of teachers with students. In other respects, the Internet is somewhat different from previous generations of instructional technologies. The Internet facilitates two-way communication. Not only can Local Academy students and teachers receive exams and new versions of the curriculum over the Internet but also they can send back to the central Academy team information critical to evalu- ating the quality of the exams and the curriculum. A second difference is that the Internet markedly increases the ability of teachers and students to search for information that can be useful in solving problems and constructing arguments. In so doing, it reduces the importance of memorizing facts that can be quickly retrieved from online sources. It also dramatically increases the importance of learning to search efficiently, to sort through the vast amount of information the Web can make available, and to synthesize and make sense of an abundance of information (NRC, 1999~. It is too early to tell exactly how the Internet will alter the demand for particular skills or the capacity of human societies to teach the critical skills. However, the experience of the Cisco Networking Academies suggests that it is a mistake to see technology as substituting for effective teachers in helping students to learn. Instead, the appropriate perspective is that technology will be most valuable in complementing the skills of highly motivated teachers who know their subject matter well and know how to engage students in learning. REFERENCES Adelman, C. (2000~. A parallel postsecondary universe: The certification system in information technology. Washington, DC: U.S. Department of Education, Office of Educational Research and Improvement. Autor, D.H. (2001~. Wiring the labor market. Journal of Economic Perspectives, 15, 35-50. Bishop, J.H. (1995). Expertise and excellence. Ithaca, NY: Cornell University, School of Industrial Relations, Center for Advanced Human Resource Studies. Bishop, J.H., Mane, F., Bishop, M., and Moriarty, J. (2001~. The role of end-of- course exams and minimal competency exams in standards-based reforms. In D. Ravitch (Ed.), Brookings papers in education policy 2001. Washing- ton, DC: The Brookings Institution. Cisco Networking Academy Program. (2000~. Curriculum alignment document. San Jose, CA: Cisco Systems. Cuban, L. (1986~. Teachers and machines: The classroom use of technology since 1920. New York: Teachers College Press. Gabelhouse, G. (2000, DecemberJ. CertMag's salary survey. Certification Magazine. Available: http://www.certmag.com/issues/decO0 [December 12, 2000]. Grossman, P.L. (1990~. The making of a teacher: Teacher knowledge and teacher education. New York: Teachers College Press. Hanushek, E.A. (1994~. Making schools work: Improving performance and control- ling costs. Washington, DC: The Brookings Institution. Johnstone, W.B., and Packer, A.E. (1987~. Workforce 2000: Work and workers for the twenty-first century. Indianapolis, IN: Hudson Institute. 156 A ROLE FOR THE INTERNET IN AMERICAN EDUCATION?

OCR for page 127
Kershaw, J.A., and McKean, R.N. (1962~. Teacher shortages and salary schedules. NewYork: McGraw-Hill. Marshall, R., and Tucker, M. (1991~. Thinking for a living: Education and the wealth of nations. New York: Basic Books. Mayer, D.P. (1998~. Do teaching standards undermine performance on old tests? Educational Evaluation and Policy Analysis, 20, 53. Murnane, R.J., and Levy, F. (1996~. Teaching the new basic skills. New York: Free Press. Murnane, R.J., and Levy, F. (2001~. Will standards-based reforms improve the education of students of color? National Tax Journal, 54, 401-415. Murnane, R.J., Willett, J.B. and Levy, F. (1995~. The growing importance of cogni- tive skills in wage determination. Review of Economics and Statistics 77, 25 1-266. National Research Council. (1999~. How people learn: Brain, mind, experience, and school. Committee on Developments in the Science of Learning. Bransford, J.D., Brown, A.L., and Cocking, R.R. (Eds.~. Commission on Behavioral and Social Sciences and Education. Washington, DC: National Academy Press. National Institute for Literacy. (2000~. Equipped for the future content standards: What adults need to know and be able to do in the 21st century. Washington, DC: Author. Parrado, E., and Wolff, E. (1999~. Occupational and industrial mobility in the United States, 1969-92. New York: New York University. Secretary's Commission on Achieving Necessary Skills. (1991~. What work requires of schools: A SCANS reportfor America 2000. Washington, DC: U.S. Department of Labor. Zehr, M.A. (1998~. Computer giants look to students. Education Week, April 15. RICHARD MURNANE, NANCY SHARKEY, AND FRANK LEVY 157

OCR for page 127
;- learning with understanding. The Jenkins framework reminds us that a change in learning goals and assessments often requires a change in teaching and learning strategies as well. In order to learn with understanding, students need to understand why veins and arteries have certain characteristics. For example, arteries carry blood Dom the heart, blood that is pumped in spurts. This helps explain why they would need to be elastic (to handle the spurts). In addition, arsenal blood needs to travel uphill (to the brain) as well as downhill, so the elasticity of the arteries provides an additional advantage. If they constrict behind each uphill spurt, they act as a type of one-way valve that keeps the blood from flowing downhill. Learning to understand relationships such as why artenes are elastic should facilitate subsequent transfer. For example, imagine that students are asked to design an artificial artery. Would it have to be elastic? Students who have only memonzed that arteries are elastic have no grounded way to approach this problem. Students who have learned with understanding know the Unctions of elasticity and hence are freer to consider possibilities like a non-elastic artery that has one-way valves (Bransford and Stein, 19931. Overall, this example illustrates how memorizing versus understanding represent different reaming goals in the Jenkins framework, and how changes in these goals require different types of teaching strategies. The details of one's teaching strategies will also need to vary depending on the knowledge, skills, attitudes, and other characteristics that students bring to the learning task. For example, we noted earlier that some students (e.g., those in the lower grades) may not know enough about pumping, spurts, and elasticity to learn with understanding if they are simply told about the functions of artenes. They may need special scaffolds such as dynamic simulations that display these properties. As a different kind of example, imagine that we want to include mnemonics along with understanding and one of the students in our class is overweight and named Art. Under these coalitions, it would seem unwise to use the mnemonic sentence about Artistry) that was noted above. The Importance of Workina Backwards lo The Jenkins model fits well with a recent book by Wiggins and McTighe entitled Understanding by Design (1997). They suggest a C`working backwards,, strategy for creating high- quality learning experiences. ~ particular, they recommend that educators (~) begin with a careful analysis of reaming goals; (2) explore how to assess students' progress in achieving these goals; and (3) use the results of ~ and 2 to choose and continually evaluate teaching methods. (Assumptions about steps ~ and 2 are also continually evaluated.) When using a "working backwards" strategy, one's choice of teaching strategies denves Tom a carefill analysis of reaming goals, rather than vice versa. In the discussion below, we attempt to clarify the importance of working backwards by discussing some imaginary universities that each set different goals for their students. These different goals have strong effects on what and how the universities teach. The Knowledge Economy and Postsecondary Education: Report of a Workshop Chapter 6 158