The Knowledge Economy and Postsecondary Education: Report of a Workshop (2002)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

5 A Role for the Internet in American Education? Lessons from Cisco Networking Academies Richard Murnane, Nancy Sharkey, and Frank Levy As of July 2001, Cisco Networking Academies located in all 50 states and 130 countries were teaching more than 160,000 students how to design, build, and maintain computer networks. The students learn from a common curriculum offered that is in nine languages and delivered over the Internet. Since the majority of Cisco Net- working Academies in the United States reside in public high schools and community colleges institutions that educate the vast majority of American youth the team developing the Academy program con- fronted many of the same problems that beset American education. This paper describes how the Cisco Academies team dealt with these problems and particularly the use it made of information technology in crafting solutions. We begin by providing a brief description of changes in the American economy that pose new educational challenges. The next section explains why we chose to study the Cisco Networking Academy Pro- gram and the methodology of our case study. We follow this with a brief history of the Cisco Networking Academies. The fourth section provides the core of the paper and describes how the Cisco Academies Richard Murnane is the Thompson Professor of Education and Society at Harvard University's Graduate School of Education. Nancy Sharkey is a doctoral candidate at Harvard University Graduate School of Education. Frank Levy is the Daniel Rose Professor of Urban Economics in the Department of Urban Studies and Planning at M.I.T. Financial support for the research and writing of this paper was provided by the National Research Council and the Russell Sage Foundation. We would like to thank the many people connected to the Cisco Networking Academy Program who provided information and allowed us to observe their work. We appreciate the helpful comments on an earlier draft of this paper provided by Clifford Adelman, Lisa Lynch, and Linda Roberts. 127

team dealt with seven challenges that face American education. The last section provides a summary of key lessons. CHANGES IN THE AMERICAN ECONOMY In recent decades, changes in the American economy have pro- duced striking changes in the labor market earnings of American workers. Since 1979 the real earnings of workers with no postsecondary educa- tion, especially males, have declined markedly, both in absolute terms and relative to the earnings of workers with college degrees. Since 1970 the variation in earnings among workers with the same educa- tional credentials has increased. changes in the distribution of earnings reflect to a significant extent an increase in the demand for skills. The logic is that the college/ high school earnings differential has increased because college graduates are more likely to possess the skills employers increasingly demand, and the variation in earnings among workers with the same educa- tional credentials has increased because the premiums employers pay to specially skilled workers has increased (see Murnane, Willett, and Levy, 1995~. This "skills" explanation for the changes in the distri- bution of labor market earnings raises the obvious question: Which skills are increasingly valued by employers? A number of blue ribbon commissions (Secretary's Commission on Achieving Necessary Skills, 1991; National Institute for Literacy, 2000) and several research groups (Marshall and Tucker, 1991; Murnane and Levy, 1996), have produced lists. While varying in details, all lists include mathematics, reading, writing skills, problem-solving skills, computer skills, and the ability Most economists believe that the to work prociuct~ve~y with people from different backgrounds. Among the driving forces underlying current educational reform efforts is evidence from the National Assessment of Educational Progress (NAEP) that a great many American students leave high school with- out mastery of these critical skills. Of particular concern are patterns in NAEP test score data showing that children of color, the most rapidly growing groups in the U.S. student population and in the next generation's labor force, are especially likely to leave school lacking critical skills. While many factors including low family income con- tribute to the low average academic achievement of children of color, low-quality education plays a key role. In response to concern about skills, almost every state in the country is engaged in standards-based educational reforms aimed at improving the skills of American students. It is too early to know how much of a difference standards-based reforms will make to the skills of American students (see Murnane and Levy, 2001~. How- ever, it is clear that the reform efforts are encountering a number of significant problems. These include: Designing and implementing professional development efforts that increase teachers' effectiveness. 128 A ROLE FOR THE INTERNET IN AMERICAN EDUCATION?

Recruiting and retaining skilled teachers, especially in subject areas that pay well outside of teaching. 3. Retaining a focus on developing critical skills for all students and avoiding the plethora of disjointed programs that compete for resources and student attention and do not add up to a coherent educational experience. 4. Improving curricula and keeping them up-to-date. 5. Designing and administering assessments that not only measure students' mastery of critical skills but also provide incentives for teachers to improve instruction. 6. Maintaining high-quality instruction in all classrooms. 7. Providing students with credentials that employers and colleges value, thereby providing incentives for students to do the hard work that skill mastery requires. These problems are not new to American education. Yet their salience is particularly great today as schools struggle to provide all students with the critical skills they will need to earn enough to support families and to participate effectively in a changing demo- cratic society. Can new technology contribute to solving these difficult problems? Over the last 80 years, many innovators have thought so. In 1922 Thomas Edison stated: "I believe that the motion picture is destined to revolutionize our educational system and that in a few years it will supplant largely, if not entirely, the use of textbooks" (cited in Cuban, 1986, p. 9~. In the 1930s, enthusiasts touted radio as "textbooks of the air" (p. 19~. In the first decades after World War II, advocates saw instructional television as bringing the world to the classroom. More recently computers were seen by some as revolutionizing edu- cation. In 1984 the computer pioneer Seymour Papert wrote: "There won't be schools in the future.... I think the computer will blow up the school. That is. the school defined as something where there are . O classes, teachers running exams, people structured in groups bv age. following a curriculum all of that." A, ~ , A, Contrary to these prophecies, the educational historian Larry Cuban concludes that none of these new technologies have had a major impact on how education takes place in American schools. None have replaced the teacher and his or her daily work with 15 to 40 students in a physical classroom as the core technology of American education (Cuban, 1986~. The most recent attempt to improve schooling by making use of technology focuses on the Internet. Will the Internet be different from past technological innovations that did find their way into schools but did not revolutionize education? Only time will tell for sure. However, we can gain insights by examining how one group, the team that developed the Cisco Networking Academies, used the Internet to develop and implement a curriculum currently studied by more than 160,000 students around the world. RICHARD MURNANE, NANCY SHARKEY, AND FRANK LEVY 129

THE CISCO NETWORKING ACADEMY PROGRAM Why Study This Program? There were five reasons why we chose to study the Cisco Net- working Academies. First, the program has grown extraordinarily rapidly, passing the market test of whether a great many high schools, community colleges, and not-for-profit organizations find it valuable. Second, the program is aimed primarily at high school students and other people who do not have a four-year college degree. As such, it is an exception to the general pattern in the United States that the most in-depth training goes to workers who have the most formal education. Third, in the United States the program is delivered primarily in public high schools and community colleges, institutions central to the effort to prepare the next generation of Americans for life in a rapidly changing society. Understanding how the Academies program achieved such rapid growth within existing institutions may provide insights about ways to improve the performance of these institutions. Fourth, materials describing the program state that it is aligned with national skills standards. This is intriguing because it suggests the possibility that the Academies program may not only prepare students to build and maintain computer networks but also might teach more generic skills useful in other occupations. Finally, students who com- plete the program and pass examinations administered by an independent organization receive credentials that may improve access to good jobs. The opportunity to earn a credential is interesting because an inability to signal skill mastery to potential employers may be an important reason why many American high school students do not do the hard work that skill mastery requires (see Bishop, Mane, Bishop, and Moriarty, 2001 ). Until quite recently the Academy program consisted of a four- semester curriculum that prepares students to take the examination for the Cisco Certified Network Associate (CCNA) credential. Recently, new offerings have been added. First, a second four-semester sequence of courses now leads to a more advanced credential (the Cisco Certified Network Professional, or CCNP). The second new offering features partner-sponsored courses on the fundamentals of UNIX and Web design (developed by Sun Microsystems and Adobe Systems respectively, but delivered through the Cisco Networking Academy Program). We focus this paper solely on the initial four-semester sequence of courses. We want to be clear on the limits of this case study. We do not seek to evaluate the effectiveness of the Cisco Networking Academies in providing marketable skills to students. .. . . . . . . . ~ . . . · . . . . · . ~ In fact, Cisco does not collect data on students, demographic information and post-program outcomes that would be needed for a systematic evaluation of the program. We do not attempt to generalize from this case study about how education and training programs using the Internet work. We lack the resources to investigate systematically how different the Cisco Academies are from the many other information technology-related 130 A ROLE FOR THE INTERNET IN AMERICAN EDUCATION?

education and training programs (see Adelman, 2000, for a discussion of the dramatic growth in IT-related training programs). Despite these limitations we believe that a description of the develop- ment of the Cisco Academies program is valuable for two reasons. First, its extraordinary rate of growth prompts questions about how the program works and why it is appealing to high schools and com- munity colleges. Second, learning how the Academies team dealt with generic problems that hinder improvement in American educa- tion may provide ideas about how to deal with obstacles to progress. Methodology We began work on this case by learning as much as we could about the history and organization of the Academies program from materials available on the Internet. This made us realize the impor- tance of interviewing instructors in Local and Regional Academies, participating students, and Cisco personnel who played key roles in the development and administration of the program. We requested and received permission from Cisco Systems to conduct such inter- views. We then developed semi-structured interview protocols to guide the interviews. We interviewed instructors and students in six Local Academies and four Regional Academies (all terms defined below) in the New England area. Approximately half of the Academies were located in high schools, including two in central city school districts. One was located in a community-based organization. The rest were located in publicly supported community colleges. We also spent several hours observing classes in each of two Academies located in urban high schools and two located in community colleges. We spent two days at the Cisco Academies curriculum and assessment development center in Phoenix, interviewing the Cisco personnel who started the program and those currently responsible for curriculum, student assessment, and instructor training. We also conducted tele- phone interviews with several other Cisco employees and consultants working on the Academies program. We sent an early draft of this paper to the Cisco employees and Academy instructors whom we interviewed, requesting comments and corrections of factual errors. We used their feedback in revising the paper. GENESIS AND DEVELOPMENT Cisco Systems sells routers, switches, software, and advice on building and maintaining computer networks to organizations around the world, including educational institutions. In 1993 John Morgridge, then CEO of Cisco Systems, hired George Ward to help build Cisco's market in educational institutions. Ward, a consulting engineer who had been head of World Wide Networks at Motorola, was well suited RICHARD MURNANE, NANCY SHARKEY, AND FRANK LEVY 131

to the task. Not only did he understand networks; but he also liked talking to educators and helping them to design networks that met their schools' needs. Cisco's sales to schools increased. With the sales growth came an increasingly serious business problem. Many schools and school districts lacked the expertise to maintain the networks. Nor did they have the funds to contract out network main- tenance. In Ward's words, "I'd go in and design Enetworks] and build them and leave, and they would crash. There was limited support staff, and even more limited training" (Zehr, 1998~. Ward's response waLs to develop a 4()-hour training nro~ram aimed at teaching networking . ·.. . . . . .. . . · . · .. · . . of. SKlllS to school personnel so they COU1d maintain their networks. -l-ne program included a PowerPoint presentation and a variety of hands- on activities. Ward spent much of the next year presenting the train- ing program at schools around the country. He quickly learned that his training program was popular; however, he also learned that he could not meet the burgeoning demand for training. Cisco needed another approach to supporting school districts' needs for networking expertise. An experience during Ward's year of travel provided an idea for a new approach. At some sites, schools allowed high school students to sit in on the training. Ward found that the students often grasped the critical skills more quickly than the adults. This led him to wonder whether Cisco should train students to maintain their schools' net- works. To test this idea, Ward asked the principal of Thurgood Marshall High School, an inner-city public school in San Francisco, if he could offer a one-week training program to students in August 1996. The principal, always looking for new ways to engage students, asked one of his teachers, Dennis Frezzo, to make the arrangements. Frezzo was an interesting choice. An optical and electrical engl- neer by background, Frezzo had decided in the early 1990s that he wanted to try teaching. He earned a master's degree in education from Stanford and started teaching at Thurgood Marshall in Septem- ber 1994. Over the next two years, he developed several hands-on courses aimed at developing students' interest in science. Because Frezzo knew which Thurgood Marshall students were interested in science, the school principal asked him to recruit students for the summer program. However, with only two weeks to recruit students, Frezzo found that most of the highly motivated students he had in mind were not available. This left him to dig quite deep into the pool of motivation and academic talent to recruit a group of students for Ward's training program. O — - - - o 1 0 ~ . . _ _= _ _ _ -- ---- ------- --- -- -or - _ _ _ _ _ the training program, which was a mixture of Ward's PowerPoint presentation and many hands-on activities, worked well. Most sur- prisingly, Ward learned that some students at risk of dropping out due to a lack of interest in their academic programs were intrigued by his hands-on training program. This led him to return to Thurgood Marshall in December and propose that Frezzo and a colleague, Jai Gosine, try teaching a one-semester course for students. Frezzo and Gosine piloted 132 A ROLE FOR THE INTERNET IN AMERICAN EDUCATION?

the course in the spring of 1997, starting out with no curriculum and no equipment but lots of enthusiasm. The results were sufficiently positive to lead Ward to take the next steps. Ward took the idea of training students to John Morgridge, who was deeply interested in education and believed that technology could play a key role in improving it. While the CEO had some doubts about whether high school students could master the relatively complex skills needed to trouble-shoot network problems, he was intrigued by Ward's idea and agreed to provide corporate funding for the next steps. Now that Ward had support to develop his idea, he needed help. Recognizing that he knew little about how to develop a good educa- tional program, he recruited Alex Belous, the director of technology education for the state of Arizona, to work with him. Belous's career history was very different from Ward's. After graduating from college in 1973, Belous started teaching first grade in Cave Creek, a rural community 40 miles from Phoenix. While teaching elementary school children in Cave Creek over the next 12 years, Belous came to appre- ciate the difficulty rural school districts with very limited resources faced in providing students with a high-quality education. This became even more apparent to Belous when he became the guidance counselor at Cave Creek's first high school, built in 1985. A great many of Cave Creek's high school students were not going to college and their high school education did not prepare them to prosper in a changing economy. An entrepreneur at heart, Belous went to the Arizona State Department of Education and asked how he could get resources for his school to better serve these students. The advice was to develop a vocational education program. So Belous developed a program to teach computer-based office skills to Cave Creek high school students. This experience led him to believe that technology could play a role in improving education for rural children. From Cave Creek, Belous moved to the Arizona State Department of Education. Through this job, he met George Ward. Alex Belous was a great find for Ward. He understood curriculum development and the types of challenges high school teachers would face in teaching technological skills to students. He also understood state education bureaucracies. Perhaps most importantly, he shared Ward's interest in using technology to improve the quality of educa- tion provided to underserved children. Working initially as a Cisco consultant, Belous led the effort to turn Ward's 40-hour training program for adults into a curriculum for high school students. They agreed on four principles that would guide their effort: 1. The curriculum would be delivered online; there would be no printed version. 2. Well-trained instructors would teach the curriculum. 3. The curriculum would be updated frequently to keep it abreast of RICHARD MURNANE, NANCY SHARKEY, AND FRANK LEVY 133

include both content and pedagogy. changing technology and to improve it as students and instructors pointed out problems. 4. Assessment of student skills would be done online. Beyond these general principles, Ward and Belous had no master plan. When they began to work together, neither imagined that their ideas would develop over the next five years into an eight-semester program serving more than 160,000 students. Encouraged by the interest Thurgood Marshall students showed in the pilot program, Ward and Belous invited interested educators (many of whom had participated in Ward's training courses) to a meeting at Cisco headquarters in San Jose in April 1997. Ward laid out to the group the idea of an educational program for high school students. To bring the idea to life, Frezzo described what he and his colleague had learned while trying out the one-semester program at Thurgood Marshall High School. Ward then sketched out the possibility of a four-semester program with an online curriculum and online assessment of student skills. He emphasized that this would be a partnership. Cisco would not just provide the curriculum. It would also provide training on how to teach the curriculum and ongoing support. In return, it would hold educators accountable for student learning. He asked the partici- pants whether they would be interested in this partnership. While some participants were wary of the accountability concept, many expressed enthusiasm about the potential relationship. Ward and Belous concluded from the meeting that a significant number of high schools were interested in teaching courses on design- ing, building, and maintaining computer networks. Thus, they went ahead in developing the curriculum and recruiting school partners. When John Morgridge officially announced the Cisco Networking Academy Program in October 1997, 64 high schools in seven states were teaching the first semester of the Academies curriculum. Following the public announcement, the number of inquiries grew rapidly. While delighted by the interest, Ward and Belous quickly realized that they faced new questions. Who was going to train the instructors? What would be the financial arrangements of the partnerships? Over the next months, Ward and Belous and their collaborators worked out answers to these questions, while at the same time pushing ahead with the development of Semesters 2, 3, and 4 of the curriculum. The answer to the training question was to develop a highly leveraged "train the trainer" model, as illustrated in Figure 5-1. Training would The first instructors Ward had trained became the heads of Cisco Academy Training Centers (CATCs). Cisco would pay them to provide training to instructors at Regional Academies, who in turn would provide training to instructors at Local Academies. Initially Ward and Belous envisioned that the CATCs and Regional Academies, as well as the Local Academies, would all be in high schools. Soon, however, they found that community colleges were 134 A ROLE FOR THE INTERNET IN AMERICAN EDUCATION?

| Cisco Academy Training Center ~ | Regional Academy Regional Academy Local Academy Local Academy Local Academy Local Academy Local Academy Local Academy FIGURE 5-1 Cisco Networking Academies organizational structure. interested in becoming CATCs and Regional Academies and that state governments were interested in having community colleges play this role. By 2001, half of the Regional Academies in the United States were located in community colleges. Table 5-1 provides a summary of the types of institutions in the United States in which Local Academies and Regional Academies are located. Table 5-2 provides information on the worldwide growth in the number of Cisco Academies. TABLE 5-1 and Location Number of Cisco Acaclemies by Type of Acaclemy, Type of Institution, Within the United States Location Local Academies Regional and Local Regional Only High schools 2,424 67 20 Vocational programs [a] 337 23 1 Junior high schools (grades 7-9) 11 4 2 Two-year community colleges 486 211 20 Four-year colleges 101 35 7 Postsecondary technical schools 72 12 1 Nonprofit organizations 61 7 3 Military service institutions 50 0 1 Other types of organizations 82 19 22 U.S. Total 3,624 378 77 Outside United States Local Academies Regional and Local Regional only Non-U.S. Total 2,197 442 174 Worldwide Total 5,821 820 251 SOURCE: Compiled in April 2001 from information available on the Academy locator portion of the Cisco Academy Web site: http://cisco.netacad.net/cnacs/pub-doc/locator.shtml NOTE: From the available information, it is not possible to distinguish clearly between Cisco Academies in vocational high schools serving teenagers and Academies in institutions with the label of vocational education that serve adults. RICHARD MURNANE, NANCY SHARKEY, AND FRANK LEVY 135

TABLE 5-2 indicators of the Growth of Cisco Networking Academies WorIc3wicle Academy Statistics Past (FY '98) Current (FY '01 Future (Forecast) Local only 395 6,419 Regional / local 61 859 Subtotal local 396 7,278 Regional 184 271 CATCs 39 Total locations 580 7,559 10,140 Countries 8 130 SOURCE: Cisco Networking Academy Program (2000~. Over time the answer to the financial arrangements question emerged. Local Academies receive the online curriculum at no cost. Regional Academies may charge Local Academies for instructor training, on a cost-recovery basis. Local Academies need to buy from Cisco a rack of hardware containing five routers and two switches at a highly sub- sidized price of less than $10,000. Additional costs to the Local Academies may include several thousand dollars for testing equip- ment and disposable materials (such as cables). They are also re- quired to purchase from Cisco the "SmartNet" service contract for approximately $1,200 per year after the first year. (The cost of SmartNet is covered in the Lab Bundle cost in the first year.) This entitles the Academies to "24/7" technical support for rapid service on their rout- ers and switches. Exceptions to these financial arrangements are for Academies in economic empowerment zones, which are rural and urban areas desig- nated by the federal government as high unemployment areas, and for Academies in Native American communities. Cisco provides the package of routers and switches at no cost to the 100 Academies that are located in empowerment zones and approximately 13 Academies lo- cated in Native American communities. Regional Academies receive one set of routers and switches from Cisco at no cost. The Cisco Learning Institute, a public charity funded and formed by Cisco, pays for CATCs to provide training to two Regional Academy instructors. In return, Regional Academies as- sume three responsibilities. First, they must make a "good faith ef- fort" to recruit 10 Local Academies. Second, they provide instructor training and ongoing support to the Local Academies they recruit. Third, they monitor the quality of the Local Academy programs. Since the training, supporting, and monitoring functions use significant resources, Cisco allows Regional Academies to charge Local Academies annual support fees and fees for training sufficient to recover their costs. These fees are set by Regional Academies and range from zero to several thousand dollars per year. 136 A ROLE FOR THE INTERNET IN AMERICAN EDUCATION?

THE ACADEMY PROGRAM'S EXPERIENCE WITH AMERICAN EDUCATIONAL PROBLEMS Since most Cisco Networking Academies in the United States are located in public high schools and community colleges, the program confronts many of the challenges that face all attempts to improve the quality of education provided by these institutions. In this section we describe how the Cisco Academies program has dealt with some of these problems. Providing High-Quality Instructor Training Providing teachers with training (usually called professional develop- ment in education circles) that improves their effectiveness in help- ing students to master critical skills has been an ongoing challenge for American public education. While school districts typically de- vote significant resources to professional development, most of the money goes to fund one-day workshops that have little effect on how teachers teach (see Murnane and Levy, 1996, Ch. 7~. The Cisco Networking Academy Program faced a particularly difficult training challenge. Local Academies chose to teach the Cisco networking curriculum. Many instructors chosen by the Local Academies were high school teachers or community college instructors with considerable teaching experience but no knowledge of computer networking. Others were adjunct faculty drawn from industry who understood computer net- working but had no teaching experience. The challenge was to pro- vide both the technical knowledge needed to teach the curriculum and the teaching skills needed to motivate students and explain Quite difficult technical concepts. It had no control over the instructors The training program that the Academy team designed with the help of professional educators provided intensive hands-on instruc- tion, tests of skill mastery, and ongoing support. Newly designated Local Academy instructors receive eight days of full-time instruction on the first semester's curriculum at a Regional Academy. Participants study the same curriculum that they will subsequently teach. They are expected to have read the online curriculum before coming to the training and to review the relevant chapters each night during the training. Participants do the same hands-on labs their students will do and they take the same Internet-based end-of-chapter tests aimed at pro- viding feedback on their skill mastery. The training guidelines specify that each participant must do a "teach-back" in each semester. The teach-back consists of preparing a lesson plan for a particular chapter and teaching the chapter to other participants and the Regional Academy instructor. In order to receive authorization to teach the first semester curriculum, participants must pass the online final examination with a score of at least 80, as well as successfully complete the teach- RICHARD MURNANE, NANCY SHARKEY, AND FRANK LEVY 137

r · 1 , ~ back. (Aspiring instructors who score below 80 may take the exam again but must score at least 85 on a second try and 90 on a third try.) The schedule recommended by the Cisco Academies Program is that participants teach each semester's curriculum immediately after completing the appropriate training and before training to teach the subsequent semester. Instructors have 24 months to complete training for teaching all four semesters of the curriculum and to pass the CCNA examination, prepared by Cisco Systems but administered by an inde- pendent testing organization (Prometric).~ This CCNA examination is the same examination that students who complete the four-semester Academy curriculum are encouraged to take and that adults who pay for commercially provided training take to obtain the industry-recognized CCNA credential. After completing the training, Local Academy instructors have access to four types of support. First, they receive from Cisco an instructor's version of the online curriculum that includes detailed suggestions for how to teach each class and how to set up hands-on labs. Second, they have access to the Cisco Academies Web site, which provides answers to frequently asked questions and provides a bulletin board on which they may pose new questions. Third, as purchasers of the "SmartNet" maintenance agreement, they are entitled to help from Cisco on technical issues. Finally, they may request assistance from their Regional Academy on both technical and pedagogical issues. Regional Academy instructors told us that, at the request of Local Academy instructors, they sometimes modeled the teaching of a particularly difficult chapter, or observed an instructor teaching a chapter for the first time. While instructor training focuses primarily on teaching the techni- cal material in the curriculum, the design is for Regional Academy instructors to model best teaching practices. The emphasis is on mini- mizing lecturing and maximizing hands-on involvement. Participants also receive an online manual describing best practices. This manual begins with a description of the six levels of Bloom's taxonomy (knowledge, comprehension, application, analysis, synthesis, and evaluation), and describes ways in which teachers can reach each level of the tax- onomy in the CCNA curriculum. For example, a sample of knowl- edge is, "Identify how many bits comprise an IT address (Semester 1~." A sample evaluation question is, "Your company has decided to use Category 6 UTP (instead of CAT 5 or 7) support their decision." The Best Practices Manual also offers guidance on using journals, portfolios, student presentations, and group work in helping students to master critical skills. For example, the Manual states that "the types ot Journal entries most applicable for Networking Academies' lithe CCNA examination antedates the Academy program and was originally designed by a Cisco group unconnected to the Academy program. The Academy's assessment group, headed by John Behrens, has contributed to improving the exam. The exam includes some drag-and-drop graphical items as well as multiple-choice questions. 138 A ROLE FOR THE INTERNET IN AMERICAN EDUCATION?

students include: daily reflections, troubleshooting details, lab pro- cedures and observations, equipment logs, hardware and software notes, router configurations, contacts and resources, questions, designs." How well does the training work? While the quality of the training inevitably varies, Regional Academy instructors and people design- ing the training for the Cisco Academies tended to share the same opinion. The training is effective in teaching the technical skills to instructors who know how to work with students and are highly moti- vated. Each of the regional instructors we interviewed had stories about enthusiastic instructors with unlikely backgrounds English teachers, culinary arts teachers who became effective Academy instructors. This does not mean that the training is easy. One very experienced teacher at a vocational high school told us that the Academies in- structor training was "the most difficult professional development I have ever done." She reported spending 17 hours each day for three weeks to complete Semester 1. The pressure to pass the end-of-semester examination makes the training intense. One member of the Cisco training team estimated that one in 10 people who start the training to become a first-semester Academy instructor either does not complete the training or do not pass the end of the semester exam. Cisco Academy trainers reported that the training was less suc- cessful in imparting pedagogical skills to participants who tended to lecture rather than to engage participants in hands-on activities. This problem was most common among community college adjunct in- structors who worked in industry. However, some high school teach- ers tended to spend an inordinate amount of time lecturing as well. The best practices document is very clear on the importance of engaging students in hands-on activities and on limiting the amount of time teachers spend lecturing. However, this does not happen in every Academy any more than it happens in every high school or commu- nity college science class. How critical is technology in the Academy's professional devel- opment? The Internet plays an important role in providing support to instructors. The Cisco Academy Community Server is a valuable resource for instructors, enabling them to obtain answers to questions quickly and to access materials developed by other instructors.2 The common online examinations provide information on the extent to which aspiring instructors have mastered critical technical skills. These are important contributions of technology to professional development. At the core, however, professional development in the Academies program depends on the quality of the Regional Academy instructors who provide the training. Technology complements the skills of the Regional Academy instructors; it is not a substitute for instructors who model best teaching practices. 2As discussed below, the community server also keeps track of students' grades on chapter tests and the semester examination, eliminating the bookkeeping activities that consume a great deal of time for most teachers. RICHARD MURNANE, NANCY SHARKEY, AND FRANK LEVY 139

Recruiting and Retaining High-Quality Instructors Many public schools have difficulty finding skilled teachers in , O technical fields. As the economists Joseph Kershaw and Roland McKean (1962) explained 40 years ago, the problem lies in the uniform salary scale. In almost all public school districts, teachers' pay depends solely on years of experience and number of academic degrees. College graduates with training in fields such as chemistry and physics that command relatively high salaries in industry are not paid more to teach in public schools than are graduates trained in fields that provide less attractive salaries outside of teaching. The proliferation of infor- mation technology applications in the last 20 years has created a boom market for college graduates with training in computer science. When they see starting salaries typically twice those offered by public schools, few graduates with training in computer networks find public school teaching attractive. Most community colleges face the same problem created by the uniform salary scale. One approach that the Academies program might have chosen was to attempt to "teacher-proof" the curriculum, that is, to produce a curriculum that students could learn from without the aid of skilled teachers. This would have been consistent with previous attempts to use technology to deal with the shortage of skilled teachers. How- ever, Ward and Belous rejected this option from the outset. The Cisco Academies solution to the shortage of technically trained instructors is to "grow its own." Since the idea for the Academy program stemmed from George Ward's realization that schools lacked the skills to maintain their computer networks, he knew that providing the technical skills needed to teach the Academy curriculum was criti- cal to the success of the Academy program. This realization led to Cisco's investment in instructor training that is described above. . . There is little question that the instructor training has been inte- gral to the growth of the Academy program. In effect, the training and the follow-up support have increased the supply of people who understand the fundamentals of computer networking. Of course, this knowledge is useful not only in teaching the Academy curriculum, but also in a wide range of jobs in industry. As a result, many schools and community colleges find that after investing in Cisco Academy training for faculty members, they lose many of these same faculty to higher paying positions in industry.3 Since the Academy program has no influence over the salaries schools and community colleges pay to Cisco Academy instructors, it has little leverage for solving this prob- lem. Its only policy instruments are to make training available to new instructors and to insist that potential instructors complete the training before teaching the curriculum. sin our interviews we heard about students in the Academy program who subsequently became instructors, so there appears to be some "grow your own" potential. 140 A ROLE FOR THE INTERNET IN AMERICAN EDUCATION?

Local Academies deal with the attrition problem using the same creative ad hoc measures that schools have used for many years to retain skilled faculty. Some offer extra compensation for extra teaching or administrative work. Some seek to provide especially attractive working conditions, such as well-equipped computer laboratories. One community college that hosted a Cisco Academy provided a more direct approach. Instead of offering the Cisco classes as for-credit courses at the usual tuition of $280, this college offered them as noncredit workforce development courses for $900. Since instructors in these courses were not covered by the union contract, the college could pay Cisco Academy instructors much more than they would have received as instructors of for-credit courses. In summary, a good training program is essential to creating a supply of instructors capable of teaching the Cisco Academy courses effectively. At the same time, it does not immunize Local and Regional Academies from the problem of retaining teachers whose skills provide them with higher paying opportunities in industry. Retaining a Focus on Developing Critical Skills for All Students The decade of the 1990s was a period of policy change in Ameri- can education. Sparked by growing realization that a great many students leave school without mastery of basic cognitive skills and that labor market opportunities for these workers had declined markedly over the past 20 years, almost every state in the country engaged in standards-based educational reform initiatives.4 While the plans vary across states, they have in common specification of the skills all students should master in each core subject at particular grade levels and tests to measure whether students have mastered the critical skills (Murnane and Levy, 2001~. It was into this policy environment that Cisco introduced its Networking Academy Program. From the beginning, Alex Belous saw the Academy program as contributing to educational reform. He believed passionately in the importance of education in equalizing economic opportunity. With many years of experience as a teacher in rural schools, he knew the difficulty in providing all children with rich opportunities to learn. He felt that the Internet could play an important role in equalizing educational opportunities. John Morgridge and his successor as Cisco CEO, John Chambers, shared these beliefs. 4The initial concern voiced in the 1980s in documents such as Workforce 2000 (Johnstone and Packer, 1987) was that skill deficiencies in the American workforce would hinder productivity growth. The quite rapid surge in productivity during the l990s showed that this concern was misplaced, at least in the medium run. However, earnings data from the l990s also showed that not all workers benefited from the growth in productivity. In particular, the earnings of workers who lacked post- secondary education and those with weak cognitive skills did not increase markedly during the l990s and continued to be significantly below the real earnings of compa- rable workers in the late 1970s. RICHARD MURNANE, NANCY SHARKEY, AND FRANK LEVY 141

As the director of the Academy curriculum development effort, Belous led the effort to develop a curriculum that taught students to design, build, and maintain computer networks and that also taught cognitive and social skills that would be valuable to students even if they chose to pursue other occupations. In Semester 1 of the four- semester curriculum, students learn about industry standards, network topologies, IF addressing, networking components, and basic network design. Activities in semester one include learning how to convert hexadecimal numbers into binary numbers and examining building blue prints to determine where computers and their supporting hard- ware can be positioned in the building. At one high school we visited, students were also conducting a survey of all of the computers in the school. This survey included all hardware specifications as well as the types of software available to run on the machines. Students kept their records in Microsoft Access and Excel, programs they learned to use in the Academy course. Ultimately, the students' computer survey would be used to help determine the school's technology needs and capabilities. In Semester 2, students learn about beginning router configura- tions and routing protocols. A sample activity from this semester is for students to work in teams to design a network tonolo~v and IP .. . . .. . . . . ^. - - r - - - on actctress~ng scheme that ~nc~ucres live routers. Students then use soft- ware to create a diagram of the networks they have designed. As part of this activity, students answer a series of questions aimed at having them reflect on their work in writing. These questions range from asking students about their group experience (for example, "What did you learn from designing a topology with such a large group of people?") to what they learned about the routing process (for example, "Could you have done it any other way? If so howdy. Semester 3 of the Cisco Networking Academy Program includes advanced router configurations, local area network (LAN) switching theory and VLANS, Advanced LAN, and LAN switched design. Stu- dents also learn about Novell IPX, a protocol commonly used in the networking industry. In this semester students also participate in a "threaded case study" that involves a simulated real-world problem. For example, students create a network for a fictional school district, including a LAN at each site in the district, as well as a wide area network (WAN) connecting to more than 30 sites. This project re- quires students to consider everything from wiring schemes at indi- vidual schools to security for the whole system. Semester 4 covers WAN theory, design, and technology; network troubleshooting; and, again, a threaded case study, in which students continue to improve upon the network created in Semester 3. Additionally, this semester includes reviews for the "Network+" certification exam (a vendor-neutral certification exam) as well as Cisco's own CCNA certification exam. To date there are no systematic studies evaluating the long-term benefits to students from participating in the Cisco Networking Academy 142 A ROLE FOR THE INTERNET IN AMERICAN EDUCATION?

Program and the extent to which benefits are contingent on choosing an occupation connected to computer networking. However, the cur- riculum does appear to offer a variety of opportunities for students to develop skills, including writing clearly and working productively in croups. that are valued in a wide range of jobs (see Murnane and Levy, 1996, for a discussion of the importance of these skills). Our conversations with Academy students indicate significant variance in the implementation of the curriculum. Not all instructors devote significant class time to hands-on activities. Nor do all Academy - instructors know how to facilitate group work effectively. Nor do all Academy instructors pay attention to the quality of the writing in students' engineering journals and work with students to improve their writing. In this respect, the Academy program illustrates a pattern replicated again and again in American education: hich-oual- ity curriculum is an essential complement to well-trained. hi~hlv mo- tivated teachers; it is not a substitute for them. - ~ O , Belous and his colleagues engage in continuous improvement of the curriculum, working to eliminate ambiguities and keep the diffi- culty of the text to the ninth-grade reading level. They also empha- sized that since students access the curriculum online and take their tests online, the design of the courses makes it easy for students to be able to work at their own pace. Despite the attention paid to the curriculum design, some states and school districts were reluctant to adopt the Academy program. Some of the wariness stemmed from past experiences in which cor- porations' contributions to school improvement had been donations of used equipment and curricula taken from commercial training pro- grams developed for adult professionals, with no support for how to develop and teach a curriculum that made sense for high school students. Another source of caution was the perception that the Cisco Networking Academies provided a vocational curriculum that would not provide students with skills transferable to occupations other than networking computers. This matters because a large proportion of students who participate in vocational education programs do not enter the occupa- tion for which they train, (Bishop, 1995) and a large and growing proportion of American workers change occupations over their work lives (Parrado and Wolff, 1999~. In response to these concerns, Belous contracted with research organizations to document the number of activities in the Cisco Academy curriculum that address national academic standards for science and mathematics prepared by professional organizations and critical skills described in the report prepared by the Secretary's Commission on Achieving Necessary Skills (1991), known as the SCANS report. The research groups commissioned teachers and college faculty who were familiar with both computer networking and the national standards to read the Academy curriculum and document points of alignment. The curriculum alignment document reported that Cisco Academy cur- riculum activities address 12 of 14 mathematics standards, either partially RICHARD MURNANE, NANCY SHARKEY, AND FRANK LEVY 143

or fully; five of eight categories of science standards; and many of the skills listed in the SCANS report.5 This evidence went a long way toward convincing state departments of education that students should receive credit toward high school graduation for completing Cisco Networking Academy courses. This is still an ongoing issue in some states. For example, Texas currently does not count Cisco Academy courses toward students' high school graduation requirements. The enthusiasm of vocational educators and community colleges provided additional support. This enthusiasm was rooted in the struggles of vocational schools and community colleges to respond to the "integration" mandate expressed in the 1990 Perkins Act, the federal legislation that reauthorized federal support of vocational education. Troubled by charges that vocational education prepared students only for narrowly defined jobs jobs that often disappeared in a changing economy reformers called for integration of occupational and academic education in high schools and for programs that enabled students to move from career-oriented high school programs to community colleges. The Cisco Academy Program was tailor-made for high schools and community colleges working to respond to the Perkins Act man- dates. The program was attractive to students seeking preparation for relatively high-wage jobs in the growing field of computer network- ing. It offered instructor training, a critical need in schools trying to keep vocational offerings up-to-date. The alignment document dem- onstrated that the Academy curriculum taught not only technical skills relevant to networking, but it also taught math, science, and writing skills needed in a wide range of occupations. Thus, the program offered the promise of integrating occupational and academic educa- tion. Finally, because all students in Networking Academies studied the same curriculum and took the same end-of-course examinations. community colleges felt justified in offering college credit and dual enrollment to high school students who completed one or more semesters of the Academy curriculum. This was exactly the type of cooperation between high schools and community colleges that the Perkins Act and the School to Work Opportunities Act of 1994 envisioned. In 1998 Colorado became the first state to endorse the Cisco Networking Academies. Other states soon followed. The net effect of the state endorsements was that high schools could add the four semesters of the Cisco Networking Academy program to their curriculum without fear that state education agencies would question offering these courses for credit toward graduation. In several states the endorsements meant the state department of education would pay the training and support costs associated with setting up Cisco Academies. 5As described in cisco Networking Academy Program <2000y, Educational cYsER- CONNECTIONS, Inc. was the research firm that documented the alignment between the cisco Academy curriculum and national science and math standards; V-TECS was the research firm that documented the extent to which the Academy curriculum addressed skills described in the 1991 SCANS document Secretary Commission, 1991~. 144 A ROLE FOR THE INTERNET IN AMERICAN EDUCATION?

From our visits to five Local Academies, we have seen that the institutions teach the curriculum at very different paces (within the framework prescribed by Cisco that the four semesters must include a minimum of 180 hours of instruction, including lab time). For example, at one community college, Networking I, a three-credit course that meets for three hours a week (plus an equal amount of lab time) for 15 weeks, covers the first two semesters of the Academy curriculum. Networking II covers Semesters 3 and 4 of the curriculum. In these courses students are expected to read two chapters each week before coming to class. Class time is devoted to discussion of the concepts in the text and to hands-on activities. Students have one day after each class to complete the online chapter tests. At the other extreme, we observed a Local Academy located in an urban high school that meets for two hours, four days a week for 15 weeks. Students spend much of the class time reading the curriculum. Students work at their own pace. They report doing little work on the course outside of class. Fewer than half of the students had completed the Semester 1 curriculum by March of the second semester. At least part of the explanation for the radical difference in the pace of the two classes concerns differences in the students. The students in the fast-paced community college course were in their twenties, thirties, and forties; were paying their own tuitions; and were acutely aware that they needed marketable skills to improve their earnings prospects. They had also demonstrated initiative in gaining places in the oversubscribed Networking class. In contrast, the students in the slow-paced urban high school class were teenagers whose interests seemed to focus primarily on sports and social activities. Most had no career plans. None had a com- puter at home. Some of the students were enthusiastic about their Cisco class and hoped it would lead to college. However, others had been assigned to it by a guidance counselor and would have preferred another program. Cisco is developing a primer curriculum to help prepare students for the Academy curriculum. It also is developing new modules on topics such as hexadecimal and binary numbers that many students find difficult to master. This module would expand on the Semester 1 lab activity explaining how binary numbers are used in IF addresses, demonstrating how to convert numbers to binary numbers, and requiring students to make conversions. Improving the Curriculum and Keeping It Up to Date Providing teachers with a curriculum that focuses on the skills students are expected to master, that is accessible to students with limited reading ability,6 and that interests a wide range of students 6We learned from Academy instructors in high schools that many students print off the online curriculum rather than read the explanations from a computer screen. RICHARD MURNANE, NANCY SHARKEY, AND FRANK LEVY 145

has been a constant challenge for American education. The challenge of providing up-to-date curricula is especially great when technologi- cal changes alter the critical skills that students should master. The team that developed the Cisco Networking Academy curricu- lum faced a set of particularly difficult challenges. Networking technology is changing quite rapidly, necessitating frequent changes in curriculum designed to teach students to design, build, and maintain networks. The curriculum teaches technical skills and could easily become accessible only to students with very strong reading comprehension skills. Alex Belous and his colleagues were determined to keep the reading level at the ninth-grade level so as to make the curriculum accessible to as many high school students as possible. They wanted to make the curriculum available in several languages (currently nine) so that it could be used in many countries. Including many diagrams and illus- trations contributed to making the curriculum comprehensible. However, the more graphics included in the online curriculum, the slower the response time for students accessing the curriculum, especially for those using less than state-of-the-art computers. To our surprise, keeping the Networking curriculum up-to-date is a relatively straightforward matter. To keep the CCNA exam up-to- date, Cisco contracts with an outside group every two years to do a task analysis of the work involved in designing, building, and main- taining computer networks. The analysts observe the work of people holding the CCNA credential and ask them how often they carry out different tasks. The results of the task analysis are used by Cisco to remove questions from the CCNA exam that pertain to tasks that no longer are central to the work of CCNA holders and to add questions that pertain to tasks that have become more important. The changes in the CCNA exam drive changes in the Cisco Networking Academy curriculum. The Internet plays a key role in improving the Academy curriculum. Curriculum developers use the Internet to elicit rapid feedback from members of the curriculum review committee, consisting of Academy instructors who have demonstrated an interest in curricular improve- ment. Instructors and students use the Internet to report problems they have encountered with the curriculum. The curriculum develop- ment team has a formal process for evaluating the feedback it receives and making changes in response to criticisms and suggestions. While this process could take place using regular mail, it would be vastly slower. There is a trade-off in the design of a Web-based curriculum between quantity and quality of graphics and the speed with which files can be accessed. The use of more graphics makes the curriculum appealing. However, it can render the curriculum inaccessible to sites using less than state-of-the-art computers and to sites that lack resources to purchase new software. Belous and his colleagues have been sensitive to this trade-off, and have been careful not to make changes in the curriculum that would require Academies to purchase new software or hardware. 146 A ROLE FOR THE INTERNET IN AMERICAN EDUCATION?

Fortunately, technological advances in displaying and transmitting graphical images have made it possible to increase markedly the number of graphical illustrations in the curriculum while also reducing the size of the curriculum files that Academies download over the Internet.7 Including references to informative Web sites is another way that Belous and his colleagues have enriched the curriculum without increasing its size. One important lesson Ward and Belous have learned is that there are trade-offs in the timing of curriculum revisions. Initially they made changes as often as daily. While this provided Academy sites with the most up-to-date version, the frequent changes raised havoc with instructors who would prepare lesson plans based on one ver- sion of the curriculum only to find that their students were reading a revised version that did not match their plans. Belous and Ward's response has been to make available new versions of the curriculum no more often than twice a year. Making Assessment a Valuable Part of the Education Process In recent years, assessments of students' skills and knowledge have come to play a critical role in standards-based educational reform efforts. In a growing number of states, students who do not score well on state-mandated assessments of core skills are not promoted to the next grade. High school students who do not pass exit examinations do not receive high school diplomas. Teachers and administrators in schools with consistently low test scores may find their jobs in jeopardy. With the increased attachment of stakes to student test score results, tests have come under increased scrutiny. One question is whether the tests assess the students' mastery of critical skills. A second is whether focusing instruction on the skills needed to do well on the tests improves or worsens the quality of instruction students receive. From the beginning of the Academy program, Ward and Belous planned to develop online assessments of student skills. Students complete an online multiple-choice examination at the end of each of the 15 chapters that are part of each semester's curriculum. They also must complete an end-of-semester online multiple-choice exam. As the Academy program grew, George Ward realized that it needed more expertise on assessments. This led the program to recruit John Behrens, a senior psychometrician at Arizona State University. In January 2000, Behrens became director of assessment, research, and evaluation for the Academies. Behrens has led the effort to improve assessments of students' and instructors' skills. There are many ways that Behrens and his colleagues use tech- nology to improve skill assessments. While all students currently 'Academy instructors download the curriculum over the Internet from the Academy central server. Then students access the curriculum on LANs. RICHARD MURNANE, NANCY SHARKEY, AND FRANK LEVY 147

answer the same questions at the end of each chapter and at the end of each semester, software randomly assigns the order of the questions for each test taker. Instructors value the random ordering because it increases the likelihood that student's responses reflect their own knowl- edge. Student's responses to the multiple-choice questions go directly to the Cisco Academy server in Arizona and students receive back their exam grade in a minute or two. The exam grades for each student are automatically entered into a spreadsheet that the instructor can access to check on student progress and to assign final course grades. Student responses on individual test items are recorded on the Academy server and are analyzed by the assessment team to identify problematic ques- tions. Since the exams are provided to students online from the Acad- emy server, the assessment team can easily test new questions and replace inadequate questions with better ones. The assessment team uses two strategies to align the assessments with curriculum goals. First, the person who prepares each exam question documents the particular curriculum goal that the question addresses. Then each item goes through a multistage quality review. Second, the end-of-course multiple-choice exam for each semester is complemented by a hands-on practical examination. While individual instructors have discretion in designing the hands-on examinations, the instructor's guide outlines the types of tasks that should be assessed. For example, the task for the second semester is to build a network that uses five routers. The hands-on exam for the fourth semester is comprehensive in that it assesses mastery of skills taught in all four semesters. For example, students are asked to use IF addresses (Semester 1), apply a routing protocol (Semester 2), make an access list (Semester 3), and construct a wide area network (Semester 4~. Students must pass the hands-on examination as well as achieve a satisfactory score (as defined by the instructor) on the end-of-course multiple-choice exam in order to receive credit for the course. Behrens has several initiatives under way to improve the Academy assessment program and to incorporate technological advances devel- oped by Educational Testing Service and other organizations on the frontier of test development. One initiative will overcome the current limitation that students only learn their score on each examination, not which items they answered incorrectly. In the future, feedback will not only include identification of questions answered incorrectly, but also links to the relevant sections of the curriculum and to addi- tional resources. A second initiative is the development of multiple forms of each end-of-semester and chapter test, with comparable scor- ing across the different forms. This will allow instructors to use different forms of the tests for different purposes. A third initiative is the development of new online assessment formats, including micro- simulations that require students to program emulated routers. A fourth is the development of adaptive testing that tailors the difficulty of the 148 A ROLE FOR THE INTERNET IN AMERICAN EDUCATION?

test items to the skill of the test taker. This permits more accurate assessment of the skills of individual students. Maintaining High-Quality Instruction A premise underlying standards-based educational reforms is that all children will receive consistently good instruction. Indeed, fulfilling this premise is necessary to justify withholding grade promotions and high school diplomas from students who do not score well on state- mandated standardized assessments. Yet a great many studies over the last 30 years have documented that the quality of teaching in American schools varies enormously.8 Assuring that all students enrolled in Cisco Networking Academies receive consistently good instruction has been a goal of the program since its inception. In fact, the one uneasy response to George Ward's initial presentation of the Academy concept to a group of educators in spring 1997 dealt with accountability. Educators wanted to know what Ward meant by the term and what responsibilities they would incur if they signed up to host a Local Academy. The structural design of the Academy program makes quality assurance a particularly difficult challenge. Administration is very decentralized. CATCs are responsible for monitoring the quality of Regional Academies; in turn, the Regional Academies are responsible for monitoring the quality of Local Academies. Cisco does not select the instructors in Local and Regional Academies, nor does it pay them. The only sanction Cisco can impose on a Local Academy that does not provide consistently high-quality instruction is to work with the Regional Academy to deny program affiliation. As a result, the primary strat- egy for assuring instructional quality is to identify problems as quickly as possible and then to offer a variety of opportunities for improving instruction. The Academy program makes extensive use of information col- lected online in identifying problems. Student's end-of-semester exam scores provide one source of information. The Academy Quality Assurance Team monitors the distribution of scores in each class, looking for classes in which a large percentage of students earn low scores. Student surveys provide a second source of information. Every student completes an online survey at the end of every semester that asks for ratings of the instructor, the curriculum, and the course assignments. Instructors can learn about their performance by accessing online the average rating their students gave on each of the 15 ques- ~Researchers from different disciplines using different methods have documented the wide variance in teaching quality in American schools. See Hanushek (1994) for a discussion of the evidence by economists. See Grossman (1990) for evidence on differences in effectiveness among English teachers and Mayer (1998) for a discus- sion of differences among math teachers. RICHARD MURNANE, NANCY SHARKEY, AND FRANK LEVY 149

tions in the survey. (See Table 5-3 for an example of feedback to an instructor from student surveys.) The single number describing the average rating students gave on the 15 survey questions is sent to the Regional Academy overseeing the particular Local Academy and to the Academy Quality Assurance team. In addition, participants in instructor training complete online evalu- ations of the quality of their training. The survey asks participants to rate the class content ("The order of course topics aided my learn- ing"), as well as the instructor's pedagogical style ("Class participation was enhanced through effective use of questions") and content knowl- edge ("Analogies and real-life experiences of the instructor added value to the course". These ratings are available to the trainer, and a summary of these ratings is also transmitted to the Quality Assurance Team. The Academy team also uses the Internet to evaluate com- plaints about training quality. When the director of training receives an e-mail message complaining about the quality of a training pro- gram, he sends an online survey to all participants in that particular training class. The responses help him to judge whether the initial e- mail reflects a significant training problem or an isolated personality conflict. A primary mechanism for quality assurance is the annual audit. A three-person team (including one consultant with expertise in peda- gogy, a second Know~eagean~e anout tecnno~ogy, and one member of the Cisco Academy team) audits each CATC. It inquires whether the CATC fulfilled its contractual obligation to conduct annual audits of the Regional Academies under it. Part of this process involves learn- ing whether the Regional Academies conducted audits of the Local Academies under them. The auditors of CATCs also examine the student test score distributions and average student survey grades for all Regional and Local Academies under the particular CATC. The auditing team asks whether the CATC had identified patterns of low student scores or poor class ratings. It also explores whether the CATC brought the potential problem to the attention of the relevant Regional Academy and whether it followed up to see whether the Regional Academy worked with the relevant instructor to diagnose the source of the problem and develop a remediation plan.9 Another part of each audit is to check that instructors of all Academy courses have kept their CCNA credential up-to-date, a process requiring them to pass the CCNA exam every three years. This requirement that instructors periodically demonstrate mastery of subject matter content is a bone of contention with many instructors. To date, however, the program has stuck by this requirement. _ _ . · . . . 90f course, there are several reasons students could have low scores on the end-of- course examinations, including low reading skills, inadequate facilities, and poor teaching. The Regional Academy is responsible for exploring the reason for low scores among students in a Local Academy in its jurisdiction and for suggesting improvement strategies. 150 A ROLE FOR THE INTERNET IN AMERICAN EDUCATION?

TABLE 5-3 Sample of Course Feedback Results Min Max Mean Question 5 5 5 5 5 5 5 4.85 4.77 4.54 4.54 4.23 4.69 4.69 4.69 4.92 4.15 3.85 4.15 4.38 4.31 4.62 The instructor was adequately prepared to teach this course. Analogies and real-life experiences of the instructor added value to this course. Presentations were clear and easy to understand. Answers to questions were provided in a timely manner. Class participation was enhanced through effective use of questions. The class was interesting and enjoyable. "Best Practices" and good teaching strategies were modeled during the training. Grouping strategies were utilized effectively. Class members felt comfortable approaching the instructor with questions/ ideas. The order of course topics aided my learning. The course schedule allowed me to complete the stated course objectives. The activities and labs helped me achieve the stated course objectives. The lesson assessment tools helped me evaluate my knowledge of the lesson. Group work aided my learning. Overall, the course materials were of high quality. SOURCE: Cisco Academy instructor. Quality assurance in the Cisco Academy program depends criti- cally on the efforts of Regional Academies, the majority of which in the United States are in community colleges. While Cisco allows Regional Academies to recover the costs they incur in supporting Local Academies and training their instructors, and a few charge an annual support fee as high as $15,000, many charge little or nothing. The rationale, several Regional Academy instructors told us, is that state legislatures expect community colleges to serve as community assets. Supporting Cisco Local Academies in area high schools is one way of demonstrating that commitment. Of course, this arrange- ment means that the quality of support Local Academies receive and the integrity of the quality assurance plan depend critically on the good will of the community colleges and high schools that serve as Regional Academies. Signaling Skills Employers in the United States find it very difficult to obtain reliable information on the skills of high school graduates who apply for jobs. The high school diploma indicates only that students com- pleted four years of high school, not that they mastered particular skills. High schools typically do not respond quickly to requests for student transcripts. A consequence of the lack of information on the skills of high school graduates is that employers typically offer the same wages and conditions of employment to high school graduates with strong skills that they offer to graduates with much weaker skills. RICHARD MURNANE, NANCY SHARKEY, AND FRANK LEVY 151

Bishop et al. (2001) have argued that this hiring pattern reduces high school students' incentives to do the hard work that skill mastery requires. Graduates of the four-semester Cisco Networking Academy may obtain the Cisco Certified Networking Associate (CCNA) credential by passing the same examination that graduates of commercial train- ing institutes take. The CCNA exam is a multiple-choice exam administered by an independent testing organization, Prometric.~° Academy graduates may also obtain the vender-neutral Network+ credential by passing a different examination offered by an industry trade organization, CompTIA. We have no way of assessing whether the opportunity to acquire industry- recognized credentials has motivated students to enroll in Academy classes and to work hard to acquire critical skills. However, many Academy students in community colleges told us that their reason for enrolling in the Academy courses was to prepare for the CCNA exam. Credentials that can be earned by passing examinations predate the Internet by many years. However, the online format of the CCNA exams mean that scoring is done almost instantaneously and the cre- dential can be awarded much more quickly than if paper answer sheets were sent to a central scoring office. This may increase the attractive- ness of the credential and the Academy program to potential students. Economists have two questions about the economic value of credentials like the CCNA. The first is whether it permits the recipient to earn more than he or she otherwise would have earned. The second is whether any wage premium that is earned stems from skills the individual acquired while studying for the credential or whether the premium stems from skills that the individual already had before obtaining the credential but could not signal to employers in the absence of the credential. It has proven extremely difficult to answer these questions for most education and training credentials, and the CCNA credential is no exception in this regard. One survey reports that recipients of the CCNA credential earn approximately 10 percent more than IT workers without this credential (Gabelhouse, 2000~. However, since individuals voluntarily decide whether to try to obtain the credential, it is possible that those who do obtain it differ from those workers who do not in dimensions such as motivation that affect earnings. Consequently the earnings differences could stem from differences between workers who obtain the CCNA and those who do not that are not observed by the researcher but are observed by employers. Thus, all that can be said is that the survey evidence is consistent with community college students' perception that the CCNA credential brings significant economic benefits. Was explained by Adelman (2000), the CCNA is one of a great many IT creden- tials that can be obtained by passing examinations, typically administered by one of three large organizations (including Prometric). The examinations for some creden- tials include hands-on performance assessments. 152 A ROLE FOR THE INTERNET IN AMERICAN EDUCATION?

To learn more about employers' treatment of the CCNA creden- tial, we conducted a computer search for job postings that listed it as required or desired. We found many such jobs, with annual salaries ranging from $35,000 to $70,000. However, as illustrated in Box 5-1, almost all of the jobs, even those paying the lowest salaries, required RICHARD MURNANE, NANCY SHARKEY, AND FRANK LEVY 153

a minimum of one year of experience in computer networking. Adelman (2000) has documented that this is the common pattern for IT jobs. To help solve the experience problem, Cisco has begun a process of matching Academy students and graduates desiring work experience with employers who want help with particular projects. Students in- terested in jobs or internships post resumes on the Academy Web site. (If a student is under 18, he or she is identified only by ID number, not by name.) Employers who want student help post internship and job listings on a Cisco Web site (http://wpl.netacad.net). They imme- diately receive a list of all Academies within 50 miles. The employer then requests that the help-wanted listing be sent to all nearby Acad- emies or to a selected subset. Academy instructors then make avail- able to employers the resumes of those students whose skills seem to fit the job requirements. Employers may contact students at least 18 years of age directly. Academy instructors arrange interviews for students under the age of 18. This example illustrates two points. The Internet can be a valuable resource in improving matchmaking between employers and students (Autor, 2001~. By increasing students' opportunities to gain work experience that is a necessary qualification for most jobs, the Internet may improve incentives for students to do the hard work that skill mastery requires. . . lo. The second point is quite different. Considerable attention is needed to assure that the enormous search potential of the Internet does not result in violations of federal and state laws regarding the privacy of information pertaining to minors. Concern with these laws explains why Cisco collects no information on the characteristics of Academy students, not even names. LESSONS The Academy program differs from state standards-based reform efforts in two notable respects. First, it has a common national cur- riculum. While the merits of this approach can be debated, one ad- vantage we observed is that mobile students find no difficulty in ob- taining credit in one state for Cisco Academy courses completed in another state. Another difference is that aspiring instructors must demonstrate mastery of the curriculum they will teach by passing the same end-of- semester exams that the students take and by passing the national CCNA exam every three years. While this does not guarantee that teachers know how to teach the curriculum, it does provide some assurance of subject matter mastery. The brief history of the Cisco Networking Academies demonstrates that the Internet has the potential to be a valuable resource for improving education. For the Academies, the Internet has been important in developing and improving curriculum, in distributing an up-to-date curriculum to underserved populations, in assessing student skills, in 154 A ROLE FOR THE INTERNET IN AMERICAN EDUCATION?

monitoring the quality of instruction, and in providing teachers with advice on technical and pedagogical issues and with a resource for keeping track of student progress and exam grades. At the same time, the Academies' experience shows that technology is not a substitute for highly motivated teachers with command of their subject matter and with the pedagogical skills to help students master critical skills. To the credit of George Ward and Alex Belous, they never intended that online curriculum and assessment would substitute for first-rate instructors. From the beginning they envi- sioned the Cisco Academies program as blending online resources with high-quality classroom instruction. The brief history of the Academies program given here shows that technology cannot insulate high schools and community colleges from the challenge of attracting and retaining talented teachers who typically are offered salaries that are considerably lower than those they could earn in other jobs. Another lesson is that it takes time to figure out how best to make use of technology. For example, the Academies team has learned that it would be better to have students fill out the course satisfaction surveys before they receive their end-of-course exam grades rather than after. The reason is that analysis of survey responses suggests that students who do not do well on the final exam are particularly critical of the course and the instructor. The team has also learned that retaining for auditing purposes only the single number that repre- sents the average rating students in a class give on all 15 rating questions is a mistake. While a low average grade may identify a quality problem, average responses on individual questions are needed to diagnose the source of the problem. The Academy team also learned that there is a trade-off between keeping the curriculum up- to-date and keeping it stable so that instructors can make lesson plans ahead of time. Too frequent curriculum revisions create more prob- lems than they solve. These examples illustrate that even with a team that included experienced professional educators and experts on information tech- nology, figuring out how to make the most effective use of the Internet to improve education required much trial and error. The important gains do not come from simply substituting online resources for books. They come from developing new teaching and learning roles. For example, in the Academy program the most effective instructors are not lecturers who spend most class time talking while students take notes. They are coaches who engage students in learning from each other and in developing the confidence to assume responsibility for their learning and for commenting constructively on the quality of instruction. For many teachers and students, these are new roles that are not easy to assume. Figuring out how to bring about these cul- tural changes and how to use the Internet to facilitate these changes is a slow process. A final question to consider is whether the Internet is different from the many technological innovations radio, instructional films, RICHARD MURNANE, NANCY SHARKEY, AND FRANK LEVY 155

TV formerly seen as transforming education. In perhaps the most critical respect, the Internet is not different in that it is not a substitute for effective interaction of teachers with students. In other respects, the Internet is somewhat different from previous generations of instructional technologies. The Internet facilitates two-way communication. Not only can Local Academy students and teachers receive exams and new versions of the curriculum over the Internet but also they can send back to the central Academy team information critical to evalu- ating the quality of the exams and the curriculum. A second difference is that the Internet markedly increases the ability of teachers and students to search for information that can be useful in solving problems and constructing arguments. In so doing, it reduces the importance of memorizing facts that can be quickly retrieved from online sources. It also dramatically increases the importance of learning to search efficiently, to sort through the vast amount of information the Web can make available, and to synthesize and make sense of an abundance of information (NRC, 1999~. It is too early to tell exactly how the Internet will alter the demand for particular skills or the capacity of human societies to teach the critical skills. However, the experience of the Cisco Networking Academies suggests that it is a mistake to see technology as substituting for effective teachers in helping students to learn. Instead, the appropriate perspective is that technology will be most valuable in complementing the skills of highly motivated teachers who know their subject matter well and know how to engage students in learning. REFERENCES Adelman, C. (2000~. A parallel postsecondary universe: The certification system in information technology. Washington, DC: U.S. Department of Education, Office of Educational Research and Improvement. Autor, D.H. (2001~. Wiring the labor market. Journal of Economic Perspectives, 15, 35-50. Bishop, J.H. (1995). Expertise and excellence. Ithaca, NY: Cornell University, School of Industrial Relations, Center for Advanced Human Resource Studies. Bishop, J.H., Mane, F., Bishop, M., and Moriarty, J. (2001~. The role of end-of- course exams and minimal competency exams in standards-based reforms. In D. Ravitch (Ed.), Brookings papers in education policy 2001. Washing- ton, DC: The Brookings Institution. Cisco Networking Academy Program. (2000~. Curriculum alignment document. San Jose, CA: Cisco Systems. Cuban, L. (1986~. Teachers and machines: The classroom use of technology since 1920. New York: Teachers College Press. Gabelhouse, G. (2000, DecemberJ. CertMag's salary survey. Certification Magazine. Available: http://www.certmag.com/issues/decO0 [December 12, 2000]. Grossman, P.L. (1990~. The making of a teacher: Teacher knowledge and teacher education. New York: Teachers College Press. Hanushek, E.A. (1994~. Making schools work: Improving performance and control- ling costs. Washington, DC: The Brookings Institution. Johnstone, W.B., and Packer, A.E. (1987~. Workforce 2000: Work and workers for the twenty-first century. Indianapolis, IN: Hudson Institute. 156 A ROLE FOR THE INTERNET IN AMERICAN EDUCATION?

Kershaw, J.A., and McKean, R.N. (1962~. Teacher shortages and salary schedules. NewYork: McGraw-Hill. Marshall, R., and Tucker, M. (1991~. Thinking for a living: Education and the wealth of nations. New York: Basic Books. Mayer, D.P. (1998~. Do teaching standards undermine performance on old tests? Educational Evaluation and Policy Analysis, 20, 53. Murnane, R.J., and Levy, F. (1996~. Teaching the new basic skills. New York: Free Press. Murnane, R.J., and Levy, F. (2001~. Will standards-based reforms improve the education of students of color? National Tax Journal, 54, 401-415. Murnane, R.J., Willett, J.B. and Levy, F. (1995~. The growing importance of cogni- tive skills in wage determination. Review of Economics and Statistics 77, 25 1-266. National Research Council. (1999~. How people learn: Brain, mind, experience, and school. Committee on Developments in the Science of Learning. Bransford, J.D., Brown, A.L., and Cocking, R.R. (Eds.~. Commission on Behavioral and Social Sciences and Education. Washington, DC: National Academy Press. National Institute for Literacy. (2000~. Equipped for the future content standards: What adults need to know and be able to do in the 21st century. Washington, DC: Author. Parrado, E., and Wolff, E. (1999~. Occupational and industrial mobility in the United States, 1969-92. New York: New York University. Secretary's Commission on Achieving Necessary Skills. (1991~. What work requires of schools: A SCANS reportfor America 2000. Washington, DC: U.S. Department of Labor. Zehr, M.A. (1998~. Computer giants look to students. Education Week, April 15. RICHARD MURNANE, NANCY SHARKEY, AND FRANK LEVY 157

;- learning with understanding. The Jenkins framework reminds us that a change in learning goals and assessments often requires a change in teaching and learning strategies as well. In order to learn with understanding, students need to understand why veins and arteries have certain characteristics. For example, arteries carry blood Dom the heart, blood that is pumped in spurts. This helps explain why they would need to be elastic (to handle the spurts). In addition, arsenal blood needs to travel uphill (to the brain) as well as downhill, so the elasticity of the arteries provides an additional advantage. If they constrict behind each uphill spurt, they act as a type of one-way valve that keeps the blood from flowing downhill. Learning to understand relationships such as why artenes are elastic should facilitate subsequent transfer. For example, imagine that students are asked to design an artificial artery. Would it have to be elastic? Students who have only memonzed that arteries are elastic have no grounded way to approach this problem. Students who have learned with understanding know the Unctions of elasticity and hence are freer to consider possibilities like a non-elastic artery that has one-way valves (Bransford and Stein, 19931. Overall, this example illustrates how memorizing versus understanding represent different reaming goals in the Jenkins framework, and how changes in these goals require different types of teaching strategies. The details of one's teaching strategies will also need to vary depending on the knowledge, skills, attitudes, and other characteristics that students bring to the learning task. For example, we noted earlier that some students (e.g., those in the lower grades) may not know enough about pumping, spurts, and elasticity to learn with understanding if they are simply told about the functions of artenes. They may need special scaffolds such as dynamic simulations that display these properties. As a different kind of example, imagine that we want to include mnemonics along with understanding and one of the students in our class is overweight and named Art. Under these coalitions, it would seem unwise to use the mnemonic sentence about Artistry) that was noted above. The Importance of Workina Backwards lo The Jenkins model fits well with a recent book by Wiggins and McTighe entitled Understanding by Design (1997). They suggest a C`working backwards,, strategy for creating high- quality learning experiences. ~ particular, they recommend that educators (~) begin with a careful analysis of reaming goals; (2) explore how to assess students' progress in achieving these goals; and (3) use the results of ~ and 2 to choose and continually evaluate teaching methods. (Assumptions about steps ~ and 2 are also continually evaluated.) When using a "working backwards" strategy, one's choice of teaching strategies denves Tom a carefill analysis of reaming goals, rather than vice versa. In the discussion below, we attempt to clarify the importance of working backwards by discussing some imaginary universities that each set different goals for their students. These different goals have strong effects on what and how the universities teach. The Knowledge Economy and Postsecondary Education: Report of a Workshop Chapter 6 158