Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 127
5
A Role for the Internet in
American Education?
Lessons from Cisco Networking Academies
Richard Murnane, Nancy Sharkey, and Frank Levy
As of July 2001, Cisco Networking Academies located in all 50
states and 130 countries were teaching more than 160,000 students
how to design, build, and maintain computer networks. The students
learn from a common curriculum offered that is in nine languages
and delivered over the Internet. Since the majority of Cisco Net-
working Academies in the United States reside in public high schools
and community colleges institutions that educate the vast majority
of American youth the team developing the Academy program con-
fronted many of the same problems that beset American education.
This paper describes how the Cisco Academies team dealt with these
problems and particularly the use it made of information technology
in crafting solutions.
We begin by providing a brief description of changes in the American
economy that pose new educational challenges. The next section
explains why we chose to study the Cisco Networking Academy Pro-
gram and the methodology of our case study. We follow this with a
brief history of the Cisco Networking Academies. The fourth section
provides the core of the paper and describes how the Cisco Academies
Richard Murnane is the Thompson Professor of Education and Society at Harvard
University's Graduate School of Education. Nancy Sharkey is a doctoral candidate
at Harvard University Graduate School of Education. Frank Levy is the Daniel Rose
Professor of Urban Economics in the Department of Urban Studies and Planning at
M.I.T. Financial support for the research and writing of this paper was provided by
the National Research Council and the Russell Sage Foundation. We would like to
thank the many people connected to the Cisco Networking Academy Program who
provided information and allowed us to observe their work. We appreciate the
helpful comments on an earlier draft of this paper provided by Clifford Adelman,
Lisa Lynch, and Linda Roberts.
127
OCR for page 128
team dealt with seven challenges that face American education. The
last section provides a summary of key lessons.
CHANGES IN THE AMERICAN ECONOMY
In recent decades, changes in the American economy have pro-
duced striking changes in the labor market earnings of American workers.
Since 1979 the real earnings of workers with no postsecondary educa-
tion, especially males, have declined markedly, both in absolute terms
and relative to the earnings of workers with college degrees. Since
1970 the variation in earnings among workers with the same educa-
tional credentials has increased.
changes in the distribution of earnings reflect to a significant extent
an increase in the demand for skills. The logic is that the college/
high school earnings differential has increased because college graduates
are more likely to possess the skills employers increasingly demand,
and the variation in earnings among workers with the same educa-
tional credentials has increased because the premiums employers pay
to specially skilled workers has increased (see Murnane, Willett, and
Levy, 1995~. This "skills" explanation for the changes in the distri-
bution of labor market earnings raises the obvious question: Which
skills are increasingly valued by employers? A number of blue ribbon
commissions (Secretary's Commission on Achieving Necessary Skills,
1991; National Institute for Literacy, 2000) and several research groups
(Marshall and Tucker, 1991; Murnane and Levy, 1996), have produced
lists. While varying in details, all lists include mathematics, reading,
writing skills, problem-solving skills, computer skills, and the ability
Most economists believe that the
to work prociuct~ve~y with people from different backgrounds.
Among the driving forces underlying current educational reform
efforts is evidence from the National Assessment of Educational Progress
(NAEP) that a great many American students leave high school with-
out mastery of these critical skills. Of particular concern are patterns
in NAEP test score data showing that children of color, the most
rapidly growing groups in the U.S. student population and in the next
generation's labor force, are especially likely to leave school lacking
critical skills. While many factors including low family income con-
tribute to the low average academic achievement of children of color,
low-quality education plays a key role.
In response to concern about skills, almost every state in the
country is engaged in standards-based educational reforms aimed at
improving the skills of American students. It is too early to know
how much of a difference standards-based reforms will make to the
skills of American students (see Murnane and Levy, 2001~. How-
ever, it is clear that the reform efforts are encountering a number of
significant problems. These include:
Designing and implementing professional development efforts that
increase teachers' effectiveness.
128
A ROLE FOR THE INTERNET IN AMERICAN EDUCATION?
OCR for page 129
Recruiting and retaining skilled teachers, especially in subject
areas that pay well outside of teaching.
3. Retaining a focus on developing critical skills for all students and
avoiding the plethora of disjointed programs that compete for
resources and student attention and do not add up to a coherent
educational experience.
4. Improving curricula and keeping them up-to-date.
5. Designing and administering assessments that not
only measure
students' mastery of critical skills but also provide incentives for
teachers to improve instruction.
6. Maintaining high-quality instruction in all classrooms.
7. Providing students with credentials that employers and colleges
value, thereby providing incentives for students to do the hard
work that skill mastery requires.
These problems are not new to American education. Yet their
salience is particularly great today as schools struggle to provide all
students with the critical skills they will need to earn enough to
support families and to participate effectively in a changing demo-
cratic society.
Can new technology contribute to solving these difficult problems?
Over the last 80 years, many innovators have thought so. In 1922
Thomas Edison stated: "I believe that the motion picture is destined
to revolutionize our educational system and that in a few years it will
supplant largely, if not entirely, the use of textbooks" (cited in Cuban,
1986, p. 9~. In the 1930s, enthusiasts touted radio as "textbooks of
the air" (p. 19~. In the first decades after World War II, advocates
saw instructional television as bringing the world to the classroom.
More recently computers were seen by some as revolutionizing edu-
cation. In 1984 the computer pioneer Seymour Papert wrote: "There
won't be schools in the future.... I think the computer will blow up
the school. That is. the school defined as something where there are
.
O
classes, teachers running exams, people structured in groups bv age.
following a curriculum all of that."
A, ~ , A,
Contrary to these prophecies, the educational historian Larry Cuban
concludes that none of these new technologies have had a major
impact on how education takes place in American schools. None
have replaced the teacher and his or her daily work with 15 to 40
students in a physical classroom as the core technology of American
education (Cuban, 1986~. The most recent attempt to improve schooling
by making use of technology focuses on the Internet. Will the Internet
be different from past technological innovations that did find their
way into schools but did not revolutionize education? Only time will
tell for sure. However, we can gain insights by examining how one
group, the team that developed the Cisco Networking Academies,
used the Internet to develop and implement a curriculum currently
studied by more than 160,000 students around the world.
RICHARD MURNANE, NANCY SHARKEY, AND FRANK LEVY
129
OCR for page 130
THE CISCO NETWORKING ACADEMY PROGRAM
Why Study This Program?
There were five reasons why we chose to study the Cisco Net-
working Academies. First, the program has grown extraordinarily
rapidly, passing the market test of whether a great many high schools,
community colleges, and not-for-profit organizations find it valuable.
Second, the program is aimed primarily at high school students and
other people who do not have a four-year college degree. As such, it
is an exception to the general pattern in the United States that the
most in-depth training goes to workers who have the most formal
education. Third, in the United States the program is delivered primarily
in public high schools and community colleges, institutions central to
the effort to prepare the next generation of Americans for life in a
rapidly changing society. Understanding how the Academies program
achieved such rapid growth within existing institutions may provide
insights about ways to improve the performance of these institutions.
Fourth, materials describing the program state that it is aligned with
national skills standards. This is intriguing because it suggests the
possibility that the Academies program may not only prepare students
to build and maintain computer networks but also might teach more
generic skills useful in other occupations. Finally, students who com-
plete the program and pass examinations administered by an independent
organization receive credentials that may improve access to good jobs.
The opportunity to earn a credential is interesting because an inability
to signal skill mastery to potential employers may be an important
reason why many American high school students do not do the hard
work that skill mastery requires (see Bishop, Mane, Bishop, and Moriarty,
2001 ).
Until quite recently the Academy program consisted of a four-
semester curriculum that prepares students to take the examination for
the Cisco Certified Network Associate (CCNA) credential. Recently,
new offerings have been added. First, a second four-semester sequence
of courses now leads to a more advanced credential (the Cisco Certified
Network Professional, or CCNP). The second new offering features
partner-sponsored courses on the fundamentals of UNIX and Web
design (developed by Sun Microsystems and Adobe Systems respectively,
but delivered through the Cisco Networking Academy Program). We
focus this paper solely on the initial four-semester sequence of courses.
We want to be clear on the limits of this case study. We do not
seek to evaluate the effectiveness of the Cisco Networking Academies
in providing marketable skills to students.
.. . . . . . . . ~
. . . · . . . . · . ~
In fact, Cisco does not
collect data on students, demographic information and post-program
outcomes that would be needed for a systematic evaluation of the
program. We do not attempt to generalize from this case study about
how education and training programs using the Internet work. We
lack the resources to investigate systematically how different the Cisco
Academies are from the many other information technology-related
130
A ROLE FOR THE INTERNET IN AMERICAN EDUCATION?
OCR for page 131
education and training programs (see Adelman, 2000, for a discussion
of the dramatic growth in IT-related training programs).
Despite these limitations we believe that a description of the develop-
ment of the Cisco Academies program is valuable for two reasons.
First, its extraordinary rate of growth prompts questions about how
the program works and why it is appealing to high schools and com-
munity colleges. Second, learning how the Academies team dealt
with generic problems that hinder improvement in American educa-
tion may provide ideas about how to deal with obstacles to progress.
Methodology
We began work on this case by learning as much as we could
about the history and organization of the Academies program from
materials available on the Internet. This made us realize the impor-
tance of interviewing instructors in Local and Regional Academies,
participating students, and Cisco personnel who played key roles in
the development and administration of the program. We requested
and received permission from Cisco Systems to conduct such inter-
views. We then developed semi-structured interview protocols to guide
the interviews.
We interviewed instructors and students in six Local Academies
and four Regional Academies (all terms defined below) in the New
England area. Approximately half of the Academies were located in
high schools, including two in central city school districts. One was
located in a community-based organization. The rest were located in
publicly supported community colleges. We also spent several hours
observing classes in each of two Academies located in urban high
schools and two located in community colleges.
We spent two days at the Cisco Academies curriculum and assessment
development center in Phoenix, interviewing the Cisco personnel who
started the program and those currently responsible for curriculum,
student assessment, and instructor training. We also conducted tele-
phone interviews with several other Cisco employees and consultants
working on the Academies program.
We sent an early draft of this paper to the Cisco employees and
Academy instructors whom we interviewed, requesting comments and
corrections of factual errors. We used their feedback in revising the
paper.
GENESIS AND DEVELOPMENT
Cisco Systems sells routers, switches, software, and advice on
building and maintaining computer networks to organizations around
the world, including educational institutions. In 1993 John Morgridge,
then CEO of Cisco Systems, hired George Ward to help build Cisco's
market in educational institutions. Ward, a consulting engineer who
had been head of World Wide Networks at Motorola, was well suited
RICHARD MURNANE, NANCY SHARKEY, AND FRANK LEVY
131
OCR for page 132
to the task. Not only did he understand networks; but he also liked
talking to educators and helping them to design networks that met
their schools' needs. Cisco's sales to schools increased.
With the sales growth came an increasingly serious business problem.
Many schools and school districts lacked the expertise to maintain the
networks. Nor did they have the funds to contract out network main-
tenance. In Ward's words, "I'd go in and design Enetworks] and build
them and leave, and they would crash. There was limited support
staff, and even more limited training" (Zehr, 1998~. Ward's response
waLs to develop a 4()-hour training nro~ram aimed at teaching networking
. ·.. . . . . .. . . · . · .. · . . of.
SKlllS to school personnel so they COU1d maintain their networks. -l-ne
program included a PowerPoint presentation and a variety of hands-
on activities. Ward spent much of the next year presenting the train-
ing program at schools around the country. He quickly learned that
his training program was popular; however, he also learned that he
could not meet the burgeoning demand for training. Cisco needed
another approach to supporting school districts' needs for networking
expertise.
An experience during Ward's year of travel provided an idea for a
new approach. At some sites, schools allowed high school students to
sit in on the training. Ward found that the students often grasped the
critical skills more quickly than the adults. This led him to wonder
whether Cisco should train students to maintain their schools' net-
works. To test this idea, Ward asked the principal of Thurgood Marshall
High School, an inner-city public school in San Francisco, if he could
offer a one-week training program to students in August 1996. The
principal, always looking for new ways to engage students, asked one
of his teachers, Dennis Frezzo, to make the arrangements.
Frezzo was an interesting choice. An optical and electrical engl-
neer by background, Frezzo had decided in the early 1990s that he
wanted to try teaching. He earned a master's degree in education
from Stanford and started teaching at Thurgood Marshall in Septem-
ber 1994. Over the next two years, he developed several hands-on
courses aimed at developing students' interest in science. Because
Frezzo knew which Thurgood Marshall students were interested in
science, the school principal asked him to recruit students for the
summer program. However, with only two weeks to recruit students,
Frezzo found that most of the highly motivated students he had in
mind were not available. This left him to dig quite deep into the pool
of motivation and academic talent to recruit a group of students for
Ward's training program.
O
— - - - o 1 0
~ . .
_ _= _ _ _ -- ---- ------- --- -- -or -
_ _ _ _ _
the training program, which was a mixture of Ward's PowerPoint
presentation and many hands-on activities, worked well. Most sur-
prisingly, Ward learned that some students at risk of dropping out due
to a lack of interest in their academic programs were intrigued by his
hands-on training program. This led him to return to Thurgood Marshall
in December and propose that Frezzo and a colleague, Jai Gosine, try
teaching a one-semester course for students. Frezzo and Gosine piloted
132
A ROLE FOR THE INTERNET IN AMERICAN EDUCATION?
OCR for page 133
the course in the spring of 1997, starting out with no curriculum and
no equipment but lots of enthusiasm. The results were sufficiently
positive to lead Ward to take the next steps.
Ward took the idea of training students to John Morgridge, who
was deeply interested in education and believed that technology could
play a key role in improving it. While the CEO had some doubts
about whether high school students could master the relatively complex
skills needed to trouble-shoot network problems, he was intrigued by
Ward's idea and agreed to provide corporate funding for the next
steps.
Now that Ward had support to develop his idea, he needed help.
Recognizing that he knew little about how to develop a good educa-
tional program, he recruited Alex Belous, the director of technology
education for the state of Arizona, to work with him. Belous's career
history was very different from Ward's. After graduating from college
in 1973, Belous started teaching first grade in Cave Creek, a rural
community 40 miles from Phoenix. While teaching elementary school
children in Cave Creek over the next 12 years, Belous came to appre-
ciate the difficulty rural school districts with very limited resources
faced in providing students with a high-quality education. This became
even more apparent to Belous when he became the guidance counselor
at Cave Creek's first high school, built in 1985.
A great many of Cave Creek's high school students were not
going to college and their high school education did not prepare them
to prosper in a changing economy. An entrepreneur at heart, Belous
went to the Arizona State Department of Education and asked how he
could get resources for his school to better serve these students. The
advice was to develop a vocational education program. So Belous
developed a program to teach computer-based office skills to Cave
Creek high school students. This experience led him to believe that
technology could play a role in improving education for rural children.
From Cave Creek, Belous moved to the Arizona State Department of
Education. Through this job, he met George Ward.
Alex Belous was a great find for Ward. He understood curriculum
development and the types of challenges high school teachers would
face in teaching technological skills to students. He also understood
state education bureaucracies. Perhaps most importantly, he shared
Ward's interest in using technology to improve the quality of educa-
tion provided to underserved children.
Working initially as a Cisco consultant, Belous led the effort to
turn Ward's 40-hour training program for adults into a curriculum for
high school students. They agreed on four principles that would
guide their effort:
1. The curriculum would be delivered online; there would be no
printed version.
2. Well-trained instructors would teach the curriculum.
3. The curriculum would be updated frequently to keep it abreast of
RICHARD MURNANE, NANCY SHARKEY, AND FRANK LEVY
133
OCR for page 134
include both content and pedagogy.
changing technology and to improve it as students and instructors
pointed out problems.
4. Assessment of student skills would be done online.
Beyond these general principles, Ward and Belous had no master
plan. When they began to work together, neither imagined that their
ideas would develop over the next five years into an eight-semester
program serving more than 160,000 students.
Encouraged by the interest Thurgood Marshall students showed in
the pilot program, Ward and Belous invited interested educators (many
of whom had participated in Ward's training courses) to a meeting at
Cisco headquarters in San Jose in April 1997. Ward laid out to the
group the idea of an educational program for high school students. To
bring the idea to life, Frezzo described what he and his colleague had
learned while trying out the one-semester program at Thurgood Marshall
High School. Ward then sketched out the possibility of a four-semester
program with an online curriculum and online assessment of student
skills. He emphasized that this would be a partnership. Cisco would
not just provide the curriculum. It would also provide training on
how to teach the curriculum and ongoing support. In return, it would
hold educators accountable for student learning. He asked the partici-
pants whether they would be interested in this partnership. While
some participants were wary of the accountability concept, many expressed
enthusiasm about the potential relationship.
Ward and Belous concluded from the meeting that a significant
number of high schools were interested in teaching courses on design-
ing, building, and maintaining computer networks. Thus, they went
ahead in developing the curriculum and recruiting school partners.
When John Morgridge officially announced the Cisco Networking
Academy Program in October 1997, 64 high schools in seven states
were teaching the first semester of the Academies curriculum. Following
the public announcement, the number of inquiries grew rapidly. While
delighted by the interest, Ward and Belous quickly realized that they
faced new questions.
Who was going to train the instructors? What
would be the financial arrangements of the partnerships? Over the
next months, Ward and Belous and their collaborators worked out
answers to these questions, while at the same time pushing ahead with
the development of Semesters 2, 3, and 4 of the curriculum.
The answer to the training question was to develop a highly leveraged
"train the trainer" model, as illustrated in Figure 5-1. Training would
The first instructors Ward had
trained became the heads of Cisco Academy Training Centers (CATCs).
Cisco would pay them to provide training to instructors at Regional
Academies, who in turn would provide training to instructors at Local
Academies.
Initially Ward and Belous envisioned that the CATCs and Regional
Academies, as well as the Local Academies, would all be in high
schools. Soon, however, they found that community colleges were
134
A ROLE FOR THE INTERNET IN AMERICAN EDUCATION?
OCR for page 135
| Cisco Academy Training Center ~
| Regional Academy
Regional Academy
Local Academy Local Academy Local Academy Local Academy Local Academy Local Academy
FIGURE 5-1 Cisco Networking Academies organizational structure.
interested in becoming CATCs and Regional Academies and that state
governments were interested in having community colleges play this
role. By 2001, half of the Regional Academies in the United States
were located in community colleges. Table 5-1 provides a summary
of the types of institutions in the United States in which Local Academies
and Regional Academies are located. Table 5-2 provides information
on the worldwide growth in the number of Cisco Academies.
TABLE 5-1
and Location
Number of Cisco Acaclemies by Type of Acaclemy, Type of Institution,
Within the United States
Location Local Academies Regional and Local Regional Only
High schools 2,424 67 20
Vocational programs [a] 337 23 1
Junior high schools (grades 7-9) 11 4 2
Two-year community colleges 486 211 20
Four-year colleges 101 35 7
Postsecondary technical schools 72 12 1
Nonprofit organizations 61 7 3
Military service institutions 50 0 1
Other types of organizations 82 19 22
U.S. Total 3,624 378 77
Outside United States
Local Academies Regional and Local Regional only
Non-U.S. Total 2,197 442 174
Worldwide Total 5,821 820 251
SOURCE: Compiled in April 2001 from information available on the Academy locator portion of the Cisco
Academy Web site: http://cisco.netacad.net/cnacs/pub-doc/locator.shtml
NOTE: From the available information, it is not possible to distinguish clearly between Cisco Academies in
vocational high schools serving teenagers and Academies in institutions with the label of vocational education that
serve adults.
RICHARD MURNANE, NANCY SHARKEY, AND FRANK LEVY
135
OCR for page 136
TABLE 5-2 indicators of the Growth of Cisco Networking Academies
WorIc3wicle Academy Statistics
Past (FY '98)
Current (FY '01
Future (Forecast)
Local only 395 6,419
Regional / local 61 859
Subtotal local 396 7,278
Regional 184 271
CATCs 39
Total locations 580 7,559 10,140
Countries 8 130
SOURCE: Cisco Networking Academy Program (2000~.
Over time the answer to the financial arrangements question emerged.
Local Academies receive the online curriculum at no cost. Regional
Academies may charge Local Academies for instructor training, on a
cost-recovery basis. Local Academies need to buy from Cisco a rack
of hardware containing five routers and two switches at a highly sub-
sidized price of less than $10,000. Additional costs to the Local
Academies may include several thousand dollars for testing equip-
ment and disposable materials (such as cables). They are also re-
quired to purchase from Cisco the "SmartNet" service contract for
approximately $1,200 per year after the first year. (The cost of SmartNet
is covered in the Lab Bundle cost in the first year.) This entitles the
Academies to "24/7" technical support for rapid service on their rout-
ers and switches.
Exceptions to these financial arrangements are for Academies in
economic empowerment zones, which are rural and urban areas desig-
nated by the federal government as high unemployment areas, and for
Academies in Native American communities. Cisco provides the package
of routers and switches at no cost to the 100 Academies that are
located in empowerment zones and approximately 13 Academies lo-
cated in Native American communities.
Regional Academies receive one set of routers and switches from
Cisco at no cost. The Cisco Learning Institute, a public charity funded
and formed by Cisco, pays for CATCs to provide training to two
Regional Academy instructors. In return, Regional Academies as-
sume three responsibilities. First, they must make a "good faith ef-
fort" to recruit 10 Local Academies. Second, they provide instructor
training and ongoing support to the Local Academies they recruit.
Third, they monitor the quality of the Local Academy programs. Since
the training, supporting, and monitoring functions use significant resources,
Cisco allows Regional Academies to charge Local Academies annual
support fees and fees for training sufficient to recover their costs.
These fees are set by Regional Academies and range from zero to
several thousand dollars per year.
136
A ROLE FOR THE INTERNET IN AMERICAN EDUCATION?
OCR for page 137
THE ACADEMY PROGRAM'S EXPERIENCE WITH
AMERICAN EDUCATIONAL PROBLEMS
Since most Cisco Networking Academies in the United States are
located in public high schools and community colleges, the program
confronts many of the challenges that face all attempts to improve the
quality of education provided by these institutions. In this section we
describe how the Cisco Academies program has dealt with some of
these problems.
Providing High-Quality Instructor Training
Providing teachers with training (usually called professional develop-
ment in education circles) that improves their effectiveness in help-
ing students to master critical skills has been an ongoing challenge
for American public education. While school districts typically de-
vote significant resources to professional development, most of the
money goes to fund one-day workshops that have little effect on how
teachers teach (see Murnane and Levy, 1996, Ch. 7~.
The Cisco Networking Academy Program faced a particularly
difficult training challenge.
Local Academies chose to teach the Cisco networking curriculum.
Many instructors chosen by the Local Academies were high school
teachers or community college instructors with considerable teaching
experience but no knowledge of computer networking. Others were
adjunct faculty drawn from industry who understood computer net-
working but had no teaching experience. The challenge was to pro-
vide both the technical knowledge needed to teach the curriculum
and the teaching skills needed to motivate students and explain Quite
difficult technical concepts.
It had no control over the instructors
The training program that the Academy team designed with the
help of professional educators provided intensive hands-on instruc-
tion, tests of skill mastery, and ongoing support. Newly designated
Local Academy instructors receive eight days of full-time instruction
on the first semester's curriculum at a Regional Academy. Participants
study the same curriculum that they will subsequently teach. They
are expected to have read the online curriculum before coming to the
training and to review the relevant chapters each night during the
training.
Participants do the same hands-on labs their students will do and
they take the same Internet-based end-of-chapter tests aimed at pro-
viding feedback on their skill mastery. The training guidelines specify
that each participant must do a "teach-back" in each semester. The
teach-back consists of preparing a lesson plan for a particular chapter
and teaching the chapter to other participants and the Regional Academy
instructor. In order to receive authorization to teach the first semester
curriculum, participants must pass the online final examination with
a score of at least 80, as well as successfully complete the teach-
RICHARD MURNANE, NANCY SHARKEY, AND FRANK LEVY
137
OCR for page 148
answer the same questions at the end of each chapter and at the end of
each semester, software randomly assigns the order of the questions
for each test taker. Instructors value the random ordering because it
increases the likelihood that student's responses reflect their own knowl-
edge.
Student's responses to the multiple-choice questions go directly to
the Cisco Academy server in Arizona and students receive back their
exam grade in a minute or two. The exam grades for each student are
automatically entered into a spreadsheet that the instructor can access
to check on student progress and to assign final course grades. Student
responses on individual test items are recorded on the Academy server
and are analyzed by the assessment team to identify problematic ques-
tions. Since the exams are provided to students online from the Acad-
emy server, the assessment team can easily test new questions and
replace inadequate questions with better ones.
The assessment team uses two strategies to align the assessments
with curriculum goals. First, the person who prepares each exam
question documents the particular curriculum goal that the question
addresses. Then each item goes through a multistage quality review.
Second, the end-of-course multiple-choice exam for each semester
is complemented by a hands-on practical examination. While individual
instructors have discretion in designing the hands-on examinations,
the instructor's guide outlines the types of tasks that should be assessed.
For example, the task for the second semester is to build a network
that uses five routers. The hands-on exam for the fourth semester is
comprehensive in that it assesses mastery of skills taught in all four
semesters. For example, students are asked to use IF addresses (Semester 1),
apply a routing protocol (Semester 2), make an access list (Semester
3), and construct a wide area network (Semester 4~. Students must
pass the hands-on examination as well as achieve a satisfactory score
(as defined by the instructor) on the end-of-course multiple-choice
exam in order to receive credit for the course.
Behrens has several initiatives under way to improve the Academy
assessment program and to incorporate technological advances devel-
oped by Educational Testing Service and other organizations on the
frontier of test development. One initiative will overcome the current
limitation that students only learn their score on each examination,
not which items they answered incorrectly. In the future, feedback
will not only include identification of questions answered incorrectly,
but also links to the relevant sections of the curriculum and to addi-
tional resources. A second initiative is the development of multiple
forms of each end-of-semester and chapter test, with comparable scor-
ing across the different forms. This will allow instructors to use
different forms of the tests for different purposes. A third initiative is
the development of new online assessment formats, including micro-
simulations that require students to program emulated routers. A fourth
is the development of adaptive testing that tailors the difficulty of the
148
A ROLE FOR THE INTERNET IN AMERICAN EDUCATION?
OCR for page 149
test items to the skill of the test taker. This permits more accurate
assessment of the skills of individual students.
Maintaining High-Quality Instruction
A premise underlying standards-based educational reforms is that
all children will receive consistently good instruction. Indeed, fulfilling
this premise is necessary to justify withholding grade promotions and
high school diplomas from students who do not score well on state-
mandated standardized assessments. Yet a great many studies over
the last 30 years have documented that the quality of teaching in
American schools varies enormously.8
Assuring that all students enrolled in Cisco Networking Academies
receive consistently good instruction has been a goal of the program
since its inception. In fact, the one uneasy response to George Ward's
initial presentation of the Academy concept to a group of educators
in spring 1997 dealt with accountability. Educators wanted to know
what Ward meant by the term and what responsibilities they would
incur if they signed up to host a Local Academy.
The structural design of the Academy program makes quality assurance
a particularly difficult challenge. Administration is very decentralized.
CATCs are responsible for monitoring the quality of Regional Academies;
in turn, the Regional Academies are responsible for monitoring the
quality of Local Academies. Cisco does not select the instructors in
Local and Regional Academies, nor does it pay them. The only
sanction Cisco can impose on a Local Academy that does not provide
consistently high-quality instruction is to work with the Regional
Academy to deny program affiliation. As a result, the primary strat-
egy for assuring instructional quality is to identify problems as quickly
as possible and then to offer a variety of opportunities for improving
instruction.
The Academy program makes extensive use of information col-
lected online in identifying problems. Student's end-of-semester exam
scores provide one source of information. The Academy Quality
Assurance Team monitors the distribution of scores in each class,
looking for classes in which a large percentage of students earn low
scores. Student surveys provide a second source of information. Every
student completes an online survey at the end of every semester that
asks for ratings of the instructor, the curriculum, and the course
assignments. Instructors can learn about their performance by accessing
online the average rating their students gave on each of the 15 ques-
~Researchers from different disciplines using different methods have documented
the wide variance in teaching quality in American schools. See Hanushek (1994) for
a discussion of the evidence by economists. See Grossman (1990) for evidence on
differences in effectiveness among English teachers and Mayer (1998) for a discus-
sion of differences among math teachers.
RICHARD MURNANE, NANCY SHARKEY, AND FRANK LEVY
149
OCR for page 150
tions in the survey. (See Table 5-3 for an example of feedback to an
instructor from student surveys.) The single number describing the
average rating students gave on the 15 survey questions is sent to the
Regional Academy overseeing the particular Local Academy and to
the Academy Quality Assurance team.
In addition, participants in instructor training complete online evalu-
ations of the quality of their training. The survey asks participants to
rate the class content ("The order of course topics aided my learn-
ing"), as well as the instructor's pedagogical style ("Class participation
was enhanced through effective use of questions") and content knowl-
edge ("Analogies and real-life experiences of the instructor added
value to the course". These ratings are available to the trainer, and a
summary of these ratings is also transmitted to the Quality Assurance
Team. The Academy team also uses the Internet to evaluate com-
plaints about training quality. When the director of training receives
an e-mail message complaining about the quality of a training pro-
gram, he sends an online survey to all participants in that particular
training class. The responses help him to judge whether the initial e-
mail reflects a significant training problem or an isolated personality
conflict.
A primary mechanism for quality assurance is the annual audit. A
three-person team (including one consultant with expertise in peda-
gogy, a second Know~eagean~e anout tecnno~ogy, and one member of
the Cisco Academy team) audits each CATC. It inquires whether the
CATC fulfilled its contractual obligation to conduct annual audits of
the Regional Academies under it. Part of this process involves learn-
ing whether the Regional Academies conducted audits of the Local
Academies under them. The auditors of CATCs also examine the
student test score distributions and average student survey grades for
all Regional and Local Academies under the particular CATC. The
auditing team asks whether the CATC had identified patterns of low
student scores or poor class ratings. It also explores whether the
CATC brought the potential problem to the attention of the relevant
Regional Academy and whether it followed up to see whether the
Regional Academy worked with the relevant instructor to diagnose
the source of the problem and develop a remediation plan.9 Another
part of each audit is to check that instructors of all Academy courses
have kept their CCNA credential up-to-date, a process requiring them
to pass the CCNA exam every three years. This requirement that
instructors periodically demonstrate mastery of subject matter content
is a bone of contention with many instructors. To date, however, the
program has stuck by this requirement.
_ _
. · . . .
90f course, there are several reasons students could have low scores on the end-of-
course examinations, including low reading skills, inadequate facilities, and poor
teaching. The Regional Academy is responsible for exploring the reason for low
scores among students in a Local Academy in its jurisdiction and for suggesting
improvement strategies.
150
A ROLE FOR THE INTERNET IN AMERICAN EDUCATION?
OCR for page 151
TABLE 5-3 Sample of Course Feedback Results
Min Max Mean Question
5
5
5
5
5
5
5
4.85
4.77
4.54
4.54
4.23
4.69
4.69
4.69
4.92
4.15
3.85
4.15
4.38
4.31
4.62
The instructor was adequately prepared to teach this course.
Analogies and real-life experiences of the instructor added value to this course.
Presentations were clear and easy to understand.
Answers to questions were provided in a timely manner.
Class participation was enhanced through effective use of questions.
The class was interesting and enjoyable.
"Best Practices" and good teaching strategies were modeled during the training.
Grouping strategies were utilized effectively.
Class members felt comfortable approaching the instructor with questions/
ideas.
The order of course topics aided my learning.
The course schedule allowed me to complete the stated course objectives.
The activities and labs helped me achieve the stated course objectives.
The lesson assessment tools helped me evaluate my knowledge of the lesson.
Group work aided my learning.
Overall, the course materials were of high quality.
SOURCE: Cisco Academy instructor.
Quality assurance in the Cisco Academy program depends criti-
cally on the efforts of Regional Academies, the majority of which in
the United States are in community colleges. While Cisco allows
Regional Academies to recover the costs they incur in supporting
Local Academies and training their instructors, and a few charge an
annual support fee as high as $15,000, many charge little or nothing.
The rationale, several Regional Academy instructors told us, is that
state legislatures expect community colleges to serve as community
assets. Supporting Cisco Local Academies in area high schools is
one way of demonstrating that commitment. Of course, this arrange-
ment means that the quality of support Local Academies receive and
the integrity of the quality assurance plan depend critically on the
good will of the community colleges and high schools that serve as
Regional Academies.
Signaling Skills
Employers in the United States find it very difficult to obtain
reliable information on the skills of high school graduates who apply
for jobs. The high school diploma indicates only that students com-
pleted four years of high school, not that they mastered particular
skills. High schools typically do not respond quickly to requests for
student transcripts. A consequence of the lack of information on the
skills of high school graduates is that employers typically offer the
same wages and conditions of employment to high school graduates
with strong skills that they offer to graduates with much weaker skills.
RICHARD MURNANE, NANCY SHARKEY, AND FRANK LEVY
151
OCR for page 152
Bishop et al. (2001) have argued that this hiring pattern reduces high
school students' incentives to do the hard work that skill mastery
requires.
Graduates of the four-semester Cisco Networking Academy may
obtain the Cisco Certified Networking Associate (CCNA) credential
by passing the same examination that graduates of commercial train-
ing institutes take. The CCNA exam is a multiple-choice exam administered
by an independent testing organization, Prometric.~° Academy graduates
may also obtain the vender-neutral Network+ credential by passing a
different examination offered by an industry trade organization, CompTIA.
We have no way of assessing whether the opportunity to acquire industry-
recognized credentials has motivated students to enroll in Academy
classes and to work hard to acquire critical skills. However, many
Academy students in community colleges told us that their reason for
enrolling in the Academy courses was to prepare for the CCNA exam.
Credentials that can be earned by passing examinations predate
the Internet by many years. However, the online format of the CCNA
exams mean that scoring is done almost instantaneously and the cre-
dential can be awarded much more quickly than if paper answer sheets
were sent to a central scoring office. This may increase the attractive-
ness of the credential and the Academy program to potential students.
Economists have two questions about the economic value of credentials
like the CCNA. The first is whether it permits the recipient to earn
more than he or she otherwise would have earned. The second is
whether any wage premium that is earned stems from skills the individual
acquired while studying for the credential or whether the premium
stems from skills that the individual already had before obtaining the
credential but could not signal to employers in the absence of the
credential. It has proven extremely difficult to answer these questions
for most education and training credentials, and the CCNA credential
is no exception in this regard. One survey reports that recipients of
the CCNA credential earn approximately 10 percent more than IT
workers without this credential (Gabelhouse, 2000~. However, since
individuals voluntarily decide whether to try to obtain the credential,
it is possible that those who do obtain it differ from those workers
who do not in dimensions such as motivation that affect earnings.
Consequently the earnings differences could stem from differences
between workers who obtain the CCNA and those who do not that are
not observed by the researcher but are observed by employers. Thus,
all that can be said is that the survey evidence is consistent with
community college students' perception that the CCNA credential brings
significant economic benefits.
Was explained by Adelman (2000), the CCNA is one of a great many IT creden-
tials that can be obtained by passing examinations, typically administered by one of
three large organizations (including Prometric). The examinations for some creden-
tials include hands-on performance assessments.
152
A ROLE FOR THE INTERNET IN AMERICAN EDUCATION?
OCR for page 153
To learn more about employers' treatment of the CCNA creden-
tial, we conducted a computer search for job postings that listed it as
required or desired. We found many such jobs, with annual salaries
ranging from $35,000 to $70,000. However, as illustrated in Box 5-1,
almost all of the jobs, even those paying the lowest salaries, required
RICHARD MURNANE, NANCY SHARKEY, AND FRANK LEVY
153
OCR for page 154
a minimum of one year of experience in computer networking. Adelman
(2000) has documented that this is the common pattern for IT jobs.
To help solve the experience problem, Cisco has begun a process
of matching Academy students and graduates desiring work experience
with employers who want help with particular projects. Students in-
terested in jobs or internships post resumes on the Academy Web site.
(If a student is under 18, he or she is identified only by ID number,
not by name.) Employers who want student help post internship and
job listings on a Cisco Web site (http://wpl.netacad.net). They imme-
diately receive a list of all Academies within 50 miles. The employer
then requests that the help-wanted listing be sent to all nearby Acad-
emies or to a selected subset. Academy instructors then make avail-
able to employers the resumes of those students whose skills seem to
fit the job requirements. Employers may contact students at least 18
years of age directly. Academy instructors arrange interviews for
students under the age of 18.
This example illustrates two points. The Internet can be a valuable
resource in improving matchmaking between employers and students
(Autor, 2001~. By increasing students' opportunities to gain work
experience that is a necessary qualification for most jobs, the Internet
may improve incentives for students to do the hard work that skill
mastery requires.
. . lo.
The second point is quite different. Considerable attention is needed
to assure that the enormous search potential of the Internet does not
result in violations of federal and state laws regarding the privacy of
information pertaining to minors. Concern with these laws explains
why Cisco collects no information on the characteristics of Academy
students, not even names.
LESSONS
The Academy program differs from state standards-based reform
efforts in two notable respects. First, it has a common national cur-
riculum. While the merits of this approach can be debated, one ad-
vantage we observed is that mobile students find no difficulty in ob-
taining credit in one state for Cisco Academy courses completed in
another state.
Another difference is that aspiring instructors must demonstrate
mastery of the curriculum they will teach by passing the same end-of-
semester exams that the students take and by passing the national
CCNA exam every three years. While this does not guarantee that
teachers know how to teach the curriculum, it does provide some
assurance of subject matter mastery.
The brief history of the Cisco Networking Academies demonstrates
that the Internet has the potential to be a valuable resource for improving
education. For the Academies, the Internet has been important in
developing and improving curriculum, in distributing an up-to-date
curriculum to underserved populations, in assessing student skills, in
154
A ROLE FOR THE INTERNET IN AMERICAN EDUCATION?
OCR for page 155
monitoring the quality of instruction, and in providing teachers with
advice on technical and pedagogical issues and with a resource for
keeping track of student progress and exam grades.
At the same time, the Academies' experience shows that technology
is not a substitute for highly motivated teachers with command of
their subject matter and with the pedagogical skills to help students
master critical skills. To the credit of George Ward and Alex Belous,
they never intended that online curriculum and assessment would
substitute for first-rate instructors. From the beginning they envi-
sioned the Cisco Academies program as blending online resources
with high-quality classroom instruction. The brief history of the
Academies program given here shows that technology cannot insulate
high schools and community colleges from the challenge of attracting
and retaining talented teachers who typically are offered salaries that
are considerably lower than those they could earn in other jobs.
Another lesson is that it takes time to figure out how best to make
use of technology. For example, the Academies team has learned
that it would be better to have students fill out the course satisfaction
surveys before they receive their end-of-course exam grades rather
than after. The reason is that analysis of survey responses suggests
that students who do not do well on the final exam are particularly
critical of the course and the instructor. The team has also learned
that retaining for auditing purposes only the single number that repre-
sents the average rating students in a class give on all 15 rating
questions is a mistake. While a low average grade may identify a
quality problem, average responses on individual questions are needed
to diagnose the source of the problem. The Academy team also
learned that there is a trade-off between keeping the curriculum up-
to-date and keeping it stable so that instructors can make lesson plans
ahead of time. Too frequent curriculum revisions create more prob-
lems than they solve.
These examples illustrate that even with a team that included
experienced professional educators and experts on information tech-
nology, figuring out how to make the most effective use of the Internet
to improve education required much trial and error. The important
gains do not come from simply substituting online resources for books.
They come from developing new teaching and learning roles. For
example, in the Academy program the most effective instructors are
not lecturers who spend most class time talking while students take
notes. They are coaches who engage students in learning from each
other and in developing the confidence to assume responsibility for
their learning and for commenting constructively on the quality of
instruction. For many teachers and students, these are new roles that
are not easy to assume. Figuring out how to bring about these cul-
tural changes and how to use the Internet to facilitate these changes
is a slow process.
A final question to consider is whether the Internet is different
from the many technological innovations radio, instructional films,
RICHARD MURNANE, NANCY SHARKEY, AND FRANK LEVY
155
OCR for page 156
TV formerly seen as transforming education. In perhaps the most
critical respect, the Internet is not different in that it is not a substitute
for effective interaction of teachers with students. In other respects,
the Internet is somewhat different from previous generations of instructional
technologies. The Internet facilitates two-way communication. Not
only can Local Academy students and teachers receive exams and
new versions of the curriculum over the Internet but also they can
send back to the central Academy team information critical to evalu-
ating the quality of the exams and the curriculum.
A second difference is that the Internet markedly increases the
ability of teachers and students to search for information that can be
useful in solving problems and constructing arguments. In so doing,
it reduces the importance of memorizing facts that can be quickly
retrieved from online sources. It also dramatically increases the importance
of learning to search efficiently, to sort through the vast amount of
information the Web can make available, and to synthesize and make
sense of an abundance of information (NRC, 1999~.
It is too early to tell exactly how the Internet will alter the demand
for particular skills or the capacity of human societies to teach the
critical skills. However, the experience of the Cisco Networking Academies
suggests that it is a mistake to see technology as substituting for
effective teachers in helping students to learn. Instead, the appropriate
perspective is that technology will be most valuable in complementing
the skills of highly motivated teachers who know their subject matter
well and know how to engage students in learning.
REFERENCES
Adelman, C. (2000~. A parallel postsecondary universe: The certification system in
information technology. Washington, DC: U.S. Department of Education,
Office of Educational Research and Improvement.
Autor, D.H. (2001~. Wiring the labor market. Journal of Economic Perspectives, 15,
35-50.
Bishop, J.H. (1995). Expertise and excellence. Ithaca, NY: Cornell University, School
of Industrial Relations, Center for Advanced Human Resource Studies.
Bishop, J.H., Mane, F., Bishop, M., and Moriarty, J. (2001~. The role of end-of-
course exams and minimal competency exams in standards-based reforms.
In D. Ravitch (Ed.), Brookings papers in education policy 2001. Washing-
ton, DC: The Brookings Institution.
Cisco Networking Academy Program. (2000~. Curriculum alignment document. San
Jose, CA: Cisco Systems.
Cuban, L. (1986~. Teachers and machines: The classroom use of technology since
1920. New York: Teachers College Press.
Gabelhouse, G. (2000, DecemberJ. CertMag's salary survey. Certification Magazine.
Available: http://www.certmag.com/issues/decO0 [December 12, 2000].
Grossman, P.L. (1990~. The making of a teacher: Teacher knowledge and teacher
education. New York: Teachers College Press.
Hanushek, E.A. (1994~. Making schools work: Improving performance and control-
ling costs. Washington, DC: The Brookings Institution.
Johnstone, W.B., and Packer, A.E. (1987~. Workforce 2000: Work and workers for
the twenty-first century. Indianapolis, IN: Hudson Institute.
156
A ROLE FOR THE INTERNET IN AMERICAN EDUCATION?
OCR for page 157
Kershaw, J.A., and McKean, R.N. (1962~. Teacher shortages and salary schedules.
NewYork: McGraw-Hill.
Marshall, R., and Tucker, M. (1991~. Thinking for a living: Education and the wealth
of nations. New York: Basic Books.
Mayer, D.P. (1998~. Do teaching standards undermine performance on old tests?
Educational Evaluation and Policy Analysis, 20, 53.
Murnane, R.J., and Levy, F. (1996~. Teaching the new basic skills. New York: Free
Press.
Murnane, R.J., and Levy, F. (2001~. Will standards-based reforms improve the education
of students of color? National Tax Journal, 54, 401-415.
Murnane, R.J., Willett, J.B. and Levy, F. (1995~. The growing importance of cogni-
tive skills in wage determination. Review of Economics and Statistics 77,
25 1-266.
National Research Council. (1999~. How people learn: Brain, mind, experience,
and school. Committee on Developments in the Science of Learning. Bransford,
J.D., Brown, A.L., and Cocking, R.R. (Eds.~. Commission on Behavioral
and Social Sciences and Education. Washington, DC: National Academy
Press.
National Institute for Literacy. (2000~. Equipped for the future content standards:
What adults need to know and be able to do in the 21st century. Washington,
DC: Author.
Parrado, E., and Wolff, E. (1999~. Occupational and industrial mobility in the
United States, 1969-92. New York: New York University.
Secretary's Commission on Achieving Necessary Skills. (1991~. What work requires
of schools: A SCANS reportfor America 2000. Washington, DC: U.S. Department
of Labor.
Zehr, M.A. (1998~. Computer giants look to students. Education Week, April 15.
RICHARD MURNANE, NANCY SHARKEY, AND FRANK LEVY
157
OCR for page 158
;-
learning with understanding. The Jenkins framework reminds us that a change in learning
goals and assessments often requires a change in teaching and learning strategies as well.
In order to learn with understanding, students need to understand why veins and
arteries have certain characteristics. For example, arteries carry blood Dom the heart,
blood that is pumped in spurts. This helps explain why they would need to be elastic (to
handle the spurts). In addition, arsenal blood needs to travel uphill (to the brain) as well
as downhill, so the elasticity of the arteries provides an additional advantage. If they
constrict behind each uphill spurt, they act as a type of one-way valve that keeps the
blood from flowing downhill.
Learning to understand relationships such as why artenes are elastic should
facilitate subsequent transfer. For example, imagine that students are asked to design an
artificial artery. Would it have to be elastic? Students who have only memonzed that
arteries are elastic have no grounded way to approach this problem. Students who have
learned with understanding know the Unctions of elasticity and hence are freer to
consider possibilities like a non-elastic artery that has one-way valves (Bransford and
Stein, 19931.
Overall, this example illustrates how memorizing versus understanding represent
different reaming goals in the Jenkins framework, and how changes in these goals require
different types of teaching strategies. The details of one's teaching strategies will also
need to vary depending on the knowledge, skills, attitudes, and other characteristics that
students bring to the learning task. For example, we noted earlier that some students (e.g.,
those in the lower grades) may not know enough about pumping, spurts, and elasticity to
learn with understanding if they are simply told about the functions of artenes. They may
need special scaffolds such as dynamic simulations that display these properties. As a
different kind of example, imagine that we want to include mnemonics along with
understanding and one of the students in our class is overweight and named Art. Under
these coalitions, it would seem unwise to use the mnemonic sentence about Artistry) that
was noted above.
The Importance of Workina Backwards
lo
The Jenkins model fits well with a recent book by Wiggins and McTighe entitled
Understanding by Design (1997). They suggest a C`working backwards,, strategy for
creating high- quality learning experiences. ~ particular, they recommend that educators
(~) begin with a careful analysis of reaming goals; (2) explore how to assess students'
progress in achieving these goals; and (3) use the results of ~ and 2 to choose and
continually evaluate teaching methods. (Assumptions about steps ~ and 2 are also
continually evaluated.) When using a "working backwards" strategy, one's choice of
teaching strategies denves Tom a carefill analysis of reaming goals, rather than vice
versa. In the discussion below, we attempt to clarify the importance of working
backwards by discussing some imaginary universities that each set different goals for
their students. These different goals have strong effects on what and how the universities
teach.
The Knowledge Economy and Postsecondary
Education: Report of a Workshop
Chapter 6
158
Representative terms from entire chapter:
cisco networking
