Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 65
4
Summaries of Individual Presentations
4.1 COMPUTATIONAL THINKING AND
SCIENTIFIC VISUALIZATION
4.1.1 Questions Addressed
• What are the relevant lessons learned and best practices for improv-
ing computational thinking in K-12 education?
• What are examples of computational thinking and how, if at all,
does computational thinking vary by discipline at the K-12 level?
• What exposures and experiences contribute to developing compu-
tational thinking in the disciplines?
• How do computers and programming fit into computational
thinking?
• What are plausible paths and activities for teaching the most
important computational thinking concepts?
Presenters:
Robert Tinker, Concord Consortium
Mitch Resnick, Massachusetts Institute of Technology
John Jungck, Beloit College, BioQUEST
Idit Caperton, World Wide Workshop
Committee respondent: Uri Wilensky
65
OCR for page 66
66 PEDAGOGICAL ASPECTS OF COMPUTATIONAL THINKING
4.1.2 Robert Tinker, Concord Consortium
The Concord Consortium is a non-profit research and technology
development group that focuses on applying technology to improve
learning at different grades. Robert Tinker, the founder of the Concord
Consortium, argued that computational efforts in K-12 should be inte -
grated around a science focus rather than a focus on either mathematics
or engineering.
Elaborating on this argument, he suggested that computational think-
ing offers an alternative new way of finding out about the world, which
is important for citizenship, for future work, and for professionals of
all types. Nevertheless, he believes that neither the computer science
community nor the education community has yet clearly articulated the
essence of computational thinking. As usually presented, computational
thinking involves abstractions upon abstractions, which are difficult to
make concrete.
At the core of computational thinking, Tinker argued, is the ability
to break big problems into smaller problems until one can automate the
solutions of those smaller problems for rapid response. (It is for this rea -
son that Tinker believes that engineering is not an appropriate integrating
focus for attempts to teach computational thinking—engineering taught
at the K-12 level is not particularly amenable to decomposition.) This
core, he argued, indicates a possible route for introducing computational
thinking into K-12 education.
Tinker’s view is that science is the right focus because modern science
often uses computational models that are based on scientific principles
and whose use depends on visualizations. Understanding these models
requires computational thinking—scientific models and visualizations
allow students to visualize the computations that are going on in near real
time. Tinker noted that students learn better by seeing models and inter-
acting with them, and that by exploring the model in a spirit of inquiry,
they learn about the science in the model in much the same way that
scientists learn about nature by using the scientific method. He argued
that students can learn complicated, deep concepts this way rather than
through the more “off-putting” and often confusing approach of formal-
istic equations.
Tinker proposed an approach across the K-12 curriculum that uses
simple models of scientific concepts such as temperature, light, and force
to teach computational thinking. A progression of concepts could start
in early elementary grades with basic ideas such as “there are numbers
associated with things you observe.” (See Figure 2.1.) In later grades, stu -
dents might manipulate and refine models to reflect more sophisticated
understanding of the concepts represented in the models. Finally, in high
OCR for page 67
67
SUMMARIES OF INDIVIDUAL PRESENTATIONS
school, a student might be able to select, modify, and apply both hardware
and software models as a key part of an extended investigation.
Tinker suggested that the following learning progression could fit
into the K-12 curriculum, improve science teaching and learning, and
introduce important aspects of computational thinking:
1. There are numeric values associated with every object and their
interactions.
2. These values change over time.
3. These changes can be modeled.
4. Models involve lots of simple steps defined by simple rules (e.g.,
the molecular dance).
5. Models can be tested to find their range of applicability.
6. You can make models.
7. Many other applications of computers share the same features.
When asked whether students perform better when learning through
computational modeling and visualization as opposed to a more tra-
ditional approach, he replied that such a distinction is not particularly
important. Rather than worry whether one method is better than the
other, Tinker pointed out that it is a good outcome if a teacher has an
additional tool in his or her arsenal to teach a complicated concept.
Tinker noted that because students begin as concrete thinkers, it
remains a challenge to identify the age or grade level at which children
can handle abstraction. As an example, he said that although he has
worked with second graders by hooking up a probe to measure tempera-
ture, it is only at fourth grade that students demonstrate reliable results
of learning and comprehension with such methods.
According to Tinker, students involved in a very tightly packed K-12
curriculum do not have the time to master programming in order to
manipulate models. Rather, he recommends a programming environ-
ment such as NetLogo or AgentSheets partially populated with general
tools, but still needing interconnection and “tuning,” that were designed
to focus users on the concepts represented rather than on the details of
programming. Another option is to use an existing piece of software in
which the student can manipulate important parameters.
4.1.3 Mitch Resnick, Massachusetts Institute of Technology
Mitch Resnick of the MIT Media Lab said that computational think-
ers must be able to use computational media to create, build, and invent
solutions to problems. He framed this approach in terms of students being
able to express themselves and their ideas in computational terms, and
OCR for page 68
68 PEDAGOGICAL ASPECTS OF COMPUTATIONAL THINKING
emphasized that indeed this should be part of the motivation to learn
computational thinking. “When young people learn about language, we
don’t just teach them linguistics or grammar; we let them express them -
selves. We want a similar thing with computational thinking.”
Moreover, he argued, most people work better on things they care
about and that are meaningful to them, and so embedding the study
of abstraction in concrete activity helps to make it meaningful and
understandable.
Resnick pointed out that for students to express themselves meaning-
fully with computational media, they need to learn new concepts as well
as develop new capacities. He argued that computer science classes often
overemphasize computational thinking concepts (such as recursion) at
the expense of helping students develop computational thinking capaci -
ties for design and social cooperation. Computational concepts include
concepts such as conditionals, processes, synchronization, and recursion.
Design capacities deal with skills like prototyping, abstracting, modu -
larizing, and debugging. Social-cooperative capacities include sharing,
collaborating, remixing, and crowd-sourcing. These social-cooperative
capacities are becoming increasingly important as new computing and
networking technologies open up new possibilities for widespread coop -
eration.
Resnick’s computational environment of choice for supporting com -
putational expression is Scratch. The MIT Media Lab developed Scratch
and a companion online community to help engage people in creative
learning experiences and to support the development of computational
thinking. Scratch is a graphical programming language, giving the user
the ability to build programs by snapping together graphical blocks that
control the actions of different dynamic actors on a screen. (Such an
approach to program construction enables users to avoid issues of syn -
tax and other details that often distract users from the critical processes
of designing, creating, and inventing. Resnick believes this construction
process serves an important grounding function for learning abstract
computational concepts, making concepts more concrete and understand-
able.) Scratch also facilitates social cooperation by making it very easy for
a user to share his or her design with others for comment and feedback.
(See Figure 2.2.)
Resnick provided several examples emphasizing the role of expres -
sion through construction and social cooperation from one particular
member of the Scratch online community who goes by the username
MyRedNeptune. MyRedNeptune was a young student from Moscow and
joined the Scratch online community shortly after it went live in 2007.
“One of the first projects that she created,” Resnick said, “was a type of
interactive greeting card for the holidays.” Each time a person clicked
OCR for page 69
69
SUMMARIES OF INDIVIDUAL PRESENTATIONS
on one of the reindeer, it would begin playing “We Wish You a Merry
Christmas” on a musical instrument in concert with the other animated
reindeer. Creating the card required modularization and synchronization,
as well as a number of core computational concepts.
Next, MyRedNeptune began offering her consulting services to
develop animated characters upon request. Another community member
requested that she develop a cheetah for a project. Resnick continued,
“She went to the National Geographic website and she found a video of
a cheetah. She used that to help guide the graphic of an animation that
she developed, and then someone else used her graphic and integrated
it into her project.”
When yet another community member requested that she show how
she developed her animations, she began to develop tutorials in Scratch
on how to program animated characters. One of her first tutorials was on
how to animate a bird to make its wings flap back and forth. Later she was
asked to participate in an international collaboration with five or six other
kids in three different countries. Working together, they developed a type
of adventure video game, with each child working on different parts of
the activity. Resnick noted: “I think you can see from these examples how
MyRedNeptune developed as a computational thinker, learning to think
creatively, reason systematically, and work collaboratively.”
Scratch is used both inside and outside formal school curricula. Ini -
tially used in homes, after-school centers, community centers, and muse -
ums, it is now moving into the schools and is being used today to teach
basic concepts in university-level introductory computer science classes
in a number of universities.
Resnick shared that “one thing we’ve seen is that different kids have
different trajectories. Some will spend a lot of time continuing to work on
the same types of projects, over and over. You might think that they are
stuck, but there’s a lot of things happening in their minds, and suddenly
they’ll start working on new types of projects and ideas.”
Resnick and his colleagues are working on many new initiatives to
support the development of computational thinking through Scratch,
including an online community (called ScratchEd) specifically for teachers
who are helping students learn with Scratch.
4.1.4 John Jungck, Beloit College, BioQUEST
John Jungck and his collaborators founded the BioQUEST Curricu -
lum Consortium 24 years ago to bring computation and mathematics
into the undergraduate biology curriculum. Jungck noted that although
there are many reasons for using computation in biology education, the
rationale he presented at the workshop focused on the power of visual -
OCR for page 70
70 PEDAGOGICAL ASPECTS OF COMPUTATIONAL THINKING
ization in a biological context. He noted the evolution of paradigms for
scientific investigation from empirical (experiment, observation) to theo -
retical (models, theoretical generalizations) to computational (simulation)
to data exploration and e-science (collection of data on a massive scale:
exploration facilitated by theory, mathematics, statistics, and computer
science). In this context, biology education needs to provide students
with ways of understanding biological data—environmental data and
genomic data, for example—that is multivariate, multidimensional, and
multicausal and that exists at multiple scales in enormous volume (tera -
bytes of data per day).
The philosophy of BioQUEST rests on three pillars:
• Students take an active role in posing problems to examine, much
as a scientist has to learn to pose good problems. Good problems must be
appealing, have significance, and be feasible to address.
• Students solve problems iteratively. They must learn to appreciate
the nature of scientific hypotheses as answers as well as to develop heu-
ristics for achieving closure to scientific problems.
• Students must persuade their peers that a solution is useful and
or valid, a process that mirrors the role of publication and extensive peer
review in biological research.
The primary challenge for learning in accordance with this philoso -
phy is that in focusing on the student as problem-poser, teachers lose
much of the control they traditionally have over the learning process. Stu-
dents engaging in self-directed collaborative processes may make some
teachers uncomfortable. Furthermore, students in this environment may
have more technical skill than their teachers, and so peer review from
other classmates may be more important than teacher feedback as far as
advancing learning.
Jungck briefly described a number of BioQUEST projects. For exam-
ple, one project sought to develop student facility with the idea of biologi-
cal modeling with equations. For this purpose, the Biological ESTEEM
project (ESTEEM stands for Excel Simulations and Tools for Exploratory
Experiential Mathematics) seeks to provide students with a mathemati -
cal vocabulary for describing common modeling concepts (e.g., linear,
exponential, and logistic growth1).
Another BioQUEST project (BEDROCK) focuses on bioinformatics.
The BEDROCK project requires students to use a supercomputer tool
1 The Biological ESTEEM Collection, website, BioQUEST Curriculum Consortium, http://
bioquest.org/esteem/index.php. Last accessed February 7, 2011.
OCR for page 71
71
SUMMARIES OF INDIVIDUAL PRESENTATIONS
called Biology Workbench,2 which allows biologists to search many popu-
lar protein and nucleic acid sequence databases. Database searching is
integrated with access to a wide variety of analysis and modeling tools.
Students can align multiple sequences of a particular gene from different
organisms onto one three-dimensional structure and see the evolutionary
conservation involved; they can thus relate the comparative biology of
sequences to structure, function, and phylogeny.
Yet another project is BIRDD (Beagle Investigations Return with Dar-
winian Data), whose goal is to provide a variety of resources related
to evolutionary research. Labs are rare in courses dealing with evolu -
tion, largely because evolutionary phenomena involve temporal and
geographic scales that make it difficult for instructors to develop labs
comparable to those in biochemistry, physiology, or behavior. BIRDD
addresses this problem by providing raw data (e.g., bird songs, sequence
data, rainfall, breeding sites, and so on) and pedagogical ideas to help
instructors structure appropriate pedagogical experiences for their stu-
dents. BIRDD helps students generate questions and look at, for instance,
whether character displacement happens when the species co-occur or
when they inhabit different islands.
To illustrate the special relationship between biology on one hand and
mathematics and computation on the other, Jungck noted 10 equations
that have driven substantial amounts of biological research and for which
numerous educational materials have been developed:3
1. Fisher’s fundamental theorem of natural selection,
2. Cormack-Hounsfield computer assisted tomography,
3. Genetic mapping (units = morgans; the Haldane function),
4. Fitch-Margoliash little maximum parsimony algorithm (Penny
and Hendy—Molecular Phylogenetic Trees—Bioinformatics),
5. Lotka-Volterra interspecific competition logistic equations,
6. Hodgkin-Huxley equations for neural axon membrane potential,
7. Michaelis-Menten equation for enzyme kinetics (Jacob and Monod),
8. Allometry (e.g., MacArthur-Wilson species area law and conservation),
9. Hypothesis testing (e.g., Luria-Delbrück fluctuation test), and
10. Crick-Griffith-Orgel comma-free coding theory.
2 BEDROCK (Bioinformatics Education Dissemination: Reaching Out, Connecting, and
Knitting-together), website, BioQUEST Curriculum Consortium, http://bioquest.org/
bedrock/about.php. Last accessed February 7, 2011.
3 John Jungck, 1997, “Ten Equations That Changed Biology: Mathematics in Problem-
Solving Biology Curricula,” Bioscience 23(1):11-36.
OCR for page 72
72 PEDAGOGICAL ASPECTS OF COMPUTATIONAL THINKING
He also noted that the typical biology textbook contains only a hand-
ful of equations, and even those are linear equations, and expressed his
surprise that biological visualizations, important as they are to the way
biologists think about the world, are not accompanied by the tools needed
to interpret different kinds of multivariate, multidimensional biological
data.
Finally, Jungck discussed the Visible Human Explorer (VHE). Accord-
ing to the VHE website,4 the VHE is an experimental user interface for
browsing the National Library of Medicine’s (NLM’s) Visible Human data
set, which is based on two digitized cadavers in the National Institutes
of Health Visible Human data set. The interface allows users to browse
a miniature Visible Human volume, locate images of interest, and auto -
matically retrieve desired full-resolution images from the NLM archive.
Jungck concluded by noting that computers and computation have
transformed biology. He noted a quote from Michael Levitt (a structural
biologist at Stanford) that “computers have changed biology forever,
even if most biologists don’t yet realize it.” Educationally, he stressed
the work of di Sessa, Parnafes, and others who emphasize the impor-
tance of engaging students in constructing, revising, inventing, inspect -
ing, critiquing, and using rich visualizations for promoting conceptual
understanding.
4.1.5 Idit Caperton, World Wide Workshop, Globaloria
Idit Caperton described Globaloria as a platform, a transformative
social media learning network, with a comprehensive hybrid course
(online/in class) for playing and making games. It includes a customiz -
able curriculum, community-developed resources, tools, tutorials, and
expert support. Students and educators learn how to create their own
web games, produce wikis, publish rich-media blogs, and openly share
and exchange ideas, game code, questions, and progress using the lat-
est learning methods and digital communication technology. Globaloria
is a project-based learning environment for stimulating computational
creativity as well as inventiveness in youth and educators as a necessary
skill for the 21st century. Computational projects are built around a range
of topics, such as health, climate, alternative energy, civics, mathematics,
biology, social studies, and literature.
The World Wide Workshop’s innovative R&D and pedagogical
approaches to platforms and tools for cultivating computational think -
4 Human Visible Explorer, website, Human Computer Interaction Lab, University of
Maryland, http://www.cs.umd.edu/hcil/visible-human/vhe.shtml. Last accessed Febru -
ary 7, 2011.
OCR for page 73
73
SUMMARIES OF INDIVIDUAL PRESENTATIONS
ing and computational inventiveness have roots in Caperton’s MIT and
Harvard research, and in educational theories about the value of project-
based, multidisciplinary, innovative and creative learning (of any subject)
through software design and programming.5
Caperton also described Globaloria as a customizable textbook com-
prising three main units. An introductory or “getting started” unit pro-
vides students with the opportunity to establish their own project spaces
on the wiki network and to review existing games’ operation and their
codes. A following unit is “game design,” in which students design an
original game about a complex topic (in science, math, health, civics)
and a social issue that matters to them. Students come up with an idea,
assemble teams, do research, build and videotape their paper prototypes,
and construct a concept and a demonstration that they present, both
physically and online via web conferencing. Using Flash text and drawing
and animation techniques, they program an interactive demonstration of
their game concepts. A third unit is “game development,” in which stu -
dents develop their game concepts and demonstrations into a complete,
interactive game. Each unit contains a structured set of learning topics, as
well as projects and assignments structured to help students create critical
parts for their own original game.
Globaloria seeks to impart to students six contemporary learning
abilities: the ability to imagine, design, prototype, and program an edu -
cational game, wiki, or sim; the ability to use project management skills
in developing programmable wiki systems in a Web 2.0 environment; the
ability to produce animated media, programming, publishing, and dis -
tributing interactive purposeful digital media in social networks; the abil-
ity to learn in a social constructionist manner and to participate actively
in the public exchanges of ideas and artifacts; the ability to undertake
information-based learning, search, and exploration as they relate to the
abilities above; and the ability to surf websites and use web applications
thoughtfully as they relate to the earlier abilities enumerated. Caperton
argued that these abilities go beyond the typical media literacy skills,
since they emphasize a bundle of complex and sophisticated construc -
tionist digital literacies and involve longer-term engagement (students
are required to use Globaloria daily, over two semesters, for a minimum
of 100-150 hours6).
The Globaloria approach emphasizes constructionist collaboration
5 The canonical examples of such research are Idit Caperton, 1991, Children Designers,
and Idit Caperton and Seymour Papert, 1991, Constructionism, both published by Ablex,
Norwood, New Jersey
6 Caperton recommended repeating the use of Globaloria year after year for greater effects
on computational thinking in learners.
OCR for page 74
74 PEDAGOGICAL ASPECTS OF COMPUTATIONAL THINKING
within a transparent community. Participants in the community—teach-
ers, students, staff, and game teams—maintain public blogs as design
journals, share resources, and publish completed games on the commu-
nity wiki. They can also submit created games for competitions or for
publishing on the school’s Globaloria network.
Caperton suggested that it is possible to learn any subject and to
master complex topics or social issues by creating functional, representa-
tional, educational multimodal computer games involving that subject’s
content. She provided “10 design principles for implementation ‘The
Globaloria Way.’” For example, developing educational games requires
students to spend significant time, engaging daily on personally chosen
projects involving open-ended and creative design tasks. A transparent
and collaborative studio environment facilitates the sharing of work and
provides many opportunities for social expression and discussion about
game projects. Students thus learn through four modes simultaneously:
(1) through design and teaching, (2) through peer-to-peer interactions,
(3) through co-learning with teachers (and also from watching the teach-
ers themselves learn), and (4) from online research and consultation with
other experts (just-in-time learning) via pre-scheduled web conferencing
and a help desk. (See Figure 2.3.)
The basic technology underlying the Globaloria platform is open-
source MediaWiki with customized MediaWiki extensions, PHP, MySQL,
Tumblr, Blogger and multiple Google tools. Students learn to program
their games much like professionals in the real world using Adobe Flash
Actionscript. The World Wide Workshop Foundation’s team (creators
of Globaloria) chose Flash for students’ programming for a number of
reasons, including:
• They themselves are expert developers in Flash;
• Flash provides a wide variety of tools, such as interfaces and video
tutorials, to support users and thus can support a range of skill levels
from novice to professional;
• Flash’s capability is present on many websites and in simulations
and media devices;
• Flash is an industry professional standard in game development
and multimedia programming, and so proficiency in Flash is likely to help
provide students with internships and job opportunities in the future.
Finally, Caperton described research she and colleagues conducted
on the impact of implementing models of Globaloria for fostering com -
putational thinking and inventiveness among low-income rural students
and low-income minority urban schoolchildren: (1) Model 1 in 45 schools
throughout the public school system in 20 counties in the state of West
OCR for page 75
75
SUMMARIES OF INDIVIDUAL PRESENTATIONS
Virginia, where 1,300 students in rural middle schools, high schools,
community colleges, and alternative education institutions participated
with 55 educators in 2010 for credit and a grade; and (2) Model 2 within a
charter middle school system in East Austin, Texas, where every student
in that school took Globaloria once a day for 90 minutes for the entire
school year. She provided an overview of selected research results7 and
shared video case studies.8 Caperton argued that these were powerful
demonstrations of plausible paths and activities for teaching compu-
tational thinking concepts to low-income rural and urban students of
underserved communities.
4.2 COMPUTATIONAL THINKING AND TECHNOLOGY
4.2.1 Questions Addressed
• What are the relevant lessons learned and best practices for improv-
ing computational thinking in K-12 education?
• What are examples of computational thinking and how, if at all,
does computational thinking vary by discipline at the K-12 level?
• What exposures and experiences contribute to developing compu-
tational thinking in the disciplines?
• How do computers and programming fit into computational
thinking?
• What are plausible paths and activities for teaching the most
important computational thinking concepts?
Presenters:
Robert Panoff, Shodor Education Foundation
Stephen Uzzo, New York Hall of Science
Jill Denner, Education, Training, Research Associates
Committee respondent: Yasmin Kafai
4.2.2 Robert Panoff, Shodor Education Foundation
Robert M. Panoff, founder and executive director of the Shodor
Education Foundation, is a proponent of teaching computational think -
ing through computational science. At the same time, he stresses the
7 For more information see www.WorldWideWorkshop.org/reports. Last accessed Febru -
ary 7, 2011.
8 For more information see www.worldwideworkshop.org/programs/globaloria/vftf.
Last accessed February 7, 2011.
OCR for page 122
122 PEDAGOGICAL ASPECTS OF COMPUTATIONAL THINKING
ing ideal gases in physics. In particular, he noted that the laws of ideal
gases traditionally entail equations such as the Maxwell Boltzmann dis -
tribution and the relationship between pressure and volume. Blikstein
offered an alternative restructuration based on computational thinking
that represents a gas as a collection of molecules moving in a gas cham -
ber governed by a simple rule: a molecule will move forward until or
unless it bumps into another molecule or wall, at which point it will
bounce back. This simple rule applied in this agent-based model results
in aggregate behavior of the collection of gas molecules that is identical
to that described by the formal gas law equations. Blikstein asserted that
the computational representation of the gas laws is simpler and easier to
learn than are the equations.
He also described how to reformulate a number of complex con-
cepts from undergraduate-level materials science. Blikstein noted that
students in traditional introductory materials science courses encounter
new equations at a very rapid rate (one new equation every 150 seconds,
not counting intermediate steps in a derivation). It is often that many dif -
ferent equations and models are needed to develop an understanding of
a particular concept. These equations must be combined and manipulated
to arrive at the final result.
Blikstein argued that an agent-based approach helps students to
explore these complex and intertwined concepts more easily, and further
that the rules and mechanisms governing the behavior of individual
atoms can be used to understand a number of different crystal phenom-
ena in materials science, such as growth, solidification, diffusion, and so
on. An example of a relevant mechanism might be for molecules to “look
around and see if they are surrounded by different neighbors or equal
neighbors” and then cluster or disperse “based on their neighborhood.”
Similarly, solidification follows a comparable process except that an “atom
in the liquid is kind of going around and looking for solid neighbors
where it can attach itself.”
Blikstein also described some of the challenges in assessing and giv -
ing objective feedback on open-ended projects with varying levels of com-
plexity and explanatory power. These challenges included the following:
• How do we go about looking at various artifacts and understand-
ing what students are doing?
• How do we assess the relative levels of complexity of the artifacts?
• How do we use assessment to provide feedback to students to
improve their models as well as their understanding of concepts?
Blikstein described several tools to facilitate assessment—rubrics and
maps, coding patterns over time, and representational shifts.
OCR for page 123
123
SUMMARIES OF INDIVIDUAL PRESENTATIONS
• Rubrics and maps. Based on the actual code that students generate,
maps can be created that track the programming steps and decisions stu-
dents were making. These maps capture many dimensions of students’
decision making, such as how they define the system, how they define the
rules of the system, how they define what the agents are doing, and so on.
From these large maps, it is possible to categorize the rules embedded in
the system and assess the sophistication of the rules. Evaluators can check
each map to see if a student used various affordances of the programming
language. For example, is this student using collisions? Is this student
using neighborhood checking? Agents moving? Agents seeking agent
clusters, walls, or energy? Blikstein argued that the greater the number
of affordances appearing in a map, the more sophisticated the underlying
model is likely to be, although this measure is not absolute and in many
respects depends on the phenomenon being modeled.
• Coding patterns over time. Such patterns document how a student’s
code changes over time (e.g., what is added or deleted, what is found each
time compilation is attempted). For example, one can count the number
of characters in a program submitted for compilation. Some students—
typically novice students—exhibit a pattern in which the code is more or
less constant for several compilations but then jumps significantly in size.
For other students (typically more expert students), there are fewer large
increases in code size—code size increases more or less linearly over time.
Blikstein asserted that such knowledge can be exploited to help tailor the
most effective way to give feedback to different kinds of students.
• Representational shifts. Changes in how a student represents or
depicts physical phenomena can indicate differences in the level of sophis-
tication of his or her understanding. For example, Blikstein compared two
groups of students, one that had been exposed to computational model-
ing and one that had not. Each group was asked to sketch the process
involved in a scientific phenomenon different from the one they were
modeling, such as the impact of a change in temperature. The students
with computational modeling experience drew and described a mecha-
nism showing the behavior of the atoms as the temperature changed.
Students who were not exposed to the activity instead drew a graphical
curve showing the aggregate behavior of the atoms as the temperature
changed.
4.6.3 Christina Schwarz, Michigan State University
Christina Schwarz, an associate professor in the College of Educa-
tion at Michigan State University, described her work with elementary
and middle school students using scientific modeling and practices.
The MoDeLS (Modeling Designs for Learning Science) project works
OCR for page 124
124 PEDAGOGICAL ASPECTS OF COMPUTATIONAL THINKING
to involve students in science through the use, revision, and creation of
models. Although not explicitly focused on computational models, some
of her work, Schwarz believes, may apply to the ongoing dialog about
computational thinking.
In the context of her work, a model is an abstract, simplified repre -
sentation of some phenomenon which could include but is not limited to
computational representations. Models also include physical models and
diagrammatic models. Modeling involves constructing a representation
that embodies aspects of theory or evidence; evaluating that representa-
tion or testing it against empirical evidence and scientific theory; using it
to illustrate, predict, and explain; and revising the representation.
Schwarz and her colleagues believe that the underlying concepts
of modeling are powerful for sense-making and for communication in
science. She further noted an overlap between modeling practice and
computational thinking, particularly the ideas of abstracting and decom -
posing systems, testing the model against actual data, and so on.
Schwarz argued that models can make important aspects of science
accessible by helping students to understand invisible processes, mecha -
nisms, and components in phenomena. Models promote both subject-
matter and epistemological understanding, and they develop systems
thinking skills. Most importantly, models can generate predictions and
explanations for scientific phenomena.
Schwarz walked through a generic MoDeLS curriculum sequence that
would be given to students. The first step is for the researchers to provide
some sort of anchoring phenomenon in a scientific context. For example,
a fifth grade unit starts with a question like, Would you drink liquid that
collected in a solar still?, and continues, “You can’t test it, you don’t want
to drink it, because you might get sick, so you have to design an initial
model that you can use to begin thinking through what is going on.”
The unit then provides some discussion about the nature and purpose
of models. Such dialog is essential to abstracting knowledge for transfer
to other kinds of systems and contexts, and to motivate and support the
kinds of skills and habits of mind essential to computational thinking.
The third element of the unit is an investigation of the subject phe -
nomenon through data gathering and students’ testing of their models.
Students evaluate their models and discuss the criteria for evaluation.
Evaluation is thus another strategy for teasing out modeling practice and
scientific thinking.
Last, the unit introduces scientific ideas that students can use to
revise their models again. Here, students often use visual, diagrammatic
models and computer simulations in different ways. Students may look
at simulations and then use some of those ideas from the simulations in
their diagrammatic models. Model design, revision, and analysis occur in
OCR for page 125
125
SUMMARIES OF INDIVIDUAL PRESENTATIONS
the context of a small scientific learning community (i.e., their classmates).
The students debate data and concepts, as well as evaluate and peer-
review each other’s work. Finally, students develop a consensus model
at the end of the unit and explore applying the model developed to other
contexts that they care about.
Schwarz noted that different science disciplines use different aspects
of modeling to explore scientific phenomena. For example, flow diagrams
and process diagrams might be most appropriate for modeling relation -
ships between components of a biological system. Most of the MoDeLS
effort focuses on various aspects of physical science, but the group is
looking at exploring modeling in other areas.
Schwarz uses a four-level learning progression to guide the interpre-
tation of student activities. This progression is continually revised and
improved based on their assessment outcomes.
• Level I focuses on students’ reflections on their existing practices
of modeling around the idea that children often begin modeling practice
by drawing literal illustrations but have yet to really grasp the purpose
of or use for models.
• Level II characterizes student use of models and shows that stu-
dents are constructing and using models to illustrate and explain to an
audience how phenomena occur. Although students at Level II are still
somewhat literal, they are moving closer to the use of abstraction.
• Level III is even more sophisticated, as students move farther along
the literal-to-abstract scale closer to the abstract end of the spectrum.
• Level IV students are constructing and using models spontane-
ously in a range of domains to help their thinking and problem solving.
For example, students might be prompted to consider, before they test
their model, how the world would behave. Schwarz argued that this
fourth level is most similar to the types of modeling a computer scientist
would do.
Schwarz also commented on assessment. Specifically, she noted
that her assessments seek evidence in student work of engagement in
modeling:
• Around different content knowledge for which students did not
receive explicit instruction—to determine what aspects of modeling prac -
tices might be used across contexts.
• By applying their models to familiar and less familiar contexts.
Schwarz described an example in which a student noted that she was
actually applying the condensation and evaporation model to simple
experiences at home like boiling water.
OCR for page 126
126 PEDAGOGICAL ASPECTS OF COMPUTATIONAL THINKING
• Mapping between representations and the real world, as illustrated
by students’ application of their models in a specific context.
• Evaluating and revising their models for items like relevancy or
saliency, evidentiary support, communicative power, and so on.
Schwarz and her colleagues use a variety of tools to obtain such evi -
dence. Although there is some use of written pre-test and post-test items
involving scientific modeling, they also use reflective interviews with
students and in-person or videotaped observations of in-class student
interactions. These qualitative instruments are designed to probe content
that was both explicitly and not explicitly taught to examine transfer to
other disciplines and the time evolution of student modeling practices
and thinking.
Nevertheless, she was aware that their assessment efforts had a num-
ber of limitations. For example, many young students often see modeling
and scientific thinking as a school-only activity that is unrelated to daily
life rather than thinking of models as tools useful for their own purposes.
Although they understand in principle the notion of evaluating each
others’ models according to relevant objective criteria, in practice they
sometimes fail to do so in the classroom environment, instead deferring
to the classmate they like better or the classmate who is the loudest.
Students also sometimes focus on the external audience when com-
municating through a model; that is, they may formulate their comments
and responses based on what they think their teachers want to hear and
what they think are “correct” answers, rather than what they themselves
think.
Last, Schwarz noted that pedagogical constraints often result from the
curricular and learning approaches determined by the various schools.
As an example, Schwarz explained that in one school, the state-wide
curriculum mandated that before fifth grade, science teachers are not to
discuss phenomena at the cellular or atomic level because they are invis -
ible. In response, Schwarz and her colleagues developed a special unit on
evaporation and condensation that was actually an attempt to bridge the
project’s elementary learning goals to a particular state guideline prohibit-
ing discussion of atoms.
4.6.4 Mike Clancy, University of California, Berkeley
Mike Clancy, from the Department of Computer Science at the Uni-
versity of California, Berkeley, addressed the topic of assessment for intro-
ductory programming classes. His top-level goals for students could be
characterized as knowing when given aspects of computational thinking
OCR for page 127
127
SUMMARIES OF INDIVIDUAL PRESENTATIONS
are applicable, when they are not applicable, and how these aspects are
applied when they are applicable.
Clancy described two complementary approaches that are useful in
assessment and evaluation. The first approach is based on case studies.
A case study is an expert solution to a problem that is accompanied by
a narrative of how that solution came to be. The expert, who may be a
faculty member or a teaching assistant, provides a solution that addresses
questions like why one approach to solving the problem was chosen over
another and how problems originating in the first implementation of a
solution were fixed (debugged).
Case studies are intended to make the expert’s thinking visible to
expose his or her design and development decisions. They demonstrate
how abstract concepts are manifest in specific situations. They encourage
reflection and self-monitoring, and they support collaborative learning
and emphasize links among various problem solutions.
A typical problem might be to find the number of days spanned by
two dates in the same year. (This problem arises in the third week of
Berkeley’s introductory programming course for non-majors, at which
point they have been exposed to conditional programming structures
such as “ifs” and how to deal with data but have not yet encountered
recursion.)
One approach splits the solution into three situations—those in which
the dates occur in the same month, those in which the dates occur in con-
secutive months, and those in which the dates are further apart. The first
two situations are relatively easy to address, but the third is harder. Spe -
cifically, the solution for the third case depends on whether the months
involved (including the intervening months, if any) have 28, 29, 30, or 31
days. Sometimes it is possible to kludge a solution when the dates are
about 2 months apart, but if they are any further apart, a more systematic
approach is needed.
At this point, the expert is faced with the question of crafting a solu -
tion to the third case that builds on the work already invested in crafting
a solution to the first two cases. If one realizes that the day-span com-
putation is essentially a subtraction of one date from another, a sensible
approach is to change the representation of the dates involved into things
that are easier to subtract—specifically, the date in month-day format is
transformed into the number of days past January 1 for the year.
Using this idea, that is, finding a uniform representation for dates,
students are then asked to address a number of related problems, such
as computing the difference between two heights, finding the number
of Saturdays spanned by two dates, and finding the number of days
between dates in different centuries. In practice, their task is to under-
stand the original solution (for the problem of computing the number of
OCR for page 128
128 PEDAGOGICAL ASPECTS OF COMPUTATIONAL THINKING
days between two dates in the same year) well enough so that they can
modify the approach accordingly.
This case study also includes a debugging exercise; debugging is
of course another key aspect of computational thinking. Imagine that
the day-span program has been accidentally modified (e.g., one word is
changed). Given the change in the output of the program as a starting
point, students are asked to figure out what was changed and how to fix
the problem.
The second approach used for assessment and evaluation involves
lab-centric instruction, which emphasizes hands-on lab hours supervised
by a teaching assistant rather than lecture and discussion. This instruction
entails a variety of traditional programming tasks, such as writing, modi -
fying, and analyzing a program. But because there is more lab time than
in most lecture/discussion courses, the course also has room for a number
of embedded assessment activities. For example, a lab period often starts
with a quiz, and it provides opportunities for self-tests. “Gated collabo -
rations” enable instructors to pose a question to students, and after any
given student answers, s/he sees the answers of his or her lab mates.
In this environment, lab instructors can monitor most of what the stu-
dents are doing and have a window into much of their thinking and not
just their finished work. Thus, lab instructors can notice confusion when
it occurs and address it immediately to provide targeted tutoring. The
result is that instructors can nip confusion and misconceptions in the bud
rather than having to wait for them to be revealed in some later venue.
4.6.5 Derek Briggs, University of Colorado, Boulder
Derek Briggs of the School of Education at the University of Colo-
rado, Boulder, began by suggesting several questions that he believed
should guide any assessment of computational thinking. His first ques -
tion is, What is being assessed? A prerequisite for assessment is a common
understanding of the important constructs and concepts of the topic being
assessed. In the case of computational thinking, Briggs noted a lack of
consensus on its essential elements and commented that even if one isn’t
willing to put down a thorough definition of what constitutes computa -
tional thinking, there has to be some common ground on the topic. What
are the important elements?
Second, he argued for clarity about why the topic is being assessed.
Briggs identified several possible reasons for assessing student under-
standing of a subject:
• Evaluating a program. If a pedagogical activity purports to promote
OCR for page 129
129
SUMMARIES OF INDIVIDUAL PRESENTATIONS
student learning, the students involved in the activity must be assessed
to see if the claimed learning indeed took place.
• Grading of students. In graded courses, a student’s understanding
of a topic often relates to the grade s/he receives.
• Diagnosing a student’s understanding of a subject in detail. Pinpoint-
ing a student’s misunderstanding of a particular subject-matter point
provides feedback to a teacher about how to direct his or her pedagogi -
cal efforts to address that particular misunderstanding. For this particu-
lar application, multiple concepts of learning progression are helpful.
A learning progression can be regarded as an ordered description of a
specific student’s understanding of a given concept as that student learns
more about it; a description of successively more sophisticated under-
standing of a concept or ways of reasoning in a content domain; and also
an ordered description of a typical student’s understanding of a given
concept as students learn more about it.
• Developing a better intellectual understanding of a subject. It sometimes
happens that an attempt to assess a student’s understanding of a subject
demonstrates that the expert’s understanding of the subject is incomplete,
and it is through the act of developing an instrument, and developing
questions for students that are intended to elicit information about the
subject, that the expert gains insight as to what it is that the expert really
meant.
Third, an instrument for the assessment must be appropriate to the
purpose of the assessment. For example, if the purpose of the assessment
is to grade students, an instrument may need only to record the percent-
age of correct answers provided by a student. However, if the purpose of
the assessment is to diagnose a student’s misunderstandings, the instru -
ment must be constructed in a way that sheds light on the specific nature
of those misunderstandings. Briggs also noted that diagnosing student
misunderstandings does not necessarily entail open-ended interactions
with students—carefully designed multiple-choice items can provide
diagnostic information that is as meaningful as or more meaningful than
that obtained through open-ended interviews.
Finally, Briggs argued for the importance of validating an instru-
ment, contrasting the notion of validity to the notion of reliability. A valid
instrument is one that accurately reflects a student’s knowledge of the
specific concepts of interest (i.e., what the investigator really wishes to
assess), whereas reliability is concerned with the consistency with which
an instrument can produce a given measure. He further noted that low
reliability of an instrument was not necessarily problematic in the context
of formative evaluations for real-time informing of in-class pedagogy or
group-level comparisons.
OCR for page 130
130 PEDAGOGICAL ASPECTS OF COMPUTATIONAL THINKING
4.6.6 Cathy Lachapelle, Museum of Science,
Engineering is Elementary Project
Cathy Lachapelle, director of research and evaluation for the Engi-
neering is Elementary (EiE) project at the Museum of Science, discussed
her assessment and evaluation experiences with that project. EiE is a cur-
riculum development and improvement effort that develops engineering
guides and activities for children in grades 1-5.
Assessments of EiE activities are focused on what students learn and
measure specific learning objectives.46 Lachapelle noted that there is no
existing standard “yardstick” against which to assess student learning
about engineering. Thus, assessment efforts compare progress toward
learning objectives in an EiE activity group to progress among students
in a control group.
Lachapelle suggested that a variety of methods are available for
assessing student learning, depending on the purpose of the assessment:
• Class observation that focuses on collecting qualitative data. Such data
include information obtained from helping the teacher implement EiE,
interviewing students to try to understand their attitudes with respect
to the learning objectives, and observing how they perform against the
learning objectives. To illustrate, Lachapelle noted that one of the learn -
ing objectives is to be able to reason from a model and understand that
a model is representing something in the real world. During class obser-
vation, assessors talk to the teacher and students to see if the students
are grasping the concept. (They might also point out different ways to
structure the lessons so that students better understand the learning objec-
tives.) A degree of uniformity in data collection is obtained by using the
same standards and criteria in each observation.
• Embedded assessments, which are often used by teachers to understand
the pedagogical impact they are having on students as they go along. Embed-
ded assessment can be as simple as examining individual student perfor-
mance on a particular worksheet, so that a teacher can better understand
which students need more help, whether he or she should give clearer
instructions, and so on.
• Paper-and-pencil assessments, which are very difficult to construct but
provide an excellent source of feedback. EiE typically uses these paper-and-
pencil assessments for summative evaluation. A great deal of work is
involved in constructing assessments and testing them, piloting them,
checking them for reliability, and then using them with hundreds of
46 Not all investigation of student learning requires such objectives—specifically, some
research is useful for understanding what students know in general and what they can do
on average.
OCR for page 131
131
SUMMARIES OF INDIVIDUAL PRESENTATIONS
students. For example, developing multiple-choice questions that yield
insight into student thinking is sometimes problematic. Lachapelle and
her colleagues often ask students how they would answer a question,
and unusual or incorrect student answers become alternative choices for
answering the question. For example, Lachapelle said, “We asked kids
what is the function of leaves in a plant and the kids said, to make food.
We would say well why did you choose that answer? And they said
because they make salad. You have learned that things are not always as
they might seem or as you might expect.” Ultimately, they discarded that
particular question.
• Performance assessments, which can be used either by teachers for their
own understanding of what their students are learning (in formative evalua -
tion) or by the curriculum development team as a summative evaluation of what
students learned. This type of testing is also time- and resource-intensive
because the assessment must be administered and scored. EiE uses this
type of assessment in the final project design exercise.
Speaking more broadly, Lachapelle addressed formative and sum-
mative evaluations in the EiE project. All work products require regular
evaluation, including teacher guides, student exercises and activities,
the learning goals, and teacher professional development materials and
activities.
As is usually the case, formative evaluation is used to inform the
development and improvement of products and processes. In the EiE con-
text, formative evaluations seek evidence of growth in students’ under-
standing and skills as stated in EiE learning objectives, determine the
age-appropriateness of lessons and activities, and examine the ease of use
of lessons and materials. Formative evaluation for EiE usually relies on
feedback from teachers and students. Therefore, it is critical that research-
ers make sure that the lines of communication are open and that feedback
received is considered in light of the project’s set evaluation criteria.
Lachapelle explained that if a researcher receives feedback that the project
was great but too troublesome to clean up afterward or the standard for
an activity was that a teacher be able to manage the activity the following
year without any support staff, the activity would be revised accordingly.
The purpose of summative evaluation is to provide evidence to EiE
stakeholders, including funders, school districts, teachers, and parents,
that implementation of specific EiE activity is worthwhile. Robert Panoff
was particularly struck by this concept of “being worthwhile” and argued
that this concept is a key factor in terms of scaling, adoptability, and moti-
vation for using the materials or the exercises. Lachapelle stated that one
criterion for this type of evaluation is to show improved learning of target
concepts among students as compared to a control group of students. The
OCR for page 132
132 PEDAGOGICAL ASPECTS OF COMPUTATIONAL THINKING
control group consists of students in a comparable classroom taught the
same science and engineering topics but not with EiE curriculum materi -
als and tools. In the ideal scenario, EiE has a large pool of teachers from
which part are admitted to the EiE project and the other part remain as
control groups. This process does not always work because of constraints
of funding and time. Another example criterion is that teachers express
increased efficacy and interest in teaching engineering to their students.
Randomized, controlled studies with external evaluators are the pre-
ferred method for evaluating and comparing efforts in education, said
Lachapelle. NSF, for example, prefers this approach when seeking sum-
mative assessments in projects it funds. Unfortunately this type of assess -
ment is very expensive to execute because there is usually a need for a
fairly large number of students in order to randomize whole classrooms
into different testing groups. Also external evaluators are an added cost
and bear their own pros and cons. Although external evaluators are likely
to be more objective in their assessments, they do not have the advantage
of an ongoing relationship with the teachers, administrators, and students
whom they are engaging and thus may miss subtleties that more familiar
evaluators might observe.
In her discussion, Lachapelle cautioned that assessments and evalu -
ations of computational thinking activities and materials require clearly
specified learning objectives, which in turn require some community con-
sensus regarding the content of computational thinking—that is, what is
it that the community wants children at various ages to know (from early
elementary school to college)? In the EiE context, some learning objectives
include being able to identify a process, to explain what a process is in an
engineering context, and to explain why the order of steps in a process
is important.
She also argued that the learning objectives should align with psycho-
logical and developmental learning progressions, since doing so provides
some guidance over time as to where students should be at each stage.
Thus, learning objectives are and should be the object of research and
design. She noted that EiE does extensive literature searches and local
interviews with kids before beginning the design of each of its units in
order to learn more about what kids know. For example, for a unit on
sinking and floating, developers would do a literature search and then
interview local students by asking them things like, “Do you know what
it means to float?,” “Do you understand why things float?,” and so on.
Finally, Lachapelle commented that their assessments are also
designed to address student attitudes toward science and engineering.
Broadly speaking, these assessments indicate that girls tend be interested
in engineering things when framed as helping to improve people’s lives
and boys tend to be interested in engineering things when framed in
terms of constructing engineering artifacts.
