Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 7
2
The Field of Cognitive Psychophysiology
In this chapter we define cognitive psychophysiology in terms of
its two parts, cognitive science and psychophysiology.
WHAT IS coGNITWE SCIENCE:?
The Oxford Dictionary of the English Language defines cogni-
tion as "the action or faculty of knowing taken in its widest sense,
including sensation, perception, conception, as distinguished from
feeling and volition; also, more specifically, the action of cognizing
an object in perception proper." Thus, cognitive science is the body
of scientific knowledge pertaining to cognition, defined to include all
forms of knowing.
Cognitive science focuses on questions about how information
must be stored internally and processed in order for an organism to
recognize objects, learn, use language, reason, or navigate. Theories
are tested in part by attempting to build computer programs that
mimic human performance (the so-called computational approach)
and in part by using the experimental methods of cognitive psychol-
ogy.
The computational approach characterizes the nature of infor-
mation processing at two levels of analysis. At one level, theorists
decompose the processing system into sets of "processing modules,"
each of which performs some part of the processing used to accom-
plish a task. Modules are "black boxes," specifying how specific types
of input are transformed to produce appropriate output. Sternberg
7
OCR for page 8
8
BRAIN AND COGNITION: SOME NEW TECHNOLOGIES
(1969), for example, postulated one module that compares an input
stimulus in short-term memory to a set of items on a list.
At another level, theorists attempt to Recover the way in which
processing is actuaBy accomplished within the modules. In some
cases processing is characterized by step-by-step sequential manip-
ulation of stored symbols, as is done in conventional computers,
whereas in other cases, processing corresponds to the formation of
patterns of activation in a network of interconnected nodes, as is
done in parallel distributed processing systems (Rumelhart and Mc-
Cleliand, 1986~. For example, the list-compar~son module posited by
Sternberg could operate either by storing the items in memory as
symbols in a list and then comparing an input symbol against each
stored symbol, or by establishing a pattern of weights distributed
through a neural network. In this latter case, comparison of input
to stored items is accomplished simply by discovering whether the
network settles into a specific state when a given input is presented.
In either case, the computational approach leads one to posit a set of
modules and to characterize how they serve to transform information.
Cognitive psychology has contributed to cognitive science so-
phisticated methodologies, a rich data base on characteristics of
human performance, and techniques for modeling such data. The
methodologies of cognitive psychology are based on observing rela-
tive response times, error rates, or types of judgments. For example,
cognitive psychologists have developed techniques for inferring prop-
erties of processing by analyzing trade-offs between speed and accu-
racy (i.e., the inverse relationship between times and errors, which
reflects how careful a subject is when responding); they have used
signal detection theory in the analysis of errors to determine what is
stored. They have also developed numerous methods for obtaining
judgments of perceived similarity among stimuli. These judgments in
turn can be submitted to multidimensional scaling and cluster analy-
ses, allowing one to draw inferences about the processing underlying
the judgments. Hypotheses derived from theories that embody dif-
ferent modular structures or types of processing are tested against
data. For example, if there is a discrete module that compares input
to lists stored in short-term memory, then it should be possible to
find brain-damaged patients with focal lesions who have lost this
specific ability. In short, then, the result of the alliance between
computational theorizing and cognitive psychology is the develop-
ment of detailed theories of information processing that are not only
consistent with the available data about human performance, but
OCR for page 9
COGNITIVE PSYCHOPHYSIOLOGY
9
that also make empirically testable predictions (see Anderson, 1983;
Kosslyn, 1980; Rumelhart and McClelland, 1986~.
WHAT IS PSYCHOPHYSIOLOGY?
Cognitive psychophysiology refers to the study and use of mea-
sures of physiological functions for the purpose of elucidating pro-
cesses and mechanisms that underlie cognition. The physiological
processes studied include both central nervous system (CNS) and au-
tonom~c nervous system (ANS) activities. Traditionally, psychophys-
iologists interested in the ANS measure such variables as changes in
heart rate or sweat gland activity (Coles, Donchin, and Forges, 1986~.
Studies of the CNS have been dominant in cognitive psychophysi-
ology and are based on more widely developed technologies than
are studies of ANS activity related to cognition. For that reason,
the report concentrates on those activities designed to clarify CNS
mechanisms involved in human cognition.
The human brain ~ largely inaccessible to the sort of fine-grained
analysis other organisms can be subjected to in the pursuit of knowl-
edge about how neuronal activity relates to psychological processes
and states. We already know enough about brain and behavioral
processes to see that a full understanding of another organism is
insufficient to allow a complete appreciation of how the human brain
carries out its appointed tasks. While it is imperative for the student
of human behavior to keep an eye on the developments in under-
standing brain and behavioral processes in nonhuman species, it is
also becoming clear that an understanding of human psychological
processes will require studying human brains at work. This is an
ambitious goal and one not easy to achieve.
THE INTERFACE BETWEEN coGNITWE SCIENCE
AND PSYCHOPHYSIOLOGY
Three fields are currently engaged in the empirical study of men-
tal activity: computational theorists attempt to understand seeing,
remembering, reasoning, and so on by building virtual machines that
mimic such processes. Cognitive psychologists conduct experiments
to measure differences in behavior under different circumstances and
attempt to fit models to account for response times, error rates, or
various types of decisions. Psychophysiologists try to gain insight
into the mind by observing the activity of its neural substrate.
OCR for page 10
10
BRAIN AND COGNITION: SOME NEW TECHNOLOGIES
In addition, it should be noted that scholars in anthropology,
linguistics, and philosophy also address issues about the mind, and
some aspects of cognitive science draw heavily on these fields. How-
ever, this work currently is difficult to connect to psychophysiology;
hence we do not consider these facets of cognitive science further here
or in the sections to follow.
Each of the three fields listed above has virtues as well as lim-
itations. Cognitive science has to a large extent grown out of an
alliance between the computational approach and cognitive psychol-
ogy. The weaknesses of each field taken in isolation are to a large
degree corrected for by the strengths of the others. It seems likely
that psychophysiology has much to gain from interactions with this
new amalgam, and vice versa.
In this section we first treat the virtues and limitations of each
of the major fields, taken singly, to the study of cognition. We then
propose an alliance among them, which would take advantage of each
one's unique strengths—empirical, technological, and theoretical-
while compensating for the limitations inherent in each single field.
The section concludes with a discussion of the advantages of com-
bining them, leading to the suggestion, made in Chapter 6, for an
enlarged study of the interface between the disciplines.
[~ tations and Virtues of a Psychophysiological Approach
Psychophysiological data may be especially useful for identifying
the structure of information processing in the brain. But to be max-
imally useful, they must be used in conjunction with sophisticated
theories and methodologies that are capable of discriminating among
such theories.
Attempts to program computers to behave with the intelligence
of even a field mouse have been of limited success. One thing we
have learned from such efforts is just how complicated cognitive
processing is. Even the simplest task, such as deciding whether a
dot is inside or outside a closed boundary, requires sophisticated
processing (Uldman, 1984~. If we are to understand the neural ba-
sis of cognition, we must be prepared to formulate rather complex
theories. Until very recently, however, this has not been done in psy-
chophysiology. For example, "localizing oneself in space" is typically
considered a single function in the psychophysiological literature,
whereas a computationally-oriented theorist would be inclined to de-
compose this process into many disparate encoding, representational,
OCR for page 11
COGNITIVE PSYCHOPHYSIOLOGY
11
and retrieval operations. Similarly, visual agnosia ("m~ndblindness")
is described and the underlying causes of the deficit are explained by
reference to damage of anatomical areas and their connections—but
exactly what is done by these areas is never clearly specified.
Thus, to expand the contribution of a psychophysiological ap-
proach it is of interest to consider what the two major strands of
cognitive science, computational theorizing and cognitive psychol-
ogy, can offer.
Limitation and Virtues of the Computational Approach
Although the brain clearly is not a standard digital computer,
brain activity can be conceptualized as the carrying out of computa-
tions.
Computational modeling of brain activity occurs at multiple
levels of analysis. The most appropriate level for present purposes
focuses on the decomposition of processing into modules, each of
which may correspond to a distinct neural network. Any given task
presumably recruits many such modules to work together, and the
ways in which modules interact determines task performance. Al-
though specifying the precise operation of the individual modules is
of course critical for a theory of information processing, at the current
level of technology we are unlikely to be able to use methods of as-
sessing brain activity to directly test theories at this level of analysis.
The main contribution of the computational approach to cognitive
psychophysiology will therefore probably be to offer guidelines for
how one formulates theories of processing modules.
An example is the work of Marr (1982~: according to Marr, the
most important task is to formulate the "theory of the computation,"
a theory of what is computed by a processing module. Marr argues
that the information available and the purpose of a computation
often virtually dictate what the computation must be. This sort
of theory can be likened to a solution to a mathematics problem,
arising through logical analysis of the nature of the problem to be
solved and of the input available to solve it. That is, if the task
is very well defined and the input is highly restricted, a specific
computation may almost be logically necessary. Furthermore, Marr
claims that once a computation is defined, the task of characterizing
the representations and processes used in carrying out the step-by-
step processing itself is now highly constrained: the representation
of the input and the output must make explicit the information
OCR for page 12
12
BRAIN AND COGNITION: SOME NEW TECHNOLOGIES
necessary for the computation to serve its purpose (e.g., picking out
likely locations of edges), and the representations must be sensitive
to the necessary distinctions, be stable over irrelevant distinctions,
and have a number of other properties (see Marr, 1982, Chapter 5~.
Marr's strong claims about the importance of the theory of the
computation do seem appropriate for some of the problems of low-
leve! vision, but only because there are such severe constraints on the
input posed by the nature of the world and the geometry of surfaces
and because the purpose of a computation is so well defined (e.g., to
detect places where intensity changes rapidly, to derive depth from
disparities in the images striking each eye, to recover structure from
information about changes on a surface as an object moves).
In broader areas of cognition, the situation is different. First,
the basic abilities in need of explanation, analogous to our ability
to see edges or to see depth, must be discovered. For example,
with the advent of new methodologies, our picture of what can
be accomplished in mental imagery has changed drastically (e.g.,
see Shepard and Cooper, 1982~. Second, the input to a "mental"
computation is often not obvious, not necessarily being constrained
by some easily observed property of the stimulus. One must have a
theory of what is represented before one can even begin to specify
the input to the computations. Third, the optimal computation
will depend in part on the kinds of processing operations that are
available and the type of representation used. For example, if a
parallel-distributed processing network is used, computing the degree
to which an input is similar to stored information should be relatively
easy, whereas serial search through a list will be more difflcult-and
vice versa if symbols are stored as discrete elements in lists that are
operated on by distinct processes.
The consequences of these difficulties are illustrated by problems
with some of Marr's own work on "higher-levein vision. Marr posits
that shapes must be stored using "object-centeredn descriptions,
as opposed to "viewer-centeredn descriptions. In an object-centered
description, an object is described relative to itself, not from a partic-
ular point of view. Thus, terms such as dorsal and ventral would be
used in an object-centered description, rather than top and bottom,
which would be used in a viewer-centered description. Marr argues
that because objects are seen from so many different points of view, it
would be difficult to recognize an object by matching viewer-centered
descriptions to stored representations. However, this argument rests
on assumptions about the kinds of processing operations that are
OCR for page 13
COGNITIVE PSYCHOPHYSIOLOGY
13
available. If there is an Orientation normalization" preprocessor, for
example, the argument is obviated: in this case, a viewer-centered
description could be normalized (e.g., so the longest axis is always
vertical) before matching to stored representations. And in fact, we
do mentally rotate objects to a standard orientation when subtle
judgments must be made (see Shepard and Cooper, 1982~. The fact
that we do seem to normalize the represented orientation, at least in
some cases, casts doubt on the power or generality of object-centered
representations. In fact, when the matter was put to empirical test,
Jolicoeur and Kosslyn (1983) found that people can use both viewer-
centered and object-centerec} coordinate systems in storing ~nforma-
tion, and they seem to encode a viewer-centered one even when they
also encode an object-centered one, but not vice versa.
The point is that a logical analysis of the computation is not
enough. At least for high-level cognitive functions, the specifics of a
computation will depend to some extent on what types of processing
operations are available in the system. One can only discover the
actual state of affairs empirically, by studying the way the brain
works.
Although the computational approach is not sufficient in itself
to lead one to formulate a correct theory of information processing,
it does have a lot to contribute to the enterprise. Analyzing how one
could build a computer program to emulate a human function is a
very useful way of enumerating alternative processing modules and
algorithms. Not only does this approach raise alternatives that may
not have otherwise been considered, but it also eliminates others by
forcing one to work them out concretely enough to reveal their flaws
(the Guzman approach to vision is a good example; see Winston,
1978~.
[nnitatione and Virtues of the Cognitive Psythology Approach
The predominant approach in cognitive psychology is solidly
empirical: researchers have developed methodologies that make use
of response times, error rates, and various judgments and have at-
tempted to develop models that account for these data. The method-
ologies used have become very sophisticated and powerful, allowing
researchers to observe quite subtle regularities in processing. As
we saw in the previous section, such data place strong constraints
on theories of processing: since processing takes place in real time,
OCR for page 14
14
BRAIN AND COGNITION: SOME NEW TECHNOLOGIES
there will always be measurable consequences of any given sequence
of activity.
Although cognitive psychologists occasionally focus on the na-
ture of the step-by-step process a subject ~ using to carry out an
entire task (e.g., see Simon and Simon, 1978), more typically they are
interested in studying how information is represented and processed
within a single stage of processing. However, it has proven difficult
to draw firm conclusions about the representations or processes used
in even one stage of processing because of two general problems:
structure/process trade-offs and task demand artifacts.
Anderson (1978) demonstrated that structure/process trade-offs
are in principle always possible, so that, given any set of data, more
than one theory can be formulated to account for the data. That is,
what are, In one theory, properties of a given representation operated
on by a specific process are, in another theory, properties of a different
representation operated on by a different process. For example,
consider the memory scanning results described by Sternberg (1969~.
He asked subjects to hold lists of digits in mind, with lists varying
from 1 to 6 in length. Shortly thereafter, a probe digit was presented,
and subjects were to decide as quickly as possible whether the probe
was a member of the list. The tone to make this decision increased
linearly with increasing set size (by about 39 ms per additional item).
One theory of this result posits that the list of digits (the structure)
is held ~ nd and then scanned serially (the process) when the
probe arrives. Alternatively, one could posit an unordered collection
(the structure) with each item being compared simultaneously with
the probe (the process). In this case, all one needs to do is assume
that the comparison process slows down as more things need to be
compared, and the two theories wall mimic each other. More time is
required when more items are on the memorized list to be compared
with the probe.
In this example, the two theories seem to account for the data
equally well but they were created entirely ad hoc simply to account
for the data. Constraints on the theories are required, a source of
motivation for selection of the specific representations and processes.
Why should information be represented as an ordered list or as an
unordered collection? Why is more time required if one compares
more items simultaneously? Computational considerations are one
possible source of constraint. However, we saw in the previous section
that computational constraints in themselves are not sufficient, and
OCR for page 15
COGNITIVE PSYCHOPHYSIOLOGY
15
in fact the observation of how the system functions puts constraints
on computational theories themselves.
Anderson (1978) drew some very pessimistic conclusions from the
possibility of structure/process trade-offs, but others such as Hayes-
Roth (1979) and Pylyshyn (1979) were less gloomy. The upshot of the
debate seems to be that while it is possible to derive inferences about
processing mechanisms from behavioral data, it is very difficult to do
so. One argument to be developed here is that psychophysiological
data are powerful supplements to the usual behavioral data, and
would greatly constrain the use of structure/process trade-offs to
develop alternative theories.
Another problem in interpreting behavioral data is the possibility
of distorting behavior because of perceived task demands. That
is, subjects may respond in a manner congruent with their beliefs
concerning acceptable behavior to the task and the situation. If they
do so, then data from many studies of, for example, mental imagery
may say nothing about the nature of the underlying mechanisms, but
may only reflect the subjects' understanding of tasks, knowledge of
physics and perception, and ability to regulate their response times.
Although the problem of task demands h" been brought to the
attention of cognitive psychophysiologists primarily in the literature
of mental imagery, it is applicable to many domains in cognitive psy-
chology and, indeed, in other areas of psychology. There is no way to
ensure that subjects are not unconsciously producing data in accor-
dance with their tacit knowledge about perception and cognition and
their understanding of what the task requires them to do. In con-
trast, not only do neurological maladies produce behavioral deficits
of various types, but often the patients are not aware of the nature of
the deficits. Thus, psychophysiological data might profitably supple-
ment the usual cognitive data, if for no reason other than to rule out
task demand as a source of explanation. And such data are useful
for other purposes, as discussed in the following section.
The Strength of a Co~bmed Approach
Psychophysiological approaches can be used to circumvent some
of the difficulties inherent in the traditional measures user] by cog-
nitive psychologists, which are based strictly on the observation of
overt responses. First, structure/process trade-offs are greatly min-
imized if neurophysiological data are used. By relating processing
to anatomical areas, many of the degrees of freedom are removed
OCR for page 16
1
16
BRAIN AND COGNITION: SOME NEW TECHNOLOGIES
from cognitive science theories: in all cases in which a given area
is active or damaged, the consequences must be the same. When
one has fixed the properties of some area, those properties cannot
be changed at whim by a theorist in order to account for new data.
Second, difficulties due to task demands are virtually eliminated if
brain activation measures are used, because subjects cannot respond
to explicit task demands by directly altering the activity of specific
regions of the brain. Whereas a person can regulate the time taken to
press a button, it is not so easy to regulate intentionally the activity
of the right parietal lobe, for example. In addition, psychophysi-
ological measures can be used to monitor on-line and in real time
the activity of processing entities that are not directly manifested by
overt behavior.
The computational approach, by contrast, especially as con-
stra~ned by data from cognitive psychology, is useful for generating
hypotheses about processing mechanisms. Analyzing the require-
ments of the task at hand and how one would need to program a
computer to perform it is a good way to generate alternative possi-
bilities. In addition, this approach provides a way of testing complex
theories by actually building a computer program that emulates cog-
nitive processing (see Newell and Simon, 1972~. Precise theories of
on-line brain functioning may be so complex that many of a theory's
implications will be derives} only by using simulation models.
Furthermore, once there are prior reasons for positing a spe-
cific modular composition of the system, the standard techniques of
cognitive psychology become more powerful. When a module is de-
fined, the number of degrees of freedom is reduced for possible struc-
ture/process trade-offs. That is, without modularity constraints, any
part of the system can be invoked in combination with any other part
to explain a specific result; but if a result can be shown to rest on the
operation of a specific module, the explanation of the result is limited
to fewer alternatives. When well-specified classes of alternative theo-
ries are defined, cognitive psychologists will be better able to specify
which phenomena will distinguish among competing accounts (for an
example see the mental rotation case noted above in Kosslyn, 1980:
Ch.8~.
One example of progress following from such a combined ap-
proach began with computational analyses suggesting that spatial
localization should be decomposed into at least two types of pro-
cesses. On one hand, if one were to build a machine to recognize
semirigid objects (e.g., a human form) it would be desirable to include
OCR for page 17
COGNITIVE PSYCHOPHYSIOLOGY
17
a module to encode representations of rather broad categories of spa-
tial relations among parts. Such representations would be constant
over different contortions of the object. For example, the forearm
and upper arm remain connected (a categorical relation) no matter
how they are configured. On the other hand, if the machine is also
intended to navigate and reach for objects, it is desirable to include a
module to encode representations of the specific metric coordinates
between parts or objects. For these purposes, a broad category of
relations (e.g., one object ~ cleft of" another) is not useful; one
needs to know precise positions. The possible Extinction between
these types of representation has been investigated by noting that
categorical representations are I~guage-like (all can be easily named
by a word or two) and hence might be processed more effectively in
the left cerebral hemisphere. In contrast, coordinate representations
are critical for navigation, which appears to draw in large part on
right hemisphere processes. And in fact, it has been found that cat-
egorical spatial relations are apprehended more effectively ~ the left
hemisphere, whereas coordinate relations are apprehended more em
fectively in the right hemisphere (Kosslyn, 1987' 1988~; this inference
is based in part on work using some of the technologies discussed in
this report. This dissociation provides evidence for the existence of
distinct processes underlying the two types of spatial representation,
which was not obvious until computational analyses led to the dim
tinction between the two and specific brain-based hypotheses were
tested.
In summary, psychophysiological data offer constraints both on
theories of processing modules and theories of the algorithms used.
The logic of dissociations and associations in deficits or patterns of
brain activation ~ a powerful way of developing and testing compu-
tational theories, particularly so if it is supplemented by the method-
ologies and analytic techniques of cognitive psychology. The method-
ologies developed by the cognitive psychologists for the most part can
be adapted for use in psychophysiological studies.
Representative terms from entire chapter:
cognitive psychophysiology
