Page 14
Synthesis and Next Steps forthe Committee
What does it all mean? A final luncheon session was designed to help the committee synthesize what they heard and to raise questions and issues the committee should address in conducting further work. Commentary from two rapporteurs who attended each of the workshop sessions concluded the event.
MAXIMIZING RESEARCH QUALITY AND UTILITY
Research quality and utility were highlighted again as distinct concepts with independent properties, both of which are important to maximize. On a similar issue, however, one rapporteur cautioned against limiting research dollars to areas that seem most likely to have immediate impact on practice. He suggested that this lens would severely limit the potential of research to improve incrementally our understanding of the underlying processes of teaching, learning, and schooling.
THE NATURE AND PROGRESSION OF SCIENCE
Picking up on a discussion from the first workshop session, a rapporteur challenged the basis of the criticism that research in education has a high “waste tolerance.” Reflecting on the nature of science as an iterative, non-linear, process of knowledge accumulation, he argued that a view of unproductive paths as “waste” was the wrong model. Trying things out in the face of uncertainty is a natural part of the scientific process. To illustrate the point that reasoning always takes place amid uncertainty, the rapporteur used a business analogy. He argued that the quality of an investment decision depends on how well the investor reasoned with the information he or she had at the time. The eventual outcome of that decision—financial loss or gain—is irrelevant to any consideration of whether the decision was a sound one at the time it was made.
If in hindsight you look at an investment decision that went belly up, whether that decision was good or bad depends on whether or not based on what you knew at the time you should have known better...if not, you had to try it out to find out what would happen... it all depends on if you reasoned well in the context of that decision. —David Klahr |
Page 15
THE BARRIERS TO LINKING RESEARCH AND REFORM: IS QUALITY ONE OF THEM?
Commenting that education practice seems impervious to well-established bodies of research in cognition, learning, and memory, a rapporteur posed the question if we fully understand the reasons why education is reluctant to change in the face of such evidence. He suggested that more research about the diffusion of innovation in school settings was needed to better understand the role of quality in research use.
MODELING COMPLEXITY: JUST TRY THINGS OUT
Reflecting on a workshop participant's suggestion that some new “instructional engineering” career may need to be invented to bridge research and practice in education, a rapporteur provided an example from physics to illustrate the point. Although the laws of motion are well understood and well documented, when automotive engineers want to determine whether one car is more crashworthy than another, they don't “solve simultaneous differential equations.” Instead, they drive each car into a wall and examine the wreckage. While this may seem to be an expensive way to do a comparison, when things get complex—even in the “hard sciences”—the only way to find out what will happen is to find out what happens. He went on to say that when educators take this approach, they are often unfairly faulted for being unscientific.
...when we want to find out if a car will survive a crash, we don't do simultaneous differential equations, we drive a car into a wall...when things get complex, we just try things out...education shouldn't feel so bad about where it is... —David Klahr |
CLARITY ON TERMINOLOGY
Both rapporteurs urged that the committee make clear the important distinctions among the various genres of research in education while deliberating about quality. They argued that terms like research, intervention, evaluation, and clinical need to be articulated and elucidated. In particular, a rapporteur further asserted, the committee should distinguish between science as an act of inquiry and science as an act of design. There is obvious overlap, but one emphasizes understanding of underlying phenomena while the other is more focused on problem-solving.
NEXT STEPS
The workshop served as an information collection exercise for the committee. The next phase of the committee's work involves intensively deliberating and writing to set forth the committee's response to its three framing questions (see preface), to forge the consensus of the group, and to articulate that consensus in a report of its findings. The committee expects to release its consensus report in fall 2001.