National Academies Press: OpenBook

Science, Evidence, and Inference in Education: Report of a Workshop (2001)

Chapter: Synthesis and Next Steps for the Committee

« Previous: Theme 3: Evidence and Inference: Consistency and Variation Revisited
Suggested Citation:"Synthesis and Next Steps for the Committee." National Research Council. 2001. Science, Evidence, and Inference in Education: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10121.
×

Page 14

Synthesis and Next Steps forthe Committee

What does it all mean? A final luncheon session was designed to help the committee synthesize what they heard and to raise questions and issues the committee should address in conducting further work. Commentary from two rapporteurs who attended each of the workshop sessions concluded the event.

MAXIMIZING RESEARCH QUALITY AND UTILITY

Research quality and utility were highlighted again as distinct concepts with independent properties, both of which are important to maximize. On a similar issue, however, one rapporteur cautioned against limiting research dollars to areas that seem most likely to have immediate impact on practice. He suggested that this lens would severely limit the potential of research to improve incrementally our understanding of the underlying processes of teaching, learning, and schooling.

THE NATURE AND PROGRESSION OF SCIENCE

Picking up on a discussion from the first workshop session, a rapporteur challenged the basis of the criticism that research in education has a high “waste tolerance.” Reflecting on the nature of science as an iterative, non-linear, process of knowledge accumulation, he argued that a view of unproductive paths as “waste” was the wrong model. Trying things out in the face of uncertainty is a natural part of the scientific process. To illustrate the point that reasoning always takes place amid uncertainty, the rapporteur used a business analogy. He argued that the quality of an investment decision depends on how well the investor reasoned with the information he or she had at the time. The eventual outcome of that decision—financial loss or gain—is irrelevant to any consideration of whether the decision was a sound one at the time it was made.

If in hindsight you look at an investment decision that went belly up, whether that decision was good or bad depends on whether or not based on what you knew at the time you should have known better...if not, you had to try it out to find out what would happen... it all depends on if you reasoned well in the context of that decision.

—David Klahr

Suggested Citation:"Synthesis and Next Steps for the Committee." National Research Council. 2001. Science, Evidence, and Inference in Education: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10121.
×

Page 15

THE BARRIERS TO LINKING RESEARCH AND REFORM: IS QUALITY ONE OF THEM?

Commenting that education practice seems impervious to well-established bodies of research in cognition, learning, and memory, a rapporteur posed the question if we fully understand the reasons why education is reluctant to change in the face of such evidence. He suggested that more research about the diffusion of innovation in school settings was needed to better understand the role of quality in research use.

MODELING COMPLEXITY: JUST TRY THINGS OUT

Reflecting on a workshop participant's suggestion that some new “instructional engineering” career may need to be invented to bridge research and practice in education, a rapporteur provided an example from physics to illustrate the point. Although the laws of motion are well understood and well documented, when automotive engineers want to determine whether one car is more crashworthy than another, they don't “solve simultaneous differential equations.” Instead, they drive each car into a wall and examine the wreckage. While this may seem to be an expensive way to do a comparison, when things get complex—even in the “hard sciences”—the only way to find out what will happen is to find out what happens. He went on to say that when educators take this approach, they are often unfairly faulted for being unscientific.

...when we want to find out if a car will survive a crash, we don't do simultaneous differential equations, we drive a car into a wall...when things get complex, we just try things out...education shouldn't feel so bad about where it is...

—David Klahr

CLARITY ON TERMINOLOGY

Both rapporteurs urged that the committee make clear the important distinctions among the various genres of research in education while deliberating about quality. They argued that terms like research, intervention, evaluation, and clinical need to be articulated and elucidated. In particular, a rapporteur further asserted, the committee should distinguish between science as an act of inquiry and science as an act of design. There is obvious overlap, but one emphasizes understanding of underlying phenomena while the other is more focused on problem-solving.

NEXT STEPS

The workshop served as an information collection exercise for the committee. The next phase of the committee's work involves intensively deliberating and writing to set forth the committee's response to its three framing questions (see preface), to forge the consensus of the group, and to articulate that consensus in a report of its findings. The committee expects to release its consensus report in fall 2001.

Suggested Citation:"Synthesis and Next Steps for the Committee." National Research Council. 2001. Science, Evidence, and Inference in Education: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10121.
×
Page 14
Suggested Citation:"Synthesis and Next Steps for the Committee." National Research Council. 2001. Science, Evidence, and Inference in Education: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10121.
×
Page 15
Next: Appendix A: Workshop Agenda »
Science, Evidence, and Inference in Education: Report of a Workshop Get This Book
×
Buy Paperback | $21.00 Buy Ebook | $16.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Research on education has come into the political spotlight as the demand grows for reliable and credible information for the guidance of policy and practice in the education reform environment. Many debates among the education research community feature questions concerning the nature of evidence and these questions have also appeared in broader policy and practice arenas. Inquiry has generally, over the past years, created bodies of scientific knowledge that have profound implications for education. Dramatic advances in understanding how people learn, how young children acquire early reading skills, and how to design and evaluate educational and psychological measurements is a good example of this. However, the highly contextualized nature of education and the wide range of disciplinary perspectives that rely on it have made the identification of reducible, generalizable principles difficult and slow to achieve.

Due to this, the U.S. Department of Education's National Educational Research Policy and Priorities Board (NERPPB) has asked the NRC to establish a study committee to consider the scientific underpinnings of research in education. The committee consists of members with expertise in statistics, psychology, sociology, anthropology, philosophy of science, history of education, economics, chemistry, biology, and education practice. The committee worked with the three questions in mind: What are the principles of scientific quality in education research?, How can research-based knowledge in education cumulate?, and How can a federal research agency promote and protect scientific quality in the education research it supports?.

A workshop was held on March 7-8, 2001 that was organized into three main sessions: Supporting Scientific Quality at the Federal level, The Interface of Research and Practice in Education, and Evidence and Inference. Science, Evidence, and Inference in Education: Report of a Workshop summarizes this workshop through these three ideas. The report also includes what the committee plans to do next, the workshop agenda, and information on the workshop's participants and speakers.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!