Scientific Research in Education (2002) / Chapter Skim
Currently Skimming:
5 Designs for the Conduct of Scientific Research in Education
5 Designs for the Conduct of Scientific Research in Education
Pages 97-126
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 97... ...
RESEARCH DESIGN Our scientific principles include research design the subject of this chapter as but one aspect of a larger process of rigorous inquiry.
|
From page 98... ...
A comprehensive explication of a hierarchy of appropriate designs and analytic approaches under various conditions would require a depth of treatment found in research methods textbooks. This is not our objective.
|
From page 99... ...
Also, in the concluding section of this chapter, we make a few targeted suggestions for the kinds of work we believe are most needed in education research to make further progress toward robust knowledge. TYPES OF RESEARCH QUESTIONS In discussing design, we have to be true to our admonition that the research question drives the design, not vice versa.
|
From page 100... ...
. Throughout the discussion, we provide several examples of scientific education research, connecting them to scientific principles (Chapter 3)
|
From page 101... ...
Similarly, lines of research encompassing collections of studies may be more or less productive and useful in advancing knowledge. An area of research that, for example, does not advance beyond the descriptive phase toward more precise scientific investigation of causal effects and mechanisms for a long period of time is clearly not contributing as much to knowledge as one that builds on prior work and moves toward more complete understanding of the causal structure.
|
From page 102... ...
Policy makers at the national, state, and sometimes district levels depend on this method to paint a picture of the educational landscape. Aggregate estimates of the academic achievement level of children at the national level (e.g., National Center for Education Statistics tNCES]
|
From page 103... ...
As committee member Paul Holland quipped during the committee's deliberations,"Casual comparisons inevitably invite careless causal conclusions." To illustrate the problem with drawing causal inferences from simple correlations, we use an example from work that compares Catholic schools to public schools. We feature this study later in the chapter as one that competently examines causal mechanisms.
|
From page 105... ...
For example, since Catholic schools can screen children for aptitude, they may have a more able student population than public schools at the outset. (This is an example of the classic selectivity bias that commonly threatens the validity of causal claims in nonrandomized studies; we return to this issue in the next section.)
|
From page 106... ...
, for example (see Box 5-2) , a number of theoretical models were developed and tested to explain how women decide to pursue or abandon nontraditional careers in the fields they had studied in college.
|
From page 108... ...
As we argue throughout this volume, research designs can often be strengthened considerably by using multiple methodsintegrating the use of both quantitative estimates of population characteristics and qualitative studies of localized context. Other descriptive designs may involve interviews with respondents or document reviews in a fairly large number of cases, such as 30 school districts or 60 colleges.
|
From page 109... ...
We have organized the remainder of this section into two parts. The first treats randomized field trials, an ideal method when entities being examined can be randomly assigned to groups.
|
From page 110... ...
The key difference between randomized field trials and other methods with respect to making causal claims is the extent to which the assumptions that underlie them are testable. By this simple criterion, nonrandomized studies are weaker in their ability to establish causation than randomized field trials, in large part because the role of other factors in influencing the outcome of interest is more difficult to gauge in nonrandomized studies.
|
From page 111... ...
However, randomized trials that show reductions in health detriments and improved social and behavioral functioning strengthen the causal links that have been established between drug use and adverse health and behavioral outcomes (Moses, 1995; Mosteller, Gilbert, and McPeek,1980~. In medical research, the relative effectiveness of the Salk vaccine (see Lambert and Markel, 2000)
|
From page 112... ...
Many nonexperimental methods and analytic approaches are commonly classified under the blanket rubric "quasi-experiment" because they attempt to approximate the underlying logic of the experiment without random assignment (Campbell and Stanley, 1963; Caporaso and Roos, 1973~. These designs were developed because social science researchers recognized that in some social contexts (e.g., schools)
|
From page 113... ...
For example, researchers might come across schools that vary in the size of their classes and compare the achievement of students in large and small classes, adjusting for other differences among schools and children. If the class size conjecture holds after this adjustment is made, the researchers would expect students in smaller classes to have higher achievement scores than students in larger size classes.
|
From page 114... ...
) , we note that some kinds ot correlational work make important contributions to understanding broad patterns of relationships among educational phenomena; here, we highlight a correlational design that allows causal inferences about the relationship between two or more variables.
|
From page 115... ...
The key difference between simple correlational work and modelfitting is that the latter enhances causal attribution. In the study examining teacher compensation and dropout rates, for example, researchers introduced a conceptual model for the relationship between student outcomes and teacher salary, set forth an explicit hypothesis to test about the nature of that relationship, and assessed competing models of interpretation.
|
From page 117... ...
what might explain the Tennessee and other class-size effects. That is, what was the causal mechanism through which reduced class size affected achievement?
|
From page 118... ...
Previous and highly controversial work on Catholic schools (e.g., Coleman, Hoffer, and ~ 78 ~ SCIENTIFIC RESEARCH IN EDUCATION
|
From page 119... ...
A series of models were developed (sector effects only, compositional effects, and school effects) and tested to explain the mechanism by which Catholic schools successfully achieve an equitable social distribution of academic achievement.
|
From page 120... ...
and the second is a "design study" that resulted in a theoretical model for how young children learn the mathematical concepts of ratio and proportion. After generating a rich description of women's lives in their universities based on extensive analysis of ethnographic and survey data, the researchers turned to the question of why women who majored in nontraditional majors typically did not pursue those fields as careers (see Box 5-2~.
|
From page 122... ...
277~. O J This genre of method and approach is a relative newcomer to the field of education research and is not nearly as accepted as many of the other 722 ~ SCIENTIFIC RESEARCH IN EDUCATION
|
From page 123... ...
call to ensure a careful application of the scientific principles we describe in this report in the conduct of such research.9 CONCLUDING COMMENTS This chapter, building on the scientific principles outlined in Chapter 3 and the features of education that influence their application in education presented in Chapter 4, illustrates that a wide range of methods can legitimately be employed in scientific education research and that some methods are better than others for particular purposes. As John Dewey put it: We know that some methods of inquiry are better than others in just the same way in which we know that some methods of surgery, arming, road-making, navigating, or what-not are better than others.
|
From page 124... ...
These studies are often extremely valuable in themselves, and they also provide the critical theoretical grounding needed to conduct causal studies. We believe that attention to the development and systematic testing of theories and conjectures across multiple studies and using multiple methods a key scientific principle that threads throughout all of the questions and designs we have discussed is currently undervalued in education relative to other scientific fields.
|
From page 125... ...
Establishing cause is often exceedingly important for example, in the large-scale deployment of interventions and the ambiguity of correlational studies or quasi-experiments can be undesirable for practical purposes. In keeping with our arguments throughout this report, we also urge that randomized field trials be supplemented with other methods, including in-depth qualitative approaches that can illuminate important nuances, i°The federal Interagency Education Research Initiative was developed to tackle this thorny issue.
|
From page 126... ...
identify potential counterhypotheses, and provide additional sources of evidence for supporting causal claims in complex educational settings. In sum, theory building and rigorous studies of implementations and interventions are two broad-based areas that we believe deserve attention.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.