Improving Indicators of the Quality of Science and Mathematics Education in Grades K-12 (1988) / Chapter Skim
Currently Skimming:
4 Indicators of Learning in Science and Mathematics
4 Indicators of Learning in Science and Mathematics
Pages 40-72
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 40... ...
AN APPRAISAL OF CURRENT TESTS OF STUDENT ACHIEVEMENT The most direct indicators of the quality of science and mathematics education are the scores based on tests that measure what students have learned. Currently available indicators of student learning are typically obtained from standardized achievement tests made up of multiple-choice items.
|
From page 41... ...
Some involve the evaluation of individual students for grading, student counseling, placement, promotion, awards, scholarships, and so onimportant for educational purposes but not always well suited to the development of indicators. The use of tests most closely identified with assessing the condition of science and mathematics education is for the evaluation of learning achieved by populations of students; a related purpose that is of interest to the committee is the use of tests in improving the quality of instruction.
|
From page 42... ...
Although such instructional uses of tests are not directly related to their use as indicators, they are as important and provide equally valid reasons for developing better tests. Criticisms of Current Testing In the early years of this century, the assessment of student achievement was generally based on teachers' judgments, which were in turn based on teacher-made tests, homework, and impressions of classroom performance.
|
From page 43... ...
Some of the criticisms most relevant to the development of indicators of science and mathematics education are discussed below. Multiple- Choice Tests Penalize Creative Thinking This is a welltaken criticism, since most multiple-choice items do not provide much opportunity to generate new ideas.
|
From page 44... ...
The two formats were similar with regard to their relationships to verbal ability and reasoning, but only for the free-response version were there substantial relations to a factor called ideational fluency, which represents the skills involved in making broad searches of the memory store in order to retrieve information relevant to a situation (Ward et al., 1980~. In at least one instance, converting a test intended to measure productive thinking to multiple-choice format eliminated the need to broadly search the memory store for ideas that might be relevant, evidence that the multiple-choice format is not conducive to measuring productive thinking.
|
From page 45... ...
Another kind of coaching attempts to improve the ability measured by the test; a review of fractions and percentages, for example, might improve both test scores and the student's underlying competence in arithmetic. Test makers should attempt to construct tests that are coachable only in the sense that coaching and teaching are indistinguishable.
|
From page 46... ...
At a conference on elementary science education held by the National Science Resources Center at the National Academy of Sciences in 1986, participants representing school districts with innovative programs expressed concern "that standardized achievement tests do not do a good job of assessing what students learn in elementary school science. There is a need to develop improved tests and alternative evaluation techniques to assess student progress in science, with more emphasis on the development of process skills and attitudes" (National Science Resources Center, 1986:3~.
|
From page 47... ...
An examination of the results of state testing programs in mathematics provides further documentation: children score well on items dealing with computation but less well on items dealing with concepts and problem solving, because the learning of these higher-order skills is not stressed in classroom instruction (Suydam, 1984~. The National Assessment of Educational Progress (NAEP)
|
From page 48... ...
But none of this test theory is concerned with the content of the test items. Another kind of theory, one that grows out of work in cognitive psychology and artificial intelligence, does provide a potentially usefuT basis for the development of tests based both on content and the cognitive processes that are involved in doing science and mathematics.
|
From page 49... ...
Matrix sampling is analogous to the methods used in public-opinion polling, in that it requires drawing random or representative samples of sum jects. But in addition to drawing random samples of subjects, matrix sampling also involves independently drawing random samples of test items (Wilks, 1962; Lord and Novick, 1968~; thus random subsamples of students are given different subsamples of items.
|
From page 50... ...
Another virtue to note is that current testing methodology makes possible comparisons over time. The collection of data on learning indicators is of limited value unless the measurement can be repeated, since the purpose of school evaluation is to detect change-to see if student performance is improving.
|
From page 51... ...
The other set of procedures yields processing measures, since they are descriptive of the information-processing components that influence the develop ment of conceptual knowledge and overt performance of the student. GIo bat Assessment A frequently used alternative to a multiple-choice test is an essay test in which the items elicit fairly long written responses.
|
From page 52... ...
Flexible computer environments are being developed that permit students to generate text based on their retrieval, generation, and manipulation of declarative knowledge in a knowledge-rich domain. The use of syntactic and semantic parsers makes it possible to analyze a student's responses to a task and to make their grammatical structure explicit on the screen.
|
From page 53... ...
For example, students can be given the necessary materials and equipment and asked to design and carry out models of scientific investigations that demonstrate understanding of such scientific concepts as density, conductivity, and capillarity. Such tests are already in use on a limited scale by NAEP (Blumberg et al., 1986; National Assessment of Educational Progress, 1987)
|
From page 54... ...
They can be the kind of tests that should be taught to-which by their use will generate higher-quality science instruction. It appears entirely practical to use simulations for classroom learning and to draw on a subset of the same group of simulations for local, state, and national assessment.
|
From page 55... ...
The development of conceptual knowledge is supported by specific processing skills that assist in the absorption of information and its organization and use; they include processing speed, memory capacity, memory organization, factual knowledge, and procedural knowledge (KyIlonen, 1986~. Procedural knowledge includes not only knowledge of algorithms but also the ability to plan and use various heuristics and strategies.
|
From page 56... ...
in comparing chess grand masters with ordinary chess players. He found that grand masters were able to reproduce correctly the positions on a board of 20 to 25 chess pieces in a midgame position after seeing them for a few seconds, while ordinary players
|
From page 57... ...
One cannot hope to discover how all the information in memory is organized, but methods are available for assessing the structure of knowledge in particular domains. One method is to ask students to recall items of information and to time the responses a method analogous to that used to investigate the size and nature of clusters of chess pieces as perceived by grand masters.
|
From page 58... ...
Thus, it seems feasible to develop for a variety of subject-matter areas assessment methods that provide some information about the organization of information in memory for individuals or for groups of students. Skill in Retrieving Information The accessibility of information stored in memory has for many years been assessed by means of aptitude tests presumed to measure the fluency with which associations and ideas are generated.
|
From page 59... ...
Another method of studying problem representations involves asking experts and novices to sort a set of problems into categories. The results in physics, where the method has been applied, indicate that novices tend to sort the problems on the basis of superficial characteristics of the problems, such as the use of inclined planes or pulleys, while the experts categorized the problems in terms of the physical principles that were involved (Chi et al., 1981~.
|
From page 60... ...
Based on diagnostic models that are integral parts of the system, computer programs can be designed that offer advice to the student and at the same time provide detailed assessments of his or her capabilities (e.g., Burton and Brown, 1979; Anderson et al., 1985~. Computerized
|
From page 61... ...
The Development arid Use of New Methods None of the assessment methods described in this section can compete with multiple-choice tests from the standpoint of economy and efficiency, although matrix sampling makes their use more feasible. However, investment in the development of the recommended new methods and the cost of using them is, in the committee's view, justifiable not only because these methods would provide information for a far more accurate and complete assessment of instruction and student learning, but also because they are likely to be useful in the instructional process itself (see, e.g., Linn, 1986~.
|
From page 62... ...
The develop ment of new assessment materials is costly, in both money and intellectual resources; needless duplication of effort must be avoided. This implies that the areas most in need of research and development of assessment techniques must be defined, newly developed instruments must be evaluated for their quality, and facilities for the distribution of materials to schools and teachers must be created.
|
From page 63... ...
Summary Currently available multiple-choice tests are adequate primarily for assessing student learning of the declarative knowledge of a subject. They are not adequate for assessing conceptual knowledge, most process skills, and the higher-order thinking that scientists, mathematicians, and educators consider most important.
|
From page 64... ...
The committee urges that the development of science tests at the K-5 level receive immediate attention. Techniques to be developed include problem-solving tasks, as exemplified by the College Board Advanced Placement Tests; penciland-paper tests of hypothesis formulation, experimental design, and other tasks requiring productive-thinking skills, as exemplified by questions in the British Assessment of Performance Unit Series; hands-on experimental exercises, as exemplified by some test materials administered by the National Assessment of Educational Progress (NAEP)
|
From page 65... ...
Key Indicator: The committee recommends that assessment of student learning using the best available tests andtesting methods continue to be pursued in order to provide periodic indicators of the quality of science and mathematics education. Tests should be given to students in upper-elementary, middle, and senior high school (for example, in grades 4, 8, and 12~.
|
From page 66... ...
. The assessment approaches based on hands-on investigation and computer simulation that would evolve from the proposed resource center could serve two functions for states and local communities.
|
From page 67... ...
ASSESSING ADULT SCIENTIFIC AND MATHEMATICAL LITERACY There are several reasons why assessment of student learning should be extended to assess trends in the science and mathematics literacy of the entire population. First, one of the reasons to care about the quality of mathematics and science instruction in school is that it will influence mathematics and science literacy throughout the population; trends in the mathematics and science literacy of adults will in time provide information about the long-term consequences of attempts to improve the science and mathematics education provided in the nation's schools.
|
From page 68... ...
Such results increase the need for future study of the population's scientific literacy and the long-term effects of science education. Desired Attributes of Indicators Any plan to generate indicators of scientific and mathematical literacy should try to estimate the degree to which a population possesses the kind of knowledge and intellectual skills outlined in Chapter 2.
|
From page 69... ...
Target Populations for Assessment The committee considers education policy makers for elementary and secondary schools at state, local, and also national levels to be prime users of indicators of the quality of science and mathematics education. This has implications and raises interesting issues for the design of a set of indicators to assess the scientific and mathematical literacy of adults.
|
From page 70... ...
Although the program would be targeted to adults 17 years of age and older, it should be expanded to include children in elementary and secondary schools as in-schooT testing programs begin to include the measurement of scientific and mathematical literacy. The objective would be to provide links between school and household measures, as well as to provide a household-based unit of analysis for adults and children.
|
From page 71... ...
Finally, because the questions are open-ended and recursive, they permit assessment of both breadth and depth. Although it may be difficult to do so, it would be important to establish to what extent people's responses are based on knowledge gained in school and to what extent they draw on knowledge gained fi:om subsequent reading, television programs, museum visits, and so on, even given
|
From page 72... ...
They should also be aggregated by age, gender, race, ethnicity, socioeconomic status, and geographic region so as to establish to what extent there are systematic inequities in the distribution of scientific and mathematical literacy.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.