Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 43
PART II Objectives of Biology Education and Measurement of Achievement

OCR for page 43

OCR for page 43
7 Issues in Objectives and Evaluation JAMES T. ROBINSON GOALS, OBJECTIVES, AND OUTCOMES Biology is taken by most students during the high-school years. It is incumbent on us to re-examine why biology is important for most or all students and what we expect the benefits of biology education to be, both for the individual and for society at large. Several issues in the field of goals, objectives, and outcomes of biology education will need to be resolved as the Committee on High-School Biol- ogy Education addresses its tasks. "Scientific literacy" has been espoused as a social imperative for a society affected so importantly by science and technology. The American Academy of Arts and Sciences (1983) devoted an entire issue of its proceedings to elaborate the meanings of scientific literacy. That same year, in Educating Americans for the 21st Century (Na- tional Science Foundation, 1983), the National Science Board Commission on Precollege Education in Mathematics, Science and Technology "found that virtually every child can develop an understanding of mathematics, science and technology if appropriately and skillfully introduced at the elementary, middle and secondary levels." The commission recommended the following criteria for improving high-school science (National Science Foundation, 1983, p. 98~: James T. Robinson is a former executive director for curriculum and evaluation in the Boulder Valley (Colorado) School District. He served as a staff officer for the Biological Sciences Cur- riculum Study. 45

OCR for page 43
46 HIGH-SCHOOL BIOLOGY · Drastically reduce the number of topics covered in high-school science courses. - ogy? Direct attention toward the integration of the remaining facts, concepts, and principles within each discipline and with other sciences and such areas as mathematics, technology, and the social sciences. · Select ideas that can be developed honestly at a level comprehen- sible to high-school students. · Develop ideas out of experimental evidence that high-school stu- dents can gather or, at least, understand. · Tie ideas into other parts of the course, so that their use can be reinforced by practice. · Let all courses provide opportunities to develop the ability to read scientific materials. These criteria raise issues for biology education that pervade all areas of our concern at this conference. If outcomes are to be determiners of curricula and evaluation, then the other subjects of this conference are derivative from the goals, objectives, or outcomes to be formulated as a major function of the committee. Several questions are proposed for consideration here. Should high- school biology goals and objectives: · Be designed for all students, or should separate courses be devel- oped for students with different interests and goals? · Be formulated in the context of a science and contribute to public understanding of science or as a separate discipline independent of other sciences? · Include the application of knowledge and understandings or be limited to the acquisition of knowledge? · Include attitudes toward science and technology and developing interest in biology and other sciences? · Include ethical and societal issues of science, biology, and technol · Specify the development of problem-solving, critical thinking, and other `'higher-order" thinking skills? manner? Be measurable or assessable in some objective and C`practical" The literature is fairly consistent in an affirmation of positive positions on these questions, but in the classrooms in high schools these issues are not settled at all, in stated objectives, actual practice in instruction, or testing and evaluation. Also, coverage of subject matter dominates instruction (Stake and Easley, 1978~; it is questionable whether retaining the current breadth of coverage will permit students to attain the other outcomes specified above.

OCR for page 43
ISSUES IN OBJECTIVES AND EVALUATION 47 It is ironic that Educating An~er~cans for the 21st Century lists drastic reduction of content as a major need in high-school science and then, in the statement of outcomes, includes all the major areas currently included in high-school biology. For example, the National Science Board commission, in discussing science education and high-school biology, proposed that scientific education programs in K-12 should be designed to produce the following outcomes (National Science Foundation, 1983, p. 44~: . Ability to formulate questions about nature and seek answers from observation and interpretation of natural phenomena. · Capacities for problem-solving and critical thinking in all areas of learning. · Innovative and creative thinking skills. · Awareness of the nature and scope of a wide variety of science- and technology-related careers open to students of varied aptitudes and interests. · Basic academic knowledge necessary for advanced study by students who are likely to pursue science professionally. · Scientific and technical knowledge needed to fulfill civic responsi- bilities and improve students' own health, life, and ability to cope with an increasingly technical world. · Means to judge the worth of articles presenting scientific conclu- sions. The commission proposed that general biology in high schools should emphasize biology in a social and ecological context. Biology should enable students to attain the following outcomes (National Science Foundation, 1983, p. 98~: Understanding of biologically based personal or social problems and issues, such as health, nutrition, environmental management, and human adaptation. Ability to resolve problems and issues in a biosocial context involv- ing value or ethical consideration. Continued development of students' skills in making careful obser- vatic~ns collecting and analv~in~ data thinking lnnicaliv and critically, and ~-~ ~ D ~--~ ~--- - - -D ~ -my -a -----O --O--~--~ making quantitative and qualitative interpretations. · Ability to identify sources of reliable information in biology that they may tap long after formal education has ended. · Understanding of basic biological principles, such as genetics, nu- trition, evolution, reproduction of various life forms, structure-function relationships, disease, diversity, integration of life systems, life cycles, and energetics.

OCR for page 43
48 HIGH-SCHOOL BIOLOGY The problems associated with formulating goals, objectives, or out- comes are formidable. First, a national consensus on such a statement would be extremely difficult to attain; and second, evidence seems to sup- port the observation that classroom instruction Is determined more often by the textbook used by teachers than by statements of goals in curriculum guides (Stake and Easley, 1978, pp. 13:59-64~. The issues implied here have included the question of the target population for high-school biology, its range of content, its context (social, technological, scientific), and its attention to application of knowledge and to the inclusion of higher-order thinking skills. Sorting these Issues out Is essential and is related to all the other dimensions of high-school biology. EVALUATION STUDIES The preliminary report by the International Association for the Eval- uation of Educational Achievement (IEA, 1988) presents International comparisons of student achievement. A biology test of 30 items was given to ~velfth-graders In 17 countries. Table 1 shows the numbers of items in the various topics. The U.S. sample taking the biology test was drawn from 43 schools with a total of 659 students taking a second year of high-school biology. There are no U.S. data on first-year biology students, nor for conscience students. Validity of the biology test was measured by three indexes (IEA, TABLE 1 Biological Content Areas and Naumbers of Items Given to Twelfth-Grade Students in 17 Countries Biological Topic No. of Items Transport and cellular material Concept of gene Diversity of life Metabolism of the organism Regulation of the organism Behavior of the organism Reproduction and development, plants Reproduction and development, animals Human biology Natural environment Evolution Total 3 3 2 2 6 30 a rive items, undesignated, were cut from the test given to students in the United States.

OCR for page 43
ISSUES IN OBJECTIVES AND EVALUATION 49 1988, p. 93~: a curriculum-relevant index (0.76), a test-relevant index (1.00), and a curriculum-coverage index (1.00~. Interpreting the results of the IEA biology test cannot be straightfor- ward, because of several conditions. Five items were dropped from the test given to U. S. students, and "the scores (comparing countries) are presented in percentage frequencies but it must be noted that the United States with 25 items is being compared with other countries with 30 items. The reduced number of items in percentage form will result in a reduced range" (IEA, 1988, p. 46~. A second year of biology may be inferred to be an advanced course for able students, but in the district in which I recently worked, a second biology course is offered for students who do not want to take chemistry or physics, but wish to take more science. I do not know how prevalent this practice is. However, the biology scores are reported as scores of the "elite" (IEA, 1988, p. 73~. The mean achievement of students in the United States for the 1986 administration was 37.9%, with a K-20 reliability of 0.669, which indicates that the items are not very homogeneous in difficulty. The highest national score reported was for Singapore, with a mean of 66.8%. With the limita- tions of the test data, the United States had the distinction of having the lowest mean percentage score on the biology test. The next lowest mean percentage was attained by Italy, with a mean of 42.3%. ~ give you a flavor of the test, one item asked, "What initially determines whether a human baby is going to be a male or a female?" Response options and percentages of U. S. students selecting them were (IEA, 1988, p. 120~: The DNA in the sperm. B. The DNA in the egg. C. The RNA in the sperm. D. The RNA in the egg. E. The DNA and RNA in both sperm and egg. No response. 48.44% 6.00% 9.17% 2.72% 33.25% 0.42% I reviewed Modern Biology (Otto and ldwle, 1985) and Biological Sciences: An Ecolog~calApproach (BSCS, 1982) to find out how they treated the subject. In both books, although they treat the subjects differently, sex inheritance is explained through X and Y chromosomes, and the more extensive presentation of DNA is associated with the function of DNA and RNA in gene action. This linking of DNA and RNA in gene action could have led students to select response E. The preliminary report of the IEA study will be followed by more de- tailed analyses of the test data and other variables not currently processed. The main report will be published in 1989. The National Assessment of Education Progress (Blumberg et al.,

OCR for page 43
so HIGH-SCHOOL BIOLOGY 1986) piloted the development and testing of higher-order thinking skills in science and mathematics for potential use in future national assessments. Exercises included hands-on activities of students to solve problems. Three modes of administration were used: intact classes with paper-and-pencil tasks, but with materials as stimuli; station activities with students rotating from station to station, each station having apparatus and investigations; and full investigations administered to individual students with an observer using a checklist to record what students did as they performed an inves- tigation. Third-, seventh-, and eleventh-graders were tested in 12 school districts. In one example of a station problem, eleventh-graders were to examine a set of 11 vertebrae, put them into three groups, and explain the similarities of the bones in each group. Cat, rabbit, and dog vertebrae were used. Fifty-four percent of the students were able to place the thoracic, cervical, and lumbar vertebrae into their proper groups. Another 20% grouped all but the atlas vertebra appropriately. Sixty-seven percent of the students provided at least one distinguishing feature for each group of vertebrae (Blumberg et al., 1986, Part II). BIOLOGY TEACHERS Only one recent study was found regarding biology teachers' knowl- edge of biological concepts. This study was reported in Cleveland, Ohio, newspaper, The Plain Dealer (Epstein, 1987), and found that only 12% of biology teachers surveyed correctly defined the modern theory of evolution. This study was based on written responses to items about evolution from 404 Ohio high-school biology teachers, about one-third of the biology-teacher population. Michael Zimmerman, a biology professor at Oberlin College who conducted the study, also found that 37.7% of the teachers surveyed favored teaching creationism and three-fourths felt that creationism was a favorable explanation for the origin of life (Epstein, 1987~. From these two studies and from those reported by other panelists, I believe we can conclude that major reconsideration of the goals and objectives of high-school biology education and of methods of assessing student interests, achievement, and attitudes is important. EVALUATION IN lIIGH-SCHOOL BIOLOGY Schools and such courses as biology are continuously subjected to informal evaluation by their many publics: parents, students, administrators, teachers, scientists, business men and women, and national groups. These informal evaluations carry great weight about the quality of education in each community and in the country as a whole. Efforts to inform these many judgments by more objectives measures and indicators of student

OCR for page 43
ISSUES IN OBJECTIVES AND EVALUATION 51 achievement have been low-technology, low-budget items. My judgment here is based on comparison of expenditures for accurate instruments for measurement in astronomy, physics, biology, medicine, and space activities. As I looked over evaluation instruments for biology, I saw little change in the last 50 years. A few efforts, such as those of the Educational Testing Service (Dresser and Nelson, 1956) and the Biological Sciences Curriculum Study (Schwab, 1963; Klinckmann, 1970; Mayer, 1978), provided teachers with resources for improving multiple-choice test items in biology. These resources provided sample items for going beyond pure recall and enabling students to demonstrate their capabilities of interpreting experimental data, applying knowledge to novel situations, and interpreting graphed data. More recently, the National Research Council (Raizen and Jones, 1985; Murnane and Raizen, 1988) has broadened the discussion of evaluation to include indicators of quality in science and mathematics education. The major issues in evaluation revolve around purposes and related instruments. Do we want to sort students on test scores similarly to the way we can sort students on height or weight? If so, we have norm- referenced tests (most standardized tests) that are designed to do just that. Norm-referenced tests are constructed, and items selected, to provide a normal distribution with mean and median at the 50th percentile. Most standardized tests are renormed about every 10 years. The new tests may be more or less difficult than the previously normed tests, but the new norms have statistical characteristics similar to those of the old. Another characteristic of the commonly used standardized tests is that they are designed to measure general knowledge and are not directly related to what is taught in any particular classroom. Within the last 20 years, criterion-referenced tests have been devel- oped, especially as part of the "minimal-competence" movement. Criterion- and domain-referenced tests are directly interpretable in terms of a "stan- dard." One problem with these tests is determining what the standard should be, other than in arbitrary ways. A second problem is the desire to make inferences about student competence by generalizing beyond an ability to achieve similar scores on similar paper-and-pencil tests (Haertel, l9SS). This identifies a second issue: "Can a single instrument serve all the purposes desired?" Among the purposes are diagnosis and guiding instruction, rank-ordering students, judging instructional quality, judging curricular quality, forcing curriculum and instruction to move in a particular direction, predicting future performance of individuals, and formulating policies for schools, districts, or states. Another issue is measuring student performance in a way different from the "recognition knowledge" that is assessed in multiple-choice formats. A great deal of interest is developing in generating alternatives to both the

OCR for page 43
52 HIGH-SCHOOL BIOLOGY commonly used forms of testing. One such alternative is performance testing: assessments that call on the examinee to demonstrate specific skills and competences and to apply them to novel situations (Stiggins, 1987~. Performance assessments have "four basic components: a reason for assessment, a particular performance to be evaluated, exercises to elicit that performance, and systematic rating procedures" (Stiggins, 1987, p. 344. Laboratory work is considered to be an important and necessary means of enabling students to attain the essential goals of biology education, but assessment of any unique contributions of laboratory work is rare (Robin- son, 1979~. Laboratory practicals have been used, but Gallagher (1987) commented that, despite the prevalence of laboratory work in science, we know very little about its effects on high-school biology achievement in the United States. Indeed, both effective and comprehensive evaluation prac- tices and evaluative instruments are a critical need for the improvement of high-school biology. Tamir and co-workers (Tamir, 1974; 1hmir et al., 1982) developed and have placed in use a laboratory practical in the schools of Israel, but evidence of its use outside Israel is lacking. A science-test review panel convened by the National Research Council (Murnane and Raizen, 1988) carefully examined nine science tests. The panel consisted of 12 scientists and high-school science teachers. They made three recommendations to avoid the misuse of science-test results (Murnane and Raizen, 1988, p. 180~: . Results from tests constructed for one purpose . . . should not be used for a quite different purpose. · School or classroom average test scores should not be applied to individuals, and individual test scores should not be interpreted as a rating or ranking of the persons, but only of performance on a test that assesses specific skills. · Test results or tests of the kind reviewed should not be used as the major force driving curriculum and instruction. CONCLUDING REMARKS Goals, objectives, and outcomes and the evaluation procedures used to assess them are two critical aspects of any proposal for policy formulation for high-school biology. I did not mention accountability earlier, but the accountability movement has stimulated the development of evaluation processes and can pressure curriculum and instruction to be concerned with only the aspects of biology that can be easily measured. In many instances, especially with many standardized testing programs, the student is forgotten in the process. It would seem that a first criterion of evaluation programs would be that they have significance to the students themselves.

OCR for page 43
ISSUES IN OBJECTIVES AND EVALUATION 53 The technology of assessment needs to have infusions of creativity, research, and development. Surely, computers and associated technologies can provide for more useful, instructive, and informative evaluation ~n- formation. Devising more effective evaluation instruments and procedures requires that we be clear and specific about the purposes of biology ed- ucation and the outcomes that we can reasonably be expected to attain with the approximately 134 hours we have to help a very diverse group of adolescents attain the understanding we propose. REFERENCES American Academy of Arts and Sciences. 1983. Scientific literacy. Daedalus 112:Spring issue. Blumberg, F., M. Epstein, W. MacDonald, and I. Mullis. 1986. A Pilot Study of Higher- Order Thinking Skills Assessment Techniques in Science and Mathematics. Final Report. Part II. Princeton, N.J.: National Assessment of Educational Progress. BSCS (Biological Sciences Curnculum Study). 1982. Biological Science: An Ecological Approach. BSCS Green Version. 5th ed. Boston: Houghton Mifflin. Dressel, P. L., and C. H. Nelson. 1956. Questions and Problems in Science. Test Item Folio No. 1. Princeton, NJ.: Educational Testing Service, Cooperative Test Division. Epstein, K C. September 3, 1987. Many Ohio science teachers favor study of creationism. The Plain Dealer. Cleveland, Ohio. Gallagher, J. J. 1987. A summary of research in science education-1985. Sci. Educ. 71:271-455. Haertel, E. 1985. Construct validity and criterion-referenced testing. Rev. Educ. Res. 55:2346. IEA (International Association for the Evaluation of Education Achievement). 1988. Science Achievement in Seventeen Countries. A Preliminary Report. New York: Pergamon Press. Klinckmann, E. 1970. Biology Teachers' Handbook. 2nd ed. New York: John Wiley and Sons. Mayer, W. V. 1978. Biology Teachers' Handbook. 3rd ed. New York: John Wiley and Sons. Murnane, R. J., and S. A. Raizen. 1988. Improving Indicators of the Quality of Science and Mathematics Education in Grades K-12. Report of the National Research Council Committee on Indicators of Precollege Science and Mathematics Education. Washington, D.C.: National Academy Press. National Science Foundation. 1983. Educating Americans for the 21st Century: A Plan of Action for Improving Mathematics, Science and Technology Education for All American Elementaty and Secondary Students So That Their Achievement Is the Best in the World by 1995. A Report to the American People and the National Science Board. Washington, D.C.: National Science Board Commission on Precollege Education in Mathematics, Science and Technology. Otto, J. H., and A. Awls. 1985. Modern Biology. New York: Halt, Rinehart and Winston. Raizen, S. A., and Lo V. Jones. 1985. Indicators of Precollege Education in Science and Mathematics. A Preliminary Review. Report of the National Research Council Com- mittee on Indicators of Precollege Science and Mathematics Education. Washington, D.C.: National Academy Press. Robinson, J. T. 1979. A critical look at grading and evaluation practices. In M. B. Rowe, Ed. What Research Says to the Science Teacher. Vol. 1. Washington, D.C.: National Science Teachers Association. Schwab, J. J. 1963. Biology Teachers' Handbook. New York: John Wiley and Sons.

OCR for page 43
1 ~ The NABT-NSTA High-School Biology Examination: Its Design and Rationale BARBARA SCHULZ EVOLUTION OF THE BIOLOGY TEST The evolution of the national high-school biology test is especially interesting. Early in this decade, there was much discussion about the need for such a test by both the National Science Teachers Association (NSTA) and the National Association of Biology Teachers (NABT). Each group ini- tiated preliminary discussions about whether such a test should be written, how it would be written, who would write it, and how it might be abused. In 1982, the high-school division of NSTA, under the direction of Linda Perez of Texas and Angelina Romano of New Jersey, conducted a needs assessment among the membership. The response was overwhelmingly in favor of a test. At the same time, NABT appointed a small committee to discuss the feasibility of such a test. This group, including Joe McInerney of Colorado and Ken Bingman of Kansas, discussed the notion of developing a test bank of questions available to the membership. In 1984, the NSTA board of directors passed a motion that NSTA and NAB T proceed with the development of a national test. The motion also asked that the president Barbara Schulz has been a high-school science teacher for 22 years and department chair since 1974. She was a recipient of the Outstanding Biology TeacherAward in 1981 and the Presidential Award for Excellence in Science Teaching in 1983, and she was a semifinalist for the Teacher in Space Award. Ms. Schulz was president of the Washington Science Teachers Association in 1987- 1988. 100

OCR for page 43
TlIE NAB T-NSTA HIGH-SCHOOL BIOL~YE~INATION 101 of each organization appoint four persons to serve on a joint test devel- opment committee. So it was that the first NABT-NSTA test development committee met in Chicago in June 1985 to begin the test construction. It was with a great deal of hesitation that this joint committee of nine biology educators proceeded with the test development. Some states have competence tests; others are considering such a move. We, the professional biology teachers, feel best qualified to design the test and help to set the direction of the biology curriculum. With the great demand for accountability being felt by all, it was clear that a test would be developed. The following statement of rationale and purpose was prepared and printed in both News & News and The Science Teacher Journal In 1985: A Standardized Test for First-Year High School Biology Rationale and Statement of Purpose There is an increasing demand for accountability in science education, and science educatom, through their professional organizations, should assume responsibility for establishing the mechanisms for that accountability, lest the responsibility fall to lay persons with vested interests. One such mechanism is a student education instrument. Accordingly, the National Association of Biology Teachers and the National Science Teachers Association are collaborating on a project to develop a standardized test for high school biology. This objective, year-end test will be intended for first-year high school biology students and will address a core of basic biological concepts, processes, and thinking skills. The joint committee has agreed on the following principles: a. The test should be used to improve science education; questions will be oriented toward inquiry and other higher-level cognitive functions; b. The test should not be used to evaluate teachem; c. The test should not become an end in itself, that is, the biologic content reflected in the test items should not be interpreted as the final word on a complete conceptual framework for an introductory biology course; and d. The test should be updated every two years. VALIDATION BY THE MEMBERSHIP Having declared that the purpose of this test is to drive curriculum forward, the committee looked at the question of content and level of difficult. The following concepts were decided on: Cell structure and function Sample concept: Biological systems vary in their degree of spe- cialization.* B. Bioenergetics Sample concept: Biological systems cannot exist without energy input.* *Biology Assessment Review Workshop. Biology Test Domains, Objectives and Content Speci- fications. Wisconsin Department of Public Instruction. 1983.

OCR for page 43
102 HIGH-SCH~L BIOLOGY C. Genetics Sample concept: Organisms pass on characteristics to the next generation through genetic material.* D. Evolution Sample concept: Organisms change through time. E. Systems, physiology to morphology Sample concept: Structure and function complement each other in biological systems. F. Ecology Sample concept: Organisms are interdependent, and their inter- actions result in the flow of energy and the cycling of matter. Taxonomy Sample concept: Biological systems are grouped on the basis of similarities that reflect evolutionary history.* H. Behavior Sample concept: The response of an organism to its environment has both a genetic and an environmental basis. I. Science, technology, and society Sample concept: Advances in science and technology have impli- cations for personal and societal decision-making. It was also decided that the following list of processes and skills should be represented in the way questions were designed: 1. Inquiry Process science Experimental design "Science as a way of knowing" (John ~ Moore) History Probabilistic thinking Creative problem-solving These concepts, processes, and skills were published in the same article with the rationale. A response card was published concurrently in both journals, and readers were asked to validate the conceptual framework and intent of the test. The test committee received a good response; more than 400 cards were returned. Also, sessions were held at the three NSTA regional meetings, the NSTA national convention, and the NAB T national convention for the purpose of concept validation. As a result of the meetings, the conceptual area of science, technology, and society was added to the test by popular demand. A call for test questions from the membership was made and a good response received. Each question was reviewed by a college-level content specialist and a high-school biology teacher for validity and appropriateness. From the solicited questions, 120 3. 4. 5. 6. 7.

OCR for page 43
THE NAB T-NSTA HIGH-SCHOOL BIOLOGY EXAMINATION 103 were selected for review and field-testing. Of those, 80 were selected for the final test. TEST CONSTRUCTION Armed with more than 300 questions, the committee turned to the issue of test content. The following numbers of questions were agreed on: Concept Cell structure and function Bioenergetics Genetics Evolution Systems, physiology to morphology Ecology Mono my Behavior Science, technology, and society No. questions 8 10 12 12 8 8 6 8 8 In addition, the committee decided that each concept area should be written at three levels of difficulty, ranging from knowledge to synthesis. A field test of 120 items was given in the spring of 1986 to students in 12 states covering all regions of the country. The Lertap test analysis was done on the field-test data. Eighty questions were then selected for inclusion in the first edition of the test. A second field test was done on the 80-item test in the fall of 1986 after some editing of the positively correlated distracters. THE RESULTS ARE IN By the spring of 1987, the test was published, and more than 30,000 copies were sold. The Educational lasting Service (ETS), Princeton, New Jersey, was hired to do the data analysis on the first commercial edition of the test. ETS analyzed 895 answer sheets from Form A and 1,075 answer sheets from Form B. Form A Form B Mean 43.8 40.7 Standard deviation 13.3 12.4 Median 44.3 40.2 Reliability (alpha) 0.91 0.91 Committee member Juliana Texley provided the analysis shown in Figures 1, 2, and 3.

OCR for page 43
104 10 5 o <' HIGH-SCHOOL BIOLOGY DATA - FORM B _ ~ rid . 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 ETn~ FIGURE 1 Jest Form B. 20 an c, 1 5 Oh 11 o 1 0 z G LL 5 o FIGURE 2 Test Form STUDENT SCORES DATA - FORM A Inn <5 10 15 20 Pnl I 55 60 65 70 75 80 30 35 40 45 50 STUDENT SCORES The test committee was pleased with the results. There was a strong interest by science teachers in using this test, and we have learned much from it. The committee made some minor revisions to correct misleading language in the 1987 test. The revised edition will be available for the next 2 years. In 1990, at the request of the memberships of NABT and NSTA, a completely new test will be designed and tested. The same format will be followed.

OCR for page 43
105 DATA - FORM A REVISED THE NAB T-NSTA HIGH-SCHOOL BIOLOGY EXAMINATION 25 In Z 20 g lL 15 o z IIJ llJ 10 5 o FIGURE 3 1bst Form A Revised. an n n n 65 STUDENT SCORES CONTINUED CHANGE THROUGH TIME The test will be recreated every 2 years. New committee members will rotate on to the test committee to serve a 3-year term, the time it takes to build, field-test, and publish the test. Questions will continuously be sought from practicing high-school biology teachers. Thus, the curricular emphasis within the test can change to reflect the concepts deemed relevant by biology teachers. The test may help to focus a solid core curriculum in biology, stated in terms of broad and significant concepts, rather than encyclopedic facts. The biology test, as an effort of professional associations, is not a product, but an on-going process. The first administration of the test has generated at least as many questions for the committee as we provided for our students. As the data from the first test were analyzed, the group was already planning a schedule for soliciting and field-testing new items for future tests. The committees will change, and the evaluation will evolve with the help and input of members. And as they do, we will learn more about our students, our classrooms, and ourselves. After 2 years of development, the first test results for biology are in, and another benefit of the test development process has also surfaced. We quickly realized that we have been given a valuable insight into what students know- about the science of biology. Scanning the answers of over 2,000 randomly selected subjects, the committee was able to peek into some widely held misconceptions and to hypothesize about the classroom procedures that might be perpetuating them. ~ a large extent, the test's validity derives from the many years of joint experience the committee

OCR for page 43
106 HIGH-SCHOOL BIOLOGY members share; their interpretations of the data they have received from the first test administration come largely from the same experience base. The group hopes to engender further discussion to improve not only the test, but the process of biology education. A PEEK INTO STUDENTS' MINDS Every teacher knows that the hardest part of test construction is choosing the wrong answers (the distracters or foils)-not too easy to spot, not so outlandish that no one would choose them. When the constructors of the national biology test received their item analyses, one of their most significant data sets was the percentage of students that chose each of the wrong answers. When one foil attracted a very large number of respondents, the obvious question always came up: Why? In a few cases, the foil was found to be marginally correct, given a slightly nontraditional reading, in a way the group had not foreseen. This type was changed by editing. But there was nothing "right" about many of the most frequently chosen foils. What was happening, it seemed, was that the foil touched . on a widespread student misconception or a teaching technique that often misfired. Some of the common errors seemed mnemonic; they seemed to result from verbal associations that we repeat too often in the teaching of biology: · When we asked students to complete the phrase. "The cell is a unit of structure and a unit of ," in the first field test, we were amazed to find that the majority chose "organ system." Had we taught the sequence "cell, tissue, organ, system" by rote once too often? Students demonstrated common vocabulary confusions, such as mistaking "cell membrane" and "cell wall." · When we asked a question about meiosis, the most popular choice was one that contained the word 'Gamete''-even though the sense of the answer was completely incorrect. The results suggested to the committee that far too many of our students are relying on word associations to weave their way through biology. Do such tricks work on classroom tests? Do we encourage them? Other common errors that the students demonstrated told us that some of our most important conceptual goals were often not met. In ques- tions about evolution, the Lamarckian explanation for an adaptation was consistently chosen as often as the explanation based on natural selection. This result is backed up by a number of research studies that show that the idea of inheritance of acquired characteristics is both intuitively appealing and surprisingly persistent in biology students of all ages. Similarly, the concepts of energy and entropy were difficult for students,

OCR for page 43
THE NAB T-NSTA HIGH-SCHOOL BIOLOGY EXAMINATION 107 despite the relatively simple and straightforward wording of questions. The idea that energy dissipates and does not cycle in the environment was a difficult one for students in several contexts. Perhaps too much emphasis was placed on cycles and too little on energy. We saw evidence of misconceptions that were and are text-perpetuated. Students believe (on the basis of misinformation in many texts) that muta- tions are always recessive and weak (Mahadeva and Randerson, 1982~. We also found that it was dangerous to assume that students had ex- perienced some of the more common laboratory investigations in first-year biology texts. Students found two questions about surface-to-volume ratio very difficult; it seemed that they had not explored the relationship between cell-membrane size and cell division. Any questions about hypertonic and hypotonic solutions were quite challenging when the terms themselves were not used. It seemed that students relied on the words, rather than the experiences, to influence their predictions. OTHER VARIABLES IN THE TESTING PROCESS In analyzing an evaluation tool, test-makers must be conscious of the other factors that can contribute to the variance in student performance. In the construction of the national biology test, the authors paid careful attention to the reading level and vocabulary of each question. In many cases, judicious editing was effective. But there was still evidence that the longer questions were harder than the shorter ones a result that was not expected. What was surprising, and what may provide the basis for more detailed research by the group, was evidence suggesting that questions involving visual or graphic analysis were harder as a group than the others in the instrument. The students who took the test seemed consistently confused by graphs and diagrams. In one item set based on an enzyme graph, the independent variable was increasing left to right, but students commonly erred by assuming that the enzymes represented left to right were in the order of their presence in the alimentary canal; that is, many believed the first were mouth enzymes, the second stomach enzymes. In a diagram of predator- prey relationships in a prairie, many students guessed that the prey of coyotes in that community would be jackrabbits-despite the evidence provided in a graph and clear directions to answer the question from that graph. In 1988, the committee examined seven demographic questions relevant to performance on the high- school biology examination. Each of the questions was preceded onto the test forms by students in self-selected classrooms and analyzed by means of one-way analysis of variance at a 0.05 level of significance. We randomly selected 882 tests. Although Forms A

OCR for page 43
108 [IIGH-SCHOOL BIOLOGY and B were distributed that year, only Form A responses were available in numbers suitable for random selection for analysis. (Previous analysis of test scores indicated that the forms were parallel, since only the order of the answers had been changed.) Of the students in the sample, ninth-graders and eleventh-graders did significantly better than the tenth-graders who would normally be enrolled in standard-level biology classrooms. In analyzing the data further, we found that students who indicated that they "never" experienced laboratory work did significantly more poorly than those who did laboratory work "some of the time." The frequency of laboratory work was not an important factor. However, those who had a laboratory experience did better than those with no laboratory experience or those who said they had laboratory all the time. While this identifies laboratory experience as necessary, it also brings into question student perception of "seldom," "frequent," and "most of the time." There is little evidence of standardization among advanced-biology sections, and some of these students may have been in courses tailored to individual research. The committee found no significant difference based on structure of schools. However, there was some evidence that students in smaller schools-500 or fewer performed significantly better. Our results on item difficulty gave us a clue to what was and what was not generally taught in the classrooms where our normative data were developed. Botany questions were uniformly more difficult for students than zoology questions. Mendelian genetics was surprisingly easy; modern genetic engineering was often very difficult. Taxonomy questions were the easiest (even though the test did not ask any specific taxa). And questions about the societal implications of modern biology and environmental prob- lems like acid rain were answered correctly by very few subjects, suggesting that teachers may be reluctant to add this emphasis to their curriculum. FUTURE TESTING For the immediate future, the committee has opted to add clearer pic- tures and diagrams for students who need such help. In years to come, both teaching and testing may be enhanced by far more visual stimuli; videotape and real-life examples may help students to reason more effectively with broader comprehension. Perhaps the most important implication of such a national test is not the result, but the point from which the committee started. With the recognition that we can't teach and students can't really learn everything in the commercial texts, the joint position of the associations is that the test establishes a core of nine concepts that should be a part of every student's first-year biology experience. It was this list and not

OCR for page 43
THE NABT-NSTA HIGH-SCHOOL BIOLOGY EXAMINATION 109 the questions themselves that seemed to elicit the most interest in the members, many of whom would rely on such a statement to guide their own choices. REFERENCE Mahadeva, M., and S. Randerson. 1982. Mutation mumbo jumbo. Sci. Teach. 49~3~:34-38.

OCR for page 43