Analysis of the AP and IB Programs Based on Learning Research
In this chapter, what is known about how people learn, especially the principles of learning presented in Chapter 6, is used as a lens for analyzing and assessing the Advanced Placement (AP) and International Baccalaureate (IB) programs in mathematics and science. In Chapter 9, this analysis is extended with respect to the design principles of curriculum, instruction, assessment, and professional development presented in Chapter 7. The analysis highlights some major strengths and shortcomings in the programs as currently implemented in U.S. high schools and indicates directions for change that would help bring the program elements into closer alignment with the principles of learning set forth in Chapter 6 and with the goals of advanced study. Although the analysis is often critical, the committee recognizes that these advanced programs challenge some of the nation’s most talented and highly motivated students and some of its most capable and creative high school teachers to accomplish more than is generally expected of a high school education. Both programs have provided models of high expectations for students and of recognized and valued external assessments of student achievement. Regular and even honors courses are often not sufficiently challenging for many of the best students in the sciences and mathematics.
The AP and IB programs are dynamic and were moving toward significant change even as the present study was being conducted, as exemplified by the report of the Commission on the Future of the Advanced Placement Program (CFAPP) (2001) and the revision of the IB Experimental Sciences curriculum, discussed in Chapters 3 and 4, respectively. Both programs are being adapted to serve broader purposes than those for which they were originally designed. Sometimes, however, the programs are misused through no fault of their developers, as discussed in Chapter 10.
Another set of caveats is in order. Neither the College Board nor the International Baccalaureate Organisation (IBO) articulates the principles of learning on which its program is based. This is understandable, as both
programs were created well before the nature of human cognition and learning became the focus of intensive research or the potential for using this research to inform the design of educational programs was emphasized. Nonetheless, now that research is providing a clearer picture of how people learn, it is important to use this information to consider how to improve the programs’ courses, assessments, and professional development activities.
Systematic information is lacking about the AP and IB programs as they are actually implemented in U.S. high schools, including the instructional strategies used in individual classrooms, the structure of the syllabi in different schools, the quantity and quality of the facilities available, the preparation of teachers who teach the courses, and the ways in which students are prepared prior to advanced study.1 What is known, however, is that there is wide variation among teachers and schools. Yet data on the nature of this variation and its effect on student learning are scant, as is information about the cognitive processes elicited by the AP and IB assessments. Because important data about the programs have not yet been published by either the programs or independent researchers, the committee focused its analysis on what the programs say they do, using available program materials such as course guides, released examinations, teacher manuals, program goals and mission statements. and expert testimony from program officials and experienced AP and IB teachers.2
The discussion in this chapter is organized around the seven principles of learning set forth in Chapter 6 and is based on the evidence noted above, as well as the findings of the four panels that conducted in-depth appraisals of the AP and IB programs in mathematics, biology, chemistry, and physics for this study.3 The analysis is focused on determining the extent to which the AP and IB curricula in science and mathematics and the associated instruction and assessments are aligned with the principles of learning and goals for advanced study outlined in previous chapters. This analysis (and that of Chapters 9 and 10) serves as the foundation for the recommendations offered in Chapter 11 for improving advanced study in general and the AP and IB programs in particular. In presenting these analyses, we emphasize that AP and IB are different programs designed for different purposes, and
The College Board is beginning to undertake some new research related to how the AP courses are implemented in schools. See www.collegeboard.org/ap/research/index.html (November 28, 2001). The IBO has also established a research committee to oversee studies on the nature of learning in IB classrooms. See www.ibo.org (February 8, 2002).
The College Board and the IBO provided the committee with a considerable range of program materials, such as mission statements; course outlines; teacher guides; sample syllabi; released examinations; scoring rubrics; and research results from studies conducted under the auspices of their researchers, as well as by independent researchers.
therefore that findings related to one program do not necessarily apply to the other.
PRINCIPLED CONCEPTUAL KNOWLEDGE
A vital goal of advanced study is to help students progress from novice understanding toward deeper understanding of the fundamental content and unifying concepts of a discipline and toward greater expertise in the processes of inquiry and problem solving. Because they lack an organizing structure, novices tend to approach problems by seeking formulas or algorithms they have used in the past, by focusing on surface features of the problems, or by relying on their intuitions and preconceptions. Experts, on the other hand, think in terms of core concepts and underlying principles in approaching and solving problems. The evidence gathered for this report indicates that, in constructing their programs, AP and IB do not make adequate use of what is known about differences in the ways in which experts and novices structure knowledge. The committee’s findings in this area are based on several key observations about the programs.
Breadth Versus Depth
According to the mathematics panel, the breadth of coverage in AP Calculus is appropriate. The curriculum for the IB Mathematical Methods Standard Level (SL) and Mathematics Higher Level (HL) courses, however, encompasses an introduction to elementary calculus (similar to the AP program’s AB-level calculus course) and additional areas of study that the teacher selects from among available options. The small number of hours suggested for study of each topic (20 hours for Mathematical Methods SL and 50 hours for Mathematics HL according to the IBO’s 1998 course descriptions) leads to the concern that students study each topic only at a procedural level.
The inclusion of too much content is a major issue in the sciences, especially the AP sciences. The 2-year format of the IB HL science courses makes the issue of breadth of content coverage less of a concern for these courses. However, the 1-year IB SL science courses suffer from some of the same shortcomings as the AP science courses in this regard.
The content that is expected to be taught and ultimately assessed in the final examinations for many of the AP and IB HL science courses is quite broad. According to the findings of the biology, physics, and chemistry panels, most teachers, especially AP teachers, find they have insufficient time for more than superficial coverage of some topics before moving on to others. These three science panels are unanimous in expressing their concern that,
in an effort to prepare students for an examination by covering all of the topics on the course outline, teachers may place undue emphasis on the presentation and memorization of disconnected facts.
In the science disciplines, both AP and IB materials emphasize concepts, key ideas, and unifying themes. However, the biology, chemistry, and physics panels emphasize that this commitment is largely unrealized in the programs’ assessments. Although the science panels find that the examinations do not adequately assess depth of conceptual understanding and place disproportionate emphasis on recall, the panels are encouraged by the fact that the College Board is moving toward making improvements in this area. The 2002–2003 AP biology course description states, “Questions on future AP Biology Examinations can be expected to test students’ ability to explain, analyze, and interpret biological processes and phenomena more than their ability to recall specific facts” (College Entrance Examination Board [CEEB] 2001a, p. 7). Although the IB examinations sample the range of assessment statements included in the syllabus for each course, the science panels cite as weaknesses what they consider to be undue emphasis on factual knowledge and failure to target misconceptions and adequately measure conceptual understanding.
In their review of the influence of external tests on the curricula provided in the schools of several nations, Madaus and Airasian (1978) conclude that “most studies have found that the proportion of instructional time spent on various objectives was seldom higher than the predicted likelihood of their occurrence on the external exam” (p. 21). Since both AP and IB courses are designed to prepare students for high-stakes, end-of-course assessments, an instructional focus on “big ideas” will likely remain unrealized until these assessments focus on key concepts in each domain, as has occurred to a considerable extent in AP calculus.
When an advanced course occupies only a single year, as in the AP program, the broad scope of the curriculum and the demands of the assessments can create a conflict between two goals: providing the broad base of content knowledge that is perceived to be necessary for the exam, and promoting conceptual understanding that provides the framework for future learning and application of knowledge. Program developers can continue to address this issue by devising focused curricula and by working to equip teachers to make judgments that support depth of understanding.
Organizing Complex Content
A second observation of the committee and the panels is that the topic outlines for AP and IB science courses sometimes provide insufficient guidance to teachers about how to organize the enormous amount of recom-
mended content around important central ideas. The course objectives are not uniformly structured to reflect the organizing themes of the discipline; each objective is presented in isolation, rather than as part of a larger network. Yet it is precisely this network—the connections among objectives—that is important (National Research Council [NRC], 2000b).
In biology, for example, although the AP course description explicitly addresses principles and themes, teachers are merely given a brief, open-ended framework for conceptual knowledge using themes that cut across different topics and must organize the content themselves. The committee is concerned that students, given the sheer volume of material with which they are confronted in a typical AP biology course, may be unable to use the organizing themes effectively, even when they know what those themes are. Being able to state a theme or memorize a statement is a far cry from understanding it.
The IB Biology Guide, like the guide for all of the IB experimental sciences, identifies “essential principles of the subject” that are common to both the SL and HL courses in a discipline, but the assessment statements used to define what should be taught are grouped by topic rather than thematically. Here also teachers are expected to organize and integrate the material on their own. While it is important not to specify course designs too rigidly, further guidance on organizing the recommended content might produce more consistent results—especially when teachers are undergoing changes in their practice.
In contrast, in the opinion of the mathematics panel, the topics in the AP calculus syllabi are appropriate and well connected. In its current formulation, AP calculus pays careful attention to the central concept of function and to connecting the common ways (numerical, graphical, analytical, and verbal) of representing functions. There is likewise careful attention to developing the main concepts of differentiation and integration, including several interpretations of and applications for each.
Examinations and Conceptual Learning
The AP and IB examinations could be improved if they required students to demonstrate a deeper level of conceptual understanding to earn passing grades. Two of the panels (calculus and chemistry) attribute this problem, of limited depth, at least for AP, in part to the relatively predictable nature of many items on the respective examinations. This characteristic allows or even encourages teachers to instruct students on how to answer particular types of questions, a practice that might not occur if tests consisted of widely varied items whose solutions required the integrated application of key principles.
For example, the mathematics panel is concerned that teachers may be encouraged to focus on teaching their students specific problem types instead of engaging them in modeling and problem-solving activities that would lead to greater understanding of the underlying concepts of the subject. As a result, students could compute a right answer in a limited range of situations but be unable to explain why their solution is correct. Students who do not understand the concepts underlying procedures will have difficulty using mathematics to solve unexpected problems or applying their knowledge in other disciplines, such as physics or economics, or in real-world settings.
The mathematics panel notes that the AP examination has improved under the new syllabus. The effort to promote conceptual understanding by asking nonstandard questions and requiring verbal explanations is an excellent step in the right direction. The fact that there is now a wider variety of applications of integration (and not from a prescribed list) encourages students to think about the meaning of an integral. The panel recognizes and applauds the fact that that the College Board is taking such steps to encourage more modeling activities (applications) and to make its tests less predictable (see, for example, CEEB, 2001b). The mathematics panel notes that many problems on the IB mathematics examinations could also be solved procedurally if students were taught to do so or if they had solved enough practice problems. The panel concludes, however, that there is such a broad range of topics that it would be difficult for students to do well on the examination as a whole without understanding the underlying concepts. Indeed, the panel’s analysis of the IB examinations suggests that considerable conceptual understanding is required for students to do well. It appears, however, that opportunities are missed throughout the exam to connect procedural knowledge with conceptual knowledge; this emphasis on procedural knowledge could lead to superficial instruction in IB classrooms.
The science panels call for final assessments that are better tests of conceptual understanding, with less emphasis on students’ ability to remember discrete facts, formulas, and procedures. The inclusion of novel tasks that require the application of key principles could encourage teachers to focus on students’ development of conceptual understanding and ability to make interdisciplinary connections. The use of a wider range of assessment strategies also might encourage effective instruction (Rice, Ryan, and Samson, 1998). The science panels note that both AP and IB are taking positive steps in this direction. For example, some AP science examinations ask students to design experiments and to demonstrate their solutions to problems. Doing well on this task requires that students have significant prior experience with designing experiments to solve a variety of problems. Teachers thus have an excellent incentive to engage students more regularly in these types of activities.
Research on cognition has strongly confirmed the importance of prior knowledge for all aspects of thinking and learning. One reason prior knowledge is important is almost self-evident: new knowledge is built on what learners already know. Thus before entering AP or IB courses, students need a solid knowledge base.4 To focus the curriculum and instruction appropriately (a critical factor in maintaining student motivation, as well as in developing understanding), teachers need a clear idea of the depth of knowledge, skills, and experiences their students bring to the classroom.
Both the AP and IB programs provide fairly detailed information about the necessary prerequisite knowledge and skills in mathematics.5 However, neither the College Board nor the IBO clearly specifies the competencies students must master in advance in the sciences. In both mathematics and the sciences, the College Board suggests (but does not require) that students complete a previous course or combination of courses before enrolling in AP sciences. Schools are free to adhere to these suggestions or not. Statements about the need for such courses are of limited value for several reasons. First, similarly titled courses vary widely in content and rigor, so students may not have the necessary background even if they follow this advice. Second, some students acquire the knowledge and skills necessary for success through independent study or related experience, not through traditional course taking. Finally, even when prerequisites are clearly specified, it may be difficult to ensure that students have met them.
The IBO does not specify any prerequisites for the experimental sciences. IB HL courses are taught over 2 years. Therefore, the IBO does not believe it necessary to specify prerequisite courses for the experimental sciences, assuming that a 2-year course of study provides ample opportunity for students to both build a solid foundation in the discipline and pursue advanced work. However, these courses also are built on prior knowledge to some extent, and this is worth specifying.
Identifying the prerequisite student knowledge and skills needed for success in each course is of considerable importance to schools in providing adequate preparatory experiences (tools for gauging student mastery of these skills would also be useful). A clear articulation of the body of necessary
prior knowledge could help teachers and counselors advise students about their readiness to take specific AP or IB courses, and appropriate assessments could help them identify areas of weakness that should be addressed before advanced study begins. The programs attempt to provide this information to varying degrees. However, identifying the full range of prerequisite content knowledge, conceptual understanding, and skills is a complex task requiring the combined efforts of experts from the science disciplines, cognitive psychologists, master teachers, and staff from the advanced study programs. The committee believes that the benefits for student learning are well worth the effort. At the same time, while this task is important, it is also essential that prerequisites not become barriers to participation for students from particular ethnic or socioeconomic groups (see Chapters 2 and 10, this volume).
An alternative to the use of more testing to measure readiness for advanced study might be to adopt a developmental approach, such as that devised by the Australian Council for Educational Research. Central to this approach are models of learning known as progress maps, which provide a sequential description of skills and knowledge indicating both the goal and the steps necessary to achieve it. Readiness for AP or IB courses could be calibrated with a position on such a map (as cited in NRC, 2001a, p. 181). Use of such an approach also might be beneficial for students seeking to prepare themselves for advanced work through independent study.6
The American Association for the Advancement of Science’s (AAAS) Project 2061 program for science literacy recently produced an atlas of learning maps that indicate the interrelationships among science concepts. Although limited in scope at present, these maps provide a useful set of guidelines for building upon science concepts as students advance through school (AAAS, 2000). Adapting aspects of these maps could assist experts in developing appropriate measures of prior knowledge and readiness for advanced study.
Coordination of Courses
The fact that students need to acquire certain prerequisite knowledge and skills before participating in advanced study implies that greater attention should be paid to across-grade curricular and instructional planning. If advanced study is viewed as an extension of the regular curriculum, it follows that students’ success in such programs depends on knowledge and skills acquired much earlier.
To date, progress maps have been developed primarily for elementary and middle school mathematics and science. If they are to be useful for the purpose proposed here, teams of experts will be required for their development. Examples of progress maps can be found at www.acer.org/.
For example, the study of calculus logically builds on foundational knowledge acquired in previous courses in mathematics. Yet the mathematics panel points out that there are not enough checks in the system to ensure that students have mastered the algebra, trigonometry, and precalculus concepts and skills necessary for success in calculus and in courses beyond. The performance of many AP calculus students is undermined by the fact that they do not learn precalculus thoroughly, or learn adequately how to solve problems and think mathematically. This situation is not a defect of the AP calculus course per se, but of the system in which it is embedded.
An example of prior foundational knowledge is the periodicity of trigonometric functions, such as sine and cosine. While these are not calculus concepts, an understanding of these functions is needed in the study of calculus. If these ideas are learned poorly or not at all in previous coursework, the calculus teacher must devote valuable instructional time to their development, or some students will experience difficulty.
The science panels argue that a syllabus for a year-long advanced course that also serves as an introduction to the discipline allows too little time for students to develop the depth of conceptual understanding that should be the fundamental goal of advanced science courses. Thus they unanimously recommend that students study the biology, chemistry, or physics that is suggested for grades 9–12 in the National Science Education Standards (NRC, 1996) before enrolling in advanced high school courses in those disciplines.7 At the same time, to ensure that students who complete introductory courses do not face excessive redundancy in subsequent courses requires active coordination among teachers at different grade levels.
Implications of Prior Knowledge for Instruction
Because prior knowledge is so important to students’ ability to understand new material, care must be taken to structure advanced courses around a coherent curriculum. That is, the presentation of topics must be sequenced so that the concepts and processes imparted to students gradually become more complex (AAAS, 2000). Neither the AP course descriptions (see for example, CEEB, 2001b, p. 8) nor the more detailed IB program guides provide guidance on sequencing concepts or topics. In fact, the IB Chemistry Guide notes that the order in which topics are presented on the course outlines does not constitute a recommended sequence: “It is up to the individual teachers to decide on an arrangement which suits their circumstances”
(IBO, 2001b, p. 35).8 Similar statements are included in IB program guides for mathematics, biology, and physics (IBO, 2001a, p. 5; IBO, 2001c, p. 9). This flexibility can be a strength in the hands of good teachers because they can adjust their activities in response to what their students understand. On the other hand, less-experienced AP or IB teachers may try to follow college textbooks too closely, and according to the chemistry panel, their doing so could contribute to the problem of excessive breadth.
Misconceptions and Formative Assessment
One important subset of prior knowledge involves misconceptions and naive theories, as discussed in Chapter 6. Even well-performing students often retain and use deeply flawed prior beliefs. The AP and IB programs could address this problem in several ways. First, using emerging research on student misconceptions, they could include in their course guides information about typical misconceptions that students may already harbor or that may arise during study of each of the topics included in the courses. There is considerable research and detailed description of the common types of misconceptions held by learners in mathematics and physics (see for example, Driver, Squires, Rushworth and Wood-Robinson; 1994; Gabel, 1994; Minstrell, 2000b). Information might be provided in the teachers’ guides about these common misconceptions, as well as strategies that could be used to address them. Second, a focus on detecting and addressing common misconceptions in content areas could be built into professional development programs. Third, items could be included in the AP and IB final assessments that would reveal common misconceptions, thus focusing teachers on the goal of identifying and remediating misconceptions that persist even after instruction. While the AP physics examination now does this to some extent, the mathematics panel notes that AP calculus examinations miss opportunities to include questions that address common student misconceptions, such as those that arise in attempting to understand the derivative, slope fields, and functions.
One effective strategy for identifying student misconceptions, especially those that endure after instruction, is the use of curriculum-embedded (formative) assessment, which enables teachers and students to set intermediate goals for learning and to select appropriate materials and tasks for pursuing those goals. Formative assessment can also provide information about students’ preconceptions and misconceptions, thereby informing subsequent instructional choices (see, for example, Kozma and Russell, 1997; Mestre, 1994; Minstrell, 2000a; NRC, 2001a). The College Board and the IBO could
develop or help teachers develop the tools needed for this purpose. Although some excellent teachers apply this strategy already, it probably cannot be presumed that most do so.
The IBO helps teachers integrate formative assessments of students’ practical (laboratory) work and portfolios into instruction on an ongoing basis. However, it does not provide guidance on the development or structure of other forms of ongoing assessment. The AP program provides no explicit guidance on the structure or development of classroom-based assessments, although course guides and teachers’ guides mention the usefulness of such measures. The panel’s appraisal of professional development materials provided by the AP and IB programs indicates that classroom assessment receives limited attention.
Lacking appropriate interim assessments, many teachers use previously released final examination questions throughout the school year to measure student achievement. This strategy may be useful for predicting students’ performance on the AP or IB examinations or providing practice with the test format and item types, but it is of little use as a tool for identifying misconceptions or appraising students’ conceptual understanding because the test items are not designed for this purpose. AP and IB teachers also would benefit from professional development opportunities directed toward improving their knowledge of common misconceptions and assisting them with the development of activities or formative assessments that could be used to detect and address those misconceptions. Experienced teachers of advanced study would be needed to staff such workshops.
Advanced study courses present a significant opportunity for providing students with instruction in metacognition, or self-monitoring of learning (Mintzes, Wandersee, and Novak, 2000). Research has shown that strong metacognitive skills are characteristic of experts in any field, as well as of effective school-age learners, and that they are developed most effectively in the context of a discipline. For example, White and Frederickson (1998) demonstrate that teaching students metacognitive strategies in physics improves their understandings of the concepts of that discipline. Similarly, Schoenfeld (1983, 1984, 1991) shows that a metacognitive approach to instruction improves students’ heuristic methods for mathematical problem solving.
Minimally, AP and IB course materials should recognize proficiency in metacognition as a goal of advanced study. Teachers should elicit metacognitive activity in classroom discourse by posing questions and modeling reasoning. In addition, suitable opportunities for students to demon-
strate metacognitive strategies should be built into curricular materials and assessments. The goal should be to help students become proficient in monitoring their own thinking and learning.
Although the AP course outlines and professional development programs for mathematics and science teachers currently pay little explicit attention to metacognition, the teachers’ guides suggest that teachers help students develop skills that are important components of metacognitive strategies. For example, according to the teacher’s guide for AP chemistry (Mullins, 1994, p. 8), “It would be unreasonable to expect that all AP students come into the class with strong analytical and critical thinking skills; therefore the AP teacher must strive to strengthen these skills within the students. A process of problem solving should be continually modeled and reinforced.” Likewise, the teacher’s guide for AP calculus (Kennedy, 1997, p. 48) suggests that “teacher modeling of questioning and exploration, multiple approaches to problem solving, correct use of terminology, clarity of work, and openness to the ideas of others are all essential.” The committee views these suggestions as positive steps toward addressing the important role of metacognition. However, the statements in the teacher’s guides are isolated, and the committee believes the importance of metacognition currently is substantially underrepresented in both the teacher’s guides and the types of questions students encounter on the final AP examinations in science. As is the case for principled conceptual knowledge, an instructional emphasis on metacognition will not be realized until that emphasis is reflected in the assessments.
The committee views as promising the fact that in future AP calculus examinations, students will be required to justify and explain their solutions to all calculator problems. Scoring rubrics will consider both the correctness of the solution and whether the reasons for using the selected procedures are justified. Given the strong influence of the assessments on the content and structure of curriculum and instruction, it is likely that AP calculus teachers will provide frequent opportunities for students to make their thinking visible. The committee commends the AP calculus development committee for its attention to this important principle of learning and encourages other AP development committees to follow suit.
The IB program also provides significant opportunities for students to learn and develop metacognitive skills. As discussed in Chapter 4, the IB Theory of Knowledge course9 is designed to provide students with knowledge about cognitive processes and strategies, practice in using those strategies, and opportunities to evaluate the outcomes of their efforts. “It encour-
This course is a requirement for students seeking an IB Diploma (see Chapter 4, this volume, for other requirements).
ages students to become aware of themselves as thinkers, to become aware of the complexity of knowledge, and to recognize the need to act responsibly in an increasingly interconnected world” (IBO, 2000b, p. 3).
DIFFERENCES AMONG LEARNERS
The existence of differences among learners and the extraordinary complexity of such differences have implications for how educational programs are structured, how students are taught, and how learning is assessed. They are also relevant to teacher professional development.
Research on how students differ in knowledge representation and expression is difficult work that is not yet complete. Nonetheless, programs such as AP and IB can begin to address student variations by building corresponding variability into their curricula and assessments and by addressing learner differences as part of their professional development opportunities for teachers. While variety will not ensure a perfect match of instruction or curriculum to the preferences or abilities of each student, it will likely provide many more students with opportunities to learn in ways best suited to their strengths, at least some of the time.
AP or IB teachers who creatively mix pedagogical approaches with sensitivity to the profile of learners in particular classes are likely to reach a broad range of students with varying learning modes and may motivate more students as well. The IB program guides for the experimental sciences recognize the importance of this tenet by indicating that there is no single best approach to teaching IB courses and that teachers should provide a variety of ways of acquiring information that can be accepted or rejected by each student, allowing different routes through the material (IBO, 2001a, 2001b, 2001c, p. 11). A combination of approaches is also likely to result in students learning the same concepts in multiple ways, a strategy that is conducive to building deep understanding. An example of such a strategy is the current “rule of four” emphasis in calculus, which suggests that students, with the help of their teachers, see the links among four representations of mathematics: numerical, graphical, symbolic, and verbal.
Using Differences Among Learners in Assessment
As discussed in Chapter 6, students with different learning styles may need different ways to demonstrate their knowledge and skills. For example, some students work well under pressure, while the performance of others is significantly diminished by time constraints. Some students express themselves well in writing, while others do not. Using one form of assessment will work to the advantage of some students and to the disad-
vantage of others. Research (see, for example, Linn, 2000; Mintzes et al., 2000; NRC, 2001a) also indicates that the use of different measures administered over a period of time will offer a more accurate picture of what students know than any single measure can provide (Linn and Hsi, 1999). Using information gathered from multiple sources shows respect for the diversity among learners.
The internal assessment that is part of all of the IB experimental science courses and the portfolio that is part of the IB mathematics courses are examples of how large-scale assessment can incorporate the results of different types of measures into a final evaluation of student achievement. However, the scores earned on the internal assessment and the portfolio are not reported separately, but are combined with scores earned on the written final examination and contribute only a fraction of the student’s total score. Therefore, some information is lost that might be useful in providing a fuller picture of students’ capabilities.
The AP reporting levels of 1 to 5 do not differentiate well among students at higher levels of achievement. A score of 5 putatively represents a commendable level of proficiency, but it fails to capture just how proficient a student is or what a student knows qualitatively. At all levels of achievement on the AP assessments, the score by itself does not speak to what students actually understand. For example, a student who earns a 3 on an AP biology examination might know a great deal about a few topics or less about many things. A student who earns a score of 1 might either know little or have failed to make an effort on the examination. The physics panel addresses this issue at length, indicating that two students who earn scores of 5 can have vastly different levels of understanding. Scores on the 2000 AP Physics B examination that result in a 5 range from 104 to 180. The panel states unequivocally that students at the higher end of the score range are far more competent than those at the lower end. Institutions receiving student scores could benefit from information that reflects this difference. The committee notes that even students who achieve a score of 5 could have serious gaps or inconsistencies in their understanding, and these gaps ought to be revealed as well.
With some planning and effort, the College Board could do much better in assigning qualitative meaning to summary scores.10 One possibility is that a numerical summary of performance—say a score of 4—could be linked to a description of what students who earn that score are likely to understand.
It also may be possible to review the specific responses of examinees and evaluate them for gaps in understanding, or to provide subscores for specific skill and knowledge areas. Currently, most AP examinations do not include enough items targeting the same concept or skill to allow development of a meaningful subscore for individuals.11 An exception is the AP Calculus BC exam, which already provides a useful subscore indicating proficiency on the AB section of the material. To develop useful subscores on other exams, the assessments might have to cover less material more deeply or include more items. Reducing the breadth of material covered on the assessments could potentially accomplish two important goals: providing more meaningful and useful information about student performance, and reducing the breadth of content teachers are expected to cover and students are expected to learn in AP classes.
Learners’ academic motivation influences their choices about which tasks to undertake, the persistence with which they pursue these tasks, the level of effort they expend, and their thoughts about the usefulness of the tasks and their performance on them. Eccles and colleagues (Eccles, Wigfield, and Schiefele, 1998) organized research about motivation into three broad questions students might ask themselves when confronted with a new task: Can I do this task? Do I want to do this task and why? What do I have to do to succeed on this task? These questions are used below to frame an analysis of the ways in which the AP and IB programs help or impede students in developing and maintaining their motivation in advanced study programs.
Believing in the Possibility of Success
Factors that contribute to students’ sense that they can be successful in AP or IB courses have been discussed earlier in this report. These factors include students’ positive beliefs about their own competence, their expectations for success or failure, their sense of control over the outcomes, their belief that others think they are competent, their belief that support of various kinds is available if they need it, and a strong sense that what they have learned before is adequate preparation for the challenges they will face in AP or IB. The IBO’s Middle Years Programme and the College Board’s Vertical Teams, Equity 2000, and Pathways Programs are designed to help students believe they can be successful in AP and IB courses.
Deciding to Enroll in an Advanced Course
Many able high school students and others take AP and IB courses because they seek opportunities to challenge themselves academically—opportunities that might otherwise not be available in high school courses. Others select these courses because they believe extrinsic rewards will accrue to them for doing so. These rewards include, for example, a potential advantage in the college admission process at competitive colleges,12 opportunities for academic recognition, the possibility of earning college credit, and the possibility of placing out of introductory college courses.13 Most students, however, are motivated by a combination of internal and external factors.
Some students take these courses because their parents want them to do so. Parents’ reasons may include those mentioned above, as well as beliefs that AP and IB courses are taught by better teachers and produce more learning, that there are fewer discipline problems in these courses, and that students will have greater access to resources and opportunities in these courses than in others. Peers also exert a strong influence on students’ decisions on whether to enroll. Students who are members of peer groups that value AP or IB participation are probably more likely to enroll even when they are not personally interested in a course. This observation may have serious implications for able students who attend schools where their peers do not value academic achievement. Still others take AP and IB classes because no other options exist in their high schools. This is particularly the case in mathematics: students who complete precalculus often have no other choice but to take AP or IB calculus if they want to continue with mathematics.14
Investing Effort for Success
Even when students are prepared to take AP or IB courses and have determined that they want to do so, they still must ask themselves whether
Responses to a survey of college admission officers conducted by the committee clearly indicate that student perceptions in this area are accurate and that the more selective a college is in terms of admission, the more important it is that a student participate in AP and IB courses if their high school offers such courses (see Chapter 2, this volume).
The committee conducted a survey of biology and mathematics departments at a sample of colleges and universities to ascertain how AP test scores are used for placement and credit (see Chapter 2, this volume).
In September 1986, the Mathematical Association of America and the National Council of Teachers of Mathematics released a joint statement advocating that calculus courses taken in high school be at the college level, in other words, at the level of AP or IB. The statement was published in Calculus for a New Century (Steen, 1988).
they are willing to expend the effort required for success. Researchers have begun to examine the relationships among motivation, cognition, and the social context in which learning takes place. The results of some of this work suggest that educational programs designed in accordance with the principles of learning described in this report can enhance students’ motivation to engage in advanced study even when considerable effort is involved. Research also has shown that when computers, teachers, and peers are integrated as learning partners, not only does greater achievement result, but also more positive affective changes occur (Canas, Ford, Novak, Hayes, and Reichherzer, 2001; Linn and Hsi, 1999).
Motivation and the Final Examination
It is interesting to note that a far larger percentage of IB than AP students take the final examinations.15 The IBO promotes the idea that IB courses prepare students for success in college and in real life. The examinations, students are told, are an integral part of the course and are the best way for them to demonstrate to themselves and others that they have achieved competence. In contrast, AP materials focus primarily on the usefulness of AP test scores for college credit and placement. If students lose interest in earning credit or placement or the colleges at which they plan to matriculate do not accept AP credits,16 they may choose not to sit for the examinations.
The College Board is concerned about the number of students who do not take the AP examinations and is considering ways to address this issue (CFAPP, 2001). All four of the panels and the committee believe that sitting for the examinations should become an integral part of AP courses (see Chapter 11). Otherwise, students will miss an important opportunity to validate their performance, colleges and universities will lack information that can be highly useful in deciding on appropriate placements, and AP will be less credible as a rigorous program for high school students.
Learning is mediated by the social environment in which students interact with others. Although students also learn individually, research indicates
that learners benefit from opportunities to articulate their ideas to others and listen to and challenge each other’s ideas, and in so doing reconstruct their thinking. Teamwork and collaborative investigations similar to those in which professional scientists engage help students refine important concepts and principles and apply them in real-world contexts. Students build on the learning of others through interaction. For example, students who are using an ineffective problem-solving strategy can learn by observing others who are using more productive methods.
The panels see learning communities as an important avenue for students to develop advanced understanding in their disciplines. The panels recognize the importance of discourse to both student construction of understanding and scientific discovery (Dunbar, 1995). They are concerned, however, that the push to cover too much content during the course of a school year can discourage teachers from permitting students to engage in lengthy discussions, ask questions, make conjectures, and challenge each others’ ideas. The science panels’ reviews of the broad scope of AP and IB course outlines, sample course syllabi, school scheduling options, and their own experiences lead them to believe that in many (but not all) AP and IB science classrooms, a lecture format is used extensively. Lectures can be an efficient tool for transmitting information, but they are not very useful for eliciting students’ misconceptions, helping them understand the conditions for which principles are valid, or providing them with opportunities to practice skills and receive feedback. Traditional lecture formats limit the ability of students to be active participants in their own learning because they cannot control the pace of the information that is being presented, and interaction between lecturer and learner is limited at best.
The science panels are also concerned that students’ opportunities to engage in collaborative work with peers in the laboratory components of science courses may be limited by the brevity of the periods within which the laboratories must be offered in today’s high schools. They note that most college students engaged in the same courses have laboratory periods that are 2 to 3 hours long. The panels commend the College Board for advocating longer laboratory periods (CEEB, 2001a, 2001c, 2001d), but are skeptical about the frequency with which schools are able to implement this suggestion because of other scheduling constraints. The science panels note that a reduction in the breadth of science curricula could allow more time for collaboration and inquiry.
The panels note approvingly that the internal assessment criteria for IB experimental science courses require teachers to structure the classroom and laboratory environments so that students have opportunities to acquire and develop skills in working with a team. It is likely that both the AP and IB programs could make more deliberate use of interactions in promoting students’ understanding of science and mathematics through such strategies as
group case studies, project-based science activities, and use of technology that facilitates discourse (Ryder, Leach, and Driver, 1999). Finally, it may be noted that although class time is limited, collaboration can be encouraged outside of class on homework and other assignments, at the discretion of the teacher.
LEARNING IN CONTEXT
The situative perspective recognizes that learning occurs best in the context of performing problem-oriented tasks analogous to those encountered in real work environments. Active, inquiry-based learning should therefore be a major component of the learning environment.
There are many instructional techniques teachers can use to situate learning, such as case-based or problem-based instruction and simulations. However, as mentioned in Chapter 6, care must be taken with all of these approaches to provide multiple opportunities for students to engage in activities in which the same concept is at work; otherwise learning could become overly contextualized, and students might not be able to discern its usefulness in solving different types of problems. The IB program recognizes this: “Students need to be exposed repeatedly to the application of basic concepts to new situations. This can be done through examples used in the classroom, by homework assignments which provide a variety of appropriate situations requiring skills beyond recall of information, and by tests and examinations which use questions similar to those used in the IB examination” (IBO, 1999a, p. 401). There are two primary reasons, however, why multicontext learning is unlikely to occur in AP and IB classrooms: there is a great deal of material to cover in preparation for the examinations, and the AP and IB assessments do not measure students’ understanding of underlying principles in unfamiliar contexts frequently enough.
In the real world, problems are not organized neatly into discipline-specific packages. Thus from the situative perspective, advanced study teachers should provide opportunities for students to explore the interdisciplinary connections relevant to complex tasks. Doing so not only makes learning more interesting and motivating, but also develops students’ capacity to generalize their learning. In this regard, the mathematics panel concludes:
The AP program does not make a sufficient attempt to connect calculus with other fields in a realistic way. There is a tendency to use applications of rather ritualistic and formulaic kinds, and of limited difficulty. The test concentrates on a few prescribed applications (e.g., determination of volumes) or gives applications that consist largely of interpretation of symbols or computations in a new context. Its problems have a “whiff” of application, but they are often jarringly unreal at a deeper level.17
See the panel’s report at www.nap.edu/catalog/10129.html.
While the IB external assessment lacks realistic connections to other disciplines, the IB portfolio, which was added to the mathematics program in 2000, includes a modeling exercise that emphasizes interdisciplinary connections with science. The committee and the mathematics panel applaud this new emphasis and encourage the IBO to expand this emphasis to the external assessment. The IB science courses include interdisciplinary topics in some of the topic options and in the required Group 4 interdisciplinary project (see Chapter 4, this volume, for more information about the IB Group 4 Project). In addition, the selection of curricular options in IB experimental science courses is based on resources that are available locally. For example, an IB chemistry curriculum in a school with access to a university’s analytical laboratory might include higher physical organic chemistry or modern analytical chemistry. Similarly, a curriculum in a school located in an area where there are chemical manufacturing plants could include the Chemical Industries option. Thus students develop knowledge that is situated in the relationships, practices, and tools of a community. The AP science curricula do not encourage links to other science disciplines, probably as a result of the independent development of each AP course.
The laboratory components of AP and IB science courses can provide excellent opportunities for teachers to situate learning in activities that reflect the kinds of thinking and problem solving in which scientists engage. The physics panel notes that experimental work in advanced courses should provide experience with the ways scientists use experiments, both for gathering data to build theoretical models and for exploring the applicability of these models to new situations. Although both the AP and IB programs emphasize the central role laboratory experiences should play in students’ learning, the science panels note that the potential for helping students learn how science is conducted is not realized to an optimal extent in either program. For example, the biology panel notes that the AP required biology laboratories tend to be more “cookbook” (i.e., a set of prescribed procedures) than inquiry-based and consequently do not replicate the ways scientists work. The physics panel is similarly concerned about the “cookbook” nature of many physics laboratories commonly used in schools. In these laboratories, students do not have adequate opportunities to make the kinds of decisions scientists make from the conception of an experiment to the critical review of its results.
Another aspect of situated cognition as related to laboratory work is the ability to use instrumentation. The science panels note that skilled use of instrumentation is an important aspect of the context in which scientific discovery occurs. Through the use of authentic activities, including laboratory work and the skilled use of appropriate instrumentation, the situated nature of learning is respected. Students are less likely to memorize words or phrases that hold no meaning except for helping them obtain high exam
scores or grades. Instead, learning can be embedded in more authentic contexts and realistic activities, using tools appropriate to the subject. The biology panel notes that the AP biology laboratory exercises do not require much instrumentation. The physics panel indicates that neither AP nor IB curricula reflect the growing use of computer modeling for the study of physics.
IB science and mathematics assessments, as a function of the internal assessment18 and portfolio19 components, respect the situated nature of learning better than do AP assessments.20 This is because the AP examinations rely exclusively on a single assessment that uses only multiple-choice and free-response questions to measure student achievement. The chemistry panel contends that paper-and-pencil assessments, such as the AP examinations, are not sufficient to measure students’ understanding of laboratory methods or interpretation of laboratory data. The biology and physics panels concur. The mathematics panel, while appreciative of the portfolio component of the IB mathematics courses, does not believe the portfolio alone eliminates the need to include on the final examinations questions that require students to use their knowledge and skills to solve real and unfamiliar problems.
This chapter has presented an analysis of programs for advanced study through the lens of research on learning, as summarized in Chapter 6. The focus has been on the themes of conceptual learning, prior knowledge, differences among learners, motivation, learning communities, and the context of learning. In general, although the programs have important strengths, they do not yet effectively utilize what is known about how people learn. In many instances, they insufficiently emphasize conceptual learning that uses inquiry-based methods, nor do they adequately take into account the importance of prior knowledge (including student preparation in earlier courses). Some of the courses are too broad to allow mastery. Most could do more to help students gain the ability to apply their learning in unfamiliar situations.
The internal assessments in the experimental sciences are focused on the candidates’ skills in laboratory investigation, which include planning, data collection and processing, evaluation of procedures and results, and manipulative skills and personal skills, which include working with a team (IBO, 2001a, 2001b, 2001c, pp. 20–22).
The purpose of the mathematics portfolio is to “provide candidates with opportunities to be rewarded for mathematics carried out under ordinary conditions, that is, without the time limitations and stress associated with written examinations” (IBO, 1998b, p. 47).
As discussed in Chapter 4, the IB internal assessment allows teachers to evaluate students as they use their knowledge and skills to solve real problems in settings other than testing.
Although the analysis here has included examples from the four disciplines on which this study is focused, the discussion has been fairly general. Readers who are interested in one of the disciplines in particular will find an in-depth assessment in the corresponding panel report. Readers are encouraged to peruse these reports individually or to examine the summary of their findings presented in Appendix A.
Chapter 9 examines the AP and IB programs from a different but related point of view: the four elements of educational programs—curriculum, instructional methods, ongoing and end-of-course assessments, and opportunities for teacher preparation and professional development. The discussion in that chapter considers the extent to which these elements of the AP and IB programs nurture deep conceptual understanding among students—the committee’s view of the primary goal for advanced study.