with the purpose of the program, AP courses are designed by a development committee comprised of university faculty, mathematicians, and master calculus teachers to be equivalent to many introductory college calculus courses with respect to the range and depth of topics covered, the kinds of textbooks used, the kinds work done by students, and the time and effort required of students (College Entrance Examination Board, 1999a). This approach ensures that students have a smooth transition to college, but also means that any shortcomings in college calculus are likely to be transferred to advanced high school students.
AP courses are designed to represent generic introductorylevel college courses. The determination of the content to include in AP calculus courses is based on information gathered from responses to an annual survey of college and university departments that offer general calculus courses in their institutions. It should be noted that the response rate to the survey is rather low (30–40 percent), calling into question the consistency of survey results with actual course syllabi in individual schools and in different types of institutions.
Each year the College Board publishes the Advanced Placement Program Course Description: Calculus, AB and BC (often referred to as the acorn book). This booklet includes a topical outline for the two calculus courses,^{14} sample examinations, and information about how previous AP mathematics examinations have been graded (see for example, College Entrance Examination Board, 1992; 1996b; 1999b). Although the booklet is published annually, changes from year to year are usually small, and sometimes there are no changes.^{15}
There are no AP calculus curricula in the true sense of the word; there are only topic outlines. The topic outlines are intended to indicate the scope of the courses, but they do not necessarily suggest the order in which the topics are to be taught, nor do they specify any particular pedagogy.^{16} Teachers are free to determine the order in which topics are introduced, and they are also encouraged to enrich their courses with additional topics as appropriate. Thus as implemented, the curricula for AP courses can vary from school to school in both sequence and emphasis. The resulting curricula are often more a function of the decisions made by individual teachers than decisions made by a centralized curriculum development group.
The topical outline for AP Calculus BC includes all of the topics from AP Calculus AB plus additional topics (see Appendices B and C, respectively, for the AB and BC topic outlines for May 2000/May 2001). AB and BC students are expected to understand topics common to both courses at the same conceptual level. Both AB and BC calculus are designed to be taught over one school year.^{17} Decisions about which AP calculus course an individual student should
select are generally based on both the school’s offerings and the individual’s abilities, achievements, and mastery of the prerequisite material.
The International Baccalaureate Diploma Programme
The International Baccalaureate Diploma Programme, created in 1968, was originally designed to serve a geographically mobile population, primarily children of diplomats, who relocate frequently but do not want to lose the continuity of their educational experiences. Thus, the program incorporates elements of several national systems without being based on any particular one. IB courses do not reflect the content and structure of college courses, but rather what is believed by the program developers to be the full range of content knowledge and skills that should be mastered by wellprepared students prior to study at the university level. Thus the courses are developed not to replicate college courses, but to reflect an international conception of a welldesigned college preparatory program of study among a small group of master teachers and discipline experts, who design the courses using both their expertise and information gathered systematically from IB teachers in every IB school. A curriculum review committee with international membership is responsible for articulating, implementing, and maintaining the vision in each subject area. The mathematics committee is also responsible for specifying the presumed knowledge and skills (prerequisites) for each of the four IB mathematics courses.
The IB mathematics curriculum is substantially broader than the AP calculus curricula. IB offerings include four mathematics courses: Mathematical Studies SL, Mathematical Methods SL, Mathematics HL, and Further Mathematics SL. Each course has been designed to meet the needs of a particular group of students, and schools and teachers are encouraged to exercise care in selecting the appropriate course for each student.^{18} Courses that are designated as SL require a minimum of 150 hours of teacher contact time, while those designated as HL require a minimum of 240 hours of teacher contact time. As noted earlier, it is the hours of contact time, not necessarily the level of course difficulty, that determines whether a course is designated as SL or HL (International Baccalaureate Organisation, 1993). For example, Further Mathematics SL is the most advanced mathematics course in the IB program. Both SL and HL courses include a common core of compulsory topics (105 hours for the SL courses and 195 hours for the HL courses), as well as one or more optional topics that are selected by individual teachers for their classes (45 hours for both HL and SL courses). Because the IB program has an international mission, this flexibility in curriculum enables schools to satisfy higher education entrance requirements in many countries.
The IB Mathematics HL course is taught over two years and includes as core material a substantial amount of calculus, roughly equivalent to the AP Calculus AB course, as well as substantial treatment of probability, algebra and trigonometry, complex numbers, mathematical induction, vectors, and matrices. In addition, the HL curriculum has optional units, one of which is taught in addition to the common core. There are units on abstract algebra, graphs and trees,
statistics, analysis and approximation, and Euclidean geometry and conic sections. With the analysis and approximation option, the calculus coverage becomes roughly equivalent to that of AP Calculus BC. With the statistics option, the statistics coverage becomes roughly equivalent to the purely mathematical parts of AP Statistics (note, however, that AP Statistics devotes substantial attention to the nonmathematical aspects of statistics).
The IB program guides for the mathematics courses present the common aims and objectives for all Group 5 (mathematics) courses (see Box 21). The guides also provide a syllabus outline for each course (see Boxes 22 through 24). In addition to the syllabus outline, IB teachers are provided with a detailed syllabus that defines more explicitly what will be covered on the final examinations. The detailed syllabi include estimated teaching hours for each topic, but do not recommend a sequence for the presentation of topics (see Appendix D for an example of a detailed syllabus). IB mathematics program guides provide general guidance on instruction and offer specific suggestions regarding instructional strategies. The teaching notes also include suggestions for linking content to help students see connections, such as linking the study of the second derivative in the further calculus option to the study of exponents and logarithms in the core content. Teachers use the guides as the basis for determining the structure of their own curriculum. Thus as implemented, the curricula for IB courses, like those for AP courses, can vary from school to school in sequence and emphasis. Little is known about the nature of these variations and their effects on student learning and achievement.
Finding: The AP and IB programs are both designed to meet the educational needs of highly motivated and wellprepared students, but the origins, goals, purposes, missions, organizations, and structures of the two programs are very different. These differences contribute to variations in the educational expectations, opportunities, and experiences of students and teachers participating in the two programs.
AP AND IB TEST DEVELOPMENT
AP Program
AP calculus examinations are designed by a development committee comprised of high school teachers and university faculty in consultation with statisticians and psychometricians who attempt to create examinations that meet accepted standards for technical quality (American Educational research Association [AERA]/American Psychological Association [APA]/National Council on Measurement in Education [NCME], 1999). The development process incorporates the judgments of both disciplinary and psychometric experts. The AP calculus examinations are timed, with about 50 percent of the total time devoted to multiplechoice questions and the rest to freeresponse problemsolving questions.^{19} The development committees are responsible for deciding the general content of the examination and the ability level to be tested.
Content specifications for AP calculus examinations are determined at the same time as the topic outlines for the courses are developed. Examinations are constructed using the topic
percentages from the AP course descriptions as a guideline for the distribution of questions. The development committee helps write and review test questions, as well as materials (including the AP course descriptions) that are distributed to schools. They work closely with College Board and Educational Testing Service (ETS) content specialists and psychometricians to ensure that the examination scores will mean the same thing from year to year and from student to student (College Entrance Examination Board, 1999a).
For optimal measurement, the development committee endeavors to design a multiplechoice section such that the average raw score is between 40 and 60 percent of the maximum possible raw score. Questions at varying levels of difficulty are included. Some previously administered questions are included to link the current form of the examination to previous forms, thus maintaining comparability from year to year and examination to examination. The committees write, select, review, and refine freeresponse questions. One important aspect of test development is determining which item type and format are best for assessing a given topic or skill area. AP development committee members work with AP content experts and ETS statisticians in making this determination (College Entrance Examination Board, 1999). However, the College Board has not conducted any systematic research to determine whether particular test items actually measure the cognitive processes they purport to measure.
IB Program
IB mathematics assessments have two components—external and internal. The external assessment consists of a written examination that is administered internationally over a period of two days in May of each year (in November for splitsession schools). The examinations test knowledge of both the core and optional topics. The internal assessment is a portfolio that represents the teacher’s formative assessment of students’ practical work judged against established assessment criteria.^{20} This component is conducted by teachers within the school environment and is moderated externally by the IBO. The Mathematical Methods SL portfolio consists of five assignments, based on different areas of the syllabus, representing the following three activities: mathematical investigation, extended closedproblem solving, and mathematical modeling. The Mathematics HL and Further Mathematics SL courses add mathematical research to the list of activities from which students and teachers can draw portfolio activities.
The course guide, which includes a detailed description of the content, skills, and understanding students are expected to master during the course, serves as the test blueprint. Amplifications and exclusions are provided for each topic. They provide more explicit information on specific subtopics and help define what is and is not required in terms of preparing students for the examination (see, for example, International Baccalaureate Organisation, 1997; 1998a; 1998b; 1998c).
New questions are written for each external examination. Unlike the College Board, the IBO does not develop its examinations in consultation with psychometricians. Rather, it relies on the expertise of its senior examining team to prepare a different form of the examination that is approximately of the same level of difficulty as previous examinations. Test items are not
field tested prior to administration, and reliability is not calculated. Items are not repeated from one administration to another. After each examination session, a sample of candidate responses is closely scrutinized to determine whether they are in line with expectations for each question. In addition, the IBO solicits comments from teachers about the suitability of the examination papers in achieving the examination’s objectives. Information gained in this manner is used in the preparation of future examinations. Like the College Board, the IBO does not conduct systematic research to determine whether particular test items actually measure the cognitive processes they are intended to measure (Pook, 2001).
3
Analysis of AP and IB Curricula and Assessments
To evaluate the appropriateness of the AP calculus and IB mathematics programs for advanced high school students, the panel considered the degree to which the curricula and assessments of each emphasize four areas listed below. These areas represent the kind of mathematical learning the panel believes advanced mathematics students should experience in their high school calculus course:

Technical skill, including flexibility with symbolic manipulation, graphical representation, numerical approximation, and function notation

Conceptual understanding, including, for example, the derivative as a rate of change

Theory, including precise definitions, carefully reasoned mathematical arguments, and mathematical rigor

Applications and modeling, including situations in which students must set up the model (for example, choose a formula that represents a real situation)
CURRICULUM
AP Program
In the panel’s opinion, the topics on the AP syllabi are appropriate and well connected. A strength of the AP curricula and examinations is their focus on reasoning that integrates graphical, algebraic, numerical, and modeling viewpoints. In its current formulation, AP calculus pays careful attention to the central concept of function and to connections among the common ways (numerical, graphical, analytical, and verbal) of representing functions. There is likewise careful attention to developing the main concepts of differentiation and integration, including several interpretations of and applications for each. However, because AP calculus courses are designed to be generalpurpose courses, the applications to realworld problems and situations tend to be limited to those that are most manageable, both conceptually and technically. There are relatively few applications requiring deeper investigation that might prove more memorable and provide better opportunities for interpretation and modeling.
The panel views the content of the AP courses as sufficient, given the assumption that the preparatory coursework provides a solid foundation for calculus. There is considerable
anecdotal evidence, however, that students intending to take calculus are rushed through prerequisite courses without learning the material well enough. Mathematical sophistication takes time and should not be confused with knowledge of a catalog of mathematical facts and techniques. Additionally, because the AP examinations create the focus for the curriculum, and they do so quite effectively, material not on the AP assessments tends not to be taught either in calculus or in the prerequisite courses.
Further, particularly able AP students could profit from an AP course enriched by modeling activities and more attention to proof. In this sense, the panel sees the AP calculus programs as minimal rather than maximal courses for advanced study. Teachers should be encouraged to challenge their students with problems that may go beyond the scope of the syllabi, teaching a course that is more demanding than the tests. Doing so would result in students having a richer learning experience. For the preparation of mathematicians, the panel views the slight attention paid to proof by the AP courses as a deficiency.
IB Program
The panel found that a broad factual base of information is provided by the IB curricula for Mathematical Methods SL and Mathematics HL. The curricula encompass an introduction to elementary calculus (similar to the AP program’s Calculus AB course) and additional areas of study selected by the teacher from among available options. However, the small number of hours suggested for study of each topic (50 hours for the introduction to calculus for Mathematics HL and 20 hours for Mathematical Methods SL) leads to the concern that students study each topic only at a procedural level. As compared with the AP program, the panel views the calculus portions of the IB curricula as “very traditional.” For example, question #6 on the Mathematics HL exam, Paper 1, November 1999, reads:
The area between the graph of y = e^{x} and the xaxis from x = 0 to x = k (k > 0) is rotated through 360° about the xaxis. Find, in terms of k and e, the volume of the solid generated.
The panel views the branching away from calculus as a positive aspect of the IB program. Mathematics HL coupled with Further Mathematics SL requires students to show competence in elementary calculus, as well as an introductory understanding of probability, matrices and vectors, abstract algebra, graphs and trees, statistics, analysis and approximation, and either mathematical foundations or mathematical proof. While it would be wonderful for students to have an understanding of all these topics, the panel is concerned that the breadth of the curricula may force somewhat superficial coverage. The panel believes that a more beneficial alternative would be to help students develop a deeper and richer understanding of one or two of the mathematical topics and their applications.
The advanced options present teachers with an enormous challenge in trying to cover advanced material in a highly limited time frame. The abstract algebra option of Mathematics HL provides an example. In this option, both the extent and depth of the study of the subject are quite limited, necessarily so in view of the time available and the prior mathematical experience
of the candidates. Therefore, the study of abstract algebra is confined to basic definitions and a few elementary properties. The focus is formal; no significant applications of groups are given. The difficult part of learning about groups is not the formal definitions and elementary properties, but understanding why the ideas are important. The panel believes it is more useful for students to study deeply topics that are closer to the high school curriculum, such as the study of solutions of polynomial equations (with emphasis on 3^{rd} and 4^{th} order equations), the use of transformations in plane geometry, or elementary number theory. Students with these topics in their background will be better prepared to appreciate later the significance of group theory.
The IB curricula provide a strong foundation in the key concepts of college preparatory mathematics. The syllabi and the assessments, however, do not appear to promote the relationships among the mathematical concepts learned. The syllabus for Further Mathematics SL states: “At this level, less emphasis should be put on departmentalizing the various topics and more emphasis should be given to the links between topics. For example, the knowledge of complex numbers, recurrence relations and Newton’s method could be brought together in a brief introduction to fractals.” However, the assessment materials do not support this linkage, but are strictly departmentalized. Waiting until the final year of advanced study to emphasize the links among topics with the portfolio assessment is too little, too late.
Finding: The AP curricula are largely sound. The recently revised syllabi with more emphasis on conceptual understanding have significantly improved the program, although further change in this direction is desirable. The panel also believes the focus on reasoning should be increased.
Finding: The IB curricula are largely sound. The portfolio requirement, with its emphasis on applications of mathematics, is likely to introduce a focus on modeling that will benefit IB students. However, the calculus sections of the syllabi do not place enough emphasis on conceptual understanding. The panel also has some concern that the breadth of the curricula, although an attractive feature of the program, could lead to superficial learning.
Finding: AP and IB curricula are designed to prepare students for successful performance on endofcourse examinations. The content and structure of the examinations, therefore, have a profound effect on what is taught and how it is taught in AP and IB classrooms.
ASSESSMENTS
Research indicates that the problems students solve as part of their classes have a significant effect on the strength of the background they acquire. Since the types of problems that are on the endofcourse assessments largely determine the structure and content of classroom and homework problems, evaluation of the effectiveness of the programs must include an evaluation of the problems on the examinations.
AP Program
On the basis of the four areas of emphasis described above, the panel believes the AP examinations should include questions that emphasize the following:

Complex problems that may require significant technical skills

Problems that probe students’ conceptual understanding and take into account what research tells us about common misconceptions

Problems that require mathematical reasoning, clear exposition, and the ability to write precise mathematical statements

Problems in which students must construct a mathematical model from a verbal description
The panel reviewed 5 years of publicly released AP Calculus AB and AP Calculus BC examinations, related course descriptions, teachers’ guides, and scoring rubrics (College Entance Examination Board, 1989; 1993; 1997a; 1998a; 1999b; 1999d; 1999e). This review focused primarily on the examinations that were administered after 1998, when the revised AP calculus courses were fully implemented. The following findings with regard to the AP calculus examinations (both AB and BC) emerged from the panel’s review.
Technical Skill
The goals of the AP program explicitly support the development of strong computational skills, both by hand and using technology. There is a reasonable balance of symbolic and graphical problems on AP calculus examinations. However, there are not many numerical problems, and more would be an improvement. The division of the examinations into two parts—one that allows the use of a graphing calculator and one that does not—is useful, ensuring that students acquire skills with both symbols and numerical approximations.
The need to cover in one 1¾hour multiplechoice test all of the material from two semesters of calculus makes the tests broad with limited depth; thus, for example, only a modest amount of time is available for solving each problem. As a result, the chain rule is tested at a superficial level, and only routine integrationbyparts questions are given. Seldom do Taylor series questions go beyond the basics, for example, rarely to a composite series such as the first five terms of sin (x^{2})e^{x}.
Student achievement on the examinations confirms that the goals for technical skills are not being met at a high level. The College Board’s analysis of grades earned on the 1998 AB examination indicated that fewer than 15 percent of the students who earned a 3 on the examination obtained a score above 23 out of 45 on the multiplechoice questions.^{21} Most of these multiplechoice questions tested very basic calculus knowledge. The panel speculates that the underlying reason students could not answer these questions was poor algebra skills rather
than a lack of understanding of the necessary calculus. For example, on the 1998 AB exam, 75 percent of students who earned a grade of 3 were able to evaluate the integral in question #3, but only 38 percent were successful when they had to separate the fraction and then integrate in question #7,
It also appears that AP students taking the examination had not had enough practice solving nonstandard problems. For instance, on the 1998 AB exam, 75 percent of the students were able to differentiate the “harder” implicit function in question #6:
However, only 36 percent were able to figure out that the second derivative of a linear function is zero and then integrate in question #11:
Question #3 of the freeresponse section of the 1998 AB examination was a problem about motion in which information was given in a graph and table, rather than the more usual formula. The score on this question (average 2.9 out of 9) was much lower than the scores on the two multiplechoice questions in which the position was given by a formula (questions #14 and #24, with 94 percent and 52 percent correct, respectively). This result suggests that although students had learned how to answer questions like #14 and #24, they had not fully understood the concept. The panel includes these data to illustrate its contention that some AP students can do very well on AP calculus examinations without understanding the underlying concepts. Further analysis of this kind is necessary and should be part of a systematic program of validity research.
Conceptual Understanding
In recent changes to the syllabi for AP calculus, the emphasis on calculus as a collection of techniques has decreased, and the emphasis on conceptual understanding of fundamental principles has increased. The panel applauds this change. The current syllabi and examinations acknowledge the importance of techniques, but do not allow them to overwhelm the course.
AP assessment items emphasize the major concepts of calculus (e.g., functions, derivative, integral, the Fundamental Theorem). Furthermore, the items incorporate current perspectives on learning by emphasizing multiple representations (i.e., the rule of four) and by placing some items in realworld contexts. However, there are missed opportunities to include questions that address common student misconceptions identified by educational research (for example, in understanding the derivative, slope fields, and functions). In addition, the test does not place enough emphasis on ensuring that students are fluent with the symbolic language.
AP examination problems also sometimes reinforce the idea that mathematics is essentially procedural. The questions may be divided into so many parts that they become exercises in following instructions rather than in choosing a strategy to solve a problem. For example, in 1998, the following problem was given on the AP Calculus BC exam:
6. A particle moves along the curve defined by the equation y = x^{3}– 3x. The xcoordinate of the particle, x(t), satisfies the equation
for t ≥ 0 with initial condition x(0) = – 4.

Find x(t) in terms of t.

Find in terms of t.

Find the location and speed of the particle at time t = 4.
Note that parts (a) and (b) of this problem lead the student to the solution of part (c). Thus, solving this problem successfully involves executing each step correctly and seeing the connections among them. But suppose parts (a) and (b) were omitted. The problem would then take on a very different character: it would test whether a student can find a path to the solution rather than execute steps. The most talented students—those in BC Calculus—should have more practice with problems in which they must design a method or solution.
Theory and Proof
Theoretical questions do not appear explicitly on AP examinations; there is little emphasis on proof in the AP syllabi. In the last decade, however, the emphasis on conceptual understanding has increased in the examinations. For example, in the 1998 syllabi, the movement away from a “laundry list” of topics encouraged students to think about the key ideas rather than memorize problem types. The panel applauds this change. Indeed, further change in this direction would be welcome. The panel recommends the inclusion of one or two items that assess students’ understanding of the definition of the derivative. By this we mean including problems that assess understanding of the meaning of the definition of the derivative, not the ability to compute using the definition.
Applications and Modeling
The AP program does not make a sufficient attempt to connect calculus with other fields in a realistic way. There is a tendency to use applications of rather ritualistic and formulaic kinds, and of limited difficulty. The test concentrates on a few prescribed applications (e.g., calculation of volumes) or gives applications that consist largely of interpretation of symbols or