Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
6 Indicators of Teaching Quality TEACHERS AS KEY ACTORS In its earlier report (Ralzen and Jones, 1985), the committee dis- cussed potential indicators relating both to the quantity and quality of teachers responsible for science and mathematics instruction. One of the major conclusions in that report is that "the construction of . . . an indicator on teacher demand and supply is at present not feasible at the national level because of the lack of a meaningful common measure of qualification" (p. 71~. At the state and local levels, stan- dards on teacher quality vary among school districts within a state and among schools within a district- appropriately so, if the schools or districts serve student populations with different needs (Wise et al., 1987~. Yet a panel, set up under the committee's aegis to de- velop better models for estimating teacher demand and supply, is stressing "that satisfactory models of supply and demand for science and mathematics teachers must be specific regarding teacher quali- fications" (Pane! on Statistics on Supply and Demand for Precollege Science and Mathematics Teachers, 1987:58~. Obviously, questions concerning the adequacy of instruction in science and mathematics cannot be answered until some measures of teaching effectiveness are developed and found acceptable. What constitutes effective teaching of mathematics or science? On what should indicators of teaching effectiveness focus-character 90
INDICATORS OF TEACHING QUALITY 91 istics of the teachers themselves? Measures of what teachers do in the classroom? In attempting to resolve this issue, the committee devoted considerable attention to the research literatures on the char- acteristics of effective teachers and on the determinants of effective teaching. We found strong research support for parents' conviction that teachers matter. This support comes from studies showing clearly that children enrolled in different schools, and even in dif- ferent classrooms within the same school, learn different amounts during the school year (Hanushek, 1972; Murnane, 1975; Armor et al., 1976~. While these studies by themselves do not demonstrate that differences among teachers alone account for why more learning takes place in some classrooms than in others, it is reasonable to infer from these and other studies that differences among teachers are one important factor contributing to these differences in student - earnmg. The evidence that teachers matter led us to turn to the studies that have attempted the more difficult research task of exploring the specific characteristics of teachers and the specific teacher behaviors that are related to high student achievement. Unfortunately, we concluded that such studies (whether in traditions known as input- output studies or process-product studies) do not provide significant guidance for the development of indicators of effective mathematics and science teaching. In part this may be the case because the studies are largely based on current conceptions of teaching that emphasize the learning of procedural skills rather than the larger vision of the teacher's role set out by, for example, the Holmes Group Consortium (1984) and the Carnegie Forum on Education and the Economy (1986~. It is conceivable that the research results would be different if student scores on tests of higher-order thinking skills were used to measure teaching effectiveness. This hypothesis has not been tested, however, since all existing studies have measured teacher effectiveness by student scores on multiple-choice tests that, as Chapter 4 on learning assessment explains, do not measure the full range of higher-order thinking skills. The following section explains why the results of input-output studies and process-product studies do not provide guidance for indicator development.
92 INDICATORS OF SCIENCE AND MARTHA TICS EDUCATION Fir~clings from the Literature One type of study, called educational production functions or input-output studies, has explored the extent to which gains in stu- dent achievement can be explained by information on teachers' de- mographic characteristics, education, test scores, and teaching expe- rience. There are a few relatively consistent findings. For example, teachers with at least three to five years of experience are more effec- tive on average than beginning teachers (Hanushek, 1972; Murnane, 1975; Murnane and Phillips, 1981), and this appears to hold true for science teachers (Druva end Anderson, 1983; Penick and Yager, 1983~. A somewhat less solid finding is that teachers with high scores on tests of verbal ability may be more effective than teachers with Tower scores (Coleman et al., 1966; Hanushek, 1972), although there are exceptions to this pattern (Summers and Wolfe, 1977~. While it is common to focus attention on positive findings, the dominant conclusion from input-output research is that the vast majority of the variables user] to depict teachers, including sex, race, possession of a master's degree, and whether the teacher was an education major as an undergraduate, are not consistently related to teaching effectiveness, whether measured by student gains on standardized achievement tests or by evaluative judgment (see, e.g., Schalock, 1979~. A second type of research, sometimes referred to as process- product studies or studies of teaching electiveness, has examined whether specific actions of teachers are systematically related to teaching effectiveness. In recent years, this research has provided support for the sensible proposition that students' achievement in a specific subject is positively related to the amount of in-cIass time de- votect to instruction in the subject, as noted in the preceding chapter and the committee's earlier report. The research also supports the proposition that just as important as the amount of time allocated to mathematics, science, or other subjects is how the time is used (Weber, 1978; Evertson et al., 1980; Good, 1983~. This has led to studies of how best to use instructional time, including how to de- velop lessons and how to manage the classroom. These studies have produced insights that are helpful in teacher education, for exam- ple, by demonstrating the importance of presenting all students with challenging work and expecting them to complete it, and making smooth transitions from one activity to another (Good and Grouws, 1979; Brophy and Good, 1986~.
INDICATORS OF TEACHING QUALITY 93 If the process-product research had found that teachers who de- velop lessons effectively and manage their classrooms well do so by engaging in particular well-defined actions, then these actions could provide the basis for indicators of teaching effectiveness. Observa- tional techniques could be used to record the extent to which teachers of mathematics and science employ these superior techniques. In fact, such a mapping of concepts that characterize effective teaching to well-defined teaching actions has not been possible, however. Con- sequently, the process-product literature provides little guidance for the development of indicators of teaching effectiveness. There are at least two reasons for this: first, effective teaching requires carrying out more than one action. Carrying out requisite actions in isolation may not result in effective teaching; and, as a result, observations of the frequency with which teachers carry out a single particular action would not provide the basis for a reliable indicator. Second, the set of particular actions that results in effective teaching may depend on the type of classroom situation the teacher is in. The actions that are most effective would be expected to vary with grade level and with the subject matter and skills being taught. In addition, there is some evidence that effective teaching of children with different characteris- tics and backgrounds requires different sets of actions by the teacher (Cronbach and Snow, 1977; Brophy and Good, 1986~. These com- plexities in mapping concepts to actions would make it very difficult to base reliable indicators of teaching effectiveness on observations of whether teachers carry out specific, well-defined actions (Brophy, 1986~. In summary, review of the research on the determinants of teach- ing electiveness led us to the conclusion that neither input-output studies nor process-product studies provide sure guidance for the development of indicators of the quality of mathematics and science instruction in school. In one sense this is discouraging, because it makes the task of developing reliable indicators of teaching effec- tiveness more difficult. In a different sense, however, the results are encouraging, because they underline the fact that effective teachers cannot be defined merely as individuals with specific demographic characteristics who have earned particular academic degrees, or as people who have been trained to behave in predictable, routinized ways in the classroom. Such definitions obscure the characteristics that effective teachers have in common the skills and attitudes of professionals (Holmes Group Consortium, 1984; Carnegie Forum on
94 INDICATORS OF SCIENCE AND MATHEMATICS EDUCATION Education and the Economy, 1986; Darling-Hammond and Hudson, 1986). The Professional Teacher In recent years, at least 44 states, several major commissions, and the national teachers' unions have moved toward a definition of the professional teacher. The following attributes are generally included in the definition: professional teachers understand the subject matter they teach and its relation to other subjects in the curriculum. They possess a high degree of intellectual curiosity, which is reflected in how they spend their time. Professional teachers also have the desire to help students increase their skills and self-confidence, and they have the skills to achieve these goals, including being able to adapt the curriculum to fit the needs of their students (Good and Weinstein, 1986~. Finally, professional teachers continue to learn new things as they progress through their careers. It is teachers with these attributes that are wanted and needed to provide instruction in mathematics and science. Schools cannot attract and retain professional teachers unless they provide the support that professionals need and can find in other occupations (Darling-Hammond, 1984; Rosenholtz, 1985~. This sup- port includes competitive salaries, opportunities for professional de- velopment, and significant control over the time, space, materials, and curriculum needed to teach effectively (Lightfoot, 1983; Purkey and Smith, 1983~. The committee's recommendations for indicators of the effective- ness of science and mathematics teaching are based on this concep- tion of the professional teacher and the support that the schools must provide to attract and retain such teachers. The rest of the chapter is organized into three categories of information about professionalism in science and mathematics teaching: 1. What are the educational backgrounds and knowledge levels of individuals who teach science and mathematics? 2. How do these individuals spend their time? 3. What are the working conditions for teachers of science and mathematics?
INDICATORS OF TEACHING QUALITY EDUCATIONAL BACKGROUNDS AND LEVELS OF KNOWLEDGE College Education 9s One attribute of professional teachers is that they understand the subjects that they teach. To assess the extent to which the nation's secondary school science and mathematics teachers have adequate subject matter preparation, NSF has sponsored two surveys that have collected information on the education that teachers received in college in the subject matter fields that they teach tWeiss, 1978; Research Triangle Institute, 1985~. More than half the states also collect information on college courses in science and mathematics taken by newly hired teachers. At present, the Center for Educa- tion Statistics of the U.S. Department of Education is considering plans for collecting information on teachers' undergraduate major and minor fields of preparation and, for both secondary and elemen- tary teachers, on the number of college courses taken in mathematics and science and in teaching mathematics and/or science (Darling- Hammond et al., 19863. The premise underlying this sort of survey is that high school physics teachers, for example, who have taken little physics in college are unlikely to have a solid understanding of physics and consequently are unlikely to have the knowledge needed to teach physics well. The committee recognizes that the extraordinary variety of un- dergraduate institutions in the United States that prepare teachers makes it virtually impossible to assess accurately the subject matter preparation of the nation's teachers. Nonetheless, we support con- tinuation of the collection of information on teachers' college courses and degrees because it will provide at least basic information on the preparation of the teachers who teach science and mathematics to dif- ferent types of children in the United States. Moreover, information on changes over time in teacher preparation and in the distribution of teachers among diffferent types of students will provide a sense of direction about the nation's success in staffing all schools with teachers who are well prepared in science and mathematics. The committee does suggest one major change in the data col- lection and reporting method: information on teacher preparation should be collected and reported according to different subgroups of students taking mathematics and science courses, so that the in- formation will be more useful in assessing the distribution of well
96 INDICATORS OF SCIENCE AND MATHEM24 TICS EDUCATION prepared teachers among groups of students with different charac- teristics. For this purpose, data collected on individual students should include gender, race, ethnicity, socioeconomic status, grade level, type of community (urban, suburban, rural), and region or state. Reporting by student subgroups will allow the following types of questions to be addressed: . ~ it, ~ What proportion of the students taking high school physics are taught by teachers who have an undergraduate major or unknot in physics? ~ Is the proportion of black students studying biology with teachers who have an undergraduate major or minor in biology dif- ferent from the percentage of white students studying biology with teachers with the same preparation? ~ What proportion of elementary school students in particular grades and with particular characteristics are taught science by a teacher who has taken at least six college courses in science? What proportion are taught mathematics by a teacher who has taken at least six college courses in mathematics? As the questions indicate, collecting and reporting information on teacher preparation by student subgroups permits one to examine whether the college education of the teachers who teach science and mathematics to students with particular characteristics differs from the college education of teachers teaching children with other characteristics. This strategy supports the focus on equity and access that the committee endorses. It will make it possible to learn whether the teachers with the most substantive college backgrounds are being selected to teach certain categories of students rather than others. This strategy also reduces the problem of how to assess the subject- matter knowledge of teachers teaching both mathematics and science and of high school teachers who teach more than one type of science. Subject-Matter Knowledge It has proven very difficult to establish that teachers with supe- rior subject-matter knowledge are more effective in teaching students than are teachers who have merely an adequate knowledge of the material they teach to students (Byrne, 1983~. For example, the ev- idence on the relation between graduate credits or advanced degrees and electiveness is tenuous (Begle, 1979; Shymanski et al., 1983;
INDICATORS OF TEACHING QUALITY 97 U.S. General Accounting Office, 1984~. Nevertheless, it is reason- able to believe that teachers who have mastered the material that they teach to their students are more effective than teachers who have not mastered this material. Therefore, some appropriate mea- sure of subject-matter knowledge should be used as an indicator of teacher effectiveness, even though agreement on specifics of optimal preparation for teaching a subject at a given grade level or in a particular course remains difficult. For this reason, the committee endorses periodic sample testing of teachers' basic competency in the subject matter they teach. The problem to date has been the development of an appropriate measure. Even if the relationship be- tween subject-matter knowledge and effective teaching of a subject were better understood, there would still be problems with current tests analogous to those discussed in Chapter 4 with respect to tests of student learning. The committee suggests that the tests used to establish basic subject-matter competency of teachers should probe essentially the same domain as the tests used to assess students' mas- tery of science and mathematics. The results of this testing should be reported in summary statistical distributions rather than as in- dividually identifiable scores, since the purpose is to establish an indicator of teachers' knowledge of the subject matter being taught, not to evaluate individuals in order to make decisions on hiring, promotions, or pay. In implementing the committee's recommendation to test teach- ers' basic subject-matter competence in science and mathematics, it will be important to retain linkages not only to changes in the dis- ciplines themselves but also to changes in science and mathematics curricula and in the content and form of student tests. In Chapter 4, the committee recommends that new tests be designed that more adequately assess students' higher-order thinking skills than existing tests and that are more closely tied to exemplary curricula. As the tests used to assess students' science and mathematics knowledge and skills change, so should the tests used to assess teachers' basic subject-matter competence. In this manner, any systematic deficien- cies can be uncovered in teachers' mastery of the changing material on which their students are being assessed. As with subject-matter preparation and for the same reason, the results of the teacher tests should be reported by student subgroup. Reporting the percentage of students with particular characteristics who are taught mathematics or science by teachers who possess basic
98 INDICATORS OF SCIENCE AND MATHEMATICS EDUCATION subject-matter competence supports the committee's desired focus on equity concerns in the development of useful indicators of the quality of science and mathematics education. Clearly, a measure of teachers' mastery of the same knowledge and skills on which their students are tested provides only a mod- est amount of information about their subject-matter competence. Even so, results of such tests may show that not all teachers have mastered the basic knowledge and skills. It is important to recognize that not all the reasons one might posit for this possible outcome blame teachers. For one thing, school district responses to declining enrollments during the 1970s- and in some parts of the country, dur- ing the 1980s led to many teachers being reassigned from such fields as history or social science, in which there was a surplus of teachers, to such fields as mathematics and the physical sciences, in which there were vacancies (Darling-Hammond, 1984; Flowers, 1984~. Of- ten the preparation of these teachers in science or mathematics was very limited and outdated. Unfortunately, not all mandated changes in curriculum or in skill emphasis are accompanied by adequate in- service programs for the teachers who are required to implement the new ideas. Future shortages of qualified mathematics and science teachers may continue to induce some school districts to staff science and mathematics courses with teachers with little preparation or knowledge in these subject areas. It is important to reiterate that the reason we recommend test- ing teachers' basic subject-matter competency in science and math- ematics is to assess the extent to which students are taught science and mathematics by teachers who have mastered the knowledge and skills they teach, not to denigrate the ability or aptitudes of par- ticular teachers. Thus, although the committee advocates collecting information on the characteristics and backgrounds of the students who are taught by the teachers sampled for testing, we do not sug- gest that comparable data be collected on the individual teachers being tested. However, since the ultimate goal is to provide teachers competent in science and mathematics to all students, individual states may wish to collect demographic data on teachers in order to examine the question of whether out-of-field teacher placement, ac- cess to in-service opportunities, and high-quality teacher preparation programs are evenly distributed among different population groups of teachers.
INDICATORS OF TEA CLING QUALITY TABLE 6-1 Suggested Schedule for Assessing Subject-Matter Knowledge of Teachers of Science and Mathematics Teacher Survey Year (elem. and sec.) New Hires (sec.) Survey Foilow-Up 1988 X 1989 X 1990 X 1991 X 1992 X 1993 X 1994 X 1995 X 1996 X Sampling Strategy 99 The subject-matter preparation and subject-matter knowledge of a random sample of the nation's science and mathematics teachers ought to be assessed at least every four years. The sample should be drawn so that it is possible to discern trends not only in the prepa- ration and subject-matter knowledge of the nation's science and mathematics teachers as a whole, but also trends in the preparation and knowledge of such critical subsets of teachers as those teach- ing particular sciences, those teaching remedial mathematics, those teaching science in the elementary schools, those teaching minority group children, and those teaching special education students. In addition, the subject-matter preparation and subject-matter knowledge of a sample of newly hired secondary school science and mathematics teachers should be assessed every two years, with a follow-up survey administered one year after the original survey to determine whether the new hires are still teaching and, if not, why they left teaching (see Table 6-1 for suggested survey scheduler. Newly hired teachers in this context are defined as those teachers employed to teach mathematics or science within the last year who did not teach mathematics or science in the year prior to this em- ployment. There are three reasons to focus particular attention on newly hired teachers. First, collecting information every two years on the college backgrounds and subject-matter knowledge levels of this group is one way to provide early warning of incipient changes in the backgrounds and skills of the profession. Second, the newly hired
100 INDICATORS OF SCIENCE AND MATHEMATICS EDUCATION teachers are the most likely to leave teaching (Charters, 1970; Green- berg and McCall, 1974; Murnane, 1981~. By learning which newly hired teachers leave teaching after one year, it is possible to examine whether those who leave have better preparation and subject-matter knowledge than those who stay, as one study of North Carolina teach- ers has found (Schiechty and Vance, 1983~. Moreover, by learning what teachers who left did in the year after they left, it may be possible to make inferences concerning whether changes in salaries or working conditions might have induced these teachers to stay in the classroom. A third reason to study newly hired teachers is that the resulting information could shed light on sources of supply of new teachers in mathematics and science. For example, recent stud- ies of newly hired science and mathematics teachers in Connecticut (Connecticut State Department of Education, 1985), Illinois (Illinois State Board of Education, 1983), and New York (New York State Ed- ucation Department, 1983) indicate that the majority were teachers with previous teaching experience- members of the much discussed but elusive "reserve pool" of individuals who are certified to teach but are not currently employed by any school system. Very little is known about the size of the reserve pool or about the backgrounds and skills of individuals in this pool. In fact, the U.S. Department of Education's current mode! for national teacher supply and demand does not even acknowledge the reserve pool as a source of supply (Pane] on Statistics on Supply and Demand for Precollege Science and Mathematics Teachers, 1987~. By collecting information biennially on the backgrounds and subject-matter knowledge of newly hired mathematics and science teachers and determining whether these teachers are still in the classroom in the next year, it would be possible to learn: . whether the reserve pool is a greater source of supply of science and mathematics teachers in some parts of the country than others; ~ whether the significance of the reserve pool as a source of supply of science and mathematics teachers changes over time; whether the educational backgrounds and knowledge levels of the newly hired coming from the reserve pool differ from the educational backgrounds and skills of the newly hired coming directly from teacher education programs; and 0 whether the newly hired teachers coming from the reserve pool are more or less likely to remain in the classroom than those coming directly from teacher education programs.
INDICATORS OF TEACHING QUALITY Needed Research 101 More research is needed on the impact that teachers' knowledge of science and mathematics has on their effectiveness in teaching these subjects to students. Current findings give no clear indica- tion on the optimal breadth and depth of preparation for a given teaching assignment, whether an elementary grade or a secondary school course. Results of studies attempting to relate measures of teacher knowledge to measures of student achievement have been mixed (Summers and Wolfe, 1977; Begle, 1979; Byrne, 1983; Druva and Anderson, 1983~. It is particularly important to learn whether new directions in curricula, especially increased attention to the de- velopment of students' analytical and critical thinking skills, will -increase the importance of teachers' subject-matter knowledge. To address this question, teachers' effectiveness must be measured by students' scores on tests that assess these higher-order thinking skills and are closely tied to curricula. As noted above, such improved tests of students' skills should inform the tests that are used to measure teachers' basic subject-matter competence. It would also be valuable to learn more about the roles that early home and school experiences play in determining the interest of elementary school teachers in science and the time they spend teaching science to their students. It would also be useful to learn whether early home and school experiences influence the decisions of college students to become science or mathematics teachers, and how long individuals who do start to teach science or mathematics remain in the teaching profession. Current evidence is mixed concerning whether there is greater attrition among science and mathematics teachers than among teachers in general (Cavin, 1986; Murnane, in press). Some potential for such studies already exists, using data from the National Longitudinal Study of the High School Class of 1972 (NLS72) and the data base of the High School and Beyond Survey. A special supplement to the NES72 fifth follow-up study, which was administered in 1986 to those members of the original sample who trained to become teachers, will increase the potential for such stud- ies. The possibilities will be further enhanced during the l990s by the National Educational I,ongitudinal Study (NELS), which will follow a large sample of students from their eighth school year in * This section draws on the material presented in Haggstrom et al. (19863.
102 INDICATORS OF SCIENCE AND MATHEMATICS EDUCATION 1988 through further schooling and into the labor market. Research should be supporter! that will use these and other data sets to explore the impact of early school and home experiences on the career choices of potential teachers. Recommendations :Key Indicators: The committee recommends that sam- ples of teachers be selected to take tests that probe the same content and skills that their students are expected to master. For this purpose, tests for teachers should be developed to include the same kinds of improvements that the comrn~ttee recommends for tests of student learning. The distribution of teachers' test scores should be reported by student background and characteristics (race, ethnicity, gender, so- cioeconomic status, type of community-urban, suburban, rural). This will provide information about the distribution across different student subgroups of teachers who are in command of the mathemat- ics and science they are expected to teach. Both current distribution and change over time are of interest; therefore, tests should be given every four years to a sample of all teachers and every two years to a sample of newly hired secondary school mathematics and science teachers. Supplementary Indicator: The committee recommends reorganization of the information currently being collected on teacher preparation (college courses in mathematics and science, majors and minors, advanced degrees), using the various student groups taught as the reporting groups of interest. The information reported should display the percentage of stu- dents with particular backgrounds and characteristics who are being taught mathematics and science in elementary school as well as courses in these domains in secondary school by teachers with spe- cific college preparation. For this indicator also, four-year cycles are appropriate for collection and analysis of information.
INDICATORS OF TEACHING QUALITY Research and Development: The committee recommends that research should be undertaken on two issues: the impact of teachers' knowledge of subject-matter on their effective- ness in teaching these subjects to students, and the role of early home and school experiences in determining decisions to become a teacher and on how and what to teach. TEACHERS' USE OF TIME 103 Among the most significant professional decisions that teachers of mathematics and science make is how to spend their time. These decisions influence the skills, energy levels, and experiences that teachers share with their students. For this reason, the comrn~ttee wishes to focus attention on collecting information in two relevant categories: time-use outside the classroom, both during and beyond the school day, and time-use within the classroom. Time-Use Outside the Classroom One of the attributes of professional teachers is that they con- tinue to learn as they teach and continue to evince interest in the subjects they teach. Thus, one question that should be explored is whether schools are making progress in attracting and retaining teachers of science and mathematics who spend some of their time outside the classroom in activities that demonstrate their continued intellectual curiosity about the subjects they teach. A second ques- tion related to time-use is whether school policies are changing the way in which teachers of mathematics and science at various levels are spending their time in school when not actually teaching in the classroom. Teachers who have no time to develop collegial relation- ships or to plan mathematics and science activities will be less able to exhibit characteristics of fully professional teachers (Rosenholtz, 1985~. A third question is whether the salary increases that are taking place in many states are making it possible for more teach- ers to live on their teaching salaries, lessening the pressure to seek second jobs unrelated to teaching that reduce the time and energy these teachers can devote to their profession. (Currently about 10 percent of teachers earn additional income in jobs outside the school system; 20 percent earn additional pay within the school system for noncIassroom functions; iNational Center for Education Statistics, 19853.)
104 INDICATORS OF SCIENCE AND MATHEMATICS EDUCATION A difficulty involved in collecting information on teachers' time- use is the diversity of ways that teachers can engage in intellectually rewarding pursuits-far more ways than could be anticipated in a closed-ended survey instrument. Similarly, it may be difficult to anticipate in a closed-ended questionnaire the number of tasks or assignments that could inhibit a teacher's ability to use time in professional ways. For this reason, we suggest that information on science and mathematics teachers' time-use be collected every fourth year through a time-budget study. In this approach, a sample of teachers of mathematics and science at different grade levels would be asked to keep a diary, recording how they spend their time during a particular period, perhaps a week. The study should be structured so that data are collected on teachers' time-use during different parts of the year. While time-budget studies have little precedent as a data col- lection strategy in education, they have been used extensively and informatively in research in other areas. Research based on data from time-budget studies has revealed important, and unsuspected, pat- terns in how families (Juster and Stafford, 1985), children (Medrich et al., 1982), and college professors (Institute for Research in Social Behavior, 1984) spend their time. Analogous research on teachers' use of time would be particularly valuable at a time when states and local school districts are engaging in myriad activities that change the incentives and constraints influencing teachers' behaviors salary increases, career-ladder plans, periodic testing of teachers' subject- matter knowledge, and more intensive evaluations of teachers' in- cIass performance (Goertz et al., 1984; Goertz, 19863. There are many ways in which teachers could respond to these new incentives and constraints, and some of the responses could be quite different from those intended. Learning about changes in how teachers of science and mathematics spend their time outside the classroom will throw light on teachers' responses to the many policy changes aimed at improving the quality of education in these subjects. The advantage of the open-ended nature of the time-budget ap- proach is that little prior categorization need be imposed that is, it is not necessary to prepare a list of how teachers might spend their time before beginning the study. In analyzing the data gen- erated by any such time-use study, however, some assumptions will have to be made as to which activities contribute to a teacher's professional competency. Candidate activities include engagement in professional association activities, work on publications related
INDICATORS OF TEACHING QUALITY 105 to science or mathematics, graduate study on the use of computers, time spent in preparation of lessons and new courses, work on curricu- Jum development, professional relations with colleagues, and working with students beyond classroom hours. In addition, time-use might be identified that might contribute negatively to a teacher's energy and enthusiasm for teaching science and mathematics. Analyses of data generated in time-use studies will reveal interesting patterns and provide suggestions for future indicators with specific policy im- plications. The establishment of trends over time will be important for assessing the effects of policy changes. Time-Use in the Classroom The use of classroom time involves both teacher and student behavior. At the elementary level, teachers have a great deal of con- tro] over student behavior in that they control instructional time. Thus, when their teachers allocate more classroom time to teaching, say, mathematics, elementary school students learn more mathemat- ics (Wiley and Harnischfeger, 1974~. The committee is concerned by evidence indicating that some elementary school teachers devote almost no classroom time to science instruction (Weiss, 1978~. For this reason, we reiterate here the recommendation made in Chapter 5 that data be collected every four years on the percentage of ele- mentary students whose teachers devote at least a minimal amount of classroom time to science instruction. This minimal level needs to be chosen with care and should reflect the time needed to teach a meaningful science curriculum at different grade levels. As noted in Chapter 5, student groups with various characteristics should be the unit of analysis in this data collection, and the information should be reported in terms of the percentage of students with particular characteristics and backgrounds in different grade levels who spend at least so many minutes of the school instructional day on science. It will be particularly important to observe whether changes take place over time in the reported distributions as state policies mandating greater attention to science instruction are put into place. The dis- cussion here elaborating the recommendation in Chapter 5 with re- spect to minimal time is framed in terms of elementary school science instruction and puts less emphasis on time devoted to mathematics instruction because it is the committee's sense that mathematics is treated as a major subject by most elementary school teachers. Nev- ertheless, the variability in time spent on mathematics instruction in
106 INDICATORS OF SCIENCE AND AfAITI~MA TICS EDUCATION different grades (Romberg and Carpenter, 1986) makes this measure important for both subjects. Homework also entails both teacher and student behavior. Stu- dents learn more when they do more homework in mathematics and science (Walberg et al., 1986), and homework is more effective when it is relevant to the student learning desired, teachers check it reg- ularly, and students are given feedback about the quality of the homework they complete (Walberg and Rasher, 19863. Within these qualifications, the committee in Chapter 5 recommends the collection of data every four years on the fraction of students with particular characteristics and backgrounds who are regularly assigned effec- tive homework by their teachers. If desired, the benchmarks for assessments of desirable quantities of homework may be determined by expert panels, although both geographic (e.g., international) and temporal comparisons are in themselves useful. For example, changes over time in the reported distributions will provide useful information about one important predictor of students' science and mathematics achievement. Clearly, how much science and mathematics students learn in school depends not only on the amount of time devoted to science and mathematics instruction and on the amount of homework as- signed, completed, and corrected, but also on how classroom time devoted to science and mathematics is used. While the committee has recommended observation of student behavior in this respect (see Chapter 5), we do not recommend the development of indicators that assess teachers' use of classroom time. The reason is that our interpretation of the process-product and teacher effectiveness litera- ture leads us to the aforementioned conclusion that effective teaching requires the orchestration of a variety of strategies suited to the spe- cific instructional context and therefore cannot be characterized by the routine use of particular well-defined actions. Consequently, it would not be useful to base indicators on counts of how often teachers engage in such actions. Needed Research Recent state and local initiatives are making important changes in the intended science and mathematics curricula. How these changes influence the curriculum content that students actually en- counter depends to a large extent on teachers' responses to the changes in the intended curricula. Past research indicates that these
INDICATORS OF TEACHING QUALITY 107 responses can vary, depending on the subject-matter, the skills of the teachers, the adequacy of in-service education programs, the avail- ability of facilities and materials, and the attitudes of administrators (Berman and McLaughlin, 1974-1975; Sarason, 1985~. Learning more about the factors that influence teachers' responses to changes in the intended curriculum will require expensive, painstaking obser- vational research. The research is needed, however, to understand how state and local curricular initiatives affect the quality of science and mathematics instruction provided to students. In the committee's view, there are two dimensions of teachers' use of time in the classroom that may merit significant research at- tention, possibly because they might one day lead to new indicators, but more significantly because they- might improve the quality of science and mathematics instruction. The first of these dimensions is the use of concrete materials, laboratory experiments, and computers in the classroom. Such hands-on components of the curriculum may be particularly important in the development of higher-order think- ing skills. Consequently, in evaluating the effectiveness of hands-on instruction, it is important to use student tests that measure higher- order skills. A second dimension of teachers' use of time involves questioning techniques. There is evidence that students' learning increases when teachers wait at least five seconds for student answers to questions (Rowe, 1983~. Learning more about the accomplishments and lim- itations of increases in waiting time may provide the basis for an indicator of one dimension of effective teaching. More important, such research may result in improvement in the quality of science and mathematics teaching in the schools. Recommendations Supplementary Indicator: The committee recommends that time-budget studies be conducted, asking teachers to record how they spend time related professionally to their present or future classroom activities, other than in the classroom itself, during a particular period, perhaps a week. The information collected should be evaluated against sets of activities identified by experts as advancing or hindering effectiveness in the classroom in teaching mathematics or science. Investigations of the relationships between professional activities reported by teachers
108 INDICATORS OF SCIENCE AND MATHEMATICS EDUCATION and teaching effectiveness should be conducted to help refine this indicator. Research and Development: The committee recommends research on the following aspects of the behavior of teachers in science and mathematics instruction (see also the related research recommendations in Chapter 5 on student behav- ior): the factors affecting teacher responses to changes in the intended curriculum; the use of hands-on experiences involving concrete ma- terials, laboratory experiments, and computers; and allowing an adequate period of time for students to for- mulate responses to questions. The recommendations in Chapter 5 on the amount of time given to the study of science and mathematics in elementary school and on the amount of homework can be considered indicators of teacher behavior as well as student behavior. In either case, we consider them important indicators of the quality of science and mathematics education. IMPLICATIONS FOR STATE EDUCATION AGENCIES Up to this point, the emphasis in implementing teacher evalu- ation schemes in the various states has been on knowledge of the subject matter rather than on other characteristics. A major ex- ception is Tennessee, which more comprehensively than other states has developed an on-site observation and interview schedule to com- plement simple subject-matter knowledge. This approach needs to be more fully explored if a more complete picture of science and mathematics education is to be drawn. The main data source currently available to states for analyzing teacher effectiveness is subject-matter knowledge of teacher candi- dates. What is not known (because it is not systematically analyzed) includes the following: . Are there significant variations among objectives that all new mathematics and science high school teachers as well as elementary teachers need to know, as reflected in teacher job-analysis surveys, polls of college of education faculty, test questions, and test results?
INDICATORS OF TEACHING QUALITY 109 Are the variations greater from state to state, between school sys- tems of different types (e.g., large urban versus rural) within a state, between different sorts of institutions preparing teachers? Is a na- tional consensus emerging on what individuals need to know to be effective science or mathematics teachers? . Is science knowledge part of the requirements for elementary teachers? Tests for elementary teachers generally lack science con- tent; typically, they are dominated by questions on general pedagogy. The low expectation for instruction in science at the elementary level may be a contributing factor, as may be the absence of any agreement as to what the science content of the elementary school curriculum should be, even when science is being taught. . With regard to testing for certification: Are there fewer minorities, proportionately or in actual numbers, entering teacher- preparation programs than in the past especially those training to be future mathematics and science teachers? Are tests and test results such that they systematically discourage members of some population subgroups from choosing teaching careers? Are there patterns in geographic distribution of the Towest-scoring test takers- for example, are they entering urban schools or small rural ones in greater proportion than suburban schools? The periodic collection and analysis of even this small part of the information needed about the potential education work force could have the following state-level policy implications: . Recruiting and preparing minority teacher candidates may need to begin in the junior year of high school; special scholarship programs may have to be initiated especially in mathematics and science if the number of rn~norities in these fields fails significantly below a predetermined standard. . Approval procedures for undergraduate teacher-education programs could be revised to ensure that prospective teachers are exposed to sufficient mathematics and science experiences. . Entrance examination systems for teacher-education pro- grams may need to be structured in such a way as to provide diagnos- tic information about the strengths and weaknesses in mathematics and science of entering candidates; such profiles could be used to guide candidates to specific academic sequences that would ensure that they had at least been exposed to appropriate mathematics and science courses.
110 INDICATORS OF SCIENCE AND MATHEMATICS EDUCATION . Analysis of the mathematics and science test results from successful teacher candidates could lead to targeted regional and state staff-development programs if it is found that the least prepared teachers are locating in certain areas. If the committee's recommendation to follow up the candi- ciates who pass the certification tests and become new hires were to be implemented, it could establish a useful data base on the abil- ity of the education establishment to provide conditions that induce teachers to stay, thereby assisting in future projections of supply and demand. The Recertification information can be collected and used an- nually in those states that possess the requisite data base. However, few states carry out systematic testing of certified teachers, and it is unlikely that this approach will become more widespread. Even if it did, the results would not enrich the general knowledge about teachers because current tests typically avoid science and touch only the basics of mathematics. The committee's recommendation on teacher testing rejects any connection between the use of a nationwide sampling of teachers' mathematics or science knowledge and any use of the information for purposes of personnel decisions. Instead, the data from the tests recommended by the committee would provide a national benchmark on the continuing intellectual growth of school faculty and whether they are staying current. Such data would provide to the states as well as other units of government information that could drive the creation of relevant staff development programs and materials. Finally, assuming some consensus within a state on curriculum, observation of how teachers of mathematics and science organize and present the material and the context in which they present it (time spent on planning and presenting, availability of equipment, etc.) become important indicators of teaching quality in a state's schools. This is especially so since more state legislative bodies are requiring local as well as state "report cards" to document class time spent in subject areas. By themselves, the statistics on minutes spent per day or week on a curriculum area are almost meaningless; they can become indicators only in conjunction with information on other variables. For example, collecting information on whether pupils are asked weekly to write a Misword science laboratory report is quite superficial; it takes on meaning only when one also knows how often these same reports are actually read and critically evaluated, with the
INDICATORS OF TEACHING QUALITY 111 results returned to the student. Only then is the writing requirement likely to help improve the quality of student understanding of science. WORKING CONDITIONS FOR TEACHING SCIENCE AND MATHEMATICS Resources for Teaching Science and Mathematics Effective teaching is best sustained if schools are places where professional teachers like to work and places that provide support for activities that characterize effective teaching. Consequently, it is important to develop indicators of the extent to which the nation's schools are able to provide the resources and support needed to sustain fully professional teaching of science and mathematics for all , . ~ · . . . students. for reasons explained in Chapter 8, the committee does not recommend the collection of data on per-pupi} expenditures devoted to science and mathematics or on specific budgets available to science and mathematics teachers. What we do see as important, however, is to collect detailed information on the uses to which money devoted to mathematics and science instruction is put within a school and within a classroom. The following information on working conditions in schools is pertinent: the availability and use of equipment, materials, textbooks and laboratory facilities appropriate to the intended curriculum; 1, the number of students and different types of courses taught by each teacher; the availability and use of professional time for planning dur- ing school hours, and support for professional activities (further education, curriculum development, collegial exchanges) during the year and during summers; and ~ the availability and use of assistance such as classroom or laboratory aides. At first glance, this information may appear relatively easy to collect using closed-ended questionnaires. This may not be the case, however, for several reasons. First, the mere presence of a facility or materials and equipment does not ensure their use. Even in 1965, most secondary schools had, for example, some facility that was called a laboratory (Coleman et al., 1966~. Analyses of the data indicated relatively minor differences among schools in the number of facilities. Most analysts believe, however, that in 1965 and in 1987
112 INDICATORS OF SCIENCE AND MATHEMATICS EDUCATION as well, there were and are significant differences in the quality of the equipment, materials, and laboratory facilities present in different schools. It is very difficult to capture these differences in quality with closed-ended survey instruments. In addition, as school district officials pointed out (see Appendix C), a secondary school may have adequate laboratory facilities, but only students taking advanced science courses may have access to them. An elementary school may have a few classrooms with provisions for hands-on work, but these may not be available to all grade levels or all classes at a single grade level. It is difficult to learn from closed-ended surveys the extent to which all students taking science and mathematics have access to a school's equipment, materials, and laboratory facilities. Similarly, materials and equipment may be present in a school, but the procedures for making use of them may be so bureaucratic that teachers forego the opportunity to use the potentially available equipment and supplies in their teaching. This suggests the impor- tance of learning about teachers' control of equipment and supplies, and whether teachers actually employ the equipment, supplies, and laboratory facilities in their teaching. For these reasons, we suggest that pilot studies be conducted to explore whether a macro-level indicator can be developed using information on the conditions under which teachers of science and mathematics work. The information should be collected through the use of open-ended interviews. All teachers and administrators who are interviewed would be asked the same questions, with spe- cial attention to probing teachers' open-ended answers. While it will be more difficult to organize these open-ended responses than it would be to tabulate teachers' responses to closed-ended question- naire items, we consider the open-ended questionnaires to be a much more effective strategy for gathering reliable information about the conditions under which science and mathematics teachers work, the number of students taught under inadequate conditions, and changes over time in teachers' access to the resources needed to do their job well. If pilot studies indicate the feasibility of developing an indica- tor on resource use and working conditions, the information should be collected every four years. Such an indicator, as other indica- tors described in this chapter and elsewhere, should be expressed in terms of percentages of students of different backgrounds and char- acteristics who are being served. Careful attention will have to be given to sample design to achieve comparability over time as well as generalizability.
INDICATORS OF TEACHING QUALITY 113 Salaries as Incentives Teacher salaries tend to rank relatively low among professional salaries. This may discourage individuals from entering or staying in teaching, particularly those with training in mathematics and the physical sciences who may have attractive. alternative career opportunities. Even if potential teachers' career decisions were not sensitive to the financial rewards in teaching relative to those in other professions, it would still be somewhat anomalous to pay poorly the members of a profession who potentially can have such marked effects on chil- dren's futures. Nevertheless, one might argue to retain the low pay for financial reasons if it did not affect the decisions that teachers and potential teachers make. There is strong evidence, however, that teachers' decisions are influenced by salaries. For example, Freeman (1976) and Zarkin (1985) have shown that the number of college students who study to become teachers is very sensitive to relative salaries. In addition, Manski (1985) found that the number of aca- demically talented college students who enter teaching is affected by salaries. Subsequent career decisions are also influenced by salaries, for example, teachers' decisions to move from one school district to another and their decisions on whether to leave teaching entirely (Eberts and Stone, 1984~. Thus, salaries appear to provide incentives that have measurable impacts on the career decisions of teachers and prospective teachers and consequently influence the ability of the nation's school districts to staff schools with competent teachers. Since salaries in business and industry vary by subject-matter field, comparative salary data need to be collected by field of spe- ciaTization. This is illustrated by Figure 6-1, which displays data on average starting salaries in business and industry, expressed in 1967 constant dollars, for college graduates with bachelor's degrees in par- ticular subjects. These data are derived from surveys administered by the College Placement Council. For the purpose of comparison, Figure 3 also displays data on average starting salaries for elementary and secondary school teachers expressed in 1967 dollars. In inter- preting the teachers' starting salary data, which stem from surveys administered by the National Education Association (NEA), it is important to keep in mind that more than 99 percent of U.S. public school teachers work in school districts using uniform salary scales, under which field of specialization has no effect on salary. As a result, in any given district, the starting salary of a physics teacher is the same as the starting salary of a history teacher.
114 '~ a: t t ~7 t /~* I\ t x- ~ ~ ~ t Ti 1 ~ t i! If. f t \ T t L ~ | t 5, ~ ~ ~ 11 Aft 1 it if ~ At ~t . , , , 1 *, ~ / it, t - t \ i,' \ ! I'm x 71 1 - L o o o o o o o o o o o o o o o o o o ~no ~Lo 0 o · 0 .C~ . to · ~ 0 ; - ~ 00 a, ~ Vim U) + ~I _ (O T Cal _ _ _ _ - ~ -m - ~ c~ - ~ - ~ ~o * t q ~q 0 ·H :~= ~ >6 6o - o - ~ o · - a) ~ {) ·- ~ o 2 · - u) x ~ .= o o ~ - o o cs ~ ~ z u) · - · - · o v - a.~ bO ~ ·- c-) ~ - v: ~ b4 ~ - ~ - v ~ · ~ c) ~ o
INDICATORS OF TEACHING QUALITY 115 Figure ~1 illustrates two points. First, how much more a college graduate earned by taking a job in business or industry than by tak- ing a teaching position depends on the graduate's subject speciality. For example, in 1974, college graduates specializing in mathematics, chemistry, or physics who entered business or industry were paid 36 to 39 percent more on average than college graduates who became teachers, while college graduates trained in the humanities who en- tered business or industry were paid only 7 percent more on average than college graduates who became teachers; for graduates trained in biology, the differential was 12 percent. Second, the salary differentials between business and industry and teaching have changed over time, and the pattern varies among subject specialties. In general, the differential between teaching and other occupational alternatives has increased more for graduates trained in mathematics or the physical sciences than for graduates trained in the humanities or biology. For example, in 1985, the start- ing salary advantage that business and industry offered over teaching had risen to 59 percent for graduates trained in mathematics, but it had risen to only 13 percent for graduates trained in one of the hu- manities, and had actually fallen by one percentage point for biology graduates. These data indicate the importance of considering each field separately. Comparative salary data need to be collected every two or three years because salaries in different occupations can change signifi- cantly from year to year, and changes over tune in the salaries offered in different occupations are more informative than salary compar- isons at one point in time. In fact, it is not possible to judge from comparisons of starting salaries at one point in time whether the schools are able to attract talented college graduates into teaching. One reason is that working conditions may differ between jobs in teaching and jobs in business or industry. A second reason is that the comparative salary figures are very sensitive to the method of calculation. For example, when daily salaries are compared by divid- ing annual salaries by number of required work days (180 to 200 for teachers; 240 for college graduates working in business or industry), teachers' salaries appear more attractive than when annual salaries are compared. There is no one right way to do the calculation: teach- ers' work days during the school year may be very long days (the proposed time-budget study would address this issue), and many teachers do not have work opportunities during the summer at the same rate of pay. In contrast to the difficulty of making inferences
116 INDICATORS OF SCIENCE AND MATHEMATICS EDUCATION from comparative salaries at one point in time, trends in comparative salaries do provide important information about changes in the abil- ity of the schools to attract talented college graduates with particular types of training. The following salary data should be collected at least every three years (preferably every two years) for each field of study (for exam- ple, mathematics, biology, physics, chemistry): (a) information on starting salaries in teaching and in business and industry and (b) information on salaries after 15 years of experience. The latter infor- mation is important because, in choosing fields of specialization and occupation, college students do compare not only starting salaries, but also streams of earnings (Zabalza et al., 1979~. Moreover, differ- ences in starting salaries between occupations do not always reflect differences in salary streams. For example, the average salary ad- vantage of industry over secondary school teaching was 49 percent ($32,100 compared with $21,600) for individuals with 0 to 4 years of work experience after earning a master's degree in physics; the differential was 70 percent ($50,300 compared with $29,500) for in- dividuals with 15 to 19 years of work experience after earning a master's degree (American Institute of Physics, 1983~. The informa- tion on starting salaries and on salaries after 15 years of experience should include median salaries and the interquartile range of salaries. Median salaries provide a measure of central tendency-an indicator of what the average person in a particular occupation with a particu- lar amount of experience earns, while the interquartile range reflects the amount of variation, for example, in the earnings of a particular group. A large interquartile range may make a particular occupation less attractive, in that college students cannot count on receiving a particular level of compensation if they choose that occupation. There are important differences between the comm~ttee's pro- posals for salary comparisons and comparisons of average salaries in different occupations. The latter comparisons, which are often cited in the media, can be deceiving because they are sensitive to the distribution of experience in each occupation. For example, average salaries in teaching grew more rapidly during the 197()s than start- ing salaries did because the teaching force became older during the decade, since relatively few new teachers were hired. Thus, average salaries do not necessarily reflect the attractiveness of teaching to college graduates who are making occupational choices. The salary comparisons proposed by the committee will throw
INDICATORS OF TEACHING QUALITY 117 the most light on the competitiveness of secondary school teaching salaries, at least for the present, since it is mainly secondary school teachers who have college majors in the subjects that they teach. This may be changing, however, as some states and institutions of higher education follow current proposals to eliminate undergraduate degree programs in elementary school education. Developing the suggested indicator of salary differentials can take advantage of a number of already existing salary surveys. For example, the College Placement Council collects data annually on the salary offers made to a sample of college graduates with particular subject-matter specialties. For many years, the Northwestern Endi- cott Report (1985) has provided annual information on the salaries that a sample of large business and industrial concerns pay to college graduates with particular subject-matter specialties. The U.S. De- partment of Labor also makes available biennial reports of starting salaries in private industry for college graduates with certain spe- cialties. Several professional associations, including the American Chemical Society, the American Institute of Physics, and the Amer- ican Mathematical Society, publish annual reports of the average or median starting salaries earned by their members, broken down by highest degree earned (e.g., American Institute of Physics, 1983~. Much of the salary information collected in individual surveys is pre- sented in a biennial publication of the Commission on Professionals in Science and Technology (formerly, the Scientific Manpower Com- mission) entitled Salaries of Scientists, Engineers, and Technicians. The NEA, which is the primary source of data on starting salaries in teaching, does not routinely report average salaries for teachers with a bachelor's degree and 15 years of experience. However, the salary schedules that are used for the calculation of starting salaries would support generation of this information. It would be preferable to have the data on comparative salaries generated by a single organization using one method. It is difficult to determine, for example, the extent to which differences in the median starting salaries of chemists and biologists reported by the respective professional societies stem from differences in survey method. One strategy that should be explored is the use of data from the U.S. Census Bureau's Current Population Survey to generate comparative salary data. Until a uniform method is developed, however, salary data can be reported using information generated by the sources cited above.
118 INDICATORS OF SCIENCE AND MATHEMATICS ED UCATION Recommer~dations Supplementary Indicator: The committee recommends that data be collected on a four-year cycle through open- ended surveys on the materials, facilities, and supplies avail- able and used by teachers in mathematics and science in- struction. An indicator can be constructed from this information by report- ing on the levels of resources being used in the classroom by student subgroups of different backgrounds and competencies. Key Indicator: The committee recommends collection at least every three years (preferably every two years) of de- tailed information on the salaries paid to college graduates with particular subject-matter specialties who choose to en ter various occupations. The information should include data on starting salaries and on salaries after 15 years of experience. These data should be reported in a manner that facilitates comparisons of salaries in teaching with salaries in other occupations for college graduates trained in partic- ular sciences and mathematics.
