Appendix F Alternative Assessment Case Studies

PERFORMANCE ASSESSMENT OF EXPERIENCED TEACHERS BY THE NATIONAL BOARD FOR PROFESSIONAL TEACHING STANDARDS

The National Board for Professional Teaching Standards (NBPTS) provides an example of a large-scale, high-stakes performance assessment of teaching that draws on portfolio and assessment center exercises. While NBPTS assessments are intended for voluntary certification of experienced teachers (teachers who have been practicing in their subject areas for at least three years), this case is relevant to the committee’s focus on assessment of beginning teaching in at least two ways. First, it provides an example of a model that states may choose or have chosen to emulate. Second, states may decide to grant licenses to candidates certified by the NBPTS. Finally, NBPTS certification is viewed by its proponents as an integral phase in a teacher’s career development that can and should be consonant with earlier phases of assessment and development.

The NBPTS is an independent organization governed by a 63-member board of directors, most of whom are classroom teachers. Its mission is to “establish high and rigorous standards for what accomplished teachers should know and be able to do, to develop and operate a national voluntary system to assess and certify teachers who meet these standards, and to advance related education reforms for improving student learning in American schools” <www.nbpts.org>. In addition, NBPTS seeks to forge “a national professional consensus, to reliably identify teachers who meet [these] standards,” and to communicate what accomplished teaching looks like” (Moss and Schutz, 1999:681). The NBPTS is in the



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 298
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality Appendix F Alternative Assessment Case Studies PERFORMANCE ASSESSMENT OF EXPERIENCED TEACHERS BY THE NATIONAL BOARD FOR PROFESSIONAL TEACHING STANDARDS The National Board for Professional Teaching Standards (NBPTS) provides an example of a large-scale, high-stakes performance assessment of teaching that draws on portfolio and assessment center exercises. While NBPTS assessments are intended for voluntary certification of experienced teachers (teachers who have been practicing in their subject areas for at least three years), this case is relevant to the committee’s focus on assessment of beginning teaching in at least two ways. First, it provides an example of a model that states may choose or have chosen to emulate. Second, states may decide to grant licenses to candidates certified by the NBPTS. Finally, NBPTS certification is viewed by its proponents as an integral phase in a teacher’s career development that can and should be consonant with earlier phases of assessment and development. The NBPTS is an independent organization governed by a 63-member board of directors, most of whom are classroom teachers. Its mission is to “establish high and rigorous standards for what accomplished teachers should know and be able to do, to develop and operate a national voluntary system to assess and certify teachers who meet these standards, and to advance related education reforms for improving student learning in American schools” <www.nbpts.org>. In addition, NBPTS seeks to forge “a national professional consensus, to reliably identify teachers who meet [these] standards,” and to communicate what accomplished teaching looks like” (Moss and Schutz, 1999:681). The NBPTS is in the

OCR for page 298
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality process of developing standards and assessments for more than 30 certification fields identified by the subject or subjects taught and by the developmental level of the students. For each certificate, development work starts by articulating a set of content standards, based on the five core propositions set forth in the NBPTS central policy statement (National Board for Professional Teaching Standards, 1996:2– 3; see Box F-1). The drafting of the content standards for each certificate based on these core propositions is handled by a committee comprised primarily of teachers experienced in the relevant subject area along with experts in child development, teacher education, and the relevant academic discipline. Public review and comment are obtained for the content standards, and the feedback received is used in the final revision of the standards (Moss and Schutz, 1999). The final version of a content standards document states that the standards “represent a professional consensus on the critical aspects of practice that distinguish exemplary teachers in the field from novice or journeymen teachers. Cast in terms of actions that teachers take to advance student outcomes, these standards also incorporate the essential knowledge, skills, dispositions, and commitments that allow teachers to practice at a high level” (e.g., National Board for Professional Teaching Standards, 1996:1). Moss and Schutz (1999:682–683) describe the Early Adolescence/English Language Arts Standards: [T]here are 14 distinct Early Adolescence/English Language Arts (EA/ELA) Standards…Standard II states that: “Accomplished EA/ELA teachers set attainable and worthwhile learning goals for students and develop meaningful learning opportunities while extending to students an increasing measure of control over how those goals are pursued.” . . This standard is then elaborated into a full-page description that includes statements such as, “Educational goal-setting is an interactive process in the middle-grades English teacher’s classroom…. These activities often include a strong mixture of student involvement and direction”; “in carrying out learning direction activities, accomplished teachers adjust their practice, as appropriate, based on student feedback and provide many alternative avenues to the same learning destinations”; or “the planning process is inclusive, no one is allowed to disappear.” Box F-2 provides an overview of the standards for EA/ELA. These content standards are then used to guide all aspects of assessment development for that certification assessment. Once the content standards are created for any given certificate, the assessment developers (now primarily at Educational Testing Services, or ETS) are joined by a second committee of approximately eight experienced teachers in the development and pilot testing of assessment tasks and the accompanying rubrics that will be used to score candidates’ performances. Although each assessment

OCR for page 298
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality BOX F-1 Five Propositions of Accomplished Teaching The National Board for Professional Teaching Standards seeks to identify and recognize teachers who effectively enhance student learning and demonstrate the high level of knowledge, skills, abilities and commitments reflected in the following five core propositions. Teachers are committed to students and their learning. Accomplished teachers are dedicated to making knowledge accessible to all students. They act on the belief that all students can learn. They treat students equitably, recognizing the individual differences that distinguish one student from another and taking account of these differences in their practice. They adjust their practice based on observation and knowledge of their students’ interests, abilities, skills, knowledge, family circumstances and peer relationships. Accomplished teachers understand how students develop and learn. They incorporate the prevailing theories of cognition and intelligence in their practice. They are aware of the influence of context and culture on behavior. They develop students’ cognitive capacity and their respect for learning. Equally important, they foster students’ self-esteem, motivation, character, civic responsibility and their respect for individual, cultural, religious and racial differences. Teachers know the subjects they teach and how to teach those subjects to students. Accomplished teachers have a rich understanding of the subject(s) they teach and appreciate how knowledge in their subject is created, organized, linked to other disciplines and applied to real-world settings. While faithfully representing the collective wisdom of our culture and upholding the value of disciplinary knowledge, they also develop the critical and analytical capacities of their students. Accomplished teachers command specialized knowledge of how to convey and reveal subject matter to students. They are aware of the preconceptions and background knowledge that students typically bring to each subject and of strategies and instructional materials that can be of assistance. They understand where difficulties are likely to arise and modify their practice accordingly. Their instructional repertoire allows them to create multiple paths to the subjects they teach, and they are adept at teaching students how to pose and solve their own problems. Teachers are responsible for managing and monitoring student learning. Accomplished teachers create, enrich, maintain and alter instructional settings to capture and sustain the interest of their students and to make the most effective use of time. They also are adept at engaging students and adults to assist their teaching and at enlisting their colleagues’ knowledge and expertise to complement their own.

OCR for page 298
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality Accomplished teachers command a range of generic instructional techniques, know when each is appropriate and can implement them as needed. They are as aware of ineffectual or damaging practice as they are devoted to elegant practice. They know how to engage groups of students to ensure a disciplined learning environment, and how to organize instruction to allow the schools’ goals for students to be met. They are adept at setting norms for social interaction among students and between students and teachers. They understand how to motivate students to learn and how to maintain their interest even in the face of temporary failure. Accomplished teachers can assess the progress of individual students as well as that of the class as a whole. They employ multiple methods for measuring student growth and understanding and can clearly explain student performance to parents. Teachers think systematically about their practice and learn from experience. Accomplished teachers are models of educated persons, exemplifying the virtues they seek to inspire in students—curiosity, tolerance, honesty, fairness, respect for diversity and appreciation of cultural differences—and the capacities that are prerequisites for intellectual growth: the ability to reason and take multiple perspectives, to be creative and take risks, and to adopt an experimental and problem-solving orientation. Accomplished teachers draw on their knowledge of human development, subject matter and instruction, and their understanding of their students to make principled judgments about sound practice. Their decisions are not only grounded in the literature, but also in their experience. They engage in lifelong learning, which they seek to encourage in their students. Striving to strengthen their teaching, accomplished teachers critically examine their practice, seek to expand their repertoire, deepen their knowledge, sharpen their judgment and adapt their teaching to new findings, ideas, and theories. Teachers are members of learning communities. Accomplished teachers contribute to the effectiveness of the school by working collaboratively with other professionals on instructional policy, curriculum development, and staff development. They can evaluate school progress and the allocation of school resources in light of their understanding of state and local educational objectives. They are knowledgeable about specialized school and community resources that can be engaged for their students’ benefit, and are skilled at employing such resources as needed. Accomplished teachers find ways to work collaboratively and creatively with parents, engaging them productively in the work of the school. SOURCE: National Board for Professional Teaching Standards, 1996.

OCR for page 298
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality BOX F-2 Standards Overview: Early Adolescence/ English Language Arts (EA/ELA) The following standards are presented as facets of the art and science of teaching English language arts to young adolescents. They are an analytical construct, created to provide a closer accounting of the critical aspects of accomplished practice. However, in real time these segments of teaching occur concurrently because teaching is a seamless activity with many disparate purposes being served in the classroom at any given moment. Preparing the Way for Productive Student Learning Knowledge of Students Accomplished EA/ELA teachers systematically acquire a sense of their students as individual language learners. Curricular Choices Accomplished EA/ELA teachers set attainable and worthwhile learning goals for students and develop meaningful learning opportunities while extending to students an increasing measure of control over how those goals are pursued. Engagement Accomplished EA/ELA teachers elicit a concerted effort in language learning from each of their students. Learning Environment Accomplished EA/ELA teachers create a caring, inclusive and challenging environment in which students actively learn. Instructional Resources Accomplished EA/ELA teachers select, adapt and create curricular resources that support active student exploration of literature and language processes. Advancing Student Learning in the Classroom Reading Accomplished EA/ELA teachers engage their students in reading and responding to literature, and in interpreting and thinking deeply about literature and other texts. Writing Accomplished EA/ELA teachers immerse their students in the art of writing. Discourse Accomplished EA/ELA teachers foster thoughtful classroom discourse that provides opportunities for students to listen and speak in many ways and for many purposes. Language Study Accomplished EA/ELA teachers strengthen student sensitivity to and proficiency in the appropriate uses of language. Integrated Instruction Accomplished EA/ELA teachers integrate reading, writing, speaking and listening opportunities in the creation and interpretation of meaningful texts. Assessment Accomplished EA/ELA teachers use a range of formal and informal assessment methods to monitor student progress, encourage student self-assessment, plan instruction and report to various audiences.

OCR for page 298
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality Supporting Student Learning through Long-Range Initiatives Self-reflection Accomplished EA/ELA teachers constantly analyze and strengthen the effectiveness and quality of their teaching. Professional Community Accomplished EA/ELA teachers contribute to the improvement of instructional programs, advancement of knowledge, and practice of colleagues. Family Outreach Accomplished EA/ELA teachers work with families to serve the best interests of their children. SOURCE: National Board for Professional Teaching Standards, 1996. task is designed in light of the standards specific to a given certificate, NBPTS has adopted a framework that generalizes across certificates to guide the work of all development teams. Each assessment consists of two major parts: a portfolio to be completed by candidates in their home schools and a half-day of testing at an assessment center. The school-based portfolio consists of (1) three entries that are classroom based and include two videos that document the candidate’s teaching practice through student work and (2) one entry that combines the candidate’s work with students’ families, the community, and collaboration with other professionals. The six assessment center exercises require candidates to demonstrate their knowledge of subject matter content. Box F-3 provides a detailed overview of the EA/ELA assessment tasks. A formal multistate pilot test is administered before the assessments are released for the first operational use. Each assessment task is scored in accordance with a rubric prepared during the task development phase. NBPTS scoring rubrics encompass four levels of performance on a particular task, with the second-highest level designated as meeting the standards of accomplishment.1 Further, the terminology used in each scoring rubric closely mirrors the language of the relevant content standards. Following initial use of the assessment, at least three extensively trained assessors and other teaching experts select a sample of responses to be used in training and certifying the scorers. The small group charged with selecting the 1   The rubrics for each exercise describe four different performance levels representing score “families” of 1 (low), 2, 3, and 4 (high). Within each score family, an assessor can choose to assign a “+” or a “−” .25 indicating somewhat higher and lower levels of performance. The passing score for each exercise is designated as the lower end of the 3 family, “3−,” or 2.75.

OCR for page 298
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality BOX F-3 Early Adolescence/English Language Arts The NBPTS has developed standards for what accomplished Early Adolescence/ English Language Arts teachers should know and be able to do. This certificate is designed for teachers of students ages 11–15. The assessment is performance-based and designed to evaluate the complex knowledge and skills of teaching described in the standards. The assessment process consists of two components: the portfolio entries and the half-day assessment center exercises. The certification decision is based on candidate performance as judged against the NBPTS standards for accomplished practice. The Portfolio The portfolio for the Early Adolescence/English Language Arts assessment provides teachers with the opportunity to present a sample of their actual classroom practice over a specified time period. The portfolio consists of four entries*: Analysis of Student Growth in Reading and Writing—Through written commentary and samples of students’ responses, teachers provide evidence of their ability to engage students in the study of literature and writing. Two writing samples from each of two students provide evidence of how the teacher uses the writing process to improve the students’ abilities in written expression. The teacher analyzes each of the work samples in relation to specified goals and to identify the students’ growth as a reader and a writer. The analysis includes assessment and evaluation of the impact of feedback on the students’ growth. The reflection includes consideration of how the teacher can improve his/her practice and implications for future instruction. Instructional Analysis—Whole Class Discussion—Through written commentary and a 15-minute videotape, teachers demonstrate the strategies that they use to engage students in whole-class discussion. The commentary includes analysis of the goals and strategies for instruction. The video provides evidence of instruction, the learning environment, and engagement of the students. Teachers demonstrate how they engage all the students in meaningful discourse about a topic, concept, or text important to their English Language Arts curriculum. The commentary also includes reflection on the teacher’s practice, analysis of successes and ways of improving, and implications for future instruction. Instructional Analysis—Small Groups—Through written commentary and a 15-minute videotape, teachers demonstrate their practice of small group instruction about a significant English language arts concept. Students may be discussing their reading, sharing writing, collaborating on projects, or participating in other learning activities which are appropriate for small groups. The videotape should show the teacher circulating among the groups and interacting with the students. The commentary includes discussion of the teacher’s knowledge about the students as well as an analysis of the goals for the instruction. The reflection addresses implications for future instruction. Documented Accomplishments: Contributions to Student Learning— Teachers demonstrate their commitment to student learning through their work with students’ families and the community and their development as

OCR for page 298
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality learners, leaders, and collaborators. This entry is designed to capture evidence of the ways in which a teacher’s role is broader than what the teacher does in the classroom. Teachers submit descriptions and analyses of their activities and accomplishments. The commentary includes discussion of why the activities and accomplishments are significant in the particular teaching context and what impact they had on student learning. In addition, teachers are asked to compose a brief interpretive summary related to these accomplishments. The Assessment Center The half-day of assessment center exercises examine subject matter content. There are six written exercises, and teachers are given 30 minutes to complete each. One score is reported for each exercise. The exercises measure knowledge in the following areas: Literary analysis and structure, including literary devices Universal literary themes and literary works Reading strategies Knowledge of language study Analysis of writing conventions and components of effective writing • exercises are currently undergoing revision; these descriptions are subject to change SOURCE: NBPTS Next Generation Certificate Overviews, April 2001. benchmarks is instructed to choose them so that assessors see that there are different ways to achieve a score. Those who select benchmarks and score portfolios are subjected to a series of training exercises designed to assist them in identifying and controlling any personal biases that might influence their selection. Most exercises are scored independently by two assessors with a third, more experienced assessor used to adjudicate scores that differ by more than a prespecified amount. For some exercises, where interrater reliability is deemed sufficient, only a sample of exercises is double scored. A weighting strategy is employed to combine scores on the exercises and to form the total scores on the assessment. This weighting strategy is selected by another committee of teachers who must choose from among four predetermined weighting strategies. To make certification decisions (accomplished/not accomplished), the total score is compared to a predetermined passing score that is uniform across certificates. This performance standard is equivalent to receiving a just-passing score on each

OCR for page 298
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality of the exercises, although high scores on one exercise can compensate for low scores on others. The uniform performance standard set by the NBPTS was based on a series of empirical standard-setting studies. These studies explored a number of different standard-setting methods (see Jaeger, 1998; Educational Testing Service, 1998). In all of the investigations, panels of experienced teachers were asked to make decisions based on profiles of scores across exercises. The process was “typically iterative, in which individual panelists—thoroughly familiar with the tasks, rubrics, and standards—were given an opportunity to discuss and revise their individual decisions about score profiles in light of feedback on other panelists’ decisions and on the practical implications of their own decisions. From this set of revised individual decisions, the assessment developers computed the ‘recommended’ performance standard” (Moss and Schutz, 1999:685). The NBPTS adopted the uniform performance standard for all certificates based on the results of these early studies along with consideration of the practical implications of setting the standard at different levels (e.g., minimizing adverse impact for groups of candidates, minimizing the anticipated proportion of candidates who are misclassified as failing the exam due to measurement error). In addition to documentation of the development process, five other kinds of validity evidence are routinely gathered and examined for each assessment: Content-related evidence of validity is examined by convening a panel of experienced teachers in the subject area to independently rate (a) the extent to which each of the content standards describes a critical aspect of highly accomplished teaching and (b) the importance and relevance of each exercise and rubric to each content standard and to the overall domain of accomplished teaching. “Scoring validation,” as defined by the assessment developers, is evaluated by assembling another panel of experienced teachers in the subject area to rank randomly selected pairs of exercise responses. These rankings are then compared to the rankings obtained from the official scoring. Information regarding reliability and errors of measurement is reported as (a) error associated with scores given by different assessors, (b) error associated with the sampling of exercises, and (c) misclassification estimates (i.e., estimates of the proportion of candidates incorrectly passing and failing) due to both of these sources of error. Confirmation of the predetermined passing standard is obtained by convening a panel of experienced teachers, who have also served as assessors. The panel examines different possible profiles of exercise scores, rank ordered by total score, and draws lines where they believe the passing standard should be. Evidence regarding bias and adverse impact is provided by (a) reporting certification rates by gender, ethnicity (where sample sizes permit), and teaching

OCR for page 298
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality context and (b) investigating the influence of having exercises scored by assessors of the same and different ethnicity as the candidate. The test developer, ETS, publishes a technical manual (Educational Testing Service, 1998) that describes the methodology for these studies, provides annual updates of technical information for each certificate, and publishes outcomes for each administration. The technical manual is available to the public. The annual updates are available only to NBPTS members, staff, consultants, and advisory panel members. Special studies are frequently presented at national conferences. The NBPTS’s validity research agenda, technical reports, and reports of special studies are routinely reviewed and commented on by the Measurement Research Advisory Panel, a panel of independent scholars whose backgrounds are primarily in educational measurement. In addition to these routine studies, NBPTS has undertaken a number of special validity/impact studies, many of which are still in progress. These include (1) an external review of complete portfolios by a diverse panel exploring differences in performance between African American and white candidates (Bond, 1998); (2) a study of rater severity and the effects of different scoring designs (Englehard, et al., 2000); (3) a survey of how teachers prepare for the exam (Lynch et al., 2000); (4) a survey of assessors’ impressions about the impact of training and scoring on their professional development (Howell and Gitomer, 2000); and (5) a comparison of NBPTS evaluations with others based on classroom observations and interviews (Wylie et al., 2000). A recently released study compared the teaching practices of NBPTS certified teachers with other teachers and compared samples of student work from classrooms of the two groups of teachers (Bond et al., 2000). NBPTS assessments are voluntary; passing the assessment results in a certification of accomplishment. To be eligible for NBPTS certification, a teacher must have completed a baccalaureate degree; must have a minimum of three years of teaching experience at the early childhood, elementary, middle, or secondary levels; and must have a valid state teaching license for each of those years or, where a license is not required, the teacher must be teaching in a school recognized and approved to operate by the state. For the 2000–2001 testing year, the examination fee was $2,300. A number of states and other education agencies have programs in place to subsidize this cost, and NBPTS publicizes that information on its website. Candidates are given approximately 10 months to complete the portfolio tasks and attend the assessment centers. Since each exercise is scored independently, a candidate who does not pass the assessment may bank scores on the exercises passed and retake exercises on which a failing score was received (below a 3 on a four-point scale) for a period of two years after being notified of his or her initial scores. NBPTS publishes a list of newly certified teachers each year. States and local education agencies have their own policies about how the scores are used, in-

OCR for page 298
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality cluding how teachers are recognized and rewarded for receiving NBPTS certification. These include the granting of a state license for teachers transferring into the state, salary increases and bonuses, and opportunities to assume new roles. (See <www.nbpts.org> for a summary of state incentives.) The NBPTS’s direct involvement in professional development and support activities is limited. For instance, the board offers short-term institutes for “facilitators” who plan to support candidates for certification and a series of “Teacher Development Exercises” that can be purchased and used in local workshops. NBPTS does, however, work informally with state and local education agencies and with teacher education institutions to support and publicize local initiatives (see Box F-4). For instance, NBPTS provides a list of ways that state and local education agencies and institutions might support its work and offers contact information for local agencies to interested teachers. In addition, NBPTS’s web page contains information about the activities of its affiliates and provides examples of the variety of professional roles that board-certified teachers have assumed. It should be noted that while NBPTS supplies information on local contacts and activities, it does not monitor the quality, relevance, or usefulness of this information. Through its certification assessments and related activities, NBPTS hopes to “leverage change” in the contexts and culture of teaching. The board hopes to (1) make “it possible for teachers to advance in responsibility, status, and compensation without having to leave the classroom” and (2) encourage “among teachers the search for new knowledge and better practice through a study regimen of collaboration and reflection with peers and others” (National Board for Professional Teaching Standards, 1996:7). CONNECTICUT’S TEACHER PREPARATION AND INDUCTION PROGRAM Overview Connecticut, working in collaboration with the Interstate New Teacher Assessment and Support Consortium (INTASC), provides an example of a state that has implemented a licensing system that relies on performance assessments. Connecticut’s Beginning Educator Support and Training program is a comprehensive three-year induction program that involves both mentoring and support for beginning teachers as well as a portfolio assessment. The philosophy behind Connecticut’s teacher preparation and induction program is that effective teaching involves more than demonstration of a particular set of technical skills. The program is based on the fundamental principle that all students must have the opportunity to be taught by a caring, competent teacher and that, in addition to command of subject matter, effective teaching requires a deep concern about students and their success, a strong commitment to student achievement, and the

OCR for page 298
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality The next interaction is called Profiles of Practice—Observation of Classroom Practice. Two observation events take place in which the mentor teacher observes the beginning teacher leading an instructional lesson. The mentor then provides feedback based on the observation and on review of planning materials, oral and written reflections, and examples of student work. Three inquiry tasks are also scheduled for beginning teachers. These tasks require beginning teachers to explore specific aspects of their teaching practice. To complete each event, beginning teachers must gather information about a selected aspect of teaching using a variety of resources, including their colleagues, research journals, and texts. Together with the mentor, the beginning teacher develops a plan of action to try out in the classroom, implements the plan, and reflects on the experience. The inquiry events focus on ways to establish a positive classroom environment, design an instructional experience, and analyze student work. Teachers also participate in two events called Individual Growth Plans. The individual growth plans are designed to help beginning teachers determine how best to focus their efforts throughout the entire induction process. To complete these events, beginning teachers must prepare a plan for professional learning that takes into account their teaching practices, school or district initiatives, and other challenges they may face. The final induction activities are referred to as Closure Events: Assessment and Colloquium. These interactions help bring closure to the year by engaging beginning teachers in self-assessment, encouraging a final evaluation of professional learning, and promoting the sharing of professional knowledge with other beginning teachers. A total of 10 events occur over the course of the school year. All of the inquires and observations make use of planning, teaching, reflecting, and applying what is learned. The activities encourage beginning teachers to participate in reflective writing, conversations with experienced colleagues, and ongoing examination of teaching in relation to student learning (Educational Testing Services, 1995a). Training of Mentors and Assessors Since the PATHWISE Induction Program-Praxis III Version relies on effective use of observational data, extensive training is required. ETS offers three levels of training for the program. An initial two-day training in the PATHWISE Classroom Observation system acquaints individuals with the 19 criteria that form the basis for the program and is a prerequisite for other levels of training. Participants receive instruction in recording observational data, analyzing written contextual information, using written and observational information to evaluate performance, writing summaries of teacher performance, and providing

OCR for page 298
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality feedback to beginning teachers. Training relies on simulations, case studies, sample evaluation forms, and videotapes. A four-day training session familiarizes individuals with the PATHWISE Induction Program-Praxis III Version. This seminar focuses on training mentors in the uses and purposes of 10 ten events and in developing their mentoring and coaching skills. The seminar for Praxis III assessors is the final level of training and is five days in duration. The training consists of a series of structured activities during which participants learn to recognize the presence of each of the 19 Praxis III criteria. Participants learn to evaluate written information provided by the teacher, take notes during classroom observations, and conduct semistructured interviews. The training process utilizes a variety of stimuli, including worksheets, sample records of evidence, simulations, case studies, and videotapes. Trainees receive feedback from instructor and fellow participants. They also engage in professional reflection through journal writing. Implementation of Pathwise Induction Program-Praxis III Version in Ohio ETS offers training for mentors and assessors but also provides instructional modules for individuals to learn how to conduct the training sessions. Individuals in Ohio have learned the ETS procedures, and the state now offers its own training sessions. Training in the PATHWISE Classroom Observation System has been incorporated into Ohio’s preservice program to introduce students to the 19 criteria. Many state institutions have incorporated the domains into their preservice education programs, and their focus will be on using the PATHWISE rubrics to evaluate students’ progress. PATHWISE Classroom Observation System training is also available for teachers who plan to become mentors. Approximately 15,000 teachers have completed this training (John Nickelson, Ohio Department of Education, personal communication, March 19, 2001). Implementation of the Praxis III Performance Assessment Beginning teachers in Ohio will be required to pass Praxis III starting in 2002. Ohio teachers who do not pass it after their first year may try again during the second year. Failure to complete the Entry-Year Program’s requirements successfully after the second attempt will result in loss of the provisional license until such time as the candidate completes additional coursework, supervised field experiences, and/or clinical experiences as designated by a college or university approved for teacher preparation and is recommended by such college or university.

OCR for page 298
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality ABILITY-BASED TEACHER EDUCATION AT ALVERNO COLLEGE Overview In this case study the committee provides a description of teacher education and assessment as practiced at Alverno College, in Milwaukee, Wisconsin. Alverno College undertook development of a performance-based baccalaureate degree over 20 years ago (Diez, et al., 1998). This change resulted in an overhaul of the college’s curriculum and its approach to teaching. The approach is characterized by publicly articulated learning outcomes, realistic classroom activities and field experiences, and ongoing performance assessments of learning progress. Alverno’s program is of interest because it provides an example of a system in which a party other than a state or district could warrant teacher competence. The focus here is on Alverno as a working program that can expand the debate about other models for warranting teacher competence. Alverno College has an enrollment of approximately 1,900 students in 66 fields of study, 300 of whom are education majors. With the exception of a postbaccalaureate teacher certification program and the master of arts in education program, all programs admit only female students. Two-thirds of Alverno’s students are from the Milwaukee area, and about 30 percent are members of minority groups. There are about 100 faculty members, and the average class size is 25. The next section provides a brief overview of Alverno’s philosophy and practices.2 Abilities and Learning Outcomes for the Baccalaureate Degree At Alverno College an ability is defined as “a complex integration of knowledge, behaviors, skills, values, attitudes, and self-perceptions” (Diez et al., 1994:9). The general education courses provide students with the opportunity to expand and demonstrate each of eight abilities: Communication—an ability to communicate effectively by integrating a variety of communication abilities (speaking, writing, listening, reading, quantitative, media literacy) to meet the demands of increasingly complex communication situations. Analysis—an ability to be a clear thinker, fusing experience, reasoning, and training into considered judgment. 2   Mentkowski and Associates (2000) gives a comprehensive picture of Alverno’s practices and philosophy, and Zeichner (2000) provides an independent description of Alverno’s teacher education practices.

OCR for page 298
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality Problem solving—an ability to define problems and integrate a range of abilities and resources to reach decisions, make recommendations, or implement action plans. Values within decision making—an ability to reflect and to habitually seek to understand the moral dimensions of decisions and to accept responsibility for the consequences of actions. Social interaction—an understanding of how to get things done in committees, task forces, team projects, and other group efforts. Global perspective—an ability to articulate interconnections between and among diverse opinions, ideas, and beliefs about global issues. Effective citizenship—an ability to make informed choices and develop strategies for collaborative involvement in community issues. Aesthetic responsiveness—an ability to make informed responses to artistic works that are grounded in knowledge of the theoretical, historical, and cultural contexts. The abilities cut across disciplines and are subdivided into six developmental levels. The six levels represent a developmental sequence that begins with objective awareness of one’s own performance process for a given ability and specifies increasingly complex knowledge, skills, and dispositions. Students must demonstrate consistent performance at level 4 for each of the eight abilities prior to graduation. An example of the development levels for problem solving appears in Box F-7. Each of Alverno’s educational programs also defines a set of abilities distinctive to each major and minor area. These outcomes, identified by faculty as essen- BOX F-7 Levels of Learning Outcomes for Problem Solving Level Benchmark 1 Articulate own problem-solving process, making explicit the steps taken to approach the problem(s) 2 Analyze the stucture of discipline- or profession-based problem-solving frameworks 3 Use discipline- of profession-based problem-solving frameworks and strategies 4 Independently examine, select, use, and evaluate various approaches to develop solutions In majors and areas of specialization: 5 Collaborate in designing and implementing a problem-solving process 6 Solve problems in a variety of professional settings and advanced disciplinary applications SOURCE: Alverno College Faculty, 1973/2000.

OCR for page 298
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality tial learning outcomes, relate to and extend the general education abilities (Loacker and Mentkowski, 1993). Within the major area of study, students are expected to achieve at least a level 5 for each of the program’s abilities (Zeichner, 2000). Abilities and Learning Outcomes in Teacher Education Alverno’s Department of Education offers degree programs in elementary, early childhood, secondary, bilingual, music, art, and adult education. All education programs are designed to foster the same set of teaching abilities. These teaching abilities define professional levels of proficiency that are required for graduation with a major in any of the teacher education programs. The teaching abilities refine and extend the general education abilities into the professional teaching context. While the professional teaching abilities are introduced in the first year, they receive heavy emphasis during the junior and senior years. The teaching abilities include: Conceptualization—integrating content knowledge with educational frameworks and a broadly based understanding of the liberal arts in order to plan and implement instruction. Diagnosis—relating observed behavior to relevant frameworks in order to determine and implement learning prescriptions. Coordination—managing resources effectively to support learning goals. Communication—using verbal, nonverbal, and media modes of communication to establish the environment of the classroom and to structure and reinforce learning. Integrative interaction—acting with professional values as a situational decision maker, adapting to the changing needs of the environment in order to develop students as learners. Each of the above education abilities is further described for faculty and candidates through maps (Diez et al., 1998). In the maps, development of the ability is defined in terms of what teachers would be expected to do with their knowledge and skills at various stages of their development. An example based on skill in integrative interaction follows: the beginning teacher would demonstrate ability in integrative interaction by showing respect for varied learner perspectives; the experienced teacher would provide structures within which learners create their own perspectives; and the master teacher would assist learners in the habit of taking on multiple perspectives. This type of mapping is intended to “capture the interactions between knowing and doing” (Diez et al., 1998:43). Alverno’s Program for Education Majors Alverno’s program for education majors is designed to address the developmental needs of learners. Concepts are addressed in an integrated fashion, across

OCR for page 298
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality multiple courses and settings to enable a “deepened understanding” that comes with repetition of concepts (Diez, 1999:233). The program is characterized by extensive opportunities for field experiences that require candidates to apply what they have learned. Coursework and field experiences are sequenced to build developmentally across the years of the program. For example, candidates begin with coursework and field experiences that require them to apply the frameworks they are learning with individual students or small groups in tutorial settings. They progress to more complex tasks with larger groups and whole-class instruction. The assignments gradually increase in complexity, requiring candidates to attend to multiple factors in their planning, their analysis of the classroom, and their implementation of learning experiences (Diez, 1999:233). Self-reflection and self-assessment skills are emphasized at Alverno. Faculty have developed a set of reflective logs that guide students in each of four semester-long field experiences prior to student teaching. These logs are intended to help students develop their skills in the five education abilities. According to Diez (1999), the logs direct students to make links between theoretical knowledge and practical application (which develops skill in conceptualization and diagnosis), to observe processes and environments of learning (coordination skills), to translate content knowledge into suitable short presentations or learning experiences (communication skills), and to begin to translate their philosophy of education into decisions regarding all aspects of teaching environments and processes (integrative interaction). The first stage of the education program is the preprofessional level. To apply to the preprofessional level, students must have completed one year of coursework, a required one-credit human relations workshop, and a portion of the math content requirements (Zeichner, 2000). The preprofessional stage lasts two semesters. During this time, education students begin to integrate the knowledge bases of the liberal arts disciplines with the process for applying the material from these disciplines (Alverno College Institute, 1996). The subject area methods courses are taught during this stage. These courses connect teaching methods with material learned in liberal arts general education courses. Performance assessments during this stage may consist of such activities as requiring teacher candidates to create a lesson for a given grade that incorporates knowledge about developmental psychology. Other performance assessments involve simulations in which prospective teachers take on the various roles that teachers play, such as conducting a parent-teacher conference, being part of a multidisciplinary evaluation team, or working with district planning activities. This period includes two pre-student teaching experiences. After two semesters in the preprofessional stage and completion of two of the pre-student teaching experiences, teacher candidates can apply for admission to the professional level. To be admitted, they must successfully complete the first two field experiences (and provide letters of recommendation), demonstrate a specific ability, meet the statewide minimum cutoff scores on the required

OCR for page 298
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality Praxis exams, and pass several standard assessment exercises that are spread throughout the program (Zeichner, 2000). An example of one these standard assessments is the Behavioral Event Interview and Self-Assessment described by Zeichner (2000:10): [This assessment is] an hour-long interview conducted in the second semester of field experiences. Each education department member interviews two students each semester. The aim of the interview is to give students a chance to talk about their actions and thinking in relation to working with pupils. It focuses on stories elicited by questions (e.g., Can you tell me about the time you came to a realization about children’s development through an experience with a child or children?). The students then are asked to use their stories as data for a self-assessment process focusing on the five advanced education abilities (e.g., Where do you see yourself drawing upon x ability? Where do you see a need to strengthen this ability?). The interview is audiotaped and students take the tape with them to complete a written self-assessment. They set goals for their next stage of development in the teacher education program and then meet for a second session with the faculty interviewer. The final two semesters are considered the beginning of professional practice, during which student teaching occurs. To be accepted for student teaching, students must demonstrate communication ability at level 4, successfully complete all four pre-student teaching experiences, and pass another standard assessment exercise—the Professional Group Discussion Assessment. Zeichner (2000:11) also describes this assessment: Students compile a portfolio that includes a videotape of their teaching together with a written analysis of that teaching in relation to the five advanced education abilities, cooperating teacher evaluations, etc. The student then participates in a half-day interview with principals and teachers from area schools who are part of a pool of over 400 educators helping to assess students’ readiness for student teaching. Assessment as Learning The philosophy behind Alverno’s program is based on the premise that only through integrating knowledge, skills, and dispositions in observable performances can evidence of learning be shown. Assessment is treated as integral to learning. It is used both to document the development of the abilities and to contribute to candidates’ development. Alverno’s faculty describe their approach as assessment as learning (Diez et al., 1998). As practiced at Alverno, assessment as learning has the following features: Expected learning outcomes or abilities are stated, and candidates are aware of the goals toward which they are working. Explicit criteria for performance are outlined to guide candidates’ work and to provide structure for self-assessment.

OCR for page 298
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality Evaluations are based on expert judgments using evidence of candidates’ performance and weighing it against established criteria. Feedback is intended to be productive; it is not aimed at judgment alone but on ongoing development. All assessments include the experience of reflective self-assessment. Assessment is a process involving multiple performances. Candidates experience many assessments using multiple modes, methods, and times to provide a cumulative picture of their development. Assessment begins during orientation and continues through graduation. For instance, all new students complete an initial communications assessment that includes writing and presenting. The presentation is recorded on videotape, and each student evaluates her own performance before receiving diagnostic and prescriptive feedback from expert assessors. Alverno faculty believe that performance assessments should be as realistic as possible and should closely mimic the experiences of practicing teachers. In developing the curriculum, they have identified the variety of roles that teachers play. Performance assessments include simulations of parent-teacher interactions, multidisciplinary team evaluation, the teacher’s work with district or building planning, and the teacher’s citizenship role, as well as actual classroom teaching (Diez et al., 1998:2). Each course is structured around the assessments and learning outcomes that must be demonstrated to claim mastery of the course material. Box F-8 contains two examples: the learning outcomes for a course at Alverno and a course assessment designed to evaluate mastery of one of the outcomes. Coursework is intentionally sequenced to reflect developmental growth and to provide for cross-course application of concepts. For example, a mathematics methods course assessment might ask students to (1) create a mathematics lesson plan for first graders that incorporates concepts from developmental psychology, (2) teach the lesson, and (3) describe the responses of the learners and the adaptations made. Alverno requires that student teachers perform all of the duties of a teacher effectively, assuming full responsibility for the classroom for a minimum of four weeks in each placement. They start and end each day of teaching on the same schedule as the cooperating teacher. Their performance is assessed on the five professional teaching abilities by the cooperating teacher, the college supervisor(s), and the student teacher herself. Research and Program Evaluation Alverno has documented in detail the ways in which its program abilities were developed. The eight general education abilities were identified during discussions with faculty members about “what it means to say that a student has

OCR for page 298
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality BOX F-8 Learning Outcomes for Integrated Reading 3 Analyze and apply learning theory in designing and implementing literacy instruction. Assess literacy development of intermediate students and prescribe appropriate teaching strategies. Use knowledge of writing workshop and reading workshop strategies and implement workshops in the intermediate classroom. Evaluate text and trade books for use with intermediate learners. Integrate technology to enhance the writing process in the classroom. Show refined communication skills to support professional growth. Demonstrate knowledge of classroom research with understanding of the dynamics of the classroom and of how to improve practice. Assessment of Learning Outcome 2: Assess literacy development of intermediate students and prescribe appropriate teaching strategies. To assess mastery of the second outcome, students are asked to develop and use a rubric to assess actual intermediate-grade writing samples. Students analyze writing samples for strengths and weaknesses, and plan appropriate teaching strategies based on that analysis. Students then collaborate in formulating a teaching plan. Finally, they evaluate their own efforts on each component of the task. Successful performance in this assessment depends on satisfying four criteria: (1) assessing the developmental level of the learner’s performance and providing sufficient evidence to support the judgment, (2) diagnosing areas requiring attention/instruction and providing an appropriate teaching plan, (3) contributing to the group discussion of the process, and (4) assessing one’s own performance on all components of the task. Each of these four criteria is mapped to one or more of the education abilities at a specified level of performance (e.g., Diagnosis at level 5, Coordination at level 5) and noted as such in the course syllabus. Hence, assessment criteria are always publicized, and the paths connecting particular concrete activity and general abstract ability can be easily traced. Assessments and their links are reviewed on an ongoing basis by interdisciplinary teams. Similar assessments and sets of criteria exist for each level of outcome for each of the abilities. SOURCE: Adapted from Alverno College Institute (1996:7). completed a liberal arts baccalaureate degree” (Diez, 1999:41). The abilities emerged from faculty discussions focused on such questions as: What should a woman educated in liberal arts be able to do with her knowledge? How does the curriculum provide coherent and developmental support to her learning? What counts as evidence that a student has achieved the expectations of the degree, the major, and the professional preparation? Once the abilities were defined, the discussions centered on pedagogical, developmental, and measurement ques-

OCR for page 298
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality tions. Descriptions of the abilities and corresponding performance levels have evolved over the years as a result of many factors. There is ongoing interdisciplinary consultation and review. Each faculty member is expected to serve on a program evaluation and development committee. Alverno’s performance-based assessments are designed to reflect the actual work of beginning professional teachers. They cover the content and skills considered relevant to the tasks that teachers are expected to perform. In addition, the context for assessments is intended to reflect real-life teaching situations, representing a broad sample of performance situations (broader than would be expected for assessments that focus on basic skills and subject matter knowledge). Committees of faculty members routinely audit the contents of the assessments (during regularly scheduled departmental meetings) to verify that they are appropriate and that they reflect current thinking about what teachers should know and be able to do. Multiple judgments are obtained of each student’s skills and knowledge. Each student is observed and assessed hundreds of times as they participate in classroom and field activities. Evaluations utilize multiple contexts, multiple modes, and multiple evaluators. There are formal, “milestone” assessments staged at relevant points in the curriculum, such as the Behavioral Event Interview and Self-Assessment and the Professional Group Discussion Assessment described above, as well as less formal, ongoing, in-class assessments. The institution has processes in place for refining and updating criteria for judging students’ performance. There have been both internal and external reviews of the program. The institution has maintained an Office of Research and Evaluation since 1976 (Mentokowksi, 1991). Through this office, a comprehensive longitudinal study of 750 students was conducted that tracked students from entry into the program through two years after graduation. For this study, researchers collected information on (1) student performance in the curriculum on college-designed ability measures; (2) student perceptions of reasons for learning, the process of learning, and its value for their own career and life goals; and (3) students’ personal growth after graduation (Mentkowski and Doherty, 1984). Another piece of the longitudinal study involved collecting data from a group of alumnae five years after graduation. This research focused on how abilities learned in college transferred to the work setting, the extent to which alumnae continued to grow and develop after college, and how graduates were doing in their careers and further learning in the long term. This study used multiple research instruments (17) to collect data, including questionnaires, tests, essays, and interviews (Alverno College, 1992–1993). In addition, surveys have collected data from Alverno graduates on their perceptions about the college’s program and the extent to which they felt prepared to teach upon graduation (Zeichner, 2000). Surveys and focus groups with principals have examined employers’ perceptions of the preparedness of Alver-

OCR for page 298
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality no graduates (Zeichner, 2000). Surveys have also compared longevity in the teaching field for Alverno alumnae with graduates of other programs. Licensing Practices in Wisconsin Wisconsin currently requires that students admitted into its teacher education programs pass a basic skills test. Alverno also has developed its own assessments for reading, listening, writing, speaking, and quantitative literacy through four levels, and these are required for graduation. Entering the classroom requires endorsement from the institution from which one graduates. Alverno uses successful completion of field experiences and course requisites and its many formal and informal assessments of students as the basis for warranting readiness to teach. In 2004 the state of Wisconsin will require candidates for teaching positions to present portfolios demonstrating their performance in relation to several professional teaching standards. Candidates who pass the portfolio review will be granted provisional licenses to teach for three to five years while pursuing professional development goals related to the standards. The portfolios are meant to comprise performance assessments compiled near the time of graduation. In addition, the state’s standards make provision for a yet-to-be-specified content examination.