Appendix F Alternative Assessment Case Studies
PERFORMANCE ASSESSMENT OF EXPERIENCED TEACHERS BY THE NATIONAL BOARD FOR PROFESSIONAL TEACHING STANDARDS
The National Board for Professional Teaching Standards (NBPTS) provides an example of a large-scale, high-stakes performance assessment of teaching that draws on portfolio and assessment center exercises. While NBPTS assessments are intended for voluntary certification of experienced teachers (teachers who have been practicing in their subject areas for at least three years), this case is relevant to the committee’s focus on assessment of beginning teaching in at least two ways. First, it provides an example of a model that states may choose or have chosen to emulate. Second, states may decide to grant licenses to candidates certified by the NBPTS. Finally, NBPTS certification is viewed by its proponents as an integral phase in a teacher’s career development that can and should be consonant with earlier phases of assessment and development.
The NBPTS is an independent organization governed by a 63-member board of directors, most of whom are classroom teachers. Its mission is to “establish high and rigorous standards for what accomplished teachers should know and be able to do, to develop and operate a national voluntary system to assess and certify teachers who meet these standards, and to advance related education reforms for improving student learning in American schools” <www.nbpts.org>. In addition, NBPTS seeks to forge “a national professional consensus, to reliably identify teachers who meet [these] standards,” and to communicate what accomplished teaching looks like” (Moss and Schutz, 1999:681). The NBPTS is in the
process of developing standards and assessments for more than 30 certification fields identified by the subject or subjects taught and by the developmental level of the students.
For each certificate, development work starts by articulating a set of content standards, based on the five core propositions set forth in the NBPTS central policy statement (National Board for Professional Teaching Standards, 1996:2– 3; see Box F-1). The drafting of the content standards for each certificate based on these core propositions is handled by a committee comprised primarily of teachers experienced in the relevant subject area along with experts in child development, teacher education, and the relevant academic discipline. Public review and comment are obtained for the content standards, and the feedback received is used in the final revision of the standards (Moss and Schutz, 1999).
The final version of a content standards document states that the standards “represent a professional consensus on the critical aspects of practice that distinguish exemplary teachers in the field from novice or journeymen teachers. Cast in terms of actions that teachers take to advance student outcomes, these standards also incorporate the essential knowledge, skills, dispositions, and commitments that allow teachers to practice at a high level” (e.g., National Board for Professional Teaching Standards, 1996:1). Moss and Schutz (1999:682–683) describe the Early Adolescence/English Language Arts Standards:
[T]here are 14 distinct Early Adolescence/English Language Arts (EA/ELA) Standards…Standard II states that:
“Accomplished EA/ELA teachers set attainable and worthwhile learning goals for students and develop meaningful learning opportunities while extending to students an increasing measure of control over how those goals are pursued.” . .
This standard is then elaborated into a full-page description that includes statements such as, “Educational goal-setting is an interactive process in the middle-grades English teacher’s classroom…. These activities often include a strong mixture of student involvement and direction”; “in carrying out learning direction activities, accomplished teachers adjust their practice, as appropriate, based on student feedback and provide many alternative avenues to the same learning destinations”; or “the planning process is inclusive, no one is allowed to disappear.”
Box F-2 provides an overview of the standards for EA/ELA. These content standards are then used to guide all aspects of assessment development for that certification assessment.
Once the content standards are created for any given certificate, the assessment developers (now primarily at Educational Testing Services, or ETS) are joined by a second committee of approximately eight experienced teachers in the development and pilot testing of assessment tasks and the accompanying rubrics that will be used to score candidates’ performances. Although each assessment
BOX F-1 Five Propositions of Accomplished Teaching
The National Board for Professional Teaching Standards seeks to identify and recognize teachers who effectively enhance student learning and demonstrate the high level of knowledge, skills, abilities and commitments reflected in the following five core propositions.
SOURCE: National Board for Professional Teaching Standards, 1996.
BOX F-2 Standards Overview: Early Adolescence/ English Language Arts (EA/ELA)
The following standards are presented as facets of the art and science of teaching English language arts to young adolescents. They are an analytical construct, created to provide a closer accounting of the critical aspects of accomplished practice. However, in real time these segments of teaching occur concurrently because teaching is a seamless activity with many disparate purposes being served in the classroom at any given moment.
Preparing the Way for Productive Student Learning
Advancing Student Learning in the Classroom
Supporting Student Learning through Long-Range Initiatives
SOURCE: National Board for Professional Teaching Standards, 1996.
task is designed in light of the standards specific to a given certificate, NBPTS has adopted a framework that generalizes across certificates to guide the work of all development teams.
Each assessment consists of two major parts: a portfolio to be completed by candidates in their home schools and a half-day of testing at an assessment center. The school-based portfolio consists of (1) three entries that are classroom based and include two videos that document the candidate’s teaching practice through student work and (2) one entry that combines the candidate’s work with students’ families, the community, and collaboration with other professionals. The six assessment center exercises require candidates to demonstrate their knowledge of subject matter content.
Box F-3 provides a detailed overview of the EA/ELA assessment tasks. A formal multistate pilot test is administered before the assessments are released for the first operational use.
Each assessment task is scored in accordance with a rubric prepared during the task development phase. NBPTS scoring rubrics encompass four levels of performance on a particular task, with the second-highest level designated as meeting the standards of accomplishment.1 Further, the terminology used in each scoring rubric closely mirrors the language of the relevant content standards.
Following initial use of the assessment, at least three extensively trained assessors and other teaching experts select a sample of responses to be used in training and certifying the scorers. The small group charged with selecting the
BOX F-3 Early Adolescence/English Language Arts
The NBPTS has developed standards for what accomplished Early Adolescence/ English Language Arts teachers should know and be able to do. This certificate is designed for teachers of students ages 11–15. The assessment is performance-based and designed to evaluate the complex knowledge and skills of teaching described in the standards. The assessment process consists of two components: the portfolio entries and the half-day assessment center exercises. The certification decision is based on candidate performance as judged against the NBPTS standards for accomplished practice.
The portfolio for the Early Adolescence/English Language Arts assessment provides teachers with the opportunity to present a sample of their actual classroom practice over a specified time period. The portfolio consists of four entries*:
The Assessment Center
The half-day of assessment center exercises examine subject matter content. There are six written exercises, and teachers are given 30 minutes to complete each. One score is reported for each exercise. The exercises measure knowledge in the following areas:
• exercises are currently undergoing revision; these descriptions are subject to change
SOURCE: NBPTS Next Generation Certificate Overviews, April 2001.
benchmarks is instructed to choose them so that assessors see that there are different ways to achieve a score. Those who select benchmarks and score portfolios are subjected to a series of training exercises designed to assist them in identifying and controlling any personal biases that might influence their selection.
Most exercises are scored independently by two assessors with a third, more experienced assessor used to adjudicate scores that differ by more than a prespecified amount. For some exercises, where interrater reliability is deemed sufficient, only a sample of exercises is double scored. A weighting strategy is employed to combine scores on the exercises and to form the total scores on the assessment. This weighting strategy is selected by another committee of teachers who must choose from among four predetermined weighting strategies. To make certification decisions (accomplished/not accomplished), the total score is compared to a predetermined passing score that is uniform across certificates. This performance standard is equivalent to receiving a just-passing score on each
of the exercises, although high scores on one exercise can compensate for low scores on others.
The uniform performance standard set by the NBPTS was based on a series of empirical standard-setting studies. These studies explored a number of different standard-setting methods (see Jaeger, 1998; Educational Testing Service, 1998). In all of the investigations, panels of experienced teachers were asked to make decisions based on profiles of scores across exercises. The process was “typically iterative, in which individual panelists—thoroughly familiar with the tasks, rubrics, and standards—were given an opportunity to discuss and revise their individual decisions about score profiles in light of feedback on other panelists’ decisions and on the practical implications of their own decisions. From this set of revised individual decisions, the assessment developers computed the ‘recommended’ performance standard” (Moss and Schutz, 1999:685). The NBPTS adopted the uniform performance standard for all certificates based on the results of these early studies along with consideration of the practical implications of setting the standard at different levels (e.g., minimizing adverse impact for groups of candidates, minimizing the anticipated proportion of candidates who are misclassified as failing the exam due to measurement error).
In addition to documentation of the development process, five other kinds of validity evidence are routinely gathered and examined for each assessment:
Content-related evidence of validity is examined by convening a panel of experienced teachers in the subject area to independently rate (a) the extent to which each of the content standards describes a critical aspect of highly accomplished teaching and (b) the importance and relevance of each exercise and rubric to each content standard and to the overall domain of accomplished teaching.
“Scoring validation,” as defined by the assessment developers, is evaluated by assembling another panel of experienced teachers in the subject area to rank randomly selected pairs of exercise responses. These rankings are then compared to the rankings obtained from the official scoring.
Information regarding reliability and errors of measurement is reported as (a) error associated with scores given by different assessors, (b) error associated with the sampling of exercises, and (c) misclassification estimates (i.e., estimates of the proportion of candidates incorrectly passing and failing) due to both of these sources of error.
Confirmation of the predetermined passing standard is obtained by convening a panel of experienced teachers, who have also served as assessors. The panel examines different possible profiles of exercise scores, rank ordered by total score, and draws lines where they believe the passing standard should be.
Evidence regarding bias and adverse impact is provided by (a) reporting certification rates by gender, ethnicity (where sample sizes permit), and teaching
context and (b) investigating the influence of having exercises scored by assessors of the same and different ethnicity as the candidate.
The test developer, ETS, publishes a technical manual (Educational Testing Service, 1998) that describes the methodology for these studies, provides annual updates of technical information for each certificate, and publishes outcomes for each administration. The technical manual is available to the public. The annual updates are available only to NBPTS members, staff, consultants, and advisory panel members. Special studies are frequently presented at national conferences. The NBPTS’s validity research agenda, technical reports, and reports of special studies are routinely reviewed and commented on by the Measurement Research Advisory Panel, a panel of independent scholars whose backgrounds are primarily in educational measurement.
In addition to these routine studies, NBPTS has undertaken a number of special validity/impact studies, many of which are still in progress. These include (1) an external review of complete portfolios by a diverse panel exploring differences in performance between African American and white candidates (Bond, 1998); (2) a study of rater severity and the effects of different scoring designs (Englehard, et al., 2000); (3) a survey of how teachers prepare for the exam (Lynch et al., 2000); (4) a survey of assessors’ impressions about the impact of training and scoring on their professional development (Howell and Gitomer, 2000); and (5) a comparison of NBPTS evaluations with others based on classroom observations and interviews (Wylie et al., 2000). A recently released study compared the teaching practices of NBPTS certified teachers with other teachers and compared samples of student work from classrooms of the two groups of teachers (Bond et al., 2000).
NBPTS assessments are voluntary; passing the assessment results in a certification of accomplishment. To be eligible for NBPTS certification, a teacher must have completed a baccalaureate degree; must have a minimum of three years of teaching experience at the early childhood, elementary, middle, or secondary levels; and must have a valid state teaching license for each of those years or, where a license is not required, the teacher must be teaching in a school recognized and approved to operate by the state.
For the 2000–2001 testing year, the examination fee was $2,300. A number of states and other education agencies have programs in place to subsidize this cost, and NBPTS publicizes that information on its website. Candidates are given approximately 10 months to complete the portfolio tasks and attend the assessment centers. Since each exercise is scored independently, a candidate who does not pass the assessment may bank scores on the exercises passed and retake exercises on which a failing score was received (below a 3 on a four-point scale) for a period of two years after being notified of his or her initial scores. NBPTS publishes a list of newly certified teachers each year. States and local education agencies have their own policies about how the scores are used, in-
cluding how teachers are recognized and rewarded for receiving NBPTS certification. These include the granting of a state license for teachers transferring into the state, salary increases and bonuses, and opportunities to assume new roles. (See <www.nbpts.org> for a summary of state incentives.)
The NBPTS’s direct involvement in professional development and support activities is limited. For instance, the board offers short-term institutes for “facilitators” who plan to support candidates for certification and a series of “Teacher Development Exercises” that can be purchased and used in local workshops. NBPTS does, however, work informally with state and local education agencies and with teacher education institutions to support and publicize local initiatives (see Box F-4). For instance, NBPTS provides a list of ways that state and local education agencies and institutions might support its work and offers contact information for local agencies to interested teachers. In addition, NBPTS’s web page contains information about the activities of its affiliates and provides examples of the variety of professional roles that board-certified teachers have assumed. It should be noted that while NBPTS supplies information on local contacts and activities, it does not monitor the quality, relevance, or usefulness of this information.
Through its certification assessments and related activities, NBPTS hopes to “leverage change” in the contexts and culture of teaching. The board hopes to (1) make “it possible for teachers to advance in responsibility, status, and compensation without having to leave the classroom” and (2) encourage “among teachers the search for new knowledge and better practice through a study regimen of collaboration and reflection with peers and others” (National Board for Professional Teaching Standards, 1996:7).
CONNECTICUT’S TEACHER PREPARATION AND INDUCTION PROGRAM
Connecticut, working in collaboration with the Interstate New Teacher Assessment and Support Consortium (INTASC), provides an example of a state that has implemented a licensing system that relies on performance assessments. Connecticut’s Beginning Educator Support and Training program is a comprehensive three-year induction program that involves both mentoring and support for beginning teachers as well as a portfolio assessment. The philosophy behind Connecticut’s teacher preparation and induction program is that effective teaching involves more than demonstration of a particular set of technical skills. The program is based on the fundamental principle that all students must have the opportunity to be taught by a caring, competent teacher and that, in addition to command of subject matter, effective teaching requires a deep concern about students and their success, a strong commitment to student achievement, and the
BOX F-4 Support for Teachers Seeking NBPTS Certification
Teachers who seek—and those who achieve—NBPTS Certification deserve recognition for their efforts and accomplishments. More and more school districts, state legislatures, state boards of education and colleges of education are supporting and encouraging teachers in their quests for NBPTS Certification. There are many ways policymakers can help.
BEFORE a teacher seeks NBPTS Certification
DURING a teacher’s participation
AFTER a teacher achieves NBPTS Certification
States and State Departments of Education
AFTER a teacher achieves NBPTS Certification, states can…
Undergraduate Colleges of Education
BEFORE a teacher achieves NBPTS Certification…
Colleges of education can base program curriculum on the propositions of the NBPTS for Professional Teaching Standards.
AFTER a teacher achieves NBPTS Certification…
Colleges of education can recognize certification as criterion for adjunct faculty.
SOURCE: National Board for Professional Teaching Standards, 1996.
conviction that all students can attain high levels of achievement (Connecticut Department of Education, 2000). This philosophy is reflected in Connecticut’s Common Core of Teaching (CCT), which is intended to present a comprehensive view of the accomplished teacher, detailing the subject-specific knowledge, skills, and competencies the state believes teachers need in order to ensure that students learn and perform at high levels. The CCT encompasses: (1) foundational skills and competencies common to all teachers from prekindergarten through grade 12 and (2) discipline-based professional standards that represent the necessary knowledge, skills, and competencies (Connecticut Department of Education, 1999). The specific components of the CCT appear in Box F-5.
Connecticut’s program covers three aspects of teachers’ development: preservice training, beginning teacher induction, and teacher evaluation and continuing professional growth. The CCT guides state policies related to each of these phases, which are described below.
At the preservice phase, teacher education programs are expected to demonstrate that teacher candidates are knowledgeable about the CCT as well as the state’s achievement test batteries, the Connecticut Mastery Tests, and the Connecticut Academic Performance Test (Connecticut State Department of Education, 1999). The training requirements for prospective teachers are specified in terms of a set of standards, as distinct from a list of required courses. The standards encompass the body of knowledge and skills the state believes individuals should develop as they progress through the teacher education programs. The approval process for teacher education programs is based on these standards, and teacher education programs are expected to demonstrate that students achieve them. Prospective teachers in Connecticut must pass Praxis I, have a minimum B minus average to enter a teacher preparation program, and must pass Praxis II to be recommended for initial licensure (Connecticut State Board of Education, 2000).
Beginning Educator Support and Training (BEST) Program
BEST is a comprehensive three-year induction program that is required for all beginning teachers. The program has two components: (1) support and instructional assistance through mentorship, seminars, distance learning, and support teams over a two-year period and (2) assessment of teaching performance through a discipline-specific portfolio assessment (Connecticut State Board of Education, 2000). The goals of the BEST program include:
ensuring that all students have high-quality, committed, and caring teachers;
promoting effective teaching practice leading to increased student learning;
BOX F-5 Components of Connecticut’s Common Core of Teaching
Connecticut’s Common Core of Teaching: Foundational Skills and Competencies
SOURCE: Connecticut Department of Education [May 7, 2001].
providing effective support and feedback to new teachers so that they continue to develop their knowledge base and skills and choose to remain in the profession;
providing standards-based professional development for both novice and experienced teachers; and
developing teacher leaders by recognizing and relying on experienced teachers to support, assess, and train beginning teachers.
As part of the program, school administrators assign new teachers to mentors, provide opportunities for new teachers to work collaboratively with more senior teachers, provide time for professional development, ensure that new teachers have access to resources and support structures, evaluate teachers and offer constructive feedback, and provide an ongoing orientation program <www.state.ct.us/sde>.
During their first year, beginning teachers are evaluated at the district level through live and video observations. They learn how to systematically document their teaching practices and to use student work to demonstrate student learning, and they develop a long-term plan for professional growth. During year two, teachers use feedback from year one to improve their teaching techniques, and they begin to plan for portfolio development. With guidance from their principal, teachers design a professional development plan that supports the BEST portfolio process and that uses the portfolio components as an avenue to support growth in planning, instructing, student work analysis, and reflecting. The portfolio is submitted in the spring of the second year. During the third year of the induction phase, teachers continue to work on the portfolio, as needed, and share their portfolio experience with other beginning teachers. Third-year teachers begin to expand their goals to focus on district and school improvement goals. They also begin to document their professional responsibilities in preparation for professional growth and tenure <www.state.ct.us/sde>.
The Portfolio Assessment
The portfolio assessment is intended to provide a thorough representation of a teacher’s performance through documentation of teaching over time and by focusing on a specific content/discipline area. In the portfolio, teachers document their methods of lesson planning, teaching and facilitation of student learning, assessment of student work, and self-reflection within a unit of instruction. The portfolio includes evidence from multiple sources, such as lesson plans, videotapes of teaching, teacher commentaries, examples of student work, and formal and informal assessments (Connecticut Department of Education, 1999). Portfolio assessment requirements vary depending on the beginning teacher’s teaching assignment and license endorsement. Content-focused seminars, run by trained teachers, help beginning teachers learn about ways to meet subject spe-
cific standards so as to demonstrate content-specific teaching practices (Connecticut State Department of Education, 1999:4).
Scorers are trained to evaluate the portfolios using criteria based on content-focused professional teaching standards. Each portfolio is evaluated by at least two experienced educators with extensive teaching experience in the same disciplinary areas as the beginning teacher. Scorers receive up to 70 hours of training and must meet a proficiency standard in order to be eligible to score a portfolio. In scoring the portfolio, scorers first review the included materials to make notes about the evidence provided. Scorers then organize the evidence around a series of guiding questions derived from the discipline-based standards. The two scorers independently evaluate the quality of the teaching documented in the portfolio according to scoring rubrics and benchmarks and then convene and decide on a final portfolio score. Any portfolio that does not meet the standard of “Competent” is rescored by another pair of assessors. If the second evaluation results in a different score, a “Lead” assessor adjudicates the final score. Teachers whose portfolios do not meet the competency standards are eligible for a personal conference with a portfolio assessor during which they receive individualized feedback about the evaluation (Connecticut State Board of Education, 2000). Such teachers may submit another portfolio during the third year of teaching. If a teacher fails to meet the standard by the third year, the teacher is ineligible to apply for a provisional certificate and cannot teach in Connecticut public schools. To regain certification, an individual needs to successfully complete a formal program of study as approved by the state.
Support for Mentors
School districts are responsible for appointing mentors to beginning teachers. Mentors must have at least three years of teaching experience and must be willing to make the time commitment involved in mentorship activities. Mentors are required to attend a series of training workshops designed to help them prepare for their role in offering guidance to the beginning teacher and in preparing for the portfolio assessment. Mentors and support team leaders receive 20 hours of training focused on the CCT and 10 hours of annual update training.
Throughout the continuous professional growth phase, the components of the CCT are used to establish standards for the evaluation of teachers (according to the Guidelines for Comprehensive Professional Development and Teacher Evaluation) and to guide teachers in selecting appropriate professional development to meet individual as well as local district goals (Connecticut Department of Education, 1999). Teachers who hold either a provisional or professional license must also complete 90 hours of professional development every five
years to maintain their licenses. Professional development activities should be directly related to improving teaching and learning.
Role of Other Staff
A number of staff play a role in Connecticut’s induction and evaluation program. The school principal is responsible for ensuring that beginning teachers are aware of the BEST program’s requirements. Principals facilitate opportunities for mentors and beginning teachers to meet (and to observe each other’s classrooms); assist with arranging classroom coverage; and provide resources for portfolio preparation (e.g., videotaping equipment). District facilitators are responsible for providing BEST program orientation sessions, ensuring that beginning teachers receive adequate support from mentors, and arranging for release time so that the mentor and beginning teacher can meet. Release time is acquired by using substitute teachers for the beginning teacher and the mentor(s). The state department of education (through six regional educational service centers) provides distance learning seminars to train mentors and portfolio assessors on the portfolio assessment. The department manages the program; sets policies, standards, and procedures; and assures that the assessment portfolios are reliable and valid.
Studies of Technical Characteristics
The state has conducted numerous studies associated with developing the portfolio assessment, evaluating its psychometric characteristics, and assessing its overall impact on teacher competence in the state. For all subject areas, the discipline-specific standards were developed by committees of experts in the particular subject area, including teachers, curriculum specialists, administrators, and higher-education representatives. As part of the development process, job analysis studies were conducted. Representative samples of teachers, administrators, and higher-education faculty participated in surveys in which they rated the standards in terms of their importance for beginning and experienced teachers, their relevance to the job of teaching, and their importance for advancing student achievement. The standards were further validated by having teachers complete journals to document the degree to which the standards were represented in their actual teaching.
Experts’ judgments and portfolio performance data are used in setting the standards for the portfolio assessment and determining the cutpoints. Bias and sensitivity reviews are conducted for the portfolio handbook (the guidelines given to beginning teachers for constructing their portfolios and all related scoring materials), and portfolio performance is broken down by race, gender, type of community, and socioeconomic status to examine disparate impact. Generalizability studies are implemented to estimate reliability of portfolio scores.
A number of studies have been conducted to collect construct-related evidence of validity. These studies have examined the relationships between teachers’ performance on the portfolio and (1) other quantitative measures, such as their grade point averages, SAT scores, and Praxis I and II scores; (2) case studies of teachers’ performance in the classroom; and (3) student achievement test results in English/language arts, reading, and mathematics.
Additional studies have examined the consequences associated with program implementation. Program effectiveness is evaluated through surveys of mentors, portfolio assessors, school administrators, principals, central office personnel, beginning teachers, and higher-education faculty. In addition, portfolio scorers evaluate their own teaching skills before and after participating in scorer training activities. This information is used qualitatively to judge the impact of scorer training on actual classroom teaching practices.
OHIO’S TEACHER INDUCTION PROGRAM
Ohio provides an example of a state that plans to incorporate a commercially available program into its licensing system. Ohio is in the process of redesigning its licensing system. The new system, which will be implemented in 2002, includes a comprehensive induction program for beginning teachers, requirements for performance assessment during the induction year administered by the Ohio Department of Education, and procedures to ensure continued professional development for experienced teachers. The new system will eliminate procedures through which teachers were awarded permanent “lifetime” licenses and instead will implement a two-stage system. During the first stage, individuals who complete an approved teacher preparation program and pass required tests (Praxis II, Principles of Teaching and Learning, and the appropriate Praxis II subject matter examination) receive a provisional license. The provisional license is good for two years, is renewable, and entitles individuals to work as full-time or substitute teachers. Teachers who successfully complete Ohio’s new Entry Year Program and pass a performance assessment (Praxis III) will receive a professional license (the second stage). The professional license is good for five years, and renewal requires completion of an approved professional development plan and a master’s degree or the equivalent after 10 years of teaching.
The Entry Year Program
The new system will require beginning teachers to participate in the Entry Year Program, an induction program designed to provide mentorship, support, and additional learning experiences for new teachers. Ohio has spent the past seven years piloting the Praxis III Classroom Performance Assessment for be-
ginning teachers and approaches for teacher induction programs and is in the process of standardizing approaches to its Entry Year Program. Development and implementation of an Entry Year Program will be the responsibility of individual school districts, but the state will provide guidelines and financial support for development activities. Districts are required to develop an implementation plan, provide orientation sessions, identify and train mentors, arrange for release time for mentors to work with entry-year teachers, conduct assessment activities, and coordinate with higher-education institutions.
As part of the Entry Year Program, experienced teachers serve as mentors to beginning teachers. Whenever possible, mentors teach the same subject or grade level and are located in the same building as the beginning teacher. Serving as a mentor can be incorporated into a veteran teacher’s professional development plan and can count toward licensure renewal. Mentor teachers work on an ongoing basis with beginning teachers. They engage in activities designed to help beginning teachers develop their skills in instructional planning and preparation, presentation of various learning activities, and assessment of students’ learning.
According to state guidelines, individuals selected to be mentors should be experienced teachers and should demonstrate an awareness of instructional methods and professional responsibilities needed to improve teaching skills and increase student learning. Districts are required to operate an in-depth training program for mentors on ways to conduct observations, provide feedback, and offer professional guidance and support to beginning teachers.
Ohio is focusing on using ETS’s PATHWISE Induction Program-Praxis III Version as the basis for its Entry-Year Program. In Ohio this program is called the Ohio FIRST (Formative Induction Results in Stronger Teaching) Year Program.
The PATHWISE Induction Program-Praxis III Version
The PATHWISE Induction Program-Praxis III Version grew from a need for mentors to assist new teachers to focus on successful teaching under the Praxis III framework (D.Gitomer, ETS, personal correspondence, July 1999). The two pieces of the program—the observation/induction system (provided by the district) and the performance assessment (administered by the Ohio department of education)—are built on the same domain of teaching skills. These domains are:
Organizing content knowledge for student learning—how teachers use their understanding of students and subject matter to establish learning goals, design or select appropriate activities and instructional materials, sequence instruction, and design or select evaluation strategies.
Creating an environment for student learning—the social and emotional
components of learning as prerequisites to and context for academic achievement, including classroom interactions between teachers and students and among students.
Teaching for student learning—making learning goals and instructional procedures clear to students, making content comprehensible to students, encouraging students to extend their thinking, monitoring students’ understanding of content, and using instructional time effectively.
Teacher professionalism—reflecting on the extent to which the learning goals were met, demonstrating a sense of efficacy, building professional relationships with colleagues, and communicating with parents or guardians about student learning.
Each domain consists of a set of four or five assessment criteria (for a total of 19) that represent critical aspects of teaching (see Box F-6). These criteria were developed over a six-year period based on information collected from job analyses, reviews of empirical and theoretical research, and examinations of states’ licensing requirements for teachers (Wesley et al., 1993).
The job analyses involved asking teachers and others familiar with teaching about the importance of the various tasks beginning teachers perform. The job analyses were conducted separately for elementary (Rosenfeld et al., 1992b), middle (Rosenfeld et al., 1992c), and secondary school teachers (Rosenfeld et al., 1992a). A series of literature reviews were also conducted to document what is known both empirically and theoretically about good practice for teachers in general and for beginning teachers in particular (Dwyer, 1994). Finally, information on the teacher licensing requirements of all 50 states and the District of Columbia was compiled. A nationwide content analysis of state performance assessment requirements also was carried out to determine the content overlap among the systems and to highlight distinctive differences (Dwyer, 1994).
Based on the findings of these studies, the test developers drafted an initial set of assessment criteria which were presented to a national advisory committee for comment. Committee comments led to revisions in the criteria, followed by a pilot test of the criteria in two states (Minnesota and Delaware). The results of this field testing formed the basis for finalizing the assessments made available for inspection and initial use by states in the fall of 1992.
According to the ETS developers, the criteria were developed so as to “infuse a multicultural perspective throughout the system” (Dwyer, 1994:4). This perspective is based on the premise that “effective teaching requires familiarity with students’ background knowledge and experiences (including their cultural resources), and effective teachers use this familiarity to devise appropriate instruction” (p. 4). The criteria are intended not to prescribe a particular way of teaching but to allow for flexibility in how they can be demonstrated in various classroom contexts.
BOX F-6 PATHWISE Induction Program-Praxis III Version Assessment Criteria
Domain A: Organizing Content Knowledge for Student Learning
A1: Becoming familiar with relevant aspects of students’ background knowledge and experiences.
A2: Articulating clear learning goals for the lesson that are appropriate to the students
A3: Demonstrating an understanding of the connections between the content that was learned previously, the current content, and the content that remains to be learned in the future.
A4: Creating or selecting teaching methods, learning activities, and instructional materials or other resources that are appropriate to the students and that are aligned with the goals for the lesson.
A5: Creating or selecting evaluation strategies that are appropriate for the students and that are aligned with the goals of the lesson.
Domain B: Creating an Environment for Student Learning
B1: Creating a climate that promotes fairness
B2: Establishing and maintaining rapport with students
B3: Communicating challenging learning expectations to each student
B4: Establishing and maintaining consistent standards of classroom behavior
B5: Making the physical environment as safe and conducive to learning as possible
Domain C: Teaching for Student Learning
C1: Making learning goals and instructional procedures clear to students
C2: Making content comprehensible to students
C3: Encouraging students to extend their thinking
C4: Monitoring students’ understanding of content through a variety of means, providing feedback to students to assist learning, and adjusting learning activities as the situation demands.
C5: Using instructional time effectively
Domain D: Teacher Professionalism
D1: Reflecting on the extent to which the learning goals were met
D2: Demonstrating a sense of efficacy
D3: Building professional relationship with colleagues to share teaching insights and to coordinate learning activities for students
D4: Communicating with parents or guardians about student learning
SOURCE: Educational Testing Service, 1995c. Used by permission of the author.
Praxis III: Classroom Performance Assessment
Praxis III is a performance assessment designed to measure beginning teachers’ skills in relation to 19 criteria (see Box F-6). The Praxis III assessment uses
three data collection methods: (1) direct observation of classroom practice, (2) written descriptions of students and lesson plans, and (3) interviews structured around the classroom observation. Prior to being observed, the beginning teacher provides the trained assessor with written documentation about the general classroom context and the students in the class, as well as specific information about the lesson to be observed. The observation allows assessors to gain a first-hand understanding of the teacher’s practices and decisions. The written documentation provides a sense of the general classroom context and the students in the class as well as specific information about the lesson to be observed. Semistructured interviews with the teacher before and after the observation provide an opportunity to explore and reflect on decisions and teaching practices. The interviews also allow assessors to evaluate the teacher’s skill in relating instructional decisions to contextual factors, such as student characteristics and prior knowledge (Dwyer, 1994:2–3).
The design of Praxis III was based on the premise that “effective teaching requires both action and decision making and that learning is a process of the active construction of knowledge.” This guiding conception makes explicit the belief that “because good teaching is dependent on the subject matter and the students, assessments should not attempt to dictate a teaching method or style that is to be applied in all contexts” (Dwyer, 1994).
PATHWISE Induction Program
The PATHWISE Induction Program is designed to prepare beginning teachers for the Praxis III assessment and to provide opportunities for professional development growth. The PATHWISE system is intended to be a “flexible system responsive to an individual’s personal teaching style,” incorporating “constructive assessment that fosters growth and professional development in students and first-year teachers by recognizing their strengths as well as their weaknesses” (Educational Testing Service, 1995b:3). The system provides opportunities for beginning teachers to interact with mentors to (1) identify their strengths and weaknesses, (2) develop a plan for improving their teaching skills, and (3) improve their skill in reflecting on their teaching practices.
The program consists of a series of structured interactions between beginning teachers and their mentors. The interactions are referred to as “events,” and the activities associated with each event are designed to encourage collaboration between beginning teachers and their mentors. A brief description of the types of events follows:
The initial interaction is called the Teaching Environment Profile. This event requires beginning teachers to examine the context of their teaching by collecting information about their students and about the environment (including the district and the community) within which the school is located.
The next interaction is called Profiles of Practice—Observation of Classroom Practice. Two observation events take place in which the mentor teacher observes the beginning teacher leading an instructional lesson. The mentor then provides feedback based on the observation and on review of planning materials, oral and written reflections, and examples of student work.
Three inquiry tasks are also scheduled for beginning teachers. These tasks require beginning teachers to explore specific aspects of their teaching practice. To complete each event, beginning teachers must gather information about a selected aspect of teaching using a variety of resources, including their colleagues, research journals, and texts. Together with the mentor, the beginning teacher develops a plan of action to try out in the classroom, implements the plan, and reflects on the experience. The inquiry events focus on ways to establish a positive classroom environment, design an instructional experience, and analyze student work.
Teachers also participate in two events called Individual Growth Plans. The individual growth plans are designed to help beginning teachers determine how best to focus their efforts throughout the entire induction process. To complete these events, beginning teachers must prepare a plan for professional learning that takes into account their teaching practices, school or district initiatives, and other challenges they may face.
The final induction activities are referred to as Closure Events: Assessment and Colloquium. These interactions help bring closure to the year by engaging beginning teachers in self-assessment, encouraging a final evaluation of professional learning, and promoting the sharing of professional knowledge with other beginning teachers.
A total of 10 events occur over the course of the school year. All of the inquires and observations make use of planning, teaching, reflecting, and applying what is learned. The activities encourage beginning teachers to participate in reflective writing, conversations with experienced colleagues, and ongoing examination of teaching in relation to student learning (Educational Testing Services, 1995a).
Training of Mentors and Assessors
Since the PATHWISE Induction Program-Praxis III Version relies on effective use of observational data, extensive training is required. ETS offers three levels of training for the program. An initial two-day training in the PATHWISE Classroom Observation system acquaints individuals with the 19 criteria that form the basis for the program and is a prerequisite for other levels of training. Participants receive instruction in recording observational data, analyzing written contextual information, using written and observational information to evaluate performance, writing summaries of teacher performance, and providing
feedback to beginning teachers. Training relies on simulations, case studies, sample evaluation forms, and videotapes.
A four-day training session familiarizes individuals with the PATHWISE Induction Program-Praxis III Version. This seminar focuses on training mentors in the uses and purposes of 10 ten events and in developing their mentoring and coaching skills.
The seminar for Praxis III assessors is the final level of training and is five days in duration. The training consists of a series of structured activities during which participants learn to recognize the presence of each of the 19 Praxis III criteria. Participants learn to evaluate written information provided by the teacher, take notes during classroom observations, and conduct semistructured interviews. The training process utilizes a variety of stimuli, including worksheets, sample records of evidence, simulations, case studies, and videotapes. Trainees receive feedback from instructor and fellow participants. They also engage in professional reflection through journal writing.
Implementation of Pathwise Induction Program-Praxis III Version in Ohio
ETS offers training for mentors and assessors but also provides instructional modules for individuals to learn how to conduct the training sessions. Individuals in Ohio have learned the ETS procedures, and the state now offers its own training sessions.
Training in the PATHWISE Classroom Observation System has been incorporated into Ohio’s preservice program to introduce students to the 19 criteria. Many state institutions have incorporated the domains into their preservice education programs, and their focus will be on using the PATHWISE rubrics to evaluate students’ progress. PATHWISE Classroom Observation System training is also available for teachers who plan to become mentors. Approximately 15,000 teachers have completed this training (John Nickelson, Ohio Department of Education, personal communication, March 19, 2001).
Implementation of the Praxis III Performance Assessment
Beginning teachers in Ohio will be required to pass Praxis III starting in 2002. Ohio teachers who do not pass it after their first year may try again during the second year. Failure to complete the Entry-Year Program’s requirements successfully after the second attempt will result in loss of the provisional license until such time as the candidate completes additional coursework, supervised field experiences, and/or clinical experiences as designated by a college or university approved for teacher preparation and is recommended by such college or university.
ABILITY-BASED TEACHER EDUCATION AT ALVERNO COLLEGE
In this case study the committee provides a description of teacher education and assessment as practiced at Alverno College, in Milwaukee, Wisconsin. Alverno College undertook development of a performance-based baccalaureate degree over 20 years ago (Diez, et al., 1998). This change resulted in an overhaul of the college’s curriculum and its approach to teaching. The approach is characterized by publicly articulated learning outcomes, realistic classroom activities and field experiences, and ongoing performance assessments of learning progress. Alverno’s program is of interest because it provides an example of a system in which a party other than a state or district could warrant teacher competence. The focus here is on Alverno as a working program that can expand the debate about other models for warranting teacher competence.
Alverno College has an enrollment of approximately 1,900 students in 66 fields of study, 300 of whom are education majors. With the exception of a postbaccalaureate teacher certification program and the master of arts in education program, all programs admit only female students. Two-thirds of Alverno’s students are from the Milwaukee area, and about 30 percent are members of minority groups. There are about 100 faculty members, and the average class size is 25. The next section provides a brief overview of Alverno’s philosophy and practices.2
Abilities and Learning Outcomes for the Baccalaureate Degree
At Alverno College an ability is defined as “a complex integration of knowledge, behaviors, skills, values, attitudes, and self-perceptions” (Diez et al., 1994:9). The general education courses provide students with the opportunity to expand and demonstrate each of eight abilities:
Communication—an ability to communicate effectively by integrating a variety of communication abilities (speaking, writing, listening, reading, quantitative, media literacy) to meet the demands of increasingly complex communication situations.
Analysis—an ability to be a clear thinker, fusing experience, reasoning, and training into considered judgment.
Problem solving—an ability to define problems and integrate a range of abilities and resources to reach decisions, make recommendations, or implement action plans.
Values within decision making—an ability to reflect and to habitually seek to understand the moral dimensions of decisions and to accept responsibility for the consequences of actions.
Social interaction—an understanding of how to get things done in committees, task forces, team projects, and other group efforts.
Global perspective—an ability to articulate interconnections between and among diverse opinions, ideas, and beliefs about global issues.
Effective citizenship—an ability to make informed choices and develop strategies for collaborative involvement in community issues.
Aesthetic responsiveness—an ability to make informed responses to artistic works that are grounded in knowledge of the theoretical, historical, and cultural contexts.
The abilities cut across disciplines and are subdivided into six developmental levels. The six levels represent a developmental sequence that begins with objective awareness of one’s own performance process for a given ability and specifies increasingly complex knowledge, skills, and dispositions. Students must demonstrate consistent performance at level 4 for each of the eight abilities prior to graduation. An example of the development levels for problem solving appears in Box F-7.
Each of Alverno’s educational programs also defines a set of abilities distinctive to each major and minor area. These outcomes, identified by faculty as essen-
BOX F-7 Levels of Learning Outcomes for Problem Solving
SOURCE: Alverno College Faculty, 1973/2000.
tial learning outcomes, relate to and extend the general education abilities (Loacker and Mentkowski, 1993). Within the major area of study, students are expected to achieve at least a level 5 for each of the program’s abilities (Zeichner, 2000).
Abilities and Learning Outcomes in Teacher Education
Alverno’s Department of Education offers degree programs in elementary, early childhood, secondary, bilingual, music, art, and adult education. All education programs are designed to foster the same set of teaching abilities. These teaching abilities define professional levels of proficiency that are required for graduation with a major in any of the teacher education programs. The teaching abilities refine and extend the general education abilities into the professional teaching context. While the professional teaching abilities are introduced in the first year, they receive heavy emphasis during the junior and senior years. The teaching abilities include:
Conceptualization—integrating content knowledge with educational frameworks and a broadly based understanding of the liberal arts in order to plan and implement instruction.
Diagnosis—relating observed behavior to relevant frameworks in order to determine and implement learning prescriptions.
Coordination—managing resources effectively to support learning goals.
Communication—using verbal, nonverbal, and media modes of communication to establish the environment of the classroom and to structure and reinforce learning.
Integrative interaction—acting with professional values as a situational decision maker, adapting to the changing needs of the environment in order to develop students as learners.
Each of the above education abilities is further described for faculty and candidates through maps (Diez et al., 1998). In the maps, development of the ability is defined in terms of what teachers would be expected to do with their knowledge and skills at various stages of their development. An example based on skill in integrative interaction follows: the beginning teacher would demonstrate ability in integrative interaction by showing respect for varied learner perspectives; the experienced teacher would provide structures within which learners create their own perspectives; and the master teacher would assist learners in the habit of taking on multiple perspectives. This type of mapping is intended to “capture the interactions between knowing and doing” (Diez et al., 1998:43).
Alverno’s Program for Education Majors
Alverno’s program for education majors is designed to address the developmental needs of learners. Concepts are addressed in an integrated fashion, across
multiple courses and settings to enable a “deepened understanding” that comes with repetition of concepts (Diez, 1999:233). The program is characterized by extensive opportunities for field experiences that require candidates to apply what they have learned. Coursework and field experiences are sequenced to build developmentally across the years of the program. For example, candidates begin with coursework and field experiences that require them to apply the frameworks they are learning with individual students or small groups in tutorial settings. They progress to more complex tasks with larger groups and whole-class instruction. The assignments gradually increase in complexity, requiring candidates to attend to multiple factors in their planning, their analysis of the classroom, and their implementation of learning experiences (Diez, 1999:233).
Self-reflection and self-assessment skills are emphasized at Alverno. Faculty have developed a set of reflective logs that guide students in each of four semester-long field experiences prior to student teaching. These logs are intended to help students develop their skills in the five education abilities. According to Diez (1999), the logs direct students to make links between theoretical knowledge and practical application (which develops skill in conceptualization and diagnosis), to observe processes and environments of learning (coordination skills), to translate content knowledge into suitable short presentations or learning experiences (communication skills), and to begin to translate their philosophy of education into decisions regarding all aspects of teaching environments and processes (integrative interaction).
The first stage of the education program is the preprofessional level. To apply to the preprofessional level, students must have completed one year of coursework, a required one-credit human relations workshop, and a portion of the math content requirements (Zeichner, 2000). The preprofessional stage lasts two semesters. During this time, education students begin to integrate the knowledge bases of the liberal arts disciplines with the process for applying the material from these disciplines (Alverno College Institute, 1996). The subject area methods courses are taught during this stage. These courses connect teaching methods with material learned in liberal arts general education courses. Performance assessments during this stage may consist of such activities as requiring teacher candidates to create a lesson for a given grade that incorporates knowledge about developmental psychology. Other performance assessments involve simulations in which prospective teachers take on the various roles that teachers play, such as conducting a parent-teacher conference, being part of a multidisciplinary evaluation team, or working with district planning activities. This period includes two pre-student teaching experiences.
After two semesters in the preprofessional stage and completion of two of the pre-student teaching experiences, teacher candidates can apply for admission to the professional level. To be admitted, they must successfully complete the first two field experiences (and provide letters of recommendation), demonstrate a specific ability, meet the statewide minimum cutoff scores on the required
Praxis exams, and pass several standard assessment exercises that are spread throughout the program (Zeichner, 2000). An example of one these standard assessments is the Behavioral Event Interview and Self-Assessment described by Zeichner (2000:10):
[This assessment is] an hour-long interview conducted in the second semester of field experiences. Each education department member interviews two students each semester. The aim of the interview is to give students a chance to talk about their actions and thinking in relation to working with pupils. It focuses on stories elicited by questions (e.g., Can you tell me about the time you came to a realization about children’s development through an experience with a child or children?). The students then are asked to use their stories as data for a self-assessment process focusing on the five advanced education abilities (e.g., Where do you see yourself drawing upon x ability? Where do you see a need to strengthen this ability?). The interview is audiotaped and students take the tape with them to complete a written self-assessment. They set goals for their next stage of development in the teacher education program and then meet for a second session with the faculty interviewer.
The final two semesters are considered the beginning of professional practice, during which student teaching occurs. To be accepted for student teaching, students must demonstrate communication ability at level 4, successfully complete all four pre-student teaching experiences, and pass another standard assessment exercise—the Professional Group Discussion Assessment. Zeichner (2000:11) also describes this assessment:
Students compile a portfolio that includes a videotape of their teaching together with a written analysis of that teaching in relation to the five advanced education abilities, cooperating teacher evaluations, etc. The student then participates in a half-day interview with principals and teachers from area schools who are part of a pool of over 400 educators helping to assess students’ readiness for student teaching.
Assessment as Learning
The philosophy behind Alverno’s program is based on the premise that only through integrating knowledge, skills, and dispositions in observable performances can evidence of learning be shown. Assessment is treated as integral to learning. It is used both to document the development of the abilities and to contribute to candidates’ development. Alverno’s faculty describe their approach as assessment as learning (Diez et al., 1998). As practiced at Alverno, assessment as learning has the following features:
Expected learning outcomes or abilities are stated, and candidates are aware of the goals toward which they are working.
Explicit criteria for performance are outlined to guide candidates’ work and to provide structure for self-assessment.
Evaluations are based on expert judgments using evidence of candidates’ performance and weighing it against established criteria.
Feedback is intended to be productive; it is not aimed at judgment alone but on ongoing development.
All assessments include the experience of reflective self-assessment.
Assessment is a process involving multiple performances. Candidates experience many assessments using multiple modes, methods, and times to provide a cumulative picture of their development.
Assessment begins during orientation and continues through graduation. For instance, all new students complete an initial communications assessment that includes writing and presenting. The presentation is recorded on videotape, and each student evaluates her own performance before receiving diagnostic and prescriptive feedback from expert assessors.
Alverno faculty believe that performance assessments should be as realistic as possible and should closely mimic the experiences of practicing teachers. In developing the curriculum, they have identified the variety of roles that teachers play. Performance assessments include simulations of parent-teacher interactions, multidisciplinary team evaluation, the teacher’s work with district or building planning, and the teacher’s citizenship role, as well as actual classroom teaching (Diez et al., 1998:2).
Each course is structured around the assessments and learning outcomes that must be demonstrated to claim mastery of the course material. Box F-8 contains two examples: the learning outcomes for a course at Alverno and a course assessment designed to evaluate mastery of one of the outcomes.
Coursework is intentionally sequenced to reflect developmental growth and to provide for cross-course application of concepts. For example, a mathematics methods course assessment might ask students to (1) create a mathematics lesson plan for first graders that incorporates concepts from developmental psychology, (2) teach the lesson, and (3) describe the responses of the learners and the adaptations made.
Alverno requires that student teachers perform all of the duties of a teacher effectively, assuming full responsibility for the classroom for a minimum of four weeks in each placement. They start and end each day of teaching on the same schedule as the cooperating teacher. Their performance is assessed on the five professional teaching abilities by the cooperating teacher, the college supervisor(s), and the student teacher herself.
Research and Program Evaluation
Alverno has documented in detail the ways in which its program abilities were developed. The eight general education abilities were identified during discussions with faculty members about “what it means to say that a student has
BOX F-8 Learning Outcomes for Integrated Reading 3
Assessment of Learning Outcome 2:
Assess literacy development of intermediate students and prescribe appropriate teaching strategies.
To assess mastery of the second outcome, students are asked to develop and use a rubric to assess actual intermediate-grade writing samples. Students analyze writing samples for strengths and weaknesses, and plan appropriate teaching strategies based on that analysis. Students then collaborate in formulating a teaching plan. Finally, they evaluate their own efforts on each component of the task.
Successful performance in this assessment depends on satisfying four criteria: (1) assessing the developmental level of the learner’s performance and providing sufficient evidence to support the judgment, (2) diagnosing areas requiring attention/instruction and providing an appropriate teaching plan, (3) contributing to the group discussion of the process, and (4) assessing one’s own performance on all components of the task.
Each of these four criteria is mapped to one or more of the education abilities at a specified level of performance (e.g., Diagnosis at level 5, Coordination at level 5) and noted as such in the course syllabus. Hence, assessment criteria are always publicized, and the paths connecting particular concrete activity and general abstract ability can be easily traced. Assessments and their links are reviewed on an ongoing basis by interdisciplinary teams. Similar assessments and sets of criteria exist for each level of outcome for each of the abilities.
SOURCE: Adapted from Alverno College Institute (1996:7).
completed a liberal arts baccalaureate degree” (Diez, 1999:41). The abilities emerged from faculty discussions focused on such questions as: What should a woman educated in liberal arts be able to do with her knowledge? How does the curriculum provide coherent and developmental support to her learning? What counts as evidence that a student has achieved the expectations of the degree, the major, and the professional preparation? Once the abilities were defined, the discussions centered on pedagogical, developmental, and measurement ques-
tions. Descriptions of the abilities and corresponding performance levels have evolved over the years as a result of many factors. There is ongoing interdisciplinary consultation and review. Each faculty member is expected to serve on a program evaluation and development committee.
Alverno’s performance-based assessments are designed to reflect the actual work of beginning professional teachers. They cover the content and skills considered relevant to the tasks that teachers are expected to perform. In addition, the context for assessments is intended to reflect real-life teaching situations, representing a broad sample of performance situations (broader than would be expected for assessments that focus on basic skills and subject matter knowledge). Committees of faculty members routinely audit the contents of the assessments (during regularly scheduled departmental meetings) to verify that they are appropriate and that they reflect current thinking about what teachers should know and be able to do.
Multiple judgments are obtained of each student’s skills and knowledge. Each student is observed and assessed hundreds of times as they participate in classroom and field activities. Evaluations utilize multiple contexts, multiple modes, and multiple evaluators. There are formal, “milestone” assessments staged at relevant points in the curriculum, such as the Behavioral Event Interview and Self-Assessment and the Professional Group Discussion Assessment described above, as well as less formal, ongoing, in-class assessments. The institution has processes in place for refining and updating criteria for judging students’ performance.
There have been both internal and external reviews of the program. The institution has maintained an Office of Research and Evaluation since 1976 (Mentokowksi, 1991). Through this office, a comprehensive longitudinal study of 750 students was conducted that tracked students from entry into the program through two years after graduation. For this study, researchers collected information on (1) student performance in the curriculum on college-designed ability measures; (2) student perceptions of reasons for learning, the process of learning, and its value for their own career and life goals; and (3) students’ personal growth after graduation (Mentkowski and Doherty, 1984).
Another piece of the longitudinal study involved collecting data from a group of alumnae five years after graduation. This research focused on how abilities learned in college transferred to the work setting, the extent to which alumnae continued to grow and develop after college, and how graduates were doing in their careers and further learning in the long term. This study used multiple research instruments (17) to collect data, including questionnaires, tests, essays, and interviews (Alverno College, 1992–1993).
In addition, surveys have collected data from Alverno graduates on their perceptions about the college’s program and the extent to which they felt prepared to teach upon graduation (Zeichner, 2000). Surveys and focus groups with principals have examined employers’ perceptions of the preparedness of Alver-
no graduates (Zeichner, 2000). Surveys have also compared longevity in the teaching field for Alverno alumnae with graduates of other programs.
Licensing Practices in Wisconsin
Wisconsin currently requires that students admitted into its teacher education programs pass a basic skills test. Alverno also has developed its own assessments for reading, listening, writing, speaking, and quantitative literacy through four levels, and these are required for graduation. Entering the classroom requires endorsement from the institution from which one graduates. Alverno uses successful completion of field experiences and course requisites and its many formal and informal assessments of students as the basis for warranting readiness to teach.
In 2004 the state of Wisconsin will require candidates for teaching positions to present portfolios demonstrating their performance in relation to several professional teaching standards. Candidates who pass the portfolio review will be granted provisional licenses to teach for three to five years while pursuing professional development goals related to the standards. The portfolios are meant to comprise performance assessments compiled near the time of graduation. In addition, the state’s standards make provision for a yet-to-be-specified content examination.