Evaluations should be based on multiple sources of information, multiple methods of gathering data, and information for multiple points in time.6 The evidence collected should be reliable (i.e., consistent and accurate), valid (i.e., it should measure what it is intended to measure), and fair (i.e., it should reflect the complexity of the educator’s achievements and accomplishments).
It is equally important to note that collecting and analyzing data of this sort often demands a skill that we may need to develop further among our faculty and administrators. A good way to learn these skills might be to enlist the help of colleagues on campus who have expertise in, for example, survey design, qualitative interviewing, educational outcomes research, and so forth.
A sustainable evaluation system must not require implementation that is burdensome to faculty or administrators. However, it is important to guard against sacrificing the fairness, validity, accuracy, and reliability of the evaluation system in trying to make it as easy to use as possible.
The evaluation system itself should be evaluated periodically to determine if it is effective. These periodic reviews should be part of the development plan to ensure that evaluations provide both formative feedback that leads to improvements in teaching and data adequate for judging the quality of teaching.
If the system is successful, all stakeholders will recognize that it provides accurate and valuable information that meets the needs of various groups and creates a culture of assessment that drives teaching and learning improvements. They will also agree that an assessment is not done to faculty but is done by faculty and for faculty and that assessment supports continuous improvements in the quality of education. If stakeholders internalize the principles listed above for developing metrics, they will naturally support a culture of assessment.
Both direct and indirect measures should be used. Direct measures (e.g., exams, projects, assignments) show evidence of students’ knowledge and skills. Indirect measures (e.g., teaching evaluations) reflect students’ perceptions of teaching effectiveness and employers’ and alumni perceptions of how well the program prepares students for their jobs.