The remainder of this chapter describes steps 1 and 2 which address the broad question of what to measure. Steps 3 and 4 which relate to how to measure, are addressed in Chapter 5.
The purpose of this step in the development process, which takes place before the faculty become involved, is to define the basic terms, such as measurement and evaluation, and clarify the underlying assumptions of the evaluation, such as that the goal is to design an evaluation system that will be objective and fair.
In the physical sciences, the term measurement is generally defined as the numerical estimation and expression of the magnitude of one quantity relative to another (Michell, 1997). However, this definition makes sense only for measuring physical and observable objects or phenomena. When measurement is used in the context of an evaluation of teaching, it takes on a somewhat different meaning, because the “things” being measured do not have readily observable, direct, physical manifestations.
For example, an evaluation that measures the impact of a faculty member’s teaching on students’ cognitive skills and/or attitudes may be desired. Although there may be some direct external evidence of these, such as student performance on examinations, this measurement will likely involve gathering certain types of data (e.g. student ratings, peer opinion questionnaires) as a basis for inferring a measurement of an internal cognitive or affective condition.
The terms measurement and evaluation are not synonymous. A measurement is as objective and reliable as possible. Whereas measurement involves assigning a number to an observable phenomenon according to a rule, evaluation is defined as the interpretation of measurement data by means of a specific value construct to determine the degree to which the data represent a desirable condition (Arreola, 2007). Thus the result of an evaluation is a judgment, which, by definition, is always subjective.
A specialized field of psychology, called psychometrics, has been developed to perform the kinds of measurements used in evaluations. Psychometrics is discussed in greater detail in the next chapter on how to measure the performance elements of teaching.
When an institution undertakes to develop a faculty evaluation system, the goal is to ensure that the system is as objective as possible. However, total objectivity in a faculty evaluation system is an illusion, because the term evaluation, by definition, involves judgment, which means that subjectivity is an integral component of the evaluative process.
In fact, the term objective evaluation is an oxymoron. Even though the measurement tools used in a faculty evaluation system (e.g., student ratings, peer observation checklists, etc.) may achieve high levels of objectivity, the evaluation process is, by definition, subjective.
However, the underlying rationale for wanting an “objective” faculty evaluation system is to ensure fairness and to reduce or eliminate bias. Ideally, in a fair, unbiased evaluation system