21st century skills if such skills are important for successful performance in the job.
In a role play assessment, the candidate for promotion is provided with written information about a realistic situation that may involve a nonroutine problem. After a period of time to prepare for the role play, the candidate presents her or his response to the situation to a panel of trained assessors. The assessors rate the response using behaviorally anchored rating scales, which describe specific behaviors.
For example, a candidate for promotion may be asked to play the role of a newly promoted insurance investigator who has just been put in charge of a large-scale insurance fraud investigation and is about to meet with a claims adjuster and an FBI agent. The candidate is told that it is important to ensure that the investigation is led by his or her company, rather than the FBI, and is given an hour to prepare for the 30-minute meeting. The managers conducting the assessment also prepare another person to participate in the role play, in the role of the FBI agent. This person is instructed to be very aggressive, to interrupt the candidate frequently, to push the candidate to turn the case completely over to the FBI, and to provide the candidate with new information about the criminal past of the suspects.
Houston explained that trained assessors use a scoring system to evaluate the candidate’s adaptability, as displayed in the role play. This system guides assessors to award few points for adaptability if the candidate acts flustered or overwhelmed by new information and more points if the candidate seamlessly adjusts to new information.
Developing and administering role plays involves several challenges, Houston said. First, substantial input from subject-matter experts is necessary to identify appropriate problems or situations, and personnel testing experts are also needed to create the role play materials and behaviorally anchored ratings, so the process is labor-intensive. Second, because only one candidate can participate at a time, this form of assessment is expensive. Many role plays involve additional role players along with the candidate, and these other role players must be paid for their time. Finally, a role play can be scored only by engaging the time and expertise of multiple trained assessors. Unlike a typical multiple-choice test used in educational settings, a role play cannot be electronically scored.
The development, administration, and scoring of a group exercise are similar to the role play, Houston said. The critical difference is that candidates work in groups to address a problem or respond to a situation,