National Academies Press: OpenBook

Driver Selection Tests and Measurement (2012)

Chapter: CHAPTER THREE Review of Driver Selection Tests and Measurements

« Previous: CHAPTER TWO Overview of Driver Individual Differences
Page 24
Suggested Citation:"CHAPTER THREE Review of Driver Selection Tests and Measurements." National Academies of Sciences, Engineering, and Medicine. 2012. Driver Selection Tests and Measurement. Washington, DC: The National Academies Press. doi: 10.17226/14632.
×
Page 24
Page 25
Suggested Citation:"CHAPTER THREE Review of Driver Selection Tests and Measurements." National Academies of Sciences, Engineering, and Medicine. 2012. Driver Selection Tests and Measurement. Washington, DC: The National Academies Press. doi: 10.17226/14632.
×
Page 25
Page 26
Suggested Citation:"CHAPTER THREE Review of Driver Selection Tests and Measurements." National Academies of Sciences, Engineering, and Medicine. 2012. Driver Selection Tests and Measurement. Washington, DC: The National Academies Press. doi: 10.17226/14632.
×
Page 26
Page 27
Suggested Citation:"CHAPTER THREE Review of Driver Selection Tests and Measurements." National Academies of Sciences, Engineering, and Medicine. 2012. Driver Selection Tests and Measurement. Washington, DC: The National Academies Press. doi: 10.17226/14632.
×
Page 27
Page 28
Suggested Citation:"CHAPTER THREE Review of Driver Selection Tests and Measurements." National Academies of Sciences, Engineering, and Medicine. 2012. Driver Selection Tests and Measurement. Washington, DC: The National Academies Press. doi: 10.17226/14632.
×
Page 28
Page 29
Suggested Citation:"CHAPTER THREE Review of Driver Selection Tests and Measurements." National Academies of Sciences, Engineering, and Medicine. 2012. Driver Selection Tests and Measurement. Washington, DC: The National Academies Press. doi: 10.17226/14632.
×
Page 29
Page 30
Suggested Citation:"CHAPTER THREE Review of Driver Selection Tests and Measurements." National Academies of Sciences, Engineering, and Medicine. 2012. Driver Selection Tests and Measurement. Washington, DC: The National Academies Press. doi: 10.17226/14632.
×
Page 30
Page 31
Suggested Citation:"CHAPTER THREE Review of Driver Selection Tests and Measurements." National Academies of Sciences, Engineering, and Medicine. 2012. Driver Selection Tests and Measurement. Washington, DC: The National Academies Press. doi: 10.17226/14632.
×
Page 31
Page 32
Suggested Citation:"CHAPTER THREE Review of Driver Selection Tests and Measurements." National Academies of Sciences, Engineering, and Medicine. 2012. Driver Selection Tests and Measurement. Washington, DC: The National Academies Press. doi: 10.17226/14632.
×
Page 32
Page 33
Suggested Citation:"CHAPTER THREE Review of Driver Selection Tests and Measurements." National Academies of Sciences, Engineering, and Medicine. 2012. Driver Selection Tests and Measurement. Washington, DC: The National Academies Press. doi: 10.17226/14632.
×
Page 33
Page 34
Suggested Citation:"CHAPTER THREE Review of Driver Selection Tests and Measurements." National Academies of Sciences, Engineering, and Medicine. 2012. Driver Selection Tests and Measurement. Washington, DC: The National Academies Press. doi: 10.17226/14632.
×
Page 34
Page 35
Suggested Citation:"CHAPTER THREE Review of Driver Selection Tests and Measurements." National Academies of Sciences, Engineering, and Medicine. 2012. Driver Selection Tests and Measurement. Washington, DC: The National Academies Press. doi: 10.17226/14632.
×
Page 35
Page 36
Suggested Citation:"CHAPTER THREE Review of Driver Selection Tests and Measurements." National Academies of Sciences, Engineering, and Medicine. 2012. Driver Selection Tests and Measurement. Washington, DC: The National Academies Press. doi: 10.17226/14632.
×
Page 36
Page 37
Suggested Citation:"CHAPTER THREE Review of Driver Selection Tests and Measurements." National Academies of Sciences, Engineering, and Medicine. 2012. Driver Selection Tests and Measurement. Washington, DC: The National Academies Press. doi: 10.17226/14632.
×
Page 37
Page 38
Suggested Citation:"CHAPTER THREE Review of Driver Selection Tests and Measurements." National Academies of Sciences, Engineering, and Medicine. 2012. Driver Selection Tests and Measurement. Washington, DC: The National Academies Press. doi: 10.17226/14632.
×
Page 38
Page 39
Suggested Citation:"CHAPTER THREE Review of Driver Selection Tests and Measurements." National Academies of Sciences, Engineering, and Medicine. 2012. Driver Selection Tests and Measurement. Washington, DC: The National Academies Press. doi: 10.17226/14632.
×
Page 39

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

22 CHAPTER THREE REVIEW OF DRIVER SELECTION TESTS AND MEASUREMENTS This chapter addresses the principal topic of this report: tests and measurements for selecting safe commercial driv- ers. The previous chapter reviewed basic requirements for commercial drivers and safety-important ways in which commercial drivers vary. These are enduring human char- acteristics relevant to general personal safety, social adjust- ment, and wellness. This chapter builds on that foundation and asks how knowledge of individual differences can be applied in a company hiring setting to select the safest driv- ers. The chapter begins with a brief review of “generic” carrier hiring processes as well as general employment test characteristics and requirements. Then it describes a series of specific instruments used for driver selection, many of which are commercially available products or services. Most of these instruments are oriented toward safe driving as the criterion job performance measure. Some attempt to predict driver retention, which is known to be correlated with safety. OVERVIEW OF COMMERCIAL DRIVER SELECTION AND HIRING Minimum Required Actions Carriers must, at a minimum, take actions to ensure that any driver they hire (and keep under hire) meets general com- mercial driver qualifications. This means that carriers must maintain a qualification file for each employee. According to 49 CFR 391.51 and as summarized in FMCSA (2008), this qualification file must include the following: • Driver’s application for employment (completed and signed). • Driver’s motor vehicle report (MVR) of past crashes and violations from the applicable state agency for the preceding 3 years. • Driver’s road test certificate or the equivalent. A cur- rent CDL is evidence of road test completion. • Annual review of driving record based on state agency inquiry and carrier review. Certification that driver meets minimum requirements is signed by the carrier. • Annual driver’s certification of violations. • Medical examiner’s certificate. • Record of inquiry(ies) to previous employer(s) for past 3 years. Overall Driver Selection and Hiring Process The previous section specified the minimum actions and paperwork for hiring commercial drivers. In practice, these minimum actions are combined with voluntary company actions to form an overall system and sequence of steps for hiring. Figure 6 shows a flowchart of a systematic com- mercial driver hiring process. It outlines a multiple hurdles approach in which a candidate must pass all assessments in sequence to be employed. The flowchart is adapted from one provided by Daecher Consulting Group for CTBSSP Syn- thesis 4 (Knipling et al. 2004). The current project focuses on test and measurement procedures that can be used to improve the selection process, especially those that can be administered to drivers as part of on-site screening. Cascio (2004) and Knipling (2009a) present four general rules for selecting the highest-quality possible drivers: 1. Target high-quality applicants 2. Attract as many applicants as possible 3. Use multiple, validated selection tools and methods 4. Be as selective as possible. A high-quality applicant pool means that those selected will be the “best of the best.” Attracting more applicants means that a smaller percentage of applicants will be hired, and thus that the process has been as selective as possible. The use of multiple validated selection tools and methods results in the most accurate possible selections. Good driver selection systems usually include multiple evaluation factors beyond minimum driver requirements, such as applicant education, driving history, nondriving history (i.e., criminal record), prior drug and alcohol tests, medical conditions, personality, and job attitudes. The ATA publication SafeReturns (ATA Foundation 1999a) recommended requiring age and experi- ence minimums, conducting in-person interviews, screening for stable employment history, setting a top limit for mov- ing violations using a point system, conducting driving tests, requiring a company physical examination, and reviewing the financial reliability (e.g., credit rating) of owner-operators. Many of these methods are discussed later in this chapter and in the chapter five carrier case studies.

23 FIGURE 6 Systematic hiring process. For any job, the selection ratio is the proportion (or per- centage) of job applicants actually hired (Cascio 2004). Stated as a ratio, this is A low selection ratio leads to higher quality employees because a lower percentage of applicants have been chosen. Many high-performing motor carriers have selection ratios of 20% or even lower. Schneider National (Osterberg 2004; Knipling 2009a) uses a multilayer selection process that hires only about 13% of applicants and a much lower percentage (3%) of total driver inquiries. Figure 7 shows the Schneider numbers for 2004 at six steps of their process, beginning with recruiting calls received and ending with new hires. FIGURE 7 Selectivity of Schneider National driver hiring. (Source: Based on Osterberg 2004.) TEST CHARACTERISTICS AND REQUIREMENTS This section covers basic testing concepts, federal require- ments, and principles for carriers to better assess their current and planned use of testing to hire safer drivers. Information on employment testing can also be found in var- ious textbooks on industrial psychology and management. These include texts by Sonnentag (2001), Cascio (2003), and Spector (2008). The following information is intended to provide a basic understanding of employment testing and how it might be improved. It is not sufficient as a guide for conducting test validation studies or implementing major new selection procedures. For these, companies are advised to consult HR specialists in staffing and employment law. Key Assessment Terms and Concepts The following are some key terms and concepts in employee assessment: • Job analysis • Predictor(s) • Job performance criterion (criteria) • Test reliability • Test validity: – Content validity – Construct validity

24 – Criterion-based validity: • Predictive validity • Concurrent validity • Success ratio. One must fully understand a job to be able to accurately select the best employees for that job. Job analysis is the delineation of specific tasks and performance involved in a job. They may include job function or duties, work tasks, skills or competencies, work-related knowledge, work environment factors, decision-making authority, educa- tional requirements, communication, training, and physi- cal abilities. A job analysis is often necessary to validate the use of a selection procedure. This is especially true for any procedure that disadvantages groups of potential employees that are protected under the employment dis- crimination laws. Appendix B provides a commercial driver job description developed and used by a medium-sized regional TL carrier in Canada. This carrier uses this document to provide its appli- cants and employed drivers with full information on their driver jobs and performance expectations. Although it is not a formal job analysis, it contains many of the same elements. A job analysis document helps a carrier to identify the most important and valid elements of its selection process. These selection elements are predictors of job performance. A job performance criterion is a measure of employee success on the job. Generally, any job has multiple performance crite- ria. Of particular concern are job safety performance crite- ria, which may include such measures as crash rate, rate of preventable crashes, violation rate, and rapid decelerations captured in onboard recordings. More sophisticated compa- nies use driving behavior criteria such as hard-braking rate, speed compliance, and fuel economy. The reliability of a test or measure is the degree to which it provides consistent measurements. Measurements like height and weight are almost perfectly reliable because repeated measures will provide exactly (or almost exactly) the same result. The test-retest reliability of height, expressed as a correlation between two successive measurements, is a perfect +1.00 or nearly so. Some tests of psychological traits, such as IQ and aptitude tests, often have test-retest or split-half reliabilities of +0.90 or more (Associated Con- tent, 2010), but the reliability of tests of personality traits like impulsivity and sensation-seeking is lower. Subjectively scored interviews are likely to have even lower reliabilities. A test with a reliability of 0.00 would be worthless because it would reflect random answers or scoring. A U.S. Depart- ment of Labor publication (DOL 2000) classifies coefficients of +0.90 or above as excellent, +0.80 to +0.89 as good, and +0.70 to +0.79 as adequate. If multiple assessments are used, their combined reliability may be greater than any one test because multiple assessments capture more elements of per- formance and behavior. Measurement reliability is a concern both for selection measures and for job performance measures. To the extent that measures are unreliable, they are confounded by ran- dom error. Random error in either a predictor measure or a criterion measure means that neither can be perfectly accu- rate (valid) as a measure. Validity is the accuracy of a measure, or the degree to which it actually measures what it purports to measure. Selection test validity can be assessed in various ways. Con- tent validity is the degree to which the content of a test corre- sponds to the content of a job. A road/range driving test, for example, has obvious content validity in relation to a com- mercial driver’s job. Construct validity is more conceptual. Construct validation usually involves showing that a test measures specific personal characteristics that are known to be relevant to the performance of the job. Chapter two, for example, cites extensive evidence linking aggressive/hostile personalities to high crash rates. If a test provides reliable measures of personal aggressiveness that correlate well with other measures, then it could has construct validity in rela- tion to this element of safe driving. Criterion-based validity is the degree to which test scores correlate with actual job performance criteria. For a person- ality measure of aggressiveness, this might be its correlation with future crash rates of new hires (predictive validity) or with current/past crash rates of existing employees (con- current validity). Criterion-based validity is the practical, “bottom line” validity of a test; that is, how well it actually performs as a test. Thus, a well-conducted criterion-based validation study is generally the strongest method to demon- strate the value of a test. Evidence attesting to content or con- struct validity is generally supportive rather than definitive. Criterion-based validity also is expressed as a correlation coefficient. The term v-score is sometimes used. V-scores are almost always lower than test reliabilities because so many factors contribute to job performance and because per- formance is hard to measure. The U.S. DOL (2000) assesses v-scores as follows: • 0.35 or higher: test is “very beneficial” in assessing likely employee success. • 0.21–0.34: test “is likely to be useful” to the employer • 0.11–0.20: test may be useful, depending on circum- stances and whether other assessments are also used. • 0.11 or less: test is “unlikely to be useful.” Understanding the overall concept of prediction is as important as familiarity with prediction statistics. Figure 8 is a simplistic model of employee selection (Cascio 2004). The horizontal x-axis represents the predictor (i.e., selection test) score and the vertical y-axis is the job performance criterion. Assume that higher scores are “good” for both scales. The slanted oval represents a hypothetical population of appli-

25 cants. It is slanted upward because of the assumption that the selection test has moderate validity as a predictor (i.e., a moderate positive correlation with job performance). The vertical line down the middle is the cutoff hiring score for predictor test scores. The horizontal line across the middle is the minimum satisfactory job performance. Assume, for the sake of the model, that everyone is hired so that their predic- tor scores can be compared with their job performance. FIGURE 8 Simplistic model of the relation between predictor test scores and employee job performance. In this figure, higher is better for both dimensions. (Source: Based on Cascio 2004.) The four areas of the oval are as follows: A. Correct acceptances (high test score, and satisfactory job performance) B. Erroneous acceptances (high test score, but unsatis- factory job performance) C. Correct rejections (low test score, and unsatisfactory job performance) D. Erroneous rejections (low test scores, but satisfactory job performance). A successful selection system would have a high propor- tion of correct decisions; for example, hiring good drivers and rejecting bad drivers. In the model, zones A and C repre- sent correct decisions, whereas zones B and D are bad deci- sions. The percent of correct decisions or selection success ratio is given by the following equation: This model illustrates selection concepts rather than actual practice. It is simplistic because it considers only one predic- tor and one job criterion, and because it assumes that there is a sharp cutoff score for each. In the real world, almost every job involves multiple selection factors and multiple measures of job performance. Yet the same conceptual model of selection applies: The employer is trying to maximize zones A and C in the model and minimize zones B and D. In the model, how would a highly valid selection test look different than one that was less valid? Other factors being equal, the difference would be in the shape of the oval. A highly valid selection test would generate a “skinny” oval. A poor test would generate a “fat” oval, and a completely worthless one would generate a circle or other shape in which the sum of A + C was no greater than the sum of B + D. One can further break down a test’s performance by its “hit” rate for identifying unsafe drivers and its “false alarm” rate for rejecting safe drivers. A valid test would have a high hit rate and a low false alarm rate. In the context of Figure 8, these two measures can be defined as follows: Unsafe Driver “Hit ” Rate  Correct Rejection (C)Total Number of Unsafe Drivers (BC) Safe Driver “False Alarm” Rate  Erroneous Rejections (D)Total Number of Safe Drivers (AD) None of these statistics can be calculated based on an actual selection process where some candidates are hired and others are not hired. Nonhired drivers would have no job performance criterion data with which to classify them as “safe” or “unsafe.” A company could, however, norm a test against its existing driver force (i.e., determine concur- rent validity). For example, the worst 15% of drivers in terms of crash rates, violation rates, complaint rates, or other met- rics could be compared with the best 85%. A high hit rate for unsafe drivers and a low false alarm rate for safe drivers would indicate a valid and useful test. Federal Requirements for Employment Tests All employers have an ethical and a legal duty to treat appli- cants for employment fairly. Employers also have the same duty with respect to assessing current employees. Nongov- ernmental employers have a particular legal duty, but most state and local agencies and all federal agencies have similar requirements. The primary sources for this section are Chee- seman (2006) and Mann and Roberts (2006). Several laws shape this legal duty, the most important of which is the Fair Employment Practices Act, or Title VII of the Civil Rights Act of 1964 as amended by the Equal Employment Opportunity Act of 1972. The basic require- ment of this law is that employers shall not discriminate in hiring, promotion, wages, training, or any other term, con- dition, or privilege of employment, according to the race, color, religion, sex, or national origin of the affected per- sons. The italicized categories are the “protected classes” of individuals under the act. The text box lists this and four other laws that further define the duty of fair treatment.

26 Laws Defining the Fair Treatment of Applicants and Employees 1. The Fair Employment Practices Act (also known as Title VII of the Civil Rights Act of 1964, as amended by the Equal Employment Opportunity Act of 1972) 2. The Equal Pay Act of 1963 (EPA) 3. Title I of the Americans with Disabilities Act of 1990 (ADA) 4. The Age Discrimination in Employment Act of 1967 4. The Genetic Information Nondiscrimination Act of 2008 (GINA). Cheeseman (2006), Mann and Roberts (2006) Along with the other laws listed in the text box, the Fair Employment Practices Act is administered by a stand- alone federal agency, the Equal Employment Opportu- nity Commission (EEOC). The EEOC issues regulations that spell out the meaning of fair treatment, and accepts complaints from individuals who believe they have been unfairly treated in employment settings owing to their membership in one of the protected classes. The EEOC can bring suit in federal court to enforce its regulations or resolve complaints. Very small trucking fleets (10 or fewer trucks) are not covered by EEOC regulations and enforcement because the regulations apply only to private employers who have had 15 or more employees. Further, to be covered, employees must have worked for at least 20 calendar weeks in the cur- rent or preceding year. For age discrimination issues only, the threshold is 20 employees. But this still leaves a large number of trucking firms covered. The EEOC has issued uniform regulations governing the use of selection tests in hiring and promotion. These appear in the Code of Federal Regulations, items 29 CFR 1607.1 through 1607.18. The regulations provide uniform guidelines defining what constitutes an adverse impact on a protected class. Adverse impact would trigger federal scru- tiny and a presumption that unfair treatment may be taking place. An adverse impact occurs when a protected group is selected at less than 80% of the rate at which nonprotected applicants are selected (29 CFR 1607.4). Thus, this trigger is called the “four-fifths rule.” Four-Fifths Rule Example If a given procedure selects 91% of male applicants (screening out 9%), then the four-fifths guideline says that females, or any other protected class, must be selected for hiring at a rate of no less than 80% of that 91%, which is 73% (0.8 × 0.91 = 0.73). Or to put it the other way around, no more than 27% of females may be screened out by a method that screens out only 9% of men. If a method of selection discriminates numerically because it does not satisfy the four-fifths rule, it may still be legal to use if it can be shown to be valid with respect to the job for which applicants are applying. Validity means that the employer can show, with specific statistical evi- dence, that the selection method generates measurements that are demonstrably correlated with job performance. Or, the employer can show (typically by job analysis) that the selection method has content that is demonstrably repre- sentative of important parts of the job. There is also a third way to show validity, by showing that the method mea- sures a related set of personal characteristics (a construct) that is important in successful job performance. However, the EEOC regulations note that this approach is less well documented in the academic literature. An employer tak- ing this route might take some extra care in meeting the regulatory requirements. There is a limitation to using validity to defend a selection method that is otherwise desirable because it selects safer drivers, but that has the side effect of numerically discrimi- nating against a protected class. The employer must obtain and keep current statistical evidence of the method’s impact on its own applicant pool. Generic information provided by the vendor of a test, for instance, will not be sufficient. Also, if the employer uses either the criterion or construct meth- ods of showing the validity of a selection procedure, extra specific statistical evidence is required. The employer must show by specific statistical evidence from the job behavior of its employees that the criterion or construct used to select among applicants is statistically linked to safer driving per- formances. However, record keeping is permitted to be sim- pler if the employer has fewer than 100 employees. The fact that a selection method has an adverse impact on a protected group may not by itself be a sufficient reason for not using it, if it is valid. Firefighters in Connecticut sued their employer, the city of New Haven, in 2006 on this issue. White and Hispanic firefighters who were selected for pro- motion by an exam that appeared to be valid objected when the city dropped the use of the exam because it discrimi- nated against African American firefighters according to the four-fifths rule. In two connected cases on this issue, in 2009

27 the Supreme Court ruled that the city had illegally discrimi- nated against the whites and Hispanics, and that because the test was valid, it should use the results in deciding who to promote (Ricci v. DeStefano 2009). EEOC Links 1. The EEOC home page: http://www.eeoc.gov/ 2. An overview of the EEOC and its regulations for employers: http://www.eeoc.gov/employers/ index.cfm 3. A clear statement of the duty not to discriminate, and prohibited practices generally: http://www. eeoc.gov/laws/practices/index.cfm 4. Laws and regulations enforced by the EEOC: http://www.eeoc.gov/laws/ 5. Regulations governing selection procedure impact and validity: http://www.access.gpo.gov/nara/cfr/ waisidx_10/29cfr1607_10.html Four other laws may be relevant to selecting drivers. The Equal Pay Act of 1963 (EPA) prohibits paying different wages to men and women if they do the same work in the same workplace. Title I of the Americans with Disabilities Act of 1990 (ADA) prohibits discrimination against quali- fied individuals who have disabilities. Further, it requires that employers reasonably accommodate the known physi- cal or mental limitations of an otherwise qualified indi- vidual with a disability, unless doing so would impose an undue hardship on the operation of the employer’s busi- ness. Of potential relevance for selecting drivers for safe driving performance is this exception to the ADA: “The ADA permits an employer to require that an individual not pose a direct threat to the health and safety of the individual or others in the work-place.” Discrimination on the basis of age is prohibited by the Age Discrimination in Employ- ment Act of 1967. The law applies only to discrimination against older workers, not younger ones, and the threshold for coverage begins at age 40. Finally, the Genetic Infor- mation Nondiscrimination Act of 2008 (GINA) prohibits discrimination against applicants based on genetic infor- mation about them or their family. All of these laws are administered by the EEOC, and in general the regulations issued by the EEOC with respect to the separate potential ways in which selection and hiring may be unfair are simi- lar to those for the Fair Employment Practices Act. The EEOC offers a comprehensive set of web pages that pro- vide clear linkage to the different issues and questions that may be of interest to employers who are concerned about selection procedures. The text box contains a few of the more useful links. Principles for Improved Employee Assessment Based largely on the previously mentioned concepts and laws related to employee selection and other assessments, the U.S. DOL has produced Testing and Assessment: An Employer’s Guide to Good Practices (DOL 2000). The report describes and explains basic principles that employers should follow when considering and designing employee assessments. They include selection-related assessments of candidates as well as assessments of current employees for promotion, placement, or other actions. The DOL guide is designed to help managers and HR professionals use tests and other assessments to improve employee and organizational performance. It helps employ- ers to— • Evaluate and select assessment tools that maximize chances for getting the right fit between jobs and employers. • Correct administer and score assessment tools. • Accurately interpret assessment results. • Understand and follow professional and legal standards. The guide presents and explains 13 principles for improved and legal employee assessment. These have also been summarized by Kahle (2010). Almost all of the prin- ciples follow from the testing concepts and laws discussed previously: • Use assessment tools in a purposeful manner; that is, for the purpose for which they are designed. Misuse or improper use could be harmful or possibly illegal. • Use the whole-person approach to testing; that is, con- sider all the information you have about the candidate. No test is perfect. Use a combination of assessments that give you as much information as possible about behaviors of greatest importance. • Use tests that are unbiased and fair to all groups. Tests that deliberately or inadvertently discriminate prevent the employer from achieving the most qualified work group. • Use tests that are reliable. Will the same person get the same results each time they take the test? • Tests must be valid for the purpose they are being used. Validity is the most important criterion for selection of a proper test instrument. Validity determination may be based on content, criterion prediction, or constructs captured by the test. Criterion-based validity is the definitive test. • Tests must be appropriate (e.g., content and difficulty) for the target population. • Test instructions and other documentation must be comprehensive and easy to understand. • Test proctors, administrators, and scorers must be properly trained. Some instruments require an exten- sive certification process for these roles.

28 • It may be necessary to provide consistent and uniform testing conditions to obtain consistent results. 1. Provide reasonable accommodations for people with disabilities. No group should be disadvantaged by the test or test conditions per se. 2. Maintain test security. For example, if specific test items on a knowledge test or inventory are not secure, applicants could memorize correct answers or other- wise “game” the test. On the other hand, for some job qualifications it may be advisable to publicize specific test items which must be passed. For example, case study Carrier C has a video on its website showing all of its physical ability test items. • Maintain the security and confidentiality of test results. • Interpret test results correctly. Make sure that decision makers understand the tests and what test results mean. Ensure that all test reports are easy to understand. SAFETY-RELATED DRIVER EMPLOYMENT TESTS Job Knowledge, Skill, and Training Chapter two reviewed basic federal commercial driver quali- fications and some of the records that carriers are required to keep of required checks made during selection and hiring. Carriers must ensure that their drivers meet these require- ments. Many also make further efforts to assess driver job knowledge and skill. Carrier actions to do this are covered in the project survey results (chapter four) and in carrier case studies (chapter five). New CDL Skills Test: Required Range Maneuvers • Straight-line backing • Offset backing to the right • Offset backing to the left • Sight-side parallel parking • Conventional parallel parking • Alley dock Source: Brock et al. (2005) Carrier assessment of driver knowledge and skill focuses first on drivers’ training histories, especially for newer drivers. Entry-level drivers may receive formal training at community colleges, private truck driver training schools, or directly from carriers (FMCSA Medical Review Board 2007). This includes knowledge training in classrooms and skill training on ranges (restricted off-road lots) and on- road. New drivers must pass a CDL knowledge test to get their learners permits before behind-the-wheel training. Then they must pass a road and range driving skills test to get their CDL. A new CDL range testing regimen has been developed and is being gradually adopted by different states. The new test is intended to correspond more closely to real- world job requirements (Brock et al. 2007). The text box contains the six basic range maneuvers required in the new skills test (Brock et al. 2005). Currently, there are no specific U.S. federal training requirements except for classroom instruction on four special topics not related directly to the driving task. The four topics are HOS compliance, drug and alcohol regu- lations, driver health and wellness, and whistleblower protection. Other countries have specific training require- ments relating to duration and quality of training, and at this writing FMCSA is considering such requirements for the United States. Carriers are obviously concerned about the quality of entry-level driver training. School quality is judged by repu- tation, school certifications, and carriers’ own experiences. Duration of school training is apparently not a good predictor of driver success. Across six large fleets and nearly 17,000 entry-level drivers, the American Transportation Research Institute (2008) compared the duration and subject content of basic training with subsequent driver safety. Basic train- ing contact hours ranged from 88 to 272, but training hours did not correlate significantly with subsequent driver crashes and violations. Hours of training in various specific topic areas did not correlate well either. This finding is not sur- prising, given the many driver individual differences largely unaffected by training, the many other factors affecting on- the-job safety, and the fact that, in general, differences in training do not have long-term effects on employee on-the- job performance (Brock et al. 2007; Knipling 2009a). Driving Record Obtaining driver records is not a “test” in the usual sense, but it functions in the same manner as a screening tool. Carriers are obliged to review State Motor Vehicle Records (MVR) for traffic violations and convictions. A new national pro- gram allows carriers to voluntarily access crash and roadside inspection data as well. State Motor Vehicle Records The FMCSRs (49 CFR 391.51) require motor carriers to obtain driver applicant MVRs covering the preceding 3 years from state agencies (FMCSA 2008). This includes every state in which a driver has been licensed during that

29 period. The MVR provides information on driver moving violations, other vehicle-related violations, involvement of crashes, and license suspensions. Crash preventability or “fault” is not specifically indicated, though traffic violations associated with crashes are shown. FMCSA (2008) provides a form letter for MVR requests to state agencies. After a driver is hired, carriers must obtain the driver’s updated MVR annually, and the driver must prepare and furnish a list of driving violations for the previous year. Commercial services such as HireRight (www.hireright. com; also called DAC Trucking) provide MVRs and other driver history reports on a fee basis. Such services may pro- vide other applicant history information as well, including employment history, drug and alcohol testing history, work- ers compensation searches, criminal background checks, credit history, and education verification. Pre-Employment Screening Program The Pre-Employment Screening Program (PSP) is a new screening tool developed by FMCSA for voluntary use by carriers. PSP allows motor carriers and individual drivers to obtain driving records from the FMCSA Motor Carrier Management Information System (MCMIS). Once carriers enroll in PSP (www.psp.fmcsa.dot.gov), they can pay a $10 fee to request driver records online. PSP driver information contains the most recent 5 years of crash data and 3 years of inspection data; because it contains only information from MCMIS, it does not include traffic violation conviction data. PSP records state a driver’s total number of crashes for the past 5 years and the number resulting in fatalities, injuries, towaways, and HAZMAT releases. Inspection data include the number of driver, vehicle, and HAZMAT inspections con- ducted and the number with out-of-service violations. Spe- cific inspection violations and out-of-service violations are listed (e.g., brakes out of adjustment, flat tire/fabric exposed, driver log not current). The information on PSP was previ- ously provided by the FMCSA Driver Information Resource. Carriers are not required to use PSP, but it has been designed to be a convenient and inexpensive way to access driver records. PSP does not contain data from state DMVs such as non-safety-related license suspensions (e.g., relating to child support). Drivers can access their own PSP records without prior enrollment. PSP is a new system just completed in 2010, so its use is not yet standard operating procedure for most carriers. Industry interest in the system is high, however, and its use is increasing rapidly. Of 65 safety manager respondents in the project survey, 45 planned to use the system, 15 were not sure, and only 5 indicated that they would not use it. As noted previously, the project survey sample was not based on structured sampling procedure and thus cannot be regarded as nationally representative. Even so, it appears that PSP use will become standard procedure for most safety-conscious carriers. Some progressive carriers plan to obtain PSP records on their current drivers to further refine and inter- nally validate their selection of PSP data. Medical Conditions and Physical Capabilities Medical Conditions Chapter two outlined the minimum commercial driver physical qualification standards per federal regulations (49 CFR 391.41) and provided a general background on the relation between medical conditions and driver crash risk. Extensive information on federal commercial driver medi- cal qualifications and the latest rules and interpretations is available at http://www.fmcsa.dot.gov/rules-regulations/ topics/medical/medical.htm. Medical evidence reports, medical expert panel recommendations, and agency medi- cal review panel reports are available on the following safety-relevant health conditions: • Diabetes mellitus (endocrine disease) • Schedule II licit (prescription) medications • Cardiovascular disease • Seizure disorders • Sleep disorders • Renal disease • Vision • Musculoskeletal disease • Hearing • Psychiatric disease • Stroke • Multiple sclerosis and Parkinson’s disease • Substance abuse. These and similar reports are intended to help the agency develop qualifications rules using an evidence-based approach. The agency does not necessarily adopt panel rec- ommendations, but provides the reports online for the pur- poses of information sharing and transparency. FMCSA provides guidance to medical examiners (and motor carrier companies) in an online handbook (http:// nrcme.fmcsa.dot.gov/mehandbook/MEhandbook.htm), and also provides training specifications for medical examiners. Carriers’ first obligations are to ensure that their driver hires meet these qualifications. A carrier’s files on drivers must include a copy of the Medical Examiner’s Certificate. Many other health-related resources are available to carri- ers and drivers. The Healthy Trucking Association of Amer- ica (http://www.healthytruck.org) publishes Driver Health magazine and sponsors various driver health initiatives. The American College of Occupational and Environmental Medicine (www.acoem.org) is oriented toward physicians and other medical professions serving industry. This non-

30 governmental organization provides books, instructional programs, and webinars on various occupational safety and health issues and practices. They include a guide to commercial driver medical certification (Hartenbaum et al. 2010), which focuses on the latest DOT regulations but also includes expanded interpretations from the medical literature and recommendations from the FMCSA Medical Review Board. This private sector information supplements that provided by FMCSA. As discussed in chapter two, the medical profile of U.S. commercial drivers is generally poor. Compared with the general population, commercial drivers are more likely to be sedentary, overweight, have a cardiovascular condition, be smokers, and have poor eating habits (Krueger et al. 2007; FMCSA 2010). Medical conditions can reduce driver safety and employment success in three general ways: • Chronic performance decrements • Catastrophic performance failures (termed “critical non-performance” in crash causation studies) • Absenteeism and reduced employment longevity. Carriers are required by law to ensure that drivers meet medical qualifications, but meeting this requirement does not eliminate their concerns regarding crash risk and car- rier liability. Whether a medical condition is identified as the direct cause of a crash or is merely suspected as an associated factor, carriers have high liability exposure when unhealthy drivers are involved in crashes. In some respects, carriers are caught between two needs. On the one hand, drivers meeting all legal medical requirements can still have medical conditions that contribute to crashes and cause liability. On the other hand, employee selection meth- ods should be fair, criterion-based, and legally defensible in relation to all driver traits, including medical conditions. It is important that managers without medical training not be making medical decisions. Employers are given more leeway in regard to medical conditions than other traits, however, because the ADA does not apply to transportation safety-sensitive positions. Accommodations need not be made for commercial driver medical conditions with known linkage to safety risks. In the project safety manager survey, the condition “poor general physical health” was given an average 5-point Likert scale rating of 3.6 by safety managers. This placed it about in the middle of 12 personal characteristics listed in terms of their perceived relation to crash risk. On the safety man- ager (SM) form, 43 of 65 respondents indicated that their driver candidates completed a medical history questionnaire during the selection process. Some top carriers have their own medical units, which perform a standardized medi- cal examination of applicants. This exam may duplicate a driver’s existing medical certification or may involve higher standards. Carriers interviewed seem most concerned about detecting OSA in their driver candidates. Another concern is cardiovascular illness. Both of these conditions have well- estimated associations with elevated crash risk or proximal crash causation (NTSB 1990; Young et al. 1997; Starnes 2006; Krueger et al. 2007; Knipling 2009a). Physical Capabilities Some companies require drivers to pass a physical activity test before hire. Such tests are not intended to detect specific medical conditions, but rather to assess drivers’ and other employees’ abilities to perform the physical tasks required in the job. For example, case study Carrier C tests driver abilities to carry, lift, climb, and crawl, all tasks performed around a truck and as part of the job. A principal motivation for conducting such tests is to reduce workers compensa- tion claims associated with loading/unloading, vehicle entry and exit, and other potentially injurious tasks involved in truck and bus driving. The MediGraph Software Functional Capacity Evaluation (Medigraph FCE; www.functional- capacity-evaluation.com) is an objective procedure to test individual work capability. Its website claims that it has been scientifically peer reviewed. The full FCE requires an array of equipment, including an inclinometer/goniometer (for assessing head movement capability), treadmill, timer/ stopwatch, adjustable-height shelving, lifting box, balance beam, assorted weights, and various smaller items. Specific scored tasks are performed on each. Performance scores on individual tasks generate assessments of capabilities in vari- ous areas, including standing/walking, lifting, pushing/pull- ing, balance, dexterity, and perception. Scale scores can be compared with a government defined job class and its asso- ciated strength requirements from the Dictionary of Occu- pational Titles. Like other physical and psychomotor tests, the FCE could identify some drivers with physical deficits inconsistent with safe driving. Beyond that, it is not intended to differentiate safe and unsafe drivers. Commercially Available Safety-Relevant Selection Tests This section presents commercially available selection tests marketed for use for selecting safe fleet drivers, or that could be promising candidates for such use. Tests are described in regard to the personal traits they seek to measure, how they are administered, test content, and key findings relat- ing to their validity. The similarity of test items to job tasks determines its content validity. The degree to which the test captures conceptual human traits relevant to safety reflects construct validity. The degree to which test scores correlate with job performance criteria, especially in future predic- tions, is its criterion-related validity. Although test validity is a key concern, this project did not formally validate any selection instrument. Motor carriers wishing to use these or other selection instruments should seek more in-depth infor- mation on them, and also fully understand the legal require- ments for selection test use.

31 Disclaimer No selection test or other product or service was formally evaluated for this report. Specific products and services are described as examples for reader edification. No endorsement of any product or service by the authors or by TRB is implied or intended. Much of the information on the following tests was obtained from product websites or, in some cases, direct discussions with test vendors. The authors strive to present only objective information here. When possible, support- ing evidence from the scientific research literature has been cited. More basic scientific research presented in chapter two is also relevant. As the disclaimer also states, however, no endorsement of any product or service by the authors or pub- lisher of this report is intended. DriveABLE The DriveABLE Cognitive Assessment Tool (DCAT, www. driveable.com) is a 30-40 minute computer-based test of dynamic performance (Dobbs, 2009). It was developed and validated in relation to other cognitive tests for the purpose of identifying drivers with cognitive or related sensorimotor deficits predictive of impaired driving. Most often it is used in the assessment of older drivers, and it is effective in cap- turing “competence” errors; that is, errors made by incom- petent drivers but not by those within normal ranges. DCAT includes six kinds of tasks measuring reaction time, span of attentional field, decision making, executive functions, and hazard identification. DCAT presents the test-taker with six dynamic tasks: • Motor speed and control task • Span of attention field task (ability to notice events in the periphery of the visual field) • Spatial judgment and decision-making task (judging space and time in driving maneuvers) • Speed of attentional shifting task (among different hazards when driving) • Executive function task (planning and executing maneuvers) • Identification of driving situations task (recognizing crash threats as they arise). DCAT is not a driving simulator. Most of its tasks resemble simple computer games where the user responds by means of push buttons or touch-screen responses, although the last task presents videos of actual driving situations. Automated test scoring provides normative scores for each task and an overall probability for success in the criterion test, an on- road evaluation. The DriveABLE website reports an overall DCAT prediction accuracy of 95%, with a sensitivity of 93% and a specificity of 82% in relation to an on-road evaluation. Here, sensitivity is defined as the percentage of subjects fail- ing the road test given a test prediction of failure. Specificity is measured by the percentage passing the road test given a test prediction of passing. The test does not attempt to pre- dict success for all subjects, however. No prediction is made for those scoring in the middle, where pass-fail predictions are more likely to be incorrect. DCAT identifies individuals with cognitive impairments but is not predictive of safe driving across normal driving populations. The test may be useful, however, to obtain base- line measures of individual driver performance. These data may be useful if issues arise in the future about a driver’s fitness, such as with school bus drivers, who may drive well into their older years. The development and validation of DCAT (A. R. Dobbs, per- sonal communication, 2010) involved performance compari- sons among three groups of drivers: older cognitively impaired, older normal, and young normal. The two older groups aver- aged about age 70, versus 36 for the young group. Over a 2-day period, each subject performed 14 different timed cognitive tasks and took an on-road driving test. The six dynamic tasks were those most predictive of driving performance. The purpose of DriveABLE is not to classify the full range of drivers but rather to identify those too cognitively impaired to drive safely. Classifying drivers in just two cat- egories based on the test would result in too many incorrect classifications. Therefore, three prediction zones were estab- lished: a strong prediction of road test failure, and indetermi- nate “gray area,” and a strong prediction of road text success. These were applied to 234 older drivers referred for testing by the Florida Department of Motor Vehicles and Highway Safety because they had possible indications of cognitive incapacity for driving. The following “truth table,” orga- nized similar to the Figure 8 selection model, shows the clas- sification results. Although none of the predictions is perfect, drivers in the two extreme prediction groups had sharply dif- ferent success likelihoods in the actual road test. TABLE 2 DCAT VALIDATION “TRUTH TABLE” Road Test Result Predict Fail DCAT Prediction No Prediction Predict Pass Passed Road Test 2% 24% 32% Failed Road Test 18% 19% 4% Dobbs (personal communication 2010) presents a fuller discussion of the validation methodology and results. Simi- lar results are presented for a second validation group. Based on the research, a distinction is made between test and driv- ing errors indicative of cognitive impairment (discriminat-

32 ing errors) and those simply indicative of bad driving habits (nondiscriminating errors). Normal subjects may make mul- tiple nondiscriminating errors, perhaps indicative of care- less driving. The more serious discriminating errors seen in cognitively impaired subjects are indicative of incapacity to drive safely. Daecher Driver Profile The Daecher Driver Profile (www.safetyteam.com) is an online inventory questionnaire taken by drivers to assess their beliefs, attitudes, personality, opinions, and other per- sonal characteristics related to success as a professional driver. The Driver Profile is a 165-item questionnaire that consists of 117 true-false items relating to personality charac- teristics and 48 multiple-choice items on driver background and attitudes predictive of safe driving. Administration time is about 30 minutes for most respondents. The profile is auto- matically scored, with results (an algorithmically derived prediction of the applicant’s probability of success) provided to the employer customer. Daecher’s promotional materials state that the test is “effective in selecting commercial driv- ers who— • Have a high level of safety awareness • Follow rules and regulations • Are responsive to customer problems • Maintain a courteous and professional manner • Are more likely to be seen by their supervisors as ‘superior’ employees.” Development of the profile was funded by a national insurer of commercial vehicle operators. Daecher’s web- site states that the test has been independently validated using a concurrent criterion-related methodology. That is, working commercial drivers’ profile responses and job ratings were compared and found to correlate sig- nificantly. Each of five subtest scores (corresponding to the driver characteristics listed above) correlated mod- erately with driver job performance ratings. According to the website, the study conformed to applicable EEOC guidelines for the validation of selection procedures and does not discriminate against minorities. The company also claims that it is difficult for drivers to falsely make themselves “look good” on the test. A 7-step summary of the Daecher validation process is provided in Appendix B. Their reported validation coefficient is +0.33, putting it in the “likely to be useful” range per the DOL guide- lines discussed earlier. The company also provides a Professional Driver Hir- ing Program guide for “recruiting, screening, and selecting the best candidates.” Appendix F of CTBSSP Synthesis 1 (Knipling et al. 2003) provides related driver selection infor- mation and materials contributed by the Daecher Consulting Group to that effort. WayPoint® WayPoint is a 4-minute Internet-based sensorimotor test, similar in some ways to the Trail-making Test Form B. Both tests were introduced in chapter two. Subjects alternately connect numbered and alphabetized boxes (i.e., 1, A, 2, B, 3, C) that are presented in random spatial patterns of increas- ing complexity. Increased complexity is achieved by adding distracting icons to the mix of letters and numbers. Figure 9 shows WayPoint screens with and without distracters. The dashed line shows the path of error-free performance. FIGURE 9 Plain and embellished WayPoint worksheets. (Courtesy: WayPoint.) Haphazard, mistake-prone WayPoint performance sug- gests a similar approach to driving. When the icons are added to the test, a large decrement in performance suggests that the individual could be highly distractible; for example, by a billboard or a cell phone message. In contrast, accord- ing to company literature, little or no decrement in per- formance (undistractible) suggests that the individual has “tunnel vision” and might not notice peripheral or surprise crash hazards. Neither extreme of distractibility is associ- ated with safe driving; the middle of the distractibility scale is said to be ideal. This U-shaped relationship between distractibility and crash proneness was found in vendor validation studies involving drivers of both trucks and cars. In one study, 63 tractor-semitrailer drivers took the WayPoint assessment. When their test scores were compared with preventable col- lision data from company records, the safest drivers scored in the middle of the distractibility scale, whereas those at both extremes had higher risk. Similar results were reported for a much larger sample of noncommercial drivers. The WayPoint developer also provided unpublished data on 121 Metropoli- tan Atlanta Rapid Transit Authority transit bus operators. WayPoint scores for these operators followed the U-shaped function for preventable crashes, unpreventable crashes, cus- tomer complaints, and absent days. Median WayPoint scores were predictive of best performance per all four job criteria. The concept of a U-shaped relationship between distract- ibility and crash proneness is by no means established as fact, but it could be consistent with existing information about proximal crash causes. In the LTCCS, 19% of truck

33 at-fault multivehicle crashes had a CR of inattention (i.e., distraction, daydreaming), whereas an equal 19% had a CR of “looked but did not see.” “Looked but did not see” could be construed as “undistractibility” in this model of driver crash risk based on WayPoint. Scheig Hiring and Performance System The Scheig system (www.scheig.com) provides a three-phase hiring process based on a job analysis: (1) applicant assess- ment questionnaire; (2) applicant structured interview; and, for those hired, (3) performance evaluation. Scheig’s descrip- tion of its job analysis says that several hundred job-specific behaviors are generated for each job analyzed. Using these job behaviors, Scheig produces behaviorally based job descrip- tions and uses them to develop the assessment questionnaire. The assessment contains two sections, an “interest and will- ingness” checklist and a forced-choice questionnaire. The interest and willingness checklist lists around 100 behaviors, often with an embedded standard of acceptable performance (though one intended not to be obvious to the test-taker). An example behavior is “Seeks assistance, advice, or directions if unsure how to handle a task or situation.” The applicants indi- cate two responses for each behavior: the degree of experience they have doing the behavior, and whether they are willing or unwilling to meet that condition of the job. The forced-choice questionnaire asks applicants to choose between two actual job behaviors. Both choices are intended to sound equally “good,” though one choice actually indicates high perfor- mance and the other indicates low performance in the context of the job analysis. From the same job analysis data, Scheig says it develops a behaviorally based structured interview as a second screening step for those passing the questionnaire phase. The third phase, performance evaluation, is not part of hiring in itself but rather a check on the hiring decision for each new hire, an aid to new employee performance improve- ment, and a method of further validating and refining the whole selection process. Past clients include BASF and Chev- ron in chemicals, and SYSCO and Food Services of America in food preparation and distribution. Virtual Risk Manager Interactive Driver Systems (www.virtualriskmanger.net) incorporates various sources of information to assess indi- vidual driver risk, aggregate risk for a company, and provide risk-reduction training and interventions. Its service compo- nents include the following: • RoadRISK: Online driver questionnaire intended to tap driver safety attitudes, hazard perception, behav- iors, knowledge, and risk exposures. • DriverINDEX: Predictive model to identify clients’ most “at-risk” drivers. • RiskFOUNDATION: Carrier safety policy and prac- tices guide structured as a carrier-driver “safety con- tract” renewed every 12 months. Driver must affirm that he or she agrees to or will abide by 45 safety- related practices. • RiskCOACH: Short training and other recommended interventions aimed at specific risks. • BenchMARKING: Carrier self-audits in which they can anonymously benchmark their company’s crash data and safety standards with other organizations and network with other fleet managers. Virtual Risk Manager uses carrier and driver inputs from audits, crash data, risk assessments, training results, and electronic license checks. The company states that its prod- ucts were developed based on research, trials, and user eval- uations by two universities in the United Kingdom involving groups of 8,000, 16,000, and 26,000 drivers. It also asserts that one truck fleet reduced claims by 25% and driver at-fault incidents 75% over a 12-month period. The company’s website and promotional materials report a study on the RoadRISK application. Six different driving risk measures were compared with individuals’ numbers of collisions. Subjects were mostly engineers and managers rather than commercial drivers. The six measures of risk were as follows: • Exposure to risk, based on 26 questions about age, type of driving, and amount of driving • Attitudes about driving, based on 10 multiple-choice questions • Driving behavior, based on 10 multiple-choice questions • Knowledge of the rules of the road, based on 10 knowl- edge questions • Hazard perception, based on subject responses to the presentation of 15 pictures of potentially hazardous road situations • Total score, a composite of the above. All six scale scores were reported to vary with actual crash experience. The knowledge score was the weakest pre- dictor, whereas the exposure, behavior, and total scores were the strongest predictors. More detail on RoadRISK research was provided in a conference presentation by Rea et al. (2004). In one of several different research studies cited (the one involving 16,000 drivers), drivers with low (bad) RoadRISK scores were 2.2 times more likely to have three or more crashes during a 3-year period than those who scored high (good). The authors acknowledged that part of this effect was derived from exposure differences between the groups. Nevertheless, individual scales were each associated with crash risk. For example, the mean number of crashes over a 3-year period for three attitude-scale groups were as follows:

34 • Low (bad) attitude score (N = 3,616): 0.32 crashes • Medium attitude score (N = 6,200): 0.25 crashes • High (good) attitude score (N = 16, 106): 0.22 crashes. MindData Attitude Index MindData (www.minddata.com) offers a general-purpose employee selection test that is validated against a company’s successful and unsuccessful employees. Its use for selecting drivers for a trucking company, for example, would require administration of the test to current drivers along with objec- tive data on those drivers’ safety or other measures of job performance quality. Its core test, called the MindData Atti- tude Index 100 (MD/100), is a personality profile that gener- ates scale scores for 10 traits: • Aggressiveness—the degree to which wants or demands are made known • Compassion—the level of concern or disinterest in the needs of others • Compliance—the tendency to resist or obey rules and regulations • Diplomacy—the level of communication, from diplo- matic to blunt • Concentration—the ability to concentrate on a task despite distraction • Optimism—the level of optimism or pessimism • Sensitivity—how criticism will be handled • Commitment—the extent to which promises may be reliably kept • Sociability—the extent to which one enjoys or avoids dealing with others • Ethics—a representation of one’s value system. A longer version of the index assesses 10 additional traits: adaptability, anxiety, decisiveness, determination, drive, initiative, meticulousness, organization, stamina, and trust. Some tested traits may be strongly related to driving safety, others moderately, and others not at all. Determination of the relevance of any one scale would be based on data from current employees, as well as other studies of personal traits relevant to safety. For example, traits like aggressiveness and compliance have both face validity (apparent validity) as safety predictors, as well as extensive corroborative evidence from various studies. Other traits like diplomacy, sensitivity, and sociability may be measured reliably by the text but have little or no predictive validity in relation to driving safety. The MindData Attitude Index can be administered either online of offline. The original form of the test has been adju- dicated and approved by a federal court as meeting EEOC validation guidelines, although the company’s website does not indicate the specific jobs to which this validation applies. MindData markets its products as tools for both employee selection and promotion. ProfileXT® Like MindData, ProfileXT is a commercially available general- purpose employee selection instrument that is normed against a company’s current employees. The 60-minute test is admin- istered online and generates specific scale scores under the categories “thinking and reasoning,” “behavioral traits,” and “occupational interests.” Improved employee selection is the principal use of this and similar assessment profiles, but they can also be used for employee placement, promotion, coaching, and job description development. Case Study F describes the use of ProfileXT by a medium-size private carrier to improve its driver hiring. The carrier administered the profile to current drivers, and found that prominent behavioral traits of success- ful drivers included “manageability” and “accommodating- ness.” Occupational interests associated with good drivers included “mechanical” and “people service.” Some personal traits usually prized in employees, including assertiveness, decisiveness, and an occupational interest in enterprise, were not necessarily characteristic of successful drivers. NEO Five-Factor Inventory The NEO Five-Factor Inventory (NEO-FFI) is a 60-item questionnaire that classifies people on five scales: Neuroti- cism, Extraversion, Openness, Agreeableness, and Consci- entiousness. Secondary scales derivable from NEO data can assess additional traits like impulsivity/impatience and Type A personality. As discussed in chapter two, these personality traits are relevant to personal risk perception and risk-related behaviors. The NEO-FFI is used extensively in research, psychological assessment, and personnel selection for non- driving jobs. Its use in selecting drivers or other safety-sen- sitive jobs is probably limited, but some studies have shown that specific NEO scale scores are related to driving safety and also to employee retention. Strong safety evidence comes from a meta-analysis of 47 studies of the five NEO personality factors in occupational and nonoccupational settings (Clarke and Robertson 2005). The meta-analysis found that, across a number of different coun- tries and jobs, individuals low in both agreeableness and con- scientiousness were more likely to be involved in accidents. “The results revealed criterion-related validity for two per- sonality dimensions, agreeableness and conscientiousness, of 0.26 and 0.27, respectively, indicating that individuals low in agreeableness and low in conscientiousness are more liable to be accident-involved.” Another rationale for assessing these two personality traits is that they relate to other aspects of success as a commercial driver. Most notably, agreeableness relates to customer relations, and conscientiousness relates to load security and financial dealings. The study also found that neuroticism (anxiety level) was associated with number of accidents in occupational settings. Extraversion was also an accident predictor, but only in nonoccupational settings.

35 Driver Behavior Questionnaire The Driver Behavior Questionnaire (DBQ; Parker et al. 2001) is a questionnaire that asks subjects to indicate on a six-point scale (from “never” to “all the time”) how often they engage in faulty or dangerous driving behaviors. Exam- ple behaviors include speeding in residential areas, racing starts from traffic lights to beat other drivers, backing into other objects, skidding on a slippery road, and steering the wrong way into a skid. One version of the DBQ has 24 items and yields three measures of driver behavior—violations, errors, and lapses, defined as follows: • Violations: deliberate deviations from rules • Errors: mistakes; intended actions with unintended consequences • Lapses: executions of unintended actions. According to Sullman et al. (2002), only the “violations” score correlates significantly with past and future crash involvement. This relation has been found across many dif- ferent samples and countries, however. Sullman et al. (2002) enlisted the cooperation of five New Zealand trucking com- panies to administer the test to 378 truck drivers. Their most common admitted aberrant behaviors were disregarding highway speed limits, sounding their horns in anger, and showing other forms of anger toward other drivers. Drivers with high DBQ violations were 50% more likely than other drivers to have been involved in a crash over the previous 3 years. They also tended to be younger and less experienced. The study noted that these significant associations emerged from the study, even though truck driver subjects might have understated their bad driving behaviors on the questionnaire. This suggests that the real associations may be even greater than those measured. Unfortunately, it also suggests that if driver applicants took the test, they would “see through” the intent of many questions and minimize any indications of misbehavior and risk. A study conducted in China used a different version of the DBQ to explore relationships among risk perception, risk- taking attitudes, and behavioral history, including serious violations, ordinary violations, and crashes. Ma et al. (2010) administered the DBQ to 248 taxi and bus drivers in Wuhan, China. Subjects responded on a Likert scale to risk percep- tion and risk-taking related items such as the following: • “Worried for yourself being injured in a traffic crash?” • “Many traffic rules must be ignored to ensure traffic flow.” • “If you are a good driver it is acceptable to drive a little faster.” Several statistical methods were used to distill the mul- tiple answers into a smaller number of psychological scales: • Risk perception scales: – Worry and insecurity (emotion-based) – Assessment of crash probability – Concern (cognition-based) • Risk-taking attitudes: – Attitude toward rule violation and speeding – Attitude toward careless driving of others – Attitude toward drinking and driving. Two statistical models, the Logit model and the Structural Equation Model, were used to identify “influential paths” of influence among the scales and driver behavioral history. Inter- relationships were seen between violation history (serious and ordinary), crashes, and various risk perception and risk-taking measures. The scale “attitude toward rule violation and speed- ing” was found to have the strongest interrelationships with other risk perception and behavioral measures. Figure 10 shows these relationships. Thiffault (2007) also noted the associations of violations and attitudes about them with crash risk. FIGURE 10 Correlations among key risk attitudes, risk perceptions, and behaviors. (Source: Based on Ma et al. 2010.) Driver Stress Inventory The Driver Stress Inventory (DSI; Matthews et al. 1996, 1997) assesses driver emotions about driving, including fear, anger, and boredom. The DSI is an experimentally validated questionnaire designed to assess an individual’s vulnerabil- ity to stress during driving and to evaluate the coping meth- ods employed in stressful driving situations. The DSI has two sections. The first section contains 12 items to evaluate driving habits and history, including the number of years a driver has been licensed, the typical number of days driven in a week, the typical roads traveled on, the number of miles driven annually, and the number and severity of accidents in the past 3 years. The second section consists of 48 Lik- ert scale items describing attitudes and emotional reactions experienced while driving. These are designed to assess a driver on five dimensions of driver stress vulnerability: aggression, dislike of driving, hazard monitoring, thrill- seeking, and fatigue proneness. Sample items (each requir- ing a 10-point Likert scale response ranging from “Not at all” to “Very much.”) include the following: • Does it worry you to drive in bad weather? • At times, I feel like I really dislike other drivers who cause problems for me.

36 • I become annoyed if another car follows very close behind my vehicle for some distance. As cited in chapter two, DSI assessments of driver aggression and thrill-seeking have correlations in the +0.40 to +0.60 range with traffic violations and other driving mis- behaviors. Like the DBQ, the DSI is well validated as a pre- dictor of driving behavior and risk when used in research settings, but the intent of its questions would likely be too obvious to applicants for driving jobs. Wonderlic Mental Ability Tests Companies have at least two reasons and justifications for incorporating mental ability testing into their driver hiring pro- cedures. First, unrelated to safety, the commercial driver job involves basic math and other mental skills such as map-read- ing, distance calculations, and keeping logs and other records. For companies that use onboard safety monitoring, it is impor- tant that drivers understand statistics on their driving such as hard-braking rates and fuel economy. The other rationale is evidence, suggesting that more intelligent drivers (as measured by IQ tests) tend to be more patient and make more rational risk choices. There is also compelling evidence that newly hired drivers scoring low on IQ tests are bad retention risks. Wonderlic (www.wonderlic.com/hiring-solutions/products. aspx) is among companies marketing mental ability tests for employee selection. The company’s website states that “cogni- tive ability or general intelligence [is] the strongest single pre- dictor of employment success.” This claim is made in relation to employees and jobs in general, not in relation to driving jobs. Among tests available from Wonderlic are the Wonderlic Per- sonnel Test and the Wonderlic Basic Skills Test (WBST). The WPT-R is a 12-minute test that can be administered online or on paper (with answers faxed to Wonderlic and scored). Sample questions include the following: 1. Three individuals form a partnership and agree to divide the profits equally. X invests $9,000, Y invests $7,000, Z invests $4,000. If the profits are $4,800, how much less does X receive than if the profits were divided in proportion to the amount invested? 2. A boy is 17 years old and his sister is twice as old. When the boy is 23 years old, what will be the age of his sister? 3. PRESENT/RESENT. Do these words: a. Have similar meanings b. Have contradictory meanings c. Mean neither the same nor opposite. In the project safety manager survey, respondents gener- ally considered “low intelligence/mental abilities” to have a moderate association with driving safety. On the five-point Likert scale, responses were concentrated around “3” (Mod- erate Association). Ten of 65 respondents actually used a mental ability test (e.g., of math, reasoning) as part of their selection procedures. Three of the 10 companies profiled in the chapter five case studies use mental ability testing for selection. Safety directors using these believed that their use did contribute to hiring safe drivers. One of them believed strongly, however, that drivers’ mental abilities were far less important than their safety attitudes. TESTS FOR RETENTION LIKELIHOOD To some extent, personal characteristics associated with driving safety overlap with those associated with employ- ment longevity. In addition to this trait overlap, there is a causal link: Unsafe driving can lead directly to termination. This section describes a few employment tests used to pre- dict employee retention. Test research shows that both mental abilities and personality factors are predictive of employee retention/turnover. Some of the same personal traits known to be associated with safe driving have also been found to contribute to employment longevity. Raven’s Standard Progressive Matrices and Advanced Progressive Matrices Distributed by Pearson Assessment (formerly known as Harcourt Assessment; http://www.pearsonassessments. com/pai/), the Standard Progressive Matrices (SPM) is a long-used test of nonverbal reasoning ability. Subjects select which small image best completes a gap in a larger image by matching the pattern of each small image to the pattern in the large image. The Advanced version is suitable for sub- ject groups who have above-average intelligence. The SPM test was first introduced in 1936 by J. C. Raven. The authors of the current version say it measures the abilities to think clearly, to make sense of complexity, and to store and pro- cess information (Raven et al. 2003). The Truckers and Turnover Project (Burks et al. 2008) gave the SPM version of the Raven’s test to 1,065 driver trainees who were new to the trucking industry and study- ing to acquire their CDLs at a school run by an LTL motor carrier. Drivers in this group signed a credit agreement to pay back the market price of their training if they did not complete 1 year of service after training. Only about 35% of trainees starting the program made it through the first year. Investigators found that drivers who scored in the top quarter on the Raven’s were almost twice as likely to complete the year of work as those in the bottom quarter (Burks et al. 2009). The authors suggest that those with higher cognitive skills are better able to manage their time and effort in the face of conflicting and changing demands, such as traffic and weather, hours of service, and customer time limits.

37 The Adult Test of Quantitative Literacy This test of mathematical reasoning or quantitative literacy (also called numeracy) is distributed by the Educational Testing Service (ETS) of Princeton, New Jersey. ETS is widely known as the distributor of the Scholastic Aptitude Test, one of the most widely used standardized tests for high school students who wish to go to college. According to ETS (2010), Quantitative literacy measures how well you can use numbers found in ads, forms, flyers, articles or other printed materials. Quantitative literacy is a little different from prose and document literacy because in addition to using a text to identify needed information, you also have to add, subtract, multiply, divide or do other math to get the information you need. ETS gives several examples, including keeping score for a bowling team or calculating a 15% tip at a restau- rant. The Truckers and Turnover Project (Burks et al. 2008) also gave this test to the same 1,065 driver trainees discussed above. As with the Raven’s, those drivers who scored in the top quarter on the Adult Test of Quantita- tive Literacy were almost twice as likely to complete the year of work as those in the bottom quarter (Burks et al. 2009). The authors attributed this effect to the same rela- tionship between cognitive skills and job performance as discussed previously. NEO Five-Factor Inventory As number of the Big Five personality traits are relevant to personal risk perception and risk-related behaviors. The NEO-FFI is used extensively in research, psychological assessment, and personnel selection for nondriving jobs. A large meta-analysis of 86 empirical studies (Zimmerman 2008) found significant evidence of a relationship between personality factors and voluntary turnover (quit) decisions. Many studies also controlled for other factors, such as job satisfaction and job performance. All five factors had some correlation with quit decisions in the expected direction, with three standing out as being particularly strong: • Neuroticism: +0.18 • Agreeableness: −0.25 • Conscientiousness: −0.20. That is, employees were more likely to quit if the test indicated that they had neurotic (high anxiety) tendencies, were disagreeable, and were less conscientious than other employees tested. A “path model” developed by the author showed direct relationships between these personality traits and intentions to quit and turnover behavior that were not captured through job satisfaction or job performance mea- sures. In addition, “Personality traits had stronger relation- ships with outcomes than did non-self-report measures of job complexity/job characteristics” (Zimmerman 2008).

Next: CHAPTER FOUR Survey Methods and Results »
Driver Selection Tests and Measurement Get This Book
×
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

TRB’s Commercial Truck and Bus Safety Synthesis Program (CTBSSP) Synthesis 21: Driver Selection Tests and Measurement synthesizes information on the use of tests, measurements, and other assessment methods used by commercial truck and bus companies in the driver selection process. The report also identifies and describes driver selection methods and instruments and their potential usefulness in predicting driver crash risk.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!