National Academies Press: OpenBook

Improving Motor Carrier Safety Measurement (2017)

Chapter: Appendix B: Details of the Safety Measurement System Algorithm

« Previous: Appendix A: Agendas from Open Portions of Panel Meetings
Suggested Citation:"Appendix B: Details of the Safety Measurement System Algorithm." National Academies of Sciences, Engineering, and Medicine. 2017. Improving Motor Carrier Safety Measurement. Washington, DC: The National Academies Press. doi: 10.17226/24818.
×

Appendix B

Details of the Safety Measurement System Algorithm

DATA INPUT

The data used for input to the Safety Measurement System (SMS) comes in four separate segments of the Motor Carrier Management Information System (MCMIS): (1) carrier census data, (2) inspection data, (3) violation data, and (4) crash data. Carrier census data include U.S. Department of Transportation (DOT) number, vehicle miles traveled, address of headquarters, active status, interstate/intrastate, operational status code, equipment type and count, cargo classification, number of power units, and whether the carrier transports hazardous materials. Inspection data include: date/time of the inspection, truck or bus carrier, vehicle configuration, cargo body type, gross vehicle weight range, date, state and county, inspection level, number of violations, and whether the vehicle had enough hazardous material to require a placard. Violation data—which are able to be linked to the relevant inspection—include violation code, whether a citation was issued, whether the violation has been adjudicated and the result, whether the violation is a postcrash violation, whether the violation resulted in an out-of-service order, the associated Behavior Analysis and Safety Improvement Categories (BASICs), the associated severity weight, and the out-of-service weight (see below). Crash data include DOT number, location, time/date, sequence of events, number of vehicles involved, type of vehicle, weather, light condition, injuries, fatalities, whether a vehicle was towed-away, whether hazardous materials were released, and the severity crash weight (see below).

Suggested Citation:"Appendix B: Details of the Safety Measurement System Algorithm." National Academies of Sciences, Engineering, and Medicine. 2017. Improving Motor Carrier Safety Measurement. Washington, DC: The National Academies Press. doi: 10.17226/24818.
×

Input Data Filters. There are several filters or checks that have to be passed before data are used in the calculation of the BASIC measures. (These are in addition to the data sufficiency standards, discussed below.) They are: (1) for carrier data: carriers must be active; (2) for inspection data: the inspections must be in the past 24 months, and the type of inspection must match the BASIC they are being applied to; (3) for violation data: in cases of multiple counts of the same violations, SMS only counts a violation once, and if any of the duplicates have an out-of-service attribute, that is made an attribute of the single counted violation; and (4) for crash data: the crash must have occurred in the past 24 months, and it must have been a tow-away crash, or one with injuries or fatalities.

NUMERATORS OF MEASURES: WEIGHTINGS OF VIOLATIONS

The numerators of the seven BASIC scores are sums over violations of the product of weights. The following are the weights that make up these products:

Severity Weights. Severity weights are assigned to violations to reflect the degree of their correlation with crash occurrence and/or crash consequences, ranging from 1 for the least important to 10 for the most important. Severity weights are only comparable within and not across BASICs. In other words, a severity of, for example, 5 for one BASIC is not necessarily comparable to a severity of 5 for a different BASIC. A severity weight is increased by 2 when the violation in question results in an out-of-service order. Also, the sum of all violation severity weights for any single inspection in any one BASIC is limited to 30.

Crash Severity. Crashes are assigned different weights according to the seriousness of the crash. (This is only to compute the Crash BASIC.) The idea is to assign greater weight to crashes involving injuries, fatalities, release of hazardous materials (HM), and a lower weight is assigned otherwise. The crash severity weights are as follows:

Crash Type Crash Severity Weight
Involves tow-away but no injury 1
Injury or fatality 2
Involves an HM release Crash Severity Weight + 1

Time Weights. Inspections, violations, and crashes are weighted according to how recently they occurred. Violations and crashes that took place

Suggested Citation:"Appendix B: Details of the Safety Measurement System Algorithm." National Academies of Sciences, Engineering, and Medicine. 2017. Improving Motor Carrier Safety Measurement. Washington, DC: The National Academies Press. doi: 10.17226/24818.
×

within the past 6 months receive a weight of 3, those that took place between 6 months ago and 1 year ago are given a weight of 2, and those that occurred between 1 year and 2 years ago receive a weight of 1.

Therefore, the numerators of the BASICS are either sums of the product of severity weights and time weights for the noncrash BASICs, or sums of the product of crash severity weights and time weights for the Crash BASIC.

CALCULATION OF DENOMINATORS: MEASURES OF EXPOSURE

Since the number of crashes experienced and violations obtained during inspections are a function of time on the road, it is important to normalize the number of violations, or the number of crashes, by a measure of time driving, or exposure, in order to make comparisons meaningful. SMS makes use of three measures of exposure to construct its denominators.

For BASICs related to driver inspections, the denominators used are the total time weight associated with the relevant inspections. This normalization is used for Hours-of-Service Compliance, Controlled Substances/Alcohol, and Driver Fitness. The second type of denominator relates to vehicle inspections; the normalization is similar to the first denominator, except that it pertains only to vehicle-based inspections and the resulting violations, and is used for BASICS for Vehicle Maintenance and HM Compliance (with the latter only for inspections for Hazardous Materials carriers). Third, for the Unsafe Driving BASIC, for behaviors that usually prompt an inspection, such as speeding or swerving, the normalization

TABLE B-1 Definition of Utilization Factors for Combination Segment

Combination Segment
Vehicle Miles Traveled (VMT) per Average Power Unit (PU) Utilization Factor
< 80,000 1
80,000–160,000 Image
160,000–200,000 1.6
> 200,000 1
No Recent VMT Information 1

SOURCE: Table 3.1 in SMS Methodology, Federal Motor Carrier Safety Administration (2016).

Suggested Citation:"Appendix B: Details of the Safety Measurement System Algorithm." National Academies of Sciences, Engineering, and Medicine. 2017. Improving Motor Carrier Safety Measurement. Washington, DC: The National Academies Press. doi: 10.17226/24818.
×

TABLE B-2 Definition of Utilization Factors for Straight Segment

Straight Segment
VMT per Average PU Utilization
< 20,000 1
20,000–60,000 Image
60,000–200,000 3
> 200,000 1
No Recent VMT Information 1

NOTE: PU, power unit; VMT, vehicle miles traveled.

SOURCE: Table 3.2 in SMS Methodology, FMCSA (2016a).

is the Average Power Units (APU) multiplied by the utilization factor. (See Tables B-1 and B-2 for definitions of utilization factors for combination and straight segments.) Similarly, the Crash Indicator BASIC is also normalized by the same denominator.

DEFINITION OF THE SEVEN DIFFERENT BASICS

SMS groups violations into categories, which are referred to as BASICs. They are as follows:1

  • Unsafe Driving BASIC—This BASIC measures “Operation of commercial motor vehicles (CMVs) in a dangerous or careless manner. Example violations include: speeding, reckless driving, improper lane change, texting while operating a CMV, not wearing safety belts.2,3 The measure is defined as follows:

    Image

  • HOS [Hours of service] Compliance BASIC—This BASIC measures “Operation of CMVs by drivers who are ill, fatigued, or in noncompliance with the HOS regulations. This BASIC includes violations of regulations pertaining to records of duty status (RODS) as they relate to HOS requirements and the management

___________________

1 Taken from FMCSA (2016a).

2 In cases of multiple counts of the same violation, each violation counts only once per inspection.

3 Violations can also result from inspections motivated by traffic enforcement stops for moving violations.

Suggested Citation:"Appendix B: Details of the Safety Measurement System Algorithm." National Academies of Sciences, Engineering, and Medicine. 2017. Improving Motor Carrier Safety Measurement. Washington, DC: The National Academies Press. doi: 10.17226/24818.
×
  • of CMV driver fatigue. Example violations include: operating a CMV while ill or fatigued, requiring or permitting a property-carrying CMV driver more than 11 hours, failing to preserve RODS for 6 months/failing to preserve supporting documents.” The measure is defined as follows:

    Image

  • Vehicle Maintenance BASIC—This BASIC measures “Failure to properly maintain a CMV and prevent shifting loads, spilled or dropped cargo, and overloading of a CMV. Example violations include: inoperative brakes, lights, and other mechanical defects, improper load securement, failure to make required repairs.” The equation used to calculate the BASIC is as follows:

    Image

  • Controlled Substances/Alcohol BASIC—This BASIC measures “Operation of CMVs by drivers who are impaired due to alcohol, illegal drugs, and misuse of prescription or over-the-counter medications. Example violations include: use or possession of controlled substances or alcohol, failing to implement an alcohol and/or controlled substance testing program.” The measure is defined as follows:

    Image

    (Since all such violations are considered to be out-of-service violations, it is not the case that two additional severity points are added to any violations for this BASIC.)

  • HM [hazardous materials] Compliance BASIC—This BASIC measures “Unsafe handling of HM on a CMV. Example violations include: failing to mark, label, or placard in accordance with the regulations, not properly securing a package containing HM, leaking containers, failing to conduct a test or inspection on a cargo tank when required by the United States Department of Transportation (U.S. DOT).” The measure is defined as follows:

    Image

Suggested Citation:"Appendix B: Details of the Safety Measurement System Algorithm." National Academies of Sciences, Engineering, and Medicine. 2017. Improving Motor Carrier Safety Measurement. Washington, DC: The National Academies Press. doi: 10.17226/24818.
×
  • Driver Fitness BASIC—This BASIC measures “Operation of CMVs by drivers who are unfit to operate a CMV due to lack of training, experience, or medical qualifications. Example violations include: failing to have a valid and appropriate commercial driver’s license (CDL), being medically unqualified to operate a CMV, failing to maintain driver qualification files.” The measure is computed as follows:

    Image

  • Crash Indicator BASIC—This BASIC measures “Historical pattern of crash involvement, including frequency and severity. The BASIC is based on information from state-reported crashes that meet reportable crash standards. All reportable crashes are used regardless of the carrier’s or driver’s role in the crash. This BASIC uses crash history that is not specifically a behavior but instead the consequence of a behavior or a set of behaviors.” This BASIC uses state-reported crash data in MCMIS to form the following measure:

    Image

    A reportable crash is one that results in at least one fatality, one injury requiring transportation to a medical facility, or one vehicle needing to be towed from the scene.

DATA SUFFICIENCY STANDARDS

BASIC scores are only computed for carriers that have had sufficient activity to justify that the computation will not have an unreasonably large variance. The sufficiency standards that have been instituted by FMCSA are different for the different BASICs, reflect information from the previous 24 months, and are as follows:

  • Unsafe Driving: Carriers with at least three inspections with at least one violation
  • Hours of Service: Carriers with at least three relevant inspections and at least one inspection with at least one violation
  • Vehicle Maintenance: Carriers with at least five relevant inspections and at least one inspection resulting in one violation
  • Controlled Substance/Alcohol: Carriers with at least one violation
Suggested Citation:"Appendix B: Details of the Safety Measurement System Algorithm." National Academies of Sciences, Engineering, and Medicine. 2017. Improving Motor Carrier Safety Measurement. Washington, DC: The National Academies Press. doi: 10.17226/24818.
×
  • HM Compliance: Carriers with at least five relevant inspections and where at least one inspection resulted in a violation
  • Driver Fitness: Carriers with at least five relevant inspections and one inspection resulting in a violation
  • Crashes: Carriers with at least two applicable crashes

During 2014–2015, 38 percent of active carriers had sufficient data to be scored for at least one BASIC, and these carriers had 92 percent of the crashes during that time period. (There is also a Critical Mass Threshold Test that is similar to the data sufficiency standard, but it sets a higher minimum standard before an intervention can be taken.)

STRATIFICATION BY TYPE AND SIZE OF CARRIER

SMS makes use of two types of stratifications in the sense that its computations and rankings are carried out separately within these strata:

Type of Carrier. Unsafe Driving and Crash Indicator BASICs are stratified (referred to as being segmented) by combination trucks/motorcoaches and straight trucks/other vehicles. For carriers with both types of vehicles, if 70 percent or more of the power unit types are combination trucks/motorcoaches, the carrier is assigned to the combination stratum, and otherwise it is assigned to the straight stratum.

Safety Event Groups. Carriers are meant to be compared to other carriers that are of a comparable size. For that reason, for each BASIC, SMS places carriers in what are referred to as safety event groups based on the number of events over the previous 24 months (e.g., inspections, violations, and crashes depending on the BASIC in question) in which they have been involved during the previous 2-year period. This allows comparison of carriers that have had roughly comparable levels of exposure. The safety event groups are defined as follows for the Combination Segment (see Table B-3) and Straight (see Table B-4) Safety Event Groups.

TRANSFORMING SCORES TO PERCENTILES

FMCSA believes that SMS should, at its core, be a relative measure of carriers’ safety performance, rather than an absolute measure. This is because FMCSA’s resources in support of interventions are fixed, and therefore SMS has to function as a priority ranking system, used to identify a specific number of carriers for intervention. Toward this end, the scores for carriers within a safety event group are ranked and the percentile associated with each score are computed for each carrier.

Suggested Citation:"Appendix B: Details of the Safety Measurement System Algorithm." National Academies of Sciences, Engineering, and Medicine. 2017. Improving Motor Carrier Safety Measurement. Washington, DC: The National Academies Press. doi: 10.17226/24818.
×

TABLE B-3 Definitions of Combination Safety Event Groups

Safety Event Group Unsafe Driving BASIC Hours-of-Service Compliance BASIC Vehicle Maintenance BASIC Controlled Substance/Alcohol BASIC Hazardous Materials Compliance BASIC Driver Fitness BASIC Crash Indicator BASIC
Number of Inspections with Unsafe Driving Violations Number of Relevant Inspections Number of Relevant Inspections Number of Relevant Inspections Number of Relevant Inspections Number of Relevant Inspections Number of Crashes
1 3–8 3–10 5–10 1 5–10 5–10 2–3
2 9–21 11–20 11–20 2 11–15 11–20 4–6
3 22–57 21–100 21–100 3 16–40 21–100 7–16
4 58–149 101–500 101–500 4+ 41–100 101–500 17–45
5 150+ 501+ 501+ 101+ 501+ 46+
Suggested Citation:"Appendix B: Details of the Safety Measurement System Algorithm." National Academies of Sciences, Engineering, and Medicine. 2017. Improving Motor Carrier Safety Measurement. Washington, DC: The National Academies Press. doi: 10.17226/24818.
×

TABLE B-4 Definitions of Straight Safety Event Groups

Safety Event Group Unsafe Driving BASIC Hours-of-Service Compliance BASIC Vehicle Maintenance BASIC Controlled Substance/Alcohol BASIC Hazardous Materials Compliance BASIC Driver Fitness BASIC Crash Indicator BASIC
Number of Inspections with Unsafe Driving Violations Number of Crashes
1 3–4 See above for combination segment See above for combination segment See above for combination segment See above for combination segment See above for combination segment 2
2 5–8 3–4
3 9–18 5–8
4 19–49 9–26
5 50+ 27+
Suggested Citation:"Appendix B: Details of the Safety Measurement System Algorithm." National Academies of Sciences, Engineering, and Medicine. 2017. Improving Motor Carrier Safety Measurement. Washington, DC: The National Academies Press. doi: 10.17226/24818.
×

TABLE B-5 Intervention Thresholds for BASICs

BASIC Intervention Thresholds
Passenger Carrier Hazardous Materials (HM) General
Unsafe Driving, Crash Indicator, HOS Compliance 50% 60% 65%
Vehicle Maintenance, Controlled Substances/Alcohol, Driver Fitness 65% 75% 80%
HM Compliance 80% 80% 80%

NOTE: There are specific definitions of passenger carriers and HM carriers on pp. 2–9 of FMCSA (2016a).

FMCSA’s analysis has demonstrated that the Unsafe Driving, Crash Indicator, and HOS Compliance BASICs have the strongest linkage to crash risk, and therefore their thresholds, seen above, are lower. In addition, because the consequences of any resulting crashes are greater, the thresholds are lower for passenger carriers and for HM carriers.

There are a few additional rules regarding computation of percentile ranks. Percentiles are only generated from scores of U.S. interstate and hazardous material (HM) carriers. For the remaining carriers, percentiles are assigned based on interpolations of the percentiles for interstate and HM carriers.

Further, there is a 12-month rule, in which the percentiles for carriers that have had all of their violations (or crashes) occur more than 12 months ago are dropped (sometimes there is the added condition that no violation was recorded in the last inspection whenever it occurred). The percentiles for the remaining carriers remain unchanged after those are eliminated.

Carriers with percentiles above a designated threshold are flagged to receive an Alert, which typically indicates that they will receive an intervention. (Alerts can also be triggered as a result of investigations. Alerts were not discussed in this report.) The thresholds are summarized in Table B-5.

Suggested Citation:"Appendix B: Details of the Safety Measurement System Algorithm." National Academies of Sciences, Engineering, and Medicine. 2017. Improving Motor Carrier Safety Measurement. Washington, DC: The National Academies Press. doi: 10.17226/24818.
×
Page 135
Suggested Citation:"Appendix B: Details of the Safety Measurement System Algorithm." National Academies of Sciences, Engineering, and Medicine. 2017. Improving Motor Carrier Safety Measurement. Washington, DC: The National Academies Press. doi: 10.17226/24818.
×
Page 136
Suggested Citation:"Appendix B: Details of the Safety Measurement System Algorithm." National Academies of Sciences, Engineering, and Medicine. 2017. Improving Motor Carrier Safety Measurement. Washington, DC: The National Academies Press. doi: 10.17226/24818.
×
Page 137
Suggested Citation:"Appendix B: Details of the Safety Measurement System Algorithm." National Academies of Sciences, Engineering, and Medicine. 2017. Improving Motor Carrier Safety Measurement. Washington, DC: The National Academies Press. doi: 10.17226/24818.
×
Page 138
Suggested Citation:"Appendix B: Details of the Safety Measurement System Algorithm." National Academies of Sciences, Engineering, and Medicine. 2017. Improving Motor Carrier Safety Measurement. Washington, DC: The National Academies Press. doi: 10.17226/24818.
×
Page 139
Suggested Citation:"Appendix B: Details of the Safety Measurement System Algorithm." National Academies of Sciences, Engineering, and Medicine. 2017. Improving Motor Carrier Safety Measurement. Washington, DC: The National Academies Press. doi: 10.17226/24818.
×
Page 140
Suggested Citation:"Appendix B: Details of the Safety Measurement System Algorithm." National Academies of Sciences, Engineering, and Medicine. 2017. Improving Motor Carrier Safety Measurement. Washington, DC: The National Academies Press. doi: 10.17226/24818.
×
Page 141
Suggested Citation:"Appendix B: Details of the Safety Measurement System Algorithm." National Academies of Sciences, Engineering, and Medicine. 2017. Improving Motor Carrier Safety Measurement. Washington, DC: The National Academies Press. doi: 10.17226/24818.
×
Page 142
Suggested Citation:"Appendix B: Details of the Safety Measurement System Algorithm." National Academies of Sciences, Engineering, and Medicine. 2017. Improving Motor Carrier Safety Measurement. Washington, DC: The National Academies Press. doi: 10.17226/24818.
×
Page 143
Suggested Citation:"Appendix B: Details of the Safety Measurement System Algorithm." National Academies of Sciences, Engineering, and Medicine. 2017. Improving Motor Carrier Safety Measurement. Washington, DC: The National Academies Press. doi: 10.17226/24818.
×
Page 144
Next: Appendix C: Item Response Theory Example Using Motor Carrier Management Information System Data - Jacob Spertus »
Improving Motor Carrier Safety Measurement Get This Book
×
Buy Paperback | $60.00 Buy Ebook | $48.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Every year roughly 100,000 fatal and injury crashes occur in the United States involving large trucks and buses. The Federal Motor Carrier Safety Administration (FMCSA) in the U.S. Department of Transportation works to reduce crashes, injuries, and fatalities involving large trucks and buses. FMCSA uses information that is collected on the frequency of approximately 900 different violations of safety regulations discovered during (mainly) roadside inspections to assess motor carriers’ compliance with Federal Motor Carrier Safety Regulations, as well as to evaluate their compliance in comparison with their peers. Through use of this information, FMCSA’s Safety Measurement System (SMS) identifies carriers to receive its available interventions in order to reduce the risk of crashes across all carriers.

Improving Motor Carrier Safety Measurement examines the effectiveness of the use of the percentile ranks produced by SMS for identifying high-risk carriers, and if not, what alternatives might be preferred. In addition, this report evaluates the accuracy and sufficiency of the data used by SMS, to assess whether other approaches to identifying unsafe carriers would identify high-risk carriers more effectively, and to reflect on how members of the public use the SMS and what effect making the SMS information public has had on reducing crashes.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!