Naturalistic Driving Study: Alcohol Sensor Performance (2015)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

19 Conclusions and Suggested Research Researchers used a multifaceted approach to further inves- tigate the standard alcohol sensor in SHRP 2 vehicles and develop an initial alcohol-detection algorithm based on the sensor data. The second phase of research was designed with this goal in mind. Specifically, the objectives were to (1) deter- mine the necessary considerations in an alcohol-detection algorithm to be applied to the SHRP 2 database and (2) eval- uate the accuracy of an algorithm using these considerations at detecting and differentiating between imbibed and unim- bibed alcohol in the SHRP 2 database. Through an examina- tion of SHRP 2 data and several experimental manipulations, an alcohol-detection algorithm was developed and investi- gated. This allowed for a basic evaluation of an algorithm and an assessment of the usefulness of the sensor for identifying imbibed alcohol. A summary of conclusions and recommen- dations from this research follows. Alcohol-Detection Algorithm Accuracy To maximize its utility, this type of algorithm must detect cases of imbibed alcohol and also filter out the effects of other substances that can affect the alcohol sensor and produce false detections. After all, the sensor was only designed to detect general alcohol presence within the cabin. At the conception of this project, little was known about the alcohol sensor’s reaction in the presence of various types of alcohol. Thus, characterization of the sensor’s response to both imbibed and unimbibed alcohol was necessary. Sensor’s Ability to Detect Alcohol Presence To answer the fundamental question of the sensor’s alcohol- detection accuracy, an experimental gold standard data set was created. While it was difficult to confirm alcohol impair- ment in the SHRP 2 data set, the gold standard data set allowed for the careful distribution of alcohol into the cabin of a vehicle using a mechanical breather. This provided an ideal metric for evaluating the alcohol-detection algorithm. When tested against this data set, the algorithm was over 95% accurate at differentiating alcohol presence from no alcohol presence. The only inaccuracies were a few trips with low doses of alcohol that barely missed the detection threshold. Differentiating Unimbibed and Imbibed Alcohol Presence A variety of substances that are naturally introduced into motor vehicles contain alcohol. To reduce false alerts, an alcohol-detection algorithm must be able to differentiate these unimbibed forms of alcohol from the presence of humans in the vehicle who have imbibed alcohol. This ability was assessed by creating a naturalistic test data set from the SHRP 2 database that was heavily weighted toward positive alcohol sensor readings. A variety of substances were identified that had an impact on the alcohol sensor readings. The most common substance was windshield wiper fluid—although other substances were shown to affect the sensor when introduced into the vehicle. These substances generally had a similar effect: a steep drop in alcohol sensor readings followed by a gradual return to base- line for the given trip. However, this was not always the case. For example, fast food often resulted in a slow, less severe drop in alcohol sensor readings. As another example, when wind- shield wiper fluid was used to melt ice on the windshield, it often had a lingering, constant presence with an unusually slow or even absent return of alcohol sensor readings to baseline. It is hypothesized that this occurred because ice saturated by windshield wiper fluid remained on the windshield. This intro- duced a constant stream of alcohol vapors into the cabin, thus making it difficult to differentiate imbibed from unimbibed alcohol. While the alcohol-detection algorithm was highly accurate in determining alcohol presence within a vehicle, it was only C h A p t e r 4

20 weakly to moderately able to differentiate imbibed versus unimbibed alcohol. In particular, the alcohol-detection algo- rithm tended to classify unimbibed alcohol trips as imbibed trips, which meant a tendency for false positives. For most analyses, false positives are better than false negatives. Visual data validation can remove trips that were actually the result of unimbibed substances, but false negatives cannot be feasi- bly reduced through a manual validation process (all video would need to be reduced for signs of alcohol). The difficulty in differentiating imbibed from unimbibed alcohol stemmed from a variety of sources. The effects of unimbibed alcohol on sensor readings can be long lasting and, thus, similar to the signature left by imbibed alcohol. Additionally, many sources of unimbibed alcohol are intro- duced into the vehicle at the beginning of a trip, placing the steep spike in sensor readings (a strong differentiating char- acteristic between imbibed and unimbibed alcohol) in the sensor warm-up period. As a result, this spike is not detected or assessed by the alcohol-detection algorithm. The warm-up period is occasionally marked by a spike in sensor readings even without the presence of unimbibed or imbibed alcohol. Thus, the characteristic spike of unimbibed alcohol is masked by this warm-up period. Finally, from the data on the DAS, there is no way to determine many of the changes in the cabin that can influence the sensor and further mask unimbibed alcohol (e.g., HVAC settings and window position). Detecting BrAC Differences The primary purpose of the alcohol-detection algorithm was to provide a yes or no estimate of whether or not an impaired individual was within a vehicle. Substantial benefits emerge from this binary rating. However, it was also worth examining whether the alcohol sensor could be used to estimate level of intoxication. Results from the gold standard data set revealed a moderate negative correlation between BrAC and alcohol sensor values. Unfortunately, while readings within a sensor could reflect level of intoxication (albeit with a large margin of error), these readings were not consistent across sensors and could not be interpreted to measure BrAC. The potential remains for a rough calibration of each sen- sor individually to attempt concentration assessment. This would require fairly accurate estimates of intoxication within a vehicle and using that as a basis for interpreting other read- ings on the same sensor. This could potentially be achieved by calculating the average alcohol sensor reading per trip and standardizing those readings across trips for a given alcohol sensor. From this information, a trip could be assessed on the basis of the standard deviation of the alcohol sensor reading for that trip compared with other trips in the same vehicle (i.e., the same sensor). While this may potentially provide a very rough estimate of intoxication, it remains uncertain if such an approach would be feasible or functional. Ultimately, other challenges with calculating BrAC made this calibration unrealistic, and future research should not consider trying to calculate BrAC from the SHRP 2 database. For example, the absolute value of sensor readings could change depending on the size and position of the intoxicated individual, the number of intoxicated individuals, the presence of other sources of alcohol within the vehicle (i.e., unimbibed alcohol), air circulation, windows being up or down, humid- ity, vehicle cabin volume, and a number of other factors. It is not feasible to assess BrAC with reasonable accuracy from the SHRP 2 sensor. Imbibed alcohol detection should be thought of as a binary classification rather than as a measure of concentration. The Algorithm’s Ability to Detect Moderately Impaired Drivers from Baseline Driving Ultimately, the alcohol-detection algorithm must be accurate at determining the presence of alcohol in SHRP 2 trips. Since this is naturalistic data, it is unknown how much alcohol, if any, someone has consumed. The ground truth measure used for assessing alcohol consumption came from video review of the drivers. It was difficult to confirm alcohol impairment in SHRP 2 trips via video review when there was potentially minor impairment with few or no behavioral cues. Therefore, some trips may have been misclassified in the test data set as “no imbibed alcohol” when in fact imbibed alcohol was pres- ent but the video reduction team could not see it. In 91 trips, the driver was judged to be moderately impaired. These trips were combined with the 97 control trips to assess how well the algorithm would do at differentiating moderately impaired trips from “normal” driving. In this case, the algo- rithm performed quite well, indicating that the sensor and an associated impairment algorithm could be used to identify trips in which moderate impairment was likely. In general, the algorithm appeared to perform better as observable impair- ment increased. This result supports running an algorithm across the SHRP 2 data to isolate potential imbibed alcohol use. However, false alarms due to other alcohol sources appear to be quite common in the data set and will continue to be a problem. Therefore, running this algorithm and accepting a positive result for imbibed alcohol without further verifica- tion is not recommended. recommendations These results suggest many future directions for research and provide insight into future use of the alcohol sensor with SHRP 2 data. Several key recommendations follow.

21 Recommendation 1: Consider the Following Criteria in an Alcohol-Detection Algorithm for Use with the SHRP 2 Data Any algorithm that is developed for use on the alcohol sensor should consider the following criteria: • There is large variance in the warm-up period for this sensor before it begins to produce stable readings. This can last as long as 2 minutes. • An artifact in the data effectively creates a sensor shadow (e.g., 15 mV offset) that should be filtered out or ignored. • The absolute value of sensor readings can potentially vary from sensor to sensor. Nominally, a threshold of 3,965 mV should be established for the alcohol-detection algorithm (assuming no unimbibed alcohol). A more sensitive approach would be to obtain a stable baseline from each sensor. • Unimbibed alcohol should be considered in reducing false positives. A quick change in alcohol sensor readings (i.e., steep slopes) is often associated with unimbibed alcohol. • Many factors can influence this sensor, including temperature change, air flow and circulation, and number of passengers. Recommendation 2: Broadly Use the Alcohol-Detection Algorithm to Find Impaired Trips The applied alcohol-detection algorithm was not without error, yet it regularly performed better than chance across multiple research efforts and data sets. Indeed, the sensitivity was over 90% across all of the various data sets and approaches. This suggests that the alcohol sensor data are useful at identi- fying alcohol-impaired trips and providing an initial indicator of the potential for alcohol involvement within a trip. The SHRP 2 data set provides a unique glimpse into alcohol- impaired driving. Considering the high hit rate of the alcohol- detection algorithm, it could be of the utmost importance in gleaning information on impaired driving from the SHRP 2 database. Identified trips can then be further explored using trained data reductionists to differentiate driver impairment from other potential unimbibed alcohol sources. Recommendation 3: Always Accompany Use of the Algorithm with Trained Data Reductionists While the alcohol-detection algorithm performed well at identifying impaired trips, many known barriers restricted its accuracy. In particular, sources of unimbibed alcohol can produce significant errors in interpreting the alcohol sensor readings. These sources appear to be quite common in the SHRP 2 database. Many of these sources of unimbibed alco- hol can be visually identified by a trained data reductionist. In addition to finding sources of unimbibed alcohol, highly trained data reductionists can and should be used to vali- date the results of any alcohol-detection algorithm using the SHRP 2 alcohol sensor. The accuracy of the results of an algo- rithm could be enhanced with confirmation by trained data reductionists. The misclassification of other alcohol sources as intoxication by the algorithm might provide erroneous con- clusions regarding alcohol-impaired driving in the SHRP 2 database if trips are not visually validated by specially trained data reductionists. Recommendation 4: Do Not Disregard the Impact of Unimbibed Alcohol Presence Much of this report discusses unimbibed alcohol as a pri- mary source of error in the alcohol-detection algorithm. However, some of these substances (e.g., cigarettes, fast food, hand sanitizer) may involve distracted driving that has an impact on driver behavior and performance. Thus, a secondary benefit of the alcohol-detection algorithm may be to identify other substances that may also affect driving performance. Overall Conclusion The alcohol sensor of the standard SHRP 2 instrumentation was designed to detect alcohol vapors within a cabin. At the beginning of this effort, it was uncertain if the alcohol sensor accurately performed this function. Assuming the function was fulfilled, it was even more uncertain if alcohol sensor readings could reliably differentiate imbibed versus unimbibed alcohol within a vehicle. To answer these questions, the mechanical breather and SHRP 2 data sets were used to explore the accuracy of an alcohol-detection algorithm. Considering the scope and detail of the SHRP 2 data set, an alcohol-detection algorithm could shed valuable light on alcohol-impaired driving. How- ever, many challenges remain for the broad implementation of an algorithm using this sensor. Despite its relatively high suc- cess rate, care should be taken when using an algorithm and this sensor. Other substances can result in false positives, and visual inspection of SHRP 2 data should almost always accom- pany the algorithm’s application in scientific endeavors.