Analysis of Existing Data: Prospective Views on Methodological Paradigms (2012)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

1Background In the spring of 2007, Pennsylvania State University (Penn State) was awarded a contract to analyze existing naturalistic databases as part of the S01 Safety project within the second Strate- gic Highway Research Program (SHRP 2). The Penn State team proposed using data collected by the Virginia Tech Transportation Institute (VTTI) during the 100-car naturalistic driving study (Dingus, Klauer, et al. 2006) and data from the automotive collision avoidance system (Ervin et al. 2005) and the road departure crash warning (RDCW) system field operational test (LeBlanc et al. 2006) conducted by the University of Michigan Transportation Research Institute (UMTRI). The next two subsections describe the analyses undertaken with each data set. The final section summarizes the findings of the research in terms of the five research questions identified in the original Penn State proposal and reiterated in the final Phase 1 report to SHRP 2. Analysis of VTTI Data Two parallel tracks were pursued in the analysis of the 100-car study data: event-based modeling and driver-based modeling. The first approach modeled the occurrence of each event in detail. The focus was on understanding the interactions of the many factors that led to event occur- rence. This initiative fit nicely with the data provided by VTTI, as it allowed events to be com- pared at three levels (summary definitions provided by Dingus, Klauer, et al. 2006): • Crash—any contact with an object, either moving or fixed, at any speed, in which kinetic energy is measurably transferred or dissipated; • Near crash—a circumstance that requires a rapid, evasive maneuver by the subject vehicle, or any other vehicle, to avoid a crash; the maneuver causes the vehicle to approach the limits of its capabilities (e.g., vehicle braking greater than 0.5 g or steering input resulting in lateral acceleration greater than 0.4 g); and • Crash-relevant incident (in this report referred to as a critical incident)—a circumstance that requires a crash avoidance response on the part of the subject. Each of these events was identified by VTTI staff as part of the 100-car study, and the three event types were provided to Penn State in response to the team’s data request. Penn State devel- oped a structured analysis framework for these event-based data; the model specified driver attri- butes, the context in which the event occurred (including roadway and environmental variables), and attributes describing details about the event itself, particularly in the few seconds before and during the event. Examples of event-level variables include whether the driver was observed to be Executive Summary

2distracted just before the event and whether the vehicle crossed over the lane or road edge. One may think of these models as exploring the details of factors associated with the events. Various model formulations were used to find variables associated with crashes and near crashes, and the attributes of vehicle motion associated with such events (e.g., vehicle over lane or road edge) that could serve as surrogate measures for crashes were investigated. If these event- related measures were shown as being positively associated with a crash or near-crash event, they were considered as potential surrogates. A set of nonincident control events was received with the original data, but it was not useful in the modeling because it contained none of the predic- tors used in the event analysis. The team tested the specific measures available in the data set and attempted to supplement the available vehicle kinematic data by downloading information from the NHTSA website. Unfortunately, kinematic data were only available for a small number of crashes; near crashes and critical incidents were not represented, and this approach was, there- fore, abandoned. One weakness of event analysis is that it precludes the study of drivers who experience none of the three measured events (i.e., the safest drivers). In order to include these drivers, the second analysis track conducted by Penn State with the VTTI data was a series of models of the number of events per driver. Consistent with much of the modeling in the safety field, these analyses were conducted using a set of count regression formulations (e.g., Poisson, negative binomial [NB], and zero-inflated Poisson [ZIP]) that resulted in estimates of the probability of a driver with particular attributes having 0, 1, 2, . . . , n events during the year of the 100-car study. These models allowed comparisons to be made across all drivers. Analysis of UMTRI Data The UMTRI data consisted of a set of drivers who experienced a series of alerts from onboard systems about potential crashes. Because no crashes were recorded in the UMTRI data, the dependent variables used in the analyses were derived from a system designed to detect excessive speed entering a curve (i.e., the curve speed warning [CSW]) and an alert triggered when the subject vehicle deviated from the lane or road edge (i.e., the lateral drift warning [LDW]). After an initial screening of the data, the team decided to focus on the CSW alerts as they provided duration of time and thus contained more details about the driver response to the alert. Further, the curve speed event was more consistent with the road departure event covered in the VTTI analyses, and it was thought there may be some benefit from the similarity. Two approaches were taken in the analysis of the UMTRI data. The first used a series of piece- wise linear models to characterize the nature of the relationship between vehicle kinematics and CSW alert frequency and duration. The interest was in finding which kinematic variables were most correlated with the triggering of the alert. This information was used to gain insight about potential surrogates, under the assumption that the kinematic variables most associated with alert occurrence would be potentially efficacious crash surrogates to consider in subsequent research. A positive association between a kinematic variable and an alert being triggered could be an indication of a kinematic variable that might also be associated with (or potentially caus- ing) road departure crashes. While the team acknowledges the nature of this conceptual leap, it was believed that the exploratory nature of the S01 projects would support this type of analysis. Time–series models of the kinematic data were also attempted, but as they did not yield particu- larly meaningful results, they are not discussed in this report. The second approach taken with the UMTRI data was to use a cohort-based formulation to estimate the probability of a particular number of alerts being triggered for an individual driver (e.g., characterized by gender, years of driving experience, and mileage driven in particular con- texts). This exposure-based analysis is based on actual miles driven under specific environmen- tal and roadway conditions as measured by the CSW–LDW system. Because of the structure of the UMTRI data, the team was able to analyze alert frequency at a very detailed level of exposure.

3One of the most important outcomes of the UMTRI modeling effort is the successful estima- tion of cohort models using homogeneous trip segments. This formulation takes advantage of the unique trip-by-trip data obtained in the naturalistic study, along with geographic informa- tion system (GIS)–related factors coded by UMTRI (such as road type and environmental con- ditions), to derive a measure of alert frequency for each trip segment. The issue of interest is the ability to truly capitalize not only on the naturalistic driver behavior data, but also on detailed GIS roadway data. Since there is a plan to collect detailed roadway data as part of SHRP 2 Safety Project S04, Acquisition of Roadway Information, the team believes this formulation merits consideration for future studies. Even though the models are estimated with alerts, there is a direct parallel to the modeling of crashes or other events of interest. In addition, researchers can very flexibly define homogeneous trip segments to match their research needs. The estimated models using the cohort formulation verify the efficacy of this approach; the findings are dis- cussed in the response to Research Question 3. Research Hypotheses, Findings, and Implications The analysis of the data provided by VTTI and UMTRI was guided by the five research questions. This section states and discusses each of the five questions in sequence, specifically including the hypothesis or issue explored and a summary of what was discovered. The implications of the various findings are discussed in detail in Chapter 4. Research Question 1 What is the relationship between events (e.g., crashes, near crashes, incidents) and pre-event maneu- vers? What are the contributing driver factors, environmental factors, and other factors? The VTTI data set was primarily used to answer this question. The general structure of the event-based models was to use predictor variables representing driver, context (i.e., roadway and environment), and event attributes. Models were estimated with context-only, driver-only, and event-only variables (and combinations of only two of these components). Resulting parameter estimates changed substantially depending on how many of the three components were repre- sented in the model; importantly, the exclusion of any of the components led to major changes in estimated parameters (see Chapter 3). The exclusion of any of the set of variables (i.e., driver, context, or event) is likely to result in biased parameter estimates, obscuring the effect of any one variable on event occurrence. To avoid this bias, future analyses of SHRP 2 event-based data (such as in proposed research for the S08 project) should include variables representing driver, context, and event attributes. In addition, thorough tests should be conducted to explore changes in parameter values and significance. The Penn State team is concerned that parameter estimates may exhibit the same characteristics, even in data sets with large sample sizes. The strongest variables (i.e., those showing the greatest association with crashes or near crashes) were the driver distraction variables. These variables included distractions such as those attributed to a portable electronic device, internal distractions (such as a pet), or vehicle-related distractions (such as adjusting the climate or audio controls). Although the team used distrac- tion as a predictor variable, some distractions may be endogenous (i.e., the conditions that led to the event also led to the distraction) and may not be suitable as event predictors. A range of statistical methods to address endogeneity should be considered in these circumstances. In addi- tion, there may be a need to explore measurement periods beyond the 5-seconds-before-event criteria used in the VTTI data base. Special care should be exercised and perhaps specific models formulated to explore the nature of the endogeneity between distractions and other event-related measures. The team’s model tests indicate that distractions as predictor variables may not be valid.

4The efficacy of using categorical-outcome models (such as logit or binary hierarchical models) to compare crash and noncrash events was explored within the limits of the VTTI data by com- paring crash and near-crash events (combined) with critical incidents. The Penn State team estimated a series of models that yielded generally consistent results concerning the effects of particular parameters when using a complete model specification as described above. Given a set of event-based data, it is feasible to apply well-established categorical data analysis techniques to estimate factors that differentiate between the categorical outcomes. This method implies that such a differentiation appears feasible for crashes (or other adverse events) and a sample of comparable, similarly described nonevents. Such a comparison was anticipated, but it was not achieved because the data for nonevents in the VTTI file did not contain predictor variables consistent with the events. Gender was important in both driver- and event-based models. Many gender-related factors were revealed as main effects, but they were particularly apparent as interaction terms, especially in driver-based models. Analyses that are directly or indirectly influenced by gender should include tests of a range of main effects and interaction terms. Variables with significant promise in future modeling include level of education and years of driving experience. Several associations between number of previous crashes and violations varied with gender; these associations were not con- sistent, but they may warrant attention from researchers on gender issues. A limited number of vehicle factors rose to significance; additional research is needed con- cerning the analysis of vehicle factors, particularly in conjunction with the gender of the driver. Research Question 2 What hierarchical structure (statistically speaking), if any, exists in the manner in which these relation ships need to be explored? Two hierarchical models are reported with the VTTI data: one was applied to event modeling and the second to driver-based models. A third hierarchical model was estimated with the UMTRI data using a cohort approach. Figure ES.1 shows one hierarchy successfully applied to the analysis of event data. The sketch is intended to convey that individual drivers may have any number of events; they must have at least one, but they may have more. If one were to model this with a count regression approach, each event would enter the model as if it were independent and from a different driver. Using a hierarchical approach, driver attributes enter at the driver level, once for each driver. Event characteristics are entered as predictors for each event in which they occur. This hierarchi- cal approach provided a conceptually justifiable approach to the modeling of complex events and was applied to both VTTI and UMTRI data. A driver-based approach presents one way to analyze drivers at a separate level from the events of interest, providing a much better depiction of the physical process being investigated. A second hierarchical model (Figure ES.2) was used in the driver-based analysis of the VTTI data. In this structure, males and females are accounted for separately, and the model includes separate parameter estimates for each gender category. Figure ES.1. Hierarchy analysis of event data.

5This driver-based hierarchical model presents another example of how hierarchical approaches can be applied to naturalistic data. The benefits of obtaining gender-specific estimates of factors contributing to the risk of events are clear. Research Question 3 What kind of elucidative evidence emerges from the analysis of roadway departure crashes in terms of Questions 1 and 2? Is the illustrative hierarchy of relationships generalizable to other nonintersec- tion crash types such as leading vehicle crashes? Elucidative evidence refers to evidence of the likely effect of individual predictor variables in modeling event occurrence (including crashes). The notion of elucidative evidence includes surrogate measures and their testing. Surrogates are a special type of variable that have been discussed as a general replacement for crash data; the description and interpretation of Penn State surrogate analyses are contained in the responses to this general question. Exposure requires a predictor variable reflecting time or distance of travel; exposure-based analyses of both data sets are also described in this report. Responses to this question thus provide a sum- mary of the extent to which the modeling results provide guidance on variables to be given priority in future analysis studies. Some evidence suggests that several types of predictor vari- ables, such as precipitating event information in VTTI models, have particularly important roles in the models. One useful definition was articulated by Hauer in his more focused discussion of the traffic conflicts technique as a surrogate measure (Hauer and Gårder 1986): “one should be able to make inferences about the safety of an entity on the basis of a short duration ‘conflict count’ instead of having to wait a long time for a large number of accidents to materialize.” Shankar has argued as part of this research that surrogates have a time dimension (e.g., a measure such as time to collision has a clear time dimension; time to road departure is another) (Shankar et al. 2008). In addition, Shankar argues that a surrogate should be responsive to the same interven- tions as a crash. An example is a curve warning system alerting the driver to unsafe conditions ahead: for a surrogate like near run-off-road crash to be valid, it must be mediated by a curve warning alert in the same way as a crash. More generally, surrogates can be considered as mea- sures that can be substituted for crashes in a safety analysis: in the data for this project, they are typically vehicle kinematic– and event-related measures that offer some description of vehicle movement and/or position relative to the roadway. Potential surrogates encountered in the VTTI data include the precipitating events of subject over lane or road edge and lost control. In most of the categorical event-based models these two All Drivers FemalesMales Driver m Driver m+1Driver 1 Event 1 Event 1Event 1 Event 2 Event 2 Event 3 Driver n Figure ES.2. Driver-based hierarchical model.

6variables were strong indicators of crash or near-crash events; in hierarchical models subject over lane or road edge was the second strongest predictor associated with a crash or near-crash event. While this measure has a strong association with crash events, this measure does not have a time dimension, so it does not directly meet Shankar’s desirable criteria (Shankar 2008). Fur- ther, Hauer’s rule could not be applied because the team did not have access to the comparable set of subject behavior for noncrashes. It may not be broadly applicable outside of SHRP 2’s instrumented vehicles; nevertheless, it is clear that the measure has some potential as a surrogate. The categorical models explored in this study appear to provide a useful paradigm for explor- ing surrogates when event-based data are available. While not directly tested with VTTI data, the Penn State team believes that kinematic measures or combinations of kinematic and roadway position measures are possible measures for future testing. For example, the subject over lane or road edge variable contained position-only information and was strongly associated with crash- related events; inclusion of longitudinal or lateral velocity and lateral position information would enhance its predictive ability. A limitation of the categorical models deserves mention. Initial event-based models, both bivariate logistic and hierarchical, used improper speed as an event-based predictor. Successful model fit was obtained, but improvement was sought. Driver impairment 1 (drowsy, sleepy, fatigued) was substituted as a predictor and much better fit occurred overall, including reduced standard errors for several variables. While we were pleased by the improved fit, we were con- cerned about the apparent model instability. This may be due to the small sample size, but it may also reflect endogeneity among the predictors. As a recommendation to future SHRP 2 analysis contractors, the team suggests that care be exercised in surrogate analyses; additional empirical testing in several other sites and with other drivers should reveal more about this issue. A method for validating events containing possible surrogates for crashes is proposed and discussed in Chapter 3. The statistical predictions from the event-based model were compared with text descriptions of the event etiology derived from video and kinematic data; the com- parison showed that events originally coded by VTTI as critical incidents were statistically esti- mated to be crashes. It was posited that these events could be used to supplement crash data observed directly. The manipulation of the event-based models is proposed as a means of pro- viding useful information about whether a particular critical incident or near-crash event really was similar, statistically, to a crash event in a similar context. Such a comparison is dependent on the model being correct. An additional validation technique is discussed using the cohort formulation with hierarchical models, leading to the development of safety performance func- tions for crashes and the surrogate measure. The UMTRI analyses tested several kinematic measures, particularly longitudinal velocity entering curves, as a potential surrogate of event risk; in this case a CSW alert, instead of a crash event, was used. Initial tests of piecewise linear models applied to the data as a whole showed that the measure has some merit, but the models were weakened statistically by the presence of serial correlation in the observations (data were collected at 10 Hz). The team next explored tracking individual drivers through the same location multiple times to see if there were repeated behav- iors or learning and to explore individual variability. The models showed different results than the aggregate. While the results were not stunning, they showed potential and are recommended over aggregate approaches. The ability to explore context through the use of the detailed roadway data available through Google maps (i.e., by tying kinematic measures to specific road segments) should greatly enhance the findings. Tracking individual drivers repeatedly over the same route has potential for addi- tional insight. Specifically, a range of kinematic variables can be measured at specific points of documented high crash frequency; these can be compared with a set of individual drivers’ kine- matic signatures through the same roads. Kinematic measures at crash locations can be compared with similar measures at low-frequency crash locations and tested for their predictive capability. Cohort-based modeling also shows promise in quantifying context effects (this method is addressed in Research Question 5). The driver-based models using VTTI data used self-reported

7annual mileage as exposure. These models showed that exposure is essential to the study of the expected number of events per year for drivers. There was a strong association of exposure with the expected number of events, and the inclusion of this variable greatly improved model fit. It is clear that travel by individual drivers should be identified to the extent possible through the face camera or other technologies. The team developed a model that clearly identified drivers who were outliers with respect to the number of events they experienced. Drivers with excep- tionally high, as well as low, numbers of events can be identified using this technique. Virtually all of the event-based models showed substantial differences in the effect of distrac- tions on event occurrence. Most generally, internal distractions (e.g., reading, moving an object in the vehicle, or dealing with a pet or insect) were most strongly associated with crash or near- crash event occurrence. Passenger-related distractions and observations of the driver talking, singing, or daydreaming also had consistent positive correlations. Interestingly, the use of a wire- less device was poorly correlated to event occurrence. These findings, taken as a whole, reveal that distractions merit careful measurement in future SHRP 2 analysis efforts. Event data would be even more useful if matched with nonevent data collected from all drivers that included com- parable distraction measures. Research Question 4 In terms of elucidative evidence, what types of behavioral correlates emerge? For example, are atti- tudinal measurements indicative of revealed behavior in terms of headway maintenance and speed reductions? The principal measure of behavioral correlation was the Dula Dangerous Driving Index (DDDI) (Dula and Ballard 2003) obtained by VTTI during the original 100-car data collection effort. The DDDI consists of 28 statements to which the driver is asked to respond on a 5-point Likert scale. Example test statements include “I verbally insult drivers who annoy me”; “Pas- sengers in my car/truck tell me to calm down”; and “I will weave in and out of slower traffic.” Responses to the questions are divided into three categories: aggressive driving (AD), negative emotional (NE) driving, and risky driving (RD). Each category is intended to capture a different aspect or component of dangerous driving. The DDDI was generally associated with an increase in crashes and near crashes in the event- based models and was also positively associated with number of events in the driver-based mod- els. The results were not always easy to interpret or consistent with intuition. In driver-based models, for example, AD was associated with an increase in the number of events, but for females only. In the event-based models, this same component was associated with a reduction in crash or near-crash events (which could be interpreted as an increase in the likelihood of critical inci- dents). So, while the associations in the data were generally consistent and statistically significant (within the limits of the data), there is a concern that the findings were not as interpretable as would be desired. The testing conducted with the DDDI confirms the importance of including some measure of driver risk propensity in the remaining SHRP 2 data analyses. In addition to the DDDI, the Life Stress Index was administered to participating primary drivers in the 100-car study. This tool attempts to measure the amount of stress present in the subject’s life by using factors such as stress at work, difficulty with personal relationships, and challenges in the family environment. The Life Stress Index was positively associated with crash and near crashes in some event-based models, but it was not a predictor in the driver-based models. Although a metric for life stress provides some interesting data, it is not as important as driver-based risk-taking measurements. The proposed testing for the S07 projects, In-Vehicle Driving Behavior Field Study, includes a number of perceptual and cognitive tests; psychological tests include metrics for risk taking, risk perception, driver style and behavior, and thrill and adventure seeking. These data should provide more than ample measures of driver predisposi- tion for events.

8Research Question 5 If elucidative evidence does in fact emerge in terms of attitudinal correlates and how their interac- tions vary by context, is it plausible to parse out the marginal effects of various context variables on crash risk by suitable research design? This question bears directly on the importance of context in the analysis of naturalistic driving data. Event modeling revealed that failure to include context-related variables will yield a model with substantially biased parameter estimates. There is no way the influence of factors such as distractions and predisposition variables can be properly assessed without the inclusion of context. Several aspects of context were revealed to be associated with crash and near-crash outcomes. Roadway-related factors were important descriptors of context in the series of event-based mod- els. The presence of curves was a significant factor in differentiating critical incidents from crashes and near crashes. Horizontal curves, in general, indicated a modest increase in risk. Horizontal curve presence does not show the magnitude of influence of driver behavior variables such as distractions, but it is clearly important in defining context. Time of day, specifically dawn or dusk, was a substantial factor increasing risk and contributed importantly to the definition of context in which crash or near-crash events occurred. This vari- able was consistently significant and positive in all event-based models and had odds ratios (ORs) that exceeded some driver distraction and precipitating event factors. These findings are consistent with sleep- and fatigue-related studies of crash risk for both private drivers and the motor carrier industry. Future research projects conducted as part of SHRP 2 Safety Project S08, Analysis of the SHRP 2 Naturalistic Driving Study Data, need to seriously consider the identifi- cation of dawn and dusk driving as an important element of context. Comparing crashes, near crashes, and critical incidents with a sample of nonevents with comparable attributes would serve to validate these findings. Run-off-road crashes were consistently and negatively associated with increased traffic levels; this seems like a plausible association, as drivers are more likely to have crashes, near crashes, and critical incidents under more congested traffic conditions. This association was not as strong as the associations with the other variables. In cohort-based models formulated with the UMTRI data, context was generally more strongly associated with event outcome (i.e., CSW alerts) than driver-based variables. This general finding supports the emphasis on context that has stimulated much discussion during recent research symposia. Interestingly, the hierarchical model described in Chapter 3 identifies variability between drivers as a major factor in explaining CSW alert frequency. Taken together, these find- ings support the concept that context and driver attributes are complementary and closely linked. The cohort-based approach enables the researcher to use naturalistic driving data to examine both driver and context factors in a consistent exposure framework. Such research is only possible with the detailed data available from a naturalistic driving database, such as the UMTRI RDCW data set, which provides data on individual drivers monitored through a series of contexts. Cohort analysis represents a breakthrough in analysis paradigms for naturalistic data. The driver is tracked through a roadway network defined as homogeneous based on the needs of the analysis team. Once segments are defined, events (using appropriate screening criteria) can be allocated to the segments. The analyst can make the segment designation as fine or coarse as roadway and roadside data allow. This framework provides the measurement of the driver’s behavior throughout the driver’s travel, as well as in the seconds immediately preceding or fol- lowing a crash. A range of statistical methods was used to provide examples of how the cohort-based data structure can be used. These are intended to assist future SHRP 2 safety studies by providing guidance about data manipulation and variable formulation.