Analysis of Existing Data: Prospective Views on Methodological Paradigms (2012)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

64 C h a p t e r 4 This report contains many models (although only a portion of those estimated) and many findings. To provide structure to these findings and their implications for the SHRP 2 Safety program, the chapter is organized according to the five original research questions. The chapter concludes with suggestions for future research. research Question 1 What is the relationship between events (e.g., crashes, near crashes, and incidents) and pre-event maneuvers? What are the contrib- uting driver factors, environmental factors, and other factors? This broad question encompassed many different models yielding a variety of findings. The general structure of the event- based models was to use predictor variables representing driver, context (i.e., roadway and environment), and event attributes. A set of tests was conducted to specifically explore changes in parameter estimates if variables from only one or two of these components were included in the model. Specifically, models were estimated with context-only, driver-only, and event-only variables (and combinations of only two of the components). Resulting parameter estimates changed substantially depend- ing on how many of the three components were represented in the model; importantly, the exclusion of any of the com- ponents led to major changes in estimated parameters (see Chapter 3). Implications for SHRP 2 Safety Program: Failure to test the inclusion of context-, driver-, and event-based variables runs the risk of producing a model with biased parameters. Although the data were limited and this is only one realization of an experiment, the results of this test showed substantial param- eter changes in the tests of parameter inclusion. Both the magnitudes and SEs of the parameters changed substantially. This is clear evidence that the exclusion of any of the set of variables (i.e., driver, context, and event) is very likely to result in biased parameter estimates, obscuring the effect of any one variable on event occurrence. Based on this result, future analyses of SHRP 2 event-based data (such as in proposed research for the S08 project) should be required to include variables representing driver, context, and event attributes. In addition, thorough tests should be conducted to explore changes in parameter values and significance. The Penn State team is concerned that parameter estimates may exhibit the same characteristics, even in data sets with large sample sizes. One is left to ponder the reasons for this apparent interac- tion. One possibility relates to the nature of the variables used to predict the outcome. Among the strongest variables (i.e., those showing the greatest association with crashes or near crashes) were the driver distraction variables. These variables, which were derived from the driver face camera, included distractions such as those attributed to a portable electronic device, internal distractions (such as a pet or other creature within the vehicle), or vehicle-related distractions such as adjusting the climate or audio controls. Some may view these driver actions as endogenous to the event process (i.e., the conditions that led to the event also led to the distraction). While this may not be true in all cases, it is likely true for some. While distraction was used as a predictor variable, the team now understands, after further deliberation, that some distrac- tions may be endogenous and may not be suitable as event predictors. A range of statistical methods to address endoge- neity should be considered in these circumstances. In addition, exploring measurement periods beyond the 5-s-before-event criterion used in the VTTI database may be necessary. So, while the modeling seems feasible, a caution is in order: carefully consider event model specification. Special care should be exercised and perhaps specific models formulated to explore the nature of the endogeneity between distractions and other event-related measures. Although distractions have been used in the modeling (and by others) as predictor vari- ables, the model tests indicate their use may not be valid. An additional issue of interest is to reach conclusions, how- ever tentative, concerning the efficacy of using categorical- outcome models (such as logit or binary hierarchical models) Conclusions, Implications for SHRP 2 Safety Program, and Suggested Research

65 particularly considering the gender of the driver. The small sample size limited the ability to make inferences concerning vehicle type. research Question 2 What hierarchical structure (statistically speaking), if any, exists in the manner in which these relationships need to be explored? Figure 4.1 shows one hierarchy successfully applied to the analysis of event data. The sketch is intended to convey that individual drivers may have any number of events; they must have at least one, but they may have more. If one were to model this with a count regression approach, each event would enter the model as if it were independent and from a different driver. Using a hierarchical approach, driver attributes enter at the driver level, once for each driver. Event characteristics are entered as predictors for each event in which they occur. This hierarchical approach (described in Chapter 2, with findings in Chapter 3) provides a conceptually justifiable approach to the modeling of complex events. Implications for SHRP 2 Safety Program: There are many hierarchical approaches that may be taken with a data set such as those presented in naturalistic driving studies. Much attention has been focused on the analysis of events; the driver- based approach presents one way to analyze drivers at a sepa- rate level from the events of interest, providing a much better depiction of the physical process being investigated. A second hierarchical model was used in the driver-based analysis of the VTTI data. That data structure is shown in Figure 4.2. In this structure, males and females are accounted for sep- arately, including separate parameter estimates for each gen- der category. In a single-level model, there would typically be a dummy variable representing the difference between males and females, but not an indication of the actual parameter value for each gender specifically. The hierarchical approach provides this additional information; a model of this type is developed in Chapter 2 and described as applied to VTTI data in Chapter 3. Implications for SHRP 2 Safety Program: This presents another example of how hierarchical approaches can be applied to compare crash and noncrash events. The Penn State team explored this issue within the limits of the data by comparing crash and near-crash events (combined) with critical inci- dents. The series of models estimated by the team yielded generally consistent results concerning the effects of particu- lar parameters when using a complete model specification as described above. Implications for SHRP 2 Safety Program: Given a set of data that is event-based, such as the VTTI data file, it is feasible to apply well-established categorical data analysis techniques to estimate factors that differentiate between the categorical outcomes. In this case, the team differentiated between crashes and near crashes (combined) and critical incidents. This implies that such a differentiation appears feasible for crashes (or other adverse events) and a sample of comparable, simi- larly described nonevents. Such a comparison was expected to occur in this research, but the data for nonevents in the VTTI file did not contain predictor variables consistent with the events; as a result, the VTTI data did not permit such analysis. Several strong gender-related differences in factors contrib- ute to crash or near-crash and critical incident occurrence. Gender was important in both driver- and event-based mod- els, hardly a surprise given the extensive literature on gender- related safety differences. Many gender-related factors were revealed as main effects, but they were particularly apparent as interaction terms, especially in driver-based models. Implications for SHRP 2 Safety Program: Analyses that are directly or indirectly influenced by gender should include tests of a range of main effects and interaction terms. Vari- ables with significant promise in future modeling include level of education and years of driving experience. There were associations between number of previous crashes and traffic violations that varied with gender; these associations were not consistent, but they may warrant attention from researchers on gender issues. A limited number of vehicle factors rose to significance. There was some indication that older vehicles driven by women were associated with a reduction in event frequency. The team did not interpret this as a direct safety effect. Despite many attempts to replace this variable with others of greater intuitive appeal, this result persisted. Implications for SHRP 2 Safety Program: Clearly, additional research is needed concerning the analysis of vehicle factors, Figure 4.1. Hierarchy analysis of event data.

66 surrogates include the precipitating events of subject over lane or road edge and lost control. These two variables were derived by the VTTI data coders as part of the original 100-car data set. In most event models they were strong indicators of crash or near-crash events in the categorical models; in hierarchical models subject over lane or road edge was the second strongest predictor associated with the prediction of a crash or near-crash event. While there is strong association with crash events, this measure does not have a time dimen- sion, so it does not directly meet the desirable criterion sug- gested in the Phase 1 report and by Shankar and colleagues (2008). Further, Hauer and Gårder’s rule could not be applied because the team did not have access to the comparable set of subject behavior for noncrashes. Were such data available, the hierarchical model could be formulated to test the asso- ciation between this measure and crashes. One is also left to ponder exactly how this measure could be broadly applied outside of SHRP 2’s instrumented vehicles, but it is clear that this measure has some potential as a surrogate. Implications for SHRP 2 Safety Program: The categorical models explored in this study appear to be a useful paradigm to explore surrogates when provided with event-based data. While kinematic measures or combinations of kinematic and roadway position measures were not directly tested with VTTI data, the Penn State team believes they are possible measures for future testing. The subject over lane or road edge variable contained position-only information and was very strongly associated with crash-related events; the team believes that including longitudinal or lateral velocity and lateral position information would enhance this variable’s predictive ability. A limitation of the categorical models deserves mention. Initial event-based models, both bivariate logistic and hierar- chical, used improper speed as an event-based predictor. Suc- cessful model fit was obtained, but improvement was sought. Driver Impairment 1 (drowsy, sleepy, fatigued) was substi- tuted as a predictor and a much better fit occurred overall, including reduced SEs for several variables. While the team to naturalistic data. The benefits of obtaining gender-specific estimates of factors contributing to the risk of events are clear. research Question 3 What kind of elucidative evidence emerges from the analysis of roadway departure crashes in terms of Questions 1 and 2? Is the illustrative hierarchy of relationships generalizable to other nonintersection crash types such as leading vehicle crashes? In its proposal to SHRP 2, the Penn State team described the desirability of comparing hierarchical modeling structures and models for road departure and lead vehicle collisions. This could not be done because of a lack of available lead vehicle event data in the supplied VTTI database. The Penn State team considers the notion of elucidative evidence to include surrogate measures and their testing, as well as exposure-based models developed during the study. There is also evidence that several types of predictor variables, such as precipitating event information in the VTTI models, have particularly important roles in the models. One useful definition of crash surrogates was articulated by Hauer and Gårder (1986) in their focused discussion of the traffic conflicts technique as a surrogate measure: “one should be able to make inferences about the safety of an entity on the basis of a short duration ‘conflict count’ instead of having to wait a long time for a large number of accidents to material- ize.” Additional attributes of surrogates as having a time dimen- sion and being responsive to countermeasures (as a crash would be) have been proposed in the Phase 1 report and by Shankar and colleagues (2008). More generally, surrogates can be considered as measures that can be substituted for crashes in a safety analysis: in the data for this project, they are typi- cally vehicle kinematic—and event-related measures that offer some description of vehicle movement and/or position relative to the roadway. The first example of surrogate testing is contained in the event-based analyses conducted with the VTTI data. Potential All Drivers FemalesMales Driver m Driver m+1Driver 1 Event 1 Event 1Event 1 Event 2 Event 2 Event 3 Driver n Figure 4.2. Driver-based hierarchical model.

67 same route has potential for additional insight. Again, detailed and accurate roadway data from project S04 will be essential to these tests. Specifically, a range of kinematic variables can be measured at specific points of documented high crash fre- quency; these can be compared with a set of individual drivers’ kinematic signatures through the same roads. Kinematic measures at crash locations can be compared with similar measures at low-frequency crash locations and tested for their predictive capability. Driver-based models used self-reported annual mileage (provided by drivers during 100-car study interviews) as expo- sure. While measured miles and time of travel would have been preferred, the team felt that self-reported exposure would be a reasonable start. Measured travel from the UMTRI study was used for exposure in the cohort-based analyses described in the discussion of Research Question 5. The driver- based models using VTTI data showed that exposure (in this case, miles driven per year) is essential to the study of the expected number of events per year for drivers. There was a strong association with the expected number of events, and the inclusion of the variable greatly improved model fit. Implications for SHRP 2 Safety Program: Not surprisingly, the driver-based analyses indicated that exposure was essen- tial to the modeling of the expected number of events. It is clear that travel by individual drivers should be identified to the extent possible through the face camera or other technolo- gies. Researchers interested in identifying high-risk drivers should explore the hierarchical models formulated in Chap- ter 2 and empirically tested in Chapter 3. The team developed a model that clearly identified drivers who were outliers with respect to the number of events they experienced. Drivers with exceptionally high, as well as low, numbers of events were identified. This method can be used in the identification of outlier drivers in subsequent SHRP 2 analysis activities. In consideration of the previous comments about driver distraction and endogeneity, it is of interest to briefly discuss the findings of the analysis with respect to this variable. All distractions are not alike in their effect on event occurrence: virtually all of the event-based models showed substantial differences in the effect of distractions on event occurrence. Most generally, internal distractions (e.g., reading, moving an object in the vehicle, dealing with a pet or insect) were most strongly associated with crash or near-crash event occurrence; passenger-related distractions and occurrences of the driver talking, singing, or daydreaming also had consistent positive correlation. Interestingly, the use of a wireless device was poorly correlated to event occurrence. These findings must be considered in the context that the VTTI data were collected before 2006, when cell phone usage was at a lower level than now, and devices generally had fewer features than in 2010. Implications for SHRP 2 Safety Program: These findings, taken as a whole, reveal that distractions are an important was pleased by the improved fit, there was concern about the apparent model instability. This apparent instability may be the result of the small sample size, but it may also reflect endo- geneity among the predictors. As a recommendation to future SHRP 2 analysis contractors, the team suggests that care be exercised in surrogate analyses; additional empirical testing in several other sites and with other drivers should reveal more about this issue. A method for validating events containing possible surro- gates for crashes is proposed and discussed in Chapter 3. The statistical predictions from the event-based model were com- pared to text descriptions of the event etiology derived from video and kinematic data (using the original VTTI data cod- ing); the comparison showed that events originally coded as critical incidents were statistically estimated to be crashes. It was posited that these events could be used to supplement crash data observed directly. The manipulation of the event- based models is proposed to provide useful information about whether a particular critical incident or near-crash event really was similar, statistically, to a crash event in a similar context. Such a comparison is, of course, dependent on the model being correct. Implications for SHRP 2 Safety Program: The team offers this method as a way for future SHRP 2 researchers to supple- ment their crash data. The method was developed along the way to working with event-based data. It may be used by others as needed. The UMTRI analyses tested several kinematic measures, particularly longitudinal velocity entering curves, as a potential surrogate of event risk, in this case a CSW alert (instead of a crash event). Initial tests of piecewise linear models applied to the data as a whole showed that the measure has some merit, but the models were weakened statistically by the presence of serial correlation in the observations (data were collected at 10 Hz). The team next explored tracking individual drivers through the same location multiple times to see if repeated behaviors or learning occurred and to explore individual variability. The models showed different results from the aggregate. While the results were not stunning, they showed potential and are recommended over aggregate approaches. Part of the analysis of individual drivers tied the kinematic measures to specific road segments using Google maps. The next set of analysis contracts should have detailed roadway data available for at least a few of the study sites through SHRP 2 S04 contracts. The ability to explore context for this modeling should greatly enhance the findings. The cohort- based modeling also shows promise in quantifying context effects; this method is described further in the discussion of Research Question 5. Implications for SHRP 2 Safety Program: After estimating a great many models with aggregate data, the team believes an approach that tracks individual drivers repeatedly over the

68 the subsequent modeling and hopes that the results are more consistent than those obtained in the present analyses. In addition to the DDDI, the Life Stress Index was admin- istered to participating primary drivers in the 100-car study. This tool attempts to measure the amount of stress present in one’s life as a whole by using factors such as stress at work, difficulty with personal relationships, and challenges in the family environment. The Life Stress Index was positively associated with crashes and near crashes in some event-based models, but it was not a predictor in the driver-based models. Although the Life Stress Index is another important metric to have, it is not as important as a driving-focused metric such as the DDDI. Implications for SHRP 2 Safety Program: It would be of interest to obtain a metric for life stress, but this is not as important as driver-based risk-taking measurement. The pro- posed testing for the S07 projects includes a number of per- ceptual and cognitive tests; psychological tests include metrics for risk taking, risk perception, driver style and behavior, and thrill and adventure seeking. This should provide more than ample measures of driver predisposition for events. research Question 5 If elucidative evidence does in fact emerge in terms of attitudinal correlates and how their interactions vary by context, is it plau- sible to parse out the marginal effects of various context variables on crash risk by suitable research design? The response to this research question has two parts. First, the team discussed the effects of the various components of context, specifically roadway-related factors, time of day, and traffic levels, on the probability of crash and near-crash occur- rence. These inferences are drawn from the event-based models with the 100-car data. Second, the team considered the cohort- based analyses conducted with the UMTRI data and expanded on their possible role in SHRP 2 projects, particularly S08. Context was an extremely important factor to consider; several aspects of context were revealed to be associated with crash or near-crash outcomes. Roadway-related factors were important descriptors of context in the series of event-based models. The presence of curves was a significant factor in dif- ferentiating critical incidents from crashes and near crashes. While there was some inconsistency in the magnitude of the effect, horizontal curves, in general, indicated a modest increase in risk. Horizontal curve presence does not show the magni- tude of influence of driver behavior variables such as distrac- tions, but it is clearly important in defining context. Implications for SHRP 2 Safety Program: Context, as related to roadway and roadside geometry and features, is planned to be collected through SHRP 2 Safety Project S04. The analyses indicate this is an extremely important activity. The event modeling described in Chapter 3 reveals that failure to include factor to measure in future SHRP 2 analysis efforts. If a por- tion of SHRP 2 funds are to be used to preprocess S07 project data to produce event files, then distractions would seem to be a high-priority measure to obtain for each event. The event data would be even more useful if matched with nonevent data collected from all drivers that include comparable dis- traction measures. research Question 4 In terms of elucidative evidence, what types of behavioral corre- lates emerge? For example, are attitudinal measurements indica- tive of revealed behavior in terms of headway maintenance and speed reductions? The principal measure of behavioral correlation was the DDDI collected by VTTI during the original 100-car data col- lection effort (Dula and Ballard 2003). The DDDI consists of 28 statements to which the driver is asked to respond on a 5-point Likert scale (never, rarely, sometimes, often, and always). Each of the categories of response is assigned an integer from 1 to 5. Example test statements include the fol- lowing: “I verbally insult drivers who annoy me”; “Passengers in my car/truck tell me to calm down”; and “I will weave in and out of slower traffic.” Based on previous research, the responses to the questions are divided into three categories of driving: aggressive driving (AD), negative emotional (NE) driving, and risky driving (RD). Each category is intended to capture a different aspect or component of dangerous driving. The DDDI was generally associated with an increase in crashes or near crashes in the event-based models and was also positively associated with the number of events in the driver-based models. The results were not always easy to interpret or consistent with intuition. In driver-based models, the AD component was associated with an increase in the number of events, but for females only. In the event-based models, this same component was associated with a reduc- tion in crash or near-crash events (which could be interpreted as an increase in the likelihood of critical incidents). So, while there were associations in the data, and they were generally consistent and statistically significant (within the limits of the data), there is a concern that the findings were not as inter- pretable as would be desired. The DDDI developers cite vali- dation studies conducted on a simulator (Dula and Ballard 2003), but no additional references to the use of this index were found during a Web search. Implications for SHRP 2 Safety Program: The Penn State team believes that the testing conducted with the DDDI con- firms the importance of including some measure of driver risk propensity in the remaining SHRP 2 data analyses. The current plans for the data collection projects (S07) call for the use of other metrics for estimating risk taking. The team expects that these metrics will be shown to be important in

69 factors in a consistent exposure framework that includes both. This ability is only possible with the detailed data available from a naturalistic driving database in which an individual driver is monitored through a series of contexts (such as in the UMTRI RDCW data set). Implications for SHRP 2 Safety Program: The team believes that cohort analysis represents a breakthrough in analysis paradigms for naturalistic data. The driver is tracked through a roadway network defined as homogeneous based on the needs of the analysis team. Once segments are defined, events (using appropriate screening criteria) can be allocated to the segments. The user or analyst can make the segment designation as fine or coarse as roadway and roadside data allow. This framework provides the measurement of the driver’s actions and behavior throughout the driver’s travel, not just in the seconds immedi- ately preceding or following a crash. Nevertheless, there is likely to always be a demand to study the details of the crash process in the few seconds before and after a crash or other event. The cohort approach provides a structure for the analyst to flexibly define how the behavior of the driver can be studied. The team used a range of statistical methods to provide examples of how the cohort-based data structure can be used. These are intended to assist future SHRP 2 safety studies by providing guidance about data manipulation and variable formulation. Suggested research The analyses completed to date offer a number of lessons learned concerning methodological issues in the analysis of naturalistic driving data. Among the more important are 1. Even with the much larger data set available in the SHRP 2 S07 project, there is a need to be rigorous in the applica- tion of Poisson, NB, ZIP, and other count regression tech- niques. As documented in this report, the estimation of literally hundreds of models will be necessary to obtain consistent estimates of model parameters. Models con- taining main effects may not be sufficient. It was not until interaction-based models were tested that the count-based models started to yield consistent parameter estimates and improved goodness of fit. Model estimation following this suggestion should be considered good practice. 2. The overdispersion parameter (i.e., a) in the NB model is an important indicator of heterogeneity and needs to be thoroughly studied. Including predictors for the parame- ter yielded much improved fit with this data set. 3. Context is extremely important. The elasticity for time of day (dawn or dusk) in the event-based models was as large as the most important driver distraction variables. The presence of horizontal curves was also marginally significant. These findings reinforce the importance of the context-related variables will yield a model with substantially biased parameter estimates. Accurately assessing the influence of factors such as distractions and predisposition is impossi- ble without the inclusion of context. In the 100-car study, many of the context variables were obtained from video of the event. In the remaining SHRP 2 projects it seems to be envisioned that much of these data will be obtained from the enhanced GIS data collected as part of the S04 activity. A cost savings will certainly be realized if context data can be gath- ered in this way, but it is likely that a degree of checking will be necessary to verify roadway and roadside features obtained from the GIS with camera data from the vehicles. Time of day, specifically dawn or dusk, was a substantial factor increasing risk and again contributed importantly to the definition of context in which crash or near-crash events occur. This variable was consistently significant and positive in all event-based models and had ORs that exceeded some driver distraction and precipitating event factors. These find- ings are consistent with sleep- and fatigue-related studies of crash risk for private drivers and the motor carrier industry. Implications for SHRP 2 Safety Program: Future research projects conducted as part of SHRP 2 Safety Project S08 need to seriously consider the identification of dawn and dusk driv- ing. This is an important element of context. As before, a comparison of crashes, near crashes, and critical incidents to a sample of nonevents with comparable attributes would serve to validate these findings. Run-off-road crashes were consistently and negatively asso- ciated with increased traffic levels; this seems like a sensible association, as drivers are more likely to have other types of crashes, near crashes, and critical incidents under more con- gested traffic conditions. The association was not as strong as other variables previously discussed. Implications for SHRP 2 Safety Program: It would be advan- tageous if some measure of traffic level could be collected or available for the S08 analysis projects; measurement other than through vehicle cameras would also be beneficial. The traffic data are important but, in the team’s judgment, not as important as the measurement of roadway and roadside fea- tures and time of day (dawn or dusk). In cohort-based models formulated with the UMTRI data, context was generally more strongly associated with event out- come (i.e., CSW alerts) than driver-based variables. This general finding supports the emphasis on context that has stimulated much discussion during research symposia. Never- theless, the Penn State team wishes to caution that there is no implication that driver actions are unimportant. In fact, the team views context and driver attributes as mutually comple- mentary and closely linked. The team would like to emphasize that the cohort-based approach allows, for the first time, it is believed, the ability to use naturalistic driving data to examine driver and context

70 adaptation to technology reflected in changes in driver behavior in vehicles with and without warning systems. Adaptation has been a topic of many research papers; the evidence continues to build of its importance in any effec- tiveness analysis. Extensions of the cohort-based formula- tions offer promise in exploring linkages between driver, context, and kinematic variables. 6. Future naturalistic driving studies could define homoge- neous trip segments to facilitate inclusion of kinematic variables. As the size of the road segment shrinks, there will be an improved ability to capture potentially signifi- cant vehicle kinematics linked to geometric features. This suggestion is closely linked to the more general question of improved precision and resolution for road-segment event modeling, and it is the substantial promise of link- ing S07 and S04 databases. The Penn State team has explored the provided data sets with methods that the authors believe fit the need. The team hopes it has contributed to an improved understanding of promising methods to analyze naturalistic driving data. roadway-related data collection activity within SHRP 2 safety projects and the need to thoroughly consider how these data can be included in more precise and accurate event-prediction models. Clearly it will be important to be able to identify noncrash events with common attributes in the data to better estimate crash risk in the larger data set. 4. There is a need to continue to test models that integrate kinematic data with broader data characterizing the event, driver attributes, and context. The event-based models contained in Chapter 3 only scratch the surface of these formulations. There is a particular need to include vehicle kinematics to more closely tie vehicle location (e.g., within the lane) and movement (e.g., longitudinal speed) within event-based model frameworks. Hierarchical models offer particular advantages given their flexibility and relaxation of assumptions concerning variable probability distribu- tions. The cohort formulations discussed in Chapter 3 seem to offer particular promise with respect to kinematic variable inclusion. 5. The technology-intense group of vehicles in the SHRP 2 field study requires the careful consideration of driver