Development of a Comprehensive Approach for Serious Traffic Crash Injury Measurement and Reporting Systems (2021)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

22 5 Near-Term Solutions to Measuring Serious Injury 5.1 Overview of Near-Term Solutions The time frame for linkage in most states is too slow for the timing of implementation of MAP-21. Indeed, at the time of this writing, the FHWA has released a Notice of Proposed Rulemaking (NPRM) that proposes “A” injury from KABCO as the definition of serious injury for use through 2020. However, the NPRM also recommends putting linkage systems in place by 2020. While linkage efforts are moving forward in states—including the quarter of states that reported a 2-year time frame—meeting the goal of widespread linkage by 2020 will require significant attention. In Section 3.4, we recommended using MAIS 3+ as the definition of serious injury. However, MAIS 3+ requires medical outcome data from either a state hospital discharge or trauma registry database. In addition, if medical outcome is to be tied to any crash or roadway characteristics, the crash/roadway and medical data must be linked. About half of states surveyed have this linkage process in place or are planning to link. Indeed, the results of the state survey show that while data linkage efforts are important or mission critical in 78% of states surveyed, linkage is still a work-in-progress or in the future for most states. The challenge is that a definition of serious injury must be chosen and implemented in a short (one year) time frame, yet the appropriate linked datasets will not be in place in time. In contrast, the KABCO scale is captured in the majority of state crash databases, which themselves have complete or high levels of coverage in almost all states. The temptation, then, is to choose A-injury as the definition of serious injury for MAP-21 purposes to reduce required burden on states and ease compliance with the new reporting requirements. In fact, the NPRM proposes to use “A” injury from the MMUCC 4th Edition (USDOT, 2012) for the next six years (through 2020). However, choosing A-injury as the definition of serious injury for state reporting is likely to introduce bias in results and may reduce the motivation to implement data linkage. Flannagan & Rupp (2013), Farmer (2003) and Tarko et al. (2010) found that A-injury is biased both with respect to the conditions being considered and the usage of the scale across jurisdictions. For example, Farmer (2003) compared the percentage of A-injuries that were actually MAIS<3 and found differences as a function of geographical region, time of day, manner of collision, driver gender and driver age. Geographic region differences indicate a variability in use of the scale in general, while the other effects indicate differences in the ability or tendency of officers to identify serious injuries based on conditions of the crash or the occupant. Tarko et al. (2010) found similar differences based on vehicle type (car vs. motorcycle) and restraint use. Flannagan & Rupp (2013) detail differences between “A” injury and MAIS 3+ definitions. In addition to substantial overestimation of incidence of serious injuries, “A” injuries tend to over-represent common roadway, crash, vehicle, and occupant categories, such as belted occupants and rear-end crashes. This could lead to suboptimal allocation of countermeasures to prevent serious injuries. A solution to the short-term need to measure serious injuries and the long-term need to promote linkage between crash and medical outcome data to improve that measurement might be to select MAIS 3+ as the target definition of serious injury and then consider ways in which existing data can be used to estimate the number of serious injuries (based on the MAIS 3+ definition). In this way, the target definition of serious injury for reporting purposes will still be MAIS 3+, but the measurement process can make use of data available now, possibly including A-injury rating from crash reports.

23 The next three sections explore the possibility of using available data in the near term to estimate serious injury as defined by MAIS 3+. We focus this section on solutions whereby states can estimate serious injuries using a medical-outcome-based definition, rather than try to “fix” a non-diagnosis-based definition. We recommend two potential near-term approaches to estimating serious injuries. First, existing state-level trauma registry databases can be used to count or estimate the total number of people seriously injured in crashes in a state. This approach does not tie injury outcome to crash characteristics, but it can be used to calibrate other counts to an appropriate total. The other approach is to use sampling of a subset of medical records from crash-involved occupants in a state. Flannagan & Rupp (2013) also explored development and application of a regression equation that corrects for biases in the distribution of A-injuries relative to MAIS 3+. We do not recommend this as a way to estimate serious injuries at the state level because it continues to rely on police-reported information. However, this approach might improve some models that are forced to rely on “A” injury from older databases. 5.2 Using State Trauma Databases Many states maintain statewide trauma databases that capture MVC-related injuries as well as trauma from other mechanisms. Crash-related trauma can be isolated from other mechanisms, and trauma databases typically include injuries defined by AIS codes directly coded by trained coders using the medical records. The databases often include alcohol involvement, basic vehicle/occupant type (car, motorcycle, pedicyclist) and restraint use, along with patient age and gender. The available variables in these datasets allow state-level counting of MAIS 3+ serious injuries in crashes as well as a breakdown by alcohol, and sometimes vehicle type and restraint use. However, a challenge to using state trauma databases is that they vary in the extent of coverage within the state. In some cases, only Level 1 and 2 trauma centers are included. In others, not all hospitals that qualify for data capture have systems in place for passing data to the state registry. The not-yet-participating hospitals may or may not be a biased sample (e.g., smaller hospitals in rural areas). Before using state trauma databases for counting crash-related injury totals, statewide coverage and any bias that results from incomplete coverage needs to be assessed. In general, it should be possible to correct for low levels of incomplete coverage without difficulty, but each state must investigate this issue to effectively use state trauma databases to make states comparable. To the extent that crash descriptors may be addressed within the state trauma database (e.g., alcohol involved), those counts of serious injuries within category will be accurate. However, for data elements only available in crash databases (e.g., road type), state trauma databases can only help make totals across states comparable. Thus, “A” injury would have to remain the definition of injury for planning with respect to specific locations, roadway characteristics, behaviors, and vehicle characteristics. 5.3 Sampling Solution Sampling of some form of medical outcome information for persons in crashes offers a near-term solution that both avoids the bias and calibration issues associated with KABCO and provides an opportunity to correct for them. More importantly, it represents a potentially cost- effective (though not cost-free) approach to improving measurement of serious injury and tying serious injury to crash, vehicle, behavior, occupant, and roadway characteristics in the near term.

24 There are multiple approaches to sampling in statistics, each of which has advantages and disadvantages. This section contains a brief background on sampling followed by a recommended approach to sampling medical records associated with crash data. The discussion below focuses on selecting cases from a state crash database for follow-up to obtain medical outcome data from hospital treatment associated with that crash. The approach described would sample only those occupants who are listed on the police report as having been transported by ambulance. Sampling only transported occupants will cover almost all seriously injured (AIS 3+) cases, but may result in missing a larger number of less seriously injured occupants (for more comprehensive analyses of the cost of crashes). Having information about the ambulance service and possibly the destination hospital will make the search for patient records simpler. Sample vs. Census A census is a dataset that contains all of the cases in a given population. The Fatality Analysis Reporting System (FARS) is a census of all crashes on public roads in which someone died within 30 days as a result of crash-related injuries (NHTSA, 2013). State crash databases are intended to be censuses of police-reported crashes in a given state. A sample is a selected subset of a population on which data are collected. “Sampling” means obtaining a probability sample, defined by a) all elements in the population having a non- zero chance of being selected, and b) the selection mechanism being randomized. National Automotive Sampling System datasets (General Estimates System (GES) and Crashworthiness Data System (CDS)) are examples of probability samples of certain crashes. Cochran (1977) describes the advantages of sampling as being: reduced cost, greater speed, greater scope, and greater accuracy. In general, the arguments in favor of sampling over collecting a census revolve around limited resources for gathering information. States that are able to implement direct linkage between their state crash dataset and state trauma or hospital registry will have a census of police-reported crashes that includes serious injury as a data element. However, for the majority of states without linkage in place, a sampling approach can allow estimation of serious injury incidence under a variety of conditions of interest. Simple Random Sampling Simple random sampling represents the most basic approach and is often used as a reference to compare to other sampling approaches. In this context, drawing a simple random sample would involve selecting at random n occupants in crashes who were transported by ambulance. One advantage of simple random sampling is that analysis is fairly simple and estimates of serious injury incidence can be easily generated. The primary disadvantage is that the design can be inefficient (defined as the sample size, n, required to obtain estimates with a given precision [confidence interval size]), as well as in terms of cost (due to the large number of hospitals or trauma centers that must be contacted). Stratified Sample Design A stratified sample design is one in which a set of mutually exclusive and exhaustive categories are identified and cases are sampled at random from these strata with some known, but possibly unequal, probability. In this case, strata would be based on elements of crashes or occupants such as KABCO injury severity, alcohol involved/not involved, and/or restrained/unrestrained. Kish (1965) suggests that, for outcomes whose variance or cost of data

25 collection differs dramatically, a stratified sample design can improve efficiencies substantially over a simple random sample. The best near-term approach to correcting both bias and over-counting is sampling of medical records from crash-involved occupants in a state. Although there is some cost involved, sampling has a number of advantages over other solutions. First, sampling addresses state and local uniqueness in the way KABCO is used by allowing direct assessment of the A-to-MAIS 3+ relationship in the state rather than assuming that models developed from national data can be applied locally. Second, sampling helps build systems, relationships and capabilities that can be leveraged for large-scale direct linkage in the future. The sampling solution does not require that state-level data systems be in place already. Third, sampling allows states that are linking and states that are not to measure serious injury using the same definition. This means that the transition to linked data in states can proceed at different paces without introducing non- comparability across states. Fourth, sampling allows states who are developing linkage systems to evaluate those systems for bias. Finally, sampling is scalable. A larger sample gives greater confidence in the estimates of serious injury, but even smaller samples can be useful for better estimating serious injury incidence. Details of the sampling approach have been published in Transportation Research Record (Flannagan et al., 2014). 5.4 Regression Solution This section describes a method that uses regression to adjust for biases in A-injury that were identified in the section on the relationship between KABCO and MAIS. As discussed earlier, we do not recommend this approach for widespread “fixing” of the use of A-injuries to measure serious injury. However, the use of regression-based adjustment might improve analysis of older datasets when better approaches (e.g., sampling) are not available. The basic approach to the regression solution is to develop a regression equation that uses KABCO and other variables as predictors of the probability that an occupant is seriously injured (MAIS 3+). This estimated probability of injury can then be used instead of observed KABCO in counting serious injuries in crashes. For this analysis, we used the CDS database from 2007-2010 to develop the model and CDS from 2011 to test it. Model-based mapping between KABCO and MAIS has been developed previously. In 2002, Blincoe et al. used a KABCO-to-MAIS “translator” in an analysis of economic cost of crashes based on the National Automotive Sampling System—GES data. Separate translators were developed for belted occupants, unbelted occupants, unknown belt status occupants, and non- occupants including motorcyclists. Each translator provided the probability of each MAIS level based on the KABCO level within the designated group. Thus, each translator consisted of a 5 (KABCO) by 6 (MAIS) table of probabilities. Tarko et al. (2010) developed a regression equation based on linked crash and hospital data from the Indiana CODES program. Their model incorporated extra complexities to account for incomplete linkage in the dataset, but the key element for this discussion was an ordered logit model that used KABCO and a large number of other predictors to predict MAIS level. Although the basic approach seems promising, the Tarko et al. (2010) model used dozens of predictors. Moreover, many predictors were included that did not interact with KABCO. For example, head-on crash was a predictor in the model that, when present, increased the probability of serious injury outcome. However, the inclusion of head-on crash in this way simply accounts for the greater likelihood that someone will be injured in a head-on crash. It does not adjust for any bias in the use of KABCO for head-on crashes vs. other crash types. That is, if KABCO were a

26 perfect match to MAIS, head-on collisions would still cause more injuries and the head-on variable would still be significant in the ordered logit model. The inclusion of factors like head-on in the Tarko et al. (2010) model serve to permanently encode the relationship between head-on crashes and injury risk for future use of the model. If injury risk in head-on crashes were to decrease over time (e.g., with improvements in occupant protection or collision mitigation), this change would not be reflected in analyses of future data that use the Tarko et al. approach. Instead, a model designed to adjust for bias in the use of KABCO should only include KABCO and interactions between KABCO and other predictors (like head-on crash). This is, in effect, what Blincoe et al. (2002) did by separating their translators. Each translator is a different relationship between KABCO and MAIS, but none of the translators indicates the overall probability of injury due to belt non-use vs. belt use. Following this idea, we developed a logistic regression model based on our CDS dataset. The development dataset included data from 2007-2010 and the test dataset included data from 2011. We limited the outcome to MAIS 3+ vs. MAIS 0-2 because we were primarily interested in counting serious injuries. As with previous analyses, we removed fatalities. Our goal in this analysis is not to fully develop a final regression model for use in all states, but to explore the potential value of the regression approach to resolving problems of bias in use of KABCO. All predictors other than KABCO were implemented as interactions with KABCO. The following predictors were entered (as interactions with KABCO) in addition to KABCO itself in the original model: sex, age, alcohol involvement, restraint use, damage direction, vehicle type, number of vehicles involved, and crash configuration. All interactions with KABCO were significant except number of vehicles involved, which was removed from the final model. When KABCO is used alone as a predictor of MAIS 3+ injury, the area under the ROC curve (AUC) is 0.88. For the full model, AUC increases to 0.91. The improvement in AUC is fairly small, but significant. However, AUC as a measure of performance is insensitive to differences in total estimated serious injuries and somewhat insensitive to patterns of bias in decision rules. Thus, the real benefit of the regression approach is better seen in comparison of distributions of crash and occupant characteristics for regression approach compared to A-injury alone. To use the model, we applied the prediction equation to each occupant in the test (2011) dataset. Even occupants with “O” injury severity will have some non-zero predicted probability of having a serious injury. The count of serious injuries will then be the total predicted probability across the condition being evaluated. For example, if we want to estimate serious injuries by crash configuration, we would sum the predicted probability of serious injury for all single-vehicle crashes, then all angle crashes, and so on. Each sum is the estimated total number of serious injuries for that configuration. The regression equation will adjust for both bias and over-counting. To look at model performance, we built the model on four years of data (2007-2010) and tested it on the most recent year available (2011). Using the test data with non-missing values of predictors, we calculated the predicted total number of serious injuries for each condition of interest. We also calculated the totals for A-injury and MAIS 3+ injuries in the same set of cases. The results for crash configuration are shown in Table 10 and Figure 4. Table 10 shows that the regression approach partially calibrates the total number of serious injuries, compared to using A-injury alone. In addition, the regression approach partially corrects for bias in A-injury with respect to crash configuration. By definition, the relative distribution of configurations using the regression-based estimate falls somewhere between using exclusively A- injury and using MAIS 3+ (the ideal solution). The regression approach corrects for overestimation

27 of angle crashes and underestimation of single-vehicle crashes, but in this comparison has over- corrected for head-on risk. Table 10. Total Serious Injuries by Crash Configuration Based on Three Definitions of Serious Injury (2011 Validation Dataset) Crash Configuration Total A-Injuries Total MAIS 3+ Injuries Estimated Total MAIS 3+ from Regression Model Angle 18921 5342 7610 Head-On 2542 681 2444 Rear End 4968 1654 1900 Sideswipe/Opposite Direction 945 775 768 Sideswipe/Same Direction 889 185 373 Single 12968 9115 10577 Grand Total 41233 17751 23672 Figure 4. Comparison of relative proportions of different crash configurations for three definitions of serious injury using the 2011 test dataset. Regression approach is described in the text. Figure 4 illustrates one of the weaknesses of the regression approach. Here, the test dataset from 2011 has an unusually small percentage of head-on collisions and an unusual pattern of relationship between KABCO and MAIS for head-on collisions. The regression approach should adjust for the overall change in percentage, but is insensitive to differences in the predictive relationship between KABCO and MAIS. Thus, in the test dataset, the percentage of MAIS 3+ injuries in head-on collisions is overestimated. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% A-Injury MAIS 3+ Regression Pe rc en t o f S er io us ly In ju re d Oc cu pa nt s Serious Injury Defini on Single Sideswipe/Same Sideswipe/Opp Rear End Head-On Angle

28 Figure 5. Comparison of relative proportions of number of vehicles involved for three definitions of serious injury (2011 validation dataset). Regression approach is described in the text. Figure 5 shows the breakdown for number of vehicles involved in a crash. This figure illustrates how bias can be improved even for predictors that are not in the model. Number of vehicles was not a significant predictor, but single-vehicle crashes are included in crash configuration. As a result, bias in use of A-injury that over-counts two-vehicle collisions is influenced by crash configuration and other predictors related to number of vehicle that are included in the regression equation. As with the age analysis, the regression-based distribution falls between that of A-injury and MAIS 3+. The regression helps remove some, but not all bias in the test dataset. The graphs and tables shown illustrate the potential performance of the regression approach. If the model is carefully developed, estimates can be corrected for factors that both are and are not included in the model itself. The potential problem with the regression solution is that the data on which it is developed may or may not reflect the patterns in the data on which it is used. Specifically, a development dataset must have both MAIS and KABCO for the same occupants. Since by definition the regression solution is being offered for states that do not have crash data linked to hospital outcome, the development dataset would most likely be CDS. If the patterns of bias relative to predictors seen in CDS hold up across regions, then the regression model developed on CDS data should be appropriate for individual states to use on their data. However, the results for crash configuration shown in Figure 4 illustrate what can happen when the patterns in the state data are not consistent with those in the development dataset. Additional work is needed to better understand how much patterns vary across states, how robust the regression solution is in the context of the variation seen across states, and how the regression approach itself might be calibrated (e.g., using a Green & Blower (2010) type of approach). Our analysis indicates that the regression approach can help, but it is clearly a limited solution that should only be used in cases where better approaches (e.g. sampling) are not possible. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% A-Injury MAIS 3+ Regression Pe rc en t o f S er io us ly In ju re d Oc cu pa nt s Serious Injury Defini on Mul -Vehicle Two-Vehicle Single