Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
78 C H A P T E R 5 Study Design Introduction This chapter presents a study design to answer the primary research question: How does the existing Highway Safety Manual (1st Edition) Predictive Method (i.e., combination of SPFs and CMFs) perform for sites with similar geometry, but different access management features (e.g., access density and spacing, corner clearance, and turning restrictions) not captured in the crash prediction model? The focus is on chapter 12: Urban and Suburban Arterials. Table 55 presents a list of the access management features considered in the study design. While this research primarily focused on the crash prediction models from the Highway Safety Manual (1st Edition), it also considered the urban and suburban arterial SPFs developed for NCHRP Project 17-62, Improved Prediction Models for Crash Types and Crash Severities, that predict crashes by crash type and severity and will presumably support the Highway Safety Manual (2nd Edition). Table 55. Applicable crash prediction models from Highway Safety Manual (1st Edition) Chapter 12 (Urban/Suburban Arterials). Strategy Substrategy Site Type1 Manage location, spacing, and design of median openings and crossovers Create directional median opening 4D Segments Regulate median opening density 4D Segments Regulate median opening spacing 4D Segments Manage location and spacing of unsignalized access Establish spacing for unsignalized access 2U, 3T, 4U, 4D, 5T Segments Manage the spacing of signalized and unsignalized access on crossroads in the vicinity of freeway interchanges Establish spacing criteria for interchange ramp terminals 3ST, 3SG, 4ST, 4SG Intersections Establish functional area and corner clearance criteria Regulate driveways at signalized and unsignalized intersections 2U, 3T, 4U, 4D, 5T Segments Manage spacing of traffic signals Establish traffic signal density criteria 2U, 3T, 4U, 4D, 5T Segments Establish traffic signal spacing criteria 2U, 3T, 4U, 4D, 5T Segments Right-turn treatment Channelize right-turn lane 3ST, 3SG, 4ST, 4SG Intersections 12U = 2-lane undivided; 3T = 2-lane with TWLTL; 4U = 4-lane undivided; 4D = 4-lane divided; 5T = 4-lane with TWLTL; 3ST = 3-legged stop-controlled; 4ST = 4-legged stop-controlled; 3SG = three-legged signalized; 4SG = 4- legged signalized.
79 Methodology Overview The exercise of validating and enhancing crash prediction models involves an assessment of the effect of access management variables on crash predictions. The idea is that the cost and effort to collect and analyze data should be justified by improved crash prediction (i.e., more informed decisions). For variables that have little impact on the crash prediction, within the expected range of those variables, there is an opportunity to save the cost and effort to collect and analyze data related to those variables. The following is an overview of the general procedure to assess the crash prediction models from the Highway Safety Manual (1st Edition): 1. Evaluate if the Highway Safety Manual (1st Edition) Predictive Method shows prediction bias for access management features (i.e., does the Predictive Method over or underpredict for site characteristics related to access management?). 2. If substantial prediction bias exists, determine if there is a discernable and explainable pattern. 3. Attempt to create CMFs to remove the prediction bias for the access management features. In doing so, evaluate if the relationships make sense (i.e., is the direction and magnitude of effect reasonable?). 4. Using a validation dataset, determine if the findings are consistent among datasets. For the NCHRP 17-62 SPFs, a similar approach was taken; however, no adjustment factors are used in the NCHRP 17-62 SPFs. As such, the analysis only included sites that had the same base condition as sites used in developing the NCHRP 17-62 SPFs. Table 56 shows the base conditions for NCHRP 17-62 SPFs. Table 56. Base conditions for NCHRP Project 17-62 SPFs. Base Condition Segments Stop-Controlled Intersections Signalized Intersection No on-street parking â¢ -- -- No roadside fixed objects â¢ -- -- Median width of 15ft (for divided roadways only) â¢ -- -- No lighting â¢ â¢ -- No automated speed enforcement â¢ -- -- No left-turn lanes -- â¢ â¢ No right-turn lanes -- â¢ â¢ Lighting present -- -- â¢ No right-turn-on-red prohibition -- -- â¢ No red-light cameras -- -- â¢ Note: â¢ indicates applicable base condition; -- indicates not applicable. Source: NCHRP Project 17-62 The following sections describe the procedure to evaluate prediction bias in the existing crash prediction models, develop CMFs to adjust for bias in the crash prediction models, and validate the CMFs using a second dataset. Evaluation of Prediction Bias The evaluation of prediction bias made use of the FHWA tool âThe Calibratorâ. The tool was used to calibrate each crash prediction model to the data collected in Task 7 (Data Collection). In doing so, a number
80 of goodness-of-fit statistics are provided, indicating the general fit of the crash prediction models to the data. In addition, CURE plots and assessment tables were developed to assess levels or ranges of data for access management related variables where the crash prediction models may be consistently over- or under- predicting crashes. The CURE plots and assessment tables also provide a visual way to determine the functional relationship between a variable and crashes. Appendix C provides guidance for interpreting goodness-of-fit measures, CURE plots, assessment tables, and assessing when the bias detected is significant. The Calibrator user guide provides further instructions on how to use the tool (Lyon et al. 2018). If no significant bias was found, then it was concluded that the Part C Predictive Method performs well at predicting crashes and the addition of CMFs for those variables is not likely to improve the prediction performance. It is important to recognize that it is inappropriate to conclude that a variable has no impact on safety just because no bias is found. Instead, the safety impact may be small or otherwise is not observed in the sample of data. Development of CMFs Although the empirical Bayes before-after analysis is the current state-of-the-practice to estimate CMFs, this method was infeasible for this project because it requires data on where and when the access management strategy was implemented, a reference group of similar unchanged sites, and crash and traffic volume data for all sites. Some of these data elements are not available, especially data for multiple access management strategies that were implemented together (e.g., to estimate cumulative effects). Instead, CMF development relied on two cross-sectional regression approaches. If significant prediction bias was observed, the project team employed two alternate but related methodologies to estimate CMFs for each access management strategy. Both methodologies used the same data collected. The first approach was based on the calibration of SPFs and the second was based on estimating a new regression model. These approaches are described below. Approach 1: Calibration of Existing SPFs The use of SPFs in any jurisdiction calls for a calibration of such SPFs because SPFs are developed using data associated with a single or select group of jurisdictions and for a specific time period. If applied to another jurisdiction, or to another time period, the predictions may be biased. The purpose of calibration is to ensure that this bias is tolerably small. The bias may arise from differences in several factors, including the following: ï· Crash reporting practices (e.g., minimum reporting thresholds), ï· Socio-demographic characteristics of the driving population, ï· Weather and topography, ï· Roadway maintenance practices, and ï· Other factors affecting crash risk, which are not represented in the SPF and which may differ by location or over time. SPF calibration includes the estimation of a calibration factor. A calibration factor serves as a multiplier to adjust the original SPF estimate. For example, a calibration factor of 1.25 would increase the original SPF estimate by 25 percent, indicating that the uncalibrated SPF is under-predicting crashes and there is a need to inflate the estimates from the SPF. A calibration factor can be estimated as a single value for a dataset or obtained from a calibration function, which is an equation that provides a unique calibration factor for each site based on the site-specific variables included in the equation. To compute a calibration factor, the existing SPF is applied to estimate the predicted crashes for each site in the calibration dataset. Then, the observed and predicted crashes are summed individually over all sites. Figure 19 shows the equation to compute the calibration factor, which is the sum of observed crashes divided by the sum of predicted crashes.
81 ð¶ â ððð ððð£ðð ðððð âðð â ððððððð¡ðð ðððð âðð Figure 19. Equation for computing calibration factor. Where: ï· C = calibration factor for the SPF of a given facility type. The calibration factor is applied as a multiplier to the existing SPF for application as shown in Figure 20. ððð¹ ð¶ â ððð¹ Figure 20. Equation for applying calibration factor. Even with a calibration factor, there may still be bias in the predictions if a variable that affects safety is not included in the model. The development of CMFs is based on finding biases that exist for access management related features and accounting for them. To do so, the following steps were undertaken: 1. Apply the Highway Safety Manual (1st Edition) SPF and CMFs to the calibration dataset and estimate a calibration factor. The calibration accounts for differences between the calibration data and the data used to estimate the Highway Safety Manual (1st Edition) models in general but does not account for differences in access management features. 2. Using the calibration results, assess the overall goodness of fit measures and CURE plots for the access management related variables. Look for evidence of bias in predictions related to the access management variables. Where this is evident, the results can be used to infer the relationship between crashes and the access management strategies (i.e., the form of this relationship to include in a model). The CURE plots are used for this assessment. 3. Estimate a calibration factor for each level of the variable where there is an evident relationship between safety and an access management related variable. 4. Because part of each calibration factor accounts for differences among jurisdictions and between time periods, the CMFs need to be adjusted to a base condition. To do so, one level of the variable is defined as the base condition (e.g., lowest level of median opening density). CMFs are then updated to relate all other levels of the variable to the base condition. To illustrate, Table 57 shows a hypothetical access management related variable with three levels. The calibration factor is estimated for each level, ranging from 0.9 to 1.2. Selecting level 1 as the base condition for the access management related variable, the CMFs are estimated by dividing the factor for level 1, 0.9, by each of the other factors. The CMFs represent the effect of changing from the base condition (level 1) to another level. Table 57. Illustration of CMF estimation by calibration factors. Level of Variable Calibration Factor CMFs 1 0.9 1.0 2 1.1 1.2 3 1.2 1.3 Each estimated calibration factor is a random variable with an associated variance. The CMF is thus the ratio of two random variables. When two random variables are independent, the variance of the ratio of the two variables (i.e., the variance of the CMF) as shown in Figure 21.
82 ððð ð¥ð¦ ð¥ ð¦ ððð ð¥ ð¥ ððð ð¦ ð¦ Â Figure 21. Equation for variance of CMF. Where: ï· Var = variance of ratio of two random variables (or the CMF in this case), and ï· x and y = random variables representing two independent CMFs. The variance can be used to assess the statistical significance of differences in the estimated CMFs by level. Approach 2: Development of New SPFs Approach 2 applies generalized linear regression modeling (GLM), the same approach used to estimate the Highway Safety Manual (1st Edition) SPFs. In effect, there is an attempt to add additional variables to the existing Highway Safety Manual (1st Edition) models by modeling the number of crashes with the Highway Safety Manual (1st Edition) crash prediction models as an offset. To account for differences between the datasets (i.e., data used for this project and the data used for the Highway Safety Manual (1st Edition) SPFs) a new intercept term is estimated. Access management related variables are included in the models and the CMFs for these variables are derived from the parameter estimates of the estimated SPF. Figure 22 and Figure 23 illustrate this approach: ð¶ððð âðð ððð¡ðððððð¡ ð¦ ððð ððð¹ ð¶ðð¹ ð¶ðð¹ â¦ð¶ðð¹ ð ð´ð Â Figure 22. Equation for modeling crashes. ln ð¶ððð âðð ln ð¦ðððð ððð¹ ð¶ðð¹ ð¶ðð¹ â¦ð¶ðð¹ ðð ððð¡ðððððð¡ ln ð ð´ð Â Figure 23. Equation for modeling natural log of crashes. Where: ï· SPF = prediction from Highway Safety Manual (1st Edition) SPF, ï· CMF1, CMF2, â¦, CMFN = values of Highway Safety Manual (1st Edition) CMFs, ï· intercept = constant term to account for differences between the data used for this project and the original data used to develop the Highway Safety Manual (1st Edition) SPFs, and ï· f(AM) = function representing the relationship between crashes and the access management related variable(s). The GLM approach is preferred since the parameter estimates from which the CMFs are derived are estimated by maximum likelihood procedures. Validation of CMFs To validate the results, the project team used a second stateâs data to test the same crash prediction models and look for consistency in the findings of goodness-of-fit and any CMFs developed.
83 Chapter 5 References Lyon, C., B. Persaud, and F. Gross. 2018. The CalibratorâAn SPF Calibration and Assessment Tool Updated User Guide. Report FHWA-SA-17-016, Federal Highway Administration, Washington, DC.