Guide for Quantitative Approaches to Systemic Safety Analysis (2020)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

61 Best Practices This section highlights the experiences and best practices of several state and local highway agencies in implementing systemic safety programs to illustrate the application of a systemic safety approach in three main topic areas: systemic safety implementation approaches, applica- tion of systemic safety by local agencies, and evaluation of systemic safety programs. In part 5.1, case studies of agencies using each of the three primary implementation approaches—the FHWA Systemic Tool, the SPF approach, and the usRAP approach—are given. In part 5.2, case studies of local agency experience are illustrated for three different focus crash types: pedestrian, inter- section, and roadway departure. In part 5.3, the case studies describe methods for evaluating the effectiveness of systemic safety programs that have been used by highway agencies. These best practices and case studies are presented to show the reader a broad range of systemic safety program implementation approaches and to provide examples of how various agencies have approached systemic safety. These best practices and case studies illustrate how systemic safety is being used and the benefits and challenges that agencies have experienced along the way. The lessons learned are shared here to help guide agencies considering implementing their own systemic safety program. 5.1 Applications of the Three Primary Systemic Safety Program Implementation Approaches Section 3 presented the three primary systemic safety program implementation approaches that are being used by highway agencies: application of the FHWA Systemic Tool methodology, application of SPFs, and application of the usRAP methodology. This section documents agency best practices and experience using each of the three approaches. 5.1.1 FHWA Systemic Safety Project Selection Tool The systemic safety management approach to traffic safety, as described in the Systemic Tool, is an “approach to safety [which] involves widely implemented improvements based on high risk roadway features correlated with specific severe crash types.” The approach “supplements and complements traditional site analysis” and proactively addresses locations that exhibit crash potential due to attributes such as roadway geometry and cross-sectional design, road- side and area features, traffic control, and more (Preston et al., 2013). The Systemic Tool helps agencies identify crash contributing factors as well as identify target crash types, effective countermeasures, and priority facilities for implementation of those countermeasures. Since its publication in 2013, the systemic safety management approach described in the Systemic Tool has been utilized by several state and local agencies. This section steps through the S E C T I O N 5

62 Guide for Quantitative Approaches to Systemic Safety Analysis elements of the systemic safety planning process of the Systemic Tool, highlighting the ways some state and local highway agencies have implemented them. This section focuses on the first few steps in the systemic safety management process as illustrated in Element 1 of Figure 2. The first element of the Systemic Tool is the systemic safety planning process. With vast roadway networks, a variety of needs, and limited funding resources, agencies are constantly working to identify the best approaches to efficiently and effectively reduce severe crashes and save lives on roadways. The systemic safety planning process in FHWA’s Systemic Tool outlines steps for agencies to identify focus crash types, facility types, and crash contributing factors on their system; screen and prioritize locations for improvement; select effective countermeasures to address the crash contributing factors; and prioritize locations for the deployment of the countermeasures. 5.1.1.1 Focus Crash Types, Facility Types, and Contributing Factors In Maine, 67% of severe (fatal and serious-injury) crashes occur on two-lane, rural roads, and lane-departure crashes represent 72% of the fatal crashes. The Texas Department of Transportation (TxDOT) and Thurston County Public Works Department in Washington State also identified roadway departure crashes as a focus crash type to be addressed with a systemic safety approach because they were a large percentage of total severe crashes. The agencies collected and analyzed data to determine the contributing factors associated with this crash type. Each agency found a slightly different set of contributing factors, some of which included curve geometry, speed limit, and visual trap. As an example, Maine Department of Transportation (MaineDOT) considered the following crash contributing factors: horizontal curvature, vertical curvature, grade, posted speed, light conditions, road surface conditions, and AADT. Table 12 shows the summary of the analysis MaineDOT conducted on horizontal curvature for run-off-road crashes. It shows that this crash type is overrepresented on sharper curves. MaineDOT used five years of statewide crash history to estimate crash percentages and a random sample of highway segments to estimate exposure percentages. To develop a score for ranking the contributing factors under consideration, MaineDOT used the crash and exposure percentages to compute a “risk ratio.” Table 13 shows an example Crashes versus Exposure for Horizontal Curves Degree of Horizontal Curvature Crashes(Fatal and Serious Injury) Exposure (VMT) > 8° (<700-ft radius) 1% 0% 4°–8° (700 ft–1,400-ft radius) 14% 2% 2°–4° (1,400 ft–2,800-ft radius) 23% 12% 1°–2° (2,800 ft–5,600-ft radius) 6% 16% 0°–1° (at or near tangent) 56% 69% Table 12. Percentage of fatal and serious-injury run-off-road crashes and vehicle miles traveled (VMT) by degree of horizontal curve in Maine (Hanscom, 2018). Degree of Horizontal Curvature Crashes (Fatal and Serious Injury) Exposure (VMT) Crashes/Exposure Ratio Risk Ratio ( > 2° : < 2° ) > 2 degrees (2,800-ft radius) 38% 15% 2.5 3.5 < 2 degrees (2,800-ft radius) 62% 85% 0.7 Table 13. Example of using crashes: exposure ratios to score a contributing factor to systemic crashes in Maine (Hanscom, 2018).

Best Practices 63 of how the risk ratio was calculated for comparing horizontal curvature sharper than 2 degrees (2,800-ft radius) with horizontal curvature flatter than 2 degrees. The 3.5 risk ratio score indi- cates that the rate of severe run-off-road crashes is 3.5 times higher on a curvature sharper than 2 degrees than on a curvature flatter than 2 degrees. MaineDOT compared risk ratio scores of the contributing factors under consideration to identify which contributing factors should receive the most focus. Table 14 lists the contribut- ing factors with the highest risk ratio scores. The table shows that horizontal curvature sharper than 4 degrees (versus flatter curves) scored highest. Based on this information, MaineDOT focused its systemic treatment of run-off-road crashes on horizontal curves sharper than 4 degrees (1,400-ft radius). To identify focus facilities, agencies evaluate the facility types where crashes are most often occurring. The Minnesota Department of Transportation (MnDOT) used a crash tree diagram to determine the crash types occurring on various facility types. Figure 6 shows a portion of a crash tree diagram, which offers a simple visual way of disaggregating crash data by various facility attributes such as jurisdiction, setting (i.e., rural versus urban), intersection versus non- intersection, and more. Using crash trees, MnDOT determined that approximately 70% of severe pedestrian and bicycle crashes occur at urban intersections of county state aid highways (CSAH) and county roads (CR), especially signalized intersections. Because MnDOT’s network includes over 1,500 signalized intersections, with only a few exhibiting any recent pedestrian or bicycle crash history, a traditional hot-spot approach was not considered appropriate to mitigate potential pedestrian and bicycle crashes. As a result, this facility type (i.e., urban intersections of CSAH/CR) was selected as a key focus facility type for the systemic safety application. Thurston County Public Works Department conducted an analysis of severe curve-related roadway departure crashes to determine their distribution among various roadway functional classifications (Figure 7). Results showed the rural major collector functional class to have a greater proportion of crashes than all other facility types, highlighting it as a focus facility type. 5.1.1.2 Screen and Prioritize Candidate Locations In order to screen and prioritize candidate locations for potential countermeasure imple- mentation, a safety assessment is conducted. This assessment uses roadway and traffic data to “characterize the potential for a severe focus crash to occur at a given location or along a given segment of roadway based on certain characteristics present at these network elements” (Preston et al., 2013). The presence of crash contributing factors at these locations is scored, tabulated, and used to prioritize locations with a high number of crash contributing factors present. A sample of such an assessment conducted by MnDOT during a 2019 systemic safety analysis is shown in Table 15, which summarizes the safety assessment results for a single horizontal curve, indicating the presence of various crash contributing factors, such as a curve radius between 500 ft and 1,400 ft, a cross section width between 28 ft and 34 ft, and the absence of Contributing Factor (Run-Off-Road Crashes) Comparison Risk Ratio (Score) Horizontal curvature > 4° versus < 4° 8.5 Horizontal curvature > 2° versus < 2° 3.5 Vertical curvature Vertical curve versus tangent 2.5 Light conditions Dusk-dark-dawn versus daylight 2.5 Grade > 3% versus < 3% 2.0 Table 14. Example of a scored ranking of contributing factors to systemic crashes in Maine (Hanscom, 2018).

64 Guide for Quantitative Approaches to Systemic Safety Analysis MnCMAT – Minnesota Crash Mapping Analysis Tool ATP - Area Transportation Partnership Int - Intersection crashes Inters-Related - Intersection-Related crashes Figure 6. Minnesota crash tree example (Scurry and Preston, 2013). Figure 7. Thurston County evaluation of roadway functional class as a potential crash contributing factor (adapted from Preston et al., 2013).

Best Practices 65 lighting with a star. The number of stars associated with each location is used to rank locations based on the presence of identified crash contributing factors. Locations that have a number of stars (i.e., crash contributing factors) that exceeds a determined threshold are considered high-priority sites for improvement. A distribution of star counts for a subset of focus inter- section locations on the Minnesota county roadway system is shown in Figure 8. Additionally, a secondary ranking method is used as a tie-breaker for locations with the same number of stars, computing the comprehensive economic cost of historic crashes at each location and ranking locations from high to low. This helps differentiate among locations with similar numbers of crash contributing factors present by first prioritizing, when possible, locations that also have a history of severe crashes. The Texas Department of Transportation (TxDOT) used a unique weighting method in its 2017 systemic safety analysis for pedestrian safety improvements. The process uses an assess- ment of overrepresentation of target crashes (e.g., pedestrian crashes) associated with different location attributes, such as median type (no median, unprotected, curbed barrier), number of lanes (1 or 2, 3 or 4, 5 or more), or traffic volume (low, moderate, high), assigning weights Crash Contributing Factors Value Threshold Star Assignment Speed limit (mph) 55 45 ≤ speed limit ≤ 55 Radius (ft) 719 500 ≤ radius ≤ 1400 Traffic volume (vehicles per day) 1,650 600 ≤ traffic volume ≤ 1300 Lane width (ft) 11 11 Shoulder type Gravel None, curb, composite Total Cross Section width (ft) 30 28 ≤ total cross section width ≤ 34 Adjacent intersection None Intersection, railroad Visual trap None Present Lighting None None Outside edge risk 2 2 or 3 Total Stars Note: The 1 to 3 scale is based on a rating where 1 is low risk and 3 is high risk. Table 15. Systemic safety crash contributing factors on a Minnesota curve (MnDOT, 2019). Stars Count Percent 10 «««««««««« 0 0% 9 ««««««««« 0 0% 8 «««««««« 1 0% 7 ««««««« 1 0% 6 «««««« 1 0% 5 ««««« 11 4% 4 «««« 25 9% 3 ««« 51 18% 2 «« 83 29% 1 « 86 30% 0 26 9% Total 285 100% Figure 8. Distribution of crash contributing factor counts on Minnesota intersections (MnDOT, 2019).

66 Guide for Quantitative Approaches to Systemic Safety Analysis to each attribute, which are then summed for each location. As a result, each location has a weighted score based on its unique combination of attributes, with all locations being ranked based on these computed scores. This method creates a highly robust and data-driven process to rank locations based on crash contributing factors that have been shown to correlate with high frequencies of pedestrian crashes relative to other crash types. It also can capture the relative influence of different types of crash contributing factors that have different levels of correlation with safety performance, such as various types of medians or pavement widths, providing for more possible scoring outcomes than a binary method, which indicates the presence or absence of a defined crash contributing factor. Table 16 lists the weighting factor values used in the weight calculation formula in Equation 1. Total Weight = 10 + CT + CO – CU (1) where CT = weight based on crash total. CO = weight based on crash overrepresentation. CU = weight based on crash underrepresentation. TxDOT’s 2017 technical memorandum on their Systemic Approach to Pedestrian Safety Improvement provides the following example calculation: Taking unprotected median type in rural areas as an example, the total length of segments with such median accounts for 2.2%, whereas pedestrian crashes on the segments account for 18.9%. So, CT = 1 (total pedestrian crash, 18.9%, is between 10% and 20%) and CO = 10 (crash overrepresentation, 16.7%, is above 10%). Total weight equals to 21 (i.e., 10 + 1 + 10 – 0 (note that pedestrian crash is not under- represented)). (Geedipally, 2017) A summary of such weights, calculated based on historic pedestrian crash data and using the methodology described above, is found in Table 17. For run-off-road crashes, MaineDOT used its geographic information resources (and, more recently, Automatic Road Analyzer data) to identify curves with horizontal curvature greater than 4 degrees (less than 1,400-ft radius) for systemic treatment consideration. Prioritization of curves at the system level was based on traffic volume and horizontal curvature, which influ- ence benefit/cost potential. Table 18 shows a benefit/cost table used to help prioritize locations for centerline and edgeline (shoulder) rumble strip treatments based on traffic volume and horizontal curvature. Further screening of specific locations for rumble strip treatments was performed based on site review and alignment with schedules of paving projects (to maximize cost effectiveness). Category Weight (points) 0 1 2 3 4 5 6 7 8 9 10 Crash total ≥ 0% and < 10% ≥ 10% and < 20% ≥ 20% and < 30% ≥ 30% and < 40% ≥ 40% and < 50% ≥ 50% and < 60% ≥ 60% and < 70% ≥ 70% and < 80% ≥ 80% and < 90% ≥ 90% and < 100% 100% Crash over- representation 0% > 0% and < 2% ≥ 2% and < 3% ≥ 3% and < 4% ≥ 4% and < 5% ≥ 5% and < 6% ≥ 6% and < 7% ≥ 7% and < 8% ≥ 8% and < 9% ≥ 9% and < 10% ≥ 10% and ≤ 100% Crash under- representation 0% > 0% and < 2% ≥ 2% and < 3% ≥ 3% and < 4% ≥ 4% and < 5% ≥ 5% and < 6% ≥ 6% and < 7% ≥ 7% and < 8% ≥ 8% and < 9% ≥ 9% and < 10% ≥ 10% and ≤ 100% Table 16. Systemic safety factor weight criteria to prioritize safety improvements in Texas (Geedipally, 2017).

Best Practices 67 5.1.1.3 Countermeasure Selection In Washington State, the first systemic treatment project deployed was cable median barrier along freeway segments in 1995. This medium-cost strategy is used to reduce severe roadway departure crashes, being especially effective in addressing cross-median head-on collisions. Since then, Washington State DOT, along with MaineDOT and many other state and local agencies, have performed systemic deployment of centerline and shoulder rumble strips, target- ing roadway departure crashes on rural roads. Maine, in particular, reviewed the benefit-cost ratio of improved striping, curve signing, edgeline and centerline rumble strips to determine Contributing Factor Weight (points)Rural Urban Median type No Median 7 8 Unprotected 21 12 Curbed 10 13 Barrier 17 19 Number of lanes 1 or 2 6 5 3 or 4 23 22 5 or more 11 21 Pavement width (ft) ≤ 16 9 10 17–24 2 4 25–50 23 21 > 50 23 23 Vehicle volume Level Low 2 2 Moderate 9 5 High 27 26 Truck percentage (%) ≤ 10 ≤ 5 4 10–20 5–10 22 20–30 10–20 19 > 30 > 20 21 Table 17. Summary of calculated contributing factor weightings used by TxDOT to prioritize pedestrian improvements (Geedipally, 2017). Estimated Benefit-Cost Ratios for Rumble Strips on Rural, Two-Lane Arterials Location Centerline Outside Edgeline on Horizontal Curves Crash Type Head-On and Run-Off-Road Run-Off-Road AADT Degree of Curvature (Radius in ft)> 4° ( < 1400 ft) 2°-4° 0°-2° (>2800 ft) 500 1.4 1.6 0.4 0.2 1,000 3.0 3.3 0.8 0.3 1,500 4.7 4.9 1.3 0.5 2,000 6.6 6.6 1.7 0.6 3,000 10.7 9.9 2.5 1.0 4,000 15.5 13.2 3.4 1.3 5,000 20.8 16.5 4.2 1.6 6,000 26.7 19.8 5.1 1.9 8,000 40.2 26.4 6.7 2.6 10,000 56.0 33.0 8.4 3.2 12,000 74.1 39.6 10.1 3.8 15,000 105.7 49.5 12.6 4.8 The three shading levels represent ranges of benefit-cost ratios: dark is greater than 30:1, medium is between 30:1 and 10:1, and light is between 10:1 and 2:1. Table 18. Example of benefit-cost use for prioritizing locations for rumble strips at the system level in Maine (Hanscom, 2018).

68 Guide for Quantitative Approaches to Systemic Safety Analysis the most appropriate treatment for sites selected. Several countermeasures that agencies have implemented as part of their systemic safety management projects are listed in Table 19. 5.1.1.4 Project Prioritization To determine which candidate locations receive different treatments, a decision-making framework should be established. It should be based on the potential countermeasures as well as crash contributing factors and other characteristics of each candidate location, which impact the applicability, effectiveness, and viability of each treatment. Additionally, project cost, agency policy, and program goals should be considered during the project selection process. A sample framework in the form of a decision tree is presented in Figure 9. This flow chart shows a portion of the decision-making logic developed by MnDOT to support project selection for rural intersections in their 2019 systemic analysis. 5.1.2 Implementing Systemic Safety Using SPFs Some agencies have used SPFs to help implement their systemic safety program. A systemic safety analysis using SPFs can be designed to suit the specific needs of an agency or project as well as specific circumstances, limitations, and goals. Some of these processes are similar to those outlined in the Systemic Tool, focusing on reducing target crash types and the selection of countermeasures to address the target crash types. Another approach begins with an identified countermeasure and uses a systemic process to identify the locations where installation of the countermeasure would likely have the greatest benefit. 5.1.2.1 Using SPFs to Address Target Crash Types Nationally, roadway departure crashes account for approximately 53% of all fatal crashes; however, since about 2004, roadway departure crashes have made up over 70% of fatal crashes in the state of Kentucky. For this reason, Kentucky was identified as an FHWA Focus State for Roadway Departure, and FHWA developed a roadway departure action plan for the state in 2010. The action plan assessed statewide crash data and identified locations with concentrations of roadway departure crashes across a wide transportation network at which to deploy low- cost safety improvements. The Kentucky Transportation Cabinet (KYTC) currently allocates Context Common Countermeasures Roadway segments • Rumble strips (both shoulder and centerline) • Cable median barrier • SafetyEdge • High-friction surface treatments • Enhanced pavement markings • Curve warning signs • Chevrons/delineators • Lane/shoulder widening • Speed feedback signs • Tree/clear zone removal Intersections • Signal backplates • Crosswalk enhancements—striping, signing, rectangular rapid flashing beacons • Countdown pedestrian signals • Pedestrian refuge islands • Curb extensions • Reflective strips on sign posts • Mini-roundabouts • Lighting Table 19. Countermeasures implemented as part of systemic safety management.

Best Practices 69 Figure 9. Minnesota intersection project selection decision tree (MnDOT, 2019).

70 Guide for Quantitative Approaches to Systemic Safety Analysis $27 million (66%) of its annual HSIP funds to reduce roadway departure crashes. The roadway departure countermeasures implemented include the following: • Shoulder widening, • Cable median barrier, • Horizontal curve signing, • Guardrail and guardrail end terminal improvement on the National Highway System, and • High-friction surface treatments. KYTC worked closely with the University of Kentucky to develop SPFs, similar to those in Part B of the HSM, for systemic safety management. SPFs were developed for seven roadway types in Kentucky to identify locations most likely to see a safety benefit due to the installation of cable median barrier (see Figure 10) (Green and Fields, 2015). KYTC also adopted a road- way departure corridors approach to identify roads with a high frequency of severe roadway departure crashes. These corridors, which vary in length, target rural two-lane roadways with posted speed limits of 50 mph and greater, as a large portion of roadway departure crashes occur on these facilities. SPFs were developed for rural two-lane roadways with posted speed limits of 50 mph and greater. By utilizing SPFs, KYTC was able to analyze and prioritize the targeted facilities statewide in a consistent, statistically rigorous process based on excess expected crash frequency, focusing first on segments with the greatest potential for safety improvements. After identifying the corridors, the agency analyzed the safety needs in more detail to assess appro- priate evidence-based strategies for implementation. These improvements include widening or paving shoulders, adding centerline and shoulder rumble strips, improving roadway slope and superelevation, providing enhanced signing and delineation, and extending culverts and other roadside improvements. Kentucky identified several benefits of a systemic safety analysis approach using SPFs for reducing the frequency and severity of roadway departure crashes. These benefits included: • Implementation of proven countermeasures along the most opportunistic corridors, • Opportunity to partner with KYTC district personnel and consulting community to develop and deliver data-driven safety solutions and advance safety culture in a compressed time- frame, and • Support for the preservation and renewal of the transportation system infrastructure. Figure 10. SPFs to prioritize locations of cable median barrier installations in Kentucky (Green and Fields, 2015).

Best Practices 71 5.1.2.2 Prioritizing Installation of Select Countermeasures Using SPFs In recent years, the Illinois Department of Transportation (IDOT) has made an effort to broadly deploy rumble strips on rural two-lane roadways across the state. This countermeasure was selected as a low-cost solution that has proven highly effective in reducing roadway depar- ture crashes. Because of the breadth of the agency’s network of target facilities, it was necessary to identify a simple, systemic, method for prioritizing locations for deployment to produce the greatest improvements over time. A prioritization process was developed that utilized a network screening SPF to compute the potential for safety improvement on all state jurisdiction, rural two-lane roads with speed limits of 50 mph or greater, which was the target facility for the assessment. These results were used to rank locations based on expected frequencies of head-on and sideswipe, opposing- direction crashes for centerline rumble strips and overturned and fixed object crashes for shoulder rumble strips. Tier 1 included locations that represented the most severe crashes and the greatest potential for safety improvement through the application of countermeasures, while Tier 2 and Tier 3 included mid-priority and low-priority locations, respectively. A summary of the results of the assessment for shoulder rumble strips is captured in Table 20, and a map District Tier Assignment Miles* Percent of District Miles** Severe Focus Crashes (K+A)*** Percent of Severe Focus Crashes in District District 1 Tier 1 32 32% (35%) 27 59% Tier 2 61 60% (65%) 19 41% Tier 3 8 8% 0 0% District 2 Tier 1 145 16% (21%) 118 50% Tier 2 560 62% (79%) 118 50% Tier 3 204 22% 0 0% District 3 Tier 1 191 18% (23%) 124 47% Tier 2 650 60% (77%) 140 53% Tier 3 249 23% 0 0% District 4 Tier 1 204 17% (21%) 124 46% Tier 2 751 63% (79%) 143 54% Tier 3 238 20% 0 0% District 5 Tier 1 72 9% (12%) 54 31% Tier 2 547 68% (88%) 118 69% Tier 3 186 23% 0 0% District 6 Tier 1 173 13% (19%) 97 40% Tier 2 728 56% (81%) 147 60% Tier 3 410 31% 0 0% District 7 Tier 1 134 12% (17%) 89 41% Tier 2 650 60% (83%) 130 59% Tier 3 298 28% 0 0% District 8 Tier 1 214 23% (30%) 173 59% Tier 2 493 54% (70%) 121 41% Tier 3 207 23% 0 0% District 9 Tier 1 205 21% (25%) 145 52% Tier 2 610 61% (75%) 132 48% Tier 3 178 18% 0 0% Statewide Tier 1 1,370 16% (21%) 951 47% Tier 2 5,050 60% (79%) 1,068 53% Tier 3 1,978 24% 0 0% * Studied roads include rural two-lane state highways with a posted speed limit of 50 mph or greater. ** Percentages in parentheses represent the percent of miles if considering only analysis segments that had at least one focus crash (i.e., disregard Tier 3 analysis segments). *** Crash data from 2004–2008; only includes overturned and fixed object crashes. Table 20. Illinois statewide analysis summary: shoulder rumble strips (Kolody and Rajabi, 2018).

72 Guide for Quantitative Approaches to Systemic Safety Analysis of the locations where rumble strips were subsequently installed during the years following the assessment is shown in Figure 11. 5.1.3 Implementing Systemic Safety Using usRAP In 2013, the Utah DOT chose to implement the usRAP program, because it was presented to them as a less resource-intense alternative to Safety Analyst. Utah DOT drove a lidar-equipped van along all of its state-managed highways and collected data that they then reduced, coded, and input into the ViDA software. The data collected using the van provide approximately 80% of the data required by the software. Some additional data, such as location of bike routes, were pulled from planning data. A university was hired to reduce street-view imagery of the network and gather the remaining data and then to perform a quality control analysis and validate all of the data. The usRAP program provided Utah DOT with a star rating for all its state-managed highways as well as recommendations for safety countermeasures along each 328-ft segment of that roadway network. Figure 12 illustrates the star ratings assigned to state-maintained roads (non-interstates) around Salt Lake City and Provo, Utah, based on the usRAP methodology. Currently, federal aid routes in the counties are being added to the usRAP roadway network in Utah. One county has been completed and 12 more counties are planned for evaluation. Rumble Strips Contract Locations Illinois Districts Legend Figure 11. Illinois statewide rumble strip installation contract locations after the systemic assessment (IDOT, 2018).

Best Practices 73 5.1.3.1 Identifying and Prioritizing Safety Projects Utah has not implemented an independent systemic safety program. Instead, it uses infor- mation gathered from usRAP as one way to justify safety projects. This type of project justifica- tion allows for the possible implementation of safety treatments at locations without significant crash history. In Utah, the DOT central office aids the four regional offices in selecting and prioritizing safety projects. Generally, the regions identify projects they would like to complete and send their recommendations to the central office, where staff evaluate those projects to determine the benefit-cost ratio. The benefit-cost ratio is calculated using three methodologies: 1. Calculate expected crash reduction using HSM predictive methodology, where available. 2. Apply a CMF to the average number of crashes from the past three years. 3. Use the expected crash reduction calculated by usRAP. Utah uses its own treatment implementation costs to calculate the benefit-cost ratio (rather than entering the cost data required by usRAP and relying on the software to make the calculation). Historically, HSIP funding was divided among the regions. Any project recommended by the region and found to have a benefit-cost ratio greater than 1.0 by any of the three methods was funded. However, the DOT is now trying to ensure the most beneficial projects are funded first. Moving forward, projects will be funded based entirely on the highest benefit-cost ratio. No funds this year will be allocated directly to the regions (districts) unless their proposed project has a benefit-cost ratio within the funding cut-off limits. Figure 12. Sample of star ratings for state-maintained roads (non-interstates) in Utah.

74 Guide for Quantitative Approaches to Systemic Safety Analysis Regions generally identify potential projects based on crash history, but when regions need help identifying potential safety projects, the DOT central office provides assistance by suggest- ing projects recommended by usRAP data. The types of projects suggested often include rumble strips, shoulder widening, guardrail, and intersection turn lanes. 5.1.3.2 Other Uses of usRAP Results in Utah Because usRAP provides countermeasure recommendations at specific locations based on that location’s site characteristics associated with higher or lower crash potential and not on crash history, it can sometimes suggest projects that would not have been considered by the DOT otherwise. Utah DOT indicated that it has begun to rethink where it prioritizes adding guardrail based on usRAP’s recommendations. Within the Utah DOT, the safety office shares the star ratings that are determined by usRAP with the planning department to help them identify areas with higher crash potential in planning future projects. The DOT also shares usRAP results on local roads with the relevant metro- politan planning organizations (MPOs) to assist them in identifying safety needs and projects within the metro areas. The Utah DOT is currently working with a local university to develop a database of usRAP projects that can be mapped to help the agency track and verify the usRAP projects that have been and are being implemented. 5.1.3.3 Benefits and Challenges of Using usRAP in Utah Utah DOT’s experience implementing usRAP is that once data are gathered and in the tool, the software is easy to run, and the results are easy to understand. The ViDA software is well documented, data are easy to input and export, and results are easy to generate and share. The results of usRAP studies have led to changes in what is considered safe and unsafe on the roadway. The results are not biased to the agency’s culture and history, so the program can help avoid implementing projects with lower potential for safety improvement that may have been selected in the past based on invalid or unsupported assumptions. The Utah DOT found that initially, the districts had some resistance to implementing projects recommended by ViDA in locations where there was little history of crashes, but overall, this push back was minor. Initially gathering the data for use in the usRAP methodology, which requires coding street- view images of roadways (and potentially incorporating data from existing sources), can be resource-intensive. Utah was able to use a university to provide coding and development of a dataset to serve as the inputs for the ViDA software. Utah recommends that other agencies considering implementing usRAP either hire a university with staff trained in performing usRAP coding or pay to train staff at a local university to handle coding so that DOT staff do not have to spend time learning and performing the coding of roadway characteristics. However, it is helpful to have at least one person at the DOT understand the coding procedures so that he or she can provide quality control checks on the dataset. In addition to the effort of coding the initial dataset, Utah DOT found that providing counter- measure costs within ViDA was time consuming and difficult. Each of the more than 70 treat- ments is expected to have a high-, medium-, and low-cost estimate for both rural and urban implementations. Because costs can shift frequently, it is difficult to ensure the costs used in ViDA remain valid. Utah decided to avoid this drawback by calculating its own benefit-cost ratios using benefits calculated in ViDA with their own costing information. The DOT also mentioned the use of metric units as a barrier to using the usRAP, but did not find it especially difficult to adjust to.

Best Practices 75 The Utah DOT noted that for projects recommended by usRAP but not supported by either of the other two project justification methodologies (CMFs or HSM approaches), a minimum benefit-cost ratio of 6.0 is used. This higher requirement for justifying projects is meant to be a conservative approach to account for possible variations in cost and potentially inaccurate input data. 5.1.3.4 Evaluating Systemic Projects Justified with usRAP As required by FHWA, the Utah DOT performs an annual crash review of its safety projects, using three years of crash data from before project implementation and three years of data after project implementation. However, the justification for each project is not considered. That is, the projects chosen based on usRAP recommendations are not evaluated separately from projects chosen based on the HSM or CMF methodologies. In fact, many projects are chosen because more than one of these methodologies predicted a benefit-cost ratio greater than 1.0. The DOT noted that it can be difficult to show that a given treatment was beneficial when it did not have a substantial crash history prior to the treatment implementation. This is a general difficulty agencies have when trying to evaluate the impact of systemic safety projects. 5.1.4 Considerations for Success Agencies that have used the systemic safety management approach have made it an integral part of their statewide and agency-wide roadway safety management process. Agencies have appreciated the flexibility and adaptability of the systemic safety management approach described in the FHWA Systemic Tool with a small investment in training and data collection, and agencies that have taken the effort to get their data into Safety Analyst and usRAP ViDA appreciate the functionality and capabilities of the software tools. Some agencies have challenges implementing a systemic safety analysis approach due to data limitations, effective identification of crash contributing factors, funding limitations and public expectations, and legal implications. The systemic safety management approach is adaptable based on available data, but it can still be moderately data-intensive. When implementing the FHWA Systemic Tool methodology, agencies have utilized available data sources but in most cases still collect additional data elements for focused crash or facility types. Factors identified in the Systemic Tool are a good place to start when collecting additional data elements. Crash contributing factors are deter- mined by an overrepresentation of severe crashes, so larger datasets with multiple counties or jurisdictions may be necessary to establish a large enough sample size to determine crash contributing factors. Contributing factor determination can be somewhat subjective, requiring some expertise and familiarity with the methodology and a thorough understanding of the state of the roadway network being analyzed. When implementing systemic safety using SPFs, roadway inventory, traffic volume, and crash data are required for SPF development and/or calibration. Safety Analyst requires approximately 38 different data elements to utilize the full functionality of the software, most of which are designated as MIRE FDE. The usRAP ViDA software makes use of over 50 roadway characteristics to develop star ratings and fatal and serious-injury estimates for each individual roadway segment. One of the strengths of the systemic safety management approach is that it is adaptable based on available data. It is not necessary to have a comprehensive, integrated database of roadway inventory, traffic volume, and crash data to begin implementing a systemic safety management program. Agencies should start by understanding the purpose and use of data in the context of systemic safety analysis and the tradeoffs and opportunities that data provide, whether in the use of a ranking methodology (based on the presence or absence of a crash contributing factor)

76 Guide for Quantitative Approaches to Systemic Safety Analysis or in the use of a required data element within a software tool. Priorities can also be established to develop a more comprehensive dataset—first, for a portion of the network, then for other parts of the network. The emphasis, however, should be on developing an initial systemic safety management approach based on available data or data that can be easily collected in the short term. Priorities can then be set for incrementally collecting additional data elements and developing a more comprehensive and reliable dataset over time. As more reliable data become available, the systemic analytical or methodological approach can evolve over time and become more robust and/or be expanded to additional parts of the network. In some cases, this may mean that the full functionality or capabilities of software, such as Safety Analyst or usRAP ViDA, are not initially realized or that some assumptions are made concerning required input data. Over time, the full functionality and capabilities of the software could be utilized with an expanded and more comprehensive dataset. Finally, a number of agencies have indicated that strong leadership support and an effective communication plan were essential for establishing the systemic safety management approach and for expanding the program. In particular, it can be difficult to get executive support and funding for systemic programming at facility locations that do not exhibit a crash history. This approach is contrary to the crash-history-based safety management approach, which is reac- tive and focuses on locations with a crash history. In such cases, evaluation of systemic safety programming can play a key role, proving the effectiveness of the approach and the importance of proactively seeking out and addressing roadway safety needs. Similarly, the systemic safety management approach may be difficult to effectively communicate to the public, with projects implemented at locations without crash histories potentially being viewed as detracting from programming at locations with crash histories. In these cases, agencies have worked closely with the public to educate on the effectiveness of safety treatments to gain buy in and support to reduce severe crashes. 5.2 Local Applications of Systemic Safety Because the systemic safety management approach described in the FHWA Systemic Tool is less reliant on high-quality, comprehensive historic crash data, it provides a means for identifying and addressing highway safety needs that are not explicitly addressed with the traditional hot-spot approach. This has proven beneficial for local agencies whose inventory, traffic volume, and/or crash data are sometimes incomplete or of poor quality. This systemic safety management approach is also helpful to assess networks with low numbers of total crashes, often associated with low traffic volumes or vast roadway networks. Because a large portion of crashes tend to occur on such local roadway networks, it is essential that they not be ignored but instead are targeted through means such as the systemic safety management approach. The traditional hot-spot approach leans heavily on crash data for the identification and prioritization of network locations for safety improvement. Because of this, poor crash data limit the capabilities of some local agencies to perform robust analyses. However, the systemic safety management approach described in the FHWA Systemic Tool provides a way around this, leading to prioritization of safety improvement projects without the need for high-quality crash data. Additionally, because this basic application of the systemic safety management approach is not exceedingly complex or difficult, it is relatively easy for local agencies to employ without the need for extensive training or experience. The following sections outline a few effective applications of the systemic safety management approach on local roadway networks, focusing on local applications to address pedestrian, intersection, and roadway departure crashes.

Best Practices 77 5.2.1 Pedestrian Corridors Pedestrian crashes tend to be widespread in dense urban environments where a variety of pedestrian generators, such as office buildings, shopping centers, and schools, create high volumes of pedestrian traffic. However, due to the complex nature of pedestrian crashes and their associated contributing factors, pedestrian crashes are often difficult to predict based exclusively on crash history. The systemic safety management approach provides a means to proactively identify network locations which, regardless of crash history, are expected to exhibit a relatively high potential for future pedestrian crashes. The systemic safety management approach, when applied to pedestrian crashes, can be used to identify the types of locations and attributes that tend to correlate with an overrepresentation of pedestrian crashes. Contributing factors for intersections such as signalization, number of lanes to cross, and roadway median type are common considerations. Because local agencies’ jurisdictions often include broad networks with many intersections and roadway corridors that experience various levels of pedestrian traffic, the systemic safety management approach can help agencies narrow their analyses, pinpointing specific types of facilities for safety improve- ment. This process can also help agencies identify types of treatments that would be most effective to apply to improve pedestrian safety, addressing locations that exhibit contributing factors as opposed to chasing locations with histories of pedestrian crashes that may be unlikely to be repeated. In its 2019 County Roadway Safety Plans, MnDOT assisted county and area transportation agencies in analyzing urban intersections, working to identify locations with contributing factors found to be associated with a high likelihood of pedestrian crashes. MnDOT developed a decision tree (Figure 13) for intersections, which takes into consideration certain contributing factors and assigns one or more safety treatments. This framework guides the project identifi- cation process for all county jurisdiction roads being assessed, providing prioritization that can be used to inform safety programming. 5.2.2 Local Intersections Within a local roadway network there may be a large number of intersections with a variety of traffic volumes and safety performance levels. Because of the vast scope of such a system, it can be difficult to effectively address all intersection safety needs through a traditional hot-spot approach. In such cases, a systemic safety management approach should be considered to help identify and address crash contributing factors that may be present throughout the network, proactively improving the performance of the whole system using data-driven countermeasure selection and project prioritization. Such methods are effective for both urban and rural systems. In recent years, Texas’ five MPOs developed intersection safety implementation plans, using a systemic safety manage- ment approach to address safety needs within their metropolitan regions. This included an analysis of facility types, which highlighted specific subsets of their vast networks for focus in future projects. Data showed that urban signalized intersections on both their state and local jurisdiction roadways tended to have the highest density of crashes per intersection when compared to other intersection types (Figure 14). The analysis identified a number of crash contributing factors based on engineering judgment as well as a distribution of severe collision types most associated with their focus facilities. These were used to support the selection of countermeasure packages to be deployed at priority facilities, which were selected based on crash frequency thresholds to ensure that locations having some history of severe crashes receive priority funding (Table 21).

Figure 13. Minnesota urban intersections: pedestrian/bike-related safety project decision tree (MnDOT, 2019).

Best Practices 79 Figure 14. Distribution of Alamo Area Metropolitan Planning Organization severe (KA) intersection crashes by area type and traffic control (TxDOT, 2016). Crash Threshold KA Crashes (Fatal and Serious Injury) Intersections B Crashes (Minor Injury) Avg. Per Intersection Cost Assuming $45M funding Number Percent Number Percent 2 or more KA crashes 3,782 52.5% 1,373 28.7% 8,242 $32,775 3 or more KA crashes 2,006 27.9% 485 78.9% 4,507 $92,784 4 or more KA crashes 1,178 16.4% 209 34.0% 2,919 $215,311 3 or more KA crashes OR 2 KA crashes and 10 or more B crashes 2,162 30.0% 563 11.8% 5,518 $79,929 3 or more KA crashes OR 2 KA crashes and 8 or more B crashes 2,298 31.9% 631 13.2% 6,088 $71,315 3 or more KA crashes OR 2 KA crashes and 6 or more B crashes 2,506 34.8% 735 15.4% 6,751 $61,224 3 or more KA crashes OR 2 KA crashes and 5 or more B crashes 2,660 37.0% 812 17.0% 7,136 $55,419 3 or more KA crashes OR 2 KA crashes and 4 or more B crashes 2,846 39.5% 905 18.9% 7,508 $49,724 Table 21. Potential crash thresholds for statewide systemic treatments (TxDOT, 2016).

80 Guide for Quantitative Approaches to Systemic Safety Analysis Similarly, MnDOT utilized a systemic safety management approach to assess intersections on its local rural system as a part of its 2019 County Roadway Safety Plan. Based on historic crash data, a selection of crash contributing factors were identified to assess which intersections exhibited the greatest potential for crashes (Table 22). Because of the expansive size of MnDOT’s system, many of its rural intersections have no history of crashes, giving no basis for a reactive approach to improving safety performance at many network locations. However, this does not preclude the need for safety improvement; through the systemic safety management approach, MnDOT has taken a proactive approach, identifying opportunities to improve safety prior to the occurrence of crashes. 5.2.3 Roadway Departure Roadway departure crashes are often overrepresented on local rural roads and may make up a large portion of an agency’s total crashes, as is the case for the local agency of Thurston County in Washington State. The county identified roadway departure as a priority crash type to address using roadway safety funding. However, due to the random nature of such crashes, it is often not feasible to address roadway departure crashes through a traditional hot-spot approach, as they tend to be spread out across large sections of roadway. This is where Thurston County has found the systemic safety management approach to be very valuable, allowing them to proactively find network locations that, regardless of crash history, are expected to have a relatively high potential for future roadway departure crashes. Such locations can be found through the use of data-driven crash contributing factors, which are roadway attributes related to geometry, cross-sectional features, traffic volume, roadway environment, and more. This helps the agency avoid chasing crashes and instead, work to head them off before they occur. Thurston County Public Works Department identified nine crash contributing factors for screening and prioritizing candidate locations for improvement to reduce roadway departure crashes on horizontal curves on their rural road network (FHWA, 2013): • Roadway class of major rural collector, • Presence of an intersection, • Traffic volume of 3,000 to 7,500 AADT, • Edge clearance rating of 3, • Paved shoulders equal to or greater than 4 ft in width, • Presence of a vertical curve, • Consecutive horizontal curves, • Speed differential between posted speed and curve advisory speed of 0, 5, and 10 mph, and • Presence of a visual trap (i.e., a minor road on the tangent extended). Crash Contributing Factor Crash Contributing Factor Criteria Context zone Commercial, industrial, mixed use, or residential Total entering traffic volume Volume ≥ 2,000 vehicles per day Traffic volume cross product Greater than 1,000,000 vehicles per day Number of entering legs 4 Alignment skew Greater than 10 degrees Adjacent railroad crossing Present Adjacent curve Horizontal, vertical, or both Commercial development Present Previous stop sign Greater than 5 miles Major-road speed limit 60 miles per hour or greater Major-road lane configuration Left/through, through/right, and turn/bypass Table 22. Rural intersection crash contributing factors (MnDOT, 2019).

Best Practices 81 These crash contributing factors provide a means for identifying sections of roadway with horizontal curves throughout their system which, whether or not they have a history of severe crashes, may be expected to exhibit the greatest potential for severe crashes, especially roadway departure crashes. This empowers the agency to proactively work to reduce crashes and save lives on its rural roadway network. 5.2.4 Considerations for Success Due to HSIP requirements to improve safety on all public roads, state transportation agencies have been reaching out to local jurisdictions within their states to assist in various ways. Several states have assisted local agencies by developing local road safety plans. In most cases, limited information is available for local roads to implement a traditional crash-history- based safety management approach to identify high-crash locations off the state system. Therefore, states have been working with local agencies to implement the systemic safety management approach to utilize data-driven safety analyses to prioritize and program safety improvements. The systemic safety management approach described in the FHWA Systemic Tool is well suited for local agencies because it is less reliant on high-quality crash data. It is also relatively easy for local agencies to employ without the need for extensive training or experience. Application of the usRAP methodology, using the associated ViDA software, is also particularly well suited for use by local agencies as coding tools can be used to prepare input data for the usRAP ViDA software. However, application of the usRAP methodology using the associated ViDA software is not particularly widespread among local agencies. 5.3 Evaluation of Systemic Safety Management Programs Evaluations of systemic safety management programs have been conducted in a number of ways, depending on the type and amount of data available, the goals of the evaluation, and the agency’s available resources in terms of time and expertise. Quantitative impacts of systemic safety management programs are most commonly analyzed using a trend analysis, the simple before-after study approach (with or without traffic volume correction), the shift of proportions method, or the EB before-after study method. Agencies have implemented these evaluation methods to inform future decision making to maximize the safety program and achieve goals. The evaluation results should be tailored to meet the needs of the target audience. For example, legislators may be interested in seeing the overall trends in fatalities, while program decision makers may want to gain a more detailed understanding of the benefit-cost ratio associated with specific investments or safety countermeasures. It should be recognized, however, that the current state of practice and knowledge in terms of the evaluation approaches can only provide a limited amount of information concerning the overall effectiveness of safety treatments implemented as part of a systemic safety manage- ment approach. With the systemic safety management approach, sites across the network may be improved but may not have any crash history prior to being treated. Because the evaluation methods either directly or indirectly compare the crash history of sites before and after imple- mentation of treatments, the current evaluation methods can only provide a limited perspective on the overall safety effectiveness of treatments implemented as part of a systemic safety manage- ment approach. This becomes more of an issue as the number of evaluated sites having no crash history prior to being treated, increases.

82 Guide for Quantitative Approaches to Systemic Safety Analysis 5.3.1 Trend Analysis The most common approach to evaluating a systemic safety management program is to use trend analysis. This approach evaluates a program by computing the aggregate number of target crashes that occur on the system under consideration and tracking changes to the crash frequency before, during, and after deployment of a strategy. If the deployment is successful in improving safety performance, this will be reflected in the crash data, represented by a down- ward trend in target crash frequency over the course of the deployment period. The trend analysis approach has minimal data needs. Apart from general information about the strategies being deployed, such as target crash types and locations, it requires only annual target crash data on the target roadway system for a defined range of years before, during, and after the systemic program deployment period. Using several years of crash data may capture the broad trends in the crash data, helping to distinguish trends due to implementation of the program from trends due to external factors. This is important to avoid attributing patterns in the data to the systemic program which may be the result of broader regional, state, or national trends. Trend analysis is commonly used as a first stage of evaluation as it is easy to perform and has minimal training and data requirements. Additionally, this method can easily be supple- mented with other methods as the systemic program evolves. MnDOT, the Washington State Department of Transportation (WSDOT), and Thurston County Public Works Department (Washington) use this method as the main approach to evaluate their systemic programs. WSDOT in particular uses trend analysis to track fatal and serious-injury crashes over time by roadway jurisdiction and may disaggregate results by crash type and county in future evalua- tions. Figure 15 presents fatality trends nationally and in Minnesota. Figure 16 shows fatality trends on state and local routes in Minnesota. MnDOT’s systemic safety management program focuses on reducing fatal and serious-injury crashes on local routes, and both Figures 15 and 16 illustrate the relative success of MnDOT’s systemic safety program in reducing fatalities on local roads. 5.3.2 Simple Before-After Method A simple before-after study evaluation approach can be used to quantify the effectiveness of a safety program by comparing the total frequency of target crashes that occur prior to the deployment of a strategy to the total frequency after the deployment. Generally, before and after periods of 3 to 5 years are assessed, with the year during which deployment occurs being Figure 15. National and Minnesota fatality trends (MnDOT, 2019).

Best Practices 83 excluded from the analysis. Once aggregate crash frequencies are computed, they can either be directly compared or they can be normalized. If traffic data are available, the results can be normalized by the mean AADT values for the before and after periods. This is done by multiplying the before-period crash frequency by the ratio of the average AADT over the after period to the average AADT over the before period. This helps to account for changes to traffic volumes on the target locations, which can influence crash frequency (i.e., more traffic generally results in more crashes). The primary benefits of the simple before-after study evaluation approach are that data requirements and training needs to perform the analysis are low. To perform a crash frequency assessment, only basic strategy information and crash data for the before and after periods are necessary. If an agency elects to expand the evaluation by accounting for changes in traffic volumes, AADT values for the analysis period years are required. For example, Thurston County Public Works Department conducted a simple before-after evaluation of systemic improvements deployed at curve locations on unincorporated county roads across the county. The department aggregated fatal and serious-injury crashes on all curves during the before period, from 2006 to 2010, as well as during the after period, from 2012 to 2016, although not all investments were completed until 2015 (Figure 17). The evalu- ation showed a 35% reduction in fatal and serious-injury crashes on horizontal curves in the after period compared to the before period, representing a quantitative improvement in safety performance at the target locations. Figure 16. Minnesota state and local fatality trends (MnDOT, 2019). 0 90 80 70 60 50 40 30 20 10 Horizontal Curve Fatal and Serious Injury Crashes 2006 to 2010 2012 to 2016 Before Period After Period 35 Percent Reduction Figure 17. Horizontal curve, fatal and serious-injury crashes, before and after implementation of systemic curve treatments (Thurston County).

84 Guide for Quantitative Approaches to Systemic Safety Analysis KYTC regularly utilizes the simple before-after study evaluation approach and other evalu- ation approaches [see parts 5.3.3, Shift of Proportions Method and 5.3.4, Empirical Bayes (EB) Before-After Method below], applying them annually to all HSIP-related improvements to obtain an overview of program performance. KYTC has found substantial crash reductions and benefit-cost ratios of greater than 3:1 resulting from the deployment of systemic strategies including rumble strips, cable median barriers, and high-friction surface treatments. MaineDOT conducted a simple before-after study evaluation of its systemic implementa- tion of centerline rumble strips. After identifying lane-departure crashes as a priority crash type occurring on its roadway network, MaineDOT worked to identify an appropriate counter- measure to deploy systemically to address these crashes. Despite some public disagreement due to noise complaints, MaineDOT selected centerline rumble strips. Over several years, it deployed centerline rumble strips on roadways, and recently it performed an evaluation using a simple before-after study approach. MaineDOT used varying lengths of before and after periods, with between 6 and 10 years of before data and between 2 and 9 years of after data for its evaluation. To adjust for the varying periods, the aggregated crash data were normalized by the mileage of the sites being considered. Results were then analyzed to determine the percentage change from the before to after periods for all crash severities, fatalities, and serious injuries by all lane-departure crashes as well as head-on and run-off-road crash types. Results of this analysis are summarized in Table 23. Although the simple before-after study evaluation approach is commonly used to quantify the safety effectiveness of systemic safety management programs and treatments, the results of such evaluations are simplistic and may not be reliable, especially in cases with small datasets or few locations being analyzed where results are easily skewed by random crashes. Addition- ally, the approach does not account for traffic variability across the study period unless rates are calculated and does not account for regression to the mean or broad state or national trends. To avoid these issues, other approaches may be considered, such as the shift of proportions and EB methods. All Severities Fatalities Serious Injuries BEFORE AFTER BEFORE AFTER BEFORE AFTER Overall Lane Departure Number 727 199 31 1 71 21 Rate(/100 miles) 143.56 111.20 6.12 0.56 14.02 11.74 Percent improvement (RATE) 22.5% 90.9% 16.3% Head-On Number 145 32 28 1 42 16 Rate(/100 miles) 28.63 17.88 5.53 0.56 8.29 8.94 Percent improvement (RATE) 37.5% 89.9% -7.8% Went Off Road Number 582 167 3 0 29 5 Rate(/100 miles) 114.92 93.32 0.59 0.00 5.73 2.79 Percent improvement (RATE) 18.8% 100.0% 51.2% All corridors pro-rated on miles and before-after years of exposure (10 corridors, 55.56 miles). Rates based on crashes/road miles per year exposure in each corridor’s available before- and after-review period. Example: If a 10 mile rumble strip corridor had 8 years of before history and 4 years of after, crash rate would be based on 80 miles (8 yrs × 10 miles) before, and 40 miles (4 yrs × 10 miles) after. Exposure bases in annual miles of corridors reviewed: Before = 506.42 miles; After = 178.95 miles. Table 23. Safety performance evaluation of Maine corridors with centerline rumble strips installed between 2006 and 2014 (Brunell, 2016).

Best Practices 85 5.3.3 Shift of Proportions Method The shift of proportions method is a common approach for evaluating the effectiveness of systemic safety management programs and treatments when traffic is expected to be an influencing factor, but traffic data are not available. It is also used to supplement other analysis methods, offering additional insight in the evaluation processes. The method computes the ratio of total target crashes to total crashes of all types for both a before period and an after period for a population of project locations. These two ratios are then compared to deter- mine whether the proportion of target crashes changed between the before and after periods for each location, indicating that the deployed treatment may have influenced the rate of target crashes at the project location. Generally, before and after periods of 3 to 5 years are assessed, with the year during which deployment occurs being excluded from the analysis. Once these ratios are calculated for all sites included in the program being evaluated, a statistical test, such as the Wilcoxon Signed-Rank test, is applied to determine if the shift of proportions is significant. A primary benefit of the shift of proportions method is the ability to evaluate locations where changes in traffic may occur between the before and after periods of the evaluation without requiring traffic volume data in the analysis. This can be beneficial for evaluating treatments along corridors where traffic volume data may be limited or unreliable. However, because of the impact of traffic on crash patterns as well as the random nature of crashes, the method may not be reliable for evaluating treatments applied only at a small number of locations or at locations with very few crashes. In addition to the use of other evaluation methods, the KYTC has utilized the shift of proportions method for assessing the impact of its systemic safety management program and treatments. KYTC has evaluated the safety performance of multiple treatments included in its HSIP programming, such as cable median barrier, high-friction surface treatment, and rumble strips. Results of the shift of proportions evaluations for treatment locations using the Wilcoxon Signed-Rank test showed statistical significance for all treatments and have helped to inform programmatic decision making by the state as it continues to develop its systemic safety management program. The shift of proportions method is most effective when used in conjunction with other evaluation methods to obtain a more complete understanding of the impacts of systemic safety management programming. 5.3.4 Empirical Bayes (EB) Before-After Method The EB before-after study evaluation method is an approach that can be used to evaluate the effectiveness of a program or treatment by comparing the number of observed crashes at a selection of treated sites to the expected number of crashes that would have occurred had there been no treatment (AASHTO, 2010). The result of this evaluation produces a CMF, which offers a quantitative description of the safety performance impact of a systemic safety management program or treatment. This is an advanced methodology that is being deployed by a selection of states to evaluate the performance of safety programming as well as to develop state-specific CMFs, which can actively influence future programming and project selection. Because the approach utilizes expected crashes in its computations, it is necessary to have the appropriate SPFs developed for the local network or calibrated to local conditions. Because of the significant level of effort required to produce these SPFs, as well as the data processing and computational complexity, this method may be difficult for some agencies to implement

86 Guide for Quantitative Approaches to Systemic Safety Analysis and may require additional, application-specific training. For this reason, other evaluation methods are often selected, with agencies indicating that they plan to work towards utilizing the EB approach in future evaluations. IDOT recently completed the development of state-specific CMFs for the addition of paved shoulders with rumble strips and pavement markings (IDOT, 2018). CMF development was completed based on projects that were completed between 2008 through 2013, using crash data for the years 2005 through 2016 with before and after periods of 3 years. Results found statistically significant reductions for most crash types and severity levels. For all crash types combined, CMFs for injury crashes ranged from 0.44 to 0.51, and for roadway departure crashes CMFs ranged from 0.38 to 0.51 for different severity levels (Figure 18). These results offer quan- titative insight into the overall impact of the safety programming, which resulted in the systemic implementation of the paved shoulder with rumble strips treatment. As shown in Figure 19, All Sites – All Crash Types/Severities Figure 19. Observed versus expected crashes for all sites and all severities (IDOT, 2018). Figure 18. Crash modification factor 95% confidence intervals (IDOT, 2018) (RD = road departure).

Best Practices 87 results of the evaluation show consistent reductions in observed crashes of all severities and all types relative to expected crashes. With the CMF development using the EB before-after study approach, IDOT was able to achieve an effective evaluation of its systemic programming. 5.3.5 Considerations for Success Because all evaluation methods require several years of before and after crash data, evalua- tions tend to lag behind program deployment by a number of years. This lag is dependent on how many years of after data are considered in the evaluation, as well as the time it takes for each year of crash data to become available after the next year begins. TxDOT performs the simple before-after study when data become available but has developed databases to compile the data so that it can be used when enough after years of data are available. Because the format of crash data may change over the range of years during the evaluation period, it may be difficult to achieve consistency between before and after period datasets as well as sometimes between individual years within a single period. Crash patterns may also change over time due to circumstances outside of the scope of an evaluation, such as technology, demographics, and driver behavior. Because of this, it is important to try to isolate the impacts of the treatments being evaluated from the impacts of other factors as much as possible. This can be achieved through the use of multiple evaluation methods, the use of more rigorous approaches such as the EB approach, and the use of multiple calibration factors for different years when using SPFs. Another challenge for evaluating systemic safety management programs and treatments is dataset size and reliability. Depending on the program or treatment being evaluated, crash and network location datasets may be very small and highly variable. This can yield unreliable results and is difficult to address within the methodological approach. Additionally, if a large number of sites are being evaluated, it may be difficult to maintain consistent datasets over the study period. Agencies have also identified the lack of surrogate measures for crash data as a major limitation of systemic programming evaluation practices. Because the needs of each roadway system change over time, it is important that regular program evaluation be performed. This keeps systemic programming balanced and helps to optimize effectiveness from year to year.