In-Service Performance Evaluation of Guardrail End Treatments (2018)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

1 Study Charge and Origin Roadside safety devices (including guardrails, break-away supports for signposts and roadway lights, median barriers, crash cushions, and other devices) are designed to reduce the risk of occupant injuries when a vehicle runs off the road. The safety of new devices is evaluated by crash testing: the device is installed at a test facility, a vehicle collides with it, and en- gineers assess the consequences of the crash according to criteria related to the risk of occupant injury (e.g., whether the occupant compartment is deformed or penetrated). The American Association of State Highway and Transportation Officials’ (AASHTO’s) Manual for Assessing Safety Hardware (MASH) (AASHTO 2009) defines methods for conducting crash tests of roadside devices and roadside design features (see Box 1-1) and the criteria for determining whether the test results demonstrate that the device is crashworthy. On federal-aid highway projects, the federal government will reimburse the state for the federal share of the expense of installing a roadside device only if the device has been shown to be crashworthy ac- cording to MASH criteria (or according to earlier guidelines if the device was developed before the MASH was published). A manufacturer who wishes to produce a new device arranges for the test to be conducted at an independent testing facility and submits the results for review to the Fed- eral Highway Administration (FHWA). FHWA maintains lists of devices approved for federal aid reimbursement (FHWA 2015). Guardrails are installed at locations that do not provide a safe clear zone in which a vehicle can decelerate without striking a fixed object or encountering unsafe terrain such as a drop-off or precipice. The end of a length of guardrail must be designed so that it does not pose a hazard to 9

10 PERFORMANCE EVALUATION OF GUARDRAIL END TREATMENTS the occupants of a vehicle that strikes it. Highway agencies install end treat- ments of various designs. The end treatment is intended to allow a vehicle striking it to maintain a safe trajectory beyond the device, to redirect the ve- hicle into a safe trajectory, or (in the case of energy absorbing end treatment types) to bring the vehicle to a stop at a safe deceleration rate, depending on the impact conditions. The upstream end of a length of guardrail, that is, the end that a vehicle in the adjacent lane encounters first at the beginning of the length of guardrail, is the end more likely to be struck head-on. The more complex end treatment types (such as energy absorbing types) usually are installed only at the upstream end. Simpler end treatments are used at the downstream end of the length of guardrail if the end treat- ment is positioned so that it is unlikely to be struck by vehicles crossing the center of the road from the opposing direction. Appendix B contains illustrations of guardrail end treatments and other roadside safety devices and a glossary of terms relating to these. COMMITTEE’S TASK The Transportation Research Board formed a committee with the sponsor- ship of the National Cooperative Highway Research Program (NCHRP). The committee’s task (see Box 1-2), as requested by NCHRP, is to Box 1-1 Safety Devices and Roadside Features with Test Procedures Specified in the MASH • Longitudinal barriers • Crash cushions • Terminals (including guardrail end treatments) • Support structures (including sign supports, mailbox supports, and luminaire supports) • Breakaway utility poles • Work zone traffic control devices • Truck- and trailer-mounted attenuators • Variable message sign and arrow board trailers • Work zones longitudinal channelizers • Roadside geometric features (including drainage structures, curbs, driveways, and embankments) SOURCE: AASHTO 2009, 4–5, 13–36.

STUDY CHARGE AND ORIGIN 11 • Develop a research design for evaluating the in-service crash per- formance of guardrail end treatments currently installed; • Determine the data required to carry out the analysis and examine data systems in selected states to determine whether the required data would be available; and • Identify appropriate next steps, which may include new data col- lection for evaluation or use of existing state data resources. The statement of task specifies that the committee is to develop a method for evaluating “the more common energy-absorbing W-beam guardrail end treatments currently installed.” The committee anticipated that methods suitable for this specific category of devices also would be applicable for evaluating the safety performance of other kinds of roadside safety devices. Also, any special data collection or data systems established for evaluation purposes are more likely to be cost-effective if the data have applications for a range of evaluation and management needs. The statement of task is not specific about some of the considerations that will be most important in selecting appropriate evaluation methods. It does not identify the objective of the evaluation, that is, the decisions about highway safety practices that will be guided by the evaluation results; it does not define a measure for performance; and it does not stipulate any administrative requirements or constraints (e.g., the parties who are to perform the evaluation or budget or duration constraints). To fill in these gaps in the statement of the problem, the committee examined the context Box 1-2 In-Service Performance of Energy Absorbing W-Beam Guardrail End Treatments, Phase 1 Statement of Task An ad hoc committee will develop an appropriate research design for evaluating the in-service crash performance of the more common energy absorbing W-beam guardrail end treatments currently installed throughout the United States. The study will also determine the data required to carry out the analysis and include exploratory work in selected states to determine whether the required data, either retrospective or prospective, would be available in sufficient quantity and quality to allow meaningful inferences to be drawn about in-service performance of end treatments collectively and individually. On the basis of the results from the explor- atory work, the committee will either identify appropriate next steps for gathering data or, should it appear unlikely that the necessary data would become avail- able, advise states on the existing process of conducting in-service evaluations of guardrail end treatments.

12 PERFORMANCE EVALUATION OF GUARDRAIL END TREATMENTS of the study request, in particular, the past difficulties highway agencies have encountered in performing in-service evaluations of road features and concerns over the safety of certain guardrail end treatment designs. The committee also reviewed past evaluation studies as sources of examples of objectives, methodologies, and important practical constraints. In this report, unless otherwise defined, the performance of a road- side safety device means the distribution of severities of the outcomes of crashes—that is, the fraction of all crashes that result in property dam- age only, minor injuries only, incapacitating injuries, and fatalities. This definition applies to devices whose intended function is to mitigate crash outcomes. Some road design features—for example, rumble strips—are intended to prevent collisions rather than mitigate the consequences of collisions. For such features, the measure of performance would be the dif- ference in frequency of collisions at locations with and without the feature in place. Collision frequency also is a relevant supplemental measure for the evaluation of end treatments because differences in the design of two alternative types of end treatments—for example, location of the instal- lation with respect to the edge of the roadway—may affect collision fre- quency. Cost-effectiveness is an additional performance measure important to highway agencies: If two alternative types of devices are equally effective at reducing the risk of injury in a collision, the highway agency will prefer the device with the lowest life-cycle cost. The next section of this chapter describes objectives that have been proposed for in-service evaluation of roadside devices. The subsequent sec- tion describes methods used in past evaluations. The final section outlines the remainder of the report. OBJECTIVES OF IN-SERVICE EVALUATION The motivation for the committee’s study came from two sources: NCHRP’s recognition of the need for in-service evaluation and the concern about the performance of a commonly used end treatment type, the ET-Plus. Need for In-Service Evaluation NCHRP recognized that the states historically have not been successful in following the guidance in the MASH (see Box 1-3) and its predecessor documents regarding the need for in-service evaluation. The MASH ex- plains the need for in-service evaluation as follows (AASHTO 2009, 111): Although the crash testing guidelines set forth in this report assure that safety devices function well for the specified test conditions, there are many unknowns and concerns about the impact performance of roadside fea-

STUDY CHARGE AND ORIGIN 13 tures under real-world conditions. Differences between field performance and crash test results can arise due to many factors, including: • Field impact conditions that are not included in crash test guidelines, such as nontracking and side impacts; • Site conditions, such as roadside slopes and ditches, that adversely affect vehicle kinematics before, during, or after impact with the safety device; and • Sensitivity to installation details, such as soil resistance or barrier flare configuration. Therefore, if necessary, conduct an in-service performance evaluation to assess and monitor field performance of roadside safety features. In-service performance evaluation allows user agencies to identify the overall impact performance of a feature as well as identify potential weaknesses or prob- lems with the design. In spite of the advice in the MASH and its predecessor guidelines, the states have carried out few in-service evaluations of roadside features, and most such studies have been hampered by serious gaps in data. State da- tabases of asset inventory, crashes, and traffic have not provided adequate data to support evaluation of the relative safety performance of alternative roadside device designs. States have not perceived malfunctioning roadside devices to be a safety problem of sufficient magnitude to justify the cost of obtaining the data on crashes, devices, traffic, and other road characteristics with the timeliness, accuracy, and detail required for evaluation (AASHTO 2009, 154–155; Heimbecker and Lohrey 2016, 41–45; Ray et al. 2003, 3).1 Concern Regarding the ET-Plus Guardrail End Treatment The second motivation for the committee’s study was a controversy that arose in 2012 about a particular guardrail end treatment, the ET-Plus, manufactured by Trinity Industries. The ET-Plus has been installed in large numbers across the United States. Concerns have been raised that occupants of vehicles in collisions with this device were at greater risk of serious injury than occupants in collisions with alternative end treatment designs (see Box 1-4). FHWA and the states were faced with an immediate need to verify the safety of this device and the validity of their crash-test-based evaluation procedures. A task force formed by FHWA and the state departments of transportation conducted investigations of 161 serious crashes involving collisions with extruding W-beam guardrail end treatments, including 131 involving the ET-Plus, 20 involving other types, and 10 involving unidenti- 1 King Gee presentation to the committee April 14, 2015.

14 PERFORMANCE EVALUATION OF GUARDRAIL END TREATMENTS fied types (Joint AASHTO-FHWA Task Force on Guardrail Terminal Crash Analysis 2015, 23). Also, FHWA led an effort with several states to begin to collect detailed data on end treatment crashes immediately after their oc- currence as a step toward a more definitive evaluation of the performance of these devices (FHWA n.d.). A quantitative comparison of the safety performance of alternative types of end treatments requires data on crashes for an extensive road network over an extended time period, because crashes involving colli- sions with end treatments are infrequent events. From 2010–2015, 0.3 percent of U.S. motor vehicle occupant fatalities were in crashes in which collision with a guardrail end treatment was recorded as the most harmful event (see Table 1-1). Fatalities were more frequent in crashes in which the most harmful event was a collision with a guardrail or other longitudinal Box 1-3 In-Service Performance Evaluation Procedures Proposed in the MASH The Manual for Assessing Safety Hardware (MASH) presents a conceptual frame- work for in-service evaluation and refers to NCHRP Report 490: In-Service Per- formance of Traffic Barriers (Ray et al. 2003), which is described in Chapter 2, as a source for a detailed procedure. Two kinds of evaluation are outlined: new feature evaluation and continuous monitoring. The new feature evaluation would collect information on a limited number of devices of a new design installed as a trial. The information to be collected would include the following: • Installation and maintenance checklist and inventory. The design type and location of each device would be recorded, along with the procedures followed for installation and subsequent maintenance of each device. • Crash monitoring. All crashes with the new devices in the test period would be recorded. Information on reported crashes would come from police reports and information on unreported crashes would come from mainte- nance records. • Crash investigation. In-depth investigations of all fatal and serious injury crashes would include site inspection, examination of the involved ve- hicle, and reconstruction of the crash in terms of impact configuration and conditions.

STUDY CHARGE AND ORIGIN 15 The evaluation based on this data would include the following: • Judgment of “whether the device performed as designed, and if not, whether there are extenuating circumstances”; • Identification of “problems identified with the construction, installation, and maintenance of the device and subsequent remedies”; and • “Recommended changes or modifications to the design and application of the feature.” The term of the evaluation is suggested to be a maximum of 3 to 5 years, and pooling of efforts among several states is suggested to increase the number of devices and observed crashes. The continuous monitoring system would be a set of four spatially linked databases: • Highway geometric and traffic data, • Roadside feature inventory, • Maintenance records for roadside features, including records of crashes not reported to police, and • Crash data from police reports, with indication of the roadside devices involved. The frequency and severity of crashes involving specified roadside features would be periodically tabulated by road characteristics and traffic characteristics. The continuous monitoring system would allow evaluation of the long-term performance of roadside features and could reveal problems not evident in the relatively small number of crashes included in evaluations of new devices. SOURCE: AASHTO 2009, 111–117. barrier or with a support structure, roadside devices for which the MASH also specifies test procedures. Three percent of all 2010–2015 occupant fatalities were in crashes in which a collision with one of the seven kinds of roadside devices listed in Box 1-1 was recorded as the most harmful event (see Table 1-1). (MASH does not specify a crash test for utility poles other than “breakaway” poles. Breakaway poles are not in general use in the United States.) Collisions with end treatments or with other roadside devices account for similarly small percentages of the first harmful events in crashes leading to fatalities (see Table 1-2) and of the most harmful events in all police-reported crashes, as estimated by the U.S. Department of Trans- portation’s (U.S. DOT’s) National Automotive Sampling System–General Estimate System (NASS-GES) (see Table 1-3). U.S. DOT’s Fatality Analysis Reporting System (FARS) and NASS pro-

16 PERFORMANCE EVALUATION OF GUARDRAIL END TREATMENTS vide limited information relevant to assessing the performance of roadside safety devices. If a device performs as intended (i.e., preventing or reducing the severity of injuries in a crash in which the vehicle leaves the roadway), its involvement may not be indicated as a harmful event, or the crash may not be severe enough to be included in the database. Fatal and severe in- jury crashes in which collision with a roadside device is recorded as the most harmful event may be instances of failure of the device to perform as intended, or may be events beyond the design capabilities of the device (e.g., a collision at very high speed or a collision of a tractor–semitrailer). However, crashes in which collision with a device is not coded as a harm- Box 1-4 The ET-Plus Guardrail End Treatment Controversy In 2012, a controversy developed about the safety of a particular guardrail end treatment, the ET-Plus, manufactured by Trinity Industries. As a result of media reports and statements by a private party, highway officials in some states began to analyze crashes involving collisions with that device that resulted in injuries and fatalities and raised their concerns with the Federal Highway Administration (FHWA). In 2014, a federal jury found Trinity Industries liable for damages under the False Claims Act on the basis of allegations of failure to notify FHWA about a change in the design of the device. Trinity has appealed the verdict. The states recognized that these issues with a widely used roadside device (estimated by FHWA to be installed in more than 200,000 locations) was a serious public safety concern and could necessitate a costly replacement program. The case called into question the system of crash testing that the states and FHWA have relied on to ensure the safety of all roadside devices. The American Association of State Highway and Transportation Officials (AASHTO) and FHWA formed a task force in 2014 to examine the safety of the ET-Plus. FHWA published a Federal Register notice seeking information (Federal Register 2014): The purpose of this notice is to request data and information regarding the ET-Plus guardrail end terminal (ET-Plus) manufactured by Trinity Industries, Inc. (Trinity). In 2005, the FHWA determined that [the] ET-Plus guardrail end terminal met the relevant crash test criteria and therefore was eligible for Federal-aid highway funding. This fall, a jury issued a verdict that Trinity made a false or fraudulent claim to FHWA when it sought the eligibility determination for the ET-Plus. Additionally, a number of parties have raised concerns about the in-service performance of the ET-Plus and the potential variability in the dimensions of installed units of the ET-Plus. As a result, FHWA is undertaking a number of efforts to assess these issues. The FHWA is seek- ing technical information and data to assist in this work.

STUDY CHARGE AND ORIGIN 17 In 2016, FHWA published the following summary of the work of the AASHTO- FHWA Task Force (FHWA 2016)a: FHWA, in concert with the American Association of State Highway and Transporta- tion Officials (AASHTO) and the States, has acted on multiple fronts to assess the performance of roadside safety hardware and, specifically, the ET-Plus guardrail end terminal. In particular, questions had been raised about whether the ET-Plus guardrail end ter- minal satisfies applicable safety criteria and performs as intended in the field. FHWA conducted a comprehensive evaluation of the safety performance of the ET-Plus to answer these questions. . . . FHWA and AASHTO formed two joint task forces to examine and address many of these issues. • The first joint task force examined measurements of more than 1,000 ET-Plus devices installed throughout the country to reach two important conclusions: - First, the devices tested are representative of the devices on the road. - And, second, the measurements do not support allegations that Trinity manufactured a second version of the 4-inch ET-Plus. • The second joint task force, comprised of Federal and State safety experts, ana- lyzed the data collected by FHWA to determine whether there is any evidence of unique performance limitations of the ET-Plus 4-inch guardrail terminal and the degree to which any such performance limitations extend to other extruding W-beam guardrail terminals. . . . FHWA has concluded that the device meets the appropriate NCHRP Report 350 criteria [Ross et al. 1993] and it continues to be eligible for Federal aid reimbursement. a The 4-inch ET-Plus is a design version of the ET-Plus end treatment. The ET-Plus was manufactured with feeder channels that were either 4 inches or 5 inches wide. The feeder channel is a component of the head assembly of the end treatment that aligns the end treat- ment with the guardrail behind it (Joint AASHTO-FHWA Task Force on Guardrail Terminal Crash Analysis 2015, 121). ful event may also represent failure of the device. (A hypothetical example would be a crash in which a vehicle struck another vehicle, then overrode a barrier and struck a tree. Each NASS and FARS crash record contains a crash events table, a detailed sequence of events during the crash that should indicate any contact with roadside devices.) This background to the committee’s study suggests a need for capabili- ties to conduct two kinds of evaluations with distinct objectives: • A nationally coordinated evaluation research program to respond to special problems, such as the concern about the adequacy of

18 T A B L E 1 -1 U .S . M ot or V eh ic le O cc up an t Fa ta lit ie s in C ra sh es i n W hi ch S tr ik in g a R oa ds id e O bj ec t W as t he M os t H ar m fu l E ve nt , Se le ct ed R oa ds id e O bj ec ts , an d A ll O cc up an t Fa ta lit ie s, 2 01 0– 20 15 Y ea r E nd T er m in al G ua rd ra il C on cr et e B ar ri er C ab le B ar ri er B ri dg e R ai l Im pa ct A tt en ua to r Si gn Su pp or t U ti lit y Po le / L ig ht S up po rt T re e A ll O cc up an t Fa ta lit ie s 20 10 7 1 43 6 15 4 21 80 11 10 4 1, 01 9 3, 60 2 27 ,8 89 20 11 9 6 40 2 15 4 21 78 14 13 2 91 3 3, 56 7 27 ,1 40 20 12 9 2 40 7 17 6 27 61 22 9 7 1, 01 3 3, 68 7 28 ,0 03 20 13 10 4 39 3 19 7 21 55 21 11 8 92 1 3, 61 6 27 ,1 75 20 14 11 0 37 2 20 3 17 82 21 12 7 95 7 3, 50 8 26 ,9 01 20 15 9 9 40 5 18 9 34 68 21 11 7 92 6 3, 60 5 28 ,6 71 N O T E S : E nd t er m in al f at al it ie s ar e ve hi cl e oc cu pa nt f at al it ie s in c ra sh es in w hi ch t he U .S . D ep ar tm en t of T ra ns po rt at io n’ s Fa ta lit y A na ly si s R ep or t- in g Sy st em ( FA R S) id en ti fie s th e m os t ha rm fu l e ve nt a s st ri ki ng a n en d te rm in al . F at al it ie s in vo lv in g th e ot he r st ru ck o bj ec ts s ho w n in t he t ab le a re de fin ed a na lo go us ly . G ua rd ra il fa ta lit ie s ex cl ud e en d te rm in al f at al it ie s. SO U R C E : FA R S (h tt ps :// w w w .n ht sa .g ov /r es ea rc h- da ta /f at al it y- an al ys is -r ep or ti ng -s ys te m -f ar s) .

19 T A B L E 1 -2 U .S . M ot or V eh ic le O cc up an t Fa ta lit ie s in C ra sh es i n W hi ch S tr ik in g a R oa ds id e O bj ec t W as t he F ir st H ar m fu l E ve nt , Se le ct ed R oa ds id e O bj ec ts , 20 10 –2 01 5 Y ea r E nd T er m in al G ua rd ra il C on cr et e B ar ri er C ab le B ar ri er B ri dg e R ai l Im pa ct A tt en ua to r Si gn Su pp or t 20 10 15 3 87 7 25 7 54 11 1 27 38 4 20 11 18 2 80 0 25 6 48 10 5 23 43 3 20 12 16 8 78 0 27 9 70 91 38 42 1 20 13 19 5 77 4 29 0 66 8 1 34 40 2 20 14 17 8 76 7 30 6 62 9 6 29 38 7 20 15 18 4 79 9 28 3 79 10 6 37 44 5 N O T E S : S om e fa ta lit ie s w er e in c ra sh es i n w hi ch t he U .S . D ep ar tm en t of T ra ns po rt at io n’ s Fa ta lit y A na ly si s R ep or ti ng S ys te m ( FA R S) r ec or ds t he co lli si on w it h th e ro ad si de o bj ec t as b ei ng b ot h th e fir st h ar m fu l e ve nt a nd t he m os t ha rm fu l e ve nt . F or e xa m pl e, in 2 01 5, o f th e 99 o cc up an t fa ta li- ti es in w hi ch c ol lis io n w it h an e nd t er m in al w as t he m os t ha rm fu l e ve nt ( Ta bl e 1. 1) , i n 83 t he e nd t er m in al c ol lis io n w as a ls o th e fir st h ar m fu l e ve nt . T hu s, t he n um be r of f at al it ie s in c ra sh es i n w hi ch c ol lis io n w it h an e nd t er m in al w as t he fi rs t ha rm fu l or m os t ha rm fu l ev en t, o r bo th , w as 2 00 . SO U R C E : FA R S (h tt ps :// w w w .n ht sa .g ov /r es ea rc h- da ta /f at al it y- an al ys is -r ep or ti ng -s ys te m -f ar s) .

20 T A B L E 1 -3 N um be rs o f C ra sh es i n W hi ch S tr ik in g a R oa ds id e O bj ec t W as t he M os t H ar m fu l E ve nt , Se le ct ed R oa ds id e O bj ec ts , an d A ll C ra sh es , 20 10 –2 01 5 Y ea r E nd T er m in al G ua rd ra il C on cr et e B ar ri er C ab le B ar ri er B ri dg e R ai l Im pa ct A tt en ua to r Si gn Su pp or t A ll C ra sh es 20 10 12 ,1 00 84 ,9 00 67 ,1 00 1 ,0 00 7, 80 0 2, 20 0 43 ,7 00 5, 41 9, 00 0 20 11 8 ,9 00 83 ,7 00 72 ,4 00 5 ,3 00 9, 80 0 3, 50 0 37 ,4 00 5, 33 8, 00 0 20 12 11 ,2 00 83 ,2 00 68 ,2 00 5 ,8 00 8, 30 0 3, 00 0 43 ,1 00 5, 61 5, 00 0 20 13 9 ,2 00 83 ,7 00 74 ,9 00 7 ,6 00 9, 20 0 2, 90 0 55 ,6 00 5, 68 7, 00 0 20 14 10 ,4 00 92 ,1 00 67 ,2 00 10 ,9 00 6, 80 0 2, 80 0 48 ,8 00 6, 06 4, 00 0 20 15 11 ,6 00 88 ,2 00 67 ,1 00 9 ,9 00 7, 30 0 2, 80 0 51 ,9 00 6, 29 6, 00 0 N O T E : T he N at io na l A ut om ot iv e Sa m pl in g Sy st em ( N A SS ) es ti m at es o f nu m be rs o f cr as he s ar e de ri ve d fr om a s am pl e of a ll cr as he s in w hi ch a po lic e ac ci de nt r ep or t w as c om pl et ed a nd w hi ch r es ul te d in p ro pe rt y da m ag e, i nj ur y, o r de at h. SO U R C E : N A SS -G E S (h tt ps :// w w w .n ht sa .g ov /r es ea rc h- da ta /n at io na l- au to m ot iv e- sa m pl in g- sy st em -n as s) .

STUDY CHARGE AND ORIGIN 21 designs now in use, and to meet common needs of the states and the federal government such as validation of crash testing, and • The routine in-service evaluation of end treatments and other road- side devices that MASH recommends for applications such as – Monitoring the highway agency’s inventory of roadside devices to ensure proper condition, – Planning cost-effective maintenance and replacement, and – Checking that newly adopted devices are performing as expected. Two expert groups—the AASHTO-FHWA Task Force that investi- gated guardrail end treatment crashes and the NCHRP project panel that prepared the project statement that was the origin of the present study— recently identified a need for a research effort with national applicability to evaluate the in-service performance of roadside devices. The report of the AASHTO-FHWA Task Force included the following recommendation: The Task Force recommends that comprehensive in-service performance evaluations of guardrail terminals be conducted at the national and State levels. As previously highlighted in this report, the Task Force’s assessment did not involve a complete in-service evaluation and concentrated on a lim- ited group of mostly higher severity crashes, specifically focused on crashes with the ET-Plus terminal. The findings of this report should be considered by the National Academies’ National Research Council (NRC) committee that is conducting a project entitled “In-Service Performance of Energy- Absorbing W-Beam Guardrail End Treatments.” (Joint AASHTO-FHWA Task Force on Guardrail Terminal Crash Analysis 2015, 8) In the AASHTO-FHWA Task Force’s conception, a “complete in-service evaluation” would • Be “a well-designed and executed research study to collect all of the necessary data and account for exposure and confounding factors such as proper installation” (Joint AASHTO-FHWA Task Force on Guardrail Terminal Crash Analysis 2015, 12); • “Provide relative comparisons of the in-service safety performance of individual terminal types” and “an indication of the frequency of occurrence of the individual performance limitations” (Joint AASHTO-FHWA Task Force on Guardrail Terminal Crash Analy- sis 2015, 5); and • “Determine whether crash-tested hardware [has] performance limitations that are not detected by the testing process,” provid- ing guidance for amending testing criteria (Joint AASHTO-FHWA Task Force on Guardrail Terminal Crash Analysis 2015, 8).

22 PERFORMANCE EVALUATION OF GUARDRAIL END TREATMENTS The NCHRP project statement (NCHRP 2013) describes the evaluation envisioned as a research project to improve understanding of the perfor- mance of end treatments. The objectives would be to allow • Highway agencies to select the most cost-effective device types for new construction • Agencies to maximize the cost-effectiveness of projects to upgrade end treatments (i.e., projects to selectively replace existing end treatments with better-performing devices) • Agencies to avoid installation and maintenance practices that de- grade end treatment performance The NCHRP project statement specified that the evaluation was to take into account the sensitivity of the performance of each device type to environ- mental conditions, site characteristics, impact conditions, and the quality of installation and maintenance of the device. In summary, evaluation objectives that have been proposed and that would be within the scope of a nationally coordinated program of evalua- tion research include the following: • Comparison of the effectiveness of the various designs of guardrail end treatments and other roadside safety devices now in use, tak- ing into account factors aside from design that may affect injury severity in a crash, such as roadway and roadside characteristics, vehicle type, and crash dynamics. The results would aid highway agencies in selecting designs to be installed at particular locations and in setting priorities for the replacement of devices. • Evaluation of the influence of installation and maintenance prac- tices on device performance. • Validation and refinement of crash test procedures. • Design of more effective roadside safety devices on the basis of the measured in-service performance of existing designs. Routine, state-level evaluation, as recommended by the MASH, could have the following objectives: • Assurance to the highway agency that newly adopted roadside de- vice designs provide the safety performance expected on the basis of crash testing results; • Identification of local factors that affect performance (e.g., local installation and maintenance practices or local roadside environ- ment conditions);

STUDY CHARGE AND ORIGIN 23 • Evaluation of the long-term performance of roadside safety devices with respect to safety and cost-effectiveness, as affected over time by maintenance and repair, normal deterioration, and changes in vehicle characteristics; and • Periodic evaluation of the agency’s inventory of roadside safety devices to ensure that the devices installed are appropriate for their locations and that instances of improper installation or mainte- nance that can compromise performance are minimized. As noted above, the state highway agency interviews conducted for the committee, as well as earlier studies, found that highway agencies lack resources for conducting routine in-service evaluations. They would un- likely be willing to invest in developing evaluation capabilities without clear evidence that the results of evaluations can be useful for guiding decisions on selection, maintenance, and replacement of roadside safety devices. EVALUATION METHODS The committee reviewed past evaluations of guardrail end treatments, evaluations of other highway risk mitigation measures, and analogous evaluation studies in other fields to learn about the methods used and also about the motivations and objectives of the studies and the uses of their results. Methods of evaluation studies can be grouped into three general categories: true experiment, nonexperimental comparative evaluation, and descriptive evaluation. 1. True experiment. In an evaluation with an experimental study design, a pool of test subjects is assembled and each subject is randomly assigned one of two or more alternative treatments. For example, an experimental evaluation of a new type of some category of roadside device would first select from the highway system a group of sites warranting installation of that category of device. Either the new type or the current type would be randomly selected for installation at each site, and measures of safety at the group of sites with the new design would be compared with those at the group of control sites for a set period of time. Any difference in safety outcomes between the two groups is then attributed to the difference in the performance of the two types of devices because the two groups of test sites are expected to be similar with respect to other characteristics affecting safety. True experiments are almost never conducted in highway research—in part because random assignment is logistically dif-

24 PERFORMANCE EVALUATION OF GUARDRAIL END TREATMENTS ficult in the highway construction environment—and are not con- sidered further in this report. 2. Nonexperimental comparative evaluation. In a nonexperimental study, the treatments are not assigned randomly. Instead, statistical methods are used to control for factors that may affect safety other than the treatments being evaluated. For example, a comparative evaluation of two alternative types of roadside devices might test the hypothesis that a motorist has the same risk of injury in a crash when traveling on a road segment where a roadside device of type X has been installed as on a road segment where a device of type Y has been installed. The evaluation would use data on the types of devices installed, traffic, and other characteristics that affect safety on a col- lection of road segments to control statistically for other risk factors (e.g., through multiple regression). An evaluation of this kind may be retrospective—that is, an examination of historical experience that uses existing data that were not necessarily collected for the purpose of the evaluation—or prospective—a study that uses data on crashes and on factors related to crash and casualty risks collected according to a plan designed to meet the requirements of the evaluation. 3. Descriptive evaluation. An evaluation of this kind consists of a program of regular observation of the performance of the subject treatment for a period of time, after which a qualitative judgment is made regarding the adequacy of the performance. The new feature evaluation proposed in the MASH (see Box 1-3) is an example of a descriptive or qualitative evaluation: Trial installations of a new type of device are regularly monitored for a period of time and all collisions involving the device type are investigated. At the end of the trial period, the evaluators determine by judgment whether the new device type performs as intended. No quantitative estimate is made of the effect of the device on casualty risk as compared with alternative treatments. For any particular evaluation need, the three methods will differ in cost, practicality, and data resources required and in the questions that each is capable of answering. Nonexperimental comparative research designs that seek causal relationships are the predominant method in highway safety research. This method has been used to demonstrate the safety benefits of a wide range of road design and vehicle design features and traffic control measures and the risks of specific driver behaviors. However, the new fea- ture evaluation outlined in the MASH and the 2016 FHWA end treatment pilot evaluation (described in Chapter 2) are descriptive evaluations, and highway agencies probably have used this method more often than com- parative evaluations.

STUDY CHARGE AND ORIGIN 25 Practical considerations will dictate the choice of evaluation methods that highway agencies will be willing to adopt. The past studies summa- rized in Chapters 2 and 3 illustrate the difficulties of assembling timely and reliable data on the occurrence, circumstances, and outcome of crashes in- volving roadside safety devices; the locations, designs, and installation and maintenance history of the devices; and traffic and roadway characteristics in the study area. The scale and complexity of the study design for an evalu- ation should be consistent with the potential value of the information the evaluation will produce. Undertaking an evaluation is reasonable only if the expected safety benefits justify the cost and the likely diversion of resources from other safety-related activities. Therefore, the committee took care to identify practical challenges and potential safety benefits in its comparison of alternative evaluation methods. Merely specifying data collection and analysis techniques would be insufficient guidance to highway agency administrators considering un- dertaking evaluation. A program of in-service performance evaluation of road safety features (whether a national program of evaluation research or routine evaluation by a state highway agency) will have three essential components: • Administrative framework. This framework includes definitions of the objectives of evaluations and of how the agency will use the results, protocols determining which features are to be evaluated and the schedule for evaluations, organizational arrangements for cooperation among the agencies and offices that are to be involved (e.g., police agencies and highway agency offices of maintenance, traffic and safety, and design), and funding provisions. • Data systems and procedures. The content, reliability, and timeli- ness of general-purpose databases recording roadway geometry, traffic, roadside features, maintenance activities, and crashes must be adequate to support the objectives of the evaluations. Some evaluations may require special data collection, but routine evalu- ation will be most cost-effective if the required data collection and management are integrated with the agency’s safety management, maintenance management, and asset management systems. • Evaluation methodologies. How safety performance will be mea- sured, including the data elements required and the methods for analyzing the data, must be specified. The statement of task refers to each of these components: The study is to propose a research design and identify data needs, and also to advise the states on their evaluation processes. The committee’s recommendations ad- dress each component.

26 PERFORMANCE EVALUATION OF GUARDRAIL END TREATMENTS ORGANIZATION OF THE REPORT The remainder of this report is organized as follows: • Chapter 2 describes the methods used in past in-service evaluations of guardrail end treatments and other roadside safety devices. • Chapter 3 proposes objectives and procedures for a nationally coordinated research program for evaluation of roadside safety devices. • Chapter 4 outlines methods for highway agencies to conduct rou- tine in-service evaluations of roadside safety devices. • Chapter 5 presents the committee’s conclusions about the need for in-service evaluation and recommendations regarding methods and procedures. • Appendix A describes the papers commissioned by the committee. One paper surveys highway agency practices regarding the selec- tion, installation, and maintenance of end treatments and other roadside safety devices. Two commissioned papers are literature reviews of studies that provide examples of methods applicable to the evaluation of roadside safety devices. • Appendix B contains a glossary and illustrations relating to the design, testing, and use of guardrail end treatments and other roadside safety devices. REFERENCES Abbreviations AASHTO American Association of State Highway and Transportation Officials FHWA Federal Highway Administration NCHRP National Cooperative Highway Research Program AASHTO. 2009. Manual for Assessing Safety Hardware 2009. AASHTO, Washington, D.C. Federal Register. 2014. ET-Plus Guardrail End Terminal. Vol. 79, No. 247, Dec. 24, p. 77595. https://www.federalregister.gov/documents/2014/12/24/2014-30081/et-plus-guardrail- end-terminal. FHWA. 2015. Guardrail Basics. https://www.fhwa.dot.gov/guardrailsafety/guardrailbasics. cfm. FHWA. 2016. FHWA Review of ET-Plus. https://www.fhwa.dot.gov/guardrailsafety. FHWA. n.d. Data Collection: In-Service Performance Evaluation of Guardrail End Terminals. Heimbecker, C., and E. C. Lohrey. 2016. Examples of State Highway Agency Practices Re- garding Design, Installation, Maintenance, and Evaluation of Guardrail End Treatments. Background Paper for Special Report 323: In-Service Performance Evaluation of Guard- rail End Treatments. Transportation Research Board, Washington, D.C.

STUDY CHARGE AND ORIGIN 27 Joint AASHTO-FHWA Task Force on Guardrail Terminal Crash Analysis. 2015. Safety Analysis of Extruding W-Beam Guardrail Terminal Crashes. https://www.fhwa.dot.gov/ guardrailsafety/safetyanalysis. NCHRP. 2013. NCHRP 22-30 [Pending] In-Service Performance Evaluation of W-Beam End Terminals. http://apps.trb.org/cmsfeed/TRBNetProjectDisplay.asp?ProjectID=3669. Ray, M. H., J. Weir, and J. Hopp. 2003. NCHRP Report 490: In-Service Performance Evalu- ation of Traffic Barriers. Transportation Research Board of the National Academies, Washington, D.C. Ross, H. E., Jr., D. L. Sicking; R. A. Zimmer; and J. D. Michie. 1993. NCHRP Report 350: Recommended Procedures for the Safety Performance Evaluation of Highway Features. Transportation Research Board of the National Academies, Washington, D.C.