In-Service Performance Evaluation of Guardrail End Treatments (2018)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

4 Routine In-Service Evaluation of Roadside Devices Chapter 1 cited the recommendations in the Manual for Assessing Safety Hardware (MASH) that highway agencies conduct in-service evaluations of roadside safety devices as a routine activity. The first section of the present chapter identifies potentially useful applications of the results of a highway agency program of in-service evaluation. Past studies and the committee’s interviews with state highway agencies have found that such evaluations are rare. The second section discusses reasons for this lack of commitment to evaluation. The final section describes actions that could strengthen the capacity of highway agencies to conduct evaluations and apply their results. OBJECTIVES OF ROUTINE EVALUATIONS The nationally coordinated evaluations proposed in Chapter 3 would be research to acquire new knowledge about the safety performance of road- side devices or to develop new methods of evaluation. In contrast, a state highway agency program of in-service evaluation of roadside safety devices would be a management support function to provide the agency with infor- mation to assure proper performance and to guide decisions about selection and maintenance of the devices. The MASH in-service evaluation recommendations (summarized in Chapter 1) and the committee’s reviews of state highway agency practices and of past evaluations suggest at least three applications of in-service evaluations for improving the performance of highway programs: 94

ROUTINE IN-SERVICE EVALUATION OF ROADSIDE DEVICES 95 • Providing notice that a device type is not performing as expected. Evaluation would give the highway agency assurance that any device type that was susceptible to failure in circumstances over- looked in design and testing would be identified, even if failures were rare. The agency would monitor the performance of newly adopted device types and also of the types in general use at the time the evaluation program was initiated. • Long-term monitoring to identify cost-effective practices regard- ing the selection and maintenance of roadside safety devices. Even if all alternative devices perform acceptably (as intended by the designer), some device types may be more effective than others in mitigating crash severity or have advantages with respect to life-cycle costs or ease of installation and maintenance. The best- performing device type may depend on the characteristics of the location. • Ensuring that the roadside safety devices on the agency’s roads are appropriate for their locations and have been properly installed and maintained. Because of the lack of highway agency experience in conducting evalua- tions, the practicality, appropriate scale, and utility of such activities have yet to be determined. The demonstration of evaluation methods for routine highway agency use proposed in Chapter 3 would provide information on feasibility, costs, and benefits. OBSTACLES TO EVALUATION National Cooperative Highway Research Program (NCHRP) Report 490 identifies three likely reasons why states have rarely conducted in-service evaluations of roadside devices (Ray et al. 2003, 3): 1. There was no formal process to use, so each state had to develop its own procedures; 2. Collecting and analyzing the data was labor intensive; and 3. Agencies did not perceive a benefit from performing in-service evaluations. In interviews with administrators and engineers from 10 state highway agencies conducted for the committee (Heimbecker and Lohrey 2016, 35– 37), officials cited similar reasons for not undertaking evaluations:

96 PERFORMANCE EVALUATION OF GUARDRAIL END TREATMENTS • Difficulties involving coordination and communication: - Lack of a cooperative relationship with police agencies for timely receipt of a crash report, - Restrictions on the release of data involved in police investiga- tion, and - Lack of established procedures and intraagency relationships needed for collation of maintenance records for use in evaluations. • Resource limitations: - Lack of crash investigation capability within the highway agency and - Lack of funding and staff for data collection and analysis. • Lack of perceived value from in-service evaluation. In general, the agencies appear not to regard roadside device performance as an area of major safety deficiency. The last reason that agencies do not see clear evidence of the benefits of the practice probably is primary. If the agency does not perceive a pressing need for in-service evaluation, then the activity cannot compete with other activities in the budget process and administrators will not be motivated to develop the necessary inter- and intra-agency arrangements. STRENGTHENING HIGHWAY AGENCY CAPACITY TO CONDUCT EVALUATIONS Chapter 1 identified three essential components of a state highway agency program of in-service performance evaluations of roadway safety features: • An administrative and planning structure that defines evalua- tion objectives, the scope of evaluation, and responsibilities for evaluation and oversees the application of results. The adminis- trative function would include maintaining arrangements for co- operation among the agencies and offices that would be involved (which might include police agencies and various offices within the highway agency) and determining funding needs in the highway agency’s budget process. • Data systems and procedures. The content, reliability, and timeli- ness of the agency’s general-purpose databases’ recording of road- way and traffic characteristics, maintenance activities, and crashes must be adequate to support the objectives of the evaluations. Special data collection procedures (e.g., crash site inspections) may be needed for some evaluations, but an in-service evaluation pro- gram will be practical only if it relies primarily on established data systems.

ROUTINE IN-SERVICE EVALUATION OF ROADSIDE DEVICES 97 • Evaluation methodologies. Measures of performance and data analysis methods for computing the measures must be specified. These components parallel the components of the maintenance manage- ment systems and asset management systems that many highway agencies have adopted. The objectives and procedures of in-service evaluation also are related to the objectives and procedures of these management systems. For example, a basic function of a performance-based maintenance man- agement or asset management system is to give prompt notice of assets that are not performing up to standards. An asset that is important for safety and is performing poorly will be assigned a high priority for attention in the maintenance or capital program. States that have made major investments of dollars, time, and training in management systems would hesitate to invest in parallel data collection and analysis for in-service performance evaluation. Extending the existing management systems to support in-service safety performance evaluation of roadside devices would take advantage of the investment already made in the management systems and increase their usefulness. Therefore, the most practical home for an in-service evaluation capability in many state highway agencies would be as an extension of the agency’s maintenance management and asset management systems. The discussion below refers to maintenance management systems, but it is to be understood that ca- pabilities applicable to monitoring roadside safety devices may reside in a highway agency’s asset management system or maintenance management system, depending on agency practices. The first subsection below describes extensions of maintenance man- agement systems to allow them to support in-service evaluation of roadside devices. The second subsection describes the role that the nationally coor- dinated demonstrations proposed in Chapter 3 could play in developing practical procedures. The final subsection addresses some additional kinds of evaluation activities: special purpose evaluations, evaluation through trial installations, and pooled multistate evaluations. Integrating Evaluation with Maintenance Management A maintenance management system enables a highway agency to track the condition of its inventory of facilities and to determine which mainte- nance program will keep all facilities at acceptable levels of condition and performance for the least cost. Data are the foundation of a maintenance management system. Three necessary data processes are an asset inventory, condition assessment surveys, and a work reporting system that provides tracking of the history of the maintenance performed on each asset and costs (AASHTO 2005, 21–34).

98 PERFORMANCE EVALUATION OF GUARDRAIL END TREATMENTS Categories of assets tracked in a typical maintenance management sys- tem include drainage structures, roadside features (e.g., fences, landscaping, sound barriers), pavements, traffic items (e.g., signs, pavement markings, guardrails, impact attenuators), and rest areas (Zimmerman and Stivers 2007, 15). In a 2005 survey, of 29 states responding, 27 reported having a maintenance management system in place (Zimmermann and Testa 2008, A-3). A 2015 survey of state highway agencies identified that 18 of the 28 agencies responding regularly inspected and recorded the condition of guardrail end treatments as part of a formal maintenance quality assur- ance program and that 21 of the 28 had an inventory of end treatments completed or partially completed. Similar numbers of states maintain data on other kinds of roadside safety devices (Zimmerman 2015, 22, 23, 132). A basic function of a performance-based maintenance management system is to give prompt notice of facilities that are not performing up to standards. Condition assessment surveys periodically rate assets according to performance-based measures. For example, lighting may be rated by the percentage of lights malfunctioning and drainage ditches by the frequency of blockages (Zimmerman and Stivers 2007, 12). In one state’s maintenance management system, the performance-based measure for guardrails is the percentage of length that is either damaged or “not functioning as de- signed” (Zimmerman and Stivers 2007, 23). The maintenance management system includes specifications of standards for minimum permissible per- formance ratings of assets and procedures for projecting the maintenance activities required to keep assets above the minimum. The evaluation procedures of both NCHRP Report 490 (Ray et al. 2003) and the Texas Transportation Institute (TTI) (van Schalkwyk et al. 2004) envision data collection for evaluation as an adjunct to maintenance data collection that is already occurring. “Working within existing main- tenance reporting procedures” was a design criterion of the TTI procedure (van Schalkwyk et al. 2004, 17). NCHRP Report 490 describes the roadside device evaluation as part of a state safety management system and observes that “since a successful [in-service performance evaluation] requires a large amount of data, it is most effective when data collection efforts are incor- porated into routine maintenance and repair procedures” (Ray and Weir 2002, 1–2). To add in-service safety performance evaluation to the capabilities of a maintenance management system, four improvements to data procedures would be needed: • Recording the characteristics and postcrash condition of a device involved in a crash with sufficient detail to assess its performance. It may be possible to obtain this data as part of maintenance

ROUTINE IN-SERVICE EVALUATION OF ROADSIDE DEVICES 99 work reporting, as proposed in the NCHRP Report 490 and TTI procedures. • Condition assessment surveys that check roadside safety features for incorrect installation or maintenance and for design types not suitable for their locations. For example, categories of problems with guardrail end treatment installation and maintenance that are likely to affect performance are noted in NCHRP Report 490 (Ray et al. 2003, 54), by the AASHTO-FHWA guardrail end treatment task force (Joint AASHTO-FHWA Task Force on Guardrail Ter- minal Crash Analysis 2015, 111–115), and in an FHWA technical brief (FHWA 2014; Furst 2015). Condition assessment surveys are a component of any maintenance management system, but the methods and rating schemes of the surveys are not necessarily designed to observe all the features of roadside safety devices that affect their performance. • A method of linking the police accident report to records that identify the roadside devices involved in a crash and the device’s condition and performance in the crash. Adding questions about the device type and condition to accident report forms probably would be impractical. Crash reports could be linked to device information by means of location coordinates. A third method, proposed in a presentation to the committee, would be to have the installers of the device label it with a radio-frequency identification (RFID) label or barcode tag.1 The label could be read electronically by police at a crash site and by maintenance workers repairing the device. A final method is already in use in some states (Heimbecker and Lohrey 2016, 52). Police at a crash scene affix a tag to roadside devices damaged in the crash to enable the state to bill the drivers responsible for the damage. The information in the tag links the damaged device to the crash report. • Timely and reliable highway agency access to police accident re- ports. The highway agency usually will have a timely awareness of crashes that involve significant damage to road features, through the involvement of the maintenance department. However, the police report contains information that is not available to the maintenance workers at the site. Also, crashes that involve only minor damage to roadside features might not be recorded by the maintenance department for some time. In the interviews conducted for the committee, highway agencies cited delays in obtaining police crash reports or the inability to locate reports 1 Dean Sicking presentation to the committee April 14, 2015.

100 PERFORMANCE EVALUATION OF GUARDRAIL END TREATMENTS as obstacles to in-service evaluation. Only two of 10 agencies interviewed reported that they had arrangements to receive notification of crashes di- rectly from police on a routine basis (Heimbecker and Lohrey 2016, 43, 45). Highway agency access to police accident reports is needed not only for evaluation of roadside devices, but for general management of the agency’s safety programs. For crashes investigated by state police on state roads, the necessary communication would be entirely within the state government. Crash reports completed by or filed with local police may be more difficult to obtain promptly. ROLE OF A NATIONALLY COORDINATED DEMONSTRATION Chapter 3 proposed a nationally coordinated demonstration of routine evaluation methods to test practical methods, measure the cost to the highway agency of an evaluation program, and assess whether the agency can derive benefit from evaluation in the form of improved safety and cost-effectiveness in its highway programs. If such a demonstration is con- ducted, its results should provide an authoritative guide to best practices for in-service evaluation. Highway agencies would be asked to volunteer to participate in the demonstration. Probably the best participants would be states with functioning maintenance management systems. These states will already have in place some of the resources and processes necessary for in-service evaluation. The demonstrations could be based on the two fully developed pro- posals for in-service evaluations described in the annex to Chapter 3: the NCHRP Report 490 procedures and the TTI procedure developed for the Texas Department of Transportation. Both procedures have already under- gone trials (although the TTI procedure trial was on a small scale). The two procedures involve contrasting approaches that reflect the TTI authors’ emphasis on minimizing the burden imposed by the procedure: • In the TTI procedure, a case record is initiated by a maintenance crew at the site of a crash to repair damage. The NCHRP Report 490 procedure anticipates that normally the police crash report initiates a case record: either the police agree to notify the high- way agency evaluators of crashes involving roadside devices or the evaluators regularly contact police to inquire about cases (Ray and Weir 2002, 27, 29). • The TTI procedure has a two-phase structure with limited data col- lection on a continuous basis and more detailed data collection on occasions when a device is identified as performing below standard.

ROUTINE IN-SERVICE EVALUATION OF ROADSIDE DEVICES 101 The NCHRP Report 490 procedure does not specify a second stage for increased scrutiny of selected devices. • The TTI procedure’s forms for recording information about crashes involving roadside devices are much less detailed than the NCHRP Report 490 forms. The TTI Phase I form asks only a single ques- tion about device performance: whether the device performed as intended. Issues about the methodology of in-service evaluations that highway agency demonstrations could help answer include • Trade-offs between minimal approaches to data collection and more rigorous procedures (e.g., as in the differences between the TTI and NCHRP Report 490 procedures), • The reliability of judgmental performance criteria (e.g., the performed-as-intended criterion in the TTI procedure, similar to the criterion applied in the FHWA evaluation pilot described in the annex to Chapter 3), • The value of collecting data on non-police-reported crashes in an evaluation of roadside devices, • The reliability and accuracy of alternative data sources and proce- dures for identifying crashes involving roadside devices (Chapter 2 noted wide variation and seeming inconsistencies in the results of past studies with respect to reported rates of collisions involving roadside devices), and • Sample size requirements for identifying unsatisfactory device per- formance and sample size for comparing the performance of alter- native device types (the statistical analysis methods proposed in both the TTI and NCHRP Report 490 procedures are problematic, as noted in Chapter 3; therefore, the development and demonstra- tion of appropriate methods are needed). Other Evaluation Considerations The NCHRP Report 490 and TTI procedures are applicable to a continuous process of performance monitoring, analogous to the continuous processes of maintenance management and asset management systems. Evaluations of finite duration and with special objectives could also be a part of an agency’s evaluation program. Chapter 2 cited examples of such evaluations: • A special survey of roadside features to determine the frequency of conditions believed to affect crash injury risk; for example, a sur- vey of a sample of guardrail end treatments could be conducted to

102 PERFORMANCE EVALUATION OF GUARDRAIL END TREATMENTS check for the occurrence on the state’s highway system of installa- tion and maintenance conditions that were identified as sources of performance limitations in the 2015 AASHTO-FHWA Task Force investigations of end treatment crashes (see Chapter 3); • A series of crash investigations as case studies for in-depth exami- nation of a particular performance concern; and • A trial installation of a new device. The in-service evaluation procedure proposed in the MASH (see Chap- ter 1, Box 1-2) calls for a trial installation of a newly adopted device type before a full deployment of the device on the agency’s roads occurs. Also, until 1993, FHWA’s certification of a new roadside device as eligible for federal-aid reimbursement (on the basis of crash testing) initially designated the device as experimental. FHWA instructed states to conduct in-service trials of the new device and submit the results to FHWA. FHWA’s intent was that it would then change the designation of devices that were successful in trials from experimental to operational. However, few states submitted trial results and FHWA discontinued the designation of experimental (Ray et al. 2003, 3). A 2001 summary and analysis of data from state in-service evaluations of end treatments found studies from eight states conducted in the 1980s and 1990s that recorded information on 649 end treatment crashes (Bowman and Peterson 2001). The Wisconsin trial of the ET-2000 guardrail end treatment described in Chapter 2, Box 2-5, illustrates the difficulties of trial evaluations. In a 5-year trial of 42 devices, 20 collisions were observed—barely a sufficient number to support conclusions about performance. The 5-year duration would be impractical for most purposes. An additional limitation of such trials is that installation and maintenance may be performed with greater attention and expertise in the trial than would be the case in general use. At the collision rate of the Wisconsin trial, to observe 100 collisions in a 1-year trial of a guardrail end treatment would require the installation of 1,050 trial devices. A trial of this scale seems impractical for a single agency but might be feasible as a multistate effort. In all forms of evalua- tion, highway agencies could overcome the sample size problem and avoid duplicative efforts through multistate cooperation. Care would be necessary to ensure that data collection practices were consistent in all the participat- ing jurisdictions.

ROUTINE IN-SERVICE EVALUATION OF ROADSIDE DEVICES 103 REFERENCES Abbreviations AASHTO American Association of State Highway and Transportation Officials FHWA Federal Highway Administration AASHTO. 2005. Guidelines for Maintenance Management Systems. AASHTO, Washington, D.C. Bowman, B., and D. Peterson. 2001. The Cost Effectiveness of Updating Guardrail End Treatments. Final Report IR-01-01. Department of Civil Engineering, Auburn University, Alabama. May. https://eng.auburn.edu/files/centers/hrc/IR-01-01.pdf. FHWA. 2014. Selection, Installation and Maintenance of W-Beam Guardrail End Terminals. http://safety.fhwa.dot.gov/roadway_dept/policy_guide/road_hardware/policy_memo/ memo052615/memo052615_attachment.cfm. Last modified August 14, 2014. Furst, T. 2015. Memorandum: Action: Roadside Safety Hardware. FHWA. May 26. http://safety.fhwa.dot.gov/roadway_dept/policy_guide/road_hardware/policy_memo/ memo052615/memo052615.pdf. Heimbecker, C., and E. C. Lohrey. 2016. Examples of State Highway Agency Practices Re- garding Design, Installation, Maintenance, and Evaluation of Guardrail End Treatments. Background Paper for Special Report 323: In-Service Performance Evaluation of Guard- rail End Treatments. Transportation Research Board, Washington, D.C. Joint AASHTO-FHWA Task Force on Guardrail Terminal Crash Analysis. 2015. Safety Analysis of Extruding W-Beam Guardrail Terminal Crashes. https://www.fhwa.dot.gov/ guardrailsafety/safetyanalysis. Ray, M. H., and J. Weir. 2002. Appendix D: In-Service Performance Evaluation Procedures Manual. Contractor’s Final Draft, NCHRP Project 22-13(2). Transportation Research Board of the National Academies, Washington, D.C. http://onlinepubs.trb.org/online pubs/nchrp/nchrp_rpt_490aD.pdf. Ray, M. H., J. Weir, and J. Hopp. 2003. NCHRP Report 490: In-Service Performance Evalu- ation of Traffic Barriers. Transportation Research Board of the National Academies, Washington, D.C. van Schalkwyk, I., R. P. Bligh, D. C. Alberson, D. L. Bullard, Jr., D. Lord, and S.-P. Miaou. 2004. Developing an In-Service Performance Evaluation (ISPE) for Roadside Safety Features in Texas. Texas Transportation Institute, College Station. December. http:// d2dtl5nnlpfr0r.cloudfront.net/tti.tamu.edu/documents/0-4366-1.pdf. Zimmerman, K. A. 2015. NCHRP Synthesis of Highway Practice 470: Maintenance Quality Assurance Field Inspection Practices. Transportation Research Board of the National Academies, Washington, D.C. http://dx.doi.org/doi:10.17226/22201. Zimmerman, K. A., and M. Stivers. 2007. A Guide to Maintenance Condition Assessment Systems. NCHRP Project No. 20-07, Task 206. Final Report. Transportation Research Board of the National Academies, Washington, D.C. October. http://maintenance. transportation.org/Documents/Final%20Report%2020-07%20Task%20206.pdf. Zimmerman, K. A., and D. M. Testa. 2008. An Evaluation of Idaho Transportation Depart- ment Needs for Maintenance Management and Pavement Management Software Tools. Final Report. Idaho Department of Transportation, Boise. December 18. https://itd.idaho. gov/highways/research/archived/reports/RP183%20Final%20-%20MMS%20PMS%20 Needs%20Assessment.pdf.