Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
5 Conclusions and Recommendations The committee was asked to develop a research design for evaluating the in- service crash performance of guardrail end treatments, determine the data required to carry out the analysis, examine data systems in selected states to determine whether the required data would be available, and identify appropriate next steps for carrying out such evaluations. Defining the objectives of evaluation in terms of the intended applica- tions of the evaluation results is the necessary first step in developing an evaluation method. The evaluation can then be designed to produce the results required for those applications. No single evaluation study design will be equally suitable for all objectives. Therefore, the committee began its work by examining the need for evaluation and identifying ways in which evaluation might lead to improved road and roadside safety design and management practices. The first section below presents the committeeâs conclusions on objectives and applications of in-service evaluation of end treatments and other roadside safety devices and on current capabilities and evaluation methods for conducting in-service evaluations. The second section presents the committeeâs recommendations for next steps. CONCLUSIONS The committeeâs conclusions on the following topics are summarized below: ⢠Need for in-service evaluation, ⢠Highway agency readiness for conducting evaluations, ⢠Initial steps toward the conduct of evaluations, 104
CONCLUSIONS AND RECOMMENDATIONS 105 ⢠Evaluation of roadside safety devices other than guardrail end treatments, ⢠Evaluation objectives, and ⢠Evaluation methods. Need for In-Service Evaluation In-service evaluation can help to ensure the effectiveness of roadside safety devices and reduce the risk of road injuries and fatalities. The American Association of State Highway and Transportation Officials (AASHTO) Manual for Assessing Safety Hardware (MASH), which specifies the meth- ods of the crash tests that state highway agencies rely on to determine the suitability of roadside safety devices, acknowledges that crash testing can- not provide full information about the performance of the devices in the field (AASHTO 2009, 3, 111). Testing cannot reproduce the variety of char- acteristics of crash dynamics, sites, and installations that may affect crash outcomes. The Joint AASHTO-FHWA Task Force on Guardrail Terminal Crash Analysis (2015) documented the occurrence of impact conditions and site and installation characteristics that differ significantly from those of the end treatment crash tests. In-service data are necessary to determine the frequency of various crash, installation, and site characteristics. Information on frequency is necessary to determine the crash conditions that should be included in the crash tests. In-service evaluation also is necessary to verify that devices in the field perform as they do in testing under equivalent crash conditions. A program of periodic evaluation is needed because vehicle and traffic characteristics and road conditions change over time, new device designs frequently come into use, and devices installed may deteriorate over time. Validation and refinement of crash testing would be appropriate objectives of a nationally coordinated program of in-service evaluation research. Highway Agency Readiness for Conducting Evaluations Highway agencies are not prepared to commit resources to systematic in- service evaluation of roadside safety devices. Interviews with administrators and engineers from 10 state highway agencies that were conducted for the committee identified one agency that is in the process of developing the capability for performing in-service evaluation of roadside devices and a second agency that is participating in the Federal Highway Administration (FHWA) pilot in-service evaluation of guardrail end treatments begun in 2016. The remaining eight agencies reported no in-service evaluation activi- ties for roadside devices either under way or planned. These observations
106 PERFORMANCE EVALUATION OF GUARDRAIL END TREATMENTS are consistent with the conclusions of past investigations and with the scarcity of published reports of highway agency evaluations. Reasons for not undertaking evaluations cited in the interviews were as follows: ⢠Limitations of agency data systems (e.g., the systems do not sup- port linking of crash, maintenance, and asset inventory records; no record is made of the postcrash condition of devices), ⢠Difficulties of coordination and communication (in particular, lack of arrangements with police agencies for the timely receipt of crash reports), ⢠Lack of funding and staff for data collection and analysis, and ⢠Lack of perceived benefit from in-service evaluation. In general, agencies appear not to regard the performance of roadside safety devices to be an area of major safety deficiency, compared with other safety priorities. Initial Steps Toward Conduct of Evaluations Planning and trials to demonstrate benefits and methods involving the high- way agencies that will use the results of evaluations will be a necessary first step toward establishing the capability to conduct in-service evaluations. More information is needed about the potential benefits and practicality of in-service evaluation before ambitious new data collection and analysis programs can be launched. A highway agency evaluation program such as the procedure outlined in the MASH will have costs, including time demands on maintenance and engineering staff, and may divert resources from other safety activities. If the highway agency does not recognize a persuasive need for in-service evaluation, then the activity cannot compete in the budget process and administrators will not be motivated to develop the necessary inter- and intra-agency arrangements. A highway agency considering undertaking an evaluation program will require a reasonable expectation of benefits at the outset and will expect the activity to prove its worth after it has been in operation for a given period. Also, a comprehensive comparative study of the effects on injury risk of alternative device designs and of installation and maintenance practices, such as the evaluation recommended by the AASHTO-FHWA Task Force, would have a greater chance of success if it were approached in stages. In the initial stage, studies of modest scale would develop data collection and modeling methods. The initial results would indicate whether the potential benefits justified further research investment.
CONCLUSIONS AND RECOMMENDATIONS 107 Evaluation of Roadside Safety Devices Other Than Guardrail End Treatments In-service evaluation is equally justified for all roadside safety devices now evaluated through crash testing. Uncertainty about the reliability of crash testing as an indicator of performance in service applies equally to any of the roadside devices for which the MASH provides a testing procedure, including longitudinal barriers, crash cushions, guardrail end treatments, and support structures such as sign supports, mailbox supports, and lumi- naire supports. Moreover, the cost and complexity of evaluation are unlikely to increase proportionately with the number of device types covered. For example, the cost of putting in place a procedure for maintenance crews to record the condition of end treatments and guardrails repaired following a crash may be only modestly greater than the cost of collecting postcrash data for end treatments alone. In general, any special data collection or data systems established for evaluation purposes are more likely to be cost-effective if the data have applications for a range of evaluation and management needs. The committeeâs statement of task refers only to the evaluation of guardrail end treatments. Similarly, the FHWA pilot in-service evaluation and the AASHTO-FHWA Task Force have examined end treatments exclu- sively. This attention to end treatments is a consequence of the controversy described in Chapter 1 regarding the safety of a particular end treatment design. However, the crash frequency data presented in Chapter 1 show that severe collisions with end treatments are a small fraction of all severe collisions with roadside safety devices and that methods suitable for a single category of devices also would be applicable for evaluating the safety per- formance of other kinds of roadside devices. Evaluation Objectives The committee reviewed the recommendations regarding in-service eval- uation of roadside safety devices in the MASH, the AASHTO-FHWA Task Force investigation of guardrail end treatment performance (Joint AASHTO-FHWA Task Force on Guardrail Terminal Crash Analysis 2015), and in-service evaluations carried out by highway agencies. These past ac- tivities indicate a perceived need for the capability to conduct two kinds of in-service evaluations with distinct objectives: 1. A nationally coordinated evaluation research program to respond to special problems, such as the recent concern about the adequacy of certain end treatment designs and to meet common needs of the states, and
108 PERFORMANCE EVALUATION OF GUARDRAIL END TREATMENTS 2. Routine state highway agency in-service evaluation of roadside devices for applications such as monitoring the agencyâs inventory of roadside devices to ensure proper conditions, planning cost- effective maintenance and replacement, and checking that newly adopted devices are performing as expected. Evaluation objectives that would be appropriate within the scope of a nationally coordinated program of evaluation research include the following: ⢠Validating and refining crash test procedures to improve the reli- ability of testing as an indicator of the performance of the device in use. In-service evaluation to validate testing also would provide data to support improvement of the designs of end treatments and other roadside devices. ⢠Demonstrating methods that state highway agencies could use for routine in-service evaluation. In addition to providing interested agencies with a method, such demonstrations would test the useful- ness of in-service evaluation in highway management. ⢠Evaluating the effects of design, installation and maintenance qual- ity, and deterioration in use on the performance of roadside devices to help states define cost-effective practices for selection, quality control, inspection, and replacement of devices. These nationally coordinated evaluations would be used as research to acquire new knowledge about the safety performance of roadside devices or to develop new methods of evaluation. In contrast, a highway agency evalu- ation program would be a management support function that would pro- vide information to guide decisions about selection and maintenance of the devices. Routine highway agency in-service evaluation could have at least three applications for improving the performance of highway programs: ⢠Providing notice that a device type is not performing as expected. Evaluation would give the highway agency assurance that any device type that was susceptible to failure in circumstances not considered in design and testing would be identified, even if fail- ures were rare. The agency would monitor performance of newly adopted device types and also of the types in general use at the time the evaluation program was initiated. ⢠Long-term monitoring to identify cost-effective practices regarding the selection and maintenance of roadside safety devices. Even if all alternative devices perform acceptably (i.e., as intended by the designer), some device types may be more effective than others
CONCLUSIONS AND RECOMMENDATIONS 109 in mitigating crash severity or have advantages with respect to life-cycle costs or ease of installation and maintenance. The best- performing device type may depend on the characteristics of the location. ⢠Ensuring that the roadside safety devices on the agencyâs roads are appropriate for their locations and have been properly installed and maintained. Because of the lack of highway agency experience in conducting evalua- tions, the practicality, appropriate scale, and utility of such routine evalua- tion activities have yet to be determined. Evaluation Methods A program of in-service evaluation of roadside safety devices (whether a national program of evaluation research or routine evaluation by a state highway agency) will have three essential components: ⢠An administrative and planning structure. This structure defines evaluation objectives, the scope of the evaluation, and the respon- sibilities for the evaluation and oversees the application of results. ⢠Data systems and procedures. The content, reliability, and timeli- ness of the agencyâs general-purpose databases for recording road- way and traffic characteristics, maintenance activities, and crashes must be adequate to support the objectives of the evaluations. In-service evaluation will be most practical if it relies primarily on established data systems. ⢠Evaluation methodologies. Measures of performance and data analysis methods for computing the measures must be specified. The experience of past studies has shown that the principal technical obstacles to evaluation are as follows: obtaining prompt notification of crashes so that data may be obtained at the site, obtaining notification of crashes not reported to police, obtaining a sample of crashes large enough to reveal infrequent failures and to allow inferences about deter- minants of severity, and ensuring the reliability of data. An evaluation methodology must include procedures for overcoming these obstacles. Effective notification and data quality assurance require cooperative re- lationships between the agency staff responsible for the evaluation and maintenance staff and between the evaluators and police. Past activities have laid the groundwork for a methodology for evalua- tions, especially NCHRP Report 490 (Ray et al. 2003), the AASHTO-FHWA Task Force inspections of end treatment installations and investigations of
110 PERFORMANCE EVALUATION OF GUARDRAIL END TREATMENTS end treatment crashes (Joint AASHTO-FHWA Task Force on Guardrail Terminal Crash Analysis 2015), the FHWA pilot end treatment in-service evaluation begun in 2016 (FHWA n.d.), and trials of in-service evaluation conducted in several states. Chapter 3 summarizes the methods developed. These past studies will be especially useful models in future evaluations for their definitions of required data elements and data-coding methods and for their demonstrations of notification and communication arrangements. The past studies have been less successful at demonstrating statistical meth- ods of controlling for confounding factors (i.e., the influence on crash sever- ity of factors other than the design and condition of the roadside device, such as road and traffic characteristics), methods of selecting scientifically valid samples, and auditing data quality. RECOMMENDATIONS The recommendations below call for a nationally coordinated research program made up of three projects: ⢠Validation of the crash tests now used to assess the performance of safety devices by comparing actual crash conditions with test conditions and actual crash outcomes with test results, ⢠Demonstration of routine in-service evaluation methods for use by state highway agencies, and ⢠Exploratory data analysis toward the development of statistical models of the effects of device design, installation, maintenance, and site characteristics on the risk of casualties in collisions with roadside safety devices. The research project to validate crash tests through in-service data col- lection would be practical to carry out and would provide highway agencies with valuable assurance that testing can reliably predict the performance of roadside safety devices. The second project, the demonstration of routine highway agency evaluation methods, would be a trial to determine the costs and benefits of evaluation. Highway agencies could use the results of the demonstration to decide whether such evaluations could provide useful information at a reasonable cost. The third project, a modeling approach to determine the effects of device design and other factors on casualty risk, also would be a trial using only data collected for the validation project to determine whether more in-depth research on the topic would be worthwhile.
CONCLUSIONS AND RECOMMENDATIONS 111 Validation of Crash Test Procedures The U.S. Department of Transportation (U.S. DOT) and the state depart- ments of transportation should cooperate in undertaking a research pro- gram to validate and refine crash testing of guardrail end treatments and other roadside safety devices through in-service evaluation and simulation modeling. Without validation of crash testing by comparing test conditions and results with the conditions and outcomes of actual crashes, highway agencies lack assurance that the tests reliably predict the performance of roadside safety devices. In addition, crash tests alone cannot provide all the information needed for identifying improvements in device design and use that would reduce severe injury risk. Validation of the crash testing of these devices is a responsibility of the U.S. DOT because of the departmentâs use of crash test results to determine the eligibility of devices for reimbursement in the Federal-Aid Highway Program. It is a state responsibility because the states are responsible for the performance of these devices on their roads. Evaluation to validate or improve crash testing of a roadside safety device will require assembling a database of crashes involving the device. The database should have the following properties: ⢠The sample of crashes is representative of the population of crashes. ⢠The sample is large enough that rare events can be observed and the frequency of crash characteristics can be estimated. ⢠Information about the crash scenario, crash site environment, and pre- and postcrash conditions of the roadside device is sufficient for comparing the circumstances of the crash with the characteristics of tests and the outcome with the outcomes of test crashes. The procedures of the FHWA pilot in-service performance evaluation of guardrail end treatments, including the methods of identifying crashes and coding information, should be the model for data collection in future evaluation research for this purpose. The results of the FHWA pilot will be useful for identifying refinements in data collection; for example, the pilot results may indicate that the data elements collected for each crash can be simplified without loss of information essential to the evaluation. The recommended evaluation research program should improve on two aspects of the methods of the FHWA pilot and other recent end treatment evaluations: ⢠The FHWA pilot evaluation (FHWA n.d.) and the AASHTO- FHWA Task Force crash investigations (Joint AASHTO-FHWA Task Force on Guardrail Terminal Crash Analysis 2015) assembled
112 PERFORMANCE EVALUATION OF GUARDRAIL END TREATMENTS a set of crash records from whatever sources were readily available, without consideration of representativeness. This approach was reasonable for the exploratory objectives of these activities but is not satisfactory for producing credible quantitative estimates of frequency and outcomes of various categories of crashes. The rec- ommended research program should assemble a scientifically valid sample of crashes for analysis. ⢠The performed-as-intended evaluation criterion applied in the AASHTO-FHWA Task Force investigation and some other past evaluations is problematic if used as the sole criterion, for three reasons. First, in practice, applying the criterion has sometimes relied on a judgment on the part of the analyst, rather than on objective measures of performance. Second, two devices may both perform as intended even though one provides greater reduction in risk than the other. Finally, the criterion gives no indication of whether test conditions adequately represent real-world crash con- ditions. The recommended research program should apply objec- tive and quantitative performance measures, including the severity distribution of crash outcomes. The analysis for validating crash test procedures should have two components: ⢠A comparison of actual crash circumstances with circumstances in the tests with respect to vehicle characteristics, speeds, and trajec- tories; occupant characteristics; and roadside device characteristics and installation features. ⢠A comparison of the outcomes of actual crashes that match test circumstances with the test outcomes to determine whether the tests predict real-world outcomes under similar conditions. The evaluation should estimate the frequency of each defined category of crash condition (e.g., the three impact conditions and three installation conditions that the AASHTO-FHWA Task Force identified as performance limitations for guardrail end treatments, as well as crash conditions that correspond to test conditions specified in the MASH) and the increase in injury risk associated with each condition. If the analysis confirms the AASHTO-FHWA Task Force conclusion that real-world impact conditions can vary widely from crash test assump- tions and also finds that significant numbers of casualties occur in crashes outside the range of test conditions, then the research program should identify changes in testing procedures to more accurately predict the safety performance of installed devices and to provide guidance on improving
CONCLUSIONS AND RECOMMENDATIONS 113 designs to reduce casualty risk. Similarly, if it is found that outcomes of actual crashes matching test circumstances differ from test outcomes, then the research program should identify changes in testing procedures to im- prove agreement. Assessment of Simulation Modeling The crash test validation research program should include the assessment of the usefulness of simulation models in conjunction with crash testing to certify new device designs. In-service evaluation findings and crash test results can support the specification and validation of simulation models of vehicle collisions with roadside safety devices. The research should ex- amine the application of models for evaluating device performance in crash circumstances that would be impractical to test physically and models for evaluating the effects on performance of site details such as placement of the device with respect to the roadway, slopes, curbs, and soil conditions. In addition, the program should assess the usefulness of simulation as an adjunct to in-service evaluation, as an aid in predicting the severity of infre- quent crash scenarios, and in interpreting the contributions of device design and site characteristics to outcomes of observed crashes. Demonstration of Evaluation Methods for Routine Highway Agency Use The state departments of transportation, or the U.S. DOT and states act- ing cooperatively, should conduct a demonstration of methods suitable for highway agencies to use for routine in-service evaluation of guardrail end treatments and other roadside safety devices. The purposes of the demon- stration would be to test practical methods and to determine whether evalu- ation results can be useful for improving the safety and cost-effectiveness of state highway programs. The results would be used to revise the guidance on in-service evaluation in the MASH and for preparation of manuals and training materials. Highway agencies would be recruited to participate voluntarily. The na- tionally coordinated program would provide technical support and coordi- nation. Participating agencies would independently test methods with their own personnel and resources and assess the utility of results. Participants would be agencies with maintenance management systems. Each agency would prepare a plan for data collection and analysis and for application of evaluation results in maintenance, construction, and safety management decisions. As outlined in Chapter 4, data collection, database structures, and analysis processes would be integrated with the statesâ existing maintenance management systems, although enhancements to existing data collection would be needed. Chapter 4 also cites appli-
114 PERFORMANCE EVALUATION OF GUARDRAIL END TREATMENTS cable alternatives for procedures for state-level data collection and analysis derived from the methods of NCHRP Report 490 (Ray et al. 2003), the FHWA pilot evaluation (FHWA n.d.), and procedures developed for indi- vidual states. It would be desirable for the demonstrations to include a trial of new technology for data capture, for example, electronic labeling of devices to simplify inventory data collection and to identify devices involved in crashes. A related kind of innovative approach to data collection and moni- toring would be to introduce contractual provisions giving the manufac- turer or installer of the devices responsibility for providing inventory data and for maintaining records of device performance. The demonstration should include an analysis of each highway agencyâs costs and staff time requirements for data collection and evaluation. Also, the benefits of evaluations should be characterized in terms of documented impacts on agency decisions and practices, including the scheduling of maintenance, specifications regarding roadside devices in construction plans, and programming of safety enhancements. Evaluating Effects of Design, Installation, and Maintenance Practices on Performance The U.S. DOT and the state departments of transportation should begin exploratory data analysis toward the development of a statistical modeling approach to measuring the effects of device design, installation, mainte- nance, and site characteristics on the performance of guardrail end treat- ments and other roadside safety devices. The ultimate product of this research would constitute the complete in-service evaluation recommended by the AASHTO-FHWA Task Force. The results could support highway agency decisions regarding the selection of device types to be installed at particular locations and regarding priorities for maintenance and replace- ment of the devices. Chapter 3 identified alternative study designs for this application. Esti- mating a multivariate model of crash severity would be necessary to achieve the complete evaluation proposed by the AASHTO-FHWA Task Force. Developing such a complete crash severity model will be a challenging task because of the complexity of crash events, the variety of factors affect- ing severity that must be taken into account, and the practical difficulties of data collection. Moreover, it is not evident that substantial safety gains are possible from the information that a successful crash severity model would provide (although a finding, for example, that replacement of existing de- vices would not yield large benefits would help highway administrators to avoid unproductive investments). The recommended exploratory analysis
CONCLUSIONS AND RECOMMENDATIONS 115 would provide a basis for deciding the appropriate scale and direction of future research on the problem. The exploratory analysis should begin with data collected for the crash test validation study recommended above. The data can be used to explore alternative specifications of risk models. The results of the crash test valida- tion study will provide insights into the relative importance of the effects of design with respect to effects of installation and condition that will aid in specifying the risk model. The exploratory analysis should include an assessment of potentially useful techniques for controlling for confounding factors in comparisons of relative effectiveness of alternative device types and for drawing inferences from small samples, including ⢠Case-control study design, ⢠Application of Bayesian statistical methods for comparison of the relative risks of alternative roadside safety devices or installations, and ⢠Application of available calibrated models that predict crash risk as a function of road characteristics (e.g., crash modification fac- tors) as a means of controlling for road and traffic characteristics in evaluations. Research in progress is developing run-off-road crash prediction models that may be applicable for this purpose (Persaud 2016, 5). The modeling approach to in-service evaluation of roadside safety devices should be seen as a component of longstanding efforts to develop severity risk and crash risk models aimed at guiding roadway and roadside design in general. Finally, a part of this research activity should be to develop a method of estimating the costs and benefits of a highway agency investment pro- gram to replace existing devices (according to rules for establishing priority of replacement) with devices that have been shown to produce a greater reduction in injury risk. Organization of the Nationally Coordinated Evaluation Research Program Each of the recommended projects could be carried out independently; however, the most efficient arrangement probably would be to conduct all related activities within a single administrative structure. The U.S. DOT and the states should consider at least two alternative organizational forms for planning and oversight of the nationally coordinated evaluation research
116 PERFORMANCE EVALUATION OF GUARDRAIL END TREATMENTS program: an extension of the charge and term of the AASHTO-FHWA Task Force on Guardrail Terminal Crash Analysis as a means of coordinat- ing state and federal interests and resources and an AASHTO-led effort conducted through the National Cooperative Highway Research Program. The entity overseeing the evaluation program should first develop a plan that defines the objectives of evaluations (i.e., how the results will be applied in the management of the highway system), the scope (e.g., which devices are to be evaluated), funding needs, and schedules. The entity also will be responsible for obtaining cooperation of the federal, state, and local agencies and offices that would be involved and for monitoring the conduct of evaluations and applications of results. Recruiting state cooperation will be a major effort that will depend on effective promotion of the activity and will depend on the funding arrangements. The organizational structure should provide for an independent expert review of plans, research meth- ods, and results. The most practical arrangement for the data collection required for the validation of crash testing probably would be a cooperative multistate ac- tivity. The entity coordinating the evaluation program would recruit a small number of state highway agencies (possibly four to eight) that each would agree to monitor crashes on a portion of its road system over a specified time period. The coordinating entity would oversee the selection of road segments and data collection procedures to assure consistency with the re- quirements of the crash validation study database. This multistate approach would limit the burden placed on any one highway agency, economize on data cost by limiting data collection to a small number of geographic areas, and provide confidence that evaluation results were nationally applicable. Planning and organizing an in-service evaluation program devoted solely to guardrail end treatments, which are involved in only a small share of serious run-off-road crashes, would be difficult to justify. A more cost- effective activity would be to identify in-service evaluation needs covering at least all kinds of permanently installed roadside safety devices that are crash-tested following the procedures of the MASH. REFERENCES Abbreviations AASHTO American Association of State Highway and Transportation Officials FHWA Federal Highway Administration
CONCLUSIONS AND RECOMMENDATIONS 117 AASHTO. 2009. Manual for Assessing Safety Hardware 2009. AASHTO, Washington, D.C. AASHTO-FHWA Measurement Task Force. 2015. AASHTO-FHWA Task Force on ET-Plus 4â Dimensions. March 11. https://www.fhwa.dot.gov/guardrailsafety/dimensionsreport. pdf. FHWA. n.d. Data Collection: In-Service Performance Evaluation of Guardrail End Terminals. Joint AASHTO-FHWA Task Force on Guardrail Terminal Crash Analysis. 2015. Safety Analysis of Extruding W-Beam Guardrail Terminal Crashes. https://www.fhwa.dot.gov/ guardrailsafety/safetyanalysis. Persaud, B. 2016. Critical Review of Methodologies for Evaluating In-Use Safety Performance of Guardrail End Treatments and Other Roadside Treatments. Background Paper for Special Report 323: In-Service Performance Evaluation of Guardrail End Treatments. Transportation Research Board, Washington, D.C. Ray, M. H., J. Weir, and J. Hopp. 2003. NCHRP Report 490: In-Service Performance Evalu- ation of Traffic Barriers. Transportation Research Board of the National Academies, Washington, D.C.
