Click for next page ( 10

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement

Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 9
9 CHAPTER 2 Status of Existing AMFs and Identification of AMF Needs This chapter includes a description of the processes used to Literature Review Process develop an initial list of important traffic engineering and ITS treatments, the process and criteria used to rate the quality of The project team then searched for evaluation reports for the AMFs discovered for these treatments, details of the AMFs each of these treatments in a variety of sources. These included that were judged to be of high or medium-high quality and the reference lists of the NCHRP Report 500 guides (6), docu- thus included in NCHRP Research Results Digest 299 (5), and ments identified through the Transportation Research Infor- the processes used to prioritize and select other treatments for mation Service (TRIS) and other resource search engines, and additional analyses and AMF development. references being reviewed in developmental work for the HSM. Given the broad scope of this project effort--traffic engineer- ing and ITS improvements--the challenge was to quickly iden- Extracting Information on Existing tify the most relevant studies for each treatment that required AMFs and Determining AMF Quality a more thorough review. Given that the focus of this effort is on AMFs, the initial screening criterion applied to each study was Determining the Treatments that the results must be founded on a crash-based analysis. to Be Considered Studies based on traffic behavior, survey results, or other Numerous treatments are used by state and local safety outcomes were eliminated from consideration. agencies in their efforts to reduce the number and severity of The studies for each treatment were further screened to crashes at intersection and non-intersection (i.e., segment) determine which ones included the development of AMFs or a locations. The NCHRP Project 17-25 team developed an methodology that could be used to develop AMFs. The studies initial list of 78 such treatments, categorized as intersection- meeting this criterion and believed to be the most credible were related, segment-related, ITS-related, other, and combined then subjected to a more critical review. Each critical review treatments. This list was based on treatments proposed in was undertaken with the following objectives: past safety guidance documents such as the NCHRP Report Evaluate the research approach and statistical methodol- 500 guides for implementation of the AASHTO Strategic ogy, including an examination of possible pitfalls such as Highway Safety Plan (6, 4) and on project team knowledge of regression to the mean or site-selection bias. state practices and knowledge of past treatment evaluations. Document the magnitude and assess the quality of any As part of this project, an AMF state-of-the-practice sur- AMFs produced. vey was sent to each of the 50 state DOTs. Working with Determine whether and how each particular study could FHWA, the project team identified the safety engineer and be used in Phase II of the research effort, in which addi- an ITS engineer in each DOT and attempted to reach them tional research on critical treatments was to be conducted. with the survey. Responses were ultimately received from The three possible uses were the following: 34 states. Respondents were asked to list additional treat- Source data for the study could be available for further ments for which an AMF is needed, which expanded the analysis using a more rigorous methodology. list of treatments to be reviewed to 113. After some further Study results (and/or source data) could be used as part screening based on the literature review process and collaps- of a meta-analysis. ing of redundant treatments, the final list included 100 Study results could be provided to an analysis-driven treatments. expert panel.

OCR for page 9
10 Levels of AMF Quality expert panel was able to utilize the results of at least one key study that was considered critical and very well done. As noted in the second point above, the output of this effort However, in other cases, the expert panel may not have been was to be not only the documentation of the AMF, the level of able to identify any studies without flaws and may have been effectiveness of the treatment, but also the development of a forced to rely on their collective knowledge, experience, and measure of the quality of the AMF--a measure of its level of judgment in combining the results of these less-valid studies. predictive certainty (LOPC). This estimate of the LOPC is a It is obvious that the AMFs developed without the results reflection of the study methodology used to define the AMF. from any critically valid studies have a lower level of predic- The LOPCs used in this review and the criteria for each are as tive certainty than the ones developed with at least one such follows: study. The criteria listed above indicate that an AMF produced High. The AMF was developed in a rigorous before-after from a rigorous meta-analysis may be considered to have a study that incorporated the current best study design and LOPC of medium-high. Many of the meta-analyses that have statistical analysis methods. At this time, the empirical been conducted include studies from multiple countries as Bayes (EB) methodology described by Hauer represents well as studies that are several decades old. There are enough the best available approach (7). The study must also have differences among the applications of treatments in North included a sufficiently large number of treatment sites, a America, Europe, and Australia to warrant caution in com- large reference group composed of comparable sites, and bining the results of these studies. There have also been enough crashes for statistical validity. enough changes in drivers and vehicles to warrant caution in Medium-High. The AMF was developed in an EB before- the inclusion of studies that are more than 25 years old. Thus, after study with limited numbers of treatment sites and/or the following criteria were used in assessing whether an AMF crashes or a before-after study that incorporated sound from a meta-analysis was deemed to be of medium-high (but not EB) statistical methods and/or may have been quality: reviewed and "vetted" by an expert panel of researchers. This level may include AMFs that were produced by an ex- A minimum of three North American studies (post-1980) pert research panel from the combination of findings from had to be included in the analysis, and the percentage of different (less-controlled) before-after and cross-sectional North American studies had to be at least 20 percent. There studies. The panel's judgment concerning the quality of the was an exception to this threshold if the treatment was AMF is reflected in the LOPC and did not always merit a believed to produce operational characteristics in the medium-high rating. This level also includes AMFs that United States that were different than the operational char- were developed in a rigorous meta-analysis by a recognized acteristics produced in other countries. For example, road meta-analysis expert. (Meta-analysis is the combination of shoulders in many European countries are designed and the results of various studies using statistical techniques used for passing maneuvers, which is completely different that allow the expert to overcome some of the shortcom- from the function of road shoulders in the United States. ings of the original research.) Not all meta-analysis results In cases in which the treatment was believed to produce warranted a medium-high LOPC (see discussion below). operational characteristics in the United States that were Medium-Low. The AMF was developed from a cross- different than the operational characteristics produced in sectional analysis (controlling for other factors statistically) other countries, the percentage of studies from the United or from less-than-rigorous before-after studies, but is still States needed to be substantially greater than 20 percent. judged to be of value. An example would be a before-after The treatment in the meta-analysis had to be clearly study in which regression to the mean was not viewed as a defined to be sure the results were applicable to the major potential bias because the population of treated sites "treatment of interest," including specifics on applicability included more than just "high-accident locations." to various classes of roadways. For example, if the treat- Low. The AMF was developed in a simple before-after ment of interest was speed limit reduction on two-lane study without control for biases or from cross-sectional rural roads, then a meta-analysis in which the only U.S. studies in which modeling techniques are questionable. studies were related to Interstates was not considered. Non-existent. No studies were found that included AMFs The results had to be statistically significant (i.e., the for this treatment. 95-percent confidence interval could not include 0). The intent here was to avoid including any AMF for which It is also important to understand that within each LOPC, the sign may change (i.e., the lower end of the confidence there can be a wide range of accuracy or confidence. For interval results in a crash decrease while the upper end example, some AMFs have been developed in cases where the results in a crash increase).