Automated Pavement Distress Collection Techniques (2004)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

22 This chapter addresses the various technologies employed to gather useful information from the data collected through auto- mated means. Modern high-speed computers and data storage devices have led to quantum leaps in the ability to deal with some of these issues in just a few years. The result is that there are at least three general classes of methods used to reduce pavement condition data to useful information. Depending on the degree of human intervention required to achieve useful results they are purely manual, semi-automated, and fully automated methods. For the most part, the manual and semi- automated methods apply only to pavement cracking and patching (image-collected data). The analysis of ride quality, rutting, and joint-faulting data collected with sensors has been largely automated. Only semi-automated and automated methods are discussed in this synthesis. PAVEMENT IMAGES Figure 11 is a generic schematic of the automated and semi- automated pavement cracking analysis now used in approx- imately 20% of highway agencies. Activities on side A of Figure 11 are performed in the data collection vehicle as the combination of cameras, lights, and DMI produce referenced images of the roadway. The images are then stored on an elec- tronic medium for either processing in the vehicle or delivery for off-line processing. Activities on side B of Figure 11 may take place in the data collection vehicle, sometimes as real-time processing, or be delivered to either agency or vendor personnel doing off-line (out of the vehicle) processing. In the case of real-time pro- cessing, one of the fully automatic procedures described later in this chapter will be used. For off-line processing any of the semi-automated or fully automated procedures described will be used. Although the schematic would apply to manual processing as well, those methods are not discussed in the synthesis. Questionnaire responses indicated that of 30 agencies using automated crack data collection, 20 used manual processing, 10 automated or semi-automated processing, and 1 some man- ual and some automated. Processing is done predominately by vendors. Only Connecticut, Maine, and Maryland use agency resources to both collect and process cracking data through automated means. All three use the WiseCrax technology described later in this chapter. Table 10 summarizes the reported processing methods in use by the various agencies. A typical workstation used in pavement distress data reduction is depicted in Figure 12. In that figure, the left-most monitor displays a summary of distress data collected from images displayed in the center monitor, while the right-most monitor gives a perspective or forward view of the pavement under review. This view provides the rater with a better means of identifying some distresses (such as skin patching) that often are not discernible in the downward view. (Note that a special keyboard has keys dedicated to types and severities of the various distresses to be evaluated, depending on user or client needs.) Semi-Automated Methods Semi-automated methods of distress data reduction include those methods in which there is significant human interven- tion. In some cases, the process is primarily manual and involves a trained rater sitting at a workstation where pave- ment images are systematically reviewed and the various dis- tresses identified and classified as to extent and severity. Such workstations are equipped with images players, integrated distress rating and location-reference software that can access image and database files, high-capacity storage devices, and one or more high-speed processors. Image viewing requirements depend on whether filmed, taped, or digital images are captured. The manual element of distress data reduction from images typically involves the use of multiple image monitors and at least one computer monitor for data display. Multiple image monitors are required to pro- vide a rater’s perspective for location purposes and to assist in identifying certain types of distress that are not readily dis- cernible in downward images. As with any means of imaging, there may be a substantial loss of resolution compared with what is visible to the human eye from the same source. Where the images were captured through photologging or videotaping, the control of film or tape progression and tying images to specific mile points can be an onerous task. For that reason, almost all image collection procedures now require that the images be date, time, and location stamped. The location will often be coordinates derived from GPS instru- mentation on the survey vehicle. This in turn means that agency roadway files (inventory) must also be tied to those coordinates. The identification of various distress types, as well as their severities and extents from images requires observers or raters CHAPTER THREE DATA PROCESSING TECHNOLOGIES

23 who have been well trained in both pavement distress evalu- ation and in the use of the workstation hardware and software. Such raters are not readily available in most agencies or in most job markets, such that almost all require extensive train- ing in at least some aspects of the process. Also, the rating process is extremely demanding, for raters must be able to coordinate the simultaneous use of several monitors while keeping track of the observed distresses and entering those observations into rating software. Cameras/Other Sensors Lights DMI Collect & Reference Pavement Images Store Images Process Image Cracking No Yes Classify, Quantify Cracks Deliver Cracking Data Select Image Automated Pavement Cracking Analysis Deliver Images A B In vehicle In vehicle or off-line FIGURE 11 Automated pavement cracking analysis. Agency Type Manual Semi-Automated Fully Automated State Province FHWA Total 16 3 1 20 1 1 — 2 7 0 1 8 TABLE 10 IMAGE PROCESSING METHODS IN USE FIGURE 12 Typical digital work station (Courtesy: ARA-ERES, Inc.).

Early efforts at reduction of distress data from images have been learning experiences for some agencies. The Virginia DOT (VDOT) found that distresses identifiable on the road- way do not necessarily correspond to those discernable from images. Cracking visible from walking or even low-speed windshield surveys may not be visible from images. Also, certain types of patching that blends well into the surround- ing pavement may not be detected from images. These find- ings led VDOT to develop a pavement distress rating manual specific to videotaped images (46). The essential difference between the videotape rating manual and that for field surveys is that fewer distress severities are defined, whereas some field-observed distresses (e.g., bleeding) simply do not occur in the videotape version. The VDOT work was performed using two systems: one a videotape system provided by Pavetech, the other provided by Transportation Management Technologies that used digital area and line scan images. The draft TRB circular (20) summarizes several systems that were used earlier, but not reported to be in use by agen- cies responding to the synthesis survey. One of these sys- tems was built in the late 1980s by the Japanese consortium Komatsu. The system consisted of a survey vehicle and an on- board data processing unit to simultaneously measure crack- ing, rutting, and longitudinal profile. Maximum resolution of 2,048 by 2,048 pixels was obtained at a speed of 10 km/h (6 mph). The Komatsu system worked only at night to control lighting conditions and represented the most sophisticated hardware technologies at that time. However, it did not pro- vide output for the types of cracking. From late 1980s to early 1990s, the Earth Technology Corporation created a research unit called Pavement Condi- tion Evaluation Services. The automated system created by that unit was the first to use line scan cameras at a 512-pixel resolution to collect pavement data. The necessary technolo- gies associated with the image capturing and processing were not mature enough at that time (20). For a time, the Swedish PAVUE pavement data acquisi- tion equipment was promoted in North America. The system includes four video cameras, a proprietary lighting system, and four S-VHS videocassette recorders. The image collec- tion subsystem is integrated into a Laser RST van. The off- line workstation is based on a set of custom-designed proces- sor boards in a cabinet, to analyze continuous pavement data from the recorded video images. Surface images are stored on S-VHS tapes in analog format. This system is no longer actively used in North America (20). The TRB draft circular further reported that in 1995 the FHWA awarded two continuing contracts to LORAL Defense Systems in Arizona, now a unit of Lockheed–Martin, to pro- vide an Automated Distress Analysis for Pavement, known as ADAPT. The data source is digitized images from PASCO’s 35-mm film. The delivered system, after completion of the projects, could not be used. 24 Several vendors currently market systems for the reduc- tion of cracking data from images. These systems are gener- ally configured as described earlier, and there are special features for determining distress severity and extent. Key differences between systems depend heavily on differences in software, almost all of which is proprietary. The degree of human intervention varies somewhat from system to system. The Pavement Distress Analysis System (PADIAS) used by CGH Engineering and applied to the LTPP program projects a high-resolution image onto a digitizing tablet where it is viewed by an operator (22). In using PADIAS, the operator uses a mouse and pop-up menus to select distress type and severity level for any distress viewed on the screen. The PADIAS data files are convertible into ASCII format for uploading into an agency’s existing database. Although the distresses built into the system are those used in the LTPP program, they are capa- ble of modification to meet other user needs. A second system is provided by Pathway Services and is known as the Pathview I: Video and Sensor Playback System (47 ). The system will support up to six videotape players and monitors. Examples of digital images of each distress and their severity levels are available for review to maintain con- sistency between raters and for QC purposes. The software provides data entry fields used to input the pavement distress features to be evaluated. The system also reads the number of the image being evaluated, as well as location-reference information, and it displays those data on the computer mon- itor. As the analysis is completed, a database with the distress features is created. A semi-automated system requiring somewhat less human intervention is the D-Rate Digital Distress Rating System (32). This system operates on digital images such that the “operator can locate, classify, and determine severity and extent of dis- tresses by using a mouse to draw boxes around distress areas and selecting distress type from a menu.” Then, the computer calculates lateral and longitudinal coordinates of the distresses as well as length and area of distress, and it automatically enters these into a database. The program also incorporates other computer-aided features, one of which is the definition of wheel-path areas with respect to the distresses. This feature aids in the definition of load-related distresses such as fatigue cracking. Another system requiring less intervention is the Roadview GDPlot (26). The system uses a 3-D digital imaging process. The system generates 3-D images of the pavement surface by combining the plots of successive laser scans. The vendor asserts that 3-mm (0.125-in.) cracks are clearly visible in 4.2-m (14-ft)-wide images providing continuous full-lane cov- erage of the pavement. The system is not considered to be fully automated, because through early 2003 the vendor did not address features allowing distress summaries to be developed and reported. No independent evaluation of this system has been reported.

25 A final semi-automatic system is described by Miller et al. (48), as applied to transverse cracking by the Kansas DOT. The system uses ICC Digital Imaging and Distress Measure- ment Analysis Software. The researchers reported that a 2,048-pixel digital line scan camera, a computerized con- troller, and an illumination system allowed cracks as fine as 1 mm (0.04 in.) wide to be recorded at speeds up to 96 km/h (60 mph). Comparisons of pixel grayscale ratings to pre- determined threshold values permitted a pixel to be consid- ered a crack pixel when its gray value was less than the thresh- old gray value. Manual intervention was used to determine crack severity and extent from the Kansas DOT’s algorithms. A comparison of distress data from roadway observations with that derived through semi-automated systems and from images is given in chapter six. Fully Automated Methods In the context of this synthesis, the definition of fully auto- mated is that distresses are identified and quantified through processes that require either no or very minimal human involvement. Typically, fully automated in the context of pavement cracking analysis involves the use of digital recog- nition software capable of recognizing and quantifying vari- ations in grayscale that relate to striations (or cracks) on a pavement surface. It is in these fully automated methods of distress data reduction from images that the greatest amount of research and development work seems to have occurred over the past decade. That emphasis is no doubt the result of the difficul- ties involved in manual reduction of these data as described earlier and related to resources required to accomplish the manual tasks encountered. WiseCrax From the survey, the most widely reported of the automated methods is that known as WiseCrax (32). Of 30 agencies indi- cating automated surface distress data collection, 8 reported that they own or contract for the use of processing equipment provided by the WiseCrax vendor. The vendor describes the system in this way: WiseCrax processes the pavement image video tapes from the ARAN. It will automatically detect cracks (length, width, area, orientation); classify them according to type, severity, and extent; and generate summary statistics and crack maps. In the early WiseCrax system, pavement surface images were collected with two continuous video cameras, covering the survey lane of approximately 4 m (13 ft). The video images were recorded into S-VHS format. Each camera is approxi- mately 2.4 m (8 ft) above the pavement surface and covers a 2-m (6.5-ft) wide area. Captured images have the resolution of 640 by 480 pixels after digitization to grayscale images. Since 2000, the vendor has captured digital images directly, making the analog to digital step no longer necessary. The other param- eters are similar for the new system (32). A typical digital pavement image is given in Figure 13. For crack detection, an initial setup is required where the workstation operator selects images used to determine an opti- mum set of detection parameters accounting for pixel-by-pixel grayscale variation as related to crack contrast, brightness, and surface conditions. During this setup phase, the program pro- vides visual feedback of the detection results in the form of crack maps traced over the underlying images of control pave- ments. These crack maps in turn provide instant feedback on the efficiencies of the parameters. Through an iterative process, the optimal detection parameters are selected for each control pavement. Once the settings are selected, WiseCrax is programmed to automatically process the pavement images to detect cracks. The beginning and end of each crack are loca- tion referenced using an x–y coordinate system. For each crack, the length, width, and orientation are also computed and saved. An example is a digital crack map as shown in Fig- ure 14 from the digital image given in Figure 13. Wang and Elliott (15) describe the WiseCrax crack clas- sification process as the following: Since the definitions of distress categories vary from agency to agency, WiseCrax compares the location, length, and width of cracks against (agency) criteria for various crack distress cate- gories. For instance, if cracks in a block pattern are more than 300 mm (12 in.) apart it may be classified as block cracking. If they are closer together, it may be classified as fatigue cracking. WiseCrax has the flexibility to process data as new classification definitions are developed. WiseCrax operates in two modes: automated and interactive. In automated mode, all processing is done without human inter- vention, once the initialization parameters on pavement type, camera and light settings, etc., are set. Interactive mode allows the user to review, validate, and edit the WiseCrax results. For instance, the automated mode can be run first; the display shows the pavement image with overlaid color lines indicating the pres- ence of cracks. The user can then point-and-click to add, delete, or modify the results. For quality control purposes, the inter- FIGURE 13 Typical digital pavement image (32).

active mode is normally used to perform statistical validation of automated results using random samples of data. The vendor, Roadware Group, Inc., has noted several lim- itations of the WiseCrax technology. First, all digital image analysis is limited by the quality and resolution of the images. Then, the minimum crack width that can be automatically detected by WiseCrax is approximately 3 mm (0.125 in.) or approximately 1 pixel wide. The vendor goes on to note that finer cracks may be detected manually from the same images, because the human eye can perceive finer crack lines than the image can clearly display. For this reason, cracks with non- uniform widths may be identified as several shorter cracks. Finally, certain types of pavement surface, chip seals, for example, provide poor crack visibility, as does crack sealing material. Such features typically need to be evaluated in the interactive mode, because the automated process is unable to discriminate those features without human intervention. Accuracy of the WiseCrax system is determined by sam- pling sections of roads and manually reviewing the output of the automatic processing program. WiseCrax output is com- pared with trained observers’ reviews of sampled videos. A percent accuracy is calculated from the ratio of the cracks found by WiseCrax to those found by the trained observers. In 1999, Wang and Elliott (15) conducted an evaluation of the WiseCrax system for the Arkansas Highway and Transportation Department. The objectives of that study were to evaluate both the vendor’s data collection system and WiseCrax, and to make appropriate recommendations to the agency about the capabilities and performance of WiseCrax. In that study, the comparison of data between the results from WiseCrax and the results from manual surveys demonstrated that there are still large differences between them. The authors continued that the automated system had no diffi- culty in finding cracks. The problem was the classification and quantification of the cracks. They further noted “the problem was not vendor specific and has been a research topic for years.” 26 In a later study, Groeger et al. (49) reviewed the imple- mentation of the WiseCrax system in Maryland (see case study in chapter eight). Their conclusion was that “automated network level crack detection is a viable and efficient tool. However, a strict QC/QA regime must be instituted in order to achieve consistent and repeatable results.” Digital Highway Data Vehicle In 2001, Wang et al. (50) reported on the University of Arkansas Digital Highway Data Vehicle. (The vehicle was also developed by the principal author.) The study was of distress data digitally captured and reduced in real time on a 4.5-km (2.8-mi) section of highway divided into 0.16-km (0.1-mi) subsections. The authors described the system as follows: • The vehicle is based on a full-digital design. It does not use any type of analog medium for data storage. The operating software environment is based on 32-bit tech- nology. • The vehicle includes a subsystem for pavement surface image collection. This subsystem has one frame-based digital camera and four strobe lights for illumination. • The vehicle has an automated survey subsystem that can be integrated into the data acquisition subsystem to iden- tify and classify pavement cracks at real time. • The vehicle acquires the exact location of itself through the use of a GPS device and a DMI, and it saves the data to the computer’s database. • The software system used in the on-board computers of the vehicle employs a real-time relational database engine, intercomputer communication techniques, multi- computer and multi-CPU (control processing unit)- based parallel computing, and generates multimedia databases. In regard to that study, it is important to note that driving speeds were 32 to 64 km/h (20 to 40 mph) and that imaging FIGURE 14 Example crack map (32).

27 was done at night with a van equipped to illuminate the roadway to ensure the best quality images. Distresses were classified in accordance with three different protocols: the AASHTO provisional standard (17), the World Bank Uni- versal Cracking Index (51), and the Texas DOT method (52). The emphasis was on real-time processing of the digital images to detect, classify, and quantify surface distresses. The 28 pavement subsections were each evaluated once with all three protocols. In addition, for the Universal Cracking Index protocol, a total of four runs were conducted to secure a measure of process repeatability. The results showed similar distributions of cracking extent for all three protocols. All three methods were deemed to clearly show the same problem spots on the pavement section and the same associated severity levels. Multiple runs on the same section yielded a coefficient of variation of approxi- mately 15%, suggesting reasonable repeatability. The authors concluded that the “solution to the problem of automating dis- tress survey is finally at hand” (50). They conceded that the system is not perfect and that there is more work to be done. The authors further commented that “newer versions of the crack analyzer have already demonstrated better accuracy.” Results of their applications will be published at a later time (50). In 2003, Wang reported, “With an acquisition system with twice the resolution and further improvement of imaging algorithms, such as more accurately determining lane mark- ings on both sides, it is anticipated that in the next few years widespread use of fully automated crack survey systems will be a reality” (53). Finally, it was recently reported (K. Wang, personal communication, August 2003) that an imaging sys- tem at a 4,096-pixel resolution could detect coarse surfaces. Automation for that type of survey is not “there yet,” but it can be done. GIE GIECRK GIE Technologies applies an “automated” surface distress analysis package in at least two agencies (Manitoba and Rhode Island). Information on this technology is limited, with no published user evaluations yet discovered. The GIE website description states only that “the data is digitised, synchro- nized, and correlated so as to allow an automatic analysis and diagnosis free from subjective interpretation and human error” (54). IMS-Terracon uses a laser road surface tester designed and developed by the company’s own engineers. The unit uses a video technology to capture and analyze pavement surface condition information in real time (55). The com- pany’s PAVUE technology uses advanced camera technol- ogy and high-speed image processing to automatically col- lect and assess pavement distress data. The company website relates the following: Four downward looking cameras, in conjunction with a strobe lighting system that provides consistent illumination, collects con- tinuous, high clarity pavement images that are analyzed using a patented process to determine and assess the pavement surface distresses (55). The company keeps its procedures somewhat confidential and there is no known independent published evaluation. Although a news release addressed work in New Mexico by IMS-Terracon using a 3-D technology (56 ), the pavement distress aspects of that work have not been reviewed by the agency’s DOT personnel. The state pavement management engineer (R. Young, New Mexico State Highway and Trans- portation Department, personal communication, August 2003) indicated that the vendor was collecting different dis- tresses from those used in the state’s pavement management program and that there are no current plans to use the ven- dor’s data. SENSOR-MEASURED DATA For automated data collection equipment, the processing of sensor-measured data is almost all real time and done in accordance with the collection protocols employed. Process- ing basically involves the analysis of longitudinal and trans- verse profiles and the extraction of key information from those analyses. The principal products are the IRI, rut-depth measurements, and joint-faulting measurements. Because each of these products is described fully in the data collec- tion and QA chapters (chapters two and six), they are not addressed further here. SUMMARY Although the automated collection and processing of pave- ment distress data have progressed greatly in the last decade, there still are barriers to overcome before the technologies involved can come to fruition as real-time, reliable, and gen- erally applicable tools. First is the need for development of systems capable of consistently producing high-quality digi- tal images under most data collection conditions (lighting, angle of the sun, shadowing, etc.). Although there is evidence that the technologies have progressed to the needed capabil- ity, they are not generally applied within the industry. Once good images are consistently produced, greater progress can be made in the second major problem area: that of improving the quality of data automatically reduced from those images and the speed with which data can be acquired. Again, there is strong evidence that the necessary technologies exist, but they seem to need further maturing to address both quality and speed. There may be a need for a focused effort to bring about that maturity.