Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
1This report describes the design and development of the Site Observer, an automated site-based video system for capturing and analyzing vehicle trajectories for the purpose of highway safety research. Many highway safety problems currently are unsupported in terms of high-quality objective data that adequately describe real-world traffic conflicts. Although much effort has been devoted to instrumenting vehicles for naturalistic driving studies, the vehicle-based approach has several limitations, particularly for addressing site-specific safety questions. Some examples of limitations are (a) monitoring sites with specific geometries of interest, (b) evaluating the effects of site-specific countermeasures, and (c) addressing conflicts and crashes that result from the kinematics of several vehicles simultaneously (e.g., path crossing at intersections). In such cases, it is more efficient and effective to use fixed sensors and data acquisition strategically situated at a site of interest, provided they are capable of measuring the continuous positions, speeds, and accelerations of all relevant vehicles passing through the site. The purpose of the Site Observer system is to fill this technology gap and provide research data and actionable data to support future developments in areas such as geometric design, signal timing, road markings, and signage and systematically improve highway safety. Although the site-based approach also may miss some variables, such as those associated with human factors (e.g., the status and individual actions of drivers and passengers), this limitation merely defines the site-based approach as being complementary to naturalistic driving studies and not a direct substitute. It also offers future opportunities for combining site-based vehicle trajectory measurement with vehicle-based naturalistic driving studies. Much work has been done in the area of video-based tracking of moving objects, including the tracking of highway vehicles, but to date no system has been shown to be sufficiently accurate, flexible, and automated to be used as a routine tool for safety research. The Site Observer has been designed with this broad scope of safety research as its primary goal. This is in contrast to many research-level tracking systems, which typically make use of temporary single-camera installa- tions and focus on algorithm development or in creating specific reference data sets. The progress in such research, reviewed in the main report, is important for advancing the science and technol- ogy of video-based vehicle tracking, but in the past such developments have not gone far enough to establish scalable architectures or robust hardware and real-time software. These kinds of devel- opment are crucial to enabling long-term development and routine implementation. Given the technical challenges associated with machine vision, the previous focus on algorithm develop- ment is understandable, and of course the supporting low-level video image processing algo- rithms are also important for the Site Observer. However, the philosophy of the current project has been to use state-of-the-art image processing as the starting point of the system design, albeit including some application-specific refinements; thus, the challenge has been to integrate the resulting information from multiple synchronized video streams and infer vehicle positions and velocities across time to provide the overall motion capture capability. Executive Summary
2The Site Observerâs capability is also in contrast to the many commercial video systems currently used for traffic management. One major difference is that the commercial systems are based almost exclusively on background subtraction (also a part of the Site Observerâs operation) and make use of predefined image regions to determine occupancyâwhether or not a vehicle is currently in those regions. This serves the important purpose of simplifying the vision/inference problem and effecting data reduction, but it removes too much critical information relevant to vehicle motion capture, and makes it very difficult to subsequently perform tracking. In addition, this typical approach requires well-defined camera locations relative to the highway, something that is feasible for traffic management but is not realistic for temporary site monitoring equipment that must be capable of dealing with low camera angles, oblique motion of vehicles, and relatively unstructured vehicle motions in and around intersections (where predictable lateral positioning within lanes cannot be assumed). As mentioned, the Site Observer was designed to address key safety research questions posed by SHRP 2. A number of safety research topics are identified in this work, including those relating to path-crossing conflicts at intersections and the influence of highway factors on lane or road departures. But one crucial area that spans multiple crash types is in the relationship between conflicts and crashes and the potential to use conflict measures as surrogates for crash. Using vali- dated conflict measures in place of actual crashes means that the influence of highway factors, countermeasures, and site improvements can be evaluated over relatively short monitoring peri- ods, but the surrogates must faithfully represent crash type and crash risk. The Site Observer can be used for surrogate validation and for the use of surrogates in countermeasure evaluation, but validation should come first. To achieve this requires sampling the traffic flow in an unbiased way, so it is crucial that large volumes of vehicle motions can be captured with high fidelity so that conflict rates can be ascertained and then related to crash rates. The focus of this report is on intersection conflicts and safety, mainly because intersections are sites of relatively high conflict rates and are also most challenging for the video tracking technology; however, the Site Observer can be applied to other sites of safety concern. Path- crossing conflicts are particularly challenging for the technology in terms of position, speed, and synchronization errors of different vehicle trajectories. Ideally the system should provide a high degree of positioning accuracy for the analysis of path-crossing conflicts; it has been shown that unbiased errors with root-mean-square (RMS) values on the order of 20 cm are needed at the conflict points (near the center of the intersection) and with negligible timing errors. The system performance was seen to have negligible synchronization errors (sub millisecond), but with positioning errors as far as 50 m from the intersection of approximately 40 cm RMS, which is in excess of the 20-cm target. In fact, it was not possible to fully and formally evaluate positioning errors because of the lack of a completely reliable, accurate, and independent benchmark mea- surement system. Manual review of sample trajectories showed some small sources of bias in lateral position because of shadows, but overall the RMS errors at the intersection center were of the same order of magnitude as in the stated requirement. Note that such levels of accuracy are completely impossible using radar technology. Worst-case errors occurred when only two cameras covered the vehicle movement within the intersection, whereas in most cases three cameras could simultaneously view at least part of the vehicle track, in which case positioning errors were low. Among the broader range of design requirements, the most important is that the system should be fully automated. One of the most serious deficiencies in previous systems designed to track vehicles is that manual corrections are needed; operator intervention was used to correct for faults and guide the detection and tracking subsystems. Full automation may lead to less than 100% of all vehicles being tracked, but as long as errors are flagged, bad data excluded, and data loss shown to be unbiased, this has no impact on the usefulness of the system. It is possible that this consid- eration will be important when the Site Observer is used in locations with very dense traffic, but according to the pilot study in this project negligible data loss (less than 1%) of this type was experienced.
3In addition to accuracy and automation, the third critical requirement for the system has been mentionedâthe need for a flexible and expandable architecture, with relatively unrestricted camera positioning and the ability to add additional cameras for more complex or large sites of interest. Of course, many other practical considerations exist, and these are elaborated in the report. However, the system was found to be capable of running in a variety of weather and lighting conditions, including light snow and with low sun angles. The design has been realized as a prototype system, which has been built and tested, and has captured representative data. It uses four machine vision cameras and is organized in a hierar- chical way with early stage processing taking place on site, local to where each camera is installed. To ensure precise synchronization, camera shutters are triggered via pulses from local Global Positioning System (GPS) transceivers, and there is no need for the different cameras to com- municate during operation, other than to send extracted image features to the site computer (ground station); this can happen at discrete time intervals, and although optical fiber links were used in this project, it is anticipated that wireless networks can be used in the future, reducing installation times. In practice, camera positioning depends on the availability and access to building structures, lighting poles, and so forth, and although it may be necessary in some cases to install new mount- ing poles, it was considered a design requirement that only modest camera installation heights would be available. In this case, perspective changes in vehicle outline and the effects of road surface height changes were all considered as likely sources of error for vehicle tracking. There- fore, algorithms were implemented to include these effects in the 3-D mappings used when data from multiple cameras were combined. The processing method is organized into two distinct levels: 1. Camera level. Features are extracted in the 2-D camera frame from the image (pixel) data and grouped into clusters, in this case clusters of corner features. The grouping process is effective at creating long-lived tracks in the image frame, although a certain amount of dither is intro- duced as individual corner features are either lost or added. A second set of features is also captured, namely, the boundaries of the foreground regions (so-called âblobsâ) where can- didate vehicles exist. These features are determined by background subtraction and fitted by convex polygons. Real-time data processing was developed for this task using an existing real-time software platform. 2. Site level. The recorded camera level features are projected into the 3-D space of the site. Although multiple cameras capture the same scene from different angles, there is no attempt to implement stereo vision techniques. Stereo vision requires cameras to have similar viewing angles, and the camera separation and calibration is crucial. The research team used a simpler technique (that requires fewer cameras) based on the fact that the foreground regions coincide with âsame vehicle,â whereas the cluster tracks do not. Fitting a 3-D candidate shape that is rectangular (in plan view) and aligned with the motion vector then allows a match to the inter- secting blobs and provides a high level of vehicle localization, especially when there are three or more simultaneous camera views available. This requirement is for only one location on the site, and it was found to be true for most vehicle trajectories at the test intersection. Once local- ized in space, the absolute heights of the cluster tracks can be estimated and used as virtual markers to track the vehicles. There is no requirement for real-time processing for site-level data analysis. This is the essence of the vehicle tracking system, and the result is a set of trajectories indexed by absolute GPS time. From the trajectory data, any kind of single-vehicle or multiple-vehicle conflict is found, such as path-crossing or turn-into-path conflicts. Intersection traffic signal states were captured simultaneously so that relevance to signal phase can be determined easily. Hardware for the Site Observer comprised commercial off-the-shelf components based on a PC architecture. Custom circuit boards and assembly were part of the hardware system design,
4and dedicated cabinets were used together with temperature control for all-weather use. The system was installed at a suburban intersection in Ann Arbor, Michigan, capturing on the order of 17,000 vehicle trajectories over several sessions of daytime traffic monitoring. As previously mentioned, the vertical geometry of the site was important for carrying out the necessary geo- metric mappings from camera to site views. To assist this and the choice of camera locations, the project made use of a lidar survey. The current system uses four cameras and five computers (one per camera plus one to collect feature data and apply start/stop control). This choice was made in light of balancing hardware and installation costs against the need for full coverage of the site. The cameras were mounted on traffic signal mast arms at heights of approximately 6 m aboveground (i.e., a little less than 20 ft). Limiting data volumes for storage and processing is also an important operational constraint. The data structures used for tracking exist at three stages, ranging from high volumes and low specific information content to low volumes and high information content. These are, from highest to lowest data volume: 1. Camera input data. Grayscale values are found at each pixel location, 1 byte per pixel (with 256 levels of gray). Although color cameras could have been used, they were not considered necessary and would increase the data volumes handled by the local camera computers. 2. Camera output data. Feature locations are stored in camera coordinates. These consist of 2-D positions and velocities for the clusters used for tracking, as well as the vertices of the poly- gons used for foreground blobs. Optionally compressed video can be exported for operator review. 3. Site Observer output data. This is the set of vehicle trajectories, augmented by estimates of vehicle length, height, and width. The data volumes for the above were, for the system tested, of the order (1) 25 Mb per second, (2) 50 Kb per second (without compressed video), and (3) 1 Kb per second. This final value depends on traffic volumes, here assumed to be around 1,000 vehicles per hour passing the site (somewhat higher than the actual peak traffic flow rate at the trial intersection). Thus, the over- all operation of the Site Observer can be viewed as squeezing meaningful trajectory information from vast volumes of raw video data via the two distinct processing levels. (Note that 25 Mb per second scales to roughly 2 Tb of image data during a 24-h period; by contrast the Site Observer output for 1 year of operation is a modest 0.3 Tb). As the number of cameras increases, the data volumes scale in proportion at stages 1 and 2, but so does the processing power because there is one image processor per camera station. For stage 3, there is no corresponding increase in data volumes, just an increase in positioning accuracy. Again the scalability of the system is clear. In this project, a feature database was constructed, trajectories were extracted, and sample conflict analyses were undertaken for left-turn-across-path (opposite direction) and right-turn- into-path configurations. The Site Observer is a robust prototype system that, while capable of further development and refinement, exists as a deployable system in its current state. The natural follow-on from this work is to deploy the system at a site of particular safety interest, where it is possible to test the extent to which configurations and relative frequencies of crashes match to the same patterns of conflicts. A second follow-on is to implement the system at a location where significant numbers of instrumented vehicles pass through, which would include the influence of driver factors on conflict measures obtained from the Site Observer.
