Site-Based Video System Design and Development (2012)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

21 C h a p t e r 4 This chapter considers the broad range of performance and operational requirements expected of the “ideal” video tracking system. Given the unique importance of trajectory accuracy and resolution in both time and space, based on the need to compute accurate conflict metrics, Chapter 5 will consider these aspects in greater detail and discuss the likely applicability of these conflict metrics as surrogates for crash. System requirements may cover a broad range of aspects (such as performance, operation, cost, maintenance, and user interface), and capturing requirements is an important step in the design of any system. For a prototype system, the emphasis is greater on the aspects related to performance and feasibility, but long-term viability for development into a deployable system remains important. In addition, there are many feasibility constraints that force the system design in a particular direction, so certain capabilities are derived rather than imposed: What tracking accuracy was achieved? What lighting conditions did it work under? Were there any speed or size limits, and so forth? Design requirements start with the overall system purpose, creating databases of vehicle tra- jectories that are sufficiently complete and accurate to sup- port safety analysis. They also address the costs and effort associated with achieving that purpose, including hardware and software costs, manual effort in setting up the system, level of effort in running and maintaining the system, and the technical and financial risks. These considerations apply to both the initial development project and the future use of the system in extended field studies. It is appropriate to think of the principal goal as being a proof of concept for a usable and widely deployable system, which may see expanded use in highway safety, rather than a time-limited study that generates a single fixed database. Location-specific safety issues, whether at intersections or other locations, clearly are too complex for any single data collection study to address the full range of problems. Ulti- mately, the requirement is to create a tool that can support a wide range of data collection, with many data sets made avail- able to many safety analysts—in other words to create a new tool that can spark safety improvements in the field. The importance of the technology that can create the core infor- mation for these studies cannot be overstated. The requirement for the tracking system is not solely to provide a new tool to address old problems, the very stub- born problem of traffic safety. As new technologies emerge, such as adaptive or intelligent traffic signal systems, the need for better evaluation techniques will only increase. Com- pared to technology developments in cars and trucks, where real-world testing and evaluation are becoming routine, site- based technologies lack the necessary safety monitoring and evaluation methods, and yet the need is no less great. The preceding discussion captures the broad customer need for a system that: • Produces accurate and reliable trajectory data; • Is affordable and usable in the long term by the highway community, perhaps with modest levels of support from researchers or video systems suppliers; • Is scalable for use in large and complex sites, with wide area coverage possible; and • Creates data sets that can be integrated with diverse other information sources. In the current project, while all major design decisions have been determined (and the prototype hardware built, installed, and run), it seems worthwhile to capture the motivations and considerations that went into the design. With this background and with emphasis on the robust prototype system developed, the research team developed a comprehensive list of operational and performance requirements. The technical performance of the site-based video system is centered on the trajectory data that can be acquired and the conditions under which these data can be captured. At a high Performance Requirements

22 Sensitivity to lighting and weather conditions: It must be expected that in poor lighting and weather, the availability (percentage time of error-free operation) will drop. In rare and extreme cases such as snow or dense fog, the vision- based system will certainly become inoperable. This does not present a source of bias provided researchers know and understand these effects. On the other hand, if availability is dropping significantly every time there is a change in ambient lighting, or when there is light rainfall, clearly this would be an unacceptable limitation. Nighttime use is another consideration, most easily addressed by suitable choice of camera type; in this project the focus was on daytime operation. Vehicle Information Vehicle size and type estimation: The basic requirement for trajectory capture is to extract the time history of the (x, y) coordinates of any suitable reference point on each vehicle and determine where this point is situated relative to the front, rear, and lateral boundaries of the vehicle. This means that vehicle length and width both need to be estimated, but nothing further need be considered an essential requirement. This provides the basis for at least an approximate classifica- tion into vehicle types, similar to the capability of some of the commercial systems discussed. When vehicle height or shape information is extracted, it is possible that more detailed and accurate information can be extracted relative to the existing commercial systems, but again this should be seen as a poten- tial bonus, not as a fundamental requirement for automated processing. When some event is found to be of interest, how- ever, recorded video should be available for a human reviewer to recognize vehicle types. Additional vehicle characteristics (e.g., truck loading, num- bers of passengers): Similar to the previous point, no such information should be regarded as a requirement of the automated data collection. Some information on truck loading might be inferred from acceleration performance, but the connection is only loose. It is preferred that stored video not be capable of resolving individual passengers to reduce concerns over possible invasion of privacy. Video is likely to be able to discern any very obvious external mark- ings (e.g., conspicuous trade names on commercial vehi- cles) but with limited resolution; the inability to count passengers or recognize driver characteristic is an advan- tage in the long term. Data availability Access to stored data: It is feasible that trajectory data can be made available to any team conducting relevant transpor- tation research or development. A base server can host the level, these should not be too specific; for example, it would be absurd to specify a requirement for “less than 5% of vehicle trajectories should display validation errors when the sun angle is less than 10° above the horizon.” In the following, the research team seeks to define a set of plausible benchmarks for testing performance. These can be used to assess performance and operation of the current system and provide a basis for future developments and comparison of rival systems. trajectory Information Accuracy and resolution in spatial and time coordinates: This should be sufficient to extract conflict metrics in avoidance and near-crash events (and of course in actual crashes) as well as to characterize exposure and statistical patterns in conflict metrics. Numerical targets are developed in Chapter 5. Error rates and sources of systematic bias: This relates to possible discrete errors in tracking—for example, when a trajectory is not resolved because of too few discernible fea- tures on the vehicle, sun glare, occlusion, intrusion of a flock of birds, or so forth. Clearly most trajectories need to be resolved under normal conditions of lighting, weather, and traffic density. A key point is that when an error occurs, it should not go unnoticed; validation checks are to be suffi- ciently robust to determine, for example, when some persis- tent moving feature (called a cluster track in the system design) cannot be associated with a vehicle, or to detect that a tracked vehicle enters the intersection but never leaves it. Under these circumstances, the error should be rectified using an automatic correction process or the error flagged so associ- ated trajectories are not used for what might be an erroneous conflict analysis. In a complex intersection, we might expect to be collecting valid data at least 80% of the time; for the simple intersection used in the pilot study, a much higher figure would be expected. The tradeoff here is between a fully automated system that is prone to intermittent inter- ruptions in sampling, compared with a manually supported system that is 100% available during the sampling times determined by project managers. The main concern is that sampling does not introduce bias, but otherwise it does not matter what process controls the sampling, provided the sys- tem is not prone to frequent interruptions. On the subject of potential bias, this will occur if the system cannot resolve certain specific situations relevant to crash frequencies (e.g., missing faster-moving traffic). To review missed vehicles or erroneous tracking, there is a requirement for compressed video to enable manual review of relevant flagged cases. Pro- vided interruptions are random and unconnected to the safety problem (e.g., caused by random scattering of light), they will not affect the validity of the research data. If sources of bias exist, the algorithm or other aspects of the system must simply be improved.

23 system could go to the extreme operational mode of “cap- ture by day, process by night,” in which case-sufficient stor- age would be needed for approximately 12 h of uncompressed video for each camera station, something that is no concern for a hard drive but could be a challenge for solid state memory. Scalability and parallel processing: Once the system is devel- oped and proved for an intersection of modest complexity, the same hardware and software should be capable of scaling up to a larger and more complex installation; to avoid pro- cessing bottlenecks, the major portion of the video process- ing should be parallelized (e.g., processing each video stream in parallel). Raw video: To avoid processing bottlenecks, it is a key requirement that raw video need not be exported from the data collection site. Data streaming: A streaming capability is required so that processing of trajectories can take place at the same time new data are collected. Ideally, 24 h of trajectory information should be processed into trajectory data in a 24-h period. Achieving this 24/24 (100%) capability depends on a number of factors (especially how many image features must be pro- cessed per frame to enable the data fusion to achieve the required availability and accuracy), so for now the proposed requirement is for the prototype to achieve approximately 50% of this capability based on processing speed. This will provide a useful immediate capability of processing 12 h of trajectories in a 24-h period, plus a strong expectation that a more fully integrated and efficient second-generation system will fulfill the ideal 100% capability. Modularity: Ideally, video image processing should take place adjacent to or within the camera itself, to remove the need for high-speed data networks at the intersection. For the prototype system, this level of integration is unneces- sary, but the architecture is to be sufficiently modular to support such an integration step in the next generation of the system. If network bandwidth can be sufficiently reduced, this opens the opportunity of using purely wire- less Ethernet connections linking cameras to one or more data storage hubs. Archival storage: The principal requirement is that trajec- tory data be stored in a relational database to support conflict metric and other analysis on demand. When events of inter- est are discovered in the data, researchers are likely to be interested in other aspects not readily accessible from the tra- jectory data, so it is important that compressed video is also retained. The video should be stored in a form that is com- patible with trajectory data so it can be displayed in synchro- nous fashion. Optionally, additional image-related primitives (e.g., cluster tracks) may be stored; this is not something the end user need be aware of, and the main reason will be to aid algorithm development. trajectories in a relational database for easy access by research- ers who are not necessarily part of the data collection team. Metadata and data dictionaries need to be formalized and should include information about periods with lost trajecto- ries. Data volumes should not be large, and this seems a rela- tively simple matter for the long term. error Checking and estimation Error checks: The trajectory estimation involves a high degree of data fusion. As previously mentioned, it is important that the data fusion be open to validation checks. Comprehensive error checking should be feasible in the automated analysis, to provide a validation or confidence parameter to the trajec- tory analysis. In addition, a video viewing tool is required so that bounding boxes of computed vehicle trajectories can be overlaid on stored video images; thus, a reviewer can look at samples of data to confirm success or otherwise of trajectory estimation. Derived information: Video processing can provide only so much information on the vehicle kinematics; the system should include additional estimation algorithms, including additional motion estimates, especially speed, longitudinal acceleration, and lateral acceleration. Accuracy checks: Once a trajectory has been synthesized, the question of its accuracy remains. This checking is not required to form an integral component of the system opera- tion because an independent reference is needed, which adds cost and complexity; however, such checks should be avail- able, especially as part of the system development. • Instrumented vehicles: Independent accuracy checks can be made using differential GPS (DGPS) and other on- vehicle instrumentation. The importance of onboard DGPS is clear, whereas onboard accelerometers, yaw rate sensors, and speed measurement (from wheel or transmis- sion speed) can be used to test the accuracy of derived information. • Site-based radar: This can be used to make an independent validity and accuracy test of the trajectory data. Radar may not provide an especially accurate alternative, but it does provide a method to benchmark performance. Computational aspects and Streaming Capability Data buffering: Some video buffering is expected, to provide flexibility of processing, especially when traffic volumes are high. The volume of data storage is largely irrelevant to the overall system performance, although it may affect the time over which the system can run before interruption is needed. It is possible that in very complex installations, the

24 • Data access: Time history data should be stored in a stan- dard database format, such as Microsoft SQL Server; • Analysis tools: Basic tools are needed to query the data to calculate a subset of conflict metrics, including time to col- lision and gap times for path-crossing conflicts; and • Visualization tools: A basic viewer should be provided so that events of interest, defined by event start and end time, can present video and time history data of that event. All such tools should be part of the prototype, with the expecta- tion that future generations will offer increasing user-friendly interfaces. User Interface It is important that the system provides an end-to-end con- nection between vehicle motion and safety analysis. The sys- tem was conceived as a research tool, and the expectation is that researchers using the system typically will understand vehicle dynamics, traffic flow theory, and intersection design and have some experience in data analysis and relational databases. But they should not require detailed knowledge of video formats, image processing, or camera technologies. For this reason, some basic tools are required for users to interact with the data: