Leveraging Big Data to Improve Traffic Incident Management (2019)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

2019 N A T I O N A L C O O P E R A T I V E H I G H W A Y R E S E A R C H P R O G R A M NCHRP RESEARCH REPORT 904 Leveraging Big Data to Improve Traffic Incident Management Kelley Klaver Pecheux Benjamin B. Pecheux AEM CorporAtion Herndon, VA Grady Carrick EnforCEMEnt EnginEEring, inC. Ponte Vedra, FL Subscriber Categories Highways • Operations and Traffic Management • Security and Emergencies Research sponsored by the American Association of State Highway and Transportation Officials in cooperation with the Federal Highway Administration

NATIONAL COOPERATIVE HIGHWAY RESEARCH PROGRAM Systematic, well-designed, and implementable research is the most effective way to solve many problems facing state departments of transportation (DOTs) administrators and engineers. Often, highway problems are of local or regional interest and can best be studied by state DOTs individually or in cooperation with their state universities and others. However, the accelerating growth of highway transporta- tion results in increasingly complex problems of wide interest to high- way authorities. These problems are best studied through a coordinated program of cooperative research. Recognizing this need, the leadership of the American Association of State Highway and Transportation Officials (AASHTO) in 1962 ini- tiated an objective national highway research program using modern scientific techniques—the National Cooperative Highway Research Program (NCHRP). NCHRP is supported on a continuing basis by funds from participating member states of AASHTO and receives the full cooperation and support of the Federal Highway Administration, United States Department of Transportation. The Transportation Research Board (TRB) of the National Academies of Sciences, Engineering, and Medicine was requested by AASHTO to administer the research program because of TRB’s recognized objectivity and understanding of modern research practices. TRB is uniquely suited for this purpose for many reasons: TRB maintains an extensive com- mittee structure from which authorities on any highway transportation subject may be drawn; TRB possesses avenues of communications and cooperation with federal, state, and local governmental agencies, univer- sities, and industry; TRB’s relationship to the National Academies is an insurance of objectivity; and TRB maintains a full-time staff of special- ists in highway transportation matters to bring the findings of research directly to those in a position to use them. The program is developed on the basis of research needs identified by chief administrators and other staff of the highway and transportation departments, by committees of AASHTO, and by the Federal Highway Administration. Topics of the highest merit are selected by the AASHTO Special Committee on Research and Innovation (R&I), and each year R&I’s recommendations are proposed to the AASHTO Board of Direc- tors and the National Academies. Research projects to address these topics are defined by NCHRP, and qualified research agencies are selected from submitted proposals. Administration and surveillance of research contracts are the responsibilities of the National Academies and TRB. The needs for highway research are many, and NCHRP can make significant contributions to solving highway transportation problems of mutual concern to many responsible groups. The program, however, is intended to complement, rather than to substitute for or duplicate, other highway research programs. Published research reports of the NATIONAL COOPERATIVE HIGHWAY RESEARCH PROGRAM are available from Transportation Research Board Business Office 500 Fifth Street, NW Washington, DC 20001 and can be ordered through the Internet by going to http://www.national-academies.org and then searching for TRB Printed in the United States of America NCHRP RESEARCH REPORT 904 Project 17-75 ISSN 2572-3766 (Print) ISSN 2572-3774 (Online) ISBN 978-0-309-48071-0 Library of Congress Control Number 2019947990 © 2019 National Academy of Sciences. All rights reserved. COPYRIGHT INFORMATION Authors herein are responsible for the authenticity of their materials and for obtaining written permissions from publishers or persons who own the copyright to any previously published or copyrighted material used herein. Cooperative Research Programs (CRP) grants permission to reproduce material in this publication for classroom and not-for-profit purposes. Permission is given with the understanding that none of the material will be used to imply TRB, AASHTO, FAA, FHWA, FMCSA, FRA, FTA, Office of the Assistant Secretary for Research and Technology, PHMSA, or TDC endorsement of a particular product, method, or practice. It is expected that those reproducing the material in this document for educational and not-for-profit uses will give appropriate acknowledgment of the source of any reprinted or reproduced material. For other uses of the material, request permission from CRP. NOTICE The research report was reviewed by the technical panel and accepted for publication according to procedures established and overseen by the Transportation Research Board and approved by the National Academies of Sciences, Engineering, and Medicine. The opinions and conclusions expressed or implied in this report are those of the researchers who performed the research and are not necessarily those of the Transportation Research Board; the National Academies of Sciences, Engineering, and Medicine; or the program sponsors. The Transportation Research Board; the National Academies of Sciences, Engineering, and Medicine; and the sponsors of the National Cooperative Highway Research Program do not endorse products or manufacturers. Trade or manufacturers’ names appear herein solely because they are considered essential to the object of the report.

The National Academy of Sciences was established in 1863 by an Act of Congress, signed by President Lincoln, as a private, non- governmental institution to advise the nation on issues related to science and technology. Members are elected by their peers for outstanding contributions to research. Dr. Marcia McNutt is president. The National Academy of Engineering was established in 1964 under the charter of the National Academy of Sciences to bring the practices of engineering to advising the nation. Members are elected by their peers for extraordinary contributions to engineering. Dr. John L. Anderson is president. The National Academy of Medicine (formerly the Institute of Medicine) was established in 1970 under the charter of the National Academy of Sciences to advise the nation on medical and health issues. Members are elected by their peers for distinguished contributions to medicine and health. Dr. Victor J. Dzau is president. The three Academies work together as the National Academies of Sciences, Engineering, and Medicine to provide independent, objective analysis and advice to the nation and conduct other activities to solve complex problems and inform public policy decisions. The National Academies also encourage education and research, recognize outstanding contributions to knowledge, and increase public understanding in matters of science, engineering, and medicine. Learn more about the National Academies of Sciences, Engineering, and Medicine at www.national-academies.org. The Transportation Research Board is one of seven major programs of the National Academies of Sciences, Engineering, and Medicine. The mission of the Transportation Research Board is to increase the benefits that transportation contributes to society by providing leadership in transportation innovation and progress through research and information exchange, conducted within a setting that is objective, interdisciplinary, and multimodal. The Board’s varied committees, task forces, and panels annually engage about 7,000 engineers, scientists, and other transportation researchers and practitioners from the public and private sectors and academia, all of whom contribute their expertise in the public interest. The program is supported by state transportation departments, federal agencies including the component administrations of the U.S. Department of Transportation, and other organizations and individuals interested in the development of transportation. Learn more about the Transportation Research Board at www.TRB.org.

C O O P E R A T I V E R E S E A R C H P R O G R A M S CRP STAFF FOR NCHRP RESEARCH REPORT 904 Christopher J. Hedges, Director, Cooperative Research Programs Lori L. Sundstrom, Deputy Director, Cooperative Research Programs William C. Rogers, Senior Program Officer Jarrel McAfee, Senior Program Assistant Eileen P. Delaney, Director of Publications Natalie Barnes, Associate Director of Publications Sharon Lamberton, Editor NCHRP PROJECT 17-75 PANEL Field of Traffic—Area of Safety Stephen W. Glascock, Louisiana DOTD, Baton Rouge, LA Mara K. Campbell, Jacobs, New Florence, MO Melissa L. Clark, California DOT, Sacramento, CA Steve J. Cyra, HNTB Corporation, Milwaukee, WI Edward Gincauskis, Massachusetts DOT, Boston, MA Eric J. Hemphill, North Texas Tollway Authority, Plano, TX Naveen Lamba, Grant Thornton LLP, Alexandria, VA Eileen M. Singleton, Baltimore Metropolitan Council, Baltimore, MD Pradeep Tiwari, Phoenix, AZ Douglas Mark Tomlinson, Pennsylvania DOT, Harrisburg, PA Paul Jodoin, FHWA Liaison Victor T. Hom, National Oceanic and Atmospheric Administration Liaison Richard A. Cunard, TRB Liaison

NCHRP Research Report 904 provides guidance for transportation and traffic incident management agencies to bring multiple comprehensive datasets (Big Data) together to derive useful information and relationships that could improve their efforts to reduce clearance times and increase highway safety. The ability to mine information on hereto- fore-unanticipated trends can provide significant opportunities for improving protocols, resource management, scene management, and real-time data sharing. As the nation continues its migration toward Big Data, an overwhelming volume of data can be used to improve the current state of traffic incident management (TIM). Big Data is not just “a lot more data” than what was available before. It is a fundamental change in how data are collected, analyzed, and used to uncover trends and relationships. In general, recent advances in information technology (IT) have significantly increased data quantity, improved data quality, and enhanced data analytics. Recognizing the tremendous potential of Big Data applications both within and outside the realm of transportation, many agencies are faced with the challenge of using or even identifying the rich datasets that could be leveraged to enhance or improve TIM efforts. A need exists to develop a Big Data environ- ment in which datasets from multiple sources can be managed and valued. The challenges are to discover the datasets, to merge them into a shared Big Data environment, to uncover important relationships, and to identify trends that may occur outside the traditional evalu- ation processes. In NCHRP Project 17-75, “Leveraging Big Data to Improve Traffic Incident Manage- ment,” AEM Corporation was asked to develop guidelines that (1) describe current and emerging sources of Big Data that could improve TIM; (2) describe potential opportunities to leverage Big Data that could advance TIM state of the practice; (3) identify potential challenges (e.g., security, proprietary, or inter-operability issues) for TIM agencies seeking to leverage Big Data; and (4) develop a matrix of Big Data options for transportation and TIM agencies to use based on their current capabilities. F O R E W O R D By William C. Rogers Staff Officer Transportation Research Board

1 Summary 6 Chapter 1 Introduction 7 1.1 Objective 7 1.2 Overview of Research and Organization of Report 9 Chapter 2 State of the Practice of TIM 9 2.1 State of the Practice 10 2.1.1 Establishment of Local, Regional, and Statewide TIM Committees 11 2.1.2 Implementation of TIM Legislation 11 2.1.3 Development and Implementation of National TIM Responder Training 11 2.1.4 Development of TIM Strategic Plans 12 2.1.5 Development and Implementation of Agency Operating Agreements 12 2.1.6 Implementation of Agency Policies for Safe and Quick Clearance 12 2.2 The Use of Data to Support TIM 12 2.2.1 TIM Performance Measurement and Management 14 2.2.2 Making the Business Case for TIM 15 2.3 Further Advancing the State of the Practice of TIM 16 Chapter 3 State of the Practice of Big Data 16 3.1 Big Data Definition 17 3.1.1 Volume 18 3.1.2 Variety 18 3.1.3 Velocity 19 3.1.4 Veracity 19 3.1.5 Value 19 3.2 The Move from Traditional Data Analysis to Big Data Analytics 19 3.2.1 Traditional Data Analysis 21 3.2.2 Hadoop: The Start of Big Data Tools 21 3.2.3 Current Big Data Tools 25 3.2.4 Big Data Architecture 26 3.2.5 Examples of Big Data Analytics 32 3.3 Big Data Applications in Transportation 32 3.3.1 Transportation Planning 33 3.3.2 Parking 33 3.3.3 Trucking 34 3.3.4 Public Transportation 34 3.3.5 Transportation Operations and ITSs 37 3.3.6 Emergency and Incident Management C O N T E N T S

39 Chapter 4 Big Data and TIM 40 4.1 Improve On-Scene Management Practices 42 4.2 Improve Resource Utilization and Management 44 4.3 Improve Safety 46 4.4 Enable Predictive TIM 48 4.5 Support Performance Measurement and Management 51 4.6 Support TIM Justification and Funding 53 4.7 Summary 55 Chapter 5 Assessment of Data Sources for TIM 55 5.1 Data Source Assessment Approach 56 5.1.1 Assessment Criteria 60 5.1.2 Data Maturity Assessment Approach 62 5.2 Findings 62 5.2.1 State Traffic Records Data 70 5.2.2 Transportation Data 75 5.2.3 Public Safety Data 77 5.2.4 Crowdsourced Data 81 5.2.5 Advanced Vehicle Systems Data 85 5.2.6 Aggregated Datasets 92 5.3 Summary 94 Chapter 6 Big Data Guidelines for TIM Agencies 96 6.1 Adopt a Deeper and Broader Perspective on Data Use 96 6.2 Collect More Data 98 6.3 Open and Share Data 99 6.3.1 Public Records Laws 99 6.3.2 Proprietary Data Formats 100 6.3.3 Contract Data Clauses 100 6.3.4 Benefits of Opening and Sharing Data 101 6.4 Use a Common Data Storage Environment 101 6.4.1 Data Silos 101 6.4.2 Data Virtualization 102 6.5 Adopt Cloud Technologies for the Storage and Retrieval of Data 103 6.5.1 Understand the Cost Savings of the Cloud 104 6.5.2 Understand Cloud Security 105 6.5.3 Recognize the Inherent Connection Between Big Data Analytics and the Cloud 106 6.6 Manage the Data Differently 106 6.6.1 Store the Data “As Is” 107 6.6.2 Maintain Data Accessibility 107 6.6.3 Structure the Data for Analysis 108 6.6.4 Ensure That Data Is Uniquely Identifiable 108 6.6.5 Sharing, Security, and Privacy 109 6.7 Process the Data 109 6.7.1 Process the Data Where It Is Located 110 6.7.2 Use Open-Source Software 112 6.7.3 Do Not Reinvent the Wheel 112 6.7.4 Understand the Ephemeral Nature of Big Data Analytics 113 6.8 Open and Share Outcomes and Products to Foster Data User Communities

114 Chapter 7 Summary and Next Steps 114 7.1 Summary of Findings 116 7.2 Next Steps 117 7.3 Suggestions and Priorities for Additional Related Research 118 Abbreviations 121 Glossary 125 References 132 Appendix A Data Source Assessment Tables 181 Appendix B Incident Response and Clearance Ontology (IRCO) Note: Photographs, figures, and tables in this report may have been converted from color to grayscale for printing. The electronic version of the report (posted on the web at www.trb.org) retains the color versions.