Guidebook for Managing Data from Emerging Technologies for Transportation (2020)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

2020 N A T I O N A L C O O P E R A T I V E H I G H W A Y R E S E A R C H P R O G R A M NCHRP RESEARCH REPORT 952 Guidebook for Managing Data from Emerging Technologies for Transportation Kelley Klaver Pecheux Benjamin B. Pecheux Gene Ledbetter Chris Lambert AEM CorporAtion Herndon, VA Subscriber Categories Administration and Management • Data and Information Technology • Highways Research sponsored by the American Association of State Highway and Transportation Officials in cooperation with the Federal Highway Administration

NATIONAL COOPERATIVE HIGHWAY RESEARCH PROGRAM Systematic, well-designed, and implementable research is the most effective way to solve many problems facing state departments of transportation (DOTs) administrators and engineers. Often, highway problems are of local or regional interest and can best be studied by state DOTs individually or in cooperation with their state universities and others. However, the accelerating growth of highway transporta- tion results in increasingly complex problems of wide interest to high- way authorities. These problems are best studied through a coordinated program of cooperative research. Recognizing this need, the leadership of the American Association of State Highway and Transportation Officials (AASHTO) in 1962 ini- tiated an objective national highway research program using modern scientific techniques—the National Cooperative Highway Research Program (NCHRP). NCHRP is supported on a continuing basis by funds from participating member states of AASHTO and receives the full cooperation and support of the Federal Highway Administration (FHWA), United States Department of Transportation, under Agree- ment No. 693JJ31950003. The Transportation Research Board (TRB) of the National Academies of Sciences, Engineering, and Medicine was requested by AASHTO to administer the research program because of TRB’s recognized objectivity and understanding of modern research practices. TRB is uniquely suited for this purpose for many reasons: TRB maintains an extensive com- mittee structure from which authorities on any highway transportation subject may be drawn; TRB possesses avenues of communications and cooperation with federal, state, and local governmental agencies, univer- sities, and industry; TRB’s relationship to the National Academies is an insurance of objectivity; and TRB maintains a full-time staff of special- ists in highway transportation matters to bring the findings of research directly to those in a position to use them. The program is developed on the basis of research needs iden- tified by chief administrators and other staff of the highway and transportation departments, by committees of AASHTO, and by the FHWA. Topics of the highest merit are selected by the AASHTO Special Committee on Research and Innovation (R&I), and each year R&I’s recommendations are proposed to the AASHTO Board of Direc- tors and the National Academies. Research projects to address these top- ics are defined by NCHRP, and qualified research agencies are selected from submitted proposals. Administration and surveillance of research contracts are the responsibilities of the National Academies and TRB. The needs for highway research are many, and NCHRP can make significant contributions to solving highway transportation problems of mutual concern to many responsible groups. The program, however, is intended to complement, rather than to substitute for or duplicate, other highway research programs. Published research reports of the NATIONAL COOPERATIVE HIGHWAY RESEARCH PROGRAM are available from Transportation Research Board Business Office 500 Fifth Street, NW Washington, DC 20001 and can be ordered through the Internet by going to https://www.nationalacademies.org and then searching for TRB Printed in the United States of America NCHRP RESEARCH REPORT 952 Project 08-116 ISSN 2572-3766 (Print) ISSN 2572-3774 (Online) ISBN 978-0-309-67349-5 Library of Congress Control Number 2020946198 © 2020 National Academy of Sciences. All rights reserved. COPYRIGHT INFORMATION Authors herein are responsible for the authenticity of their materials and for obtaining written permissions from publishers or persons who own the copyright to any previously published or copyrighted material used herein. Cooperative Research Programs (CRP) grants permission to reproduce material in this publication for classroom and not-for-profit purposes. Permission is given with the understanding that none of the material will be used to imply TRB, AASHTO, FAA, FHWA, FTA, GHSA, NHTSA, or TDC endorsement of a particular product, method, or practice. It is expected that those reproducing the material in this document for educational and not-for-profit uses will give appropriate acknowledgment of the source of any reprinted or reproduced material. For other uses of the material, request permission from CRP. NOTICE The research report was reviewed by the technical panel and accepted for publication according to procedures established and overseen by the Transportation Research Board and approved by the National Academies of Sciences, Engineering, and Medicine. The opinions and conclusions expressed or implied in this report are those of the researchers who performed the research and are not necessarily those of the Transportation Research Board; the National Academies of Sciences, Engineering, and Medicine; the FHWA; or the program sponsors. The Transportation Research Board; the National Academies of Sciences, Engineering, and Medicine; and the sponsors of the National Cooperative Highway Research Program do not endorse products or manufacturers. Trade or manufacturers’ names appear herein solely because they are considered essential to the object of the report.

The National Academy of Sciences was established in 1863 by an Act of Congress, signed by President Lincoln, as a private, non- governmental institution to advise the nation on issues related to science and technology. Members are elected by their peers for outstanding contributions to research. Dr. Marcia McNutt is president. The National Academy of Engineering was established in 1964 under the charter of the National Academy of Sciences to bring the practices of engineering to advising the nation. Members are elected by their peers for extraordinary contributions to engineering. Dr. John L. Anderson is president. The National Academy of Medicine (formerly the Institute of Medicine) was established in 1970 under the charter of the National Academy of Sciences to advise the nation on medical and health issues. Members are elected by their peers for distinguished contributions to medicine and health. Dr. Victor J. Dzau is president. The three Academies work together as the National Academies of Sciences, Engineering, and Medicine to provide independent, objective analysis and advice to the nation and conduct other activities to solve complex problems and inform public policy decisions. The National Academies also encourage education and research, recognize outstanding contributions to knowledge, and increase public understanding in matters of science, engineering, and medicine. Learn more about the National Academies of Sciences, Engineering, and Medicine at www.nationalacademies.org. The Transportation Research Board is one of seven major programs of the National Academies of Sciences, Engineering, and Medicine. The mission of the Transportation Research Board is to provide leadership in transportation improvements and innovation through trusted, timely, impartial, and evidence-based information exchange, research, and advice regarding all modes of transportation. The Board’s varied activities annually engage about 8,000 engineers, scientists, and other transportation researchers and practitioners from the public and private sectors and academia, all of whom contribute their expertise in the public interest. The program is supported by state transportation departments, federal agencies including the component administrations of the U.S. Department of Transportation, and other organizations and individuals interested in the development of transportation. Learn more about the Transportation Research Board at www.TRB.org.

C O O P E R A T I V E R E S E A R C H P R O G R A M S CRP STAFF FOR NCHRP RESEARCH REPORT 952 Christopher J. Hedges, Director, Cooperative Research Programs Lori L. Sundstrom, Deputy Director, Cooperative Research Programs Lawrence D. Goldstein, Senior Program Officer Anthony P. Avery, Senior Program Assistant Eileen P. Delaney, Director of Publications Natalie Barnes, Associate Director of Publications Linda A. Dziobek, Senior Editor NCHRP PROJECT 08-116 PANEL Field of Transportation Planning—Area of Planning Methods & Processes Bo Zou, University of Illinois at Chicago, Chicago, IL (Chair) Jason Bittner, Applied Research Associates, Inc., Madison, WI Kim S. Burton, Missouri Public Service Commission, Jefferson City, MO Wei Fan, University of North Carolina at Charlotte, Charlotte, NC Jared D. Kauffman, DART First State, Wilmington, DE Jianming Ma, Texas DOT, Austin, TX Pavithra Parthasarathi, Trans Planning/Puget Sound Regional Council, Seattle, WA Gustavo Serratos, Jr., Complete Streets, Detroit, MI Alison C. Stettner, Florida DOT, Ocoee, FL Joanna Wadsworth, City of Las Vegas, Las Vegas, NV Ryan J. Westrom, City Solutions/Ford Mobility, Washington, D.C. Govind Vadakpat, FHWA Liaison Richard A. Cunard, TRB Liaison AUTHOR ACKNOWLEDGMENTS J. D. Schneeberger and John Hicks of Noblis, Inc., Washington, D.C., and Brian Burkhard and Mara Campbell of Jacobs, San Francisco, California, and New Florence, Missouri, respectively, contributed to the report. 15612-00b_FM-4thPgs.indd 4 9/23/20 8:48 AM

New forms of communication and other evolving technologies have led to advance- ments in infrastructure and vehicle capabilities, new mobile applications, and a host of mobility service offerings. In turn, these changes have led to the creation of new, big, and varied data sets available for use by transportation agencies. When used effectively, data from these emerging technologies have tremendous potential to offer new insights to transportation agencies in managing, operating, maintaining, and developing transporta- tion infrastructure. NCHRP Research Report 952: Guidebook for Managing Data from Emerging Technologies for Transportation will assist state departments of transportation (DOTs) and other trans- portation planning agencies in applying these large data sets. This research report is in the form of a guidebook and provides a framework for managing big data from emerging technologies, including data from connected and automated vehicles and data linked to new mobility initiatives (e.g., smart city programs). The guidebook also outlines a process for applying that framework to incorporate these data into the decision-making process. Intelligent transportation systems (ITS) technologies have helped to advance trans- portation safety and mobility and to enhance American productivity over the past three decades. Recently, further developments in technology and communications have led to the development of more advanced infrastructure and vehicle capabilities, mobile applications, shared-use and micro-mobility services, crowdsourcing, connected and automated vehicles (CAVs), the Internet of things (IoT), and new mobility initiatives such as smart cities and communities. To demonstrate and build on these emerging technologies, a wide range of institutions, both public and private, have initiated and invested in these technologies, new data sets, and pilot projects and programs. As these activities continue to expand, the amount of data surrounding them is also expanding. Unexpected and unprecedented amounts of data will continue to become available to transportation agencies and will fundamentally alter the transportation sector. While these data sets have broad potential to offer new insights and to identify unique solutions for delivering services and improving outcomes, they are often too large, are too varied in nature, and will change too quickly to be handled by the traditional database management systems of most transportation agencies. Instead, trans- portation agencies need to adopt modern, flexible, and scalable big data methods to manage these data if these data are to be used to facilitate better decision-making. As many agencies are already forced to do more with less while meeting higher public expectations, continuing with traditional data management systems and practices will prove costly for agencies unable to make the shift. F O R E W O R D By Lawrence D. Goldstein Staff Officer Transportation Research Board

Under NCHRP Project 08-116, “Framework for Managing Data from Emerging Trans- portation Technologies to Support Decision-Making,” AEM Corporation developed this guidebook to help state DOTs and other transportation agencies begin to move from traditional data systems to advanced data systems and management practices designed to make more effective use of data from emerging technologies. In addition to this guide- book, an executive summary and a PowerPoint presentation directed to decision makers were developed. There are also two online-only products: NCHRP Web-Only Document 282: Framework for Managing Data from Emerging Transportation Technologies to Support Decision- Making, which documents the research process, and a technical memorandum that proposes steps to disseminate research findings and products to the user community. All five products are available on the TRB website at www.trb.org.

1 Chapter 1 Introduction 3 Supporting Resources and Tools 4 Terminology 5 Chapter 2 Laying the Foundation 5 Traditional Data System and Management Approach Versus the Modern Big Data System and Management Approach 9 Modern Big Data Architecture 11 Chapter 3 Roadmap to Managing Data From Emerging Technologies for Transportation 11 Overview of the Roadmap 15 Step 1. Develop an Understanding of Big Data 21 Step 2. Identify a Use Case and an Associated Pilot Project 24 Step 3. Secure Buy-in from at Least One Person from Leadership for the Pilot Project 28 Step 4. Establish an Embryotic Big Data Test Environment 34 Step 5. Develop the Pilot Project Within the Big Data Test Environment/Playground 41 Step 6. Demonstrate the Value of the Data to Other Business Units 45 Step 7. Demonstrate the Value of the Data to Executive Leadership 48 Step 8. Establish a Formal Data Storage and Management Environment 53 Chapter 4 Modern Big Data Management Life Cycle and Framework 53 Modern Big Data Management Life Cycle 53 Modern Big Data Management Framework 73 Chapter 5 Supporting Tools 73 Data Management Capability Maturity Self-Assessment 75 Big Data Governance Roles and Responsibilities 95 Data Sources Catalog Tool 99 Frequently Asked Questions or FAQs 106 Works Cited C O N T E N T S Note: Photographs, figures, and tables in this report may have been converted from color to grayscale for printing. The electronic version of the report (posted on the web at www.trb.org) retains the color versions.