Framework for Managing Data from Emerging Transportation Technologies to Support Decision-Making (2020)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Contents 1 Introduction ......................................................................................................................... 1 1.1 Research Objectives ........................................................................................................................ 1 1.2 Background ..................................................................................................................................... 1 1.3 Organization of Report .................................................................................................................... 3 2 Data Management Defined ................................................................................................... 4 2.1 Data Management Defined ............................................................................................................. 4 2.2 Why Big Data Management for Emerging Technologies? ............................................................... 6 3 Big Data Industry Review ...................................................................................................... 9 3.1 Big Data Industry Review ................................................................................................................ 9 3.1.1 Big Data Architecture Frameworks ....................................................................................... 10 3.1.2 Data Governance .................................................................................................................. 17 3.1.3 Data Security and Privacy ...................................................................................................... 23 3.1.4 Data Quality .......................................................................................................................... 26 3.1.5 Data Usability ........................................................................................................................ 27 3.1.6 Transparency and Provenance .............................................................................................. 29 3.1.7 Data Sharing .......................................................................................................................... 29 3.2 Foundational Principles of Big Data Management ........................................................................ 30 4 Overview of Approach ........................................................................................................ 36 4.1 Overview of Approach ................................................................................................................... 36 4.2 Findings ......................................................................................................................................... 37 4.2.1 Literature Review .................................................................................................................. 37 4.2.2 Online Survey and Telephone Interviews .............................................................................. 43 4.2.3 Project Documentation Reviews ........................................................................................... 52 4.2.4 Benchmark and Assessment Methodology ........................................................................... 54 4.2.5 Stakeholder Workshop ......................................................................................................... 60 4.3 Summary of Challenges and Needs ............................................................................................... 61 4.3.1 Challenges ............................................................................................................................. 61 4.3.2 Needs .................................................................................................................................... 64 5 Research Outputs ............................................................................................................... 68 5.1 Research Outputs .......................................................................................................................... 68 5.2 Recommendations for Future Research ........................................................................................ 68 5.3 Recommended Next Steps ............................................................................................................ 69

References ........................................................................................................................................... 70  Appendix A – Online Survey ................................................................................................................. 74  Appendix B – Interview Guide and Questions ....................................................................................... 82  Appendix C – Telephone Interview Findings ......................................................................................... 88  Appendix D – Project Documentation Review Findings ....................................................................... 101                                    This document contains the contractor’s revised final report for NCHRP Project 08-116 and accompanies NCHRP Research Report 952: Guidebook for Managing Data from Emerging Technologies for Transportation. Readers can read or purchase NCHRP Research Report 952 at www.trb.org.        

List of Figures Figure 1. Data Lifecycle (Spirion, 2019) ......................................................................................................... 5 Figure 2. DAMA DMBOK2 Knowledge Areas (DAMA International, 2017) .................................................... 5 Figure 3. Big Data Ecosystem: Data, Lifecycle, Infrastructure (Demchenko, Defining the Big Data Architecture Framework, 2013) ........................................................................................................ 11 Figure 4. Conceptional DW/BI and Big Data Architecture (DAMA International, 2017) .............................. 12 Figure 5. Big Data Ecosystem Reference Architecture (Levin, 2013) ........................................................... 13 Figure 6. The Multi-Dimensional Data Management Framework V4.0 (Multi-Dimensional Thinkers, 2020) .......................................................................................................................................................... 14 Figure 7. Big Data Governance Framework (Kim & Cho, 2018) ................................................................... 22 Figure 8. IBM Information Governance Council Maturity Model (Soares, 2018) ........................................ 23 Figure 9. Top Ten Big Data Security and Privacy Challenges (Demchenko, Canh, de Laat, Membrey, & Gordijenko, 2013) ............................................................................................................................. 24 Figure 10. Emerging Principles for Data Sharing (Chitkara, Deloison, Kelkar, Pandey, & Pankratz, 2020) 30 Figure 11. Framework for Improving Data Utilization for TPM (Harrison, et al., 2019) ............................... 41 Figure 12. Big Data Framework for Smart Grids (Daki, El Hannani, Aqqal, Haidine, & Dahbi, 2017) ........... 43 Figure 13. Agencies Responding to the Online Survey ................................................................................ 44 Figure 14. Survey Completeness Matrix ...................................................................................................... 45 Figure 15. What Type(s) of Emerging Technology is Your Agency Deploying? ............................................ 46 Figure 16. What is the Status of Your Emerging Transportation Technology Project? ................................ 46 Figure 17. What Type(s) of Data is the Emerging Technology Project Expected to Generate? ................... 47 Figure 18. What Organization is Primarily Responsible for Managing the Data? ........................................ 47 Figure 19. How is Your Agency Planning to Use the Data Being Collected? ................................................ 48 Figure 20. How Much of the Data is Being Archived? ................................................................................. 48 Figure 21. How is the Data Being Archived? ................................................................................................ 48 Figure 22. Benchmark Assessment Chart .................................................................................................... 58

List of Tables Table 1. Traditional Data System/Management Approach Contrasted with Modern, Big Data System/Management Approach ......................................................................................................... 8 Table 2. Synthesis of the Foundational Principles of Modern, Big Data Management ............................... 33 Table 3. Survey Responses by Type of Emerging Transportation Technology Project ................................ 44 Table 4. Project Documentation Noted by Survey Respondents ................................................................. 53 Table 5. Documentation Availability by Type .............................................................................................. 53 Table 6. Documentation Available from Other Emerging Transportation Technology Projects .................. 54 Table 7. Big Data Benchmark and Assessment Methodology ..................................................................... 55