Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
NCHRP Web-Only Document 279: Information Findability Implementation Pilots at State Transportation Agencies iv Contents 1.0 Introduction .................................................................................................................... 1 2.0 Washington State DOT Tests ............................................................................................ 2 2.1 Planning and Scoping ............................................................................................................ 2 2.2 Content Collection and Analysis ........................................................................................... 3 2.3 Solution Development and Testing..................................................................................... 21 2.4 Washington State DOT Implementation Plan ..................................................................... 46 3.0 Utah DOT Findability Tests ............................................................................................. 59 3.1 Planning and Scoping .......................................................................................................... 59 3.2 Content Collection and Analysis ......................................................................................... 62 3.3 Solution Development and Testing..................................................................................... 67 3.4 Utah DOT Implementation Plan .......................................................................................... 94 4.0 Iowa DOT Findability Tests ............................................................................................ 101 4.1 Planning and Scoping ........................................................................................................ 101 4.2 Content Collection and Analysis ....................................................................................... 102 4.3 Solution Development and Testing................................................................................... 104 4.4 Iowa DOT Implementation Plan ........................................................................................ 109 References ......................................................................................................................... 115 Appendix A. WSDOT Facets and Terms ............................................................................... 116 This document contains Technical Memo: Findability Pilots at State DOTs from the contractorâs final report on NCHRP Project 20-97A. The main body of the final report has been published as NCHRP Research Report 947: Implementing Information Findability Improvements in State Transportation Agencies. Readers can read or purchase NCHRP Research Report 947 at www.trb.org.
NCHRP Web-Only Document 279: Information Findability Implementation Pilots at State Transportation Agencies v LIST OF FIGURES Figure 1. Example section from the Temporary Erosion and Sediment Control Manual. ............. 7 Figure 2. PHP Script designed to convert a PDF document to a txt file or an html file. ................. 8 Figure 3. Boilerplate example. ........................................................................................................ 9 Figure 4. Pareto Analysis of word frequency in the WSDOT engineering manuals. .................... 10 Figure 5. Cross-referencing of clusters by manual. ...................................................................... 20 Figure 6. Culvert ontology. ............................................................................................................ 31 Figure 7. Content classification flowchart. ................................................................................... 34 Figure 8. Culvert Design Approach. .............................................................................................. 35 Figure 9. Content tagging validation process. .............................................................................. 37 Figure 10. Illustration of ontology relationships........................................................................... 38 Figure 11. Frequently occurring content types. ........................................................................... 63 Figure 12. Agreement types. ......................................................................................................... 63 Figure 13. Utility agreement types. .............................................................................................. 64 Figure 14. Utah DOT manuals cluster analysis - sample clusters. ................................................ 66 Figure 15. Utah DOT manuals cluster analysis: distribution of clusters by manual. .................... 66 Figure 16. Maintenance Agreement Example 1. .......................................................................... 71 Figure 17. Maintenance Agreement Example 2. .......................................................................... 72 Figure 18. Project ID, PIN, Date, and Party. .................................................................................. 82 Figure 19. Different patterns of Project ID, PIN, Date, and Party................................................. 83 Figure 20. Concept Report example. ............................................................................................ 86 Figure 21. Design Exception example. .......................................................................................... 87 Figure 22. Highway Easement Deed example. ............................................................................. 87 Figure 23. Quit Claim Deed example. ........................................................................................... 88 Figure 24. Warranty Deed example. ............................................................................................. 88 Figure 25. Document processing time and cost. .......................................................................... 91 Figure 26. Extracted entities and ProjectWise metadata comparison. ........................................ 94 Figure 27. IADOT Proposal example. .......................................................................................... 104 Figure 28. IADOT Plan example. ................................................................................................. 105
NCHRP Web-Only Document 279: Information Findability Implementation Pilots at State Transportation Agencies vi LIST OF TABLES Table 1. Description of the Manuals Included in the Analysis ........................................................ 3 Table 2. List of Frequently Used Terms-Single Words (18 Engineering Manuals) ........................ 11 Table 3. List of Frequently Used Terms-Two-Word Terms (18 Engineering Manuals)................. 13 Table 4. Definitions of Natural Language Processing Functions in Python NLTK ......................... 16 Table 5. Results of Cluster Analysis â 18 Manuals ........................................................................ 19 Table 6. WSDOTâs Master Deliverables List (First 2 levels)........................................................... 24 Table 7. WSDOTâs ECM Taxonomies ............................................................................................. 26 Table 8. Selected Facets and Scope for Rule Development ......................................................... 29 Table 9. Relationships among Terms in an Ontology ................................................................... 30 Table 10. Pilot Technical Components .......................................................................................... 32 Table 11. Match between Cluster Topics and Deliverables Expectation Matrix Topics............... 39 Table 12. Comparison of cluster terms between WSDOT and UDOT........................................... 67 Table 13. Maintenance Agreement Rules ..................................................................................... 72 Table 14. Utility Agreement Rules ................................................................................................ 74 Table 15. Betterment Agreement Rules ....................................................................................... 74 Table 16. Cooperative Agreement Rules ...................................................................................... 75 Table 17. Drainage Agreement Rules ........................................................................................... 76 Table 18. Machine Learning vs. Rule-based Classification Results ............................................... 79 Table 19. Text Extraction Logic ..................................................................................................... 84 Table 20. Concept Report Rules .................................................................................................... 86 Table 21. Design Exception Rules ................................................................................................. 89 Table 22. Highway Easement Deed Rules ..................................................................................... 89 Table 23. Quit Claim Deed Rules ................................................................................................... 89 Table 24. Warranty Deed Rules .................................................................................................... 89 Table 25. Exception Rule for Emails .............................................................................................. 90 Table 26. Round 2 Classification Results for the UDOT Corpus .................................................... 90 Table 27. Round 2 Entity Extraction Results for the UDOT Corpus .............................................. 92 Table 28. UDOT Implementation Plan Overview .......................................................................... 95 Table 29. Common OCR Character Errors ................................................................................... 103 Table 30. Iowa DOT Recall and Precision Results for Rule-Based Approach .............................. 106 Table 31. Iowa DOT Recall and Precision Results for Machine Learning Algorithms ................. 108 Table 32. IADOT Entity Extraction Results .................................................................................. 109 Table 33. IADOT Implementation Plan Overview ....................................................................... 110
NCHRP Web-Only Document 279: Information Findability Implementation Pilots at State Transportation Agencies vii LIST OF ACRONYMS AASHTO American Association of State Highway and Transportation Officials ADA Americans with Disabilities Act BMP Best Management Practice CMS Content Management System COTS Commercial-Off-the-Shelf DOT Department of Transportation ECM Engineering Content Management ERMS Electronic Records Management System ERP Enterprise Resource Planning FHWA Federal Highway Administration GIS Geographic Information System HTML Hypertext Markup Language IADOT Iowa Department of Transportation IT Information Technology LDA Latent Dirichlet Allocation MDL Master Deliverables List MP Milepost NCHRP National Cooperative Highway Research Program NLP Natural Language Processing NLTK Natural Language Toolkit OCR Optical Character Recognition OWL Web Ontology Language PDF Portable Document Format PIN Project Identification Number R/W Right of Way SQL Structured Query Language SME Subject Matter Expert SVM Support Vector Machines TESC Temporary Erosion and Sediment Control TF-IDF Term Frequency-Inverted Document Frequency TOC Table of Contents TRT Transportation Research Thesaurus UDOT Utah Department of Transportation WSDOT Washington State Department of Transportation
