Management and Use of Data for Transportation Performance Management: Guide for Practitioners (2019)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Foundation

Introduction • Foundation • Reporting • Insight • Cases Step 1 Specify & Define Data This step involves the up-front work to define data requirements for TPM. You may be seeking to add new performance measures, modify your existing measures, or improve the efficiency of your data gathering processes. Whenever a change to performance measures or associated data is contemplated, the following guidance can help to plan and scope improvements. Taking the time to work through each sub-step below will pay off in the form of a solid business case for change and reduced risk that re-work will be required in the future. “We don’t have time to do it right, but we always have time to do it over.” – Anon 13

Introduction • Foundation • Reporting • Insight • Cases Step 1.1 Define Need, Vision, & Scope Define business needs for data. Begin with the business objective(s) and concern(s) in mind and consider how your performance measures will be used to support them. The goal is to clearly articulate the business case for new data to agency managers and stakeholders. To do this, you need to answer three questions: • What will new data tell us? • How will we act on it? • Is the cost of obtaining the data worth the value that will be added? Define and document what information is needed to meet both internal agency decision-making requirements as well as external reporting and information-sharing requirements. • For agency decision support, document each decision: what is the decision, when is it made, who makes it, and who provides supporting information. Example decisions are which bridges to program for rehabilitation, which intersections to target for safety improvements, what percentage of available funding should be allocated to bridges versus pavements, and what strategies should be considered to address freight bottlenecks. • For external performance reporting requirements, document what needs to be reported, when reports are due, and the required format for the information. Include references to any applicable regulations or guidance documents. • For external performance information sharing, define what performance information the agency will share with the traveling public and with external partners. Document the intended uses of the information by each type of audience. Specify the data requirements. For each of the above business needs, identify the following: • What data attributes are essential for calculating performance measures, and what additional attributes might be helpful for providing context and interpreting performance results? • What scope of data coverage is needed? Tip Don’t limit your scope to the data needed to calculate performance measures. Also consider the data needed to understand trends or patterns, formulate strategies, and identify appropriate actions to improve performance. To support its TPM efforts and as part of a broader strategy for making effective use of data, the Mid-America Regional Council (MARC) data coordination committee compiled a top 10 list of high-priority data sets for automation. This top 10 created a road map for subsequent work activities to organize critical data sets at MARC. Priorities included pavement and bridge conditions, safety measures, and system performance. Case G 14

Introduction • Foundation • Reporting • Insight • Cases • What granularity of data is needed—what time period and spatial unit should each data observation represent? • What level of precision is required? • How current do data need to be—one year old? one day old? real time? Current federal TPM rules establish many of these requirements for pavement, bridge, safety, and system performance measures. However, many agencies choose to go beyond these minimum requirements or maintain additional measures to meet their own internal needs. For example, state DOTs generally track pavement condition for all paved roads they maintain, not just on the National Highway System, and use locally defined condition indices that support project-level decision making. Identify common data needs across business units. Rather than looking at the data requirements of individual business units in isolation, identify common needs. Socio-economic data, population forecasts, traffic data, roadway characteristics, and project status are examples of shared data needs. Engage each business unit in vetting of possible data sources and work toward a coordinated data gathering strategy. Carefully specify location attributes. Involve data users and geographic information system (GIS) staff in your agency in specifying location accuracy requirements and measurement methods. Be sure to understand the reasons for gathering data based on a linear reference rather than using Global Positioning System (GPS) locations. Linear references such as route-milepoint can provide convenient ways for someone without technology to find a feature. If only GPS coordinates are available, it can sometimes be difficult to determine a specific route location—for example, for complex interchanges. Consider data integration needs. Effective TPM rarely relies on a single isolated data set. Meaningful metrics often require integration of multiple data sets and types. For example, speed data alone may identify slowdowns and delay patterns, but it does little to provide insight as to why there are slowdowns. Combining speed data with incident and weather data can paint a more complete picture of system performance as it relates to recurring and non-recurring congestion. In order to integrate data sets effectively, agencies must focus on linkages between data sets using time and geography information, as well as available contextual information. For example, the location and time of the start of delay should be cross-referenced with available weather PennDOT identified five specific business goals that guided collaborative establishment of a Statewide Operations Data Warehouse that broke down agency data silos. Case J Maryland SHA found that linking disparate data sets enabled effective after action reviews to improve operational preparedness and response. Case E 15

Introduction • Foundation • Reporting • Insight • Cases conditions in that location and at that time, as well as understanding that the delay is happening during a busy holiday season in a retail area. Considering data integration needs early can save effort later that may be required to build integrated data sets for analysis. Key questions to address are the following: • What data sets need to be linked? • What data elements can be used for temporally and spatially linking the data sets? • What quality checks are required to make sure that these link elements are complete and accurate? • What data standards should be followed for newly acquired data so that these linkages can be made? Answering these questions will ensure that any new data collected are “integration-ready.” It is much easier to address integration needs up front rather than after data are collected. For more information... 1. Guide for Prioritizing Assets for Inclusion in Transportation Asset Management (TAM) Programs (FHWA, 2019—forthcoming) 2. Priorities in Roadway Safety Data Guide (FHWA, 2017) https://safety.fhwa.dot.gov/rsdp/downloads/fhwasa17032.pdf 3. Minnesota Department of Transportation Data Business Plan (Minnesota Department of Transportation, 2007) https://www.dot.state.mn.us/tda/databusinessplan.docx Tip Anticipate that an iterative approach will be needed to fully understand your data requirements. There is no substitute for having actual data in hand and trying to use it. That will inevitably lead to adjustments in specifications. 16

Introduction • Foundation • Reporting • Insight • Cases Step 1.2 Define Performance Measures Specify, test, and document performance measure calculations. For each performance measure, precisely document the data inputs and calculations needed. This documentation should have all of the information needed for a programmer/analyst to implement the calculations. Test the calculations with sample data and compare values and trends against other similar measures that may be available. Describe performance measures in plain English. Performance measures involving multiple data inputs and complex calculation logic should be documented in a manner that end users can understand. For example, the measure “Buffer Time Index” can be described as “the amount of extra buffer time a commuter needs to allow to avoid being late to work more than one day per month.” For more information... 1. National Performance Measures for Congestion, Reliability, and Freight, and CMAQ Traffic Congestion (FHWA, 2018) https://www.fhwa.dot.gov/tpm/guidance/hif18040.pdf 2. Validation of Pavement Performance Measures Using LTPP Data: Final Report (FHWA, 2018) https://www.fhwa.dot.gov/publications/research/infrastructure/ pavements/ltpp/17089/17089.pdf 3. Computation Procedure for the Bridge Performance Measures (FHWA, 2018) https://www.fhwa.dot.gov/tpm/guidance/hif18023.pdf 4. Freight Performance Measures Primer (FHWA, 2017) https://ops.fhwa.dot.gov/publications/fhwahop16089/fhwahop1 6089.pdf 5. Transportation Management Center—Data Capture for Performance and Mobility Measures Reference Manual (FHWA, 2013) https://rosap.ntl.bts.gov/view/dot/3373 Ohio DOT established “regain time” as a winter performance measure and defined it as the elapsed time from the end of the snow or ice event to the time at which speeds recover to typical levels. Regain time is the type of measure that is easily communicated to decision makers and the general public, yet it ties well with operational actions that directly influence it. Case I 17

Introduction • Foundation • Reporting • Insight • Cases Step 1.3 Identify Analysis & Reporting Requirements Understand data user needs. Consider the information needs of different audiences and identify how they want to consume this information. Conduct interviews or focus groups to learn about analysis and reporting needs and desired improvements. Specify and document data extracts and report formats. For some audiences, standard, static reports will be sufficient; others may need more flexible views of the data with the ability to drill down into details from a summary view or to obtain direct access to data via an application programming interface (API). Others may want to load detailed performance data into specialized analysis tools such as pavement management systems, safety analysis systems, or traffic simulation models. Identifying these different needs early can help to avoid unexpected data requests that may be difficult to satisfy once data systems and processes are established. For more information... 1. Road Safety Fundamentals—Unit 3: Measuring Safety (FHWA, 2017) https://rspcb.safety.fhwa.dot.gov/RSF/unit3.pdf 2. TRB Web Document 9: Meeting Critical Data Needs for Decision Making in State and Metropolitan Transportation Agencies— Summary of a Conference. Transportation Research Board Conference Proceedings on the Web (Transportation Research Board of the National Academies, 2013) http://onlinepubs.trb.org/onlinepubs/conf/CPW9.pdf To ensure MARC’s data developers understand how the data they manage is ultimately used, data personnel regularly participate in meetings with transportation planning staff, both to discuss high-level data needs and more focused detail- oriented breakout information. Case G 18

Introduction • Foundation • Reporting • Insight • Cases Capabilities Checklist: Specify & Define Data Basic o The business need for data has been identified and documentation of this need is available for future reference. o An inventory of existing agency data sources has been compiled. o Managers of the units responsible for data collection can describe the primary users and uses of that data. o Data requirements to meet internal and external performance reporting requirements are defined and documented, including attributes, scope, and granularity. o Location referencing methods for performance data are established to enable linkages with other agency data sets. o Update frequencies for new data are defined and documented. o Authoritative data sources have been designated for performance measure calculations. Advancing o Discussions about data requirements are not constrained by the status quo; they reflect what is important to know about transportation performance in order to improve. o Data needs are identified to support the entire TPM cycle (beyond performance reporting) including root cause analysis, identification and prioritization of improvements, and evaluation of impacts. o Minimum data quality standards are established considering timeliness, accuracy, completeness, consistency, and accessibility. o Data requirements are defined collaboratively across business units, including GIS and information technology. o Data communities of interest (or equivalent) have been established to identify data improvements to support different business needs. Do’s & Don’ts Do: Have a clear business case for new data that articulates how the data will be used to improve performance. Educate stakeholders on the benefits of performance measures. Put in the time and effort needed to nail down data requirements in a precise fashion—ambiguity can lead to downstream problems.  Implement small-scale pilots to test out new types of data and assess their value. Don’t:  Establish performance measures solely based on what data you have (instead, figure out what you would like to measure first and then assess options for getting the data you need).  Approach data requirements from the perspective of a single business unit (instead, try to identify common needs).  Go too lean on data requirements in order to save money in the short term and miss an opportunity to gain new insights that provide a bigger bang for the buck.  Neglect to understand location accuracy requirements (instead, investigate how location should be measured and represented). 19

Introduction • Foundation • Reporting • Insight • Cases Step 2 Obtain Data This involves activities related to obtaining the data needed to support the entire TPM process, including • data needed to calculate performance measures, • data needed to provide context necessary to understand performance trends, • data needed to understand root causes and factors contributing to performance results, • data needed to set realistic targets, and • data needed for selecting strategies to improve performance. These data may be obtained from existing internal, external, and commercial sources. New data may also be gathered using in-house resources and/or via contract. “Increasingly, data is gathered by information-sensing mobile devices, remote sensing, software logs, cameras, microphones, and wireless sensor networks. Global technological information per- capita capacity has approximately doubled every 40 months since the 1980s.” Institute of Engineering and Technology 20

Introduction • Foundation • Reporting • Insight • Cases Step 2.1 Assess & Select Data Sources Identify available data. Identify existing data sources that could be tapped to meet some or all of the requirements. Review sources within the agency, sources from external partners (federal agencies, state agencies, Metropolitan Planning Organizations (MPO), local agencies, universities), and commercial sources. Obtain detailed information about each source, including data elements and their definitions, scope, date of last update, frequency of updates, available formats, costs, and use restrictions. Evaluate available data sources against requirements. For each available data source, identify gaps between what is required and what the source can deliver. Develop estimates of the costs to fill the gaps through new data gathering activities. Use the list of gaps and the cost information to discuss • whether some of the requirements can be relaxed, • whether proposed performance measures can be modified to better align with available data, and • whether there are workarounds that can be applied (e.g., through sampling techniques or fusion of multiple data sources). Also, identify any constraints associated with data sources (e.g., usage restrictions) that could limit the value of those sources to support TPM. For more information... 1. Institution of Engineering and Technology: IET Sector Insights: Big Data in Transport https://www.theiet.org/sectors/transport/topics/intelligent- mobility/articles/big-data.cfm 2. Using Truck GPS Data for Freight Performance Analysis in the Twin Cities Metro Area (Minnesota Department of Transportation, 2014) http://www.dot.state.mn.us/research/TS/2014/201414.pdf 3. NCFRP Report 25: Freight Data Sharing Guidebook (Transportation Research Board of the National Academies, 2013) https://www.nap.edu/catalog/22569/freight-data-sharing-guidebook 4. Asset Management Data Collection for Supporting Decision Processes (FHWA, 2006) http://www.fhwa.dot.gov/asset/dataintegration/if08018/assetmgmt_web.pdf Ohio DOT wanted to track how quickly roads returned to normal speeds following a storm. They were able to leverage existing data sources, including their RWIS and commercial speed data. Case I 21

Introduction • Foundation • Reporting • Insight • Cases Step 2.2 Acquire Data It may be appropriate to launch a data acquisition effort if • current agency data sources will not meet the requirements, • there are no suitable commercial sources that meet the requirements (for an acceptable price), • there is a business case for new data collection, and • resources are available—both for initial collection and ongoing upkeep of the data. Once you have decided to collect new data, you must determine whether to collect the data with in-house personnel or outsource data collection to a vendor. This will depend on the scale of the effort and in- house staff capacity. Regardless of who will be collecting the data, it is essential to have a documented plan describing how it will be collected. Create a data collection and quality management plan. Prepare a detailed plan to guide both data collection and quality management activities (see Table 1 for suggested elements of such a plan). Data quality management takes additional time and effort but should be integral to the data collection process. Without sufficient resourcing for data quality, there is a risk that the data collected will not be usable. Remember that the people who collect the data are in the best position to ensure quality. Build in training activities so that they understand not only how to collect the data, but why the data are being collected and what the intended uses are. Check in with them during the data collection and see if they have suggestions for improving the process. Maryland SHA captures a rich set of data about highway incidents including the name of responders, the road surface conditions, lane closings/openings over the course of the incident, and operator notes. These data are then combined with data from ITS devices [Dynamic Message Signs (DMS), Closed Caption Television (CCTV) images, volume and speed detectors, signals] and probe-based speed data. Case E 22

Introduction • Foundation • Reporting • Insight • Cases Table 1. Contents of a data collection and quality management plan. Data Specifications Data element descriptions and allowable values Measurement methods Accuracy and precision Data formats Data Collection Procedures Schedule Scope Roles and responsibilities Procedures Issue reporting and resolution protocols Staffing Training and certification of data collectors Supervision and management roles Equipment Specifications Calibration Certification Quality Control Quality checks before and during collection Data Acceptance Acceptance criteria Error resolution procedures Data Review and Validation Review procedures and responsibilities Accuracy checks for sample records Independent verification Aggregation—check totals Field-level validation Record-level validation Check against prior value Visualization Reporting Data quality metrics and targets Data quality reporting protocols Error reporting procedures for data users 23

Introduction • Foundation • Reporting • Insight • Cases For more information... 1. Guidelines for Development and Approval of State Data Quality Management Programs (FHWA, 2018) https://www.fhwa.dot.gov/pavement/management/pubs/dqmp.pdf 2. Southeast Michigan Council of Governments—Innovative Traffic Data Quality Assurance/Quality Control Procedures and Automating AADT Estimation (Case Study) (FHWA, 2015) https://safety.fhwa.dot.gov/rsdp/downloads/fhwasa17035.pdf 3. Practical Guide for Quality Management of Pavement Condition Data Collection (FHWA, 2013) https://www.fhwa.dot.gov/pavement/management/qm/data_qm_gu ide.pdf 4. I-95 Corridor Coalition, Vehicle Probe Project Scope and Methodology (2009) http://i95coalition.org/wp- content/uploads/2015/02/Validation-Process-May-19-2009-distr- June-20092.pdf?x70560 24

Introduction • Foundation • Reporting • Insight • Cases Capabilities Checklist: Obtain Data Basic o Data collection procedures and protocols are defined and documented. o Data collection and processing workflows are mapped to clearly assigned responsibilities and deadlines. o Existing agency data sources are reviewed prior to collection of new data. o Available external (public and private) data sources are reviewed prior to collection of new data. o Quality management procedures are defined and documented— including training and certification for data collection personnel. o Requirements are in place that ensure new data collection adheres to agency location referencing standards. o Impacts of changes to existing data collection methods are assessed to minimize loss of consistent trend data and disruption to existing reports. o Data sources are assessed to understand usage restrictions that may limit value. Advancing o The full cost of new data acquisition is estimated, considering initial collection, ongoing updates, and supporting staff and technology infrastructure. o Funding for regular data updates (beyond the initial collection) is planned and committed. o There is regular communication with partner agencies to identify opportunities for collaboration on data collection. o Periodic scans are conducted to identify ways to improve data quality and collection efficiency. o Agency guidance and/or coordination protocols have been established to assist business units wishing to purchase commercial data sources. o Specialists with appropriate expertise (in-house or contractors) evaluate use of emerging private data sources. o Data requirements are defined with consideration of opportunities to create valuable information through integration of multiple data sources. Do’s & Don’ts Do: Coordinate across units within the agency prior to collecting new data to avoid duplication; plan for integration needs.  Invest in data quality management to make sure that data collected are reliable. Communicate with partner agencies to identify areas for collaboration. Periodically evaluate if there is a better way to get the data you need. Don’t:  Purchase private data without understanding its derivation and limitations.  Enter into a data use agreement with terms that are overly restrictive for your agency.  Change data collection methods without identifying how this may impact existing reports and the ability to understand trends. 25