National Academies Press: OpenBook

Guidance for Developing a Freight Transportation Data Architecture (2010)

Chapter: Chapter 2 - Data Sources, Systems, and Architectures

« Previous: Chapter 1 - Introduction
Page 17
Suggested Citation:"Chapter 2 - Data Sources, Systems, and Architectures." National Academies of Sciences, Engineering, and Medicine. 2010. Guidance for Developing a Freight Transportation Data Architecture. Washington, DC: The National Academies Press. doi: 10.17226/14466.
×
Page 17
Page 18
Suggested Citation:"Chapter 2 - Data Sources, Systems, and Architectures." National Academies of Sciences, Engineering, and Medicine. 2010. Guidance for Developing a Freight Transportation Data Architecture. Washington, DC: The National Academies Press. doi: 10.17226/14466.
×
Page 18
Page 19
Suggested Citation:"Chapter 2 - Data Sources, Systems, and Architectures." National Academies of Sciences, Engineering, and Medicine. 2010. Guidance for Developing a Freight Transportation Data Architecture. Washington, DC: The National Academies Press. doi: 10.17226/14466.
×
Page 19
Page 20
Suggested Citation:"Chapter 2 - Data Sources, Systems, and Architectures." National Academies of Sciences, Engineering, and Medicine. 2010. Guidance for Developing a Freight Transportation Data Architecture. Washington, DC: The National Academies Press. doi: 10.17226/14466.
×
Page 20
Page 21
Suggested Citation:"Chapter 2 - Data Sources, Systems, and Architectures." National Academies of Sciences, Engineering, and Medicine. 2010. Guidance for Developing a Freight Transportation Data Architecture. Washington, DC: The National Academies Press. doi: 10.17226/14466.
×
Page 21
Page 22
Suggested Citation:"Chapter 2 - Data Sources, Systems, and Architectures." National Academies of Sciences, Engineering, and Medicine. 2010. Guidance for Developing a Freight Transportation Data Architecture. Washington, DC: The National Academies Press. doi: 10.17226/14466.
×
Page 22
Page 23
Suggested Citation:"Chapter 2 - Data Sources, Systems, and Architectures." National Academies of Sciences, Engineering, and Medicine. 2010. Guidance for Developing a Freight Transportation Data Architecture. Washington, DC: The National Academies Press. doi: 10.17226/14466.
×
Page 23
Page 24
Suggested Citation:"Chapter 2 - Data Sources, Systems, and Architectures." National Academies of Sciences, Engineering, and Medicine. 2010. Guidance for Developing a Freight Transportation Data Architecture. Washington, DC: The National Academies Press. doi: 10.17226/14466.
×
Page 24
Page 25
Suggested Citation:"Chapter 2 - Data Sources, Systems, and Architectures." National Academies of Sciences, Engineering, and Medicine. 2010. Guidance for Developing a Freight Transportation Data Architecture. Washington, DC: The National Academies Press. doi: 10.17226/14466.
×
Page 25
Page 26
Suggested Citation:"Chapter 2 - Data Sources, Systems, and Architectures." National Academies of Sciences, Engineering, and Medicine. 2010. Guidance for Developing a Freight Transportation Data Architecture. Washington, DC: The National Academies Press. doi: 10.17226/14466.
×
Page 26
Page 27
Suggested Citation:"Chapter 2 - Data Sources, Systems, and Architectures." National Academies of Sciences, Engineering, and Medicine. 2010. Guidance for Developing a Freight Transportation Data Architecture. Washington, DC: The National Academies Press. doi: 10.17226/14466.
×
Page 27
Page 28
Suggested Citation:"Chapter 2 - Data Sources, Systems, and Architectures." National Academies of Sciences, Engineering, and Medicine. 2010. Guidance for Developing a Freight Transportation Data Architecture. Washington, DC: The National Academies Press. doi: 10.17226/14466.
×
Page 28
Page 29
Suggested Citation:"Chapter 2 - Data Sources, Systems, and Architectures." National Academies of Sciences, Engineering, and Medicine. 2010. Guidance for Developing a Freight Transportation Data Architecture. Washington, DC: The National Academies Press. doi: 10.17226/14466.
×
Page 29
Page 30
Suggested Citation:"Chapter 2 - Data Sources, Systems, and Architectures." National Academies of Sciences, Engineering, and Medicine. 2010. Guidance for Developing a Freight Transportation Data Architecture. Washington, DC: The National Academies Press. doi: 10.17226/14466.
×
Page 30
Page 31
Suggested Citation:"Chapter 2 - Data Sources, Systems, and Architectures." National Academies of Sciences, Engineering, and Medicine. 2010. Guidance for Developing a Freight Transportation Data Architecture. Washington, DC: The National Academies Press. doi: 10.17226/14466.
×
Page 31
Page 32
Suggested Citation:"Chapter 2 - Data Sources, Systems, and Architectures." National Academies of Sciences, Engineering, and Medicine. 2010. Guidance for Developing a Freight Transportation Data Architecture. Washington, DC: The National Academies Press. doi: 10.17226/14466.
×
Page 32
Page 33
Suggested Citation:"Chapter 2 - Data Sources, Systems, and Architectures." National Academies of Sciences, Engineering, and Medicine. 2010. Guidance for Developing a Freight Transportation Data Architecture. Washington, DC: The National Academies Press. doi: 10.17226/14466.
×
Page 33
Page 34
Suggested Citation:"Chapter 2 - Data Sources, Systems, and Architectures." National Academies of Sciences, Engineering, and Medicine. 2010. Guidance for Developing a Freight Transportation Data Architecture. Washington, DC: The National Academies Press. doi: 10.17226/14466.
×
Page 34
Page 35
Suggested Citation:"Chapter 2 - Data Sources, Systems, and Architectures." National Academies of Sciences, Engineering, and Medicine. 2010. Guidance for Developing a Freight Transportation Data Architecture. Washington, DC: The National Academies Press. doi: 10.17226/14466.
×
Page 35
Page 36
Suggested Citation:"Chapter 2 - Data Sources, Systems, and Architectures." National Academies of Sciences, Engineering, and Medicine. 2010. Guidance for Developing a Freight Transportation Data Architecture. Washington, DC: The National Academies Press. doi: 10.17226/14466.
×
Page 36
Page 37
Suggested Citation:"Chapter 2 - Data Sources, Systems, and Architectures." National Academies of Sciences, Engineering, and Medicine. 2010. Guidance for Developing a Freight Transportation Data Architecture. Washington, DC: The National Academies Press. doi: 10.17226/14466.
×
Page 37
Page 38
Suggested Citation:"Chapter 2 - Data Sources, Systems, and Architectures." National Academies of Sciences, Engineering, and Medicine. 2010. Guidance for Developing a Freight Transportation Data Architecture. Washington, DC: The National Academies Press. doi: 10.17226/14466.
×
Page 38
Page 39
Suggested Citation:"Chapter 2 - Data Sources, Systems, and Architectures." National Academies of Sciences, Engineering, and Medicine. 2010. Guidance for Developing a Freight Transportation Data Architecture. Washington, DC: The National Academies Press. doi: 10.17226/14466.
×
Page 39
Page 40
Suggested Citation:"Chapter 2 - Data Sources, Systems, and Architectures." National Academies of Sciences, Engineering, and Medicine. 2010. Guidance for Developing a Freight Transportation Data Architecture. Washington, DC: The National Academies Press. doi: 10.17226/14466.
×
Page 40
Page 41
Suggested Citation:"Chapter 2 - Data Sources, Systems, and Architectures." National Academies of Sciences, Engineering, and Medicine. 2010. Guidance for Developing a Freight Transportation Data Architecture. Washington, DC: The National Academies Press. doi: 10.17226/14466.
×
Page 41
Page 42
Suggested Citation:"Chapter 2 - Data Sources, Systems, and Architectures." National Academies of Sciences, Engineering, and Medicine. 2010. Guidance for Developing a Freight Transportation Data Architecture. Washington, DC: The National Academies Press. doi: 10.17226/14466.
×
Page 42
Page 43
Suggested Citation:"Chapter 2 - Data Sources, Systems, and Architectures." National Academies of Sciences, Engineering, and Medicine. 2010. Guidance for Developing a Freight Transportation Data Architecture. Washington, DC: The National Academies Press. doi: 10.17226/14466.
×
Page 43
Page 44
Suggested Citation:"Chapter 2 - Data Sources, Systems, and Architectures." National Academies of Sciences, Engineering, and Medicine. 2010. Guidance for Developing a Freight Transportation Data Architecture. Washington, DC: The National Academies Press. doi: 10.17226/14466.
×
Page 44
Page 45
Suggested Citation:"Chapter 2 - Data Sources, Systems, and Architectures." National Academies of Sciences, Engineering, and Medicine. 2010. Guidance for Developing a Freight Transportation Data Architecture. Washington, DC: The National Academies Press. doi: 10.17226/14466.
×
Page 45
Page 46
Suggested Citation:"Chapter 2 - Data Sources, Systems, and Architectures." National Academies of Sciences, Engineering, and Medicine. 2010. Guidance for Developing a Freight Transportation Data Architecture. Washington, DC: The National Academies Press. doi: 10.17226/14466.
×
Page 46
Page 47
Suggested Citation:"Chapter 2 - Data Sources, Systems, and Architectures." National Academies of Sciences, Engineering, and Medicine. 2010. Guidance for Developing a Freight Transportation Data Architecture. Washington, DC: The National Academies Press. doi: 10.17226/14466.
×
Page 47
Page 48
Suggested Citation:"Chapter 2 - Data Sources, Systems, and Architectures." National Academies of Sciences, Engineering, and Medicine. 2010. Guidance for Developing a Freight Transportation Data Architecture. Washington, DC: The National Academies Press. doi: 10.17226/14466.
×
Page 48

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

17 Introduction This chapter includes a discussion of systems, databases, and architectures that might be used as a potential reference for the development of a national freight data architecture. As previously described, the purpose of NCFRP Project 12 was to develop specifications for a national freight data architec- ture that could support the decisionmaking process by both public and private stakeholders not just at the national level, but also at the state, regional, and local levels. As the real- world examples in Chapter 1 showed, supporting such a wide spectrum of decisionmaking needs requires using a wide range of data sources. The literature review in this chapter includes a general description of available data sources as well as a detailed discussion of several systems and architectures that were of particular interest because of the processes that led to their development, which could be used as lessons learned while developing a national freight data architecture. Data Sources A variety of listings, links, and summaries of systems, data- bases, architectures, and other related documents that pertain to freight transportation data are available in the literature. A small sample of documents that contain freight transportation data-related listings includes the following: • FHWA listings of freight data sources (9, 10); • FHWA report: Quick Response Freight Manual II (11); • BTS report: Directory of Transportation Data Sources (12); • BTS report: A Preliminary Roadmap for the American Freight Data Program (13); and • Texas Department of Transportation report: State-of-the- Practice in Freight Data: A Review of Available Freight Data in the U.S. (14). Although there is a wealth of sources of information that pertain to freight transportation, a comprehensive catalog of freight-related data sources at different geographic levels (including national, state, regional, and local levels) does not exist. As a reference, the research team conducted a review of a sample of freight-related data sources at the national level to complement or otherwise extend existing listings. Table 1 provides the following information: • Freight data source/dataset. This column provides a list- ing of freight transportation data sources reviewed in this report. In total, the research team reviewed 49 freight data sources. For convenience, Table 1 groups data sources according to the following categories: public-sector data sources; private-sector data sources; freight-related archi- tectures, frameworks, programs, and standards; and initia- tives under development. • Agency in charge. This column provides the name of the agency responsible for maintaining and/or publishing the data. • Data subject covered. These columns indicate whether the data source covers specific data subjects. The data sub- jects are associated with freight transportation components (i.e., cargo or freight, vehicle or container, transportation network, and traffic control system), which are part of the freight data architecture described in this report. • Transportation mode covered. These columns indicate whether the data source covers specific modes of freight transportation (i.e., air, rail, truck, water, and pipeline). Note that the combination between cargo (under data subject covered) and transportation mode provides an indication of the mode of transportation used to trans- port commodities. It is worth noting that the list of freight data sources in Table 1 is primarily at the national level and is not comprehen- sive. In practice, state, regional, and local entities collect and maintain datasets such as truck counts, commercial vehicle inventory datasets, accident data, and facility data (e.g., data C H A P T E R 2 Data Sources, Systems, and Architectures

18 Freight Data Source/Dataset Acronym Agency in Charge Data Subject Covered Transportation Mode Covered C ar go /F re ig ht V eh ic le / C on ta in er Tr an sp or ta tio n N et w or k Tr af fic C on tr ol Sy st em A ir R ai l Tr uc k W at er Pi pe lin e Public Sector Agricultural Market Service Publications U.S. Department of Agriculture (USDA) Agricultural Marketing Service x x x x x Air Carrier Data BTS x x Automated Commercial Environment/International Trade Data System ACE/ ITDS U.S. Customs and Border Protection (CBP) x x x x x x x Automated Commercial System ACS CBP x x x x x x Automated Export System AES CBP x x x x x Bureau of Labor Statistics (BLS) Databases BLS x x x x x x Bureau of Transportation Statistics Publications BTS x x x x x x x Commodity Flow Survey CFS BTS, U.S. Census Bureau x x x x x Economic Accounts (including the National Income and Product Accounts [NIPAs]) U.S. Bureau of Economic Analysis (BEA) x x x x x Economic Census U.S. Census Bureau x x x x x x Energy Information Administration Data Services Energy Information Administration x x x x x Fatality Analysis Reporting System FARS NHTSA x x Freight Analysis Framework FAF2 FHWA x x x x x x Hazardous Materials Information System HMIS PHMSA x x x x x x x Highway Performance Monitoring System HPMS FHWA x x x Motor Carrier Financial and Operating Data FMCSA x x x Motor Carrier Management Information System MCMIS FMCSA x x x Motor Carrier Safety Status Measurement System SafeStat FMCSA x x National Automotive Sampling System NASS NHTSA x x National Hazardous Material Route Registry NHMRR FMCSA x x National Pipeline Mapping System NPMS PHMSA x x x National Transportation Atlas Database NTAD BTS x x x x x x Navigation Data Center Waterborne Commerce Data U.S. Army Corps of Engineers (USACE) x x x x x North American Transborder Freight Database BTS x x x x x x Railroad Data (including the Carload Waybill Sample) Surface Transportation Board (STB) x x x x Service Annual Survey U.S. Census Bureau x x x Statistics Canada Statistics Canada x x x x x x x x Surface Transportation Board Economic Data and Tools STB x x x TradeStats Express International Trade Administration x x x x x x TranStats BTS x x x x U.S. Census Bureau Foreign Trade Statistics U.S. Census Bureau x x x x x x Vehicle Travel Information System VTRIS FHWA x x Workforce Information Database WID States x x x x x Private Sector Association of American Railroads (AAR) Publications AAR x x x x American Trucking Associations Monthly Reports ATA x x Colography Group Services Colography Group, Inc. x x x IHS Global Insight Services IHS Global Insight x x x x x x Intermodal Association of North America Information Services Lloyd’s MIU Services Intermodal Association of North America (IANA) x x x x x Lloyd’s MIU x x x Table 1. Freight-related data sources described in this report.

19 Freight Data Source/Dataset Acronym Agency in Charge Data Subject Covered Transportation Mode Covered C ar go /F re ig ht V eh ic le / C on ta in er Tr an sp or ta tio n N et w or k Tr af fic C on tr ol Sy st em A ir R ai l Tr uc k W at er Pi pe lin e Port Import Export Reporting Service PIERS United Business Media (UBM) Global Trade x x x State of Logistics Report Council of Supply Chain Management Professionals (CSCMP) x x Worldwide Airport Traffic Reports Airports Council International (ACI) x x Freight-Related Architectures, Frameworks, Programs, and Standards Commodity, Product, and Industry Classifications Several x x x x x x Electronic Data Interchange Standards EDI Several x x x x x x x National ITS Architecture Research and Innovative Technology Administration (RITA) x x National Spatial Data Infrastructure NSDI Federal Geographic Data Committee (FGDC) x x x x x x x Initiatives under Development Electronic Freight Management EFM FHWA x x x x x Freight Performance Measurement FHWA x x Multimodal Hazmat Intelligence Portal HIP PHMSA x x x x x x x Table 1. (Continued). about ports, warehouses, and crossings). These datasets sup- plement national-level datasets. Likewise, Table 1 does not include data from trade associations, such as the National Industrial Transportation League (NITL) and the National Association of Retailers. Although the list does not include all of the potential data sources that deal with freight trans- portation, the list is useful because it provides a sample of the typical national-level data sources that may need to be evalu- ated in detail while building the national freight data archi- tecture, as well as any potential system implementations that could be derived from that data architecture. Readers should also note that some data sources in Table 1 might include multiple datasets. For compactness, the table does not disaggregate data sources into datasets. For exam- ple, Table 1 does not show all of the datasets associated with the National Transportation Atlas Database or that may be available through Statistics Canada. Furthermore, relationships between freight data, data source, and business processes are too complex for a single table or diagram. The number of business processes that deal with freight transportation at any given point in time (ranging from planning to policymaking, operations, and emergency management) is huge. Providing a single table that illustrates all of the relationships between freight data and business processes would be impractical—if not impossible— to develop. System and Architecture Review The following systems and architectures in Table 1 were of particular interest because of the processes that led to their development: • Automated Commercial Environment/International Trade Data System, • Carload Waybill Sample, • Commodity Flow Survey, • Electronic Data Interchange Standards, • Freight Analysis Framework, • Highway Performance Monitoring System, • National Income and Product Accounts, • National ITS Architecture, • National Spatial Data Infrastructure, and • National Transportation Atlas Database. Lessons learned from the development and implementation of these systems and architectures can provide invaluable infor- mation for, and help to minimize the costs of, the development

and implementation of a national freight data architecture. This section presents a summary of the analysis completed on those systems and architectures. The analysis covered several topics, including the following: • Purpose and intended benefits; • Content; • Institutional arrangements used for developing and main- taining the system or architecture; • Challenges and issues faced in creating and maintaining the architecture or system; • Strategies and methods for dealing with data integration issues, such as data quality, timeliness, and proprietary and privacy concerns; • Adaptability to serve evolving purposes and data sources; and • Assessment of how well the system or architecture works in the form of lessons learned. In reality, institutional arrangements, issues faced during development, implemented strategies, and adaptability are interrelated. The reason is that, historically, systems tend to evolve and strategies are put in place not just to meet the goals and objectives of an initial master plan but also in response to challenges and issues faced during the implemen- tation and/or maintenance phases of those systems. For con- venience, to avoid redundancy in the presentation, and for readability purposes, each analysis in this section includes three subsections, as follows: • Purpose and content; • Development, challenges, strategies, and adaptability; and • Lessons learned. Automated Commercial Environment/International Trade Data System Purpose and Content ACE is a trade data processing system that CBP is imple- menting to support customs activities at U.S. borders (15). The ACE effort is a multi-year, multi-million-dollar project that is replacing the 1984 ACS legacy system. ACE uses a secure data portal that enables the trade com- munity and participating government agencies (PGAs) to connect to the ACE database as well as to legacy databases (Figure 5). ACE provides a single, centralized access point for communications and information related to cargo shipments. Through the portal, it is possible to manage accounts, perform periodic payments, enter data for electronic truck manifests (also called e-Manifests), and generate reports. E-Manifests enable tracking of crew, equipment, shipper, consignee, and shipment data. E-Manifests are now required when entering the country through any of the 99 land border ports of entry. CBP plans to extend ACE to provide cargo processing capa- bilities across all modes of transportation, replacing existing systems with a single, multimodal manifest system for land, air, rail, and sea cargo. 20 Figure 5. ACE framework (16).

ITDS is a federal program that encourages PGA participa- tion in ACE (17). The program assists PGAs in identifying, documenting, and executing plans to improve business oper- ations through their participation in ACE. Currently, 46 PGAs are involved in the ITDS program. Nearly 500 users from 27 PGAs have access to the ACE portal. One of the mecha- nisms the ITDS program uses to support the integration of PGAs into ACE is through the development of the ACE/ITDS standard dataset. This dataset is a collection of data require- ments for international trade and U.S. border regulatory and enforcement processes. Its purpose is to ensure data harmo- nization to facilitate the full implementation of ACE across all relevant federal agencies. CBP is working to align the dataset with the international data standards developed by the World Customs Organization (WCO). Development, Challenges, Strategies, and Adaptability Critical milestones in the creation and development of ACE and ITDS are the following: • In 1993, the U.S. Customs Service commissioned a report (Future Automated Commercial Environment Team [FACET] Report) to make recommendations for the redesign of its commercial processing systems. • In 1994, a multi-agency task force composed of representa- tives of 53 agencies was formed to develop recommendations for implementing an international trade data system that could meet the needs of the federal government, business community, and public. • In 1995, vice presidential memoranda chartered the ITDS Project Office in the Department of the Treasury as well as a multi-agency ITDS board of directors. Over time, this management structure has evolved to include committees, working groups, and integrated product teams, frequently with PGA participation. • In 1998, formal design and concept of operations documents were prepared and a pilot system called the North American Trade Automation Prototype (NATAP) was approved to demonstrate the benefits of ITDS. • In 2000, the ITDS Project Office was transferred to the U.S. Customs Service and its goals were refined to fit better into the U.S. Customs Service operational environment. • In 2001, ACE started with an initial appropriation of $130 million (18). Development of ACE started the same year. In 2004, it was estimated that the full deployment of ACE would be completed by December 2007 at an estimated cost of $2.24 billion (19). • In August 2001, the ITDS pilot project went live in Buffalo, NY. The pilot project was suspended the following month due to operational considerations at the port following the September 11, 2001, terrorist attacks. • In 2003, the first ACE portal accounts were established. • In 2003, the U.S. Customs Service moved to DHS and became U.S. Customs and Border Protection. Following this transition, the ITDS program focus changed to pro- vide support to the integration of PGAs into ACE. As pre- viously mentioned, one of the mechanisms to promote this integration is through the development of the ACE/ITDS standard dataset. • In 2004, ACE e-Manifest was deployed in Blaine, WA (20). Since then, e-Manifest has been deployed at all 99 land bor- der ports of entry. In 2006, CBP conducted an evaluation of the e-Manifest initiative (21). From surveys, site visits, and telephone interviews, the study found that using elec- tronic manifests resulted in smoother border crossing oper- ations, a lower number of secondary inspections, and a higher number of post-secondary inspections. Developing the ACE/ITDS dataset was a significant chal- lenge over a 2-year period, which involved compiling a list of data elements from PGAs, clarifying data element definitions required by each PGA, working with PGAs to identify and eliminate overlapping data requirements, and translating those data elements into specific software requirements while ensur- ing consistency with ACE (22). From the 10,000 data elements that PGAs identified, ACE/ITDS staff reduced the number of required data elements by 96 percent to a standard dataset of 400 data elements. CBP is also working to harmonize the ITDS standard dataset with the WCO data model. As mentioned, 46 PGAs are currently involved in the ITDS program. Different PGAs are at different stages of ACE inte- gration (23). Several U.S.DOT operating administrations plan to access CBP data through an interface between ACE and the U.S.DOT’s planned International Freight Data System (IFDS) (24), with different levels of access depending on the statutory authority of each U.S.DOT operating administration. For example, • BTS plans to use ACE data such as entry data from importers, manifest data from carriers, and carrier contact informa- tion from ACE carrier account tables to conduct a variety of statistical analyses. • FHWA plans to access summary and manifest data to ana- lyze cargo and conveyance movements in order to better allocate resources among states. • FMCSA plans to access ACE data to analyze international truck freight flows in connection with enforcement activi- ties and the allocation of federal resources among state motor carrier safety agencies. The interface between CBP and FMCSA is currently undergoing testing to analyze the 21

volume of screening issues and system screening perform- ance. Over the next 2 years, various functions will be phased in, including screening of manifest information, notifica- tions to carriers, and warnings to send vehicles to an FMCSA inspection facility. RITA is the primary agency responsible for developing and managing IFDS. Lessons Learned Lessons learned in connection with the development, evo- lution, and maintenance of ACE and ITDS follow: • Develop robust implementation plan with adequate stake- holder involvement. By all accounts, ACE has been a huge undertaking. Developing a brand new system to replace the myriad of ad hoc programs and procedures that have evolved for decades at Customs (while taking into consid- eration the needs of all affected stakeholders) is not a triv- ial task. Despite the magnitude and complexity of the proj- ect, CBP essentially relied on its external contractors during the development phase without ensuring adequate PGA par- ticipation. This lack of coordination affected the ACE/ITDS development process. • Clearly define expected outcomes and development and coordination plan. The process to consolidate 10,000 data elements into a list of 400 data elements was highly itera- tive and required the involvement of a large number of stake- holders. Therefore, it was critical to identify the expected goals and outcomes as well as the procedures for coordina- tion and data element conflict resolution. However, the process was not clearly defined, eventually resulting in two versions of the dataset. There also were issues related to dataset ownership and responsible-party designation to modify and/or add data elements. • Address needs of stakeholders. Developing an accurate understanding of the data needs of affected stakeholders is an important project development requirement. For the development of the ACE/ITDS standard dataset, it was crit- ical to properly document the needs of all the PGAs involved. However, there were inconsistencies in the process, which resulted in some data needs not being properly identified. Carload Waybill Sample Purpose and Content The Carload Waybill Sample is a stratified sample of car- load waybills for terminated shipments at railroad carriers (25). STB is the agency responsible for the management of the Carload Waybill Sample. Railinc, Corp., a wholly-owned sub- sidiary of AAR, is under contract with STB for the production of the sample. The Carload Waybill Sample captures data about O-D points, number of carloads, tonnage, participat- ing railroads, interchange locations, and total freight revenue. The sample is one of the main sources of information for the development of trip generation estimates and is often used by regulators, planners, nongovernmental agencies, and other stakeholders. The sampling rate for carload waybill samples is a function of the number of carloads per waybill and the method the railroad uses to submit the documentation (i.e., manually or using a computerized system) (25, 26). Table 2 lists current sampling rate requirements. The vast majority of railroads submit sample data electronically. Because of the threshold for submission, the Carload Waybill Sample does not account for many Class II or III railroads. In 2007, there were 565 freight railroads in the United States, with only 63 freight railroads filing a sample of waybills (27, 28). The sample does not cap- ture data from export shipments carried on Canadian rail- roads operating inside the United States. The Carload Way- bill Sample has increased in size over the years, from 346,903 in 1986 to 666,989 in 2007 (27). The stratified sample of carload waybills provides informa- tion about shipments by rail, including STCC codes, origins, and destinations. The sample results in two types of files as follows (26): • Master file. The master file contains movement-specific confidential waybill data and is therefore limited to author- ized users as required by Code of Federal Regulations (CFR) Title 49 Section 1244 (49 CFR 1244). Current regulations 22 Reporting Method Number of Carloads per Waybill Expected Sampling Rate Manual 15 1/100 Manual 625 1/10 Manual 26 and more 1/5 Computerized 12 1/40 Computerized 315 1/12 Computerized 1660 1/4 Computerized 61100 1/3 Computerized 101 and more 1/2 Table 2. Carload waybill sampling rates (25, 26).

for use of the master file pertain to the protection of spe- cific shipper or carrier data that are considered proprietary. The master file includes 176 data items. • Public use file. The public use file is an aggregated, less detailed file that contains non-confidential data and is avail- able to the public without restrictions. This file removes several fields to shield confidential data and provides the data in a geographically aggregated manner. The public use data file includes 63 data items. Development, Challenges, Strategies, and Adaptability Shipper freight movement data have been collected and analyzed since the late 1800s (27). The Interstate Commerce Commission (ICC) was responsible for these data until 1995, when it was replaced by STB. The Carload Waybill Sample has been continuously collected since 1946. As required in 49 CFR 1241-1248, railroads must submit reports to document their operations to STB (26), as follows: • 49 CFR 1241 requires Class I railroads to submit annual financial data, covering elements such as total revenue, inventory of equipment, track and traffic conditions, and mileage (26, 29). • 49 CFR 1243 requires Class I railroads to submit quarterly reports documenting revenues, expenses, income, fuel costs, fuel consumption, and fuel surcharges. • 49 CFR 1244 requires railroads terminating at least 4,500 cars per year or that transport at least 5 percent of any state’s total traffic to submit carload waybill samples. Railroads must submit waybill samples at least quarterly. These samples are the basis for the STB Carload Waybill Sample (25, 26). • 49 CFR 1245 requires Class I railroads to submit quar- terly and annual reports of railroad employees, service, and compensation. • 49 CFR 1246 requires Class I railroads to submit monthly reports of the number of railroad employees. • 49 CFR 1248 requires Class I railroads to submit quarterly and annual freight commodity statistics using CCTS codes issued by the Bureau of the Budget (i.e., the predecessor of the Office of Management and Budget [OMB]). Railroads must report on a number of data elements for each com- modity code, including revenue, number of carloads, and tonnage. Railroads also must report on the average num- ber of miles operated and gross freight revenue. In addition, as required by 49 CFR 225, all railroads regardless of size must submit safety reports to FRA (30). As previously mentioned, the Carload Waybill Sample is one of the main sources of information for the development of trip generation estimates and is often used by regulators, planners, nongovernmental agencies, and other stakeholders. The sam- ple is also used for the calculation of the Rail Cost Adjustment Factor (RCAF), which measures the rate of inflation in railroad inputs such as labor and fuel and is therefore used to determine shipment rate adjustments (31, 32). AAR submits all of the RCAF components to STB for review and approval, first as a forecast and then actual data two quarters later. Lessons Learned Lessons learned in connection with the development, evo- lution, and maintenance of the Carload Waybill Sample follow: • Provide clear, uniform guidance. The regulations clearly identify which freight railroads are required to submit a Carload Waybill Sample. They also clearly state who is eli- gible to receive the data and the restrictions for those par- ties in which to use the data. In addition, both STB and Railinc have developed several documents that assist rail- roads in the understanding of the reporting requirements. • Develop mechanisms that facilitate use of data for vari- ous purposes but also maintain necessary confidential- ity. The Carload Waybill Sample data are both necessary and desired by a variety of users. The guiding regulations clearly define five classes of users of the data and statutory requirements for each group to use the data. This regula- tion is designed to maintain confidentiality of the data while also providing critical information. Relevance of the data is also maintained by the role that Railinc plays, which includes collecting and processing data on behalf of the railroads and submitting the data collected to STB. Rail- inc also provides real-time data exchange services to the railroad industry. Commodity Flow Survey Purpose and Content CFS is a joint effort between the U.S. Census Bureau and BTS to gather and compile data on the movement of goods in the United States (33–36). CFS is a shipper-based survey that gathers data from shipments in the United States. With the exception of operating status and the verification of name and location, CFS does not collect data on shipper or receiver descriptors. CFS includes the following shipment data: • Shipment ID number, date, value, and weight; • SCTG commodity code; • Commodity description; • Destination (and port of exit in the case of exports); • Mode(s) of transportation; • Mode of export; and • Hazardous material (hazmat) code. 23

CFS collects shipment data from a sample of establish- ments selected from the U.S. Census Bureau Business Regis- ter. These establishments are from manufacturing, mining, wholesale, select retail and service industries (electronic shop- ping, mail-order houses, and fuel dealers), and auxiliary estab- lishments (i.e., warehouses and managing offices) of multi- establishment companies. CFS does not include establishments from the following industries: crude petroleum and natural gas extraction, farms, government establishments, trans-border shipments, imports (until the shipment reaches the first domestic shipper), and remaining service industries. Many of these industries (e.g., farms and government establishments) are not included in the Business Register. Each establishment selected is mailed a questionnaire four times during the year. For each questionnaire, the establishment provides specific data about a sample of individual outbound shipments dur- ing a pre-specified 1-week period. CFS data are available at several levels of geographic reso- lution, such as national, state, metropolitan area, and census regions and divisions. Key statistics from CFS include the following: • Value, tons, ton miles, average miles per shipment; • Commodity shipped; • Modes of transportation; and • O-D flows. CFS data are used to assess demand on existing transporta- tion systems and assist with critical investments in future transportation facilities and services. For example, CFS data are used to build truck O-D trip tables, for traffic simulation analyses, to benchmark the Carload Waybill Sample (37), and as input to the Freight Analysis Framework. Commercial data- bases such as TRANSEARCH® Insight also use CFS data. Development, Challenges, Strategies, and Adaptability CFS is a component of the 5-year U.S. Census Bureau’s Economic Census. It was first conducted in 1993. Between 1963 and 1977, information about commodities transported in the United States was collected through a survey of Amer- ican businesses as part of the economic census. However, due to data reliability issues, this survey was last published in 1977 (38). Data reliability problems also affected a smaller com- modity transportation study in 1983, which caused the U.S. Census Bureau not to publish the results (38). In 1991, a TRB report identified the lack of commodity flow data as one of the greatest gaps in the U.S.DOT data program (39). Follow- ing its creation that year, BTS instituted CFS and arranged with the U.S. Census Bureau to conduct the survey as part of the economic census (38). BTS provides 80 percent of the funding, while the U.S. Census Bureau provides the remain- ing 20 percent (35). CFS has been conducted four times, as follows: • In 1993, the CFS sample size was about 200,000 establish- ments based on a Standard Industrial Classification (SIC) stratification. The 1993 CFS used STCC codes. The budget for the 1993 CFS was $15 million. • In 1997, the CFS sample size was reduced to about 100,000 establishments based on a SIC-based industry group strat- ification. The response rate was 75 percent. The 1997 CFS used SCTG codes. The reporting period was reduced to 1 week (from 2 weeks required in the 1993 CFS). The budget for the 1997 CFS was $19 million. • In 2002, the CFS sample size was reduced to about 50,000 establishments based on a North American Industry Clas- sification System (NAICS) stratification. The response rate was about 70 percent. The 2002 CFS used SCTG codes. The budget for the 2002 CFS was $13 million. • In 2007, the CFS sample size was increased to about 100,000 establishments based on an NAICS-based stratification. The 2007 CFS used SCTG codes. Although CFS is widely used for a variety of applications, it has some limitations, including the following (35, 37): • Gaps in shipment and industry coverage. CFS does not collect shipment data for certain industries and commodi- ties, and does not collect shipment data for shipments pass- ing through the United States. In addition, cross-border shipment paths only include U.S. mileage. According to a 2002 estimate, non-CFS shipments were 36 percent by value, about 40 percent by tonnage, and about 29 percent by ton-miles (36). Further, the survey does not capture route information beyond shipper and receiver locations, which makes estimating intermodal drayage components difficult. In general, intermodal freight volumes may be low due to the CFS definition of “intermodal.” • Lack of geographic and commodity detail at the state and local levels. There is widespread agreement that increased geographic and commodity detail at the state and local levels would greatly enhance the usefulness of the survey. The challenge is how to determine the optimum level of disaggregation. Geographic strata for the 1993 and 1997 CFSs included 89 national transportation analysis regions (NTARs). These regions were consolidated 1987 BEA economic areas to keep O-D tables within 8,000 cells. Geographic strata in the 2002 CFS included the top 50 metropolitan areas (MAs) based on population in the 2000 Census, with establishments not located in an MA 24

assigned to the remainder of the state. Geographic strata in the 2007 CFS will use 73 MAs, with establishments not located in an MA assigned to the remainder of the state. Several ideas have been suggested to increase CFS regions, including using three-digit zip code regions (of which there are 929 around the country) and BEA areas (of which there are 172 around the country) (37). A recent study of techniques to generate national freight analysis zones (FAZs) for transportation models recommended a system of 400 zones (40). • Insensitivity to short-term economic changes. CFS follows a 5-year cycle, which is inadequate for freight analyses in connection with phenomena such as recessions or droughts. The 2-year lag between data collection and release of results is also a weakness. Lessons Learned Lessons learned in connection with the development, evo- lution, and maintenance of CFS follow: • Continue to involve the U.S. Census Bureau for the use of survey instruments. CFS is a joint effort between the U.S. Census Bureau and BTS. According to BTS, this partner- ship has been beneficial because the U.S. Census Bureau had previous experience conducting commodity-based surveys, an establishment list, and in-house resources for data collection (37). • Create crosswalks to ensure the compatibility of survey data internally over time and externally across other datasets. Over time, key CFS characteristics have changed, such as sample size, industry classification, commodity classification, survey methodology, and data items. A doc- umented crosswalk between CFS surveys to link survey data over time is needed (37). In addition, while CFS is a shipment survey, other surveys that can be used to supple- ment CFS data (e.g., carrier surveys) contain data that are not necessarily compatible with the CFS data structure. There is also a need for an integrated data collection pro- gram and coordination on definitions for commodity codes and vehicle types (37). • Consider importance of adequate resources for data col- lection and fully understanding implications of small sample sizes. Due to delays and limited funding, the 2002 CFS design made limited use of prior surveys and did not incorporate pilot studies (37). The sample size also was reduced from 100,000 to 50,000 establishments, which degraded the quality and usefulness of the data. There also were communication issues such as not sharing sampling procedures and relevant documentation with CFS data users (35). Electronic Data Interchange Standards Purpose and Content EDI standards are data exchange standards that facilitate the exchange and interpretation of formatted data messages between computers. EDI formatted data messages are business documents, examples of which include rate proposals, invoices, purchase orders, and ship notices. The two parties in an EDI transaction are usually called trading partners. Although the term EDI can be used in connection with any formatted exchange of data between computers, EDI frequently applies to the standards developed by the American National Standards Institute Accredited Standards Committee (ANSI ASC) X12 (41). Other formatted data exchange standards used by the freight community include the following: • United Nations Electronic Data Interchange for Adminis- tration, Commerce, and Transport (UN/EDIFACT) stan- dards (42). These standards are predominant outside of North America. • Universal Business Language (UBL) (43). UBL is a library of standard extensible markup language (XML) electronic business documents developed by the Organization for the Advancement of Structured Information Standards (OASIS), which is an international non-profit organiza- tion that seeks the adoption of open interoperability stan- dards for business applications. Federal Information Processing Standards Publication (FIPS PUB) 1612 describes the requirements to use EDI standards within the federal government (44). ANSI ASC X12 standards define data message (or transac- tion set) components such as message syntax, message type, control data elements, data segments, message grouping, and message authentication. A transaction set is divided into data segments, where a segment is a collection of data elements that typically includes a segment ID, data elements separated by delimiters, and a segment terminator. A segment within a transaction set can be mandatory, optional, or conditional. Many transaction sets have three parts: header (which starts with a header segment), detail, and summary (which ends with a trailer segment). ANSI ASC X12 has sponsored the development of more than 300 EDI standard transaction sets (and, increasingly, XML schemas) in a wide range of areas such as materials, warehousing, product services, and transportation. Many EDI transaction sets are related to freight and cover topics such as rate proposals, freight details and invoices, trailer manifests, shipment information, shipment status inquiries and status messages, and tariff information. Table 3 lists a short sample of freight-related ANSI ASC X12 transaction sets. 25

Development, Challenges, Strategies, and Adaptability Significant milestones in the development of the ASC X12 standards include the following (41): • In 1979, ANSI formed Accredited Standards Committee X12 to develop uniform standards for electronic exchange of business transactions. • In 1982, ANSI published Version 1 of the American National Standards. Over the years, ANSI has published revised ver- sions of these standards, which are ANSI-certified releases of draft ASC X12 standards. • In 1986, project teams were formed as precursors to func- tional subcommittees. Currently, ASC X12 has seven sub- committees (communications and control, finance, gov- ernment, insurance, supply chain, technical assessment, and transportation) as well as several task groups. • In 1990, an alignment task group was formed to recom- mend steps to converge ASC X12 standards and EDIFACT messages. • In 1999, an XML task group was formed to draft policies and procedures related to EDI and XML. • In 2000, the Health Insurance Portability and Accountabil- ity Act (HIPAA) transaction regulation (45 CFR 160 and 162) was published adopting nine ASC X12 transaction sets for the health care industry (45). ASC X12 signed a mem- orandum of understanding (MOU) with the Department of Health and Human Services and standards development organizations to manage the EDI standards adopted under HIPAA. • In 2001, ASC X12 and the UN/EDIFACT working group started work to create a single set of core components that could work on both standards environments. • In 2005, ASC X12 published the first set of XML schemas. Although EDI standards are independent of hardware and software communication technologies, EDI implementations typically require the use of special-purpose software for the transmission and interpretation of EDI transaction sets. Tra- ditional EDI implementations use direct modem-to-modem connections. However, the number of EDI implementations that use Web-based communication protocols (e.g., hyper- text transfer protocol over secure socket layer [HTTPS] and Applicability Statement [AS]), is increasing rapidly. Many implementations rely on value-added networks (VANs) to facilitate communications between trading partners. There are several versions and releases of the ANSI ASC X12 standards (e.g., 3040, 4010, 5010, and 6010). EDI appli- cations are normally built upon specific version releases. Dif- ferent releases are not compatible, which adds complexity to the data exchange process. The decision to upgrade an EDI application to a more recent version of the standard depends on a number of factors, including cost to upgrade and what versions are used by current and potential trading partners. In large companies, it is common to have internal technical teams that support the development and maintenance of in- house applications. In smaller companies, it is more common to outsource EDI communications to third-party vendors. The alternative to upgrading is to purchase EDI translation software (which often costs in excess of $50,000) or contract with third parties to translate data formats, data elements, and qualifiers to ensure compatibility with the EDI standard versions required by trading partners in the supply chain. The disparities between different versions of EDI standards currently in use may at least partially explain why many sup- ply chain stakeholders either build their own systems to the minimum “mandatory” specifications and omit the more robust (“optional”) data elements or outsource EDI data exchange to a third-party provider. To address some of the limitations associated with tradi- tional EDI transaction sets (including proprietary software implementations, cryptic format, and implementation com- 26 No. Description 104 Air Shipment Information 109 Vessel Content Details 110 Air Freight Details and Invoice 210 Motor Carrier Freight Details and Invoice 211 Motor Carrier Bill of Lading 214 Transportation Carrier Shipment Status Message 215 Motor Carrier Pick-up Manifest 216 Motor Carrier Shipment Pick-up Notification 217 Motor Carrier Loading and Route Guide 218 Motor Carrier Tariff Information 309 Customs Manifest 310 Freight Receipt and Invoice (Ocean) 311 Canadian Customs Information 315 Status Details (Ocean) 319 Terminal Information 322 Terminal Operations and Intermodal Ramp Activity 323 Vessel Schedule and Itinerary (Ocean) 350 U.S. Customs Status Information 353 U.S. Customs Events Advisory Details 404 Rail Carrier Shipment Information 410 Rail Carrier Freight Details and Invoice 426 Rail Revenue Waybill 435 Standard Transportation Commodity Code Master 437 Railroad Junctions and Interchanges Activity 440 Shipment Weights 451 Railroad Event Report 470 Railroad Clearance 601 U.S. Customs Export Shipment Information 715 Intermodal Group Loading Plan 853 Routing and Carrier Instruction 857 Shipment and Billing Notice 858 Shipment Information 859 Freight Invoice Table 3. Sample of freight-related ANSI ASC X12 transaction sets.

plexity), ASC X12 developed a Context Inspired Component Architecture (CICA) that enables the construction of XML- based message sets that rely on reusable vocabulary across multiple industries (41). In CICA, data element definitions (e.g., date, time, and name) are XML constructs that can be reused multiple times as needed. ASC X12 has published a number of XML schemas, including the following, which are related to transportation: • Transportation freight invoice, • Transportation status—general use, • Transportation status—small package use, • Transportation status—general use request, • Transportation empty car release—rail request, • Transportation empty car release—rail response, and • Transportation price distribution—rail. Commodity, Product, and Industry Classification Standards As previously mentioned, many EDI transactions sets are related to freight. Of particular interest are transaction sets that provide information about the commodities being trans- ported. For motor carrier shipments, ANSI ASC X12 Transac- tion Set 211 describes commodity items using national motor freight classification (NMFC) codes. NMFC is a standard maintained by the National Motor Freight Traffic Association (NMFTA) that groups commodities into 18 classes according to four commodity “transportability” characteristics: density, stowability, handling, and liability (46). NMFC is one of several commodity and product classifica- tion standards available to the freight community. Widely known standards include the following: • Central Product Classification (CPC). CPC is a product classification system sponsored by the United Nations, which uses a five-digit hierarchical structure that provides three levels of product code resolution (47). With some excep- tions, CPC subclasses are groupings and rearrangements of Harmonized System (HS) codes. CPC code listings provide an indication of the corresponding HS codes, along with International Standard Industrial Classification (ISIC) activ- ity classes (47). CPC provides the base for the Standards Nomenclature for Transport Statistics (NST 2007) classi- fication system (48). • Harmonized System. HS is an international product cod- ing system developed by the WCO (49). HS uses a six-digit hierarchical structure that provides three levels of com- modity code resolution. Most countries have adopted HS, including the United States, which used HS as the basis for the Harmonized Tariff Schedule (HTS) maintained by the U.S. International Trade Commission (USITC) (50). • National Motor Freight Classification. As previously mentioned, NMFC groups commodities into 18 classes according to four commodity “transportability” charac- teristics: density, stowability, handling, and liability (46). the access and use of NMFC codes is limited by 49 U.S. Code Section 13703 (49 USC 13703) to specific regulated carriers (51, 52). • North American Product Classification System (NAPCS). NAPCS is a product classification system the United States, Canada, and Mexico are developing to complement NAICS (53). • Price Look-Up (PLU). PLU codes are used by the produce sector to describe products such as fruits, vegetables, dried fruit, herbs and flavorings, and nuts (54). Typically, sealed, containerized, or packaged produce falls outside the scope of the PLU coding system. Also excluded is produce that has undergone additional processing. • Standard Classification of Transported Goods. SCTG codes are commodity codes that were developed to support the needs of the 1997 CFS (55). SCTG uses a five-digit hier- archical structure that aggregates HS codes into categories that CFS planners considered more suitable for statistical analyses and the collection of freight movement data. • Standard Transportation Commodity Codes. STCCs are commodity codes used by the railroad industry to describe product information in waybills and other shipping docu- ments. AAR developed STCC in 1962 using a seven-digit structure that provided five levels of commodity code res- olution (56). It may be worth noting that CFS shifted from STCC to SCTG codes in 1997. As a result, other applica- tions that rely on CFS data (such as FAF) also changed to SCTG (57). Most of the product classification standards above provide a mapping of product codes to industrial classification sys- tems such as the following: • International Standard Industrial Classification of All Economic Activities. ISIC classifies industries using a four-digit hierarchical structure that provides three levels of industry code resolution (47). At the top level, the two- digit division codes are grouped into sections designated by letters (which are not included in the ISIC codes). • North American Industry Classification System. NAICS classifies industries using a six-digit hierarchical structure that provides six levels of industry code resolution (58). NAICS replaced SIC in 1997. • Standard Industrial Classification. SIC classified indus- tries using a four-digit hierarchical structure that provided four levels of industry code resolution (59). SIC has been replaced by NAICS and is no longer in use by the federal government. 27

Examples of crosswalk tables that enable the mapping of codes across systems include the following: • Five-digit CPC codes and six-digit HS codes (47), • Five-digit CPC codes and four-digit ISIC codes (47), • Two-digit HS codes and two-digit SCTG codes (57), • Two-digit SCTG codes and four-digit STCC codes (57), • Six-digit NAICS codes and four-digit SIC codes (58), and • Four-digit ISIC codes and six-digit NAICS codes (60). It is worth noting that crosswalk tables are actually snapshot views because they use specific versions of the corresponding codes linked by the crosswalk tables. Crosswalk table mainte- nance practices vary widely from agency to agency. In addi- tion, there is no centralized repository of links to current and historical crosswalk tables. Readers also should be aware that the list of classification standards provided above is only a sample. Additional classi- fication standards that should be taken into consideration for the development of a national freight data architecture include the following: • HMIS regulations and classification standards, • Federal and state vehicle type/class classification standards, • Vessel classification standards, • Railcar classification standards, and • Facility classifications. Lessons Learned Lessons learned in connection with the development, evo- lution, and maintenance of EDI standards follow: • Develop applications that rely on widely used data stan- dards. The ANSI ASC X12 EDI standards have been around for almost 30 years and are widely used in industries such as retail, transportation, education, health care, travel, and insurance. Many EDI transactions sets are related to freight and cover topics such as rate proposals, freight details and invoices, trailer manifests, shipment information, shipment status inquiries and status messages, and tariff information. Traditional EDI implementations use direct modem-to- modem connections. However, the number of EDI imple- mentations that use Web-based communication protocols (e.g., HTTPS and AS) is increasing rapidly. To support this transition, ASC X12 is beginning to develop XML schemas, which facilitate data exchange using modern communica- tion technologies. • Develop applications with backward compatibility. There are several versions and releases of the ANSI ASC X12 stan- dards (e.g., 3040, 4010, 5010, and 6010). EDI applications are normally built upon specific version releases. Unfortu- nately, different releases are not compatible, which adds complexity to the data exchange process. The disparities between different versions of EDI standards currently in use may at least partially explain why many supply chain stakeholders either build their own systems to the mini- mum “mandatory” specifications and omit the more robust (“optional”) data elements or outsource EDI data exchange to a third-party provider. • Participate in the standards development process. For motor carrier shipments, ANSI ASC X12 Transaction Set 211 describes commodity items using NMFC codes. NMFC is a standard maintained by NMFTA, which groups com- modities into 18 classes according to four commodity “transportability” characteristics: density, stowability, han- dling, and liability. NMFC codes are not compatible with other commodity classification codes commonly used by the freight community. Active participation by other freight stakeholders in the development of ASC X12 standards would be an effective mechanism to help address code incompatibility problems. Freight Analysis Framework Purpose and Content FAF is a commodity O-D database and analytical frame- work that provides estimates of tonnage and values of goods shipped according to origin, destination, commodity, and mode (57, 61). In addition to commodity O-D data, FAF pro- vides estimates of commodity movements by truck and vol- umes of long-distance trucks over specific highways. FAF was originally developed as a policy analysis tool within the U.S.DOT. Over time, FAF products have also used to convey freight profile and statistics to the states and the public, and as a tool to support economic analyses that involve commod- ity flow trends in areas other than transportation. Additional examples of FAF applications are documented in reports such as the Quick Response Freight Manual (62). FAF includes 138 origin and destination “zones,” consist- ing of 114 regions as defined in the 2002 CFS, 17 international gateways (which supplement FAF regions that are both gate- ways and domestic zones), and 7 international regions. Com- modities are defined at the two-digit SCTG level. Although the 2002 CFS defines 11 separate modes, multimodal combi- nations, and unknown modes, FAF only uses 7 aggregated modes. FAF relies primarily on data collected every 5 years as part of the economic census. Conceptually, the database of origins and destinations in FAF can be thought of as a four-dimensional matrix of ori- gins, destinations, commodities, and modes, in which each cell in the four-dimensional matrix represents tonnage or value of goods shipped (57). The actual implementation of 28

FAF uses a database composed of several tables (63), includ- ing the following: • Domestic tonnage and value tables. These tables contain the following data: – Origin: one of the 114 FAF/CFS domestic regions, – Origin state: state where the FAF origin region is located, – Destination: one of the 114 domestic regions, – Destination state: state where the FAF destination region is located, – Commodity: one of the 43 SCTG commodities, – Mode: one of the 7 aggregated modes, and – Years 2002–2035: thousand tons or million dollars for each year. • International tonnage and value tables. These tables (for transborder, sea, and transocean air) contain the following data: – Origin: one of the 7 international regions (for imports) or one of the 114 domestic regions (for exports), – Origin state: state or international region where the FAF origin region is located, – Destination: one of the 7 international regions (for imports) or one of the 114 domestic regions (for exports), – Destination state: state or international region where the FAF destination region is located, – Commodity: one of the 43 SCTG commodities, – Port: one of the 17 international gateways, – Mode: one of the 7 aggregated modes used for the domes- tic portion of the movement, and – Years 2002–2035: thousand tons or million dollars for each year. Development, Challenges, Strategies, and Adaptability FAF versions include the following: • FAF1 (or “original” FAF). This version of FAF, released in 2000, includes commodity O-D data for base year 1998 and future years 2010 and 2020. FAF1 relied partly on pro- prietary data. • FAF2 version 2.1 (FAF2.1). This version of FAF, released in January 2006, includes commodity O-D data for base year 2002. • FAF2 version 2.2 (FAF2.2). This version of FAF, released in November 2006, includes commodity O-D data for base year 2002 and future years 2010 through 2035 at 5-year intervals. Version 2.2 includes minor corrections to 2002 base year flows in Version 2.1. FAF2 also includes provi- sional data. Because the movement of goods may experi- ence shifts between economic census years, FHWA pro- duces provisional estimates of goods movement by origin, destination, and mode, using publicly available publica- tions that are less complete and detailed than the data used for the 2002 base estimate. The most recent year for which there are O-D estimates is 2007. • FAF2 version 2.3 (FAF2.3). FAF2.3, scheduled for release in 2009, will be the final version of the FAF2 series (64). This version will include minor adjustments to the 2002 O-D database, a distance matrix for estimating ton-miles, major corrections to the 1997 historical O-D file, and a Web- based tool for creating tables and extracting portions of the O-D database. • FAF3. FHWA is currently working on FAF version 3 (FAF3) (64). FHWA anticipates releasing FAF3.0 by mid 2010, including the 2007 O-D commodity flow database, the 2007 highway network database, and initial ton-mile estimates by state. FHWA also expects to release FAF3.1 by the end of 2010 with forecasts, the rail and waterway network data- bases, and detailed ton-mile estimates. The current plan is to use FAF2 to release provisional 2008 and 2009 estimates (in 2009 and 2010, respectively) and use FAF3 to release 2010 provisional estimates (released in 2011) and other future years. FAF1 was developed as a policy analysis tool within the U.S.DOT. FAF1 products were also used to convey freight profile and statistics to the states and the public, and as a tool to support economic analyses that involved commodity flow trends in areas other than transportation. However, FAF1’s shortcomings, including its reliance on proprietary data and little use of CFS data, resulted in inconsistencies between FAF1 and CFS and the inability to publish estimates of commodity flows for areas smaller than states (64). A 2004 FHWA report identified improvement needs in areas related to geographic detail, completeness, accuracy, and timeliness (65). The 2004 report also outlined six goals for FAF2, as follows (64, 65): 1. Integrate economic census data more effectively, 2. Assure quality of FAF data for the benchmark years, 3. Provide timely updates to FAF data products, 4. Assure that FAF methods and products are transparent and can be reproduced, 5. Help state and local governments make effective use of FAF products in conjunction with developing a local under- standing of freight activity, and 6. Continue to work with customers to improve the useful- ness of FAF products. Specific changes in FAF2 to meet these goals included the following: • Modes of transportation. FAF2 was expanded to include all modes of transportation, including truck, rail, water, 29

air, and pipeline. The term “intermodal” in FAF1 was based on CFS definitions, which include postal and courier shipments as well as any shipment using more than one mode of transportation. This definition was broader than other industry definitions such as trailer on flatcar or con- tainer on flatcar. As a result, FAF2 included two categories of “intermodal” shipments: truck-rail and other. • Commodity classification. FAF2 changed from STCC to SCTG to address limitations in the STCC coding structure and to ensure consistency with critical data sources in FAF2 (particularly CFS). • Timeliness. FHWA began releasing provisional estimates to address requests for more frequent updates than once every 5 years (when CFS occurs). • Public versus commercial data. FAF2 shifted from using commercial, proprietary data to public data to populate its models because of limitations in FAF1 (which relied on commercial data) that prevented the publication of all FAF data. Disseminating all data in FAF2 was a strategy to enhance transparency, credibility, and public access. Some of the issues raised in the 2004 report that FAF2 did not address but are relevant for FAF3 include the following (64): • Geographic region coverage and resolution. FAF2 used 2002 CFS geographic regions. Because the number of geo- graphic regions in CFS has increased since the 2002 census, updated CFS regions will be used for the 2007 benchmark O-D commodity flows, annual provisional estimates, and forecasts through 2040. FAF2 also excluded freight move- ments that passed through the United States. A strategy being considered is to use the North American Transporta- tion Statistics Interchange forum, which includes the United States, Canada, and Mexico, to estimate in-transit flows at the national level. FAF2 includes a temporary file that contains disaggre- gated county-to-county commodity flows. However, FHWA does not publish the temporary file because of the large errors that result from disaggregating flows from regions to counties. With the increase in the number of CFS geo- graphic regions, FHWA is considering options such as developing a standard region-to-county disaggregation method coupled with a program to collect supplemental data locally. • Transportation network coverage and resolution. FAF2 relies mainly on National Highway Planning Network (NHPN) routes for assigning O-D commodity flows to the transportation network. FAF2 does not map freight move- ments that are shorter than 50 mi to this highway network. It also does not map commodity flows to individual rail lines, waterways, or pipelines. As in FAF1, FAF2 disaggre- gates flows to counties and selected sub-county generators such as major ports, and then assigns the flows to routes. The process included matching route assignments to HPMS truck volume estimates, which revealed quality problems with HPMS data. • Temporal resolution. FAF2 provides annual commodity flows. It does not handle seasonal, daily, or hourly varia- tions in commodity flows. In addition, highway network assignment estimates are for peak period conditions. FAF2 simulates routing changes in response to bad weather by adjusting network impedances. A strategy that FHWA might consider is to use observed data from the Freight Performance Measurement initiative. • Modes of transportation. FAF2 uses multimodal CFS definitions, which include shipments by postal and courier services and shipments that use more than one mode. This categorization is broader than trailer-on-flatcar or con- tainerized service. FHWA is evaluating whether modal definitions can be developed within the confines of the 2007 CFS. • Shipper and carrier costs. Forecasting costs and evaluat- ing potential responses from the private sector, particularly shippers and carriers, requires access to transportation cost data. However, the collection of transportation cost data was largely discontinued after deregulation, and viable strategies have not been defined for obtaining this type of data in the future. • Feedback. The FAF development team is small, which lim- its its capacity to respond to user requests and feedback. To address this limitation, FAF3 will include enhanced out- reach methods to improve access to, and ease of use of, FAF products. Lessons Learned Lessons learned in connection with the development, evo- lution, and maintenance of FAF follow: • Use available data sources while keeping the system flex- ible to respond to changes and new data sources. FAF was designed around existing information, particularly CFS, which, at the time of the FAF development, was one of the most comprehensive data sources available. FAF used CFS definitions of mode and geographic resolution and adapted other data sources to fill gaps in the CFS data. In the begin- ning, FAF used STCC codes. Eventually, FHWA changed to SCTG codes in order to address limitations in the STCC coding structure and to ensure consistency with critical data sources in FAF2, particularly CFS. • Develop systems that are consistent with input data lim- itations. FAF’s geographic region definitions are based on CFS region definitions. It would have been difficult, costly, and time consuming for FHWA to develop a much more 30

detailed region definition for FAF (e.g., at the county level) because, realistically, FHWA was not in a position to con- duct a separate data collection effort to bypass CFS. By redefining those geographic regions, the quality of the FAF data would have suffered because the new commodity flow data would have relied much more on synthetic data instead of actual CFS data. • Implement strategies that promote adequate data access and transparency. FHWA published not just FAF data products online but also ample documentation about the process to develop those products. Relying on proprietary data for the production of FAF1 prevented FHWA from publishing some of the FAF1 data products. This limita- tion was solved in FAF2 by relying mainly on public data. • Keep in mind the value of customer feedback. FAF evolved from being a freight policy analysis tool into a product that is widely used by the freight community because new FAF releases took into consideration lessons learned from the use of the previous versions, including user feedback. Highway Performance Monitoring System Purpose and Content HPMS is a system that contains data about the extent, con- dition, performance, use, and operating characteristics of the U.S. highway network (66). HPMS data are used for a variety of applications, including the following: • Providing input to the production of reports to Congress on the condition, performance, and investment needs of U.S. highways, which Congress uses to establish authoriza- tion and appropriation legislation that affects the scope and size of the federal-aid highway program; • Assessing changes in highway system performance and for apportioning federal-aid highway funds to individual states; • Assembling freight corridors and determining freight move- ment performance; • Special policy and planning studies; • Travel and congestion monitoring, public road usage, and fatality rate calculations; • Investment needs and planning at the state level; and • Air quality conformance and planning. Various agencies use HPMS data, including federal, state, and local agencies, as well as research agencies. HPMS relies on annual data from state DOTs. The HPMS field manual (67) provides guidelines to state DOTs on the procedures to obtain and report data to HPMS, including precision levels and sample size estimation procedures. HPMS includes limited data on all public roads; detailed data on a sample of the arterial and collector functional systems; and area-wide summary information for urbanized, small urban, and rural areas; as follows (Table 4) (67, 68): • “Universe” data include basic inventory data on all open public road systems in the HPMS database. The basic inven- tory includes 46 data items for National Highway System (NHS) sections and 28 data items for local roads. • “Sample” data include 98 data items containing addi- tional inventory, condition, use, pavement, operational, and improvement data for 120,000 sections of roadway selected as standard samples. • “Summary” data provide travel data for all functional sys- tems in urbanized areas, small urban areas, and rural areas, as well as air quality non-attainment and maintenance areas. In addition to other HPMS data, each state is required to submit linear referencing system (LRS) data in one of the fol- lowing three category options: (1) maps and computer files, (2) LRSEDIT files and maps for new links and nodes, or (3) geographic information system (GIS) files. The HPMS field manual details these options (67). State-submitted LRS data are integrated with NHPN and are now published as part of NTAD in Environmental Systems Research Institute (ESRI™) shapefile format (see the National Transportation Atlas Data- base section in this chapter for additional information). Development, Challenges, Strategies, and Adaptability HPMS began in 1978 as a mechanism to replace annual data reports and biennial special studies that states conducted for FHWA (68). The special studies were in response to a 1965 requirement to produce a condition and performance (C&P) report on the nation’s highways every 2 years. The first C&P report was completed in 1968. FHWA used data from state annual reports and biennial special studies, and subsequently has continued to use data from HPMS to produce the C&P reports mandated by Congress. Federal legislation has continued the requirement for bien- nial reports, for example through the 1991 ISTEA (69); the 1998 TEA-21 (70); and the 2005 SAFETEA-LU (71). ISTEA introduced a requirement to provide the necessary informa- tion to enable the comparison of measures when these mea- sures change. TEA-21 moved the report requirements from 23 USC 307 to 23 USC 502. FHWA has modified HPMS several times to accommodate changes in legislation, technology, national priorities, and reporting requirements (72). For example • In 1980, HPMS was merged with the Mileage Facilities Reporting System (MFRS), which included facility mileage, 31

travel, and accident statistics. After the merge, a single system evolved to include the “universe” data attributes in MFRS and the area-wide data attributes in the original HPMS. • In 1988, HPMS was enhanced to include pavement data items, including International Roughness Index (IRI) measurements. • In 1993, HPMS underwent several modifications to address changes in FHWA analysis and simulation models, 1990 census effects, ISTEA, the Clean Air Act Amendments of 1990, and EPA requirements for vehicle miles traveled (VMT) data in air quality non-attainment areas. The 1993 revision added several “universe” data items for the National Highway System and other principal arterial highways. The amount of sample traffic data for urbanized air quality non-attainment areas increased. Truck data requirements also increased. At the same time, the revision deleted sev- eral pavement data items and sample data items for rural minor collectors. • In 1999, FHWA conducted a major HPMS reassessment (73). The reassessment, which began in 1996, removed 15 data items and changed 21 others to eliminate duplication with NHTSA’s Fatality Analysis Reporting System. The 32 No. Data Item No. Data Item 1 Year of Data 49 Standard Sample Expansion Factor 2 State Code 50 Surface/Pavement Type 3 Reporting Units – Metric or English 51 Structural Number or Thickness 4 County Code 52 General Climate Zone 5 Section Identification 53 Year of Surface Improvement 6 Is Standard Sample 54 Lane Width 7 Is Donut Sample 55 Access Control 8 State Control Field 56 Median Type 9 Is Section Grouped? 57 Median Width 10 Linear Referencing System (LRS) Identification 58 Shoulder Type 11 LRS Beginning Point 59 Shoulder Width – Right 12 LRS Ending Point 60 Shoulder Width – Left 13 Rural/Urban Designation 61 Peak Parking 14 Urbanized Area Sampling Technique 62 Widening Feasibility 15 Urbanized Area Code 63-68 Curves by Class 16 National Ambient Air Quality Standards (NAAQS) Nonattainment Area Code 69 Horizontal Alignment Adequacy 17 Functional System Code 70 Type of Terrain 18 Generated Functional System Code 71 Vertical Alignment Adequacy 19 National Highway System 72-77 Grades by Class 20 Planned Unbuilt Facility 78 Percent Passing Sight Distance 21 Official Interstate Route Number 79 Weighted Design Speed 22 Route Signing 80 Speed Limit 23 Route Signing Qualifier 81 Percent Peak Single Unit Trucks 24 Signed Route Number 82 Percent Average Daily Single Unit Trucks 25 Governmental Ownership 83 Percent Peak Combination Trucks 26 Special Systems 84 Percent Average Daily Combination Trucks 27 Type of Facility 85 K-Factor 28 Designated Truck Route 86 Directional Factor 29 Toll 87 Number of Peak Lanes 30 Section Length 88 Left Turning Lanes/Bays 31 Donut Area Sample Annual Average Daily Traffic (AADT) Volume Group Identifier 89 Right Turning Lanes/Bays 32 Standard Sample AADT Volume Group Identifier 90 Prevailing Type of Signalization 33 AADT 91 Typical Peak Percent Green Time 34 Number of Through Lanes 92 Number At-Grade Intersections – Signals 35 Measured Pavement Roughness 93 Number At-Grade Intersections – Stop Signs 36 Present Serviceability Rating 94 Number At-Grade Intersections – Other or No Controls 37 High Occupancy Vehicle (HOV) Operations 95 Peak Capacity 38-46 Highway Surveillance Systems 96 Volume/Service Flow Ratio 47 Sample Identifier 97 Future AADT 48 Donut Area Sample Expansion Factor 98 Year of Future AADT Table 4. HPMS data items (67).

reassessment reduced the HPMS sample size by 35 percent and the number of records by two-thirds through grouping. • In 2005, FHWA started another HPMS reassessment. This reassessment included a number of cross-cutting topics, such as process improvements, data quality, data models, sampling, boundaries, and functional classifications (68). Documentation related to the reassessment effort includes a final report (68), data specifications (74), and a field man- ual (75). Some of the changes in HPMS include the addi- tion of motorcycle travel data, ramps, pavement meta- data, traffic metadata, and ownership codes. Other data changes include a reduction in the number of data elements that states need to collect (from 87 to 68). Another major change in the recent HPMS reassessment is an updated data model for HPMS data, which includes several subject areas (called catalogs), including shapes, summaries, refer- ences, metadata, sections, and estimates. FHWA is cur- rently conducting training and technical support activities, and expects 2009 data submissions using the updated HPMS data model in 2010. Although state DOTs recognize that correct and complete HPMS data influence state apportionments of federal funds, a significant challenge for state DOTs is the availability of funds to support the collection and reporting of HPMS data. Unlike other federal data collection programs, there are no dedicated (i.e., earmarked) funds to support this effort (68). Within FHWA, the primary source of funding for HPMS is discre- tionary research funds. States often use state planning and research funds to collect HPMS data. HPMS data collection and processing can be expensive. The 1999 HPMS reassessment esti- mated the cost of collecting HPMS data at about $15 million per year nationwide (or about $300,000 per year per state on aver- age) (73). Collecting sample data is the most expensive compo- nent, representing 63 percent of the data collection costs. Data collection costs influence the number of collected data items, transportation system scope, and data quality requirements. FHWA is responsible for maintaining HPMS data provided by state DOTs. In turn, state DOTs are responsible for the accuracy and timely collection and reporting of HPMS data (67). Although FHWA does not specify data collection tech- niques or perform detailed quality control analyses on the data, FHWA provides assistance with the following: • Quality assurance using tools as the TranStats HPMS Map Viewer and a 5-year trend table; • A restricted server side application, which may be accessed on the Internet, that states must use to submit HPMS data (prior versions of this tool were stand-alone versions); • Annual reviews of state DOT data, including reviews of high-risk items such as traffic data and inventory data, as well as certifying that the state’s public road mileage data, VMT data, and lane-mile data are valid and suitable for use in apportionment of federal-aid highway funds; and • Coordination between FHWA and state DOTs on HPMS improvements. HPMS sample data are stratified by state, type of area, high- way functional system, and AADT group. The sample size is estimated based on AADT within each stratum, which is the most variable data item. Although the sampling error can be estimated directly based on the sample design for each stratum, this exercise has not been repeated since the 1980s because of the amount of work involved (76). The impact of non-sampling errors in HPMS is significant. For example, there are guidelines on how to measure data ele- ments such as AADT, but many states use their own proce- dures. Some data elements may be collected by agencies other than the state DOTs (e.g., MPOs and cities) following differ- ent procedures, frequently as an alternative to purchasing commercial datasets. Some data items may be difficult and/or costly to collect, and, as a result, are reported using estimates. Likewise, states use different methodologies to measure pavement condition ratings. The frequency of pavement inspections also varies from state to state (77). For example, pavement inspections for state systems vary from every year to every other year. However, inspections for non-state systems can vary from every year, to every other year, to every sixth year. The time lag from the date of the pavement inspection to the date when HPMS data are available can be several years. Lessons Learned Lessons learned in connection with the development, evo- lution, and maintenance of HPMS follow: • Involve stakeholders through a variety of mechanisms and technologies. Both the 1999 and the 2010 HPMS reassessments included extensive outreach, ongoing com- munication, and coordination with stakeholders through various approaches. One of these approaches was through steering committees comprised of federal, state, and local officials, which met several times to identify critical issues and provide oversight. Another approach was to prepare issue papers covering relevant technical issues and data areas, such as sampling, boundaries and functional classi- fication, safety, pavements, interchanges, freight, capacity, data quality, process improvement, and data models. Other methods to involve stakeholders included presen- tations before committees, such as TRB data committees, the AASHTO Standing Committee on Planning (SCOP), the Association of Metropolitan Planning Organizations (AMPO), and Washington-based organizations involved in transportation, as well as regional workshops and follow-up 33

surveys. There were also meetings and interviews with federal employees who were HPMS “customers” or were involved in federal data and policy analysis activities. Com- munication technology made the process easier, particu- larly in the case of the 2010 HPMS reassessment, which made extensive use of webinars, posting issue papers online, and receiving comments by e-mail. • Evaluate user and data needs carefully to avoid scope creep. Although systems need to be flexible and adaptable, frequent and uncontrolled changes in scope (i.e., scope creep) can result in cost overruns, missed deadlines, and loss of original goals. In 1999, there were some questions as to the need for HPMS (68). To reduce state DOT HPMS data collection costs, the 1999 HPMS reassessment reviewed the need and requirements for each data item in detail (73). Implementing the results of this review was estimated to result in potential annual cost savings of $3 million to $5 million nationwide. The data item review included the following items: – Determine if the data were needed to meet a legislative requirement, were used in analysis and policy decision activities, were included in state or local highway data- bases, were quantitative (versus subjective), could be collected with consistency, or could be simulated or estimated (as opposed to collected); – Evaluate data aggregation levels; – Evaluate if the data should be collected for all sections or for a sample of sections; – Evaluate whether the data collected met HPMS mission and objectives and constituted an improvement over the current system, including an evaluation of imple- mentation costs versus savings, potential impacts when data elements are analyzed as a group, and process and timing requirements; and – Gather public input and obtain final FHWA internal review and adoption. In addition to identifying core data items and processes to reduce the data collection burden on state DOTs, the 1999 reassessment identified a number of needed system enhancements, including the following: – Improve the quality of HPMS data; – Increase the use of new technology to collect HPMS data; – Improve training to states and other data collection agencies; – Develop better integration, linkages, and coordination among state, regional, and local databases; – Allow access to raw or disaggregated HPMS data for local use; – Design HPMS to be statistically significant at the local level; and – Include additional pavement condition data. • Phase implementation of system changes. Coordinating the implementation of HPMS changes with the annual timing of ongoing data collection programs is critical. For the 2010 HPMS reassessment, FHWA decided to imple- ment HPMS in four phases (early, immediate, phased, and late) to focus first on critical data collection and reporting requirements while allowing for future anticipated changes (e.g., changes to boundaries and functional classes follow- ing the decennial census). National Income and Product Accounts Purpose and Content BEA’s economic accounts are records of economic activ- ity in the United States that provide information about the structure and performance of the U.S. economy (78–80). BEA uses a variety of economic accounts, including national economic accounts, regional economic accounts, international economic accounts, and industry economic accounts. There are three main national economic accounts: • National Income and Product Accounts. The NIPAs document the value and composition of national output and the distributions of incomes generated in the produc- tion of that output. These accounts help to provide mea- sures of the output of the economy, the sources and uses of national income, and the sources of savings. A key sum- mary measure is the GDP. Other summary measures track personal income, corporate profits, government spending, national production, distribution, consumption, invest- ment, and savings. • Industry Input-Output (I-O) accounts. These accounts document the flow and value of goods and services by indus- try, the commodity composition of national output, and GDP by industry. • Federal Reserve Board flow of funds accounts. These accounts document the acquisition and value of finan- cial assets, nonfinancial assets, and liabilities, as well as the sources of the funds used to acquire those assets and liabilities. This review focuses on the NIPAs. The NIPA framework consists of seven summary accounts that contain data aggre- gated from approximately 300 supporting NIPA tables that contain production and expenditure data by sector, product, function, and investment source. The seven summary accounts are as follows: 1. Domestic income and product account, 2. Private enterprise income account, 3. Personal income and outlay account, 4. Government receipts and expenditures account, 5. Foreign transactions current account, 34

6. Domestic capital account, and 7. Foreign transactions capital account. The supporting tables are available on the BEA Website and also can be downloaded in .xls or .csv formats (81). The BEA Web interface enables users to access data annually or quarterly within a specified year range. Examples of NIPA uses include the following (80): • Macroeconomic analysis and forecasting, • U.S. economic measurement, • Economic policymaking and evaluation, • Federal budget and tax projection preparation, • International economy comparison, • Evaluation of interrelationships between different economic sectors, • Financial and investment planning by businesses and indi- viduals, and • Development of other economic accounts. BEA uses a number of satellite accounts that provide more detail than the NIPAs and facilitate the analysis of specific aspects of the economy. The transportation satellite accounts (TSAs), which were jointly developed by BTS and BEA, focus on transportation services and the contribution of these ser- vices to the U.S. economy (82). These accounts make a dis- tinction between hired transportation services and transporta- tion services that businesses provide for their own use, identify industries that account for most transportation activities or are the largest users of transportation services, estimate the impact of transportation in the production costs of these industries, and estimate relative expenditures in transportation infrastruc- ture and equipment by government and businesses. Development, Challenges, Strategies, and Adaptability The origin of the NIPAs can be traced back to the 1930s with the publication of the first estimates of national income, which were needed to measure the effectiveness of the strategies implemented to combat the Great Depression (83). In 1942, annual estimates of gross national product (GNP) were intro- duced and estimates were developed to detail how income was generated, received, and spent by various sectors of the econ- omy. In 1947, the national income and product estimates were integrated into a complete, consistent accounting system with 48 tables. Since then, there have been annual revisions and sev- eral comprehensive revisions. Table 5 summarizes major mile- stones associated with the development and evolution of the NIPAs (83, 84). 35 Year Major Milestones/Revisions 1934 First publication of national income estimates 1942 Annual GNP estimates introduced to complement the estimates of national income 1947 National income and product statistics presented as part of a complete, consistent accounting system 1954 Estimates of real GNP and implicit price deflators added to the NIPA tables 1958 Five summary accounts adopted Quarterly estimates of real GNP, regional estimates, and estimates of the net stock of fixed assets in manufacturing introduced Government-sector tables and foreign-transactions tables modified 1965 Components of GNP benchmarked for the first time in the 1958 I-O table 1976 Estimates of consumption of fixed capital (CFC) shifted to a current-cost basis 1985 Quality-adjusted price indexes for computers and peripheral equipment introduced 1991 National production measure changed from GNP to GDP 1993 NIPA improvements started following the System of National Accounts 1993 framework 1996 Methods for estimating changes in real GDP and for CFC calculation improved Government expenditures for equipment and structures recognized as fixed investment 1999 Several key definitions improved New method introduced for calculating real value of non-priced bank services Consumer price indexes revised back to 1978 2003 More advanced measures of insurance services and banking services adopted New treatment of government activity adopted National income definition expanded to follow international guidelines New tables including two new summary accounts added 2009 New treatments of disasters and insurance services provided by government enterprises introduced Transactions between the federal government and U.S. territories and commonwealths reclassified New classification system for personal consumption expenditures added 2002 benchmark I-O accounts incorporated Statistical measure for estimating persona proprietors’ income improved l consumption expenditures, wages and salaries, and Table 5. Major milestones in the development of the NIPAs.

Comprehensive revisions have normally taken place at 5-year intervals that correspond with the integration of updated statistics from BEA’s benchmark I-O accounts (85). The com- prehensive revisions typically introduce major improve- ments to definitions and classifications, statistical methods, and/or presentations of NIPA tables. Annual revisions com- plement the comprehensive revisions. The annual revisions generally take place each summer and cover the last 3 years. In 2010, BEA will start using “flexible” annual revisions that will retain the features of the current annual revisions while allowing for improvements normally associated with the major revisions (85). Data for the economic accounts come from a variety of sources, including the U.S. Census Bureau, BEA, USDA, BLS, the U.S. Treasury Department, the Internal Revenue Service (IRS), and OMB. Close cooperation from these agencies is critical for the production of the NIPAs. BEA complements government-produced or -maintained data with data from trade associations, businesses, international organizations, and other sources. After collecting the data, BEA processes the data and produces NIPA estimates using a combination of statistical methods. BEA uses the following criteria and methodologies to main- tain the usefulness and effectiveness of the NIPA estimates (80): • Accuracy. Accuracy refers to how close the estimates mea- sure the concepts they are designed to measure. In order to keep pace with innovations in the economy, BEA periodi- cally reviews and updates procedures and data to make sure they provide complete, consistent estimates. • Reliability. Reliability refers to the size and frequency of NIPA estimate revisions. BEA’s objective is to develop ini- tial estimates that provide reliable indicators of economic growth characteristics and where the economy is in rela- tion to the business cycle. • Relevance. Relevance refers to the length of time before estimates become available and the ability of the accounts to provide estimates that help answer relevant questions. To address the first issue, BEA has developed a release cycle for the estimates, which addresses timeliness and accuracy tradeoffs. To address the second issue, BEA has periodically incorporated improvements to the NIPAs and the other economic accounts to ensure the estimates reflect current conditions and changes in analytical and statistical practice. • Integrity. Integrity refers to the independence and objec- tiveness of the estimates. To ensure integrity, BEA strives to develop objective, timely estimates and make its processes open and transparent. In addition, BEA devotes considerable time and effort to ensure the security of the data before releasing any data to the public (86). For example, physical and computer access to sensitive information is restricted, estimates are accessible only to authorized individuals, employees are prevented from pre-releasing information, and releases follow a pre- determined schedule. BEA relies on its own research and development workforce for the preparation of the NIPAs and other economic accounts (80). BEA also relies on scholars and experts from various sources to improve definitions, presentations, and relevant statistical methods. Lessons Learned Lessons learned in connection with the development, evo- lution, and maintenance of the NIPAs follow: • Emphasize data quality, reliability, and integrity. A critical requirement for BEA has been to ensure that calculations and estimates be accurate, reliable, and relevant. Integrity in the form of objective, timely estimates and open, trans- parent processes are also key requirements. Meeting these requirements is critical to ensure the public’s trust in processes and data. • Schedule major and regular revisions effectively. The NIPA revision process consists of annual revisions and compre- hensive revisions. The comprehensive revisions enable not just the revision of estimates, but also a review of NIPA def- initions, statistical methods, and presentations of NIPA tables. Comprehensive revisions take place at regular inter- vals (every 5 years) that correspond with the integration of updated statistics from BEA’s benchmark I-O accounts. This regularity provides a sense of continuity and ensures the feasibility and relevance of the NIPA process. National ITS Architecture Purpose and Content The National ITS Architecture is a collection of tools that describe functions, entities or subsystems where these func- tions reside, and data flows that connect functions and sub- systems in connection with the implementation of systems that use computing, sensing, and communication technolo- gies in transportation operations (87). The National ITS Architecture has been used for many implementations around the country, including traffic management centers (TMCs), traffic signal systems, and tolling operations. The National ITS Architecture includes user services, a log- ical architecture, a physical architecture, and standards, as summarized below. • User services. User services represent what a system would do from the perspective of the user. A user might be the 36

public or a system operator. Currently, there are 33 user services grouped into eight categories (called bundles). • Logical architecture. The logical architecture defines processes, data flows among processes, terminators (i.e., entry and exit points such as sensors, computers, and human operators), and data stores required to satisfy the functional requirements of the 33 user services (88). The logical archi- tecture is presented to readers using nested data flow dia- grams (DFDs) that provide graphical representations of processes, data flows, terminators, and data stores at vari- ous disaggregation levels. At the highest level is a DFD called Manage ITS that has nine first-level processes, all of which are DFDs. In turn, each of these processes has subordinate processes, some of which are DFDs. Version 6.0 of the National ITS Architec- ture includes 3,475 logical data flows, of which 344 data flows have as a source node one of the Manage Commercial Vehicles processes or subprocesses. Freight-related data ele- ments typically cover vehicles and their interaction with the road environment. However, some elements address cargo data needs. • Physical architecture. The physical architecture provides a representation (although not a detailed design) of how an integrated system would provide the functionality defined by the user services and the logical architecture. This goal is achieved by defining subsystems based on functional similarity of process specifications and physical locations of functions within the transportation system. As Figure 6 shows, there are four general categories of subsystems: Centers, Field, Travelers, and Vehicles. In general, the phys- ical architecture handles subsystems, architectural flows (that connect subsystems and terminators), and equipment packages (that break up subsystems into deployment-sized pieces). The physical architecture also handles market pack- ages, which represent slices of the physical architecture that address specific services. In general, a market pack- age includes several different subsystems, equipment packages, terminators, and architectural flows that pro- vide the desired service. The physical architecture includes 13 market packages related to commercial vehicle oper- ation (CVO). 37 Figure 6. National ITS Architecture subsystems (87).

• Standards. There are 96 ITS standards in the RITA ITS standard database (without including withdrawn or sus- pended standards): 2 standards under development, 2 stan- dards in ballot, 4 approved standards, and 88 published standards (89). The standards include document types such as guides, data dictionaries, message sets, and protocols. Although the National ITS Architecture is generic, it can be tailored to meet unique local or regional transportation needs. In the architecture, functions, subsystems, and data flows have precise definitions and associated data elements, which facil- itates data exchange within and among jurisdictions at several levels. Readers should note that the National ITS Architecture is not a system architecture in that it does not prescribe spe- cific hardware or software configurations and interfaces, leav- ing that responsibility to individual agencies that implement the systems. Development, Challenges, Strategies, and Adaptability The U.S.DOT manages the implementation of the National ITS Architecture through a program governed by a board of directors called the ITS Management Council (90). In 2006, the RITA Administrator became the Chair of the ITS Man- agement Council and, in this capacity, has overall responsi- bility for the strategic direction and oversight of the ITS Pro- gram. In 2004, the ITS Management Council reorganized the functions of the ITS Program to focus on the following nine initiatives: 1. Vehicle Infrastructure Integration (recently renamed IntelliDriveSM), 2. Next Generation 9-1-1, 3. Cooperative Intersection Collision Avoidance Systems, 4. Integrated Vehicle Based Safety Systems, 5. Integrated Corridor Management Systems, 6. Clarus, 7. Emergency Transportation Operations, 8. Mobility Services for All Americans, and 9. Electronic Freight Management. The national ITS Program started with the Intelligent Vehicle/Highway Systems (IVHS) initiative in the late 1980s, which focused on the development and implementation of advanced technologies to improve mobility, enhance safety, and maximize the use of existing facilities, at a time when the bulk of the Interstate highway construction program was end- ing (91, 92). IVHS was an effort to better integrate a host of related technologies such as advanced traffic management sys- tems (ATMS), advanced traveler information systems (ATIS), advanced vehicle control systems (AVCS), and CVO. During the late 1980s and early 1990s, U.S. investments in IVHS were relatively minor, although growing. For exam- ple, U.S.DOT’s research expenditures in IVHS were $2.3 mil- lion in fiscal year 1990 but grew to about $20 million in fis- cal year 1991 (93). A number of organizations recognized the increasing role of advanced technology in transporta- tion and called for actions such as increasing the level of funding for research and demonstration programs, devel- oping organizational arrangements involving public and private sectors, and including IVHS in federal legislation. Impetus for work in this area in the United States was also the awareness of major IVHS investments in Europe and Japan and the concern that the United States might lose its competitive advantage and become dependent on foreign developments. Two significant efforts that shaped the future of IVHS were Mobility 2000 (91, 92) and TRB Special Report 232 (94). Mobil- ity 2000, an informal group of representatives of universities, industry, and federal, state, and local governments, conducted workshops in 1989 and 1990 that • Produced key recommendations (including developing an organizational structure to develop policy and legislative recommendations related to IVHS); • Estimated the investment needs in IVHS by different sectors (including federal, local, and private) to be around $34 bil- lion through year 2010; and • Identified institutional needs, including developing a frame- work to facilitate the development of standards for inter- faces and communications, noting that efforts should be made to coordinate ATMS, ATIS, AVCS, and CVO elements in a flexible manner to accommodate changes. Mobility 2000 led to the formation of IVHS America in 1990 (later to become ITS America) as a private, non-profit mem- bership organization with a mission to advise the U.S.DOT and serve as the primary representative of the IVHS community. TRB Special Report 232 was the result of an effort by the U.S.DOT, NCHRP, TRB, and private industry from 1988 to 1991 to document a vision for IVHS and formulate specific recommendations for implementing IVHS (94). The report evaluated the Mobility 2000 recommendations and identified key IVHS components and issues, including the following: • It formalized the concept of an IVHS system architecture as the framework within which individual systems and compo- nents would operate and relate to each other. The architec- ture would be a standards-based open system architecture to ensure component interoperability and interchangeabil- ity. The report also identified a requirement for the architec- ture to be flexible to accommodate changes in requirements and technology. 38

• It recognized the need to identify human-machine inter- faces, specifically, the information processing demands that IVHS would impose on human operators and users. • It highlighted the need to use a systems engineering approach for developing the architecture as well as the need to evalu- ate alternative architecture approaches. • It recommended the U.S.DOT (in a leadership role) and IVHS America (in an advisory role) work together to imple- ment the IVHS program. TRB Special Report 232 also included a summary of the development of other similar architectures, specifically, those associated with the Aeronautical Telecommunications Net- work (ATN), the Advanced Train Control Systems (ATCS) Project, and the European Dedicated Road Infrastructure for Vehicle Safety (DRIVE) Program. In 1991, ISTEA (69) resulted in significant increases in IVHS investment levels (95). As a reference, Table 6 provides a summary of critical pieces of federal legislation related to IVHS and ITS since 1991, along with the corresponding allo- cation of federal funds. In 1991, the U.S. General Accounting Office (GAO) reviewed 38 major studies conducted in the 1980s and concluded that IVHS was seen as promising, although the empirical evidence for judging its benefits was still limited (93). In addition to the need for a more detailed analysis of anticipated benefits and costs, the 1991 GAO report identified three types of barriers (cost barriers, institutional barriers, and technological barri- ers) that could affect the overall success of IVHS. In particu- lar were the following: • The report recommended finding a proper mix of burden sharing between private and government sectors to absorb the costs of developing and operating IVHS, noting that an inappropriate distribution of costs could prevent full real- ization of IVHS potential. • The report highlighted the requirement for various stake- holders to work together to ensure the success of IVHS and the need to focus on setting standards (which would also require cooperation and coordination among participants). The ISTEA mandate called for a three-pronged effort, including basic research and development, operational tests, and deployment support activities. Nonetheless, a 1994 GAO report on traffic control signal systems found that federal protocols to review state and local governments’ operations plans for signal systems were inconsistent and that technical expertise deficiencies of FHWA staff, which FHWA had iden- tified in 1990, had not improved significantly (95). During the early 1990s, the U.S.DOT developed strategic and program plans for implementing IVHS. At that time, FHWA had primary responsibility for the program, although the FTA and NHTSA also had active roles in funding and managing IVHS projects. State and local governments, private industry, and the research community were active participants in shaping the program and conducting research and opera- tional tests. Significant input also was available through IVHS America. A critical initiative in the early 1990s was the development of a national architecture and standards for IVHS. The main motivation for this development was the recognition that the absence of common IVHS architecture and standards in Europe was having a negative impact on the European Com- munity’s goal of a seamless IVHS environment across national boundaries, delaying the development of a common market for European IVHS products. The vision for the national IVHS architecture in the United States was that it would define a gen- eral framework within which IVHS system components would work, while standards would specify the technical require- ments of individual IVHS applications. Developing a national IVHS architecture would ensure compatibility among differ- ent IVHS hardware and software technologies and accelerate the implementation of IVHS by reducing the risks to private- and public-sector stakeholders. Without the assurance of compatibility, stakeholders would be reluctant to invest in IVHS infrastructure. 39 L e g i s l a t i o n C o m m e n t / A l l o c a t i o n o f F u n d s * ISTEA (ITS Program: 1992–1997) $659 million for research and testin g $564 million for deployment Total: $1.22 billion National Highway System Designation Ac t Rep laced IVHS with ITS. TEA-21 (I TS Program: 199 8–2005), including a 2-ye ar extension $823 million for research and testin g $923 million for deployment Total: $1.75 billion SAFETEA-LU (ITS Program : 2006–2009) $440 million for research and testin g Deployment was disc on tinued Total: $440 million * Fu nd allocati on in formation provided by th e U.S.DOT. Table 6. Important pieces of federal legislation related to IVHS and ITS.

To develop a common national IVHS architecture, the U.S.DOT instituted an IVHS architecture development pro- gram and contracted several key aspects of this development, including the following (96): • System architecture development. The first phase involved four contractors (Hughes Aircraft, Loral, Rockwell Inter- national, and Westinghouse) to develop candidate IVHS architectures (97). The second phase involved a consor- tium between Loral and Rockwell International to develop the most promising architecture concepts from the first phase into a single architecture. This architecture was com- pleted in July 1996. • System architecture manager. The purpose of this con- tract (awarded to NASA’s Jet Propulsion Laboratory) was to work closely with the architecture development teams, providing technical review and evaluation of the candidate architectures. • System architecture consensus building. The purpose of this contract was to develop an outreach program, includ- ing regional briefings on the progress of the IVHS architec- ture definition effort. • Commercial vehicle operations institutional issues. The purpose of these studies was to evaluate institutional issues that would impede the achievement of national commer- cial vehicle operations goals. The U.S.DOT has continued to support the architecture through deployment and maintenance contracts. According to information provided by the U.S.DOT, the total federal investment on the architecture program has been $65 million so far. During the mid 1990s, ITS—federal legislation replaced IVHS references with ITS in 1995—grew rapidly, from a few projects in 1992 to 268 projects in 1995. Appropriations also grew to more than $800 million (98, 99). By the end of 1996, the total federal funding committed to ITS since 1991 had grown to $1.2 billion. In the mid 1990s, FHWA changed the focus of the ITS Pro- gram from research and operational tests to deployment and training. FHWA viewed outreach and training as critical because of the realization (backed by several studies) that many local officials did not have the technical skills needed to operate and maintain ITS infrastructure investments (100). One of the reasons for this shortage was that most transporta- tion agencies had staff with a background in civil engineering, not electrical engineering or systems integration. Lack of ITS awareness also was common among agency managers and decisionmakers. These limitations were barriers to successful ITS deployment. Additional barriers were the lack of eco- nomic models that local transportation officials could use to determine the costs and benefits of ITS implementations, making it difficult to justify expenditures on ITS-related proj- ects (100) and a lack of funds at the local level to support these projects in light of other transportation priorities (101). In 1998, TEA-21 consolidated this trend by launching a transition to more integrated ITS application deployments. In the process, it consolidated eight ITS program areas into two subprograms: infrastructure (metropolitan infrastruc- ture, rural infrastructure, and commercial vehicle infrastruc- ture) and intelligent vehicle initiatives (including Commercial Vehicle Information Systems and Networks [CVISN]) (102). It also recognized the need to accelerate the development of standards and the identification of critical standards to ensure national interoperability. Specific strategies the U.S.DOT pur- sued at that time to address challenges affecting ITS deploy- ments included the following: • Accelerate the development of standards, • Provide professional capacity training, • Conduct ITS infrastructure and vehicle research, • Provide ITS deployment assistance, • Conduct workshops to encourage consistency with the National ITS Architecture and standards, • Showcase the benefits of integrated deployments, and • Evaluate the ITS program. In 2001, the U.S.DOT finalized a rule (23 CFR 940.9) requiring ITS projects to conform to a regional architecture (103). The purpose of the rule was to ensure compliance with national standards in a regional, integrated way. Regions and states were required to complete their regional ITS architec- tures by April 2005 (if they had ITS implementations in 2001) or within 4 years of the first ITS project advancing to final design (if the region or state did not have an ITS implemen- tation in 2001). In 2005, SAFETEA-LU ended the ITS deployment pro- gram, although it continued to support ITS research and oper- ational testing at $110 million each year through fiscal year 2009. (Note: ITS projects are still eligible for regular federal- aid highway funding.) Relevant provisions in SAFETEA-LU related to ITS in connection with this research included the following: • It required states and regions developing or updating their regional ITS architectures to address real-time highway and transit information needs, and the systems needed to meet those needs. The regional ITS architectures also had to incorporate data exchange formats to ensure the data from highway and transit monitoring systems could be made available to state and local governments as well as to the traveling public. 40

• It required the designation of a panel of experts to recom- mend ways to expedite and streamline the process of devel- oping standards and protocols. In 2005, GAO (which by then had been renamed as the U.S. Government Accountability Office) reviewed a variety of reports that documented ITS deployments around the coun- try and interviewed officials from several agencies at the fed- eral, state, and local level (104). The report concluded that, although ITS technologies could be beneficial to help relieve congestion, the original goal to deploy ITS systems to relieve congestion had not been met. The report highlighted that measures the U.S.DOT had in place to determine deployment levels (e.g., whether a metropolitan area had transportation management centers) were inadequate and did not take into consideration other factors such as operational requirements (e.g., number of hours a center had to operate each day). The report also identified a number of barriers to ITS deploy- ment, including the following: • At the state or local levels, viewing options such as adding a new highway lane more favorably than ITS when decid- ing how to spend transportation funds; • Lack of funding for both ITS installations and operations and lack of awareness that federal funds also can be used for operational costs; • Lack of technical expertise at the local and state level; and • Lack of technical standards, slow pace in standard devel- opment, or standards that do not keep pace with techno- logical advances. According to information provided by the U.S.DOT, the total federal investment on the development of standards has been $109.3 million so far ($20 million under ISTEA, $68 million under TEA-21, and $21.3 million under SAFETEA- LU). As previously mentioned, there are 96 ITS standards in the RITA ITS standard database (without including with- drawn or suspended standards) (89). Nonetheless, the lack of standards and the slow pace in standard development are fre- quently cited as important factors that explain ITS deploy- ments delays. The standards development process can be a lengthy process. In some cases, technological innovations evolve faster than standards. The RITA administrator became the chair of the ITS Man- agement Council in 2006 (90). The ITS Management Coun- cil develops and directs federal ITS policy and ensures the effectiveness of the ITS Program. Members of the council include the following: • Under secretary of transportation for policy, • Assistant secretary for transportation policy and inter- modalism, • U.S.DOT’s chief information officer, • FHWA administrator, • FMCSA administrator, • FTA administrator, • NHTSA administrator, • RITA administrator (chair), • FRA administrator, and • Maritime Administration (MARAD) administrator. The ITS Strategic Planning Group advises the ITS Manage- ment Council. The group, which is chaired by the ITS program manager, includes members at the associate administrator and office director level. The ITS program manager leads the ITS Joint Program Office (JPO), which includes program managers and coordi- nators of the U.S.DOT’s multimodal ITS initiatives. The program includes staff support for functions such as Website development and maintenance, outreach, program evalua- tion, training, architecture, and standards. The ITS Joint Pro- gram Office is administratively located in FHWA under the policy direction of RITA. Archived ITS Data The national ITS Program evolved primarily to assist real- time and near-real-time transportation operations needs. Although placeholders for transportation planning needs were included in the National ITS Architecture from the beginning, and there were examples of traffic sensor data archival efforts going back to the 1970s, the process to develop an archived data user service (ADUS) only started in 1997 after the first release of the National ITS Architecture (105). ADUS was added as a user service in Version 3 of the architecture in 1999. As with other user services, which represent what a system would do from the perspective of the user, ADUS provides tools and describes processes related to ITS data archiving. Although all ITS deployments use and/or produce data, ADUS is not mandatory. However, having ADUS in the architecture facilitates the inclusion of ITS data archival functions in ITS deployments. Over the last 10 years, the focus on ADUS development has been the development of standards. Currently, there are three ITS standards for archived ITS data. The first standard, pub- lished in 2003, provides guidance for archiving and retrieving ITS data. The second standard, published in 2006, contains a metadata standard for ITS data. The foundation for this stan- dard was the FGDC metadata standard (106). The third stan- dard, published in 2008, contains a detailed data dictionary for archived ITS data. Unfortunately, additional work on archived ITS data standards stopped due to lack of funding. This funding was used to pay a consultant to do the technical 41

work and support travel of public-sector officials to attend standards development meetings. Lessons Learned Lessons learned in connection with the development, evo- lution, and maintenance of the National ITS Architecture (and associated standards) follows: • Involve stakeholders early and often. It was critical to involve various stakeholders (federal, state, and local gov- ernments, as well as private industry) early to develop a vision for a national ITS Program. These stakeholders also had a clear picture of what a national ITS architecture (and related standards) should focus on and accomplish. Nonetheless, it took nearly 3 years to develop and docu- ment that picture. In addition, although the U.S.DOT has played a critical leadership role in the development and implementation of the national ITS Program, other stake- holders also have played a critical role in shaping that pro- gram. For example, ITS America continues to provide an advisory, advocacy role on behalf of some 450 member organizations that include public-sector agencies (including state, county, and local levels) and private-sector agencies. Roughly half of the member organizations in ITS America are public-sector agencies. • Develop and compare candidate architecture concepts. The National ITS Architecture as implemented was the result of a two-phase approach. The first phase involved having competing teams develop candidate architecture concepts. The second phase involved selecting a consortium from the first phase and developing an architecture using the best elements from the first phase. • Consider federal legislation to support and develop the program. The ITS program was a major initiative at a time when the bulk of the Interstate highway construction pro- gram was ending. Without the support of federal legisla- tion, the U.S.DOT would not have received the level of funding needed to develop and implement the program, as well as to help maintain national attention on that program. Developing and maintaining the National ITS Architecture and standards also was included in the federal legislation. • Develop long-term plans with flexibility in mind. The national ITS Program has evolved since its inception in the early 1990s. Along the way, changes have been instituted to respond to issues that were not anticipated in the original vision. For example, the requirement to develop regional ITS architectures evolved as a strategy to encourage com- pliance with national standards in a regional, integrated way. There are also ambitious goals that have not fully material- ized yet. For example, when the ITS program started, the goal was to fully deploy ITS systems at all major metropol- itan areas in the country. However, partly because of the lack of appropriate measures to determine whether that goal was being attained, the current level of ITS deploy- ment is not what the original visionaries had in mind. Like- wise, although it was clear from the beginning that national ITS standards had to be developed to ensure compatibility and interoperability, managers of the ITS Program did not anticipate the slow pace with which ITS standards would be developed. • Develop tools to measure benefits and costs early. For many years, the ITS community did not have access to prac- tical tools to measure the costs and benefits of ITS imple- mentations. To assist in this process, RITA now has on its Website benefit and cost databases (including unit costs) to help planners and engineers determine the technical and economic feasibility of their proposed projects. The need for this type of tools became critical after SAFETEA-LU ended the ITS deployment program and ITS projects had to compete for funding just like any other transportation proj- ect. A critical requirement in this process is the develop- ment of appropriate performance measures to determine the effectiveness of ITS investments. • Develop and implement professional capacity and train- ing programs early. A factor that hampered acceptance and implementation of ITS deployments was the lack of techni- cal skills in critical areas (e.g., systems integration and elec- trical engineering) to operate and maintain ITS infrastruc- ture investments. Lack of ITS awareness also was common among agency managers and decisionmakers. These limi- tations were barriers to successful ITS deployments. • Integrate archived data needs into frameworks and architectures early. The national ITS program evolved primarily to assist real-time and near-real-time transporta- tion operations needs. Although placeholders for trans- portation planning needs were included in the National ITS Architecture from the beginning, the recognition of the need for an archived data user service did not happen until the National ITS Architecture was already published. The development of ITS archived data standards also has been slow. Although there are now three ADUS-related data standards, there are no documented examples of their use yet. National Spatial Data Infrastructure Purpose and Content NSDI is a dissemination effort to “acquire, process, store, distribute, and improve utilization of geospatial data through- out all levels of government, the private and non-profit sec- tors, and academia” (106). NSDI is managed by FGDC. NSDI goals include (1) reducing duplicative efforts among agencies; (2) improving quality and reducing the costs of geospatial 42

data; (3) making the benefits of geographic data more acces- sible to the public; and (4) establishing partnerships to increase data availability. NSDI includes five major components, as follows (106): • Framework. The framework is a collaborative approach and effort to facilitate the development of datasets that are critical at the national level. The framework has three parts (sometimes presented as four components: information content, technical context, operational context, and business context [107]), as follows: – Seven data themes (also called framework data): geo- detic control, orthoimagery, elevation and bathymetry, transportation, hydrography, cadastral, and governmen- tal units; – Procedures and references for building and using frame- work data, e.g., spatial data models, permanent feature identification codes, support for multiple resolution lev- els, and a common coordinate referencing system; and – Institutional arrangements and business practices to encourage the maintenance and use of the data, e.g., through open, distributed access to framework data. • Metadata. FGDC is responsible for maintaining a meta- data standard for geospatial data called Content Standard for Digital Geospatial Metadata (CSDGM). CSDGM became mandatory for federal agencies in January 1995. Nation- wide, state and local agencies are increasingly adopting and using CSDGM, partly because of the availability of user- friendly CSDGM editors such as those included in com- monly used GIS applications. • Standards. FGDC maintains a list of FGDC-endorsed stan- dards. The standards, which are sponsored and maintained by different organizations including FGDC, cover areas such as data transfer, data content, and geospatial positional accu- racy. The status of a standard throughout its life cycle could be one of several options, including reaffirmed, to be deter- mined, not applicable, requiring changes, or retired. It is not clear to what degree FGDC-endorsed standards are used nationwide, particularly by agencies other than the federal agencies that sponsor and/or have maintenance responsibil- ity for individual standards. In the case of content standards (e.g., cadastral, digital orthoimagery, remote sensing swath data, framework data, and utility facilities—which has been retired), the GIS industry is developing and promoting spa- tial data models outside the FGDC environment, which, in practice, might render some FGDC content standards irrel- evant, particularly for state and local agencies. • Clearinghouse Network. The Clearinghouse Network is a community of distributed data providers that publish infor- mation about, and links to, available digital spatial data and services. FGDC coordinates sharing geographic data, maps, and online services through the geodata.gov portal (108). • Partnerships. Partnerships include institutional arrange- ments with federal agencies and other recognized stake- holder groups that share a common interest in critical data themes, standards, metadata, and information sharing. To support this effort, FGDC has developed an interagency organizational structure that includes a steering commit- tee, a coordination group, working groups and subcom- mittees, and partner organizations. Interaction with other agencies also takes place through a variety of initiatives, including the following: – Fifty State Initiative, which focuses on assisting states in developing strategic and business plans to facilitate pro- grams, policies, and technologies to support NSDI; – NSDI Cooperative Agreements Program, which focuses on assisting the geospatial data community through funding and other resources in implementing NSDI components; and – Geospatial Line of Business (LoB), which is a presiden- tial initiative that focuses on fostering collaboration, reducing redundancies, and improving accountability and transparency across the federal government. Institutional Arrangements, Challenges, and Strategies Important milestones in the development and evolution of FGDC and NSDI include the following: • In 1983, OMB established the Federal Interagency Coordi- nating Committee on Digital Cartography (FICCDC), from which FGDC evolved (109). • In 1990, OMB revised Circular A-16 to establish FGDC within the Department of the Interior to support the nation- wide use, sharing, and distribution of geospatial data (110). • In 1994, Presidential Executive Order 12906 made FGDC responsible for coordinating the development of a national spatial data infrastructure to address redundancy and incompatibility issues related to geospatial information (110). The same year, FGDC developed a strategy for NSDI with help from stakeholders (111). • In 1997, FGDC developed an updated strategy for NSDI to continue major components of NSDI and to increase awareness (112). • In 2002, OMB revised Circular A-16 to reflect changes in geographic information management and technology, fur- ther describe NSDI components, and assign agency roles and responsibilities for the development of NSDI (113). • In 2003, FGDC started a new initiative, called the NSDI Future Directions Initiative, to develop a geospatial strategy and implementation plan for further developing NSDI (114). • In 2007, the U.S. Geological Survey (USGS) selected a con- tractor to manage the Geospatial LoB initiative under the coordination of FGDC (115, 116). 43

The 1994, 1997, and 2003 strategic plan documents reflect efforts by FGDC to maintain and continue developing NSDI. Most of the initiatives and programs currently in place at FGDC (see previous section) are the result of those plans. However, the 2004 NSDI Future Directions Initiative report acknowledged a number of issues and needs voiced by stake- holders (114), including the following: • Need for more effective data sharing and coordination within the entire geospatial community, • Need for a level playing field in the design and implemen- tation of NSDI and a need for FGDC to play a neutral facil- itator role, • Lack of an effectively communicated shared vision, • Lack of a clear business case for stakeholder participation, and • Emphasis on isolated geospatial programs at many govern- ment agencies. Other reports have produced similar observations. For example, GAO reports in 2003 and 2004 concluded that the NSDI program was successful in promoting basic concepts, the clearinghouse, and development of several standards, including CSDGM (110, 117). The CSDGM standard, in par- ticular, is increasingly used outside federal agencies. How- ever, the reports noted a number of issues, including that developing standards to meet stakeholder needs and achiev- ing stakeholder participation remained a challenging task, and that the FGDC reporting process was not sufficiently developed. More importantly, the GAO reports concluded that the NSDI programs had not resulted in significant reduc- tions in geospatial data redundancy and costs or improve- ments in geospatial data accuracy. Reasons mentioned include the following: • Lack of up-to-date strategic plans with specific measures for identifying and reducing redundancies, • Many federal geospatial datasets not being compliant with FGDC standards or published outside NSDI clearinghouse procedures, and • Lack of effectiveness in OMB’s oversight of federal geospa- tial activities. A recent National States Geographic Information Council report concluded that there was a need to refocus national efforts to complete the development of NSDI and to devise appropriate data maintenance methods (118). This report produced several recommendations, including the following: • Formulate an effective national strategy for implementing NSDI across federal, state, and local levels; • Make federal NSDI funding contingent on compliance with collaboratively established criteria and requirements (i.e., similar to the federal highway funding model); and • Develop a national strategy to communicate about, and advocate for, NSDI. Finally, although the level of awareness about NSDI is increasing, many geospatial data stakeholders (particularly outside the federal government) have difficulty understand- ing NSDI’s purpose, its governance structure, or its products. There is plenty of documentation (e.g., reports, brochures, presentations, and papers) about NSDI and FGDC, much of it on the FGDC Website (106). However, navigating through this information is difficult because the information is not properly grouped or indexed (e.g., by subject or date of publication) which means it is common to find information without proper thematic or temporal context. An example of this situation is the description of the NSDI framework on the NSDI Website. On some pages, the framework is described as having three parts, but on other pages, it is described as hav- ing four components. Without a proper thematic or tempo- ral context, it is difficult to understand which categorization is current or if there is a difference between parts and compo- nents. Likewise, the NSDI Website provides various incon- sistent definitions even for basic terms such as NSDI, NSDI framework, clearinghouse, and partnerships. Lessons Learned Lessons learned in connection with the development, evo- lution, and maintenance of NSDI follow: • Articulate programs well and provide good documenta- tion. Of the five main NSDI components, the metadata and the clearinghouse components have been considered successful. Nationwide, state and local agencies are increas- ingly adopting and using the CSDGM metadata standard, partly because of the availability of user-friendly CSDGM editors such as those included in commonly used GIS applications. The acceptance of other NSDI components has been mixed. One of the reasons NSDI has not been more successful is the lack of adequate, consistent, prop- erly indexed information and documentation on the FGDC Website. NSDI is frequently considered an ambigu- ous concept. As a result, many geospatial data stakeholders do not really understand what NSDI is, its purpose, its gov- ernance structure, or even its products. • Develop systems that are relevant to stakeholders. As men- tioned, the CSDGM standard has been successful partly because user-friendly CSDGM editors are now included in commonly used GIS applications. Another reason is that CSDGM is more comprehensive than the relatively simple 44

data dictionaries in use at many agencies nationwide. In other words, migrating to a “better” standard, particularly when the standard implementation is already available on a user-friendly interface, is a logical step. In contrast, migrat- ing to one of the FGDC data content standards might not be desirable, particularly at the state or local level, considering that the GIS industry is developing and promoting spatial data models outside of the FGDC standards environment, which, in practice, might render some FGDC content standards irrelevant. • Provide incentives to encourage participation, particu- larly in the case of state and local entities. A major imped- iment cited in relation to the promotion of NSDI nation- wide is that states and local jurisdictions do not perceive a benefit in implementing NSDI within their jurisdictions. Lack of funding is another reason frequently cited for the lack of acceptability of NSDI nationwide. National Transportation Atlas Database Purpose and Content NTAD is a product compiled and published by BTS that contains several geographic databases of interest to the trans- portation community. The transportation atlas is available on digital video disk (DVD) and for download on the BTS Web- site (119). Table 7 shows the list of datasets included in the 2009 version of NTAD. Many of those datasets are relevant to freight planning and operations. Development, Challenges, Strategies, and Adaptability ISTEA created BTS with the mission to enhance transporta- tion data collection, analysis, and reporting (69). BTS received $90 million to support its activities over a first 6-year period starting in fiscal year 1992. NTAD was one of the early initia- tives that BTS undertook (120). TEA-21 and SAFETEA-LU reemphasized this commitment by requiring BTS to main- tain geographic databases that depict transportation net- works; flows of people, goods, vehicles, and craft over the net- works; and social, economic, and environmental conditions that affect, or are affected by, the networks (70, 71). BTS released the first NTAD version in 1995, and since then it has continued to publish yearly updates. NTAD was originally published on CDs and then on DVDs. The latest NTAD version is also available online (119). The North Amer- ican Transportation Atlas Database (NORTAD) was a special 45 Dataset Type Maintained By 111th Congressional Districts Boundaries Polygon U.S. Census Bureau Airport Runways Polyline FAA Airports Point FAA Alternative Fuels Point BTS Amtrak Stations Point FRA Automatic Traffic Recorder Stations Point FHWA Core Based Statistical Areas Polygon U.S. Census Bureau Fixed-Guideway Transit Facilities Polyline FTA Freight Analysis Framework Polyline FHWA Hazardous Material Routes Polyline FMCSA Highway Performance Monitoring System Polyline FHWA Hydrographic Features Polygon BTS Hydrographic Features Polyline BTS Intermodal Terminal Facilities Point BTS Metropolitan Planning Organizations Polygon BTS and FHWA National Bridge Inventory Point FHWA National Highway Planning Network Point FHWA National Highway Planning Network Polyline FHWA National Park System Boundary Dataset Polygon National Park Service National Populated Places Point U.S. Census Bureau Navigable Waterway Network Polyline USACE Non Attainment Areas Polygon EPA Ports Point USACE Railroad Grade Crossings Point FRA Railway Network Point FRA Railway Network Polyline FRA U.S. County Boundaries Polygon U.S. Census Bureau U.S. Military Installations Polygon Transportation Engineering Agency U.S. State Boundaries Polygon U.S. Census Bureau Urbanized Area Boundaries Polygon U.S. Census Bureau Weigh in Motion Stations Point FHWA Table 7. Datasets included in the 2009 version of NTAD (119).

version in the series, which was published in 1998 in collab- oration with Canada and Mexico (121). NORTAD contained updated NTAD datasets, border crossings, and datasets depict- ing Canadian and Mexican transportation infrastructure facilities. Early NTAD versions followed an American Standard Code for Information Interchange (ASCII) fixed-record-length for- mat that included six distinct record types, as follows (121): 1. Link (.lnk file extension); 2. Node (.nod file extension); 3. Point (.pnt file extension); 4. Area (.are file extension); 5. Geography (.geo file extension), which contains shape infor- mation for network links and area boundaries; and 6. Attribute (.txx file extensions, e.g., .t01, .t02, and .t03). Combinations of different record types defined the geom- etry, topology, and attributes of point, network, and area fea- ture types. BTS made these file formats available to GIS ven- dors on a temporary basis to facilitate the development of data conversion software tools. In practice, although BTS intended to migrate NTAD to the federal Spatial Data Transfer Standard (SDTS), the ESRI shapefile format became a de facto standard. As a result, since the early 2000s, BTS has published NTAD in this format. It is also worth noting that, although ASCII is still in use, it is quickly being replaced by more modern encoding schemes such as Unicode. A number of agencies contribute data to NTAD, includ- ing BEA, FAA, FHWA, FMCSA, FRA, FTA, the U.S. Census Bureau, and USACE (Table 7). BTS collects data from these participating agencies and then processes and publishes the data within NTAD. Data gathering for NTAD typically starts in November of each year. In general, data creation, maintenance, update, spatial positional accuracy and resolution, and quality control responsibilities remain with the data source agencies. Depending on data format, quality, and completeness, BTS processes data by checking spatial and attribute data character- istics, inserting additional attribute data, and recompiling meta- data using the FGDC metadata standard (122). Most data pro- cessing takes place in an ESRI ArcGIS™ environment. Lessons Learned NTAD is a popular data series that remains an important program within BTS and RITA, along with the operation of the FGDC transportation subcommittee (123). Budgetary changes have forced the agency to cut back other GIS programs. Lessons learned in connection with the development, evo- lution, and maintenance of NTAD follow: • Implement interagency data exchange programs with centralized data coordination. NTAD is a product led by BTS with data support from a large number of agencies. BTS collects data that those agencies produce and/or main- tain, reprocesses the data into a central repository, and then makes the data available to users. This approach takes advantage of existing resources at individual data produc- ers while avoiding data collection and data processing redundancy. BTS’s coordination role helps to improve the availability and standardization of critical transportation data and metadata. • Develop spatial data programs that use industry stan- dards. Originally, NTAD used an ASCII fixed-record-length format. However, modern encoding schemes such as Unicode started replacing the ASCII format. In addition, although data conversion tools to enable the conversion of NTAD data to other file formats were available, in practice, the ESRI shapefile format became a de facto standard. As a result, since the early 2000s, BTS has published NTAD in this format. The advantage of using a de facto standard such as the ESRI shapefile format is that this format is widely used, which facilitates data exchange. The disadvantage is that the shapefile format is an old data format that ESRI no longer maintains (although ESRI GIS software applica- tions provide shapefile format backward compatibility). With the introduction of ArcGIS 8 in 1999, ESRI started using “geodatabases” to store geographic datasets. A geo- database could be a “personal geodatabase” (in Microsoft® Access® format) or an “ArcSDE™ geodatabase” (with the data stored in a relational database such as Oracle®, Microsoft Structured Query Language [SQL] Server®, IBM® DB2®, or IBM Informix®) (124). Recently, ESRI introduced a “file geodatabase” format to address the file size limita- tions of personal databases (Access files cannot exceed 2 gigabytes in size). Although the use of geodatabases is increasing, at this point it is not clear to what degree federal agencies have started to require their use for data exchange purposes. General Observations and Lessons Learned The previous sections included a summary of data sources, systems, and architectures relevant to the freight community. Table 1 provides a listing of the freight data sources, systems, and architectures reviewed, whereas Appendix A of the con- tractor’s final report (available on the project web page) includes a detailed description of each data source, system, or architecture. Although not comprehensive, the review pro- vides a sample of the typical data sources, systems, and archi- tectures that could be included in a national freight data architecture, as well as any potential implementations that could derive from that data architecture. In general, the material in Appendix A describes the cur- rent status and characteristics of the data sources, systems, 46

and architectures reviewed. For the research, it also was impor- tant to learn about the process, institutional arrangements, and challenges associated with the development and mainte- nance of the systems, as well as any strategies that have been deployed to address challenges and other data integration issues, such as data quality, timeliness, and proprietary and privacy concerns. Given the large number of data sources, sys- tems, and architectures uncovered during the literature search (49 according to Table 1), it was not practical to conduct a comprehensive review of historical developments for all of them. However, a few systems and architectures in Table 1 were of interest because of the processes that led to their devel- opment (e.g., in the form of issues, motivations, challenges, legislative efforts, and implemented strategies). Lessons learned from those processes can provide valuable information for, and help to minimize the cost of, the development and imple- mentation of a national freight data architecture. The sample of systems and architectures selected for the detailed historical analysis covered a wide spectrum of topics related to freight transportation and included the following: • ACE/ITDS, • Carload Waybill Sample, • CFS, • EDI standards, • FAF, • HPMS, • NIPAs, • National ITS Architecture, • NSDI, and • NTAD. For this sample of systems and architectures, the analysis covered several topics, including the following: • Purpose and intended benefits; • Content; • Institutional arrangements used for developing and main- taining the system or architecture; • Challenges and issues faced in creating and maintaining the architecture or system; • Strategies and methods for dealing with data integration issues, such as data quality, timeliness, and proprietary and privacy concerns; • Adaptability to serve evolving purposes and data sources; and • Assessment of how well the system or architecture works in the form of lessons learned. In general, all of the systems and architectures reviewed in this report (including those described in detail in this chap- ter) relate to processes that affect one or more freight trans- portation aspects or components (see Chapters 3 and 4 for a more detailed discussion of those aspects and components). From this perspective, all of the systems and architectures in this report should be included at some level in the design of a national freight data architecture if this data architecture is indeed designed to address the needs of both public and pri- vate decisionmakers not just at the national level, but also at the state and local levels. However, identifying which data sources, systems, and architectures to include in a national freight data architecture depends to a large degree on the vision that is laid out for this data architecture. As Chapter 4 describes in more detail, it is possible to outline a number of competing implementation alternatives, some of them more comprehensive and ambi- tious in scope than others. For example, a freight data archi- tecture that only focuses on commodity flow aspects for plan- ning purposes at the national level requires data sources, systems, and architectures that are both commodity-related and relevant at the national level. By comparison, a freight data architecture that includes commodity, transportation network, vehicle, and safety aspects to address the needs of federal and state governments would require the inclusion of a larger number of finer resolution data sources, systems, and architectures. Likewise, a freight data architecture that includes real-time supply chain data from the private sector would require conducting an analysis of the risks the private sector would assume if it shares sensitive and/or confidential data elements that could undermine its competitive position and decisionmaking process. Chapter 4 addresses different implementation alternatives that may be possible for a national freight data architecture. However, a summarized list of lessons learned from all of the systems and architectures reviewed is appropriate here because many of the lessons learned are sufficiently generic and, consequently, could be used to guide the development and implementation of any freight data architecture, regard- less of implementation level. The summarized list of lessons learned follows: • Develop systems that are relevant to stakeholders, include adequate stakeholder participation, and provide incentives to encourage participation, particularly in the case of state and local entities (ACE/ITDS, NSDI); • Clearly define expected outcomes and development and coordination plan (ACE/ITDS); • Articulate programs well; provide clear, uniform guidance; and provide good documentation (Carload Waybill Sample, NSDI); • Develop applications that rely on widely used data standards (EDI standards, National ITS Architecture, NSDI); • Develop and compare candidate architecture concepts (National ITS Architecture); 47

• Consider federal legislation to support and develop the program (National ITS Architecture); • Develop tools to measure benefits and costs early (National ITS Architecture); • Integrate archived data needs into frameworks and archi- tectures early and develop data programs that use industry standards (National ITS Architecture, NTAD); • Implement interagency data exchange programs with cen- tralized data coordination (NTAD); • Use available data sources and develop long-term plans while keeping systems flexible to respond to changes and new data sources (FAF, HPMS, National ITS Architecture); • Schedule major and regular revisions effectively while avoid- ing scope creep (HPMS, NIPAs); • Develop systems that are consistent with input data limi- tations (FAF); • Develop applications with backward compatibility (EDI standards); • Evaluate data disaggregation level requirements to ensure statistical significance (HPMS); • Provide adequate resources for data collection, fully under- stand the implications of small sample sizes, and continue to involve the U.S. Census Bureau for the use of survey instruments (CFS); • Emphasize data access, quality, reliability, confidentiality, and integrity (Carload Waybill Sample, FAF, NIPAs); • Participate in the standards development process (EDI stan- dards, National ITS Architecture, NSDI); • Create crosswalks to ensure compatibility of survey data internally over time and externally across other datasets (CFS); • Involve stakeholders early and often through a variety of mechanisms and technologies (ACE/ITDS, FAF, HPMS, National ITS Architecture); and • Develop and implement professional capacity and training programs early (National ITS Architecture). 48

Next: Chapter 3 - Surveys, Interviews, and Peer Exchange »
Guidance for Developing a Freight Transportation Data Architecture Get This Book
×
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

TRB’s National Freight Cooperative Research Program (NCFRP) Report 9: Guidance for Developing a Freight Transportation Data Architecture explores the requirements and specifications for a national freight data architecture to link myriad existing data sets, identifies the value and challenges of the potential architecture, and highlights institutional strategies to develop and maintain the architecture.

The report also includes an analysis of the strengths and weaknesses of a wide range of data sources; provides information on the development of a national freight data architecture definition that is scalable at the national, state, regional, and local levels; and offers readers a better understanding of the challenges that might block the implementation of a national freight data architecture as well as candidate strategies for developing, adopting, and maintaining it.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!