Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 1
1
SUMMARY
Guidance for Developing a Freight
Transportation Data Architecture
Introduction
The movement of freight in the United States continues to grow, causing congestion along
corridors and at network nodes such as seaports, land ports of entry, truck and rail terminals,
and airports. It is critical to have accurate, comprehensive, and timely information about freight
movements and the impact of these movements on the transportation network in order to make
sound investment decisions to improve and optimize the freight transportation system.
This report documents the results of a study to develop specifications for content and
structure of a national freight data architecture that serves the needs of public and private
decisionmakers at the national, state, regional, and local levels. It is worth noting that the
purpose of NCFRP Project 12 was to develop requirements and specifications for a national
freight data architecture, not to develop the data architecture (which would be a logical next
step after identifying those requirements and specifications). The research team undertook
the following activities to address these research needs:
· Completed a review of systems, databases, and architectures that might be used as a poten-
tial reference for the development of a national freight data architecture;
· Conducted surveys and follow-up interviews, interviews with subject matter experts, and
a peer exchange with freight transportation stakeholders;
· Developed a formal definition for a national freight data architecture;
· Identified high-level categories of data architecture components;
· Identified potential implementation approaches;
· Developed a list of specifications for a national freight data architecture; and
· Identified challenges and strategies related to the implementation of a national freight
data architecture.
Data Sources, Systems, and Architectures
Various listings, links, and summaries of systems, databases, architectures, and other related
documents that pertain to freight transportation data are available in the literature. Although
there is a wealth of sources of information that pertain to freight transportation, a comprehen-
sive catalog of freight-related data sources at different geographic levels (including national,
state, regional, and local levels) does not exist. As a reference, the research team conducted a
review of a sample of freight-related data sources at the national level to complement or other-
wise extend existing listings. This sample is obviously not comprehensive. For example, it does
not reference datasets that state, regional, and local entities need to collect to supplement or
augment national-level datasets. Although the sample of data sources evaluated does not
OCR for page 2
2
include all the potential data sources that deal with freight transportation, it is useful because
it provides a sample of the typical national-level data sources that would need to be included
in a national freight data architecture. A few systems and architectures were of particular inter-
est because of the lessons that could be derived from the processes that led to their develop-
ment. The analysis included topics such as purpose, content, institutional arrangements used
for developing and maintaining the system or architecture; challenges and issues faced in cre-
ating and maintaining the architecture or system; strategies and methods for dealing with data
integration issues; and adaptability to serve evolving purposes and data sources.
Online Surveys, Interviews, and Peer Exchange
The research team conducted a planner and analyst survey, a shipper survey, and a motor
carrier survey (as well as follow-up interviews) to gather information about freight data uses
and needs. The research team also conducted interviews with subject matter experts to
address specific items of interest to the research. The purpose of the planner and analyst sur-
vey was to gather information from government planners, analysts, and other similar
freight-related stakeholders. Respondents were involved in all modes of transportation,
including air, rail, truck, pipelines, and water. Respondents indicated that they use freight
data to support the production of a wide range of public-sector transportation planning doc-
uments, adding weight to the notion that the national freight data architecture should sup-
port a variety of freight-related processes. Respondents reported using and/or needing data
at various levels of geographic coverage and resolution. The feedback on unmet data needs
complement similar findings in the literature.
The purpose of the shipper survey was to gather general information from the shipper
community regarding freight data uses and needs, as well as willingness to share data with
other freight-related stakeholders. Feedback from respondents indicates that the shipper
industry collects large amounts of data. Many shippers and logistics service providers trans-
mit data electronically using electronic data interchange (EDI) technologies. However,
accessing data from shippers and logistics service providers for transportation planning appli-
cations, beyond aggregated data from commercial data providers and national survey cam-
paigns such as the Commodity Flow Survey (CFS), is not necessarily straightforward. For
example, while a data record might characterize a commodity as well as origin and destina-
tion locations, the route data component may be missing unless the carrier movement data
are included. In addition, the shipper stakeholders interviewed indicated they could not com-
ment on their companies' ability or willingness to share data for freight transportation plan-
ning purposes (particularly on a load-by-load basis, given its proprietary and confidential
nature). Subsequent feedback obtained at the peer exchange (see below) highlighted a num-
ber of strategies to address this issue, including initiating discussions about data sharing at a
sufficiently high administrative level--since low-ranking personnel might know the data, but
frequently do not have the authority or permission to discuss data sharing options. Involv-
ing trade associations rather than individual firms might also be beneficial.
The purpose of the motor carrier survey was to gather information from the motor carrier
community about freight data uses and needs, as well as willingness to share data with exter-
nal freight-related stakeholders. Carriers handle large amounts of disaggregated data during
the course of their business operations. Increasingly, carriers use EDI standards and applica-
tions. However, the amount of shipment information detail varies according to the type of car-
rier. For example, truckload (TL) carriers, who tend to bill customers on a per-mile basis or
by using a flat rate, rarely collect detailed commodity data. In addition, TL carriers are less
likely to collect data on tonnage hauled or tare-level data. By comparison, less-than-truckload
(LTL) carriers typically bill customers using a rate structure based on shipment weight, origin,
OCR for page 3
3
destination, and freight classification. However, LTL carriers tend to favor a freight-all-kinds
rating structure that assigns a general freight classification to all shipments regardless of freight
commodity or type. As opposed to TL carriers, LTL carriers are more likely to track total ton-
nage. Motor carrier reservations about sharing proprietary and confidential data were related
to the need to develop mechanisms to protect proprietary and confidential information and
to maintain the anonymity of carriers and customers. In general, carriers would need to know
in advance the specific uses of the data and, in return, would expect information in the form
of industry benchmarking metrics. It is worth noting that developing metrics of interest to the
private sector is part of the scope of work of NCFRP Project 3 "Performance Measures for
Freight Transportation."
In conjunction with the 2009 North American Freight Flows Conference held in Irvine,
CA, the research team organized a peer exchange to discuss preliminary research findings;
request feedback; and facilitate a dialogue on implementation strategies to develop, adopt,
and maintain a national freight data architecture. Participants included representatives of
federal, state, regional, university, and private-sector agencies. To encourage participation and
discussion, attendees received background materials such as relevant research topic sum-
maries and breakout group agendas and discussion objectives. Feedback from peer exchange
participants included recommendations for changes to initial research findings as well as a
list of issues, challenges, and strategies to consider during the implementation of the national
freight data architecture.
National Freight Data Architecture Definition
Taking into consideration the results of the literature review, as well as feedback from sur-
veys, follow-up interviews, and the peer exchange, the research team developed the following
generic definition for a national freight data architecture:
The national freight data architecture is the manner in which data elements are organized and inte-
grated for freight transportation-related applications or business processes. The data architecture includes
the necessary set of tools that describe related functions or roles, components where those roles reside or
apply, and data flows that connect roles and components at different domain and aggregation levels.
Depending on the specific level of implementation chosen for the data architecture, this
generic definition could be fine-tuned as follows:
· Single-application approach. In this case, the national freight data architecture would
become the manner in which data elements are organized and integrated for a specific
freight application or business process at the national level (e.g., commodity flows).
· Intermediate approaches (depending on the number of applications). In this case, the
national freight data architecture would become the manner in which data elements are
organized and integrated for a specific set of applications at the national, state, regional,
and local levels. A large number of intermediate approaches is possible, depending on
the business processes and geographic levels involved. For example, an intermediate
implementation approach could include commodity flows at the national, state, and
regional levels. Another, more encompassing, intermediate implementation approach
could include commodity flows, safety, and pavement impacts at the national, state,
regional, and local levels.
· Holistic, all-encompassing approach. In this case, the national freight data architecture
would become the manner in which data elements are organized and integrated for all freight
transportation-related applications or business processes at the national, state, regional, and
local levels.
OCR for page 4
4
For any of these implementation options, the data architecture would include the necessary
set of tools that describe related functions or roles, components where those roles reside or
apply, and data flows that connect roles and components.
National Freight Data Architecture Value
From the documentation and information gathered during the research, the research
team identified the following list of benefits that, together, provide a statement of value for
the national freight data architecture:
· Better understanding of the different business processes that affect freight transportation
at different levels of coverage and resolution;
· Better understanding of the supply chain, which should help transportation planners to
identify strategies for improving freight transportation infrastructure;
· Better understanding of the role that different public-sector and private-sector stakeholders
play on freight transportation;
· Better understanding of the need for standards to assist in data exchange;
· Systematic, coordinated development of reference datasets (e.g., comprehensive commod-
ity code crosswalk tables);
· Systematic inventory of freight transportation data sources;
· Systematic inventory of user and data needs that are prerequisites for the development of
freight data management systems;
· Use as a reference for the identification of locations where there may be freight data redun-
dancy and inefficiencies;
· Use as a reference for requesting funding allocations in the public and private sectors; and
· Use as a reference for the development of outreach, professional development, and training
materials.
In practice, the value of the national freight data architecture is also a function of the costs
associated with its implementation. Quantifiable data about expected benefits and costs are
currently not available (benefit-cost analyses need to occur both at the beginning and at dif-
ferent phases of implementation of the national freight data architecture). However, it is clear
from the documentation and information gathered during the research that the "do-nothing"
alternative (i.e., not implementing the national freight data architecture) is costly, ineffective,
and unsustainable. Therefore, the research team's recommendation is to pursue the national
freight data architecture following a scalable implementation path in which the national freight
data architecture starts with one application at one or two levels of decisionmaking and then
adds applications and levels of decisionmaking as needed or according to a predetermined
implementation plan until, eventually, reaching the maximum net value.
National Freight Data Architecture Components
The research team identified the following categories of components to include in the
national freight data architecture:
· Physical transportation components,
· Cargo or freight,
· Freight functions or roles,
· Business processes,
· Data sources,
OCR for page 5
5
· Freight-related data,
· Freight data models,
· Freight data standards, and
· User interface and supporting documentation.
Figure 1 shows a high-level modular conceptualization and lists different categories of com-
ponents. The diagram recognizes the scalable nature of the national freight data architecture and
enables the production of various diagram versions (as well as tabular representations) depend-
ing on what implementation level to pursue. For example, for a single-application data architec-
ture that only focuses on commodity flows at the national level, it may not be necessary to
depict (at least not in detail) other freight functions and business processes. Similarly, not all data
standards would need to be considered, and the requirements for user interfaces to support that
data architecture would be relatively minor. The diagram in Figure 1 is only one example of
potentially many different types of diagrams that can be used to depict interactions among freight
transportation components.
National Freight Data Architecture Recommendations
and Specifications
In addition to the list of categories and components, the research team put together a list
of recommendations for the development and implementation of the national freight data
architecture. For convenience, the recommendations are written in the form of specifica-
tions to guide and monitor the implementation of the data architecture as follows:
· Adopt a definition for the national freight data architecture that is generic, scalable, and
understood and accepted by the freight transportation community (see proposed definition
above);
· Compare candidate data architecture concepts;
· Develop implementation plan for national freight data architecture components;
· Develop lists of components to include in the national freight data architecture;
· Develop and implement protocols for continuous stakeholder participation;
· Conduct data gap analysis;
· Conduct data disaggregation need analysis;
· Assume a distributed approach (as opposed to a centralized approach) to freight data
repository implementations;
· Use a systems engineering approach for developing the national freight data architecture;
· Use standard information technology tools and procedures;
· Develop and/or use standardized terminology and definitions for each data architecture
component developed;
· Implement strong privacy protection strategies; and
· Establish integration points with other data architectures and standards.
Readers should note that the list of specifications is preliminary and might need refine-
ment during the process of building the data architecture.
Challenges and Strategies
The research team identified relevant issues and challenges that might block the implemen-
tation of the national freight data architecture as well as candidate strategies for developing,
adopting, and maintaining the data architecture. The challenges were in the following cate-
gories: technical, policy, economic/financial, and stakeholder buy-in and consensus.
OCR for page 6
Physical Transportation
Components
Cargo or Freight
User Interface and Supporting
Documentation
-based information clearinghouse
Freight-Related Data
Freight Data Standards
Commodity and product Freight Functions or Roles
classification standards
CPC
HMIS Fixed infrastructure manager
HS or operator
NAPCS
NMFC Policymaker
NST 2007
PLU Regulator
SCTG Researcher
STCC Shipper or receiver
Industrial classification standards Freight-Related Data Third-party logistics or broker
ISIC
NAICS Business Processes and condition
SIC
Congestion management
Data Model Tuesday, June 02, 2009
SITC
ata exchange standards
ANSI ASC X12 standards , speed, and delay data
UN/EDIFACT standards Page 1
OASIS UBL standards Freight Data Models
FIPS PUB 161-2 routing data
National ITS standards
FGDC-sponsored standards
(including metadata) Data Sources
Other standards Administrative records International trade
ITDS SDS Logistics management
METS
Vehicle classification standards On-board security monitoring
by laws and regulations Planning and forecasting
Notes: -sector data Roadside safety inspection
1) Categories and components are provided for illustration -sector data Routing and dispatching
purposes, are not exhaustive, and may be subject to change. Safety analysis
2) Not all categories and components apply to all freight-related Transportation infrastructure
business processes. analysis, design, and construction
Transportation operations
Workforce development and training
Figure 1. National freight data architecture framework and components.
OCR for page 7
7
· Technical challenges. Technical challenges refer to issues (e.g., technological limita-
tions, hardware and software incompatibilities, and standards incompatibilities) that
might impede the successful implementation of the data architecture. Examples include
the following:
Feasibility of different implementation approaches;
Data storage requirements;
Feasibility of updated data entry protocols to eliminate data redundancies and support
standardized data entry procedures;
Conversion of commodity code classifications;
Data life cycle and usefulness to support the decisionmaking process by public and pri-
vate stakeholders;
Variability in data quality control practices, which affects data accuracy, completeness,
and timeliness;
Differences in terminology, data item definitions, and data implementations among
freight data stakeholders;
Prioritization of data architecture components;
Integration between shipper and carrier data to characterize commodity movements
properly; and
Data confidentiality and security concerns.
· Policy challenges. The national freight data architecture might fail if required policies,
both in the public and private sectors, fail or are not feasible. Examples of policy chal-
lenges include the following:
Homeland security concerns, which might limit the dissemination of certain freight-
related data;
Impact on current private-sector data collection initiatives; and
Competitive and proprietary (privacy) concerns with the concept of public-sector
agencies having access to private-sector data.
· Economic and financial challenges. The national freight data architecture might fail if
the perceived costs associated with its implementation exceed the benefits that stakehold-
ers would receive. Examples of economic and financial challenges include the following:
Cost of data collection, storage, and quality assurance;
Benefits and costs related to data disaggregation requirements for different business
processes;
Data life cycle and usefulness to support the decisionmaking process by public and pri-
vate stakeholders;
Cost to acquire private-sector data; and
Cost to implement robust data confidentiality and data security measures.
· Stakeholder buy-in and consensus. The national freight data architecture might fail if
there is no stakeholder buy-in or consensus about the potential benefits that could
result from implementing the data architecture. Examples of related issues include the
following:
Reluctance of stakeholders to participate if there is no clarity regarding justification and
anticipated benefits;
Confidentiality clauses in supply chain contracts, which might impede data sharing for
transportation planning purposes;
Perception that data collected as part of a national freight data collection program could
validate projects of national significance at the expense of small or rural communities;
Ability of carriers to provide data about loads they move;
Risk of low stakeholder participation, which could decrease data reliability; and
Adequacy of data standards.
OCR for page 8
8
Strategies to ensure a successful implementation of the national freight data architecture
include the following:
· Implementation levels
Develop and compare candidate data architecture concepts,
Identify business process and implementation level priorities,
Develop high-quality data architecture concepts and applications that address the
needs of the highest priority items first, and
Identify data needs at the finest disaggregation level and implement data collection and
data storage plans at that level.
· Relationships with leaders, champions, and stakeholders
Identify data architecture leaders and champions;
Engage the national freight data architecture champions early;
Maintain good communication channels with the various stakeholders during all
phases of the development and implementation of the national freight transportation
data architecture;
Identify funding mechanisms for the implementation of the data architecture;
Develop brochures, presentations, and other materials that explain the national freight
data architecture, its scope, high-level components, and what it expects to accomplish;
Deliver effective messages on how the national freight data architecture will assist stake-
holders in the identification of strategies to address a variety of freight-related issues
ranging from data collection to analysis and reporting;
Deliver messages that provide clear, concise answers to the various challenges high-
lighted in the previous section;
Articulate benefits of participation by the private sector; and
Identify opportunities for partnerships with the private sector (e.g., through public-
private partnerships) to make data accessible for transportation planning purposes in
a cost-effective manner.
· Performance measures and effectiveness
Develop criteria for measuring effectiveness in the implementation of the national
freight data architecture,
Identify major progress milestones,
Tie the implementation of the national freight data architecture to the development of
metrics or performance measures that could benefit the entire freight transportation
community, and
Accelerate the implementation of programs such as EFM and the freight performance
measurement program.
· Lessons learned from the implementation and maintenance of existing freight-related
systems and architectures
Develop systems that are relevant to stakeholders, include adequate stakeholder par-
ticipation, and provide incentives to encourage participation--particularly in the case
of state and local entities;
Clearly define expected outcomes and development and coordination plan;
Articulate programs well; provide clear, uniform guidance; and provide good docu-
mentation;
Develop applications that rely on widely used data standards;
Develop and compare candidate architecture concepts;
Consider federal legislation to support and develop the program;
Develop tools to measure benefits and costs early;
OCR for page 9
9
Integrate archived data needs into frameworks and architectures early and develop data
programs that use industry standards;
Implement interagency data exchange programs with centralized data coordination;
Use available data sources and develop long-term plans while keeping systems flexible
to respond to changes and new data sources;
Schedule major and regular revisions effectively while avoiding scope creep;
Develop systems that are consistent with input data limitations;
Develop applications with backward compatibility;
Evaluate data disaggregation level requirements to ensure statistical significance;
Provide adequate resources for data collection, fully understand the implications of
small sample sizes, and continue to involve the U.S. Census Bureau for the use of sur-
vey instruments;
Emphasize data access, quality, reliability, confidentiality, and integrity;
Participate in the standards development process;
Create crosswalks to ensure compatibility of survey data internally over time and exter-
nally across other datasets;
Involve stakeholders early and often through various mechanisms and technologies;
and
Develop and implement professional capacity and training programs early.
One of the strategies for implementation mentioned is to develop and compare candidate data
architecture concepts. Peer exchange participants highlighted that implementing a comprehen-
sive data architecture at once with no testing of options prior to making a decision about the cor-
rect approach would be too risky. Participants also favored the concept of developing and com-
paring several alternative approaches.
A recommendation from peer exchange participants was to use NCFRP as an avenue for
funding the development of alternative data architecture concepts. Participants indicated
that the request for proposals should outline clear objectives while leaving the definition of
approaches to the research team(s) selected. An idea discussed was to develop the data archi-
tecture around scenarios or themes, such as business areas or processes, levels of govern-
ment, or economic activity. Activities in connection with each scenario or theme would
include structuring a competition for research teams (each of which would include a uni-
versity partner, a private-sector partner, and a government-level partner) to develop and test
competing data architecture concepts, making sure to include multimodal components in
the scenarios and tests, and conduct a follow-up evaluation.
