|
Items in cart [0] |
Questions? Call 888-624-8373 |
| Copyright © 2009. National Academy of Sciences. All rights reserved. Terms of Use and Privacy Statement |
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 64
4
Ensuring Relevance
A statistical agency must not only document, evaluate, and improve the qual-
ity of the data within its subject area, but it must also ensure that there are relevant
data on topics of importance to policy makers, planners, and researchers in the
field. The previous chapter addressed the immediate necessity for the Bureau of
Transportation Statistics (BTS) to focus on data quality and to build a strong
statistical staff to carry out its responsibilities for quality improvement. This
chapter addresses areas that BTS should undertake to improve the relevance of
transportation data to meet important user needs.
The 1991 Intermodal Surface Transportation Efficiency Act (ISTEA) assigns
several functions to BTS that have the goal of ensuring the relevance of transpor-
tation data for policy making and other purposes. They include: developing
appropriate indicators for the transportation system; coordinating the collection
of transportation data by USDOT with other federal agencies and collecting data
to fill gaps; and identifying unmet information needs and ways to meet those
needs. These functions are commonly undertaken by statistical agencies, but to
date BTS has done relatively little on them. Work needs to begin.
Central to a statistical agency' s ability to improve data relevance, and more
generally to determine priorities for its work, is that it have a broad vision of a
comprehensive data system that can serve the information needs of users over the
medium and long term. In this chapter, after first defining what we mean by
"relevance," we discuss the development by BTS of a vision of a comprehensive
transportation data system and how that vision and other considerations should
factor into its development of a long-range plan for implementing all aspects of
its mandate.
We then discuss priority areas for BTS to undertake to improve the relevance
64
OCR for page 65
ENSURING RELEVANCE
65
of transportation data for policy making and other uses. They include: the devel-
opment of key national transportation indicators; an increased role in coordinat-
ing the collection of transportation data, in particular, the compilation of an
annual statistical budget as a data coordination mechanism for USDOT; the es-
tablishment of regular mechanisms for identifying user information needs, in par-
ticular, effective two-way communication channels with states and metropolitan
planning organizations, building on the work that BTS has under way in this area;
and the assessment and further development of BTS's analysis programs and
publications. Primary recommendations appear at the end of the chapter.
DIMENSIONS OF RELEVANCE
"Relevance" concerns substantive aspects of data systems that affect their
usefulness. Dimensions of relevance include the following:
· The appropriateness of concepts, which means that the concepts a data
system is intended to measure are those that can help policy makers and analysts
understand trends and behaviors of concern to them and the implications of pro-
gram and policy changes. Conceptual appropriateness must be reviewed in light
of changing conditions. For example, with concern about the effects of economic
growth on the environment and nonrenewable resources, there is growing interest
in concepts of national income and gross national product that account for natural
resource depletion, pollution, and other environmental costs. More narrowly,
beginning in December 1991, the Bureau of Economic Analysis has featured
gross domestic product (GDP), and not gross national product (GNP), as the more
appropriate concept by which to measure U.S. output for comparative analysis
with other countries in today's global economy.)
· The match between concepts, operational definitions, and measurements,
which implies that the theoretical concepts of importance to data users are
operationalized by appropriately defined empirical variables that, in turn, are ac-
curately and reliably measured (see Bonnen, 1977:395-396~. Many concepts are
difficult to operationalize. For example, the economic cost of transportation fa-
talities and injuries is an important concept for which to have data, but it may be
difficult to operationalize such a concept with an appropriate proxy variable or
combination of variables that can be measured empirically. For example, should
the costs include the immediate costs of medical treatment, emergency system
use, vehicle repair, etc.? The long-term costs over the lifetime of accident vic-
tims of health care, public assistance, lost productivity, foregone tax revenues,
etc.? Estimates of the value of lost quality of life? The choice of operational
definition affects the relevance of the concept for policy and research use and also
iGDP includes the output produced by labor and property located in the United States, including the
output of U.S.-located establishments of foreign-owned enterprises; GNP includes the output attribut-
able to labor and property supplied by U.S. residents (see Bureau of the Census, 1996b:439-440).
OCR for page 66
66 BUREAU OF TRANSPORTATION STATISTICS: PRIORITIES FOR THE FUTURE
the ability to obtain accurate measurements. In the example, the broader the
operationalization of the concept, the more difficult the measurement process.
· More generally, the appropriate level of detail in a data system in terms
of the specificity and range of subject matter and geographic detail that it pro-
vides to inform current and emerging policy and research interests. Appropriate-
ness of detail must be reviewed continually in light of changing conditions and
policy concerns. As examples, more data are needed for the fast-growing ser-
vices sector of the economy to support public- and private-sector policy making
and planning than was true in the past, and more data are needed on intermodal
transport to address increasingly important transportation policy concerns.
· Timeliness of statistical information, which denotes the length of time
between the occurrence of some event or the act of measuring some attribute of
interest and the availability of statistics to the user. For example, the Bureau of
Labor Statistics provides a monthly update of the labor market status of the
economy. Wise decisions on the periodicity of data collection are a function of
the rate of change and causes of change in estimates as well as the nature of
decisions taken on the basis of the estimates.
The implications of a statistical agency's focusing on data relevance in terms
of appropriateness of concepts and their measurement, level of subject matter and
geographic detail, and timeliness are that it must identify needs for data among
current users, gaps in available data systems, and possibilities for the agency to
inform the policy debates of the future. This process, combined with a compre-
hensive understanding of the field, permits the agency to define sets of indicators
that offer great relevance to current and future users and to provide other kinds of
useful data.
In its efforts to ensure relevance, it is important that a statistical agency seek
ways to contain the costs and burden of data collection, processing, and analysis
by keeping abreast of new methods and technologies that have the potential for
cost savings and by looking for ways to cut back on less important data (e.g.
through reductions in sample size or frequency). There will always be more data
demands than can be satisfied, particularly in an era of increasingly constrained
budgets, and it is critical for a statistical agency to evaluate data needs to deter-
mine priority areas for new and improved data and also areas for which reduc-
tions are possible.
A VISION OF A COMPREHENSIVE
TRANSPORTATION DATA SYSTEM
Key to BTS's ability to ensure the relevance of transportation data and to
make wise choices among competing activities to improve both data relevance
and quality is that it have a broad vision of transportation data (see recommenda-
tion 5 at the end of the chapter). The vision should encompass the information
OCR for page 67
ENSURING RELEVANCE
67
needs of transportation policy makers, planners, and researchers in the medium
and long term and the characteristics of a comprehensive data system that could
best serve those needs. BTS will not necessarily or even likely itself develop all
of the data that are required to implement the vision. Much of the specific work
to be undertaken will be done outside BTS: by other USDOT modal administra-
tions, by other parts of the federal statistical system, and by states, localities, and
private organizations. However, if BTS is to fill the leadership role set out for it
by the 1991 ISTEA, then it must have an overarching vision of the data require-
ments in key constituencies in the transportation field.
How can BTS, as a priority effort, go about constructing such a vision and
refreshing it periodically? One way is to ask relevant constituencies such ques-
tions as the following:
What are seen as important national policy concerns in transportation, how
are they changing, and what are the implications for data? The 1991 ISTEA
called for a reorientation of transportation planning to address intermodal and
multimodal issues and concerns. The reauthorization of ISTEA is likely to con-
tinue a cross-modal planning focus and may single out other important policy
issues for the transportation community as well. The continuing public concern
with such issues as the safety of the transportation system, the quality of the
environment, and the costs and availability of energy sources will also have im-
portant implications for transportation planning and investment. In developing a
vision of information needs and a data system to address them, BTS must assess
the data requirements of continuing and emerging national transportation policy
concerns as seen by the Congress, the administration, and others, including states,
metropolitan planning organizations, industry, and the general public.
What changes are occurring in the economy and society that suggest the
needfor new data or the reassignment of priorities among areas? The nature and
pace of future social and economic change are hard to foresee with any great
precision; however, broad trends are identifiable that are likely to have implica-
tions for a comprehensive transportation data system that can serve user needs.
Such trends include the aging of the population; the continued suburbanization of
people and industry; growing pressures on the environment; the computerization
of homes, schools, and businesses; and the globalization of the economy and
continued growth in international trade. BTS could elaborate scenarios in these
and other areas and consider the possible data implications for transportation.
While not making too much of the results of such scenario-building, BTS could
identify areas in which modest additional data collection, or somewhat different
data collection, could help the transportation community anticipate and respond
to important societal trends.
As an example, rapid growth over the next few decades in the proportion of
employed people who telecommute to work via computer, telephone, and fax at
home can be expected to change the kinds of infrastructure investments that are
OCR for page 68
68 BUREAU OF TRANSPORTATION STATISTICS: PRIORITIES FOR THE FUTURE
required for an efficient transportation system, compared with a continuation of
current work practices. If the projected differences in the kinds of needed invest-
ment are significant, then it could be important to the transportation community
to have data with which to assess more accurately the likely rates of change in
home-based employment linked with data on residence patterns. Such data in-
puts could include not only survey questions about current workplaces, but also
survey questions about the likelihood that respondents will work part or all of
their hours at home in the next year, or next 5 years, and what factors might cause
them to make such a change.
What topics and information needs are still relevantfrom the past? In addi-
tion to new and modified data to respond to emerging concerns, there is always a
need for continuing time series to support trend analysis and provide benchmarks
against which to measure change. The question is which series are critically
important to continue and which could be reduced in scope, frequency, or sample
size or redesigned in other ways (e.g., by converting an administrative records
system to sample-based reporting or using new collection technology) in order to
free up resources for other areas or to reduce the overall costs and burden of
transportation data collection.
In the transportation area, data on safety are clearly an important continuing
need, particularly for USDOT, given the extensive involvement of the federal
government in safety issues and safety regulation in all modes of transportation.
There are likely to be other such areas as well. However, there are also likely to
be areas in which the data that are currently collected are of less value to continue
in the future (e.g., because of lesser policy concern) or that could be collected just
as effectively by other organizations or other means. For example, detailed infor-
mation on financial and operating characteristics of some kinds of common carri-
ers might be one such area. Decisions to reduce or eliminate long-established
data series are always difficult to make. However, a statistical agency that is
striving to improve data relevance must have a vision of a comprehensive data
system that is dynamic and allows for the retirement of obsolescent data series
along with the emergence of new and modified series.
A BTS IMPLEMENTATION PLAN
A critically important task for BTS to undertake in the near future is the
development of a long-term strategy for implementing its mission to improve
both the relevance and quality of transportation data. BTS's mandate encom-
passes a large, almost daunting, array of functions and responsibilities. A struc-
tured implementation plan that specifies short-term, intermediate, and long-term
goals in each of BTS's main programmatic areas is a necessity in order for BTS
to work toward its vision of a comprehensive transportation data system and
evolve as a statistical agency for USDOT. Without such a plan, BTS's energies
are likely to be dissipated in striving to do more than it reasonably can. Also,
OCR for page 69
ENSURING RELEVANCE
69
without such a plan, the inevitable pressures from continuing areas of responsi-
bility (e.g., for data dissemination) may result in continued deferment of needed
initiatives in other areas (e.g., development of quality standards). BTS must have
a roadmap with a well-blocked-out route to guide its activities, help it develop the
necessary staff capabilities, and build a reputation as an effective agency that,
over time, is fulfilling its mandate from the ISTEA.
The implementation plan should identify overall priorities among BTS's
major functions for the short and longer term and, within each functional area,
identify specific activities, goals, and timetables.2 BTS's vision of a comprehen-
sive transportation data system should provide the context for the development of
the implementation plan. For example, such a vision should help determine a
priority sequence for the development of national transportation indicators. An-
other source of input to the plan is this report, which identifies broad areas of high
priority, including work to develop department-wide quality standards, increased
emphasis on documenting and evaluating data quality, and work to develop na-
tional indicators, at the same time recommending decreased emphasis on the
quantity of data disseminated. Still other sources of input, for both general and
specific priorities and goals, include the constituencies or customers for transpor-
tation data national policy makers, state and local agencies, private-sector orga-
nizations, and academic researchers.
The development of a vision of a comprehensive transportation data system
and the development of a long-term strategy for implementing BTS's mandate
are difficult, time-consuming tasks that represent added responsibilities for BTS
staff. As would be true for any attempt at a serious long-range planning process,
it is likely that BTS's initial efforts will produce areas for which it is not clear
how to proceed or for which there is less complete articulation of ideas and goals
than for other areas. Also, there must be flexibility to revise and further develop
the vision and plan as circumstances change and new knowledge and experience
are gained. Nonetheless, it is critical that BTS make its best attempt to envision
the future requirements for transportation data and to plan its own future so that,
for the long term, it has an overall sense of direction and, for the short and me-
dium term, it has a set of goals that are feasible, contribute to the long-term
agenda, and make it possible for the agency to demonstrate a solid record of
accomplishment over time.
ENSURING RELEVANCE: TRANSPORTATION INDICATORS
The 1991 ISTEA mandates BTS to establish and implement, in cooperation
with the modal administrations, the states, and other federal officials, a compre
2BTS recently outlined its goals in specific areas for fiscal 1997 and 1998 (provided in a back-
ground document for the Advisory Council on Transportation Statistics). These goals should be
reexamined as part of a longer-range planning process.
OCR for page 70
70 BUREAU OF TRANSPORTATION STATISTICS: PRIORITIES FOR THE FUTURE
hensive, long-term program for collection and analysis of data relating to the
performance of the national transportation system. The Transportation Research
Board report, Datafor Decisions (National Research Council, 1992a), urged, as a
high priority, that a new transportation data center, which is now BTS, develop a
national transportation performance monitoring system. It developed a list of
important attributes of the transportation system, for which it suggested one or
more types of indicators (this list is reproduced in Table 4-1~.
We agree that a high priority for BTS is to develop a consistent, easily under-
stood, and useful set of indicators of key aspects of the transportation system (see
recommendation 6 at the end of the chapter). Most statistical agencies produce
indicators (usually regular time series) in their areas: examples are monthly and
quarterly gross domestic product (GDP) estimates produced by the Bureau of
Economic Analysis; monthly Consumer Price Index (CPI) and unemployment
rate estimates produced by the Bureau of Labor Statistics; monthly retail sales,
monthly housing starts, and annual poverty statistics produced by the Census
Bureau; annual high school and college completion and dropout rates and peri-
odic assessments of levels of student achievement produced by the National Cen-
ter for Education Statistics; and annual vital statistics, which include estimates of
births and deaths and mortality rates by cause, produced by the National Center
for Health Statistics. Each of these indicator series is important in informing the
public and contributing to the policy debate in its area; some of them have signifi-
cant effects on the economy and public- and private-sector decision making. In
other words, appropriately developed indicators provide highly relevant data for
policy making and general public awareness.
It is a heavy responsibility to produce such important statistics. There are
often difficult conceptual, definitional, and measurement issues involved in de-
veloping a single reliable and credible indicator to represent a complicated socio-
economic phenomenon or construct (e.g., GDP, unemployment, poverty); even
two or three indicators may not be adequate. Moreover, the policy use of key
indicators can bring unwelcome publicity to a statistical agency, which may be
hard pressed to explain the proper interpretation of its statistics and to defend the
concepts and methods against politically motivated criticism and misuse.
Furthermore, the development of indicators should not be the only focus of a
statistical agency in terms of providing relevant data. Thus, indicators cannot
serve such important needs as that of researchers for rich, multivariate data sets
(e.g., longitudinal surveys) with which to analyze complex trends and behaviors.
Nonetheless, key statistical indicators are important for both the public and policy
makers, and statistical agencies can gain stature and support from the responsibil-
ity to produce them. Also, such responsibility can help an agency set priorities
for improvement of key concepts, definitions, and data sources that are needed to
support the development of indicators and to support more in-depth analysis as
well.
OCR for page 71
ENSURING RELEVANCE
7
Indicators, Not Performance Measures
In discussing BTS's role in developing transportation indicators, we explic-
itly use the term "statistical indicator" instead of "performance measure" or a
similar term. The latter has a judgmental or regulatory connotation that is inap-
propriate for a statistical agency.
Indeed, there has been considerable concern among the states about the re-
cent interest on the part of the federal government in assessing the performance of
the transportation system. The states are wary of jurisdiction-specific perfor-
mance measures that might be used for such purposes as allocating federal trans-
portation funds, particularly in light of the difficulties of developing valid mea-
sures that appropriately take account of measurement problems and varying
conditions at state and local levels. (As examples, measures of road conditions
should adjust for such factors as types and extent of usage, and measures of high-
way traffic congestion should adjust for such factors as population density and
the availability of public transit.)
Such concerns are not limited to transportation. For many years, the states
largely opposed the development of comparable crossjurisdiction indicators of
children's educational progress. A major survey, the National Assessment of
Educational Progress (NAEP), was originally designed so that only national-level
and not also state-level estimates could be produced from the data. However,
increasing public concern about educational issues has led to a willingness on the
part of the states to compare their performance, and the NAEP was recently rede-
signed to provide state-level estimates.
What all this means is that a statistical agency must approach the develop-
ment of indicators with care. To the extent that meaningful national-level indica-
tors can be developed, they should be an important focus of the agency.
BTS has already established as a priority goal for fiscal 1998 to begin work
on transportation indicators. BTS proposes a cooperative activity with the other
USDOT modal administrations, such as the Federal Aviation Administration and
the Federal Highway Administration, in which they would first identify appropri-
ate topics and concepts for indicators and BTS would then provide technical ad-
vice on implementation. A motivation for the development of indicators is the
1993 Government Performance and Results Act, which requires federal agencies
to establish performance measures of their output.
We urge BTS to move forward with its plans to help the other modal admin-
istrations identify performance indicators for their own programs. In general, as
BTS builds its statistical staff and capabilities, it should be able increasingly to be
helpful to the other modal administrations not only in the development of indica-
tors and other kinds of statistical data, but also in advising on ways to improve the
cost-effectiveness and usefulness of the large amounts of data that many of them
collect for purposes of program administration and regulation.
At the same time, we urge BTS to develop a small set of key national statis
OCR for page 72
72
.o
.tq
Cq
a'
be
VO
Cq
so
o
C)
a'
Cq
VO
.~
En
¢
En
ca
o
· ~
o
.n
ca
Cat
· ~
o
ca
~o
o ~s.
.= ~^ ~o
Y ° o E R R
ER 9 as a <, A
t>.4 ~ R ~ ~ ~ R
.~ ~ ~ ~ .
y ~' ~ ~ ~ ' ; ~'
=4 4- ~ ;^ ~ ~ ~ ·
O ~ I ~ ~ ~ O
~u ~ ~ i ~ ~ 3 ~ s ~ ~
~= ~¢~m am zmE-o ~ E-Z~
ca ca
C) C)
. ~ ·=
R ~ ~ 9 ~ ~ · > _ j R
~ ~R _
V, ~
OCR for page 73
73
ca
~i ~
R At, ~ R ~
r ~ ~ ~ a E ~ ~ ~ ~ ~ , a E E ' ~ 'a
& ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ .
~ ~ t, ~0 ~OR o ~ R = . ~ ~ ~ o ~ o ~ ~ R 5 ~ ~ ~ R ~ ~ ~ R ~ 3 o ~
To ~ ~ ~ ~ ~ ~ e~ ~ ~ o ~ ~ ~ ° ~ ~ °
O ~En Z ~ Z v, ~ ~ Z ~ ~ ~ ~ V ¢ ~ ~ V ~ ~ ~ ~ ¢ Z
;^ ~
.n ~
C) ~
~;^
ca 4=
R O ~
D C ° E a ~ a a ~@
En
. .
=b
ca
o
.~
7
..
V
O
V,
OCR for page 74
74 BUREAU OF TRANSPORTATION STATISTICS: PRIORITIES FOR THE FUTURE
tical indicators of the transportation system that are relevant to policy and public
concerns that it publishes on its own behalf as a statistical agency. (Some indica-
tors e.g., trends in airline safety may serve the purposes of both BTS and
another modal administration and could be developed and published jointly with
the appropriate administration.) BTS's vision of a comprehensive transportation
data system should inform its choice of priority areas for indicators, along with
input from the other agencies in USDOT and transportation constituencies out-
side USDOT. Because of BTS's responsibility to improve transportation data for
cross-modal, system-wide analyses, the statistical indicators it decides to develop
and publish should feature cross-modal concepts and concerns.3
What Indicators and in What Form?
The challenge for BTS is to identify important aspects of the national trans-
portation system for which it is possible to develop meaningful and reliable indi-
cators. Transportation is largely local, yet it has national effects. For example,
the functioning of the highway, rail, and air systems at a major transportation hub
like Chicago affects not only the local economy and well-being, but also the
national economy and international trade. The difficulty is to develop indicators
that have national meaning when appropriate data may be hard to obtain and to
interpret.
The Transportation Research Board report, Data for Decisions (National
Research Council, 1992a), identified several key areas for which it would be
useful to have national indicators but for which data are currently difficult to
compare across transportation modes: safety; access to services by such groups
as elderly, disabled, low-income, and rural populations; and the efficiency and
quality of service provided by the transportation system. BTS has covered some
of these topics in its Transportation Statistics Annual Reports (TSARs), as well as
other topics. The 1996 TSAR includes chapters on passenger travel and the move-
ment of freight, with tables on the physical condition of highways, runways and
aircraft, and other transportation facilities; the role of transportation in the
economy; safety; energy use; and transportation and the environment. Although
there is much material in these analyses that could support the development of
key indicators, there are also many hurdles to overcome.
In the important area of safety, the 1996 TSAR notes some of the conceptual
and measurement problems for developing meaningful trend indicators. One con
3In this regard, Data for Decisions (National Research Council, 1992a:32-37) recommends that
indicators be developed for types of markets rather than for transportation modes. As an example,
cross-modal indicators of travel delays might be developed for intercity markets, such as an indicator
that looks specifically at weak links between modes (e.g., highway or rail connections to airports).
Such an indicator could help policy makers identify a fuller range of options for improvement of
transportation infrastructure than is likely to emerge from analysis of indicators that pertain to particu-
lar transportation modes.
OCR for page 76
76 BUREAU OF TRANSPORTATION STATISTICS: PRIORITIES FOR THE FUTURE
BTS might also consider using data produced by the Bureau of Labor Statistics
on employment and combining them with data from the American Travel Survey
and the Nationwide Personal Transportation Survey on travel from place of resi-
dence to place of work, including the modes used by workers, their costs, and
availability. Another set of indicators, as noted previously, might deal with
safety in the air, on the highways, and other modes. Another approach might be
to select one of the themes used in the TSARs and develop one or more indicators
to inform the public about progress for example, changes in transportation pro-
ductivity or changes in the relationship between transportation and the environ-
ment. (See the section below on "Analysis Programs" for a discussion of how a
chartbook of indicators could relate to the TSARs and how the latter could use-
fully be reconfigured.)
COORDINATION OF DATA COLLECTION
AND FILLING GAPS
It is important for a statistical agency to coordinate data collection in its area
to the extent feasible. Coordination is necessary to make the most cost-effective
use of scarce resources to provide relevant, high-quality information for such
purposes as developing appropriate statistical indicators and directly serving the
information needs of policy makers and other users. (Relatedly, a statistical
agency should establish regular sources of input from data users, producers, and
methodologists about priority information needs and methods to supply them-
see the section below on "Identifying User Needs.") Effective mechanisms for
coordination (and input) are required to identify:
.
areas of overlap in data collection for which it may be possible and desir-
able to reduce duplication and associated costs and burdens on respondents and
thereby free up resources for other needed data;
· areas for which no data system currently provides relevant measures and
for which it may be possible to fill gaps;
.
linkages among data systems that may increase their relevance and ana-
lytical power; and
· innovations in data collection and analysis methods that may improve the
quality of measures across data systems.
There are at least three domains for coordination of data collection in the
transportation field: coordination within USDOT; coordination between USDOT
and other federal statistical and program agencies; and coordination between
USDOT and such key data providers and users as states and metropolitan plan-
ning organizations. We recommend that BTS take a major step to facilitate data
coordination among the modal administrations in USDOT through a department-
wide statistical budget.
OCR for page 77
ENSURING RELEVANCE
77
A USDOT Statistical Budget
USDOT supports a large number of data collection and analysis programs,
with significant statistical activities in almost every modal administration (see
Appendix B). To the existing programs, BTS has added new data collection
systems on intermodal flows of passengers and freight. We believe it would
help USDOT evaluate and improve the relevance and cost-effectiveness of its
large array of statistical activities to have BTS prepare each year a consolidated
statistical budget for the department (see recommendation 7 at the end of the
chapter). BTS could follow the example of the Statistical Policy Division in the
U.S. Office of Management and Budget, which brought together information on
agencies' proposed fiscal 1998 statistical budgets across the entire federal gov-
ernment for purposes of program review and decision making among competing
priorities.5
For USDOT, BTS should compile budget information from all of the modal
administrations about their statistical programs, including supporting justifica-
tion. BTS could organize this material in several ways for example, by subject
area as well as by modal administration and agency. It should add commentary
as appropriate for example, noting relationships among programs in different
modal administrations or pointing out user needs that no USDOT data collection
program currently addresses.
BTS would not determine the budget allocations for any other modal admin-
istration. Rather, the consolidated statistical budget would be available for the
secretary's use in making final proposed budget allocations to transmit to OMB.
It would help clarify for the secretary what the individual modal administrations
see as priorities for data collection and analysis. At the same time, it would help
the secretary determine how well the agencies' priorities accord with department-
wide needs and whether some reallocation of resources among data programs
within a modal administration would enable the department to be more cost-
effective in providing relevant data for policy purposes and to serve other impor-
tant needs of the transportation community. To ensure that the preparation of a
USDOT statistical budget becomes institutionalized and integrated into the
department's decision making, the reauthorization of BTS should directly assign
to BTS the responsibility for compiling the statistical budget each year.
We repeat that the USDOT statistical budget would be compiled and anno-
tated by BTS but that BTS would not make budget decisions for any other modal
administration. Also, the statistical budget would not include all USDOT data
programs. Many data collection systems in USDOT provide modal administra
5The Statistical Policy Division some years ago regularly produced cross-cutting statistical budgets
as part of the preparation of the president's budget submission to Congress. The practice then lapsed
and was just resumed this year. (Routinely, the division produces a cross-cutting description of
federal statistical activities after the budget preparation is completed i.e., to document rather than to
inform decision making see, e.g., Executive Office of the President, 1997b.)
OCR for page 78
78 BUREAU OF TRANSPORTATION STATISTICS: PRIORITIES FOR THE FUTURE
lions with information for program management and regulation and have few
statistical uses. Examples are the large number of operational databases of the
Federal Aviation Administration (e.g., the Aircraft Registration System and
Manufacturing Inspection Management Information System see National Re-
search Council, 1992a:111-114~. The budgets for operating such data systems
would not be included in the USDOT statistical budget, except for that portion
that may be devoted to statistical analysis of the data for public use.6
Practically speaking, the USDOT statistical budget would include the bud-
gets of the major statistical units in the modal administrations (e.g., the Safety
Data Services Division in the Federal Aviation Administration see Appendix
B), plus other programs that are not lodged within a separate statistical unit but
that the modal administration identifies as having an important statistical compo-
nent. Indeed, the preparation of the USDOT statistical budget may identify areas
in which it would be helpful to a modal administration and for transportation
policy analysis, planning, and research more generally to develop the statistical
applications of an operational database. As BTS enhances its statistical capabili-
ties and achieves excellence in its own operations, it should be increasingly able
to offer technical assistance to the other modal administrations in this regard.
Other Coordination Activities
There are other coordination activities that BTS should consider working
into its implementation plan, as available resources and the demands of other
priorities permit. For example, BTS could undertake periodic reviews of existing
transportation data systems to determine how well they meet the requirements for
development of indicators on specific topics and, more generally, how well they
contribute to BTS's vision of a comprehensive transportation data system. Such
reviews may identify data gaps that are important to fill. They may also identify
opportunities for linking or integrating data systems to achieve such goals as
making the combined data relevant for a broader range of analyses, improving
data quality by such means as standardizing definitions for key variables, and
reducing costs.
An obvious first priority for a cross-system data review would be for BTS to
look at sources of data on intermodal transportation, including its two flagship
intermodal surveys the Commodity Flow Survey and the American Travel Sur
6The Statistical Policy Division in the U.S. Office of Management and Budget faces a similar issue
of defining which data programs to include in the cross-cutting federal statistical budget: the criterion
used is that statistical programs of $500,000 or more in annual expenditures are to be included. As an
example, the budget includes the Statistics of Income program in the Internal Revenue Service, which
produces statistical publications and data files from tax return data, suitably processed to protect the
confidentiality of the information for tax filing units, but the budget does not include the vastly larger
costs of the Internal Revenue Service to enter the data from tax returns, calculate taxes owed and
refunds due, and monitor compliance.
OCR for page 79
ENSURING RELEVANCE
79
vey and other possibly relevant data sources. In Appendix F. we briefly review
sources of data on household travel and develop some ideas about data linkage
opportunities and remaining data gaps (see also Bureau of Transportation Statis-
tics, 1993c, 1995:96-102~.
Conducting cross-system data reviews, as well as carrying out other kinds of
coordination activities, will require that BTS involve appropriate agencies through
working groups, interagency committees, and the like. In some instances, it will
be necessary to involve not only one or more agencies in USDOT, but also out-
side agencies for example, other federal statistical and program agencies.
Experience has demonstrated the difficulty of achieving effective interagency
collaboration, particularly when the agencies involved are from different depart-
ments or levels of government. The history of a short-lived federal interagency
transportation statistics coordinating committee that was in existence in the early
1990s illustrates the problem. The committee initially attracted a large atten-
dance to exchange information; however, no action agenda was developed, atten-
dance fell off, and the committee became moribund. This is a common pattern
with interagency groups, as participants are pulled back to the agendas of their
own agencies and the activities of the interagency group become largely ones of
show and tell.
Generally, an effective interagency group requires that agencies be involved
because they want to be, believe they can accomplish more on the topic together
than apart, have an action agenda, contribute people or funding to the extent
possible, and have the support of their agency heads. These characteristics sug-
gest that it generally makes more sense to establish interagency groups on an as-
needed basis with a specific set of issues and agenda in mind than to set up an
umbrella committee. An example of such a special-purpose committee in which
BTS plays an active role is the Federal Geographic Data Committee, which is
working to standardize geographic information system (GIS) capabilities for the
federal government as a whole. It may be that other special-purpose interagency
committees will be useful to establish in the future (e.g., an interagency commit-
tee on the development and appropriate application of data for monitoring air
quality and other environmental effects of the transportation system).
IDENTIFYING USER NEEDS
It is important for a statistical agency to obtain regular input not only on the
usefulness of its current products and services (e.g., through customer surveys),
but also on unmet data needs and priorities for data, indicators, analyses, and
improved concepts and measures that are relevant to users' concerns. The agency
must assess and interpret the input it receives users are not always the best
judges of appropriate or feasible data constructs or measures; also, they will gen-
erally want more than it is possible to provide within budget constraints. None-
theless, user input is clearly central to the development by a statistical agency of
OCR for page 80
80 BUREAU OF TRANSPORTATION STATISTICS: PRIORITIES FOR THE FUTURE
its vision of the important information needs in its area and the characteristics of
a comprehensive data system to serve those needs.
There are many constituencies for transportation data, including federal sta-
tistical and program agencies inside and outside USDOT, congressional agen-
cies, state and local agencies, private-sector organizations, academic researchers,
and the public. Input from federal agencies comes (or will come) from such
activities as developing quality standards for transportation data, constructing
transportation indicators, and reviewing data systems. In addition, BTS obtains
input from the Advisory Council on Transportation Statistics (ACTS) (mandated
in the 1991 ISTEA), which meets twice a year to consider priorities for BTS's
growth and development. Although the ACTS provides a range of public- and
private-sector user perspectives, its membership is small (6 people). BTS also
sponsors six standing committees of the Transportation Research Board, which
bring together researchers and other users to exchange information about data
needs and applications in several areas (see Chapter 2~. However, the transporta-
tion data community is so large and diverse that regular communication with
many more users in state and local organizations, the private sector, and academia
will be needed for BTS to develop and refine its vision and implementation plan
for improving transportation data.
States and metropolitan planning organizations (MPOs) are particularly im-
portant constituencies for BTS to work with because of the federally based struc-
ture of planning, investment, and associated data collection and analysis for the
U.S. transportation system. States and MPOs play a vital role in developing and
implementing transportation policy and in making decisions about investments in
transportation infrastructure that have important consequences for the cost-effec-
tiveness of the transportation system as a whole. They also provide many key
transportation data sets and, in turn, use transportation data for a wide range of
purposes. A decision by Congress to devolve yet more responsibilities for trans-
portation policy planning and implementation could further strengthen the role of
states and MPOs.
BTS does not at present operate data collection systems that require working
directly with states or MPOs to obtain data; such systems (e.g., the Highway
Performance Monitoring System) are lodged with other USDOT modal adminis-
trations.7 However, as the lead statistical agency for the department, BTS should
develop regular channels of communication with these two important constituen-
cies and in the past year it has begun to do so. We recommend as a priority
effort that BTS continue with its plans for obtaining regular input from states and
MPOs and, relatedly, its plans for technical assistance to help states and MPOs
make more effective use of transportation data (see recommendation 8 at the end
of the chapter).
7See Ruddick (1996) for a comparative descriptive analysis of nine federal-state data collection
systems, including the Highway Performance Monitoring System and the General Highway Statistics
Program in USDOT.
OCR for page 81
ENSURING RELEVANCE
8
Outreach to States and MPOs
BTS began over a year ago an active outreach program of meetings with
state transportation officials, which were carried out in collaboration with the
Office of Highway Information Management in the Federal Highway Adminis-
tration and the American Association of State Highway and Transportation Offi-
cials. It then expanded these efforts to include MPOs (working through the Asso-
ciation of Metropolitan Planning Organizations). A conference held in spring
1997 brought together state and local officials with staff of BTS, the Federal
Highway Administration, and the Federal Transit Administration to discuss pri-
ority data needs, the appropriate role of each level of government in transporta-
tion data collection and dissemination, the implications of technological advances
(e.g., intelligent transportation systems) for data collection and dissemination,
and the kinds of technical assistance that could help states and MPOs make more
effective use of national transportation data sets.
We urge that the conference be followed up by considering the most effec-
tive communication channels to establish for regular, two-way interaction of BTS
and other USDOT modal administrations with states and MPOs. Such interac-
tion will be vitally important for BTS to carry out its mission in developing trans-
portation indicators and filling key data gaps in its vision of a comprehensive
transportation system that is relevant to user information needs.
Technical Assistance
As an outgrowth of its rounds of meetings with state transportation officials,
BTS has begun to conceptualize ways to provide technical assistance to states
and MPOs in obtaining, collecting, and analyzing transportation data. Technical
assistance can be a draining activity for a small statistical agency, particularly if it
involves one-on-one assistance on particular problems of individual organiza-
tions. However, it is possible to structure a technical assistance program so that
such products as user's guides and application software are developed that have
broad utility for many organizations.
In light of its mandate for intermodal data, we suggest that BTS focus its
technical assistance activities on developing tools for states and MPOs for
intermodal analysis, using data from BTS's two surveys, the Commodity Flow
Survey (CFS) and the American Travel Survey (ATS), and other relevant sources.
The 1991 ISTEA increased the planning requirements for states and MPOs, in-
cluding that they consider system-wide issues instead of focusing narrowly on
particular transportation modes. The ATS and the CFS, alone and linked with
other information, will provide rich data sets for cross-modal analyses of trans-
portation flows within and across states and metropolitan areas. The CFS also
provides data that could be useful to states in planning future economic develop
OCR for page 82
82 BUREAU OF TRANSPORTATION STATISTICS: PRIORITIES FOR THE FUTURE
ment (e.g., locating or further developing an airport that could be a hub for long-
distance shipments of specific kinds of products).
Technical assistance in using the ATS, the CFS, and related data could take
such forms as user's guides that highlight state and local applications of the data,
special analysis software, and innovative methods of data analysis. Assistance
could be offered in a variety of formats and venues, such as access via the Internet,
continuing education classes, conferences, and, occasionally, on-site work on a
particular project. Analytic tools and techniques could be developed in some
cases directly by BTS, or by working with one or a few states, or through con-
tracts with universities or other organizations.
The experience of other federal statistical agencies suggests that funding one
or a few states to develop data processing and analysis tools has the advantage
that other states may be more receptive to using a state-developed product (see
Ruddick, 1996~. If BTS uses contractors for its technical assistance activities, it
is important that some BTS analysis staff also be involved, particularly in the
development of analysis tools that exploit the information value of the ATS and
the CFS. BTS staff need hands-on experience in using the ATS and CFS data for
a variety of analysis needs in order not only to help states, MPOs, and others use
the data more effectively, but also to set priorities for improving the relevance
and quality of the data for the future.
ANALYSIS PROGRAMS
Data analysis is an important component of the work of a statistical agency-
not only analysis of quality measures and issues related to methods, but also
analysis of substantive topics. Statistical agencies should not be advocates for
particular policies, but they should engage in research that sheds light on the
effects of alternative policies and that illuminates trends and relationships in
policy-relevant areas. Careful analyses in substantive areas that explain what the
data show and qualify findings with information about the quality and appropri-
ateness of the data for particular uses are very helpful for users. Such analyses
are also critical to the statistical agency itself to help it understand the data in its
area, determine how to keep the data relevant for policy and other purposes, and
continually refine its vision of a comprehensive transportation data system to
serve user information needs. (See Bonnen, 1997, for a discussion of the analysis
roles of statistical agencies.)
Developing a substantive research program can be difficult for a statistical
agency. Such research is often a target for cutbacks when budgets are tight in
favor of preserving resources for data collection. Also, analytical researchers and
statisticians and methodologists on the staff may not always work together effec-
tively because of differing expertise and perspectives. Statistical agencies need
to address these challenges in order to have an active in-house research program
that benefits the agency and its users.
OCR for page 83
ENSURING RELEVANCE
83
Encouraging Substantive Research
At BTS, the director has emphasized the importance of substantive research
on transportation issues. The Transportation Statistics Annual Reports feature
each year a special analytical section on a particular topic of policy concern, in
addition to providing updated assessments of the state of transportation. BTS
also regularly hosts seminars and conferences on research topics, and it recently
inaugurated a new twice-yearly, peer-reviewed Journal of Transportation and
Statistics that will feature research articles.
We support BTS's research initiatives and encourage an expansion of them
as resources and the demands of other priorities permit. In particular, we encour-
age research by BTS on the substantive uses of data from the CFS and the ATS
that can help policy makers understand the problems and opportunities for cost-
effective intermodal transportation of people and goods.
At the same time, because research is labor-intensive and time-consuming,
we urge BTS to assess how it is carrying out its research activities and whether
there are more cost-effective ways to approach them. One way for BTS to aug-
ment its in-house research capabilities would be for it to announce special re-
search initiatives in a request for proposals aimed at university faculty involved
in transportation studies. Looking at the demands on its own staff, we encourage
BTS to assess the contribution of the TSARs to the agency's analysis functions.
The Role of the TSARs
BTS is mandated by the 1991 ISTEA to produce a Transportation Statistics
Annual Report, and the TSARs produced to date have contained useful data and
analyses that were not previously available to transportation planners and ana-
lysts. However, the TSAR may not be the best format with which to provide
transportation data analyses to the user community or, relatedly, to provide a set
of widely followed national transportation indicators.
BTS is still a small agency, and the preparation of each year's TSAR absorbs
substantial time and energy of BTS's in-house and contractor staff. Yet a thick
annual report of textual chapters, even with many tables and charts, does not
seem well suited to serve the information needs of policy makers and other users.
They are all too likely to lose sight of the forest for the trees and to find such a
publication too difficult to use, either for locating a key statistic or for under-
standing the relevance of trends and relationships in the data for particular policy
issues and concerns.
In contrast, such publications as the monthly Survey of Current Business,
published by the Bureau of Economic Analysis, the Monthly Labor Review, pub-
lished by the Bureau of Labor Statistics, and the quarterly Social Security Bulle-
tin regularly include standard, easily locatable tables that update key statistics. In
addition, they include articles on selected topics that amplify the material in the
OCR for page 84
84 BUREAU OF TRANSPORTATION STATISTICS: PRIORITIES FOR THE FUTURE
tables. These articles may be substantive (e.g., analyzing particular trends) or
related to methods (e.g., analyzing measurement problems for a particular vari-
able). A similar monthly publication would not be feasible for BTS at its present
stage of development and is likely not needed in any case, given that transporta-
tion indicators tend not to show pronounced movements over short periods of
time. However, a format that provides regularly updated, standardized tables and
charts of key indicators together with topical articles could be more useful and
easier to produce than the TSARs.
We suggest that BTS consider alternative formats to the TSARs and that, if an
alternative format seems workable, that it seek authority to adopt that format in
place of the required annual report. One alternative would involve the chartbook
that we earlier suggested BTS publish, together with BTS's new twice-yearly
Journal of Transportation and Statistics.
Under this alternative, the chartbook would include 10 to 20 key statistical
indicators with accompanying brief commentary and notes on methods; it would
first appear annually but, as resources permit, should be published more fre-
quently. It would be a publication that users look forward to receiving in order to
follow key trends. BTS's new journal would publish not only peer-reviewed
research and methods articles from BTS staff, other USDOT staff, and outside
researchers, but also articles on the state of transportation containing the kind of
analytical material that currently appears in the TSARs perhaps in a special sec-
tion covering selected topics in each issue. (As examples, an article on cross-
modal trends in safety might appear every December, and an article on energy
impacts of transportation might appear every June.) Our expectation is that re-
ports on particular aspects of the transportation system (e.g., safety, access, con-
dition of the infrastructure) that are presented in the form of journal articles will
be more accessible to users than the current TSAR format and less burdensome on
the BTS staff to develop.8
Together, the chartbook and the journal would fulfill the mandate in the 1991
ISTEA for BTS to provide information about the transportation system in an
annual report. (In this chapter and the preceding one, we recommend several
changes in BTS's roster of publications. Table 4-2 maps BTS's current publica-
tions to those that we recommend.) An alternative format that would accomplish
the same goal would be to fold both the chartbook and related analytical articles
into the new journal as a regularly appearing supplement. Whether this alterna-
tive is preferable to a separate chartbook depends on how often it appears useful
to publish key indicators and whether the desired publication schedule could be
accommodated by the journal. Under either alternative (folding both tables and
analyses or just analyses into the journal), a goal for the longer term should be to
The articles in the special section of the journal should be reviewed and held to high standards, but
the review process should be managed by BTS, given its mandate to produce regular reports on the
transportation system, and not by an outside editorial board.
OCR for page 85
85
a'
cq
o
son
a'
. .
o
c)
c)
o
.=
a'
cq
c)
· - ~
cq
.=
vO
o
.~
cq
EM
o
a'
Ad
¢
Em
ca
to
4=
v,
v
.~
c)
~ ~ ~ ~ l ' ~ ^ 1 l at i
) , ~ ~' ~ [ ' |, ~ ' ~ ~
ca
~o
~·> o ~ ~ o
to ~ ~
o if .= ~ ~ o
~ ~ ~o ~ c
~ ~ I ~ ~
.^ ~.^ ~ ~ .^
~3 %~ 3 ~ ~
~o
I , I l ,, ,
, . . .
OCR for page 86
86 BUREAU OF TRANSPORTATION STATISTICS: PRIORITIES FOR THE FUTURE
publish the journal on a quarterly instead of a twice-yearly basis in order to ac-
commodate the additional material.
Whatever publication format is adopted (one of those suggested or another),
we repeat that it is important for BTS to find the most cost-effective ways by
which to conduct and report the results of substantive research with its data. Such
research is an essential component of a statistical agency's mission to organize,
interpret, and communicate data so that the data become information that is rel-
evant for policy needs and other purposes.
RECOMMENDATIONS
Vision and Plan
(5) BTS should develop a long-term strategy for implementing fully all
of the areas in its mandate in order to evolve as a statistical agency that
ensures the relevance, as well as the quality, of transportation data. The
implementation plan should set priorities within the context of a vision of a
comprehensive system of transportation data.
National Transportation Indicators
(6) BTS should develop key national statistical indicators for the trans-
portation system for example, multimodal series in the areas of safety,
travel patterns, and the condition of the transportation infrastructure in
consultation with the statistical and analysis units in the other USDOT modal
administrations and the transportation community.
USDOT Statistical Budget
(7) In the reauthorization of BTS, Congress should require BTS to com-
pile, analyze, and provide to the secretary of transportation a department-
wide statistical program budget for the secretary's use in making decisions
during the budget process.
Building Relationships with States and
Metropolitan Planning Organizations
(8) BTS should regularly meet with representatives from states and met-
ropolitan planning organizations to help determine priorities for key national
statistical indicators, other data, analyses, products and services, and im-
provements in data concepts and measurements. BTS should also provide
technical advice to states and metropolitan planning organizations to help
them make more effective use of BTS and other transportation data.
Representative terms from entire chapter:
statistical agency
