National Academies Press: OpenBook

Measuring International Trade on U.S. Highways (2005)

Chapter: 3 Panel Findings and Conclusions: Data Quality and Reliability

« Previous: 2 BTS Report: Summary of Findings
Suggested Citation:"3 Panel Findings and Conclusions: Data Quality and Reliability." National Research Council. 2005. Measuring International Trade on U.S. Highways. Washington, DC: The National Academies Press. doi: 10.17226/11167.
×

3
Panel Findings and Conclusions: Data Quality and Reliability

In this section the panel selectively addresses several major aspects of the Bureau of Transportation Statistics (BTS) report, focusing on the analysis of the requirements for data quality and reliability associated with the purposes of allocation formulas. Section 4 of this report discusses potential (new) sources of data, the strengths and weaknesses of the models used in the BTS report, and the possible role of emerging technologies in obtaining needed data.

The panel concurs with the BTS report that available freight data are not of sufficient coverage and quality to permit precise determination of international trade traffic by state. The panel’s review of the availability of validated data to support estimates suggests needed improvements in data collection programs to enhance the accuracy and reliability of international trade traffic measures for formula apportionment purposes.

The panel concurs with the central finding of the BTS study (Bureau of Transportation Statistics, 2003) that the present state of available freight data does not permit precise determination of international trade traffic by state.

The panel was limited in its ability to evaluate the accuracy and reliability of the data for use as formula factors because no allocation formula that depends on ton-miles or value-miles of international trade traffic has been developed or proposed. Without knowing the formula and its purpose,

Suggested Citation:"3 Panel Findings and Conclusions: Data Quality and Reliability." National Research Council. 2005. Measuring International Trade on U.S. Highways. Washington, DC: The National Academies Press. doi: 10.17226/11167.
×

it is not possible to assess how accurate estimates of international trade traffic must be. If, for example, the central purpose of the formula is to compensate states for wear and tear on their highway infrastructure, then even highly precise measures of value-miles will bear little relationship to the objective of the formula, and ton-miles will only roughly reflect the goal of the formula. Less accuracy is needed if the purpose of the formula is insensitive to variations in ton-miles than if the formula is highly sensitive.

More generally, the accuracy that is needed for formula allocation may depend on whether the objective is constructing, maintaining, and operating infrastructure; reducing congestion; minimizing the impact of the highways on the environment; improving the safety of the network; or any of a number of other worthwhile objectives. When evaluating the quality and reliability of data, such factors as error, bias, and transparency would be weighed differently, depending on the objective of the apportionment formula.

It is instructive to compare the quality of international trade traffic data with that of the data used in existing federal highway apportionment formulas. Each major federal highway program has a legislated formula that relies on data obtained from the joint Federal Highway Administration Highway Performance Monitoring System. The primary measures used in the current allocation formulas include lane miles, vehicle miles traveled, diesel fuel data, state population, urbanized area population, and nonhighway recreational fuel use. Each of these data elements has its own error measures, as well as an established validation procedure that is documented in guidelines that were carefully developed in a federal-state cooperative venture.

It is important to note that each of the data elements used in the existing formulas is directly and independently measured. The measures are also transparent: their sources are well known, and they use carefully monitored procedures for verification. The current formula allocation factors do not rely on model-based estimation procedures; each has a directly measurable error structure in which bias and variance can be specified.

The characteristics of verifiability and transparency are extremely important for estimates used as allocation factors. Unfortunately, the estimates computed in the BTS report are neither verifiable nor transparent.

The BTS model-based estimates of ton-miles and value-miles of inter-

Suggested Citation:"3 Panel Findings and Conclusions: Data Quality and Reliability." National Research Council. 2005. Measuring International Trade on U.S. Highways. Washington, DC: The National Academies Press. doi: 10.17226/11167.
×

national trade traffic have their genesis in a multiplicity of data sources and methodologies. As the BTS report appropriately notes, “limitations in the data have a negative effect on their accuracy and reliability.” Even when it is possible to specify the error associated with one of the data inputs, such as the sampling variance for the Commodity Flow Survey (CFS), it is not possible to specify the quality and reliability of the overall estimates, which reflect the accumulation of the errors1 of several data sources. It is also not possible to compute the uncertainty that is introduced into the estimates by several of the key methodologies, such as use of the gravity model, the substate allocation procedure, and the highway network assignment procedure.

The BTS report appropriately addresses the need for separately identifying international trade traffic data for use in an allocation formula. However, the panel is concerned that in the long run, a focus on estimating ton-miles and value-miles of international trade traffic by state may distract attention from the larger issues of characterizing sources of highway wear and tear and congestion for purposes of funds allocation. Wear and tear, congestion, and other concerns are not a function of whether the traffic is international or domestic; rather, they relate to such measures as traffic volumes, axle load factors, vehicle types and categories, and weight. It was observed in the panel’s workshop that pavement doesn’t care whether traffic is international or domestic. Moreover, even total ton-miles are only roughly related to wear and tear and other concerns.

It is likely that total ton-miles of all types of freight traffic moving by state highway can be estimated more accurately than the disaggregated measures of domestic and international trade traffic.2 Part of the problem of accurately estimating international trade traffic stems from two issues of disaggregation. We offer two examples, on disaggregating diesel fuel use and on computing substate estimates.

1  

By errors due to data sources here, we mean all sources by which the estimate fails to equal the true value, which includes various types of bias and sample variance.

2  

The panel notes that the Federal Highway Administration’s Office of Freight Management and Operations uses a variety of data sources in the development of the Freight Analysis Framework (FAF), which estimates (as of 1998) and forecasts (to 2010 and 2020) total freight tonnage flows on U.S. highways, as well as U.S.-Canadian truck traffic. The base year of the FAF is the 1993 economic census year, for which CFS data are also available. However, the FAF is assembled from existing data sources. Therefore, it is subject to the same problems of low frequency of acquisition and questionable accuracy as the data sources.

Suggested Citation:"3 Panel Findings and Conclusions: Data Quality and Reliability." National Research Council. 2005. Measuring International Trade on U.S. Highways. Washington, DC: The National Academies Press. doi: 10.17226/11167.
×

Disaggregating Diesel Fuel Use. The BTS report assesses the accuracy and reliability of diesel fuel data as a measure of ton-miles of international trade traffic on state highways. Diesel fuel data is already a factor in the apportionment of funds for the National Highway System program, but the measure has several weaknesses when pressed to serve as data for allocation purposes. It is difficult to estimate the number of gallons consumed for road use in each state because fuel may be purchased in one state and used in another, and there are differences in state reporting of motor fuel sales. Procedures used to adjust sales data to use data (by means of reports submitted by carriers under the International Fuel Tax Agreement) add variability to the estimates. Furthermore, not all diesel fuel is used for on-the-road vehicles: for example, diesel fuel is also used to run construction equipment and railroad locomotives, and therefore any heterogeneity in these uses across states would further limit the utility of diesel sales for this purpose. Other factors further reduce the reliability of these estimates. Nonetheless, the diesel fuel data are currently used as a factor in the apportionment formula. The BTS report states that these estimates “inherently capture international trade traffic.”

Several computations are necessary to separate international from domestic diesel fuel use. The BTS report uses a regression analysis of the relationship between the estimate of total ton-miles by truck by state and diesel fuel sales by state to produce a formula for predicting ton-miles based on diesel fuel use. The report finds that the regression relationships for many states are weak, suggesting that “diesel fuel usage estimates do not provide good predictors of total trade ton-miles on a state-by-state basis.” The panel is concerned that the next step—to break down the weak total relationships into domestic and international components—would require comparing estimates of international trade traffic ton-miles from the BTS study with estimates of total ton-miles by state from other sources.

The panel concurs with the BTS report that the process of disaggregating diesel fuel use into its domestic and international trade traffic components would compound variability in that it would apply a ratio of two estimates with significant variability with state diesel fuel use data that have known quality problems.

Computing Substate Estimates of Import Trade Traffic Flows. Existing data sources provide estimates of imports flowing to each state, but not of the distribution of imports within states. The distribution

Suggested Citation:"3 Panel Findings and Conclusions: Data Quality and Reliability." National Research Council. 2005. Measuring International Trade on U.S. Highways. Washington, DC: The National Academies Press. doi: 10.17226/11167.
×

of imports within a state is estimated by assuming that imports to each county are proportional to annual payrolls computed from data in the 1997 County Business Patterns. The accuracy of this assumption is unknown. For discussion of this point, see National Research Council (2004).

The issue of the quality of data used to support fund allocation formula programs is a complicated one; for a broad review of the considerations involved and much more general issues concerning inputs to and use of fund allocation programs, see National Research Council (2003b).

Suggested Citation:"3 Panel Findings and Conclusions: Data Quality and Reliability." National Research Council. 2005. Measuring International Trade on U.S. Highways. Washington, DC: The National Academies Press. doi: 10.17226/11167.
×
Page 15
Suggested Citation:"3 Panel Findings and Conclusions: Data Quality and Reliability." National Research Council. 2005. Measuring International Trade on U.S. Highways. Washington, DC: The National Academies Press. doi: 10.17226/11167.
×
Page 16
Suggested Citation:"3 Panel Findings and Conclusions: Data Quality and Reliability." National Research Council. 2005. Measuring International Trade on U.S. Highways. Washington, DC: The National Academies Press. doi: 10.17226/11167.
×
Page 17
Suggested Citation:"3 Panel Findings and Conclusions: Data Quality and Reliability." National Research Council. 2005. Measuring International Trade on U.S. Highways. Washington, DC: The National Academies Press. doi: 10.17226/11167.
×
Page 18
Suggested Citation:"3 Panel Findings and Conclusions: Data Quality and Reliability." National Research Council. 2005. Measuring International Trade on U.S. Highways. Washington, DC: The National Academies Press. doi: 10.17226/11167.
×
Page 19
Next: 4 Data Sources, Models Used, and Emerging Technologies »
Measuring International Trade on U.S. Highways Get This Book
×
Buy Paperback | $29.00 Buy Ebook | $23.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

International trade plays a substantial role in the economy of the United States. More than 1.6 billion tons of international merchandise was conveyed using the U.S. transportation system in 2001. The need to transport this merchandise raises concerns about the quality of the transportation system and its ability to support this component of freight movement. Measuring International Trade on U.S. Highways evaluates the accuracy and reliability of measuring the ton-miles and value-miles of international trade traffic carried by highway for each state. This report also assesses the accuracy and reliability of the use of diesel fuel data as a measure of international trade traffic by state and identifies needed improvements in long-term data collection programs.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!