7
Dissemination
In this chapter, we consider the penultimate step in the cycle of a data collection program (see Table 1-1 in Chapter 1)—namely, disseminating the collected information to users in accessible and useful ways. We first consider the range of expertise and data needs among the users who would likely benefit from an integrated Annual Business Survey System (ABSS). We then describe current data products and dissemination practices for the annual economic surveys and ways to improve them.
In Chapter 2, we describe the current user community for the annual economic surveys and how that community is likely to grow once the surveys are integrated into an ABSS. That chapter discussed the views of users in federal agencies, state and local agencies, and the private and academic sectors on five dimensions of data quality, broadly considered: (1) accuracy, (2) timeliness, (3) relevance (including level of geographic and industry detail needed by different users), (4) consistency, and (5) accessibility. When planning improved dissemination tools and products for the current suite of surveys and an ABSS, the Census Bureau will want to be sure that they serve, to the extent possible, these different groups of uses and users. Other potential users are journalists who report on business statistics; they may become more aware of and interested in the annual economic surveys as those surveys become more harmonized and ultimately integrated into an ABSS.
In this chapter, we distinguish users primarily by their skill levels and familiarity with data from the Census Bureau. We define four types of data users, viewed from a dissemination perspective:
- Expert: Experts are frequent users with sophisticated computer skills who seek data of multiple types in high-volume and multiple formats. This category includes data intermediaries who repackage census data for their clients. Experts frequently use their own application programs to call the Census Bureau’s application programming interface (see below). Researchers in academia and elsewhere who value access to microdata may also be considered experts.
- Experienced: Experienced users, whether they access data occasionally or frequently, generally seek specific data with which they already are familiar. Experienced users are most likely to want to download text or spreadsheet files, often large, that they manipulate and analyze themselves, using software such as Microsoft Excel, R, or Stata.
- Casual: Casual users are occasional users who sometimes have a specific need for census data but also may be looking for something that they need help in locating. Casual users are apt to use formatted tables or figures that can be transferred and used directly and to download Excel files.
- Novice: Novice users are new to www.census.gov and have little notion as to what types of data are available or where to find them. They typically come looking for a specific number or statistic and need guidance as to how to find it and interpret what they find.
For expert users, the main goal is to provide efficient access to large volumes of data that are downloaded repeatedly. Expert users almost always reformat and analyze census data for their own use or that of their clients. That is, much of the work involved in moving from raw data to final knowledge occurs downstream from the Census Bureau. Programming interfaces are likely to satisfy most experts’ needs, though large files in text or spreadsheet format also may be useful to them. A priority for the Census Bureau in disseminating to this group of users is to have a programming interface that is practical and functional. For nonexpert users—experienced, casual, and novice—the Census Bureau needs to continue to provide methods to retrieve data that do not require users to be programmers.
7.1 CURRENT DISSEMINATION PRODUCTS AND PRACTICES
Current dissemination practices for the annual economic surveys vary extensively, with different formats and modes across surveys. Some variation is inevitable and in fact could well be desirable given the different needs and abilities across the user types discussed above. Thus, multiple dissemination formats and modes may be warranted under an ABSS.
The existing variation, however, which manifests itself in an assortment of siloed data products and dissemination practices for the annual economic
surveys, is far from optimal. It appears to be mostly a reflection of the historical development of practices among survey-specific staff groups that have operated somewhat independently rather than as a result of a plan to effectively distribute data to disparate users. One unfortunate consequence of the different practices is disparate data formats that are resource intensive for the Census Bureau to produce and unnecessarily complicated for users. For example, as discussed in Chapter 6, the annual economic surveys have varied cutoffs for statistical precision suppression. Some adhere to a threshold requiring a coefficient of variation below 30 percent for a cell to be reported, while others use 40 percent or 50 percent cutoffs. These differences require survey-specific suppression editing (often by hand) and create confusion for data users who want to combine data from multiple surveys. We are not aware of any benefits from this variation.
Similarly, disparate dissemination modes characterize the annual economic surveys, which use one or more of the following five modes:
- hypertext (HTML) data tables that can be viewed and downloaded with a browser;
- Excel spreadsheet files of tables;
- American FactFinder (AFF), an online tool that produces tables according to a user’s instructions for both population and economic datasets;1
- an application programming interface (API) that allows users and data intermediaries to write computer code in Python or other high-level languages to acquire data directly and to manipulate the data with the flexibility of a full programming language. Of the annual economic surveys, APIs are currently available for the Annual Survey of Entrepreneurs and the Annual Survey of Manufactures;2 and
- access through a Federal Statistical Research Data Center (FSRDC), where qualified researchers can analyze confidential microdata (stripped of basic identifiers, such as name and address) to produce aggregated tabular or analytical output that is reviewed for disclosure protection before the researcher may take it out of a center.3
___________________
1 Currently, AFF produces tables from the Annual Survey of Manufactures but no other annual economic survey: see https://factfinder.census.gov/faces/nav/jsf/pages/programs.xhtml?program=aes [November 2017].
2 See https://www.census.gov/data/developers/data-sets.html [November 2017].
3 The FSRDC network currently comprises 29 centers around the country, typically housed at universities. The network began when the Census Bureau opened its first research data center at a location other than its headquarters in 1994, and it became a network for the federal statistical system, managed by the Census Bureau, in 2014: see https://www.census.gov/fsrdc [November 2017].
No single economic survey offers every mode, and most offer three or fewer. The most common dissemination procedure is to provide data through a table template system.4 In this system, a survey-specific staff group at the Census Bureau decides on the format and organization of particular tabulations of the microdata from the survey. These data are then processed and aggregated to fill out the templates. Although some of this process has elements of automation, it is still common for much of this work to be done manually, including data review, validation, and application of suppressions. Multiple handoffs and hand-checks are typical. This process applies not just to files disseminated directly as tables, but also to those distributed through other modes. As a consequence, the tables are often not comparable across surveys.
The AFF is not satisfactory in its current form. As we indicate in Chapter 2, users told us that it is not easy or straightforward to use, even for frequent users, unless they are willing to invest considerable time to learn how to work with the system. And even experienced users would benefit from more direct access to files rather than having everything filtered through a system. Piecing together a dataset geography by geography or industry by industry that originally was comprehensive is a common and unnecessary chore for users. Less experienced users find navigating the tool to be too difficult or time intensive. Fortunately, we understand, the AFF is headed for retirement.
The existing Census Bureau API uses the industry-standard protocol for describing data, but involves complicated coding, at least in the examples that the Census Bureau provides. The Bureau of Labor Statistics provides more extensive examples of the use of a similar API, starting with an example that involves a simple function call and only a few lines of code.5 Practicality and ease-of-use should be a priority for the Census Bureau API; it need not be leading-edge technology.
The FSRDC network houses microdata from most of the annual economic surveys, as well as the economic censuses and monthly and quarterly indicator surveys; we anticipate that it would also house microdata from an ABSS as soon as they are available. We note that the network uses common protocols to provide controlled access to a wide range of population and economic data and has a record of more 20 years of operation and path-breaking research: see the annual reports of the Census Bureau’s Center for Economic Studies (e.g., U.S. Census Bureau, 2017).
We endorse the recommendations in other reports about the continuing need to streamline the process for gaining approval to carry out research in one of the FSRDCs (see Commission on Evidence-Based Policymaking,
___________________
4 See, for example, https://www.census.gov/retail/index.html [November 2017].
5 See https://www.bls.gov/developers [November 2017].
2017, pp. 36–37, 70–71, 81; National Research Council, 2007, p. 108). Further, the recommendations we make below for improved access tools and documentation for the annual economic surveys will facilitate research with the microdata in an FSRDC. The potential for cross-cutting research on business dynamics across industries and time will be greatly facilitated once ABSS microdata, based on consistent concepts and measures, are available in the FSRDC network.
Two other current dissemination practices for the annual economic surveys are not optimal for users. The first concerns timing and timeliness: the timetables for release of data from the various surveys are not set out in sufficient detail in advance, and, more important, the release dates lag considerably behind the completion of data collection for the surveys. The second concerns documentation, that is, metadata (data about the data) and user assistance. The Census Bureau provides basic documentation, such as variable names for spreadsheet files, but, according to the users we heard from, there is little additional information that would help them understand the data. There is also no assistance for users, such as webinars, to help them work with the data generally and, specifically, to help users who want to relate data from more than one survey to create a broader picture, perhaps across an industry, or to link data from successive annual files for a single survey into a consistent time series.
7.2 TOWARD IMPROVED DISSEMINATION
As dissemination tools are reconfigured looking to an ABSS, we suggest that four principles help guide this process. The first principle is for the Census Bureau to adopt a user-targeted approach to dissemination under an ABSS. The overriding goals of this approach would be simplicity for novice users and timeliness and completeness for expert users.
The second principle is for the data products (such as tables) from the separate annual economic surveys to be standardized wherever possible in terms of their content and format (see Chapter 6). An alignment of what are now disparate products and formats will reduce the staff resources necessary to make data available and provide data users with faster, easier, and more consistent access. These changes also will make it easier for users to tie data together across the surveys and from different years, tasks that are not well supported with current dissemination practices.
The third principle is for the Census Bureau to respond to expert and experienced users’ desires for ease of access and retrieval since they are often capable of solving their own problems without help from the Bureau. This low-touch access could be facilitated by the easy availability of flat data files and documentation, with more examples of download code in
Python and Matlab than currently offered. A good deal of this information is now available only from third-party websites.
The fourth principle is for the Census Bureau to recognize that casual and novice users need more value added by the Bureau that is carefully targeted. Thoughtful reconsideration of the menu of available tables, for example, could reduce the burden on census staff to make this information available with minimal loss of valuable access. Users’ interest in the tables that are currently made available is highly skewed. For example, of roughly 100 tables of results available for the Survey of Business Owners (SBO), the single table most viewed by users on AFF during the period 2012–2015 accounted for two-thirds of all views of tables for that survey. The top four tables accounted for four-fifths of all views.6 More careful targeting of the production and release of formatted products could free up resources for other purposes. When combined with an easy-to-use search tool, this targeting would create an efficient system to disseminate annual economic surveys data to less sophisticated users.
RECOMMENDATION 7-1: The Census Bureau should design access and retrieval mechanisms for data from the annual economic surveys and an Annual Business Survey System, once developed, to be as straightforward as possible; users should not be forced to use complex tools to do simple things. Expert and experienced users should find it easy and efficient to download data and produce their own analyses. The number and types of formatted tables and outputs targeted to casual and novice users should be curated to balance availability and efficiency and harmonized to the greatest extent possible among the surveys.
Simplicity for users need not imply that more work will be required of the Census Bureau. In fact, the panel believes that, while keeping things straightforward for users may require careful upfront design considerations, once put into place, the new dissemination systems will be simpler for the Bureau to administer.
Furthermore, the Census Bureau does not need to start this redesign from the ground up as the approaches already developed by other organizations provide models from which the Census Bureau can borrow. Among U.S. federal statistical agencies, examples worth studying include the dissemination tools provided by the Bureau of Economic Analysis, the Bureau of Labor Statistics, and the National Center for Education Statistics. Other
___________________
6 Information provided to the panel by the Census Bureau. The SBO (see https://www.census.gov/econ/overview/mu0200.html [November 2017]) is conducted every 5 years in conjunction with the economic censuses.
examples are Statistics Netherlands; FRED at the Federal Reserve Bank of St. Louis; and, among proprietary data intermediaries, Haver Analytics.7 Similarly, dissemination software does not need to be custom built. There are many benefits from using off-the-shelf software, including open-source software in the public domain. Although using outside tools might raise data security issues that would need to be addressed, in many cases the benefits could be large enough to outweigh any additional costs related to security.
RECOMMENDATION 7-2: In designing improved data access and retrieval mechanisms for the annual economic surveys, the Census Bureau should learn from and use successful approaches of other agencies and organizations, including, to the extent possible, the use of off-the-shelf interface designs, software, and tools.
For all data products from the annual economic surveys and an ABSS, once developed, a key companion goal is to provide users easy access to ancillary information about the data. Users need ready access to survey questionnaires, sampling information, weighting protocols, metadata (such as precision metrics), and other documentation relevant to the data they are seeking. We suggest that links to online documentation could suffice for this purpose. For questionnaires, it is important that users be able to print out a readable version that facilitates understanding of how questions are asked and pathways through the questionnaire logic (see also Chapter 4 on this point). We commend the Census Bureau on providing printable versions of electronic questionnaires for the current annual economic surveys.8
In addition to documentation for specific surveys, users should be able to obtain training and assistance in accessing and working with the data. The users who spoke with the panel strongly voiced their need for assistance through webinars, online tutorials, users’ guides, training sessions at relevant conferences, and other means.
RECOMMENDATION 7-3: The Census Bureau should take steps to ensure that users accessing data from the annual economic surveys have ready access to comprehensive documentation about the surveys and survey data, including readable hard-copy versions of electronic questionnaires. The Census Bureau also should provide
___________________
7 See, respectively, https://www.bea.gov/API/docs/index.htm; https://www.bls.gov/bls/api_features.htm; https://nces.ed.gov/datalab/; https://www.cbs.nl/en-gb/our-services/open-data;https://research.stlouisfed.org/datatools.html; and http://www.haver.com/datalink.html [November 2017].
8 See, for example, https://www.census.gov/programs-surveys/arts/technical-documentation/questionnaires.html [February 2018].
training and assistance to data users through such means as webinars and online tutorials.
7.3 DATA MANIPULATION
For casual and novice users, it is appropriate that the data interfaces on www.census.gov provide tools that can generate simple statistics, such as graphs and tabulations. It is, however, unnecessary and likely even undesirable for the Census Bureau to attempt to facilitate more complex analytics. Users who want to do more thorough analyses on the data are generally experienced or expert users and they will usually use other tools for their needs, such as the open-source statistical analysis system, R, now widely used in teaching statistics, and proprietary statistical and computational software, such as Stata and Matlab. All widely used statistical software can read spreadsheets downloaded from the Census Bureau website, and Matlab and other programming languages can use the Bureau’s API to download data automatically.
RECOMMENDATION 7-4: The Census Bureau should provide a user-friendly interface for producing basic statistics, such as percentages, means, and cross-tabulations, and for drawing charts from annual economic surveys data, but more complex analysis should be left to users.
The Census Bureau could usefully give more priority to enabling users to create consistent time series. There is a natural bias in favor of cross-sectional presentations of data because that is how they are collected, but year-by-year changes are important and indeed are what many if not most nonnovice users are interested in. Although some time series are available in spreadsheets on www.census.gov, many users want to create their own time series in a consistent manner from single-year tables, spreadsheets, and other products from the annual economic surveys. For this purpose, it will be important for the Census Bureau to make the data available for the smallest unit of analysis possible, while protecting confidentiality and satisfying precision standards for data release. For example, it would be desirable if size categories of establishments or companies for such variables as employment, payroll, and revenues were as narrow as possible while remaining consistent with confidentiality restrictions, rather than as a few broad categories, as is now the case. If this approach is pursued, it will raise issues about changes over time in the surveys. The Census Bureau will need to inform users of changes in methodology, sampling, questionnaires, and units of analysis that may have affected the comparability of data over time.
RECOMMENDATION 7-5: The Census Bureau should make data from the annual economic surveys available on the basis of the smallest possible unit of analysis subject to confidentiality and statistical quality restrictions, allowing users to create their own time series and other aggregations of the data. Documentation should inform users of changes in survey procedures that could affect data comparability over time and among surveys.
7.4 SCHEDULING OF DATA RELEASES
Users of all types would benefit from more timely data releases from the annual economic surveys. We recognize that a balance generally must be struck between speed and data quality, but improvements in timeliness are possible. Standardization of processes and data products as discussed throughout this report is likely to facilitate reduced times between data collection and dissemination without compromising data quality. As we discuss in Chapter 5, one approach would be the development of preliminary estimates of key outputs from the annual economic surveys, which would be of considerable help to users who require key statistics as soon as possible after data collection.
Data releases from the surveys also could be more precisely scheduled than they are currently. At present, the month in which data will be released is determined well ahead of time, but the particular day is not announced until later. More specificity about when data will be released could help draw more attention to data releases and help users plan for them. Given the tradeoff between speed and quality, data release times from the annual economic surveys will be driven by the Census Bureau’s ability to collect and edit the basic data.
RECOMMENDATION 7-6: The Census Bureau should schedule release dates for annual economic surveys data products, including release dates for preliminary estimates, sufficiently ahead of those dates for users to plan for their availability.
7.5 PUBLICIZING DATA AND DATA DISCOVERY
Our discussion so far has focused primarily on users who come to www.census.gov knowing that the site is likely to provide the data they seek. The Census Bureau also has a role in providing useful information to people using the web to search for economic facts. In comparison with other organizations seeking wider use of their products, the Census Bureau engages in relatively little search-engine optimization and makes only limited efforts to attract and serve casual and novice users. Some of the
Bureau’s current approaches, such as the release on social media of “gee-whiz” facts and infographics constructed from census economic data, hold promise. In contrast, however, one of the more common Census Bureau dissemination activities is the issuance of press releases, and the examples of press releases the panel has examined were disappointingly uninformative. They contained little to make the information contained in the survey data attractive or appear insightful or useful to anyone who is not already a regular user. The panel believes that annual economic surveys data are newsworthy in and of themselves and that the Bureau can do more to emphasize their value in the ways it informs the public of data availability.
RECOMMENDATION 7-7: The Census Bureau should formulate a plan to raise the profile of data from the annual economic surveys. Currently successful strategies, especially those on social media, should continue and be expanded where possible. Press releases should accompany every data release, and they should be self-contained and compelling.
Even when users are fairly sure that www.census.gov will provide data they can use, casual and novice users in particular face the challenge of discovery—that is, of tracking down the specific data that could meet their needs. The Census Bureau has a discovery tool in beta release, http://api.census.gov/data.html. It is a spreadsheet without organic search methods that can be searched in Excel or as a text file.
For casual and novice users, a better, simpler in-site discovery tool would be valuable. At present, it is unnecessarily difficult for such users to find the data they want unless they have considerable experience searching Excel or text files or navigating www.census.gov and its many legacy web-pages. A web-based topical index to the annual economic surveys data with dropdown granularity of detail to guide users to appropriate data locations would be useful. Vivid and simple examples of the nature and use of data from the surveys would be helpful to novice users.
One alternative to a comprehensive in-site discovery tool that might be simpler to develop and still be helpful is a menu-based interface. We envision a system in which a user first identifies data of interest; next obtains a dropdown menu or fill-in-the-blank search query; then selects the dataset to be accessed; then specifies specific items, sample periods, and industries or areas; and, lastly, selects a method for delivery—text file, spreadsheet, or figure. Discovery tools at data suppliers such as FRED, the service offered by the Federal Reserve Bank of St. Louis, offer examples that illustrate the design of this class of tools.9
___________________
9 See https://research.stlouisfed.org/datatools.html [November 2017].
RECOMMENDATION 7-8: The Census Bureau should develop and provide training for a user-friendly discovery tool for casual and novice users who may be interested in data from the annual economic surveys. This tool should be featured on the likely landing page for such users.
7.6 ARCHIVING
It is important for any data-producing agency, including the Census Bureau, to adopt and follow best practices for archiving data and documentation so that users can readily retrieve and work with historical information. A well-specified plan for data archiving and retrieval will need to be included as a priority design element for the recommended ABSS. It may be necessary to develop such a plan retroactively for any of the annual economic surveys for which archiving procedures are not clearly prescribed. For this purpose, the Census Bureau can take advantage of existing documentation and archiving standards, such as the Data Documentation Initiative.10 In addition, best practices, such as permanent url addresses (e.g., digital object identifiers, DOI) for datasets and associated documentation and easy-to-navigate paths to archived datasets from a landing page, will greatly facilitate user access to valuable data resources. Having permanent url addresses has the added advantage that the Census Bureau, by scraping the web, can trace users and uses of its data that cite these addresses.
RECOMMENDATION 7-9: The Census Bureau should have explicit detailed procedures for archiving data and documentation from the annual economic surveys and an Annual Business Survey System, once it is developed. The Census Bureau should also follow best practices, such as permanent url addresses, for datasets and documentation to facilitate the ability of users to access valuable historical information from the annual economic surveys.
7.7 MONITORING OF DISSEMINATION PRACTICES
Best practices for an organization are rarely static. In this chapter, we offer our ideas about basic principles for the initial design of dissemination processes and mechanisms for the recommended ABSS, many of which could be implemented for the current annual economic surveys. We recognize, however, that as user types and needs evolve, and as the Census Bureau’s own priorities and resources shift, the best ways to bring ABSS data to the public will also change. It will be important that the Census
___________________
10 See http://www.ddialliance.org [November 2017].
Bureau consistently monitors, both quantitatively and qualitatively, who is using its data and how the data are being used and uses that feedback to adjust both its data products and modes of dissemination.
RECOMMENDATION 7-10 The Census Bureau should monitor the use of data from the annual economic surveys and build feedback loops so that data availability and data access tools can be adjusted to usage. That monitoring can perhaps be partly automated, but it will certainly also involve analysis by staff. The Census Bureau should monitor its web traffic as an important part of a feedback loop.
7.8 REFERENCES
NOTE: All URL addresses were active as of November 2017.
Commission on Evidence-Based Policymaking. (2017). The Promise of EvidenceBased Policymaking. Washington, DC: Author. Available: https://cep.gov/content/dam/cep/news/2017-09-06-news.pdf.
National Research Council. (2007). Understanding Business Dynamics—An Integrated Data System for America’s Future. Panel on Measuring Business Formation, Dynamics, and Performance. J. Haltiwanger, L. Lynch, and C. Mackie (Eds.). Committee on National Statistics, Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press. doi: https://doi.org/10.17226/11844.
U.S. Census Bureau. (2017). Center for Economic Studies and Research Data Centers Research Report: 2016. Washington, DC: Author. Available: https://www.census.gov/ces/pdf/2016_CES_Research_Report.pdf.
