3

Measuring Retail Employment and Labor Productivity

This chapter discusses the task of measuring retail employment and productivity, addressing both the conceptual elements that need to be measured and the available data for doing so. Inevitably, the available data fall short of capturing the concepts, which leads to the discussion in the next chapter of possible ways of addressing those shortfalls in the context of a retail satellite account.

As a starting point for discussion, this chapter first clarifies two high-level concepts essential to measuring productivity and its components: output (including price deflators) and input. It then considers the definition of the retail sector, which structures the way data related to retail are collected, expanding on the relevant concepts and the data provided by U.S. federal statistical programs to measure output, including deflators, and employment. The chapter ends with a discussion of alternative data sources.

The concepts and data presented here derive from two main sources: documentation provided by the U.S. federal statistical agencies and the information-gathering workshop organized by the panel. The panel’s workshop included three sessions related to measurement issues. The workshop’s third session focused on key measurement and data challenges, with particular attention to the data collected and the measures produced by the Bureau of Labor Statistics (BLS), the U.S Census Bureau, and the Bureau of Economic Analysis (BEA). The workshop’s fifth session focused on quality-adjusted prices, a particularly difficult measurement issue related to deflation of output. The sixth session discussed improvements in the measurement of retail trade productivity that might be gained using microdata from the statistical agencies. Beyond those three sessions, the workshop’s

other sessions prompted a number of exchanges on the conceptual and data issues related to measuring retail employment and labor productivity.

MEASURING RETAIL EMPLOYMENT AND LABOR PRODUCTIVITY: THE HIGH-LEVEL TASK

The project focuses on the concepts and data needed to measure employment and labor productivity in the retail sector. This scope immediately raises questions about the ways such terms as “employment,” “labor productivity,” and “retail sector” should be defined. Answering those questions is the task of this chapter, and it is also the main focus of government economic statistics programs that must implement data-gathering processes to produce a set of measures of the economy.

A key point to note is that the productivity concept at issue is labor productivity, not multifactor productivity. Labor productivity concerns the amount of output produced per unit of labor input. At heart, this involves the division of industry output by labor input, once the appropriate measures have been defined. Box 3-1 summarizes the current labor productivity measures produced by BLS for the trade industries (retail and wholesale) and retail-related services. For these industries, the output measure currently used by BLS is gross sales deflated with a price index. That is, the sales figure is adjusted to convert dollars to a base year, in order to remove apparent changes in output that are actually due to price changes. Input is measured as hours worked.

Alternative approaches to measuring labor productivity for retail-related industries also use the equation shown in Box 3-1 but may define output (deflated using an appropriate deflator) as gross margin (sales revenue minus cost of goods sold), sectoral output¹ (gross output minus all inputs originating from firms within the industry being measured), or value added (gross output minus the value of all inputs originating as the output of other firms).

BLS measures labor productivity by deflating detailed revenues with corresponding price indexes, using either its own Consumer Price Index (CPI) or Producer Price Index (PPI) or else Merchant Wholesale Deflators from BEA.

BLS determines the revenue obtained from specific product classifications within each industry. Revenues for detailed product classes are deflated with corresponding price indexes. For about 97 percent of retail sales, BLS uses price indexes from CPI. For about 3 percent of the sales the Bureau uses a PPI, because pricing data for those products or services are not

___________________

¹ The BLS labor productivity measures for the manufacturing sector, individual manufacturing industries, and NIPA-level nonmanufacturing industries are calculated under a sectoral output approach.

available from the CPI. For services, BLS employs a mix of PPIs and CPIs, using a larger portion of the former depending on the industry. For wholesale output, BLS uses PPIs for the manufacturers’ sales branches and offices and merchant wholesale deflators from BEA. For industries for which BLS possesses revenue detail from the Economic Census (conducted every 5 years) to break up the annual sales, BLS applies available product- or service-specific deflators to the detailed portion of the total revenue. When the Bureau does not have the revenue detail, as for several services industries, it uses the total-industry deflator from PPI.² Alternatives would use a price deflator appropriate for the selected output measure.

The input measure currently used by BLS to estimate labor productivity is hours worked. Alternative formulations adjust hours worked for differences in labor composition (typically, education or skill). An example of a potential alternative formulation³ is used by BLS in its formulation of multifactor productivity, as described next.

___________________

² From e-mails with Jenny Rudd, BLS, December 14-16, 2020.

³ Another potential alternative way of accounting for labor composition is used by BLS/BEA in their joint Industry-Level Production Account.

Multifactor productivity involves multiple inputs in addition to labor—including various forms of capital and other purchased inputs such as energy, materials, or purchased services. In addition, the BLS multifactor productivity measure adjusts hours worked to account for differences in labor composition. As described in BLS (2020):

Because multifactor productivity involves multiple inputs, computations are more complex. This report does not address the additional conceptual and data issues related to measuring multifactor productivity.

DEFINING THE RETAIL SECTOR

This discussion starts with the last of the three terms that need to be defined—“retail sector”—because of its centrality to the motivation for the project. Specifically, the project seeks to answer the question, Has the transformation in retail affected the definition of the sector in ways that would require a different, perhaps broader, definition of the sector? The statement of task for the project asks about the creation of a satellite account that could address a “retail-related” sector that would go beyond the businesses included in retail alone.

This section first considers how the definition of industries in economic data affects the ability to identify a retail-related sector. It then turns to an important practical problem in the way industry classification is carried out in the United States across multiple statistical agencies.

Defining Retail-Related Establishments in Federal Data

One of the key organizing frameworks for federal economic data is provided by the North American Industry Classification System (NAICS),

which classifies establishments in a hierarchical coding system according to their primary activity. Box 3-2 provides more detail about NAICS. NAICS was implemented in 1997 to replace the Standard Industrial Classification

System (SIC), which classified establishments by sector using different concepts, such as production- or demand-based definitions.

A key difference between the SIC and NAICS classifications is in their treatment of auxiliary establishments. As Fort and Klimek (2018, p. 8) explain,

Hence, under NAICS, additional information needs to be used to identify whether an establishment is an auxiliary that primarily supports retail trade.

The 1992 Economic Census, the last such census that relied solely on SIC classifications, showed more than 840,000 auxiliary employees assigned to retail trade out of a total of 18 million retail trade employees. Also in 1992, BLS payroll data showed 13 million retail trade employees. In 1997, the Economic Census collected data with sufficient detail so that it could be categorized under both SIC and NAICS. That year the number of retail trade employees fell to 13 million, close to the count from BLS payroll data. Today, under the NAICS system, the auxiliary employees who had been listed in retail trade under the SIC classifications are most likely to have been moved to one or more of the following sectors or subsectors: Management of Companies and Enterprises; Administrative Support, Waste Management and Remediation Services; Warehousing and Storage; Computer Systems Design and Related Services; and Accounting, Tax Preparation, Bookkeeping and Payroll Services.⁴

Ding and colleagues (2020, p. 1) illustrated the impact of auxiliaries on the manufacturing sector, observing that

The changes in the retail trade sector discussed in the preceding chapter suggest that a broader definition of the sector than provided by NAICS

___________________

⁴ See http://www.incontext.indiana.edu/2002/july-aug02/details.asp.

might be required to be able to understand the shifts that are occurring. In particular, the restructuring that started first with the warehouse clubs and superstores and then moved on to e-commerce has begun to blur the lines between the retail industry and several other sectors, including wholesale trade, warehousing and storage, different types of transportation, and some other types of business services.

The ability to analyze these changes in an integrated way is directly affected by the structure of the economic data in the U.S. federal statistical system and, therefore, by the NAICS classification.

CONCLUSION 3-1: Given the structure of economic data in the U.S. federal statistical system, a study of the retail-related sector will require identifying those North American Industry Classification System codes that can be defined as either retail related or partially retail related. For those that are partially retail related, estimates will be needed for the portion that is related to retail.

Classification

Until this point, the discussion has described the NAICS classification of businesses in the abstract. However, the classification scheme needs to be applied to a specific set of businesses using a set procedure to determine the classification for each business unit. See Box 3-3 for a brief introduction to guidelines for classification.

Each statistical agency independently uses NAICS guidelines to classify establishments into industries on the basis of their primary activity, as measured in that agency’s data, and updates that classification on its own agency schedule. Generally, for an establishment engaging in more than one activity, the entire employment of the establishment is included under the industry indicated by the primary activity.⁵ Because business registers rely on different underlying source data, the Census Bureau and BLS may assign the same establishment to different industries or record the establishment with a different employment level.

There is even less agreement concerning the assignment of a code to enterprises, because such classification is not required under NAICS. In fact, BLS does not assign NAICS codes to enterprises. Some agencies choose to assign NAICS codes to enterprises based on their own internal data, and

___________________

⁵ Some large companies report different activities at the same location as separate profit centers. The Census Bureau’s County Business Patterns and Statistics of US Businesses (SUSB) program treats each profit center as a separate establishment. The Economic Census reporting may combine the profit centers into one establishment. This results in establishment count differences due to differences in how the data are collected. See https://www.census.gov/programs-surveys/cbp/technical-documentation/methodology.html#par_textimage_36648475.

some may ask respondents to report the code that best describes their primary business activity. The Census Bureau’s Statistics of U.S. Businesses classifies enterprises in this way:

One of the challenges with enterprise classification is that the “primary sector” is likely to change over time as business lines evolve and establishments are bought and sold.

Ideally, the NAICS coding of establishments would be applied uniformly, with all federal statistical agencies using the same code for each business unit. However, this is effectively impossible in the U.S. context, due to laws that restrict the sharing of individually identifiable information, even across federal statistical agencies. As a result, the two agencies that provide data related to business output and employment, the Census Bureau and BLS, each develop their own address lists and classifications of business establishments, with limited ability to share and compare them.⁷

___________________

⁶ See https://www.census.gov/programs-surveys/susb/technical-documentation/methodology.html. An enterprise may consist of groups of establishments that operate in different sectors. Each such group may be referred to as a “firm” for purposes of reporting to annual, quarterly, and monthly surveys described later in this document.

⁷ As a result of the Confidential Information Protection and Statistical Efficiency Act of 2002, BLS now receives annually the list of enterprises and the Employer Identification Numbers (EINs) that are associated with them as well as business names, addresses, and industry codes from the Census Bureau’s Business Register. However, the panel was told that merging the two lists is a time-consuming, resource-intensive exercise, because in many cases different EINs are recorded on the two lists. In an email from Ken Robertson on September 2, 2020, BLS observed that an enterprise can and often does have multiple EINs. Consider a large enterprise with 60 subcomponents. The company might register one EIN with IRS for the entire enterprise, or it might register one EIN for each subcomponent. It might report 20 of those 60 EINs to the Census Bureau, aggregating 3 subcomponents each into each of 20 reports, and list a different 20 when reporting on employment to BLS, or report all 60 to one agency but not the other. Even within the Census Bureau, an enterprise may use one EIN to report payroll and another to report revenue. So, even with Census and BEA data we have incom-

Business Registers

A long-standing issue for the U.S. federal statistical system has been the fact that the economic and business surveys conducted by the Census Bureau and those conducted by BLS rely on samples drawn from separate business registers (sampling frames), with different strengths and weaknesses (Fairman et al., 2008; Fixler and Landefeld, 2006). More recent updates are described in National Academies of Sciences, Engineering, and Medicine ([NASEM], 2017, p. 41):

More recently, in November 2020, the American Economic Association provided a letter to the incoming Biden-Harris transition team regarding “Necessary Improvement in the U.S. Statistical Infrastructure.”⁹ Under the seventh bullet in that letter, the association makes these points:

___________________

plete information about enterprises. An article that BLS has published profiling these data is available at https://www.bls.gov/opub/mlr/2016/article/establishment-firm-or-enterprise.htm.

⁸ The Obama administration pushed for this legislative authority (see, e.g., U.S. Department of the Treasury, 2014; U.S. Office of Management and Budget, 2016), but despite support from previous administrations and broad support from the statistical and research community no action has been taken for this limited data sharing of business tax information for exclusively statistical purposes by Census, BEA, and BLS (see http://www.copafs.org/UserFiles/fle/FederalBusinessRegistryLetterSenatewithAttach.pdf).

⁹ See https://www.aeaweb.org/content/file?id=13507.

As described in the National Academies’ report (NASEM, 2017, p. 41), “the Census Bureau’s Business Register is a listing of all legal business entities—incorporated businesses, partnerships, and sole proprietorships—operating in the United States and its territories (island areas) as identified by the U.S. Internal Revenue Service (IRS). It lists businesses that have paid employees (i.e., employer businesses), of which about 5 million have only one location and 160,000 have more than one location. It also lists non-employer businesses, of which there are about 25 million.”

The National Academies’ report (NASEM, 2017, pp. 42, 43) goes on to say that

The Business Register is updated with information from the IRS, the Economic Censuses, the Company Organization Survey (see Box 3-8), the Census Bureau’s Business and Professional Classification Survey, and the Annual Survey of Manufacturers (but no other annual economic surveys).

The Business Register for BLS is designed to support BLS’s Quarterly Census of Employment and Wages. Establishments are classified into industries on the basis of their primary activity. For an establishment engaging

___________________

¹⁰ BLS does not have access to IRS data.

in more than one activity, the entire employment of the establishment is included under the industry indicated by the principal activity. Industry information is also collected on a supplement to the quarterly unemployment insurance tax reports filed by employers.

The Quarterly Census of Employment and Wages, which consists of a monthly count of employment and quarterly counts of wage levels and business establishments, covers greater than 95 percent of the jobs available in the United States. The primary source for this census is administrative data from state unemployment insurance programs. These data are supplemented by data from two BLS surveys: the Annual Refiling Survey and the Multiple Worksite Report.

As reported by the National Academies (NASEM, 2017, p. 43),

In 2004, BLS and the Census Bureau reinitiated a project to compare and contrast the two registers. Preliminary results, reported by Becker and colleagues (2005), used an aggregate analysis comparing 2001 data from the Census Bureau’s County Business Patterns to data from BLS’s Quarterly Census of Employment and Wages. They observe that there are four main types of known differences between the two lists: differences in collection (the surveys used to maintain the lists), differences in scope (the parts of the economy covered by the data), data definitions (e.g., agencies use different definitions—for payroll and for designation of a company as “active”), and differences in reference period. If adjustments are made for these known differences, establishment counts and wages at the national level are similar, but employment differences remain. One of the challenges encountered in making comparisons at the sectoral level was that in 2001, BLS data were classified based on 2002 NAICS codes, while Census data were classified based on 1997 NAICS codes. The sectors impacted most by this difference were retail trade (NAICS 44-45), wholesale trade (NAICS 42), and information (NAICS 51).

Fairman and colleagues (2008) report on the follow-on study, a microdata match between the two registers. Of the 6.1 million unique EINs in the BLS and Census registers in 2003, only about 75 percent matched. They concluded: “while it seems likely that differences in establishment classification by Census and BLS at the same companies may explain a substantial

part of the industry differences between the two lists, the lack of a common establishment-level identifier makes matching individual establishments and comparing their industry codes very difficult” (pp. 5, 6).

One of the industry differentials they observed was an apparent misclassification of the headquarters operations for mining companies in Texas. These tended to be assigned by BLS to the category of Mining and by Census to Management (a category that includes auxiliaries). The paper concluded that “it is difficult to determine the best way to classify these establishments” (Fairman et al., 2008, p. 4). In general, BLS has limited information as a basis for designating auxiliaries.

Impact of Differences

Even when the concepts being measured are the same, some differences will emerge in the estimates by the two agencies due to their separate sources of data, separate processes for maintaining business registers, and the fact that different classifications may be assigned to the same enterprise by the two agencies. See Tables 3-1a and 3-1b for a comparison of the number of establishments and the number of employees estimated to fall under different NAICS codes as measured by three programs: the Economic Census (under the Census Bureau), the Statistics of US Businesses (also under the Census Bureau), and the Quarterly Census of Employers (under BLS).

The largest percent differences are in the establishment counts, between that of the Economic Census and that of the Quarterly Census of Employers, with the latter generally having larger counts and with percent differences ranging from –2.1 to 33.3 percent. The percent differences in employment are smaller, ranging from –5.5 to 10.6 percent. These latter percentages may provide a clue as to the impact of differences in classification on the numerator (output data collected by Census) and denominator (input data collected by BLS).

CONCLUSION 3-2: Labor productivity is measured as the ratio of change in output divided by change in input. Given that nominal output is measured by Census Bureau surveys while labor input and price deflators are measured from the Bureau of Labor Statistics’ surveys, and that the two agencies use separate business registers with separate classifications of business establishments by the North American Industry Classification System code as sampling frames for surveys, estimates of productivity are bound to contain errors. The resulting differences in statistics produced by the two agencies likely contribute to this error because different establishments may contribute to the numerator and denominator. The error most likely has a time-varying component because each agency updates its business list on a different schedule.

TABLE 3-1a Comparison of the Number of Establishments by NAICS Codes, as Measured by Three Programs, 2017

TABLE 3-1b Comparison of the Estimated Number of Employees by NAICS Codes, as Measured by Three Programs

CONCLUSION 3-3: The Bureau of Labor Statistics (BLS) annually receives a file containing Census Bureau Firm IDs, Employee Identification Numbers (EINs), and establishment detail. However, BLS does not use the Census file on a regular basis because the reconciliation of EINs between Census and BLS is labor intensive and time consuming. It would be beneficial to be able to quantify all of the activity under firm IDs that have some establishments classified as retail and for which linking BLS and Census firm and establishment data might help in identifying retail-related auxiliaries in BLS data, for example, something that is not currently possible. This has the potential for helping in the development of a satellite account on activities supporting retail trade.

CONCLUSION 3-4: A study using the Census Bureau’s firm-level microdata and other relevant information could be designed to develop factors to adjust for systematic differences between numerator and denominator to improve productivity estimates.

CONCLUSION 3-5: The ideal long-term solution to the issue of separate business registers being developed, maintained, and used by the Bureau of Labor Statistics and the Census Bureau would be to remove the obstacles to data sharing noted in National Academies of Sciences, Engineering, and Medicine (2017) and National Research Council (2007) and for the federal government to develop and use a single common business register.

MEASURING OUTPUT

Conceptual Issues in Measuring Nominal Output

A central part of measuring labor productivity is measuring the real output of an industry and the way it changes over time. This discussion is divided into two sections, because there are two substantial issues that need to be addressed: the concept of output that forms the basis of the measurement and the price indices that are used to deflate nominal measures of output to adjust for inflation. This section addresses the output concept and its measurement; the next section addresses the price indices.

As noted in the previous chapter and above in this chapter, output in the retail-related sector is defined in four different ways¹¹ in the federal

___________________

¹¹ Note that none of these output measures addresses household tastes, so they have no way of measuring the effect of increased product variety on consumer welfare, as explored by Neiman and Vavra (2019). It is not clear how the increase in product variety could be appropriately reflected in the output measure.

statistical system: (1) as total sales revenue; (2) as the difference between sales revenue and the cost of goods sold (gross margin); (3) as the difference between sales revenue and the cost of all purchased inputs (value added); and (4) as the difference between sales revenue and the cost of all inputs purchased within the sector (sectoral output). This conceptual discussion focuses on the contrasts in the first three definitions (see Box 3-4), since sectoral output varies with the definition of the sector, ranging from sales revenue (for narrow definitions of the sector) to value added (when the sector encompasses the entire economy).

BLS has said¹² that for most service industries, and in particular the retail-related service industries, sectoral output nearly equals the gross sales or revenue of the industry, because intra-industry transfers are tiny or even nonexistent. Hence, for purposes of measuring the output of retail-related industries, sectoral output is approximately equal to gross sales but is more complex to compute. Hence, gross sales provides a reasonable simple approximation to nominal output for the service industries. However, the observation that intra-industry transfers are tiny for the service industries may be a reflection of data gaps.

Conceptually, a sales revenue measure of retail output uses the retail sales price of a product as the measure. That price reflects the entire chain of processes that goes into the product, from its initial design to the raw materials to the manufacture to the multiple steps involved in providing the product for sale and delivering it to the final customer. In other words, it includes the contributions of the entire value chain in the production and distribution of the good, not just the value added by the retailer. In contrast, a value-added measure of retail output focuses on the portion of the product’s value directly provided by the retailer, subtracting the wholesale cost of the product and any other purchased inputs that the retailer does not provide. Thus, the value-added measure of output directly isolates the portion of the value chain that is produced by the retail firm’s own labor and capital. Between these two concepts, the gross margin measure removes the wholesale cost of the product, which reflects the value related to its design and manufacture, but includes the value added by other factors besides the retailer’s own labor and capital, such as the value provided by leasing a store or paid to another vendor who handles customer service.

Strictly speaking, the value-added measure of output is the one that is associated with the services provided by the retailer that derive from the

___________________

¹² Email from Jenny Rudd, BLS October 21, 2020: “In most cases the sectoral output of a service industry nearly equals the gross output of the industry. Because intra-industry transfers are tiny or even non-existent, the values for the two output concepts overlap. We have looked into the possibility of adding intra-industry adjustments to service industries. The problem is that the data to do so from the input-output (IO) tables are generally not at a detailed enough level.”

capital used and the activities of the retailer’s employees who provide the labor input used to calculate labor productivity. Of particular importance, the value-added measure of output is the only measure that makes the adjustments necessary to compare the labor productivities of large national retail chains and small mom-and-pop stores on an equal footing, in each case removing the contributions to total sales revenue that are provided by the workers at other firms. However, not all analysts are persuaded that the value-added measure is superior; some analysts are concerned that a value-added measure of output can be distorted by monopoly pricing (e.g., Walter Oi, cited by Manser, 2005). In fact, all three measures are potentially influenced by varying markups. Value-added measures are in a sense residual measures, reflecting the effect of business cycles, shifts in demand, and input cost changes, and this can make them quite volatile.

Despite the conceptual differences across these three output measures, they are clearly related, and under some conditions they will produce similar measures of labor productivity change. Specifically, in cases where the contributions of the different inputs are fixed, the three output measures

will move proportionally. For example, if the cost of the goods represents one-third of the sales price, and the cost of the store lease and other purchased services represents one-third of the sales price, then the gross margin will be two-thirds of total sales revenue and value-added will be one-third of total sales revenue. In this simple fixed case, the three different output measures will all increase by the same percentage. As a result, if the industry is relatively stable, all three measures will show roughly the same change.

However, if the industry is experiencing change, with different parts of the value chain growing more or less quickly than others, then the three different output measures are likely to show different percentage changes. It would be reasonable to expect the retail sector to show such differences at this time, given the kinds of transformation discussed in the previous chapter. And indeed, Figure 2-1 in that chapter shows that the three different measures of retail output produce three different estimates of labor productivity growth for the 1997-2018 period. That implies three different estimates of output growth, since all three estimates use the same measure of the change in labor input. Figure 2-1 also shows that sales revenue for the retail sector overall grew faster than gross margin, which in turn grew faster than value-added. Given the relationships between these measures, these inequalities imply that the cost of goods sold grew faster than the gross margin, and that the contribution of other purchased factors grew faster than the value added by retailers’ own labor and capital. These different growth rates in the other sectors related to retail—particularly those that produce other services purchased by retail firms—provide some hints of the restructuring occurring.¹³

CONCLUSION 3-6: The nominal output of the retail and related sectors is measured in several different ways in the federal statistical system. A sales revenue measure of output is the simplest to produce but does not reflect changes in a retailer’s cost structure when additional functions—like warehousing—are integrated into the business. It does not focus on the services the retail sector provides, either. A value-added measure of output is theoretically preferred for measuring labor productivity in retail, capturing the difference between gross output and intermediate inputs, but there are limits in the ability to obtain the data needed to produce value-added measures. A gross margin measure of output reflects the value of the most important input for a retailer—the cost of goods sold—while sidestepping problems related to estimating other inputs. Because the extra data on purchases and

___________________

¹³ This paragraph slightly oversimplifies the comparison, since the labor productivity comparisons in Figure 2-1 show changes in real—not nominal—labor productivity, so differences in both nominal output measures and price deflation will affect the comparison.

other inputs are not published for as many detailed North American Industry Classification System codes as for sales, gross margin and value-added measures are available for fewer detailed retail industries. For retail-supporting services that might be combined with retail trade in a broader retail-related sector, similar choices would need to be made concerning which measure of nominal output to use, and those choices would entail tradeoffs between simplicity of data and conceptual focus. When the retail sector experiences significant change, the different output measures will give different pictures of labor productivity growth, depending on the extent to which the change is occurring for the retail services themselves, the various retail-supporting services provided by other suppliers, or the products provided for sale.

Federal Data and Issues for Measuring Output

The following sections discuss the statistical programs of the U.S. Census Bureau that collect retail-related data for measuring output (sales, revenue, or value of shipments; purchases; detailed expenses; and transfers between establishments) including the Annual, Quarterly, and Monthly Economic Surveys, the Economic Census, and other every-5-year collections.

The Census Bureau, like other statistical agencies, collects information in a time sequence ranging from simple statistics published frequently to more detailed statistics published with longer delays. The quality of the early estimates is lower, because samples are smaller and respondent-provided data may consist of estimates made by businesses. There is typically more detail provided with later estimates, because they have larger sample sizes, and there is more time to clean the data. This time sequence and the use of benchmarking mean that there may be very long delays before final data are available. For example, the detailed data from the Annual Retail Trade Survey (ARTS) for 2018 were released in February 2020 and were benchmarked to the 2012 Economic Census (because 2017 Economic Census data were not yet available.)

The Census Bureau’s Monthly, Quarterly, and Annual Economic Surveys

The Monthly Retail Trade Survey and Advance Monthly Retail Trade Survey collect sales data from a sample of retail firms¹⁴ that report for their retail establishments. Data from the former are published within 50 days of the close of the reference month, while data from the latter are published

___________________

¹⁴ We use the term “firm” to distinguish a group of establishments within an enterprise, all of which are either classified in (say) retail trade or classified as supporting establishments for retail trade. An enterprise may contain many such firms.

within 20 days. Both surveys provide estimates with less NAICS detail than ARTS provides.

The Monthly Wholesale Trade Survey collects data from a sample of U.S. merchant wholesalers (excluding manufacturers’ sales branches and offices) on monthly sales, end-of-month inventories, and number of enterprises reported for. Data are released about 40 days after the close of the reference month and are provided with less NAICS detail than for the Annual Wholesale Trade Survey (AWTS).

The Quarterly Services Survey collects total sales, receipts, revenue, and total operating expenses from a sample of firms with establishments in selected services industries. Estimates are released about 50 days after the close of the reference quarter, and the data are provided with less NAICS detail than for the Services Annual Survey (SAS).

The surveys summarized in Box 3-5 collect economic detail for retail and retail-related industries in the three annual surveys mentioned above: ARTS (retail), AWTS (wholesale), and SAS (services). ARTS provides data on nominal gross sales, purchases, and gross margins for establishments classified in the retail sector. AWTS provides nominal gross sales, operating expenses, gross margins, and purchases for establishments classified in the wholesale sector.¹⁵ And SAS provides revenue and total expenses for establishments classified in the services sector, which includes transportation and warehousing. Except as noted, for all three surveys data are provided at the 4-digit NAICS level, with some 5- and 6-digit detail. The exceptions are that ARTS provides estimates of purchases, gross margins, and total expenses only at the 3-digit NAICS level, with some 4- and 5-digit NAICS level detail; for transportation industries in SAS, 3-digit NAICS level data are also provided for detailed expenses and commodity-level revenue.

For the margin industries (retail and wholesale), there is a data gap in integrating the Economic Census data with the annual surveys. In the annual surveys, gross margins are measured at the industry level of detail (mostly 3-digit). In the Economic Census data, gross revenue is collected at a detailed product-group level, but no information is collected on gross margins. This implies that gross margins at the product-group level are never directly measured but only inferred by combining this disparate

___________________

¹⁵ To create the sampling frame for the Monthly Retail Trade Survey and ARTS (same approach used for the Monthly Wholesale Trade Survey and AWTS) all employer establishments located in the United States and classified in the retail trade and accommodation and foot-services sectors are sorted by EINs or firm identifiers. The establishment data for the EIN/firm (potentially only part of an enterprise) are aggregated to become potential sampling units. The sample is selected by a stratified design and selected firms/EINs are asked to report for the aggregate of their retail establishments and retail auxiliaries. Thus, the survey data statistically represent retail establishments. See https://www.census.gov/retail/mrts/how_surveys_are_collected.html.

information measured at different frequencies. This is a potential source of measurement error, including mismatches between the price deflators at the product-group level and the measured nominal gross margins.

The sampling units for ARTS, AWTS, and SAS are enterprises that are asked to report for the aggregates of their retail, wholesale, or services establishments (respectively) plus the auxiliaries that support those industries. The only exception is that enterprises reporting on ARTS are asked to exclude auxiliaries from their retail sales aggregates. The enterprises are asked to break out the industry detail if, for example, an enterprise with retail activity has establishments in multiple retail industries. If enterprises do not provide this detail, their allocation to an industry category is based on administrative data and the Economic Censuses.

The Census Bureau’s Economic Census and Related Surveys

As illustrated in Box 3-6, the Economic Census provides measures of gross sales, payroll, first quarter payroll, and number of employees every 5 years. Data are collected at the establishment level and are available at the U.S. level with 6-digit NAICS code detail as well as at the product code level (the Business Expense Supplement to ARTS and AWTS provides detailed expense data at the enterprise level every 5 years). The Commodity Flow Survey, sponsored by the Bureau of Transportation Statistics and collected by the Census Bureau, is also conducted every 5 years to describe domestic freight shipments by establishments in the mining, manufacturing, wholesale, auxiliaries, and selected retail and services trade industries located in the 50 states and the District of Columbia. The Commodity Flow Survey is of potential importance to this project because data reported on a transported commodity may indicate whether the shipment is for retail (e.g., TVs or clothing) or wholesale (e.g., jet engines or elevator assemblies).

Differences in the unit of observation and the range of data collected in ARTS, relative to that collected in the Census for Retail Trade (CRT), create limitations in estimating gross margins at detailed industry levels at an annual frequency. The data in ARTS are collected at the enterprise level, which limits the level of detail directly available from the survey data (although the Census Bureau requests large multi-units in multiple industries within retail to break out industry detail). Given these limitations, the data released from ARTS combines information directly from the survey data with adjustments from the Economic Census.¹⁶ This is an imperfect process in a number of ways. For example, the release of the 2018 Annual Retail Trade data in February 2020 uses adjustments from the 2012 Economic Census, because the 2017 Economic Census tabulations were not yet

___________________

¹⁶ See https://www.census.gov/data/tables/2018/econ/arts/annual-report.html.

available for these tabulations.¹⁷ In addition, while ARTS collects information on gross margins, the Census for Retail Trade does not, which makes combining information from that census and ARTS more complicated. A

___________________

¹⁷ This discussion is related to the construction of the input-output accounts by BEA. The 2012 Economic Census is the most recent Economic Census data in the BEA input-output accounts used to produce 2020 statistics on economic activity.

related problem is the computation of margin prices from the PPI to match the gross margin measure of output by industry. The changing product mix within different types of retailers is captured only once every 5 years via the Census for Retail Trade. Margin prices on different products and by outlet type vary, but the changing product mix and outlet type are not well captured in the annual, quarterly, and monthly surveys of retail trade activity.

Auxiliaries

The Economic Census collects information on auxiliaries, also called enterprise support establishments, for six industries in the services sector: NAICS 48-49 (Transportation and Warehousing), NAICS 51 (Information), NAICS 54 (Professional, Scientific and Technical), NAICS 55 (Management of Companies or Enterprises), NAICS 56 (Administration and Support and Waste Management and Remediation), and NAICS 81 (other services except public administration). Information about auxiliaries in these six industries was collected in 2012 and 2017. Included are data by 3-digit sector served, number of establishments, number of employees, payroll, and external sales or revenue receipts. In the Economic Census an auxiliary is tabulated in two places: under its own establishment NAICS code and in the tabulation of auxiliaries.¹⁸

The Economic Census auxiliary questionnaire is sent to establishments that are marked as auxiliaries in the Business Register. New auxiliaries are so designated based on analyst research and other information. The questionnaire obtains information for each auxiliary establishment about the main industry that it serves, classified according to 3-digit NAICS code. Using this classification, Fort and Klimek (2018) determined that 20 percent of the payroll related to the retail industry in 1997 was represented by employment in nonretail auxiliary establishments that primarily served the retail sector.¹⁹ Most of this auxiliary payroll—15 percentage points—relates to management (NAICS 551114). The other significant industries represented in auxiliaries are Warehousing and Storage (NAICS 493, 2% of payroll), Truck Transportation (NAICS 484, 1%), Accounting Services (NAICS 5412, 1%), and Unclassified (1%).

BEA includes the Census Bureau’s information about auxiliaries in its national accounts and other products. Because an auxiliary serves its

___________________

¹⁸ Information on Census Bureau data on auxiliaries provided by Edward Watkins at the workshop.

¹⁹ When calculated using employment rather than payroll, the auxiliary portion represents only 7 percent of employment related to the retail industry in 1997. This auxiliary portion of employment is substantially lower than the auxiliary portion of payroll because employees in the establishments related to firm management are compensated at a higher rate than those in many other establishments.

enterprise, it does not report output data related to its support functions on the census. Any profits or earnings that accrue because of the contribution of auxiliaries to their enterprise are not captured in current data. BEA measures the service outputs of auxiliaries using data on expenses.²⁰

Table 3-2 shows the Census 2012 data on auxiliaries in the six sectors that report that they serve the retail trade (by 3-digit NAICS code). The table shows that in 2012, 73 percent of the auxiliaries serving retail trade were in Administration and Support and Waste Management and Remediation, and 23 percent were in Transportation.

Those auxiliaries reporting that they serve a retail firm seem to be exactly the type of establishments that should be included in a retail-related satellite account, regardless of where they are classified. However, data on auxiliaries or support establishments are currently collected and available through the Economic Census only for establishments classified in six industries. Other information about such service establishments may be available in the microdata available at the Census Bureau.

The panel proposes that a study should be undertaken, using the Census Bureau’s microdata²¹ that include firm IDs as well as establishment data, to gain insights into how enterprises are structured and how support establishments might be identified. One could imagine a new public domain data product that uses this information to quantify support services. Additionally, the study could inform further work on improving the collection of data on auxiliaries and possibly improving information to estimate the value they provide to their enterprises.

CONCLUSION 3-7: Data available from the Economic Census and the Economic Surveys for the retail trade-related industries limit the ability to estimate output for retail-related industries in important ways:

• Purchase data are needed to compute gross margins, but the only purchase data for retail are collected on the Annual Retail Trade Survey (ARTS), not the Economic Census. As a result, purchase data are not available at the establishment level for retail establishments, so benchmarking to the Economic Census requires assumptions that likely affect the quality of estimated gross margins.
• ARTS does not have the same product detail for sales as in the retail trade component of the Economic Census, and it does not request any industry breakdown of activity. However, such detailed data are needed to accurately and separately allocate

___________________

²⁰ Reported by Jon Samuels during the workshop.

²¹ Through a Federal Statistical Research Data Center.

TABLE 3-2 Number of Auxiliary Establishments That Supported Retail Trade, 2012 Economic Census

NOTES: Data on auxiliaries in 2017 Economic Census not available until September 2021.

• sales and purchases to codes; their absence may affect the quality of estimated gross margins.
• Data on operating expenses are needed to compute value added. Operating expenses for retail and wholesale trade establishments are collected as an aggregate of an enterprise’s establishments on ARTS²² and Annual Wholesale Trade Survey once every 5 years during Economic Census years. Data on expenses are not collected at the establishment level in the Economic Census.
• Auxiliaries are a key concept for quantifying the impact of vertical integration in a retail-related satellite account. Though some data are available from the Economic Census, there are limited ways to estimate the value an auxiliary establishment provides to its enterprise, and the Bureau of Labor Statistics currently has limited information to designate auxiliaries that support retail.

ADJUSTING NOMINAL OUTPUT FOR CHANGES IN PRICES

After obtaining a measure of the nominal output of the retail sector, that estimate must be adjusted to remove the effect of price changes to identify the real changes in the output of the sector. This step is crucial for determining productivity, because nominal output figures can be strongly affected by inflation or deflation, particularly in a sector undergoing rapid change.

The price adjustment required will depend on the type of nominal output measure used: a sales revenue measure of retail output is deflated using the CPI. A gross margin measure of retail output is deflated using the PPI for the retail margin. A value-added measure of retail output is deflated by using the PPI for the retail margin component and other PPI indices for the other inputs. Similarly, the price adjustment for the various retail-supporting industries, such as warehousing and transportation, uses the PPI indices for those industries for a gross output measure and the analogous PPI indices for the key inputs for a value-added output measure.

This section addresses both quality adjustment, which is implemented in current U.S. price indices, and the problem of outlet substitution bias, which has been discussed in the research literature but is not reflected in U.S. economic statistics. It then provides a brief overview of the CPI and PPI indices.²³

___________________

²² See https://www.census.gov/programs-surveys/economic-census/data/bes.html.

²³ This section draws on workshop presentations by Ana Aizcorbe on the conceptual issues related to retail price indices, Brendan Williams on the CPI, and Bonnie Murphy on the PPI.

Quality Adjustment in Price Indices

Adjusting for quality is a key issue in developing price indices because potential quality differences across similar products or services make it difficult to know whether a price difference reflects a difference in the price level (indicating inflation) or a difference in quality. In the context of retail trade, “quality” refers specifically to the quality of the retail services themselves—not the quality of the products sold by retailers—and relates to the kinds of shifts the retail sector has experienced over the past few decades. As discussed in the previous chapter, the recent changes in retail have introduced different kinds of retail outlets—including warehouse stores, e-commerce, and large retailers—that provide greater product variety and different ways of obtaining and learning about products. These changes reflect changes in the quality of the services that the sector provides, and adjusting for them is necessary to determine the real output of retail services. If a gross margin or value-added measure of output is used, the quality of retail services will be reflected in a price index for the retail margin, like the PPI. If sales revenue is used to represent retail output, the quality of retail services will be reflected as one part of a price index for the product’s overall sales price, like the CPI. Both types of price index are addressed here. The CPI is discussed first—because it is currently used by BLS and provides a more concrete example of the underlying concepts—but the PPI is more directly related to quality adjustment of retail services.

The simplest way to account for potential quality differences is to try to eliminate them by focusing price comparisons on products or services that are identical. When feasible, this strategy eliminates the confounding of price and quality because quality has been fixed.

The strategy of focusing on price changes for identical products is the starting point for the price comparisons used to construct the CPI itself, which collects price information across a broad range of consumer goods. A sample of products with specific characteristics is selected at a specific retail outlet, and the prices of those sampled products are collected monthly for a 4-year period and then rotated. To the extent possible, the exact same items are sampled over the entire period. If a selected item becomes unavailable, it is replaced with a comparable item, to the extent possible.

A similar approach to control for quality is used for the price information collected for the PPI for retail, which looks at changes in the average margin price (the average difference between the sales revenue and cost of goods per product) within “comparable product lines” of “related products that are distributed under a similar set of conditions.”²⁴ The standard

___________________

²⁴ The quotes in this paragraph come from Bonnie Murphy’s presentation at the workshop describing the guidelines for the PPI for retail.

guidelines for identifying the products that will be considered together call for products that are “classified within a single product category,” “displayed and/or marketed in a similar manner,” and “located in the same area or department of the store.” By controlling these different factors that can affect the nature of the retail services offered, the PPI for retail aims to provide margin price comparisons that essentially hold constant the quality of the retail services provided.

Of course, there are cases when it is not possible to keep quality fixed and it is necessary to explicitly adjust for quality differences when comparing prices. These problems are not new conceptually: there are well-developed techniques—specifically, hedonic price indices²⁵—that derive the price differences associated with different product features related to quality.

When it is not possible to find an identical or comparable item for a sampled product, BLS generally either imputes the prices of the missing items from other available observations in the sample or uses hedonic methods to estimate the previous period price for a replacement product.²⁶ In two important cases—computers and motor vehicles—the price adjustment is carried out using information on wholesale component costs and markups to derive a comparable price for the now-unavailable sampled product from the price of a similar but noncomparable product that is available.

Explicit quality adjustments using hedonic price techniques are not a standard feature of the PPI program that collects average margin prices for retail, although there was a program that developed a hedonic margin price model to adjust for the quality of retail services at beer, wine, and liquor stores.²⁷ Over a 12-year period, the program collected data related to the retail services of individual stores, including square footage, number of checkouts, number of full-time employees, number of products carried, whether product testing was conducted, whether classes were offered, and whether local deliveries were offered. This information was used to develop a model that related changes in margin prices to these characteristics. Unfortunately, the model did not show a meaningful relationship between these indicators of the quality of retail services and the change in the margin prices. Despite this failure to use hedonic price techniques to explain retail service quality changes within individual retail outlets, hedonic techniques have significantly improved the measures of important product

___________________

²⁵ Hedonic price indices analyze price changes for changing consumer products by estimating prices associated with each product’s different characteristics. The price changes for the different characteristics can then be used to estimate the overall change in price for the product while controlling for the shifting characteristics.

²⁶Aghion et al. (2019) argue that using prices from available observations is likely to overstate the price of missing (discontinued) items and substantially bias the CPI.

²⁷ This program was described by Bonnie Murphy in her workshop presentation.

classes—such as computers (Berndt, Griliches, and Rappaport, 1995)—and they could be important in understanding the differences in the quality of retail services across retail outlets. However, that shift raises the challenge of estimating price indices when there are shifts in consumption across retail outlets, which is addressed in the next section.

Outlet Substitution Bias

As noted above, the techniques for estimating price indices for both retail products and services—CPI and PPI, respectively—use samples of product or margin prices for individual outlets that are combined using fixed weights across outlets. However, one important aspect of the economy is the ability of consumers to move from one supplier to another, and the dynamic nature of the retail sector suggests that this type of change is clearly important in retail. When consumers shift their purchases in this way, the resulting changes in aggregate price indices for the retail sector are particularly important to reflect. The bias introduced in the price index when this shift is ignored is known in the research literature as “outlet substitution bias” (Reinsdorf, 1993).²⁸

Outlet substitution bias was originally studied in the context of the earlier wave of retail transformation involving the rise of warehouse clubs and supercenters, which was discussed in the preceding chapter. These discount stores often sell the same products as more expensive stores but at generally lower prices. The effect of these stores on price level comes from the opportunity for consumers to switch from a regular store to a discount store to buy the same products for less. The major effect on average prices comes from the “outlet substitution”—consumers moving from one retail outlet to another—and not from the smaller price changes occurring at the discount stores themselves. To the extent that e-commerce products are cheaper than their in-store counterparts, e-commerce would pose a similar outlet substitution effect on the average price level.

Outlet substitution poses a conceptual challenge for the price indices currently collected by BLS, which focuses on price changes for similar goods and services at individual retail outlets. These BLS price indices weight the price changes from different outlets by their relative share at different outlets, but the change in relative share across outlets caused by consumers switching from one outlet to another is not reflected in the price indices. This is true for both the CPI used for sales prices and the PPI used for retail margin prices. As a result, neither the CPI nor the PPI can describe how prices change—either for the overall product or for the margin price of the

___________________

²⁸ This section draws on the workshop presentation by Ana Aizcorbe on the conceptual issues related to retail price indices.

retail services themselves—when consumers systematically move from regular stores to discount stores or from discount stores to e-commerce, except when consumers move between outlets classified in different NAICS codes.

This problem of outlet substitution bias poses a direct conceptual challenge to understanding the labor productivity effects of the transformation in retail involving warehouse clubs, supercenters, and e-commerce. If the potential price decreases from outlet substitution are not reflected in the price indices, then the estimated price increases will be too large, which will in turn make the estimates of increases in real output too low. Since price indices fail to correct for outlet substitution, they effectively miss the important productivity effects of the recent transformation in retail.

One solution to the problem of outlet substitution bias involves the use of a different kind of price index—a “unit value index”—that specifically looks at the change in the average price of a product or service over time (Nakamura et al., 2015).²⁹ This price index explicitly reflects changes in the distribution of sales across different outlets, allowing it to capture the effect of consumers moving to less expensive stores. To do this, however, a unit value price index requires information about the quantity of sales of each sampled product at each outlet at each time period, in addition to information about prices.

Transaction data from retailers could be used as a way of providing information about both prices and quantities of sales. (See the discussion below about private sector data sources.) The available transaction data are incomplete, with data from aggregators such as Nielsen, IRI, NPD, Affinity, and Palantir including aggregations from both scanners and credit cards but often missing key types of outlets, such as the warehouse clubs, supercenters, and e-commerce outlets of particular interest.³⁰

A new data collection effort by the PPI division of BLS is collecting more detailed margin price data directly from large wholesale trade companies that would provide transaction-level data on a monthly basis electronically.³¹ If this approach proves workable, it could provide a model for expanding data collection in the retail sector in a way that is easier for large

___________________

²⁹ Another solution to the problem of outlet substitution bias would involve estimating the consumer utility related to the different services that retailers offer. This potential approach would have the benefit of also providing an analysis of the consumer benefits resulting from the increased variety offered by retailers. Techniques that have been developed to analyze the consumer utility from consumer goods (e.g., Diewert and Feenstra, 2019; Feenstra, 1994; and Redding and Weinstein, 2016) could be used to do this. The panel thanks an anonymous reviewer for pointing this out.

³⁰ This overview of the coverage of private data sources was provided in Ana Aizcorbe’s presentation at the workshop.

³¹ This new data collection effort was described in Bonnie Murphy’s presentation at the workshop.

companies to provide and with additional detail for price index estimation. However, the more limited coverage of this effort—likely limited to large companies that routinely make extensive use of electronic data systems—would limit its potential use for broader price indices like the CPI and PPI, which sample from all types of outlets.

The lack of the necessary quantity of information in regular reporting raises the question of understanding the size of the bias caused by outlet substitution.³² This in turn raises the question of quality adjustment, which was discussed in the previous section. In constructing the CPI and PPI margin price indices, quality needs to be controlled or adjusted within an individual retail outlet for the products or services sampled in that outlet. However, to construct a unit value index across retail outlets, it is necessary to control or adjust quality across products or services across the full range of retail outlets, which could raise substantial comparability problems.

The difficulties of adjusting for quality raise the pragmatic question of the extent to which the size of the bias from outlet substitution could be bounded by an approach that did not attempt to adjust for quality differences. Most studies agree that outlet substitution bias generally causes price increases to be overestimated (ILO/IMF/OECD/UNECE/Eurostat/The World Bank, 2004), although Nakamura and colleagues (2015) show that the bias can go in either direction, in principle. Moulton (2018) estimates that the outlet substitution effect caused by the introduction of warehouse clubs and supercenters in the 1990s led to an overestimate of price increases by only 0.08 percentage points per year. The effect of outlet substitution bias from the growth of nonstore retailers in recent years is likely to be much smaller (Hatzius, 2017), given their relatively small share of retail sales. In any case, it is important to remember that outlet substitution bias is a problem only during the transition itself, when market shares are changing across outlets. Of course, there is likely to be the most interest in productivity statistics for the retail sector precisely when market shares are changing across outlets.

Federal Price Deflators

BLS produces the primary price deflators that are used to estimate real output from the nominal measures of output derived from Census data.

The PPI is a family of indexes that measure the average change over time in selling prices received by domestic producers of goods and services. Most useful in measuring productivity is the set of indices that uses NAICS classifications, measuring changes in prices received for an industry’s output

___________________

³² This discussion of the problem of estimating the bias caused by outlet substitution was provided by Ana Aizcorbe’s presentation at the workshop.

sold outside the industry (that is, its net output). The prices included in the PPI are from the first commercial transaction for many products and some services. The PPI is the primary deflator source for manufacturers’ sales and branch offices’ sales, for truck transportation, couriers, and messengers, and for warehousing. For retail and wholesale trade, the PPI measures average margin prices for narrow product groups, calculated as the difference between the sales price and the cost of goods sold. Calculation of the PPI first captures the changes in the average margin prices for individual outlets, and then weights those individual-outlet changes according to the share of each outlet. For the PPI retail and wholesale margins, the outlet weights are based on the margin revenue for each outlet. These outlet weights are updated every 5 years using margin revenue data from the Census Bureau.³³

The CPI is a measure of the average change over time in the prices paid by urban consumers for a market basket of consumer goods and services. Indexes are available for major groups of consumer expenditures (food and beverages, housing, apparel, transportation, medical care, recreation, education and communications, and other goods and services), for items within each group, and for special categories, such as services. The CPI-U-RS is the primary deflator source for retail trade industry sales used by the BLS industry program to obtain its labor productivity measures. Price surveys capture the changes in price of a particular item at a particular store, and as such do not capture price changes when a new retail outlet enters, nor when a service is added to the sale. The weights used to sample outlets and choose the number of price quotes for each outlet are determined by the reported expenditure shares for each item category at each outlet. These weights are updated when the outlet sample is revised in a staggered rotation of eight sampled cohorts, each lasting 4 years with one cohort being revised every 6 months. However, any implicit shift across outlets that occurs when a sample is revised to reflect current sales is not reflected in average prices, because price changes are based on only the matched model that compares current and previous period prices of the same item at the same outlet.³⁴

In addition to these BLS price deflators, annual data from BEA’s National Income and Product Accounts on implicit price deflators for manufacturing and trade sales are used to deflate merchant wholesale sales. An implicit price deflator is the ratio of the current-dollar value of a series, such as gross domestic product (GDP), to its corresponding chained-dollar value, multiplied by 100.

CONCLUSION 3-8: Although the existing price indices provide a way of describing price changes that occur for services and products

___________________

³³ Bonnie Murphy, BLS, personal communication, October 28, 2020

³⁴ Brendan Williams, BLS, personal communication, October 27, 2020.

provided by individual retail outlets, they do not typically capture the aggregate price changes that result when consumers move from one type of retail outlet to another. For example, the price indices do not reflect the change in the price and quality of retail services as consumers move from a traditional department store to a warehouse store or to e-commerce, except when consumers move between outlets classified in different North American Industry Classification System codes.

CONCLUSION 3-9: The price deflator for retail-sector industries should relate to the change in services the sector provides and changes in the prices and quality of those services. This differs from price adjustment related to the products the retailer sells, which is focused on the characteristics of the goods themselves. Price deflation in the retail sector needs to consider, for example, the shifts in services in moving from a traditional department store to a warehouse store to e-commerce, which involves changes related to such things as product variety and the process for identifying and obtaining goods.

MEASURING INPUT

Compared to the difficulties involved in measuring nominal output and prices, the conceptual issues related to measuring employment are more straightforward. There are two primary factors to address, one related to the quantity of labor input and the other related to its quality. Although it is convenient to refer to “employment” as the denominator of labor productivity, the correct concept is actually quality-adjusted hours of labor, reflecting the fact that different workers provide different numbers of hours of work (because of differences in standard hours, overtime, and paid time off) and labor of different quality (because of varying skill levels). During periods of increasing labor quality, labor inputs that are not quality-adjusted could be understated and could lead to overstatements of changes in labor productivity. Alternatively, advanced technology, such as automation or the use of scanner technology, may substitute for more skilled workers in some components of retail trade, so that there is declining labor quality.

Obtaining hours of labor requires making adjustments to convert information on employment into hours worked. These adjustments need to reflect data related to standard hours, overtime, and paid time off, in addition to information related to part-time employees. Box 3-7 describes BLS employment data.

Adjusting for labor of differing quality could be done in a variety of ways, using different empirical approaches to account for different skill levels. In practice, however, only crude measures of skill are available across the entire labor force—primarily the proxies of education, age, and

compensation.³⁵ As noted above, BLS measures labor quality for the multifactor productivity program using data on average wage rates for different groups of workers defined by differences in educational attainment, age, and gender. The shift in the labor force distribution across these different categories, resulting in a shift in the weighted average wage rate across the labor force, is then used to adjust the number of hours worked by the change in quality.

One challenge in implementing a broader retail satellite account is likely to be in allocating employment (or hours worked) into retail-related and nonretail-related for some NAICS codes. It is likely that this will require new data.

A final issue regarding the measurement of employment concerns properly accounting for outsourcing, that is, an establishment’s use of workers from temporary agencies or a professional employer organization that has its own NAICS code. Professional employer organizations are under NAICS 56–Administrative and Support—one of the codes related to auxiliary establishments. Outsourcing makes it difficult to link the workforce to the sector in which the work is being done. Outsourcing has become common for some firms. For example, there are some Walmart distribution centers that are in-house warehouses with all workers outsourced. Although some data are available through the Census Bureau, this is another potential data gap that will require investigation.

Federal Data and Issues

Employment Statistics

The primary federal statistics for input (employment or hours worked) are collected by the BLS on an establishment basis through its Current Employment Statistics Survey and the National Compensation Survey (see Box 3-7). The Census Bureau collects data on employment and payroll (not hours worked) by establishment through its annual Company Organization Survey (see Box 3-8) and through the Economic Census.

CONCLUSION 3-10: While hours worked is considered to be the appropriate measure of input for measuring labor productivity, it is improved when work hours are adjusted to reflect the quality of work provided by workers with different skill sets. Current Bureau of Labor Statistics’ approaches adjust for worker quality by looking at pay

___________________

³⁵ New research by Acemoglu and Autor (2011), Autor (2013), and Acemoglu and Restrepo (2018) suggests that task-based measures of job-specific skill may be more useful measures of labor quality, which would require additional data sources.

differences across groups of workers defined by difference in educational attainment, age, and gender. However, the retail transformation is bringing substantial changes to the workforce, with large increases in workers with high-end programming and data analysis skills that support e-commerce. New research in labor economics is investigating ways to measure the skill shifts related to such changes by looking at changes in the tasks involved rather than the educational attainment, age, and gender of the workforce.

ADDITIONAL DATA SOURCES

In addition to the primary data sources described previously, federal data sources also include secondary data products: estimates prepared by federal agencies to illuminate economic concepts such as productivity and the national accounts. Two of these, the BEA Industry Accounts and the BLS BEA Integrated Industry-Level Production Account, are summarized in the first two sections below. The third section describes private and other non-federal data sources, including commercially produced data for purchase, data derived from web-scraping, and credit card data.

BEA Industry Accounts

BEA’s industry economic accounts, which are presented both in an input-output framework and as annual output by industry, provide a detailed view of the interrelationships between U.S. producers and

users and the contribution made to production across industries.³⁶ Data products include

• GDP by industry, which measures industries’ performance and their contributions to GDP;
• gross output by industry, principally a measure of sales or revenue from production for most industries, although it is measured as sales or revenue less cost of goods sold for margin industries like wholesale and retail;
• input-output accounts, a data set showing how industries interact with each other and with the rest of the economy; and
• employment by industry, which measures the nation’s number of full- and part-time workers as well as the self-employed.

BEA provides further detail about its input-output accounts:³⁷Supply tables show the total value of goods and services available in the domestic economy, including those produced by foreign as well as domestic industries. Use tables show how the supply of goods and services is used, including domestic purchases by industries, individuals, and government and exports to foreign purchasers. Requirements tables summarize the full supply chain by showing how production relies on both direct and indirect inputs. For example, flour is a direct input for a baker, while wheat (used in the production of flour) is an indirect input for the same baker. The requirements tables can be used to analyze the economic repercussions of a natural disaster or other event that changes spending patterns.

Of special interest to this study are the input-output tables, released in November of each year. The input-output accounts are represented in detailed tables showing how industries interact with each other and the rest of the economy. The input-output data, which provide information on 71 industry categories (including 4 in retail, 1 in wholesale, 7 in transportation, and 1 in warehousing), are updated each year. Detailed benchmark input-output statistics, which are further subdivided into 405 industries, are produced roughly every 5 years.

BLS-BEA Integrated Industry-Level Production Account (KLEMS)

During the panel’s workshop, Jon Samuels (BEA) told the panel that the new BLS-BEA Integrated Industry-Level Production Accounts (ILPAs)³⁸

___________________

³⁶ See https://www.bea.gov/data/economic-accounts/industry.

³⁷ See https://www.bea.gov/resources/learning-center/what-to-know-industries.

³⁸ A reference for the official BEA-BLS ILPA work is here: https://www.nber.org/papers/w22453.pdf. BEA-BLS have done research work to extend the account back to 1947. See https://www.sciencedirect.com/science/article/pii/B9780128175965000111.

were designed with the intention of capturing innovation and the importance of the trade sectors in aggregate productivity growth (Eldridge et al., 2020; Jorgenson, Ho, and Samuels, 2016). These productivity measures use as the numerator the BEA output measures (gross output and value added), which are consistent with GDP. They make use of the KLEMS approach to compute the input measures. Samuels cited work by Jorgenson, Ho, and Samuels (2016) as describing the approach that has been incorporated into the new BLS-BEA account, but noted that the paper only covers 1987 forward. At a minimum, these accounts may provide information useful to this project, with any proposed satellite account building on this prior joint work, rather than duplicating it.

As described by BEA,³⁹

Growth accounting (Jorgenson and Griliches, 1967) provides a method for using the estimates in the integrated industry-level production account to estimate how factors of production contribute to aggregate economic growth. Gollop, Fraumeni, and Jorgenson (1987) showed how to do this at the industry level, and this account uses that basic approach.

The integrated industry-level production account decomposes growth in industry gross output into contributions from growth in intermediate inputs, capital, labor, and multifactor productivity. Similarly, the account decomposes growth in aggregate economy value added into the separate contributions from industries’ growth in capital, labor, and multifactor productivity. Data on gross output and intermediate inputs by industry are drawn from BEA statistics on GDP by industry, while data on capital and labor inputs come primarily from the BLS productivity program. Total capital and labor compensation by industry are controlled to match estimates of value added by industry from BEA. Labor, capital, and intermediate inputs

___________________

³⁹ See https://www.bea.gov/resources/learning-center/what-to-know-industries.

⁴⁰ See https://apps.bea.gov/scb/2020/04-april/0420-integrated-industry-level-production.htm.

are adjusted to account for changes in composition over time. Growth in multifactor productivity is defined residually as the difference between industry output growth and the sum of the share-weighted growth in industry inputs of intermediates, capital, and labor.

Data are provided under the following headings: sector (21 NAICS codes), summary (71 codes), underlying summary (138 groups), and detailed (405 groups, but only available every 5 years). For retail, summary includes three 3-digit codes plus one aggregate for retail and six 3-digit codes for transportation. The underlying summary tables include 11 additional codes for wholesale, 6 additional 3-digit codes for retail, and 2 additional 3-digit codes for transportation. The most recent data release occurred on March 2, 2020.

During the workshop, Samuels noted that under the current NAICS classifications, retail and services are comingled; for example, 30 percent of Motor Vehicle Retailing is under services, as is 12 percent of Restaurant Food and Beverages. He noted that the total factor productivity accounts can be used to analyze these things, and can also be used to split the output of industries by commodity when there is joint production.

Private-Sector and Nonfederal Data

The Census Bureau’s Economic Surveys and Economic Census and BLS’s statistical collections, some of which were described earlier in this chapter, form the building blocks of the federal economic statistics program. Because of their limitations, however, particularly concerning timeliness and granularity, these surveys and collections are being augmented with private-sector and alternative data sources. Some of the most promising of these alternative sources are scanner data (available for purchase from the private sector), credit card transactions or bank data, and web scraping, each discussed below.

Private retailers and manufacturers have a long history of collecting consumer data, often for market research purposes. Proprietary data are collected, owned, and made available by commercial firms. Granularity is among the strengths of scanner data, and some data are available on a weekly basis. At the same time, these data are collected for marketing or other purposes, are not nationally representative, and are not well documented, and store coverage is not equal across all geographic areas.

Data originating from commercial and alternative sources provide information assets not available elsewhere. Attributes may include timeliness, granularity, geographic distribution, longitudinal information, and cross-time measures. At the same time, hurdles to the use of commercial and alternative data sources include access issues; bias in coverage and representation; perpetually dynamic algorithms; lack of documentation and transparency; fake

data and bots; limited scope of organic data sources; and privacy concerns. One of the challenges with the use of any outside data source is determining its quality and coverage, which are key to understanding how the data can best be used. (See NASEM [2020], pp. 76-79, for more detail on challenges.)

Scanner Data

Horrigan (2013) reported that BLS has explored the use of scanner data for many years. The most extensive use he reported was undertaken as comparative research between the CPI and the scanner and associated household panel data from Nielsen. There are two types of scanner data available: data that originate from retail establishments (retail scanner data, such as InfoScan, IRI Worldwide) and data that originate from consumers (household panel and scanner data, such as IRI Worldwide’s Consumer Network and Nielsen’s Homescan).

The following discussion of scanner data draws heavily on NASEM (2020, pp. 68-70). Retail “scanner data capture transactions for purchased products with a Universal Product Code (UPC) on their labels, as well as random-weight products such as fruits and vegetables” (NASEM, 2020, p. 68). The retail data include store information, including store name and corporate parent, address, and retail outlet type. “Granularity is among the strengths of scanner data” (NASEM, 2020, p. 69), some of which are available on a weekly basis. For each consumer store purchase, “scanner devices can detect and record exactly which products are purchased, the number of items, total dollars spent after discounts (if any), and total gross amount (before discount)” (NASEM, 2020, p. 68). With this information, “researchers can infer the average price paid as the ratio between dollars spent and units purchased, since many retailers do not share individual-level purchase prices with the data aggregators (Nielsen and IRI) but prefer to share average prices within a store or across geographic areas. This means the price data are not individual prices but are averages” (NASEM, 2020, p. 68).

“From the perspective of the firms IRI and Nielsen, store data are seen as a census. Whether or not this is accurate, their methods do not treat these sales data as a sample, and data available for purchase may include only those stores that have agreed to share their data. Infoscan, for example, does not include all large retailers (e.g., Costco is not included)” (NASEM, 2020, p. 68, footnote 32).

“The National Consumer Panel, a joint venture by Nielsen and IRI, is used by both these firms in their household panel data products. It comprises more than 120,000 households, which provide information on their demographic characteristics in addition to purchase information (NASEM, 2020, p. 70). “Unlike the retail scanner data collected at check-out, household

scanner data are collected using hand-held scanning devices provided to participating households or using a mobile cellphone app. In this way, purchases can be captured for the panel of households. Again, this source includes products with barcodes” (NASEM, 2020, p. 70).

The Economic Research Service (ERS) uses the Nielsen and IRI data (both retail scanner data and household consumption and scanner data) in addition to other purchased data as an integral part of its Consumer Food Data Program (NASEM, 2020). Brent Nieman and Joseph Vavra (2019) use Nielsen Homescan data to investigate changes in consumer shopping over the last 15 years. Scanner data and credit card transactions data have the potential to improve price indices by adjusting for the long lags between incorporation of the Economic Census data into the Census Bureau’s data program. Scanner data, in particular, are especially rich and available for the retail trade sector.

Credit Card Transactions, Bank Data, and Payroll Processer Microdata

One of the more exciting new applications of credit card data to improve the timeliness of estimates is BEA’s advance estimates of GDP for 2020, undertaken to try to capture the impact of COVID-19 on the economy. As noted in news releases, advance estimates are based on source data that are subject to updates. Much of the data used in these advance estimates is from the monthly and quarterly surveys that are part of the Census Bureau’s Economic Surveys. However, BEA also reported that its “assumptions were based on a variety of sources, most notably: private high-frequency credit card transactions data to better capture shifts in consumer spending, news reports on reopenings, and industry and trade association reports, that include volume data, such as health care patient visits and traveler throughput. More information on the source data and BEA assumptions that underlie the second-quarter estimate is shown in the ‘Key Source Data and Assumptions’ table on the BEA Website.”⁴¹

Cajner and colleagues (2018) show that high-frequency private payroll microdata can help forecast labor market conditions, noting that payroll employment is the most reliable real-time indicator of the business cycle. In their example, they demonstrate that using payroll microdata substantially improved forecast accuracy for current month employment and revisions to the BLS Current Employment Statistics.

Using anonymized transactions data from a large electronic payments technology company, Aladangady and colleagues (2019) created daily estimates of retail spending at detailed geographies. When aggregated to the national level, they found that these estimates had a pattern of monthly

___________________

⁴¹ From https://www.bea.gov/sites/default/files/2020-07/tech2q20_adv.pdf.

growth rates similar to that found in the official Census statistics. The daily estimates were available a few days after the transactions, and the authors provided historical estimates from 2010. They suggested that such daily estimates might be particularly useful during times of stress, such as hurricanes. Other examples include Mian, Rao, and Sufi (2013) using credit card company data, and Farrell and Grieg (2015) using accounts from a large bank.

In most of these studies, the source data were purchased or use of the data was granted through agreements. Collaborations between the government and the larger Internet-related companies in private industry, many of which have assembled massive data sets, might fruitfully expand the data available for the study of the retail trade transformation.

Web Scraping

Web scraping is the practice of extracting data from websites, typically through the use of a software program that simulates human exploration. One of the best-known examples of web scraping is the Billion Prices Project at MIT,⁴² which constructed daily price indexes for several countries using web scraping techniques to convert posted Internet prices of products to create a new daily version of a consumer price index for 22 countries (Cavallo and Rigobon, 2016).

The Bureau of Justice Statistics conducted a pilot project using web-scraped data from online articles to try to improve estimates for arrest-related deaths, finding that the “open-source methodology alone identifies the majority of law enforcement homicides, but agency surveys aid in identifying deaths by other causes (e.g., accidents, suicides, and natural causes).”⁴³

While web-scraping has great potential, it also creates policy challenges for federal statistical agencies. Unresolved questions include whether information on a company’s website can be considered publicly available data and how confidentiality protections should be applied. Policy to date has guided statistical agencies to secure permission from companies before web-scraping data from their websites.

Other Sources

Other alternative data sources include information from trade associations and commercial data on establishments. The National Establishment Time Series (NETS) is extracted from Dun and Bradstreet and available

___________________

⁴² See http://bpp.mit.edu.

⁴³ See https://www.bjs.gov/index.cfm?ty=pbdetail&iid=6626.

from Wold Associates.⁴⁴ The NPD Group provides retail tracking data including firm and store/product-level sales data.⁴⁵

CONCLUSION 3-11: Private-sector data, such as scanner data, might support capturing both quantities and prices of purchases to estimate the price effect of consumers moving between retail outlets.

CONCLUSION 3-12: Private-sector, credit card, and payroll processing data have been used to provide more timely information about economic output, prices, and input that could potentially be used to provide more timely estimates for labor productivity in the retail-related sector, though issues about the representativeness of those data will need to be addressed.

CONCLUSION 3-13: A collaboration between the government and larger Internet-related private companies has the potential to vastly expand the types of data available to study the transformation in retail trade and may support detailed analysis by population subgroup.

___________________

⁴⁴ See http://maryannfeldman.web.unc.edu/data-sources/longitudinal-databases/nationalestablishment-time-series-nets.

⁴⁵ See https://www.npd.com/wps/portal/npd/us/solutions/tracking-services/retail-tracking.