8

Toward an Integrated Annual Business Survey System

In Chapters 2–7, the panel reviews the 11 annual economic surveys that are major components of the U.S. Census Bureau’s portfolio of economic censuses and surveys. These surveys provide critically important business sector data for the National Income and Product Accounts and other key data series. The panel’s review covers the stages and major activities for data collection programs that are anchored in a survey, as well as the other sources used to achieve such goals as reducing respondent burden and improving data quality (see Table 1-1 in Chapter 1). These activities include: ascertaining data user needs and expectations (Chapter 2); constructing and updating the sampling frame, which for the annual economic surveys is the Census Bureau’s Business Register (Chapter 3); determining the concepts to be measured, data sources, including administrative records to draw on in addition to survey responses, data collection modes, and questionnaire design and content (Chapter 4); specifying sampling and estimation processes (Chapter 5); specifying procedures for data editing, imputation, disclosure control, and meeting quality standards (Chapter 6); and disseminating data products and data access tools to the user community (Chapter 7).

Our review documents numerous differences among the 11 surveys in all of these areas. Most differences appear to be the result of largely independent development of the surveys over time, rather than from a careful, evidence-based consideration of when differences are appropriate and when they are not. The result is that users cannot easily work with the different data products from the surveys for cross-industry and other kinds of analyses, businesses sampled in more than one survey are more burdened

than necessary, and the Census Bureau is not well positioned to benefit from economies of scale from standardized software, procedures, and other elements of the survey programs.

The Census Bureau recognizes that it needs to reengineer the annual economic surveys to achieve greater integration and has commendably taken steps to do so. The panel urges the Bureau to have as its ultimate goal a fully integrated and harmonized Annual Business Survey System (ABSS).

This concluding chapter begins by describing the panel’s concept of an ABSS that would be advantageous for users, respondents, and the Census Bureau. It then discusses the elements of a high-level project plan that the Census Bureau would need to articulate in detail to guide the development of an ABSS. Drawing on the recommendations in the preceding chapters (see Box 8-1), the chapter identifies opportunities for near-term improvements for the Census Bureau to pursue as steps toward an ABSS. The chapter also describes the experiences of other statistical agencies that have carried out significant reinventions of major business survey programs, noting the feasibility of such reengineering, the factors that were critical for success, and the pitfalls to be avoided.

8.1 THE CONCEPT OF AN ABSS

The panel’s concept of an ABSS is a system that would cover all industries included in the current suite of largely separate annual economic surveys. Specifically, an ABSS would integrate and harmonize the data that are currently collected in eight separate surveys: Annual Survey of Manufactures (ASM), Manufacturers’ Unfilled Orders Survey (M3UFO), Management and Organizational Practices Survey (MOPS), Annual Retail Trade Survey (ARTS), Annual Wholesale Trade Survey (AWTS), Service Annual Survey (SAS), Annual Capital Expenditures Survey (ACES), and the currently suspended Information and Communication Technology Survey (ICTS). Two other surveys covered in our review, the Company Organization Survey (COS) and the Business and Professional Classification Survey (SQ-CLASS), would be part of a reengineered system for updating the Business Register. The Annual Survey of Entrepreneurs, which pertains to people who own businesses, rather than the businesses themselves, would remain separate, with design changes as appropriate to complement an ABSS.

An integrated ABSS would contain a core set of questions asked of all respondents, with appropriate industry tailoring, plus industry-specific modules and rotating supplements on topical questions. Candidates for supplements (or for inclusion in the core or as modules) include the content of MOPS and ICTS, which were originally designed as supplements to ASM and ACES, respectively; the business expenses content that is collected annually for manufacturing and services as part of ASM and SAS, but only

once every 5 years for wholesale and retail trade as a supplement to AWTS and ARTS;¹ and the content recommended by the panel on topics for which users need more information, such as offshoring and contracting out of business functions (see Recommendation 4-4).

Although research will be required to determine the most useful and efficient sample design for an ABSS, the panel believes it should incorporate

___________________

¹ This content is termed the Business Expenses Survey (BES), which we did not review separately because it is essentially a module of questions.

an integrated approach for the selection of certainty businesses—that is, businesses that are so large that excluding them would distort estimates. For the smaller non-certainty units, the panel supports a fully or partially rotating panel sample. Under such a design, some smaller sample units would remain in the recommended ABSS for more than 1 year to facilitate consistent time-series data products and analyses. Assuming a fully rotating panel sample with a 5-year retention period (a matter for research), one-fifth of the sample would be retired and a new fifth of the sample added each year. In addition, business deaths would be dropped from the sample

whenever they occur, and samples of new businesses would be added to maintain the overall sample’s representativeness.

The sample for the recommended ABSS would be drawn from a redesigned, comprehensive Business Register that links establishments, enterprises, and alternative reporting units. The redesign currently planned by the Census Bureau, as augmented by the panel’s recommendations in Chapter 3, would support an efficient ABSS sample that enhances the usefulness of the data and reduces respondent burden. The use of the information in the register on corporate structure and characteristics, obtained from administrative records and survey sources, also would support improved data collection, editing, imputation, and estimation for the recommended ABSS. To support an enhanced Business Register, it will be important for the Census Bureau to implement appropriate changes in the COS and the SQ-CLASS.

The panel’s recommended ABSS would have a single application programming interface to access data, which would allow for improved dissemination. Correspondingly, tables, spreadsheets, and other data products would use standardized concepts and descriptors to facilitate cross-industry and time series analysis. Users of an ABSS would be supported by improved documentation and training through webinars, online tutorials, and other user-friendly approaches (see Recommendation 7-3).

In the panel’s concept, the Annual Survey of Entrepreneurs (ASE), which pertains to people who own businesses, rather than the businesses themselves, would remain separate, with design changes (e.g., harmonized content and question wording) as appropriate to complement the recommended ABSS. While not considered in depth by the panel, a useful research question for the ASE would be whether data on ownership characteristics are needed as frequently as every year.

After this report was drafted but before its release, the panel learned of a joint initiative by the Census Bureau and the National Center for Science and Engineering Statistics (NCSES) to fund a new Annual Business Survey (ABS) with the goal of improving the measurement of business dynamics in the United States. The ABS replaces the 5-year Survey of Business Owners (SBO, part of the economic censuses) for employer businesses, the ASE, and the Business R&D and Innovation Survey-Microbusinesses (BRDI-M). The ABS also replaces the innovation portion of the Business R&D and Innovation Survey (BRDIS), funded by NCSES. In addition to core content, ABS will include supplemental modules each year to collect information on emerging topics of relevance. Data collection will be electronic, with worksheets available to sampled businesses to assist them in assembling the needed information. The first mailout invitation to sampled businesses is scheduled for June 2018.²

___________________

² See https://www.reginfo.gov/public/do/PRAViewICR?ref_nbr=201712-0607-001 [March 2018].

The panel was not able to review the plans for the new ABS, but, in principle, it appears to be a welcome step forward toward survey integration and improved collection of information on business innovation and use of technology. We urge the Census Bureau to use the experience with the ABS to inform the development of our recommended ABSS and to take steps to minimize the burden on respondents from the two data collection programs and to harmonize and standardize content and procedures to the extent feasible.

There are two other important components of an ABSS. The first is the expanded use of administrative records data from the Business Register and other sources to achieve improvements in data quality, timeliness, and burden reduction (see Recommendations 4-1, 5-7, 5-10, and 6-2). Given the increasing challenges and costs of maintaining survey response rates, the panel believes it is imperative to use already collected administrative information to the greatest extent possible in place of questionnaire content to reduce respondent burden and thereby increase the likelihood that businesses will respond with completed questionnaires. Greater use of administrative records data also has the potential to improve editing, imputation, and estimation procedures.

The second important component of an ABSS is a capability to produce subnational estimates and greater industry detail than the ABSS sample sizes alone will likely support given budget constraints (see Recommendation 5-8). Such a capability, which is greatly desired by users, would entail collecting some data at subnational levels of detail and using small-area estimation techniques, which have been employed in other settings by the Census Bureau and other statistical agencies.

Finally, an ABSS, from design through implementation, could explicitly build in provisions for ongoing user input and ongoing evaluation and improvement of data quality. User input is needed not only to solicit views on proposed design features for an ABSS, but also to be sure users understand tradeoffs that may be necessary—for example, between relevance and continuity of time series or between improved timeliness from preliminary estimates and the possibility of subsequent revisions to estimates. Ongoing quality review needs to encompass potential new data sources, the processes used for data collection and estimation, and the estimates and data products provided to users.

RECOMMENDATION 8-1: To improve the utility of annual business data, reduce response burden on businesses, and more effectively use scarce resources, the Census Bureau should develop a detailed concept and implementation plan for an Annual Business Survey System (ABSS) to replace the current suite of largely separate annual economic surveys. An ABSS should

• administer a core set of questions to all respondents, with appropriate industry customization, plus modules containing industry-specific and topical questions;
• use a rotating panel sample drawn from a redesigned comprehensive Business Register;
• use standardized data collection, processing, and estimation methods for all industries in the sample;
• develop a single application programming interface for access to data for single and multiple industries;
• use administrative records to the maximum extent possible for greater efficiency and accuracy of data and reduced respondent burden; and
• incorporate small-area models to produce subnational geographic estimates with sufficient accuracy for users’ needs.

From design through implementation and continued operation of an ABSS, the Census Bureau should have ongoing programs to seek data user input and to evaluate and improve quality. Quality evaluation and improvement should encompass current and potential new data sources, data collection and processing, and resulting estimates.

The Census Bureau has commendably begun work to reengineer the annual economic surveys, acknowledging the need to integrate the separate surveys, streamline and harmonize the data collection, and make the resulting data more timely, accurate, and relevant to users. The reengineering work needs to be organized and focused toward a carefully designed outcome, and an ABSS is the concept we recommend the Census Bureau adopt for this purpose. An ABSS is feasible, as demonstrated by the experiences of other statistical agencies (see below), which offer useful lessons and factors to take into account for success. We encourage the Census Bureau to move forward with alacrity toward implementation of an ABSS in order to better serve national needs for improved economic data.

8.2 PLANNING AN INTEGRATED SYSTEM

The challenge and opportunity for the Census Bureau is to plan an integrated ABSS in ways that build momentum toward the end goal on as ambitious a schedule as resources permit. As is clear from a close examination of Table 1-1 (in Chapter 1), there will inherently be dependencies among the various stages in the design of a new ABSS. For example, the questionnaire cannot be redesigned until users’ needs have been specified; the data edits that are necessary and appropriate will depend on the ques-

tionnaire that has been developed; and the sampling and estimation methods that are developed will depend on the information that is contained in the Business Register.

A possible timetable for developing and implementing an ABSS that takes these interdependencies into account is presented in Figure 8-1. Development of an ABSS will take time; the timeline we have outlined assumes a 3-year implementation schedule. We believe it is possible to meet this schedule, but doing so will depend on commitment, resources, and effective management (see Section 8.5 below, “Lessons Learned for Implementing an ABSS”). Resources and commitment are particularly important because the current suite of annual economic surveys has to continue at the same time that research and development is under way for an ABSS.

The proposed schedule reflects lessons learned from the experience of panel members with major survey development projects. Because developing a questionnaire almost always takes more time than planned, questionnaire development needs to begin as soon as possible with the identification of key concepts (see Snijkers et al., 2013, pp. 127–163; also see Chapter 4). Modifications to the Business Register to support an ABSS also need to begin as early as possible (see Ravindra, 2016; also see Chapter 3), particularly as an enhanced register will be of great assistance to the Census Bureau’s other economic programs, including the quarterly and monthly indicator surveys and the quinquennial economic censuses. Development of the sampling and estimation system, the editing and imputation system, and the disclosure and quality review procedures can be staged for completion later in the process, along with the refinement of the questionnaire and the development of dissemination plans.

To implement a timeline for reengineering the annual economic surveys such as the one in Figure 8-1, it will be necessary for the Census Bureau to move from a broad overview of the necessary steps to a more detailed set of smaller focused projects. This will involve breaking down the overall project into its smallest components, to specify all of the resources that will be needed to achieve the desired result (Frame, 2003). For each focused project, a work breakdown structure (see Project Management Institute, 2006) or similar tracking system can be applied for planning the activities to be carried out and the resources required.

8.3 INITIAL STEPS IN HARMONIZING THE ANNUAL ECONOMIC SURVEYS

Throughout this report, the panel recommends a number of ways for the Census Bureau to improve the current annual economic surveys, including ways to harmonize them, using “tailoring within harmonization” to address significant industry differences, and ways to improve data col-

lection, estimation, and dissemination procedures generally (see Box 8-1). Below we single out some of those recommendations that we believe merit consideration by the Census Bureau for early testing and implementation, not only to improve one or more of the current surveys, but also to facilitate the development of an ABSS. Early successful action on these recommendations—covering the Business Register, data content and collection, sampling and estimation, data editing and related functions, and dissemination—will help establish momentum and maintain commitment to the overall project.

Our list of recommended first steps is meant to be suggestive and illustrative but not exhaustive or prescriptive. The Census Bureau will need to develop a detailed project plan, as suggested in Figure 8-1, with projects staged over the implementation period by their importance for an ABSS, by the necessity to include them as first steps because they are on the critical path for ultimate success, and by the ability of the annual economic surveys staff to leverage the capabilities of staff in other parts of the Census Bureau. Such leveraging will be critical to enable current production responsibilities to be met while getting research and development under way for an ABSS. In particular, the Center for Administrative Records Research and Applications (CARRA) and other centers in the Research and Methodology Directorate may be able to work in parallel with survey staff to make progress on a range of methodological issues.

We note the importance throughout the process of adhering to Recommendation 2-1, to involve data users at every stage of planning and development. Such involvement is critical to building support for the project and to ensuring that users can not only contribute to design decisions, but also understand the reasons behind them.

Finally, the panel recognizes that harmonization and methodological improvement are processes. Even when an ABSS replaces the current suite of largely separate surveys, continued review will be needed to determine when greater harmonization can be achieved or, conversely, when additional tailoring for certain industries or subject matter is appropriate. Similarly, continued review will be needed to learn which methodological changes have worked as expected, which need to be modified or dropped, and which can be further enhanced. We also recognize that up front investments in harmonization and improved methods for the current surveys and, ultimately, an ABSS cannot be expected to have immediate payoffs in terms of more efficient use of staff resources, improved timeliness, and other benefits. Yet the costs of continuing the current suite of surveys, which include duplication of staff resources, impaired ability to implement methodological improvements across industry sectors or to improve the relevance and timeliness of the data, and the absence of user-friendly data products and documentation that could help users develop time series,

comparisons across industries, and other useful analyses, are high. The benefits of harmonization and methodological improvements, leading to the development of an ABSS, in our assessment, represent a clear return on the necessary investment.

8.3.1 Business Register

Key to many needed improvements in the annual economic surveys and to the development of an ABSS is the timely completion of the Census Bureau’s planned redesign of the Business Register. The redesign should include the panel’s recommended enhancements (see Recommendation 3-1), which include a database structure that can accommodate additional information on corporate structure from surveys and alternative data sources that may become available in the future. The register should also provide for recording each time a business is included in one or more of the surveys to facilitate harmonization of sample designs that may reduce respondent burden (see Recommendation 3-2).

The urgency of the redesign of the register is not only for its use in making the annual economic surveys more efficient, leading to an ABSS, but also for efficiencies in the Bureau’s other economic data collection programs. To maintain the register, appropriate modifications to COS and SQ-CLASS will need to be part of the package of the Census Bureau’s initial steps toward an ABSS.

Recommendation 3-3 calls for an Account Manager Program for interactions with the largest businesses in the annual economic surveys to facilitate register updates and survey response. Such a program may require a longer implementation period to determine through careful evaluation the most effective methods and processes to train and support the account managers. Pilot testing in selected industries may be the best way to ensure success and achieve buy-in as the program is expanded across industries.

8.3.2 Content and Data Collection

As detailed above, content and questionnaire development for an ABSS should begin as soon as possible by a team that includes both survey methodologists and subject-matter experts (see Recommendations 4-2 and 4-3). The goal is an integrated core of content for all industries in an ABSS, with tailored question wording as appropriate for the industry, together with supplements on a rotating schedule for one or more industries. To facilitate progress toward that goal, the staff working on content and questionnaire could develop and test harmonized question blocks for key concepts, such as receipts or payroll, in one or more of the current surveys, and add more question blocks incrementally (see Recommendation 4-6). Each question

block should be based on agreed-on standard terminology for the underlying concepts with concordances that link any necessary industry-specific variations in wording to the relevant concept. The development of new questions for modules or supplements to fill important gaps in the information currently being collected also is something on which work ideally would begin early in the development process (see Recommendation 4-4).

For data collection, most of the annual economic surveys use the Centurion software platform for electronic responses. Assuming the Census Bureau believes Centurion to be the best one available, migrating those surveys that do not currently use Centurion to that platform as soon as possible will then make it possible to identify improvements that should be implemented for an ABSS. Provision for businesses to readily obtain print versions of questionnaires to facilitate their response also should be among the Bureau’s package of initial steps (see Recommendation 4-5).

Separate from the annual economic surveys program, it would be very productive for CARRA to begin the process to identify and evaluate administrative record sources that could replace or augment question content (see Recommendation 4-1). Early work in this area is essential to allow time for implementation, which will involve working out or negotiating access arrangements with agencies that house relevant administrative data (if the information is not already provided to the Census Bureau) and redesigning processing systems to combine survey responses with administrative information.

8.3.3 Sampling and Estimation

In the panel’s view, there are several changes that could improve the sample designs of the current suite of annual economic surveys in the short run and facilitate the development of an ABSS in the longer run. First, it is key (as it is for content and questionnaire development) for the Census Bureau to adopt standardized terminology, as consistent as possible with best practice in the field, for sample design features, sampling units, and estimation methods. Inconsistent terminology is an unnecessary impediment for communication and collaboration among staff of the different surveys and also can interfere with users’ understanding of the data (see Recommendation 5-1; see also Recommendation 6-6, which calls for consistent terminology for imputation procedures).

Second, it seems clear, especially for cost reasons, that an ABSS will have an enterprise-based sample design. To harmonize across its components, it will be important to determine as soon as possible the feasibility of using a two-stage design for the establishment-based ASM and MOPS. The first stage for the manufacturing sector would sample enterprises, and the second stage would sample establishments (see Recommendation 5-2). A

third and related priority is to determine whether stratified simple random sampling can be applied to the subset of the current surveys that currently use a probability proportional to size design without significant loss of statistical efficiency (see Recommendation 5-3).

Capturing and analyzing response rates in those annual economic surveys that do not refresh their samples from the Business Register more frequently than every 5–6 years—and also capturing other kinds of paradata for subsequent analysis—should be implemented as soon as possible (see Recommendation 5-5). Systematic collection and retention of paradata is essential for evidence-based decisions on a range of survey features, including sample design, questionnaire design and wording, and data collection modes (see Recommendations 4-6 and 4-7).

Other panel recommendations for improvements in sampling and estimation will likely require more research to implement. Such research (e.g., on methods for selecting certainty units, using administrative data in estimation, producing preliminary estimates) should begin as soon as feasible so that implementation is not unnecessarily delayed.

8.3.4 Editing, Imputation, Disclosure Control, and Quality Standards

For editing and imputation, which currently occupy disproportionate amounts of staff time, a necessary first step is standardization of terminology, as noted above (see Recommendation 6-6). Similarly, standardization of disclosure protection procedures and methods for indicating the quality of estimates in tables should be a high priority, as these steps will be of significant help to users in working with the data (see Recommendations 6-7 and 6-8). Other recommendations in these areas will require considerable research and development, which should begin as soon as feasible. Reducing the staff time devoted to data editing will require a multi-pronged approach of pruning unnecessary edits, automating the remaining edits as much as possible, and reengineering processes for when and how staff intervention is required (see Recommendation 6-1).

8.3.5 Dissemination

The panel stresses that it is never too soon to begin working with users, while taking into account their differing data needs and expertise, on improved data access for the current annual economic surveys, which can be enhanced for an ABSS. The text and recommendations in Chapter 7 identify a range of data products and tools that can be expected to provide a vastly improved user experience once they are implemented. The easier it is for users to work with the data, the more likely it is that the user community will grow and provide support for the importance of the data collected in

the current surveys and, ultimately, in an ABSS. Establishing a feedback loop with users to facilitate evaluation and continuous improvement of the Census Bureau’s data products, documentation, and access tools will be key (see Recommendation 7-10). The Census Bureau also should strive to learn from the experience of other statistical agencies and use standard software and methods to the extent feasible (see Recommendation 7-2).

8.4 REENGINEERING INITIATIVES IN OTHER STATISTICAL AGENCIES

As part of its work, the panel considered the experience of other statistical agencies that have reengineered their business survey programs in recent years. These agencies include Statistics Canada (Ravindra, 2016);³ Statistics Netherlands;⁴ Statistics Portugal (Saraiva, 2016); the United Kingdom’s Office for National Statistics (Clark, 2016; Penneck, 2009); and the U.S. National Agricultural Statistics Service (Clark, 2016). We draw on all of these efforts for lessons learned but focus in particular on the Statistics Canada and Statistics Netherlands experiences, which we judged to be the most relevant for an ABSS.

8.4.1 Statistics Canada

In 2010, Statistics Canada launched the Integrated Business Statistics Program (IBSP), aimed at reengineering the business processes, methods, and systems for the production of Statistics Canada’s business statistics (Ravindra, 2016).⁵ In early 2014, after 4 years of development and testing, the new approach was deployed for 60 annual business surveys, including surveys covering manufacturing, parts of retail trade, parts of wholesale trade, parts of services, and expenditure activities. In 2017, the IBSP covered 138 surveys by adding surveys on services, agriculture, mining, capital expenditures, and the energy sector. The enterprise survey that collects financial information and the agriculture census also will be incorporated into the IBSP. Only some portions of retail and wholesale trade will not yet have been migrated to the IBSP.

___________________

³ Based on presentation by Daniela Ravindra, Statistics Canada, “Statistics Canada’s Integrated Business Statistics Project,” during the panel’s October 29, 2015, meeting.

⁴ Communication from Jeroen van Velzen and Wim Vosselman of Statistics Netherlands, July 21, 2016.

⁵ Based on presentation by Daniela Ravindra, Statistics Canada, “Statistics Canada’s Integrated Business Statistics Project,” during the panel’s October 29, 2015, meeting.

All IBSP programs rest on four pillars:

1. The content for the IBSP economic programs has been harmonized, so that common variables, such as revenues and expenses, have the same definitions. As needed, however, the variables may have different names in different questionnaires in order to use the terminology a respondent is most likely to understand (“tailoring within harmonization”; see Chapter 4).
2. Statistics Canada’s Business Register is used as the frame for all of the IBSP program surveys. In addition, when updated information is obtained in the course of conducting an IBSP survey, it is fed back into the Business Register.
3. A common electronic platform is the primary instrument for data collection for all of the IBSP surveys.
4. All of the IBSP surveys use common tools for sampling, editing, imputation, and estimation.

The primary goal of the IBSP program has been to achieve efficiency improvements for all of Statistics Canada’s business surveys, and the initiative has in fact reduced staffing requirements and streamlined data collection and processing. Its scope was far broader than the scope of the panel’s recommended ABSS for the Census Bureau, but many of the considerations affecting the design of the IBSP initiative also are applicable to thinking about the implementation of an ABSS (see Section 8.5 below, “Lessons Learned for Implementing an ABSS”).

8.4.2 Statistics Netherlands

In 2000, to accommodate government-wide budget cuts, Statistics Netherlands launched a harmonization program for its annual and monthly business surveys, an initiative referred to as IMPECT (IMPlementation EConomic Transformation). IMPECT had two components—one for annual surveys (IMPECT1), which was implemented first, and the other for monthly surveys (IMPECT2). We considered only IMPECT1 in relation to an ABSS.⁶

The primary goal of IMPECT1 was to make the production of business statistics more efficient—while striving to maintain data quality—by harmonizing 12 separate annual business surveys that covered all sectors of the economy. Prior to IMPECT1, each of these surveys operated completely separately, and a previous effort at harmonization in the 1990s

___________________

⁶ Communication from Jeroen van Velzen and Wim Vosselman of Statistics Netherlands, July 21, 2016.

was not successful.⁷ The schedule called for IMPECT1 to be accomplished within 2 years, an admittedly ambitious timeline, even with some preparatory work on harmonization having been done in the 1990s. The scope of IMPECT1 was very similar to the panel’s recommended redesign for the Census Bureau’s annual economic surveys.

The IMPECT1 initiative consisted of a number of projects:

• UniQuest: Harmonizing the various questionnaires into one core questionnaire, with about 120 tailored versions for subgroups defined on the basis of industry and size class. This project’s first product was a harmonized list of concepts to be measured, which next were translated into variables for the questionnaires.
• LogiQuest: Building one integrated tool for data collection, producing the tailored questionnaires, and developing the communication materials (e.g., initial contact letters and nonresponse follow-up materials) to be mailed out according to a specified data collection schedule.
• UniEdit1: Harmonizing the data editing methods and creating a process and tool for applying the edits.
• UniEdit2: Harmonizing the methodology for imputation, outlier detection, weighting, and estimation.
• UniPub: Harmonizing the economic indicators based on the surveys to be published.
• Methods and Advice: Integrating input from a support group of methodologists. Among other things, this project harmonized the sample design and unit definitions.
• Process Control: Establishing a process control organization for the new system to ensure that it would run smoothly. This project was added to the list after IMPECT1 was already under way.

Some of the IMPECT1 projects moved along in parallel while others were sequential. Figure 8-2 shows the timeline for the IMPECT1 projects.

Because of the very tight timetable, which made the entire project in some sense a quick win, the scope of IMPECT1 and the schedule for its implementation included only essential elements. The schedule allowed no time for pretesting of the new questionnaire or the new production system, which meant that some issues needed to be worked out after the system was in place, and it was a few years before everything was working smoothly. Because Statistics Netherlands already had a centralized business register, there was no need to have a project for centralization of the register. In

___________________

⁷ Communication from Jeroen van Velzen and Wim Vosselman of Statistics Netherlands, July 21, 2016.

retrospect, however, there were important differences in the definition of units in the various surveys that needed to be reconciled and reflected in the register, and it would have been better if the work to harmonize the unit definitions had been done earlier in the process. The UniPub project, which involved standardizing the economic indicators to be published, did not start until relatively late in the process because the harmonized list of output variables produced in the UniQuest project was needed before it could begin.

8.5 LESSONS LEARNED FOR IMPLEMENTING AN ABSS

The major initiatives to redesign the business surveys of Statistics Canada, Statistics Netherlands, and other statistical agencies all identified the same major factors that must exist to accomplish a successful redesign (see Box 8-2). For illustration, our discussion follows the steps described by Ravindra concerning the Statistics Canada IBSP initiative,⁸ drawing on other statistical agencies’ experiences as appropriate.

8.5.1 Vision

As shown in Box 8-2, Ravindra identifies four main factors for success (see also Kotter, 1995, 2012). The first factor is having a clear vision. Part of developing a vision is being clear about the reasons an initiative is being pursued. In the case of Statistics Netherlands, change was driven by outside imperatives. Prior to 2000, Statistics Netherlands had two locations, one in Heerlen and the other in The Hague. In 2000, responding to large budget cuts affecting all parts of government, the Ministry of Economic Affairs decided to close down Statistics Netherlands’s Heerlen location. Instead of accepting the plan and shuttering the Heerlen offices, Statistics Netherlands proposed an alternative plan for meeting the budget targets it had been given. This plan was IMPECT, an initiative designed to reduce business survey costs through increased efficiency while endeavoring to maintain data quality. The implementation of IMPECT (both IMPECT1 for the annual surveys and IMPECT2 for the monthly surveys) resulted in a 40 percent reduction in the staffing required to carry out the surveys.⁹ Budget exigencies and the need to become more efficient also drove change at Statistics Canada.

Another part of developing the vision for this sort of project is making it clear and actionable. Statistics Canada spent considerable time writing a

___________________

⁸ Based on presentation by Daniela Ravindra, Statistics Canada, “Statistics Canada’s Integrated Business Statistics Project,” during the panel’s October 29, 2015, meeting.

⁹ Communication from Jeroen van Velzen and Wim Vosselman, Statistics Netherlands, July 21, 2016.

document that not only laid out the overall vision for the IBSP initiative, but also made it as concrete as possible. Six key components of the overall vision included (1) overall optimization rather than optimization for any specific component; (2) allowing for activity surveys as well as industry surveys; (3) allowing for enterprise- as well as establishment-based surveys; (4) providing a quicker processing turnaround through a simpler, more harmonized model; (5) increasing operational efficiency; and (6) maintaining a level of data quality within Statistics Canada norms (Statistics Canada, 2009).

Discussing and refining the vison made it Specific, Measurable, Assignable, Realistic, and Time-bound, that is, SMART. During the development stage, this vision served as an anchor. It was consulted over and over again, to make sure that the program stayed on track. In a comparable way, IMPECT1 at Statistics Netherlands was divided into several component projects, and plans for each of those projects were elaborated by the team responsible for that part of the work.

As outlined by senior Census Bureau officials to the panel, the impetuses for changes to the Census Bureau’s economic statistics programs include budget pressures; increasing difficulty in obtaining responses to business surveys; growing desire from data users for more detailed data; and the per-

ception that, in the face of increasing competition from privately produced data, official statistics from the Census Bureau are at risk of becoming increasingly less relevant. The panel believes that this report responds to those factors and offers a concept for the Census Bureau to reengineer the annual economic surveys. Further work clearly will be needed to translate the concept into an actionable implementation plan: this chapter identifies some important principles and practical considerations for developing such a plan.

8.5.2 Communication

Having a vision on paper is not the same as having a vision that is embraced by everyone in the organization. Communicating the vision and plan from the very beginning of a reengineering effort is the second key success factor identified by Ravindra (see Box 8-2) and is critical for obtaining the commitment to the plan by all of the relevant actors. Clark (2016) similarly observes that a reengineering project must include communicating the vision and motivating the staff. Communication is required not only at the start of the process, but throughout all phases of the project.

At Statistics Netherlands, there was a strong organizational imperative for starting the IMPECT project, but continuing communication was essential to ensure that it succeeded. As implementation of the plan proceeded, regular staff meetings were held to keep everyone informed about progress. At every meeting, the message was the same: “We need to do this and we need to keep it simple. We need to stay focused on the most important tasks and not try to do everything at once.” Every meeting began with a slide with one word on it: “Simple.” This continuing communication was essential to keep the project on track. Equally important to keeping team members motivated and on task was being able to communicate and celebrate successes as the project progressed.

8.5.3 Governance

The third success factor listed in Box 8-2 is strong governance. Even though this factor is listed third, it may be the most important. Strong governance, including continuing support from upper management, is an absolutely essential ingredient for any major change to succeed. Clark (2016) elaborates on this message, noting the importance of finding leadership and support for a plan at all levels of an organization undergoing change.

The strong commitment of senior management to the project was one reason for the success of the IMPECT1 initiative at Statistics Netherlands. The project was very ambitious—involving new questionnaires, new methodologies, and aggressive timelines—which meant it carried inherent risks. Senior management understood, however, that the job simply had to be

done. The deputy director of the Economics Statistics Division was assigned to be the overall manager for IMPECT, and, as IMPECT progressed, he reported regularly to the Statistics Netherlands board of directors. It is important to note that managing IMPECT was his only job.

In projects such as a major reengineering, there is often a tendency to want to preserve existing ways of doing things, even where that is not consistent with the overall goal. Clark (2016) recommends creating teams to carry out the project with members who are selected based in part on their commitment to the overall concept and the plan. At Statistics Netherlands, the people selected as project leaders were individuals who had proven themselves capable of achieving results under pressure and who did not have past involvement with any of the annual business surveys, so that they were not committed to old ways of doing business. The project leaders were permitted to pick anyone they wanted to work on their projects, which allowed them to assemble very strong teams. Project teams were delegated considerable authority to make their own decisions. The result was that everyone involved felt responsible for the outcome of the project, and the teams were fully committed to obtaining achieving results.

A similar level of commitment will be required at the Census Bureau to ensure the success of the reengineering of the annual economic surveys that the panel recommends. It will be critical that the director, as well as other senior managers, endorse and fully commit to the project. The Bureau will need to appoint a senior manager whose sole responsibility is the successful completion of the reengineering plan. Strong teams will be needed to carry out the work. Although there is obviously an important role for oversight and coordination, to the extent possible, these teams should be empowered to make decisions regarding their projects so that the work can move forward in a timely fashion.

RECOMMENDATION 8-2: The Census Bureau should create a team with clear authority to lead the program to build an Annual Business Survey System (ABSS). The team should be headed by a senior manager for whom ABSS implementation is her or his sole responsibility.

8.5.4 Consultation with Users

A final success factor listed in Box 8-2 is wide consultation with data users. Wide consultation does not mean that a full consensus among users with regard to the final products to be delivered should be sought; such a consensus may in fact not be achievable. It is important, however, that all key users be involved and a part of the process. As discussed in Chapter 2 and elsewhere, key users of data from the Census Bureau’s annual economic

survey include other statistical agencies, such as the Bureau of Economic Analysis, the Bureau of Labor Statistics, the Division of Research and Statistics of the Federal Reserve Board of Governors, and the National Health Statistics group in the Office of the Actuary at the Centers for Medicare & Medicaid Services. Others who rely either on those data or on statistics that use those data as an input include federal economic policy officials, state and local government officials, trade associations, local business organizations, and others. All should be consulted as the reengineering project moves forward. Consultation needs to include consideration of the potential for breaks in time series as the current surveys are dropped and an ABSS is implemented; cost-effective ways to evaluate and minimize the effects of any breaks, such as by administering new and old questionnaire versions to portions of an ABSS sample; and documentation and training aids that can help users make the transition (see Recommendation 2-1).

8.5.5 Lessons Learned in Implementation

In her presentation to the panel, Ravindra discussed a number of implementation issues that need to be taken into account when planning for a new ABSS.¹⁰ In another presentation she discussed four lessons learned from the Statistics Canada reengineering experience (Ravindra, 2016). Box 8-3 distills some of the lessons described in these two presentations.

As shown in Box 8-3, the first lesson learned was that the demands of developing a new survey system while continuing ongoing production work should not be underestimated. Ravindra makes two points about this:

A second lesson learned was that complexity in the plan cannot always be justified. This is something that Statistics Netherlands worried about a great deal in connection with IMPECT; hence, the project mantra, “Keep it simple.” Ravindra concludes that, in the IBSP project, too much effort

___________________

¹⁰ Based on presentation by Daniela Ravindra, Statistics Canada, “Statistics Canada’s Integrated Business Statistics Project,” during the panel’s October 29, 2015, meeting.

¹¹ Based on presentation by Daniela Ravindra, Statistics Canada, “Statistics Canada’s Integrated Business Statistics Project,” during the panel’s October 29, 2015, meeting.

initially was devoted to elements that would have small effects. For example, a number of methodological changes were introduced with the goal of increasing data quality, but these sophisticated methods were new to Statistics Canada and implementing them took much more time than had been estimated. In the words of Ravindra:

___________________

¹² Based on presentation by Daniela Ravindra, Statistics Canada, “Statistics Canada’s Integrated Business Statistics Project,” during the panel’s October 29, 2015, meeting.

A third lesson learned was that all relevant expertise needed to be represented on project teams, including information technology staff, process architects, subject-matter experts, and the clients for and users of the products. The fact that all of these people have many competing demands on their time admittedly may make team operations difficult. In the case of Statistics Canada, a shortage of agency staff with critical skills resulted in the use of contractors on project teams. This outsourcing led to some problems when these individuals’ contracts ended and they departed, as new team members had to be trained, and productivity suffered as a result.

One of the most important lessons learned was how easy it is to underestimate the time needed for integrated testing of a new system. According to Ravindra, “Because the development timelines for the project were fairly tight, a lot of testing was carried out on each component of the system but not necessarily on the entire process.”¹³ The effect of not having allowed sufficient testing time is that issues were pushed downstream and had to be dealt with during implementation.

In somewhat the same vein, in developing a project plan, it is easy to underestimate costs and overestimate potential efficiency gains. On the basis of presentations to the panel, for planning purposes, a suggested rule of thumb is to multiply estimated time and costs by one and a half and reduce estimated efficiencies by one-third.

A final lesson learned is that any plan will need to be adjusted as it is implemented. Responding flexibly as new information becomes available will be important for achieving success.

8.6 REFERENCES

NOTE: All URL addresses were active as of November 2017.

Clark, C.Z.F. (2016). Leading significant change in official statistics: A woman’s place is in the counting house! In A. Golbeck, I. Olkin, and Y. Gel (Eds.), Leadership and Women in Statistics (pp. 69–84). Boca Raton, FL: CRC Press.

Frame, J.D. (2003). Managing Projects in Organizations: How to Make the Best Use of Time, Techniques, and People. San Francisco, CA: Jossey-Bass.

Kotter, J.P. (1995). Why transformation efforts fail. Harvard Business Review, March–April, 61.

Kotter, J.P. (2012). Leading Change. Boston, MA: Harvard Business Review Press.

Penneck, S. (2009). The Office for National Statistics (ONS) Statistical Modernisation Pro-gramme: What went right? What went wrong? In Proceedings of Modernisation of Statistics Production. Helsinki, Finland: Statistics Sweden.

Project Management Institute. (2006). Practice Standard for Work Breakdown Structures (Second Edition). Newtown Square, PA: Author.

___________________

¹³ Based on presentation by Daniela Ravindra, Statistics Canada, “Statistics Canada’s Integrated Business Statistics Project,” during the panel’s October 29, 2015, meeting.

Ravindra, D. (2016). Challenges and benefits of producing business statistics within a highly centralized model. In Proceedings of the 5th International Conference on Establishment Surveys. Alexandria, VA: American Statistical Association. Available: http://ww2.amstat.org/meetings/ices/2016/proceedings/153_ices15Final00050.pdf.

Saraiva, P. (2016). Integrating Data Collection: Wins and Challenges. Presentation at the United Nations Economic Commission for Europe Workshop on Statistical Data Collection—Visions on Future Surveying, The Hague, October 1–5. Available: https://statswiki.unece.org/pages/viewpage.action?pageId=120129901&preview=/120129901/124748478/WP3-5%20Portugal%20-%20Saraiva_presentation.pdf.

Snijkers, G., Haraldsen, G., Jones, J., and Willimack, D.K. (2013). Designing and Conducting Business Surveys. Hoboken, NJ: John Wiley & Sons.

Statistics Canada. (2009). Integrated Business Statistics Program. Blueprint. Ottawa: Enterprise Statistics Division.