2
Scope and Importance of Innovation

This chapter, reporting largely on the first workshop session, addresses the question: Why is innovation in the federal statistical system needed? The main arguments of the background paper by Robert Parker (2010) are summarized first, followed by relevant points from the background paper by Hermann Habermann (2010a), and then by participants’ comments. As noted in Chapter 1, many of the participants also commented on barriers and possible remedies; those comments are summarized in Chapters 3 and 4, respectively.

CHALLENGES TO THE FEDERAL STATISTICAL SYSTEM

In his paper, Parker focuses on the importance of continued innovation by the system. He discusses several cases in which users are asking for more relevant policy data and identifies some of the challenges in providing these data. Parker notes that in looking ahead there are risks and challenges that will require the system to change. The identification of such challenges is not new, as evidenced by statements made over more than a decade:

The challenge for the 21st century is to build on to the remarkable statistical system developed in the 20th century.… We need to take advantage of new methods of information collection and dissemination and devote adequate resources to improve the quality, coverage and timeliness of federal statistical programs.… Federal statistics are not a “hot button” issue; politicians do not run for office on a strong plank for improving them (Knapp, 1996).



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 9
2 Scope and Importance of Innovation T his chapter, reporting largely on the first workshop session, addresses the question: Why is innovation in the federal statistical system needed? The main arguments of the background paper by Robert Parker (2010) are summarized first, followed by relevant points from the background paper by Hermann Habermann (2010a), and then by participants’ comments. As noted in Chapter 1, many of the participants also commented on barriers and possible remedies; those comments are summarized in Chapters 3 and 4, respectively. CHALLENGES TO THE FEDERAL STATISTICAL SYSTEM In his paper, Parker focuses on the importance of continued innova- tion by the system. He discusses several cases in which users are asking for more relevant policy data and identifies some of the challenges in providing these data. Parker notes that in looking ahead there are risks and challenges that will require the system to change. The identification of such challenges is not new, as evidenced by statements made over more than a decade: The challenge for the 21st century is to build on to the remarkable statis - tical system developed in the 20th century. . . . We need to take advantage of new methods of information collection and dissemination and devote adequate resources to improve the quality, coverage and timeliness of federal statistical programs. . . . Federal statistics are not a “hot button” issue; politicians do not run for office on a strong plank for improving them (Knapp, 1996). 9

OCR for page 9
10 FACILITATING INNOVATION IN THE FEDERAL STATISTICAL SYSTEM *** We are in the early stage of a technological revolution that will dramati - cally increase the demand for statistics and their use in policy debates, as well as in many other areas of society. This revolution is created by the advent of the Internet and the World Wide Web. . . . I believe that three principal challenges must be faced if the system is to successfully meet the demands placed on it by this technological revolution. The three chal- lenges are relevance, validity, and timeliness (Bradburn, 1999). *** Many new problems are facing the statistical agencies, and it will take an enormous effort to solve them. Indeed, the agencies are fully aware and understand there is a need for innovative thinking (Spar, 2009). One of the essential functions of any statistical agency is to produce relevant data, a requirement recognized by the Committee on National Statistics in a volume that lays out the principles that determine an effec - tive statistical agency (National Research Council, 2009). The first of these principles is that “a federal statistical agency must be in a position to pro - vide objective information that is relevant to issues of public policy.” One of the most important reasons for innovation, then, is ensuring the ability to provide users with policy-relevant data. In his paper, Parker offers four examples of areas in economic statistics in which innovation is needed to provide relevant data for major policy issues and that illustrate the need for innovation by the federal statistical community: 1. data on intangible assets (based on work by the U.S. Department of Commerce); 2. measures of economic welfare (see Stiglitz, Sen, and Fitoussi, 2009); 3. indicators related to the most recent recession and recovery (see Advisory Committee on Measuring Innovation in the 21st Century Economy, 2008; Blank, 2010; Krueger, 2010); and 4. measures of international economic activity. These areas are all critically important to the nation, as has been noted by others. For example, the problems involved in the measurement of economic progress and welfare and, more specifically, the challenge of defining and measuring gross domestic product were raised in a recent New York Times Magazine article (Gertner, 2010) on the utility of gross domestic product (GDP). Because of their importance, these examples, as developed by Parker, are described in more detail below.

OCR for page 9
11 SCOPE AND IMPORTANCE OF INNOVATION Data on Intangible Assets The need for innovation is not unique to the federal statistical system. Innovation is essential for the business community if the United States is to be economically competitive in the world. Recognizing this, in 2008 the Commerce Department’s Advisory Committee on Measuring Innovation in the 21st Century Economy issued a report calling for new measures of innovation to be prepared by the Bureau of Economic Analysis (BEA), the Bureau of Labor Statistics (BLS), and the National Science Founda- tion (NSF). For its work, the committee adopted the following defini- tion of innovation (Advisory Committee on Measuring Innovation in the 21st Century Economy, 2008, p. 3): “The design, invention, development and/or implementation of new or altered products, services, processes, systems, organizational structures, or business models for the purpose of creating new value for customers and financial returns for the firm.” The committee, then, was concerned with developing new measures of innovation in the business community that would result in improved returns for the business and added value for the customer and be more cost-effective. It is worth noting that if this definition were to be applied to the federal statistical system, statistical agencies would supply more relevant, timely, and reliable data and be more cost-effective in doing so. Carl Schramm, chair of the advisory committee and president and chief executive officer of the Ewing Marion Kauffman Foundation, described the need for new measures of innovation as “central to under- standing the economy as it evolves and responds to growing world com- petition. . . . Improvements to our measurement of innovation will help to ensure continued economic strength” (Parker, 2010). The measures the advisory committee called for include a comprehensive accounting of the effect of high-tech goods and services on growth and productiv - ity, as well as new data on research and development and innovation- related inputs.1 BEA reported that there are difficult conceptual issues to be resolved in order to collect new data on such intangible investments (Aizcorbe, Moylan, and Robbins, 2009). In addition, the collection of these data may be difficult because they may not be available from the usual business accounting records. Furthermore, it may be that firms can pro- vide such estimates only at the enterprise level, yet the data are needed at the establishment level. Thus, statistical agencies will need to develop new methods to distribute firm-level data to the establishment level for a variety of types of intangible investments. 1The annual Business R&D and Innovation Survey, just released in 2009 by the U.S. Census Bureau for the National Center for Science and Engineering Statistics, is intended to expand the available data on R&D and innovation. See http://www.nsf.gov/statistics/ srvyindustry/ [June 2011].

OCR for page 9
12 FACILITATING INNOVATION IN THE FEDERAL STATISTICAL SYSTEM GDP and Measurement of Economic Welfare and Prosperity The Report by the Commission on the Measurement of Economic Perfor- mance and Social Progress, known as the Sarkozy report, argued that there is no sufficient statistic for judging the health of the economy (Stiglitz, Sen, and Fitoussi, 2009).2 Joseph Stiglitz, one of the authors of the report, said that policy makers “focused too much attention on GDP as an indica- tor of economic success, and there was no indication in the GDP figures that a crisis was brewing. Although GDP is the best-developed broad measure of economic performance, it can provide a misleading gauge of the quality of life or sustainability of an economy” (Parker, 2010). Although not in specific response to the Sarkozy report, BEA in 2010 published an article related to the report and some of its recommenda- tions (Landefeld et al., 2010). The article reported that BEA had looked at changing its presentation of the various measures in the GDP accounts and announced that it plans to publish additional detail based on the current accounts (p. 12): This article explores each of these issues and relates them to the need for expanded or supplementary measures for the national accounts, high- lighting what such estimates might reveal relative to the conventional statistics presented by GDP and other aggregate statistics from the ac - counts. In particular, it explores how the accounts might be extended to provide new measures of (1) the distribution of growth in income across households, other sectors, and regions and (2) the sustainability of trends in saving, investment, asset prices, and other key variables important to understanding business cycles and the sources of economic growth. With regard to various alternative indicators, such as the genuine progress indicator, the world development indicators, and the Index of Sustainable Economic Welfare, BEA reported that “while these efforts have been much discussed and debated, there has never been sufficient consensus on the difficult issues involved to produce a common set of concepts or methods or a widely accepted regular set of estimates that were used for analytical or policy purposes” (Landefeld et al., 2010). This article was followed by the New York Times Magazine article by Gertner (2010), which noted: For decades, academics and gadflies have been critical of the measure [GDP], suggesting that it is an inaccurate and misleading gauge of pros - perity. What has changed more recently is that G.D.P. has been actively challenged by a variety of world leaders, especially in Europe, as well as by a number of international groups, like the Organization for Economic 2The commission was created in 2008 by French President Sarkozy to identify the limits of GDP as an indicator of economic performance and social progress .

OCR for page 9
13 SCOPE AND IMPORTANCE OF INNOVATION Cooperation and Development. The G.D.P., according to arguments I heard from economists as far afield as Italy, France and Canada, has not only failed to capture the well-being of a 21st-century society but has also skewed global political objectives toward the single-minded pursuit of economic growth. “The economists messed everything up,” Alex Michalos, a former chancellor at the University of Northern British Columbia, told me recently when I was in Toronto to hear his presenta- tion on the Canadian Index of Well-Being. The index is making its debut this year as a counterweight to the monolithic gross domestic product numbers. “The main barrier to getting progress has been that statistical agencies around the world are run by economists and statisticians,” Michalos said. “And they are not people who are comfortable with hu- man beings.” The fundamental national measure they employ, he added, tells us a good deal about the economy but almost nothing about the specific things in our lives that really matter. Parker suggested that part of the reason statistical agencies have not embraced alternative concepts is their desire that components of, for example, the index of well-being, be measures that (1) have a theoretical relationship to the concept being measured; (2) are reliable; and (3) for the weights used to combine them, are based on some reasonable calculation. Consequently, the Canadian Index of Well-Being (CIW) is not produced by Statistics Canada but by a group of nonprofit foundations. Even if the “domains” covered by the index are reasonable, many economists and statisticians do not believe there is sufficient theoretical foundation to assign the appropriate weights to the domains. This problem of calculat- ing weights also applies to other measures cited by Gertner, such as the planned Key National Indicators System3 and the Human Development Index.4 These factors do not mean that the measures do not have value in assessing progress; it does mean that more conceptual and theoreti - cal work needs to be done before they will be produced by government statistical agencies. Nevertheless, the measures discussed in the Gertner article do pose a challenge for statistical agencies. Should they attempt to develop statisti - cally valid measures of prosperity in addition to the conventional mea- sures? BEA has expressed its reasons for not doing so, but other agencies may choose to look at more narrowly defined welfare measures consistent with their areas of responsibility, such as health, education, and the envi - 3For a detailed report on the indicators project, see U.S. Government Accountability Office (2004, 2011). 4See http://hdr.undp.org/en/statistics/hdi/ [June 2011].

OCR for page 9
14 FACILITATING INNOVATION IN THE FEDERAL STATISTICAL SYSTEM ronment.5 The challenge of providing measures of welfare and prosperity is that the current methods are not those usually used by statistical agen - cies. This difference raises the question of whether the agencies should avoid those measures or perhaps limit their participation to just collecting input series. But even the latter step has significant risks, as some of the items would require the collection of opinion-type data, now usually col - lected only by polling firms. To deal with these problems Parker suggests “that agencies and related professional associations may need to develop guidelines on what the agencies should and should not collect” (p. 6). Indicators of Recession and Recovery In October 2009, Alan Krueger, then the chief economist in the U.S. Department of the Treasury, addressed the annual meeting of the National Association of Business Economists (NABE) on how the weaknesses in the financial and regulatory systems also “revealed an important weakness . . . in data and statistics that policymakers and others use to assess the performance of the economy to predict its future prospects, and to evalu- ate the effectiveness of public policies” (Parker, 2010). Krueger, who pre - viously headed the Princeton Data Improvement Initiative, which evalu- ated the reliability of the government statistical agencies’ main economic indicators, included as weaknesses a lack of timeliness, insufficient detail, and lack of relevance of certain key statistics as well as data gaps. More recently, Commerce Department Undersecretary Rebecca Blank addressed an NABE seminar on federal statistics on the same topic. The rest of this section summarizes their major concerns, as laid out by Parker. Krueger and Blank both noted that most key data series are released with a lag, making it difficult to monitor existing or proposed stabilization policies. For example, the Federal Reserve Board’s flow of funds accounts provides information on sector balance sheets and related transactions, but they are only produced quarterly and with a 3-month lag. Similarly, its Survey of Consumer Finances provides a detailed look at the state of households’ financial health only once every three years, with well more than a year’s lag between the last year of the survey and the release date. BEA’s GDP data by industry are produced annually with a 5-month lag. Krueger and Blank said that more timely data on how different industries were affected by the recent recession would have been valuable in assess- 5It also should be noted that in calculating GDP, BEA follows international guidelines, as do almost all other countries. Thus, changes to GDP would require agreement at the international level.

OCR for page 9
15 SCOPE AND IMPORTANCE OF INNOVATION ing various policy options.6 Also, more timely Census Bureau longitudi- nal firm-level data would enable policy makers to study births and deaths over the economic cycle and to determine which firms contract (or grow) more than others in a time of economic change. Likewise, more timely longitudinal household-level data would show what happens to families when one member experiences unemployment. Krueger and Blank also found that there is an almost complete lack of information for a number of key economic variables. They pointed to little or no data on investment in intangibles; almost no data relating to loan originations (because the measures on characteristics of mortgages collected by the Mortgage Bankers Association are incomplete); no data on alternative sources and characteristics of bank lending; and few data to determine why the financial collapse created a credit crisis that led to the recession. They also noted that there is limited information about the amount saved from mortgage refinancing activity, which limits the ability to estimate the aggregate economic effects of refinancing. Krueger suggested that one possible solution to filling these and related data gaps would be to establish some sort of “rapid response” data-gathering capac- ity in the statistical system that could be tailored to answer specific, one- shot questions, such as changes in consumption by households. Krueger and Blank also noted that the available GDP accounts pro- vide information as to whether the economy is contracting or expand - ing, but not about the sustainability of growth or about the distribution of income; neither do the accounts provide information as to whether resources are being used to maximize well-being. For example, neither the negative effects of pollution nor the positive effects of leisure time are valued in GDP. Because of these lacks, some analysts studying the future health of the economy also look at other statistics, such as the poverty rate, data on consumer wealth or the state of the environment, and data from household time-use surveys. In supporting his concerns about reli - ance on GDP, Krueger also referred to the findings of the Sarkozy report (Stiglitz, Sen, and Fitoussi, 2009), discussed above. Some of the issues raised by Krueger and Blank involve data cur- rently collected by the private sector or by financial regulatory agencies. Improvements might be made by statistical agencies partnering with these organizations. However, the preparation of sampling frames by statistical agencies may require additional detail on products of large financial institutions and dealing with new types of organizations. As with new data on innovation, some of these data will be available only at the enterprise level, so the agencies would need to develop new methods 6A proposal to fund quarterly industry GDP was included in the 2011 BEA budget request to Congress.

OCR for page 9
16 FACILITATING INNOVATION IN THE FEDERAL STATISTICAL SYSTEM to distribute enterprise-level data to the level of establishments. Similarly, if funding is available only to prepare national-level data, it may be nec - essary to develop other techniques to provide more detailed geographic detail. The Census Bureau recently developed small-area estimates for health insurance coverage using a model that combines survey data with population estimates and administrative records.7 Measures of International Economic Activities In the May 2010 issue of BEA’s monthly journal, the Survey of Cur- rent Business, Howell and Yuskavage (2010) discuss the agency’s planned changes in the GDP accounts to more closely align them with new inter- national guidelines. One example is the issue of whether the United States will have the source data to implement the recommended change for goods that are sent abroad for further processing and then return without change in ownership. The current treatment is to “impute” a change in ownership and record the values as exports and imports of goods. The recommendation would stop the imputation, exclude the merchandise from exports or imports, and include the difference in the two values as a service. The article reports on research to determine if data can be collected—by the Census Bureau, BLS, and BEA—to meet the new guide - lines and if users of these data are in favor of the change. Those three agencies would need to conduct significant research to determine not only if the necessary source data can be collected, but also whether those new data will be sufficiently reliable because they will affect both the U.S. international transactions accounts and the U.S. GDP. The article also discusses needed improvements to the data collected by BEA on international trade in services. For example, BEA has expanded the mailing list used for these surveys to reflect information from the Census Bureau’s Company Organization Survey and has instituted a new survey of travel expenditures both abroad and in the United States.8 Col- lecting these expenditures has proven to be a difficult task, and it is not clear whether the new survey will be successful. Both the Census Bureau and BLS play an important role in the prepa- ration of the U.S. international transactions accounts, note Howell and 7For more details on the methodology for these estimates, see the Small Area Health Insurance Estimates Program on the Census Bureau’s website, see http://www.census.gov/ did/www/sahie/index.html [May 2011]. 8Such sharing was made possible by the enactment of the Confidential Information Protection and Statistical Efficiency Act of 2002. Further sharing of these lists would be possible with the enactment of the data synchronization legislation recommended by the Department of Commerce’s Advisory Committee on Measuring Innovation in the 21st Century Economy (2008), discussed above.

OCR for page 9
17 SCOPE AND IMPORTANCE OF INNOVATION Yuskavage (2010). The Census Bureau provides the merchandise trade data used to prepare the estimates of international trade in goods, and BLS provides the price indexes used to deflate most components of those goods estimates. Each of these programs faces data collection issues that affect the reliability of the U.S. accounts. In January 2010, the Census Bureau introduced a new methodology for the estimation of low-value imports and exports—that is, small transactions whose value falls below the exemption levels for filing the administrative documents used to generate the merchandise trade data.9 It is hoped that future evaluation studies will confirm the utility of the new methodology. Parker’s paper also notes that BLS continues to face problems in col- lecting prices for intracompany transfers. In his paper, Parker states that a study conducted in 2001 concluded that BLS should make changes to improve the consistency of transfer prices by collecting data consistent with administrative definitions from the Internal Revenue Service or the Customs Service or by investigating the use of export price indexes from other countries. It does not appear that significant research has been con- ducted into new or improved data collection methods in this important area. Duplication in Federal Household Surveys Parker provided another example from household surveys, summa- rizing a report from the U.S. Government Accountability Office (2006): At the time of GAO’s review, OMB had approved 584 ongoing federal statistical or research surveys, of which 40 percent were administered to individuals and households. . . . The seven surveys GAO reviewed could be considered to contain necessary duplication. GAO identified three subject areas, people without health insurance, people with dis - abilities, and housing, covered in multiple major surveys that could potentially involve unnecessary duplication. Although they have simi - larities, most of these surveys originated over several decades, and differ in their purposes, methodologies, definitions, and measurement techniques. These differences can produce widely varying estimates on similar subjects. For example, the estimate for people who were uninsured for a full year from one survey is over 50 percent higher than another survey’s estimate for the same year. While agencies have undertaken efforts to standardize definitions and explain some of the differences among estimates, these issues continue to present challeng - es. In some cases, agencies have reexamined their existing surveys to reprioritize, redesign, combine, and eliminate some of them. Agencies have also used administrative data in conjunction with their surveys 9For details, see http://www.census.gov/foreign-trade/aip/lvpaper.html [May 2011].

OCR for page 9
18 FACILITATING INNOVATION IN THE FEDERAL STATISTICAL SYSTEM to enhance the quality of information and limit respondent burden. These actions have been limited in scope, however. In addition, two major changes to the portfolio of major federal household surveys are underway. The American Community Survey is intended to replace the long-form decennial census starting in 2010. . . . Officials are also rede - signing the Survey of Income and Program Participation which is used in estimating future costs of certain government benefit programs. In light of these upcoming changes, OMB recognizes that the federal gov - ernment can build upon agencies’ practices of reexamining individual surveys. To ensure that surveys initiated under conditions, priorities, and approaches that existed decades ago are able to cost-effectively meet current and emerging information needs, there is a need to undertake a comprehensive reexamination of the long standing portfolio of major federal household surveys. The Interagency Council on Statistical Policy (ICSP), which is chaired by OMB and made up of the heads of the major statistical agencies, is responsible for coordinating statistical work and has the leadership authority to undertake this effort. The critical challenge in this situation, Parker noted, is for the sta- tistical system to innovate, to move beyond its traditional stovepipe approaches to integrated data collection approaches across agencies. As Robert Groves noted, the traditional stovepipe approach to surveys may not be adequate for the future, and escalating costs will prevent agencies from continuing business as usual. The effort to move away from dupli- cated data collections toward integrated, more cost-effective ones is one of the major challenges for the statistical system. As the GAO report states, innovation is required to standardize definitions, reduce respondent bur- den, and increase the use of administrative records. The 2020 Decennial Census Parker also discussed the decennial census in his paper, noting that it is the only data collection required by the U.S. Constitution. The decen - nial census has also been the focus of numerous studies by the National Research Council on how to improve the data collection and make it more cost-effective, most recently looking at planning for the 2020 census (National Research Council, 2010). In an article highlighting the con- clusions of the National Research Council report, panel chair Lawrence Brown emphasized several challenges, including the exceptionally high cost of the 2010 census and growth of these costs relative to those of recent Canadian censuses; the continuing social and technological changes in the United States; the need for a focused research and development program for 2020 census planning; and reaction to the report by the Census Bureau staff (Brown, 2010). With regard to the later, he wrote (p. 31):

OCR for page 9
19 SCOPE AND IMPORTANCE OF INNOVATION We are heartened by the positive reaction of the Census Bureau to our panel’s report and by the concrete steps that the Census Bureau is taking to begin 2020 census planning now, with the development of a small number of visions of alternative ways of conducting the census and plans for R&D beginning in 2011-2012. R&D focused on these alter- natives could lead to more cost-effective ways of updating the Master Address File (to the benefit of the [American Community Survey] and other household surveys in addition to the census); the strategic use of the Internet and other response modes to save paper and improve data quality; the possible use of administrative records in nonresponse follow-up operations; and the full implementation of hand-held tech - nology for a “paperless” census. CROSS-CUTTING ISSUES In his background paper, Habermann (2010a) took a different approach to the issue of delineating the scope of the innovation problem. Rather than concentrating on specific examples, such as measurement of economic welfare, he focused on areas that cut across all the statistical agencies. The areas that he asserted warrant more attention include • ethods to improve response; m • mproving small-area estimates through models; i • etter metrics to understand the relationship between nonresponse b and bias; • se and production of synthetic data and a better understanding u of their strengths and limitations; • etter understanding of the likelihood of disclosure under different b sets of disclosure rules; • mprovements in editing and imputation; i • efining disclosure rules to foster release of small-area data; r • nderstanding the impact of communication technologies on the u dissemination strategies of agencies; • elping the public understand how to interpret official and nonof- h ficial statistics; • orking with researchers in other disciplines to understand how w to adapt advances in self-tracking technologies (the use of smart phones, other electronic equipment, and software applications to track and transmit detailed information on daily living) to sample survey design and operations; • dapting to the demands of the political community for more a timely data that are “good enough”; and • ntroducing multimode collection approaches in order to reduce i costs and cater to respondent preferences, while understanding and neutralizing mode effects.

OCR for page 9
20 FACILITATING INNOVATION IN THE FEDERAL STATISTICAL SYSTEM Habermann’s paper notes that many of these are traditional prob - lems in sample survey methodology, whereas others are created by new dynamics in both society and technology. He pointed out that some ana- lysts have gone further to assert that it is not only specific areas that are ripe for innovation, but also that the survey research field itself is decades behind other disciplines. For example, he quoted Robert Fay (2009, p. 845), who stated 1. that research on memory has achieved a significant body of findings; 2. that the results show potential promise for understanding aspects of the limitations of survey research and the potential for improving it;10 and 3. that the survey research community appears largely unaware of most of these developments. WORKSHOP DISCUSSIONS The examples in the Habermann and Parker papers were developed not to provide a complete list of issues and problems, but instead to illus - trate both the critical need for innovation and the breadth of the innova - tion that is needed. Lynda Carlson (National Science Foundation) agreed with Parker about the importance of innovation in providing policy-relevant data, stating that innovation is important in order for the statistical system to get the right data for policy makers. More than the need to provide policy- relevant data, she asserted, innovation is needed to shake up moribund agency processes and surveys. In discussing the need for innovation, Groves said that escalating costs are driving the need for innovation. The costs of surveys have been rising, and the costs are expected to continue to rise as nonresponse rates increase. He argued that these escalating costs will not be tolerated by tax- payers in the future, so that the federal statistical system will not be able to continue its current business model of surveys with current methods. Nancy Gordon (U.S. Census Bureau) noted that statistical agencies have and will continue to have real competition from other sources of information. Although this nonfederal information is generally of very low quality, the information is provided more quickly than traditional surveys, and many users are not focused on quality. She stressed that the federal statistical system must innovate in order to survive. Richard Newell (Energy Information Administration) discussed the 10For example, Fay points out that some survey questions tax the limits of memory, and the potential benefits of framing problems of recall in terms of current research on memory should seem self-evident.

OCR for page 9
21 SCOPE AND IMPORTANCE OF INNOVATION provision of information as a public good, which is one of the most important functions of a statistical agency. He noted that if competitors to the federal statistical system can provide data more cheaply and more quickly—even if of lower quality—then the fundamental nature of infor- mation as a public good can be significantly affected. One of the suggested responses to the rising costs of traditional sur- vey methods is increased use of administrative records. Clyde Tucker (Bureau of Labor Statistics) pointed out that taking advantage of admin - istrative data not only involves understanding how to make linkages and the accuracy of administrative data, but also requires organizational change and innovation in the way statistical agencies carry out surveys. With respect to the need for innovation, Jennifer Madans (National Center for Health Statistics) argued that innovation involves not only big projects—tectonic shifts in the methods and processes of statistical agencies—but also the everyday processes of doing smaller things. Graham Kalton (Westat) agreed that innovation is important to pro- duce statistics in a cost-effective way. He also suggested that it is because the statistical system is mature that big innovation projects are needed. In the early days of the statistical system, innovation was comparatively easy. Now, for example, improvements in the consumer price index are an enormous undertaking, and needed changes require substantial innova- tion projects. This point was also made in the paper by Dillman (1996). He noted that in the 20 years previous to 1996, fundamental change had come rapidly to survey methodology: for example, in 1976, mail surveys were treated as something to avoid, and telephone survey methods were not considered acceptable for any important surveys. Furthermore, although mixed-mode surveys were occasionally done, questions on whether people gave different answers using different modes were not seriously considered. Norman Bradburn (National Opinion Research Center) addressed the question of the importance of innovation by considering the workshop itself: Why is innovation the topic of an entire workshop? He answered that it is because the statistical system is mature, and people are looking for something transformative. What is needed is major innovation that will enable the federal statistical system to respond to the ever-growing challenges that it faces.

OCR for page 9