The last workshop session was devoted to possible next steps. The participants were invited to identify, in view of the previous discussion, ways that innovation could play a more prominent role in the federal statistical system.
Katherine Wallman began the discussion with an overview of several important points made throughout the workshop discussions as well as her own views:
It is important to recognize the idea of Robert Groves’s “creative misfits,” who can produce new ideas.
The federal statistical system is a mature system, and change may require major initiatives, as pointed out by Graham Kalton.
Although administrative records show great promise and are widely believed to be critical for future data collections, their use always seems to be a year away, as John Haltiwanger noted.
Sometimes the problem in fostering innovation, as Richard Newell said, is a lack of ideas, and sometimes it is the inability to implement new ideas.
Competition can be an impediment to innovation, as John Thompson (National Opinion Research Center) noted; collaboration is critical.
A key to innovation is the willingness of the senior managers present at the workshop to provide the necessary leadership and to follow through on the ideas discussed at the workshop.
The Office of Management and Budget is responsible for providing leadership in eliminating bureaucratic barriers in contracting and recruitment.
A system-wide marketing plan to academic institutions could stimulate academic work on federal statistical problems.
Case studies of best practices could be useful in providing guidance on how to stimulate innovation.
Communication within and between agencies could be improved.
Progress in innovation needs to be measured periodically.
Edward Sondik followed Wallman’s overview with some ideas on next steps for the federal statistical system:
Annual or biannual reports on key innovations and research could be developed and disseminated.
Although he does not support a centralized research program, the federal statistical system could develop a joint federal statistics research agenda.
The Interagency Committee on Statistical Policy (ICSP) could take the lead in developing a marketing program with academic institutions.
The ICSP could provide leadership in establishing an innovation culture in the federal statistical system.
Thomas Louis stressed the importance of accountability and agreed with others on the importance of a periodic review and evaluation of statistical programs. He also made the general comment that it is important for at least a subset of the agencies to work on specific innovation projects while discussion proceeds on the larger issue of innovation in the federal statistical system. He stressed that he is not implying that innovation is not taking place, but perhaps the discussion today could lead to a different angle to implementing innovation projects. For example, he suggested looking at innovation in an existing area, like seasonal adjustment, that proceeds from databases to tables and figures with reasonably seamless connections. The project need not be a new procedure but an embodiment of many new changes. It is important for such a project to be examined by a variety of agencies to disseminate the lessons learned.
Lawrence Brown (University of Pennsylvania) asked Hermann Habermann (Committee on National Statistics), who was charged with writing the summary of the workshop, what he had heard. Habermann
commented that this could be a seminal moment for the federal statistical system—an opportunity for the system to consider how it wants to pursue research and innovation. He noted that many participants favored some cross-cutting centralized research approach, although with each agency having the ability to retain local creativity and focus. In this connection, he said, the statistical system could build on past successes of the Federal Committee on Statistical Methodology. He observed that many opportunities to encourage innovation had been suggested, such as the need to remove bureaucratic barriers to contracting and recruitment, but it is not clear that the system has the necessary will to follow through.
Much of the remaining discussion focused on the importance of leadership and the ICSP in particular, interagency efforts, and case studies and best practices.
Jennifer Madans asked if the system would be able to use this workshop on innovation and create a more interactive statistical system. She noted that much of what had been discussed was under the purview of the ICSP and that it needed different ways of communicating.
Groves said he supports the development of a system-wide research agenda that identifies common research problems, but he cautioned that addressing turf issues will be difficult. He said that it probably would be necessary for the ICSP to explore legal and sustainable limits of collaboration across agencies.
Although she supports efforts to blur the traditional bureaucratic boundaries, Madans said that thought was needed to ensure that smaller agencies would not get gobbled up by some of the bigger agencies.
Marilyn Seastrom offered an example of concrete cross-agency innovation, the Statistical Community of Practice and Engagement.1 Mark Harris (National Agricultural Statistics Service) mentioned another example, the Federal Committee on Statistical Methodology, although he noted that it is more involved with the documentation of best practices than with cutting-edge innovation. He pointed out that some of its projects have been incredibly productive as a result of the creativity and participation of staff from many agencies. He also commented that often the
See http://www.apdu.org/conference/2010/Bianchi_APDUPresentation9-16-10.ppt#260,6, SCOP Goals = OMB Goals [February 2011]. SCOP was subsequently renamed SCOPE, Statistical Community of Practice and Engagement.
problem with developing successful projects is not in getting the correct people, but in getting agencies to offer any people at all.
Nathaniel Schenker (National Center for Health Statistics) mentioned the importance of transferring methods and techniques invented at one agency to others. Wallman asked how one can institutionalize the idea that interagency collaborative work is part of what is expected of statistical agency staff, rather than something extra that is not important to their jobs.
John Eltinge emphasized the importance of ensuring that interagency initiatives resonate and be consistent with the mission of an agency’s department as well as appropriate congressional committees. In this connection, he suggested that since selling risk and cost may be difficult, the statistical system could consider framing initiatives as value added rather than as innovations.
In considering next steps to innovation, David Banks stressed the point made by Schenker about the importance of transferring innovative ideas from one agency to another. Nancy Gordon supported the concept but cautioned that to be successful it is necessary to deal with the “not invented here” syndrome.
CASE STUDIES AND BEST PRACTICES
Schenker returned to the idea of case studies and suggested that what is needed is a prestigious way to publish papers on case studies and innovative ways of using existing techniques. With respect to best practices, Roderick Little cautioned that best practices can be the opposite of innovation: a best practice may be considered the best thing to do—so why try something else?
In his closing comments, Louis returned to the reason that innovation is critical for statistical agencies at this time. One of the important reasons is that the assumptions and models on which the statistical system was built are changing. He made the analogy that, at one time, “Biostatistics Department” was the equivalent of a brand name for all things in biostatistics, but that is no longer true. This analogy holds for the federal statistical system: it is no longer the only place where federal statistics is done in every sense. Increasingly, for example, there are other sources for data. He suggested that although it might not be a sufficient step, the system may find it necessary to elevate the amount and visibility of innovation and research to maintain its brand name in federal statistics.