National Academies Press: OpenBook
« Previous: 2 AGENCY METHODS
Suggested Citation:"3 COMMUNICATION ISSUES." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2001. Implementing the Government Performance and Results Act for Research: A Status Report. Washington, DC: The National Academies Press. doi: 10.17226/10106.
×
Page 27
Suggested Citation:"3 COMMUNICATION ISSUES." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2001. Implementing the Government Performance and Results Act for Research: A Status Report. Washington, DC: The National Academies Press. doi: 10.17226/10106.
×
Page 28
Suggested Citation:"3 COMMUNICATION ISSUES." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2001. Implementing the Government Performance and Results Act for Research: A Status Report. Washington, DC: The National Academies Press. doi: 10.17226/10106.
×
Page 29
Suggested Citation:"3 COMMUNICATION ISSUES." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2001. Implementing the Government Performance and Results Act for Research: A Status Report. Washington, DC: The National Academies Press. doi: 10.17226/10106.
×
Page 30
Suggested Citation:"3 COMMUNICATION ISSUES." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2001. Implementing the Government Performance and Results Act for Research: A Status Report. Washington, DC: The National Academies Press. doi: 10.17226/10106.
×
Page 31
Suggested Citation:"3 COMMUNICATION ISSUES." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2001. Implementing the Government Performance and Results Act for Research: A Status Report. Washington, DC: The National Academies Press. doi: 10.17226/10106.
×
Page 32
Suggested Citation:"3 COMMUNICATION ISSUES." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2001. Implementing the Government Performance and Results Act for Research: A Status Report. Washington, DC: The National Academies Press. doi: 10.17226/10106.
×
Page 33
Suggested Citation:"3 COMMUNICATION ISSUES." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2001. Implementing the Government Performance and Results Act for Research: A Status Report. Washington, DC: The National Academies Press. doi: 10.17226/10106.
×
Page 34

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

C H A P T E R 3 COMMUNICATION ISSUES ne of the challenges to agencies and oversight bodies is to interpret O the law itself in ways that achieve the desired results of accurate reporting and accountability. The language of the law is general; it addresses the agencies as a single population without distinguishing among them. As mentioned above, however, the variations in structure and function of agencies are considerable. For the most part, individual agencies have been left with the task of working out for themselves the best way to interpret such terms as “program activity,” “performance indicator,” and “program result” within their particular structures. In the absence of detailed, continuing discussions among the creators of the law, oversight bodies, and agencies, the agencies have little guidance on the best ways to apply such terms to existing agency procedures and research programs. In addition to its com- ments on these communication difficulties, the panel offers several specific observations about the issue of oversight. The issue of communication is fundamental to the imple- mentation of GPRA. That is, the essential objective of the law, in the context of research, is to improve the management of govern- ment services. And a key element of that management is communi- cation of federal agencies with oversight bodies about the scientific and energy research needs of the nation as a whole. To achieve that objective, long-standing terms and customs of the agencies must be translated and communicated so that they are clear outside the agencies. If this can be achieved, internal “language” and GPRA 27

IMPLEMENTING THE GOVERNMENT PERFORMANCE AND RESULTS ACT FOR RESEARCH “language” can be reconciled toward the dual goals of facilitating congressional budgeting activities and enhancing the effectiveness and efficiency of agency management. This chapter examines the communication between the agencies and the primary audiences for its GPRA reports: oversight groups, the users of research, and the public. As was the case with the previous chapter, the observations here are based on the panel’s focus groups and workshop where agency and oversight group representatives discussed agency responses to the following questions: • How is the result communicated to different audiences (e.g., S&T community, advisory committees, agency leadership, administration, Congress)? • How is the result used in internal and external decision- making? The agency responses are summarized in Appendix C and the workshop discussion is provided in Appendix D. The viewpoints of Congress, GAO, OMB, and other entities interested in the implementation of GPRA vary with their specific charges. In general, however, all of them have expressed a desire to know more about: • What procedures the federal agencies use to comply with GPRA; • How successful those procedures are; and • How the GPRA planning and reporting processes can serve agency missions and the public interest better than is avail- able in the existing documentation. 28

Communication Issues The panel’s discussions with agency and oversight repre- sentatives made it clear that communication between agencies and oversight groups must be more regular, extensive, and collaborative to facilitate agency responses to GPRA. Participants in the focus groups and workshop emphasized that improved communication about methods could hasten the implementation of GPRA, increase its value as a planning and accountability tool, and reduce the cost of compliance. One common complaint from agencies is that oversight bodies are quicker to criticize shortcomings than to suggest im- provements or specify desired outcomes. The most constructive course would be for oversight bodies to suggest how agencies can use procedures already in place without adding additional steps. GPRA should not “make extra work,” and oversight bodies should be willing to work with agencies to ensure that this does not happen. The public benefit of such a course is to eliminate unnec- essary cost and duplication of effort. As mentioned above, one objective of GPRA and of the oversight bodies is to clarify the mechanisms used by agencies to validate and verify their evaluation procedures. For example, oversight representatives would like to be assured that the review- ers of research programs are objective, experienced, and expert— again, a communication issue. Oversight bodies have expressed an inability to see or understand how those qualities are validated by agencies, and they have asked for improved communication about the procedures. In addition, agencies sometimes receive conflicting signals from oversight groups. Even from a single oversight entity, they might receive different guidance from different staff members. One agency revised its accounting to link GPRA reports more tightly with its budgets, for example, only to be told by a congressional oversight committee to return to the previous format to which the committee was accustomed. Further, different congressional committees prefer differ- 29

IMPLEMENTING THE GOVERNMENT PERFORMANCE AND RESULTS ACT FOR RESEARCH ent levels of aggregation. Some prefer a high level, in which many or all research programs are considered together; others prefer a disaggregated approach. This presents a confusing picture to agencies. Most importantly, agencies have not yet seen the use of their reports in the congressional decision-making that determines the size and form of their budgets. That could reduce the incentive of agencies to integrate their own planning and budgeting functions with the requirements of GPRA. Without such integration, agencies duplicate their reporting efforts to serve internal budgeting func- tions and GPRA requirements. During focus groups, the workshop, and interviews, it was consis- tently clear that improved communication between these two sectors could reduce the difficulties and misunderstandings experi- enced by some agencies. Agencies should provide brief, clear summaries of the procedures by which they perform expert review, aggregate programs, validate evaluation methods, set research priorities, and include user groups and other members of the public in planning and reporting. That simple step would allow a clearer view of the links between GPRA documents and agencies’ internal procedures. User groups, as described above, represent important segments of the public that are served by publicly funded research. They have important roles to play in planning and evaluating research programs. Some agencies make good use of such groups on 30

Communication Issues panels and review committees, but their activities are seldom made clear in GPRA documents or to oversight groups. Agencies can demonstrate the value and operation of their review processes better by publicly describing them to oversight groups, the poten- tial users of research results, and the general public. Good oversight need not impede agencies that are making a sincere effort at compliance. Oversight activity is likely to be lowest for agencies that have advanced in their efforts to comply with GPRA and greatest for those still struggling. Agencies, for their part, should strive to make the research clear to nonscientists among oversight bodies. They should consult, as requested, on practical ways to integrate agencies’ internal planning and reviewing practices with GPRA requirements. They should also meet more often among them- selves to coordinate their expectations of agency practices. One objective of the law is to encourage the integration of program activities and strategic planning. With or without GPRA, in fact, each agency can benefit from reviewing its research programs 31

IMPLEMENTING THE GOVERNMENT PERFORMANCE AND RESULTS ACT FOR RESEARCH in the light of how they and their individual projects serve the broader strategic plan. The strategic plan, in turn, should evolve year by year in view of the changes that are made in individual programs and projects. More effective use of strategic planning can allow oversight groups to understand the contribution of individual programs to an agency’s mission and hence improve agency- oversight communication. Although agency representatives expressed enthusiasm for using the criterion of quality to evaluate research on a regular basis, they voiced concern over the requirement to provide annual reports for their basic research programs. As explained earlier, basic research often does not produce useful results in a single year and must be monitored over several years before outcomes become apparent. A particular concern on the part of agency representatives is that programs and individual researchers might feel pressured to produce evidence of annual achievement in the form of “extra” publications or other meaningless metrics. Such activities would waste valuable resources and distract researchers from productive work. Because the value of investments in basic research can be evaluated only over long periods, retrospective methods might be more effective than annual reports. For example, NSF is experi- menting with “rolling” assessments whereby one-third of the portfolio is evaluated each year. Every research project is thus evaluated every 3 years, a reasonable period in which to expect results. 32

Communication Issues NSF suggests that that method could be useful for other agencies and that the 3-year focus be applied to performance plans, as well as performance reports. Thus, an agency would set 3-year performance targets, rather than annual performance targets, on research goals. Performance reports would still be annual, but they would cover one-third of the portfolio each year. They would also describe trends in the direction of basic research, the rate of progress of research, and the productivity of special initiatives. Management goals and short-term objectives in applied-research programs, where targets are more easily calibrated and predicted, would still be described annually in performance plans and reports. A potential benefit of GPRA is the ability to strengthen agencies’ planning procedures by making available the research results of previous years. Because of timing requirements built into the legal guidelines of GPRA, agencies find that they must begin work on future performance plans before the most recent perfor- mance reports are available. For example, in November 2000, one agency was beginning its performance plan for 2002 before it had finished its performance report for 2000. One reason for the difficult timing is that the act was designed to enable oversight groups to connect each performance plan and performance report directly with its corresponding annual budget. The timing is unfortunate for several reasons. Agencies and researchers need the flexibility to change the course of a research project if change is warranted by previous and current results. And neither agencies nor the public receive a benefit when agencies create detailed performance plans before they have sufficient recent information on the performance of current programs. 33

IMPLEMENTING THE GOVERNMENT PERFORMANCE AND RESULTS ACT FOR RESEARCH The schedules should allow agencies to use previous results when preparing performance plans. Agencies should also continue their efforts to integrate GPRA planning and evaluation procedures into current agency processes. Specifically, the panel suggests that agencies engaged in basic research make 3-year performance plans and set 3-year performance targets for research goals in their performance plans, rather than targets that refer to particular fiscal years. Management goals and short-term objectives in applied-research programs should still refer to a 1-year period. Performance reports should be annual and stress trends and indicators of the direction of basic research and the level of progress and productivity of special initiatives. Communication between agencies and oversight bodies is essential to making the GPRA process work. So far, the communi- cation process has been flawed from the viewpoint of both sides. If agencies are clearer regarding their methodology and oversight groups are clearer and more consistent regarding their expectations, a better and more useful product will result. 34

Next: 4 Conclusions and Recommendations »
Implementing the Government Performance and Results Act for Research: A Status Report Get This Book
×
Buy Paperback | $60.00 Buy Ebook | $47.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

As requested by Congress and the White House Office of Science and Technology Policy (OSTP), this report assists federal agencies in crafting plans and reports that are responsive to the Government Performance and Results Act (GPRA), OMB Guidance, and agency missions. Using a case study approach, the report identifies best practices used by individual agencies to evaluate the performance and results of their science and technology programs. The report takes into account individual agencies' missions and how science and technology programs and human resource needs are factored into agency GPRA plans. Specific applications of recommendations are included from COSEPUP's earlier report entitled Evaluating Federal Research Programs: Research and the Government Performance and Results Act.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!