In this chapter, the committee summarizes the most important issues that emerged from the review of each of the seven Themes and apply broadly to the Strategic Plan as a whole. The committee also notes certain editorial issues observed among all Themes, for example, problematic or inconsistent use of terminology in some contexts (see Appendix C), and the style or order of the material presented. However, these issues are of secondary importance in our analysis and we thus refrain from further elaboration on these concerns; they can readily be addressed in a revised Strategic Plan. Instead, the following discussion focuses on concerns about more substantive elements of the Strategic Plan: vision statement, goals and objectives, research priorities and metrics, strategy for implementation, and National Program Office. The committee considers these to be typical elements of a strategic plan and the Interagency Working Group on Ocean Acidification (IWGOA)’s Strategic Plan would benefit from strengthening each of these elements and improving, where relevant, integration among them.
To aid in its review of the IWGOA’s Strategic Plan, the committee examined other examples of strategic research plans such as the U.S. Global Change Research Program’s (USGCRP’s)1 strategic plans (Climate
1 The U.S. Global Change Research Program (USGCRP) is a multiagency program formed in response to the Global Change Research Act of 1990. The USGCRP has a national program office and dedicated national program office staff to manage the process of coordination across the agencies, the periodic assessment of the research results, and the strategic planning process. The USGCRP was called the U.S. Climate Change Science Program under the G. W. Bush Administration, but is again called the USGCRP under the current administra-
Change Science Program Strategic Plan [CCSP] 2002, USGCRP Strategic Plan 2012) along with a National Research Council review of the CCSP’s 2002 strategic plan (NRC, 2004). Strategic plans have become a requirement of the GPRA Modernization Act of 2010 and the Office of Management and Budget provides guidance for preparing and submitting an agency strategic plan (OMB Circular No A-11 2011). Based on these documents, the committee found that the following elements are generally included in a federal strategic research plan: a vision or mission statement, goals and objectives, research priorities or criteria for setting priorities, metrics for evaluation of progress and success, and a strategy for implementation of the goals and objectives of the plan. In the context of this last element, the committee is cognizant of the importance of distinguishing a strategic plan from an implementation plan, but also recognizes that there is overlap in content between a strategic and an implementation plan. Thus, several of the committee’s analyses center on the issue of how the Strategic Plan can lay out a process to ensure that the different objectives can most effectively be implemented.
As follows, the committee discusses the IWGOA’s Strategic Plan vis-à-vis the five elements of a strategic plan. The chapter concludes with a discussion of the National Ocean Acidification Program and the National Ocean Acidification Program Office, the latter of which is a critical conduit to the implementation of the Strategic Plan.
The committee found no clear ‘vision’ or ‘mission’ statement in the Strategic Plan. OMB Circular A-11 (see Section 210, page 2) specifies the structure of a strategic plan and is concerned with how a strategic plan reflects the ‘vision’ or ‘mission statement’ of an individual agency. The committee acknowledges the difficulty in framing a single mission statement that reflects the diversity of agencies involved, each representing a different mission. However, the committee believes that a concise vision or mission statement of the type found in the USGCRP Strategic Plan2 (2012; page 11) would assist, in particular, the nonspecialist reader in appreciating the thrust and importance of the Strategic Plan.
RECOMMENDATION: The Strategic Plan should include a vision statement for the National Ocean Acidification Program.
tion. This program and its origin are similar to the National Ocean Acidification Program; and thus, serves as a good example for how a strategic plan and associated metrics can be developed.
2 “Vision: A Nation, globally engaged and guided by science, meeting the challenges of climate and global change.” (USGCRP Strategic Plan, 2012)
The ten-year Strategic Plan prepared by the IWGOA is comprehensive in describing critical research and monitoring goals for the near- and long-term for seven different, but often closely interrelated, Themes. The Themes of the Strategic Plan include the five Program Elements required by the FOARAM Act and two additional Themes that are critical for the success of an overall research and monitoring program in ocean acidification. Each Theme in the Strategic Plan presents goals and objectives with a description of the rationales for their inclusion in the Plan. The committee’s review of the goals and objectives for each Theme is presented in detail in Chapter 3. At this juncture, the committee emphasizes two overarching points related to goals and objectives that apply to all Themes and are of key importance for the Strategic Plan.
First, the goals and objectives could be strengthened if they included a more integrated and comprehensive treatment across themes. This is particularly true in the context of how the natural and social sciences and monitoring components of the overall program will inform important societal questions such as food security and conservation. As discussed further in the context of the individual Themes, advancing the social sciences (e.g., research on socioeconomic impacts, adaptation, and conservation) cannot be placed on hold while the natural sciences and monitoring progress. Rather, socioeconomic goals and objectives need to be an integral part of the focus at the same time and coordinated with the natural science and monitoring efforts. This concern can be addressed by including in the appropriate Theme sections how the goals of each Theme relate to the social sciences (e.g., socioeconomic impacts, adaptation, and conservation strategies).
Second, the committee identified another common shortcoming in the Plan’s descriptions of goals: the absence of clear priorities (or a strategy for establishing priorities) for undertaking the varied efforts needed to implement the goals and attain the Plan’s objectives. This concern is discussed in the following section.
RECOMMENDATION: The Strategic Plan should describe a process that can ensure integration across the themes, coordination among the agencies, and development of priorities.
A strategic plan ideally defines the process by which priorities will be set. The long list of research and monitoring goals in the Strategic Plan is divided into short-term and long-term goals for each Theme, but does not prioritize the goals or provide criteria for doing this. Examples of
criteria include scientific and societal importance (both intrinsic to one Theme and interrelated with other Themes), logic of research sequence, evaluation of the costs and benefits, and the availability of human expertise, research infrastructure, and funding. Many frameworks for setting research criteria are available in published reports.
A National Research Council review of Charting the Course of Ocean Science in the United States for the Next Decade: An Ocean Research Priorities Plan and Implementation Strategy (NRC, 2007) proposed the following questions to identify its priorities:
• Is the proposed research transformational (e.g., will the proposed research enable significant advances in insight and application, even with potentially high risk for its success; would success provide dramatic benefits for the nation)?
• Does the proposed research impact many societal theme areas?
• Does the research address high-priority needs of resource managers?
• Would the research provide understanding of high value to the broader scientific community?
• Will the research promote partnerships to expand the nation’s capabilities (e.g., contributions from other partners, including communities outside of ocean science, such as health science; unique timing of activities)?
• Does the research serve to contribute to or enhance the leadership of the United States in ocean science?
• Does the research contribute to a greater understanding of ocean issues at a global scale?
• Does the research address mandates of governing entities (federal agencies; state, tribal, and local governments)?
As indicated above, clear societally relevant objectives could provide one important focus for development of a framework for the National Ocean Acidification Program. Such a framework could help set priorities of the social and natural science research to inform the development of solutions to problems (see Box 2.1 for an example). Framing research goals through the lens of societal needs when relevant could also help determine the appropriate allocation of resources across the different Themes and research goals.
Setting priorities is not only important to ensuring that important societal needs are met, it is also critical when the broad goals of the Strategic Plan are juxtaposed with the realities of federal funding. Although the FOARAM Act specifies a ten-year program, the Plan describes the fiscal resources available to achieve its goals only for the President’s budget for a single year, FY 12. As Figure 2.1 reveals, the majority of the funding allocation is focused on Theme 1 (monitoring) and Theme 2 (research on ocean acidification impacts), $9.65 million and $14.43 million respectively.
An Example for Setting Priorities Based on Societal Relevance
The Strategic Plan might postulate ‘food security’ as an important societal question and regard research focused on answering this question to be of high priority. For example, how can the U.S. ensure the future of a sustainable source of domestic seafood of high quality and affordability? This question would require thinking about the role of commercially wild-caught and aquaculture sources of seafood and the different potential impacts that ocean acidification would have on these sources. While natural scientists would strive to improve the understanding of direct and indirect impacts of ocean acidification on the biology of these fisheries (as outlined in Themes 1, 2, and 3), social scientists and economists would want to consider the societal preferences for the relative contribution of both sources (given their environmental footprints) to supply healthy and affordable seafood (as part of Theme 3 and 5).
In the wild-caught fishery, the indirect effects (e.g., less primary prey available through food webs) are likely to be more significant relative to the direct effects (e.g., higher natural mortality rates) and information will be needed on a scale commensurate with the ecosystem and management. These impacts could lead to a delay in the recovery of a particular fish stock. Thus, understanding management practices that could incorporate information about the direct and indirect environmental impacts on the rebuilding of fish stocks would emerge as a high priority research goal.
In the aquaculture setting, the research could focus on the direct effects on farmed species or in developing predictions of acidity on a local scale that can help aquaculture operations avoid exposing their stocks to unfavorable conditions. If it is the latter, scientists might need to advance the understanding of how accurate the predictions need to be. That is, the societal gain from better forecasts could be marginal if the increased precision does not translate into improved aquaculture practices and aquaculture yield.
Theme 3 (modeling) and 4 (technology development and standardization of measurements) receive funds on the order of $2-3 million, whereas Themes 5 and 6 are each funded below $1 million. In fact, if funds available for research on the socioeconomics and adaptation measures remain at their current levels, only a single small research project could be supported for a year; this is unlikely to generate the kind of innovation required to develop ways to adapt to the impacts of ocean acidification. Similarly, budget allocations of $152,000 for data management will not allow for a substantial effort. Input from the research community and other stakeholders could help facilitate future adjustment of the total resources available for the seven Themes as well as define a process for prioritizing among the Themes.
If the IWGOA aims to achieve all goals outlined in its Strategic Plan
FIGURE 2.1 Funding levels in the President’s FY 2012 Request and associated with each Theme of the Strategic Plan. Theme 1: Monitoring of Ocean Chemistry and Biological Impacts; Theme 2: Research to Understand Responses to Ocean Acidification; Theme 3: Modeling to Predict Changes in the Ocean Carbon Cycle and Impacts on Marine Ecosystems and Organisms; Theme 4: Technology Development and Standardization of Measurements; Theme 5: Assessment of Socioeconomic Impacts and Development of Strategies to Conserve Marine Organisms and Ecosystems; Theme 6: Education, Outreach, and Engagement Strategy on Ocean Acidification; Theme 7: Data Management and Integration.
by the end of the 10 years, new resources would be needed (see Box 2.2). However, constrained federal budgets might make it difficult to mobilize resources at the required level, therefore requiring the IWGOA to set some priorities among the listed goals. This potential scenario further illustrates the benefit of describing a clear process for setting priorities. Such a process could be facilitated, at least in general terms, by information about the likely costs of the activities associated with different goals. Using such information, the feasibility of pursuing the listed goals within a short or long time frame could be assessed. In setting priorities, it will be critical to establish a process that can engage the views of the extramural research community and the stakeholders in general. In addition, the committee concludes that a National Program Office (see next section) would be critical in the process for setting priorities using criteria set forth in the Strategic Plan. To inform this process, the National Program Office could convene representatives from the extramural research community (for
example, through an external advisory committee including social and natural scientists).
The committee was also asked to consider, as part of its review, the metrics3 that would be employed to evaluate progress toward the goals outlined in the seven Themes. The thematic sections in the Strategic Plan do not provide an explicit description of the metrics for evaluation, thus the committee found this component of its task particularly challenging.
Since issuance of the Government Performance and Results Act (GPRA) of 1993 and related Office of Management and Budget (OMB) policies, agencies have increased efforts to establish metrics and track progress (NRC, 2005). However, it is often difficult to evaluate research using strictly quantitative measures because the discovery process is complex and outcomes that matter to society are not always traceable to specific projects; these achievements often result from a combination of research findings and their use in formulating policy. Most agencies, therefore, rely on expert peer review to assess progress in research (NRC, 2005). Performance measures are also agency-specific and may include qualitative outputs or outcomes (see Box 2.3 for additional details). For example, NOAA’s National Marine Fisheries Service might use the reduction in numbers of overfished major stocks while NSF might need more general input or output metrics. Therefore, a broad range of metrics need to be used and they need to be tailored to the specific research goals and agency missions.
The FOARAM Act requires the National Ocean Acidification Program to provide biennial progress reports and a revised 10-year plan every 5 years. This requirement dictates that progress of the various elements of the program be evaluated. To this end, it is important that metrics be established to measure progress toward the Strategic Plan’s goals, and that these evaluations of the program be used to set priorities in an iterative fashion. The committee recognizes the difficulty in establishing a set of specific metrics for scientific research. It is also recognized that the Strategic Plan is not an implementation plan, which would be a more appropriate document to provide a detailed description of metrics for evaluation. However, there are many reports (NRC, 2005) that provide guidance on developing such performance measures. For example, the committee finds that many of the suggested metrics for the former CCSP, now referred to as the USGCRP, could be applied to the Ocean Acidification Program (NRC, 2005).
Lastly, the issues of “metrics” and “prioritization” are closely linked. As stated in the 2005 NRC Report, Thinking Strategically: The Appropriate
3 Government agencies use “metrics” (synonymous with “performance measures”) to assess progress toward pre-established goals (NRC, 2005).
Federal Funding Needs for Ocean Acidification (based on a report by the National Marine Sanctuaries Foundation)
A report by the National Marine Sanctuaries Foundation (NMSF) assesses whether the current investments in ocean acidification research are commensurate with the urgency and magnitude of the challenge. The NMSF’s report compares current funding levels with two estimates of the funding needed for ocean acidification research and monitoring (see Figure). One such estimate is based on informal conversations with program managers at various federal agencies (red squares) and a second set of estimates was provided by the Ocean Carbon and Biochemistry (OCB) program’s white paper. In this white paper, OCB estimates that a U.S. National Ocean Acidification Program would need $50-100 million per year to provide timely information for decision-makers (blue circles).
The green squares in the figure display actual combined funding spent on ocean acidification related research by several federal agencies (i.e., EPA, MMS/ BOEM, NASA, NOAA, USGS, NSF) (IWGOA, 2011; IWGOA report draft). NOAA and NSF’s combined actual spending exceeds what the FOARAM Act authorized to be appropriated for these two agencies. The FOARAM Act authorizes the following sums to be appropriated to NOAA: $8 M for FY10; $12 M for FY11; $15 M for FY12; and $20 M for FY13. To NSF, the FOARAM act authorized $6 M for FY10; $8 M for FY11; $12 M for FY12; and $15 M for FY13. However, what Congress appropriated for each year (NOAA, for example, $5.5 M for FY10, $6.3 M for FY11, $6.2 M for FY12) was significantly lower than what they were authorized to appropriate.
Use of Metrics for the Climate Change Science Program, it is important to realize “the potential to use metrics not just as simple measures of progress, but as tools to guide strategic planning and foster future progress.” Evaluating progress and setting priorities both benefit from using an expert peer-review process.
RECOMMENDATION: The Strategic Plan should define a process by which the goals will be prioritized, because not all the goals listed in the Strategic Plan are equally important for achieving the scientific mandates of the FOARAM Act. These priorities should be re-evaluated as part of the 5-year revision of the Strategic Plan.
SOURCE: Reprinted with permission from Cutting Edge Design (from NMSF, 2012).
The Strategic Plan is lacking detail about strategies for implementation with the exception of mentioning that the National Ocean Acidification Program Office “serve the vital role of developing and executing an implementation plan.”4 Elements to be described in a strategy for implementation may include criteria and process for prioritization (see above), individual priorities and responsibilities for each agency, a process for ensuring coordination and integration across the Themes and research disciplines, and an approach for establishment of the National Program Office. Without a clear definition of the structure of the National Pro-
4 IWGOA Strategic Plan, pg. 3.
The NRC report Thinking Strategically categorizes metrics in the following way (and uses the discovery of the Antarctic Ozone hole as an illustrative example):
“Process—a course of action taken to achieve a goal. (Example metrics include existence of a project champion and length of time between starting the research and delivering an assessment on stratospheric ozone depletion to policy makers.)
Input—tangible quantities put into a process to achieve a goal. (An example input metric is expenditures for (a) theoretical and laboratory studies on ozone production and destruction, (b) development and deployment of sensors to sample the stratosphere, (c) modeling and analysis of data, or (d) meetings and publications.)
Output—products and services delivered. (Examples of output metrics include number of models that take into account new findings on chlorofluorocarbon chemistry or number of publications and news reports on the cause of stratospheric ozone depletion and its possible consequences.)
Outcome—results that stem from use of the outputs. Unlike output measures, outcomes refer to an event or condition that is external to the program and is of direct importance to the intended beneficiaries (e.g., scientists, agency managers, policy makers, other stakeholders). (Examples of outcome metrics are the number of alternative refrigerants introduced to society to reduce the loss of stratospheric ozone and scientific outputs integrated into a new understanding of the causes of the Antarctic ozone hole.)
Impact—the effect that an outcome has on something else. Impact metrics are outcomes that focus on long-term societal, economic, or environmental consequences. (Examples of impact metrics include the recovery of stratospheric ozone resulting from implementation of the Montreal Protocol and related policies and the increase in public understanding of the causes and consequences of ozone loss.)”
SOURCE: NRC, 2005.
gram Office and the functions it will provide, the implementation of this Strategic Plan might be delayed and made more difficult.
The Strategic Plan lacks a description of the appropriate roles of the various Federal agencies in implementing the specified goals. While this kind of specificity would seem to belong in an implementation plan, such a description of how the federal agencies can strategize to leverage resources and find synergies given their complementary missions would increase the likely success of the implementation of the Strategic Plan.
RECOMMENDATION: If the Strategic Plan is to “establish the National Ocean Acidification Program” as instructed by the FOARAM Act, it should strengthen the discussion of the National Program and its mission and, as stressed in the section below, provide greater details on how the National Ocean Acidification Program and the Program Office will be established.
The committee is in full agreement with the Strategic Plan’s assessment that a National Program Office is essential to ensure that ocean acidification research is well coordinated across all the participating federal agencies, and to provide a national voice for effective international cooperation. In addition, the committee feels strongly that the National Program Office should not be perceived as being associated with a single federal agency and that (ultimately) coordination would be improved if it was the sole U.S. office responsible for coordinating ocean acidification activities. The National Program Office could be part of the federal government structure, similar to the U.S. GCRP, or be modeled after program offices such as the GLOBEC (Global Ocean Ecosystem Dynamics) or U.S. JGOFS (Joint Global Ocean Flux Study) program offices that reside within academic institutions. The committee also endorses the statement in the Strategic Plan that the National Program Office be subject to oversight from the IWGOA. In addition, the committee concludes that it is equally important that a process be established for the National Program Office to receive external advice, such as an external scientific steering committee, that is focused explicitly on the activities of the office.
The committee recognizes that the costs of setting up and operating such an office require resources that might otherwise be directed at research. The currently limited budget dedicated to the National Ocean Acidification Program exacerbates this drain on dedicated funds for ocean acidification research. However, the committee views establishing an office as an important investment in ensuring that the U.S. has an effective National Ocean Acidification Program. This is particularly important given the broad array of Themes and assuming that future budgets of the National Ocean Acidification Program will grow. Strong coordination across the Themes and between agencies through a single National Program Office will maximize efficiency and avoid wasting resources through needless duplication of activities. The cost could potentially be minimized if it remained part of the federal government, with agencies lending staff time to such an office. Furthermore, its staff size could be adjusted based on the size of the overall National Ocean Acidification Program. Alternatively, if the National Program Office is to reside at an
academic institution, it might be collocated with an already existing program office. In addition, the National Program Office will—through the various activities outlined on pg. 9 of the Strategic Plan—ensure that there is continuing effective community input into the evolving strategies and goals as the Program moves forward.
In developing the design of a National Program Office in the Strategic Plan, the IWGOA can draw on many lessons learned from previous, highly successful programs (and program offices) such as the GLOBEC Program and the U.S. JGOFS Program (NRC, 2010) and federal program offices such as the USGRCRP. Based on lessons learned from previous national research programs, the following functions are fundamental for a successful program office:
• coordinating research across the federal and state agencies;
• coordinating activities with international efforts;
• engaging the broader stakeholder community in developing research priorities and metrics for evaluating progress;
• communicating important results among agencies, policy makers, stakeholders, and the public; and
• ensuring that the nation receives the highest return from its investment in the National Ocean Acidification Program.
RECOMMENDATION: The Strategic Plan should provide clarity regarding the creation, function, and governance of the National Program Office.