Assessment of Impediments to Interagency Collaboration on Space and Earth Science Missions (2011)

Through an examination of case studies, agency briefings, and existing reports, and drawing on personal knowledge and direct experience, the Committee on Assessment of Impediments to Interagency Cooperation on Space and Earth Science Missions found that candidate projects for multiagency collaboration¹ in the development and implementation of Earth-observing or space science missions are often intrinsically complex and, therefore, costly, and that a multiagency approach to developing these missions typically results in additional complexity and cost. Advocates of collaboration have sometimes underestimated the difficulties and associated costs and risks of dividing responsibility and accountability between two or more partners; they also discount the possibility that collaboration will increase the risk in meeting performance objectives.

This committee’s principal recommendation is that agencies should conduct Earth and space science projects independently unless:

• It is judged that cooperation will result in significant added scientific (and possibly follow-on opera­tional) value to the project over what could be achieved by a single agency alone; or

• Unique capabilities reside within one agency that are necessary for the mission success of a project managed by another agency; or

• The project is intended to transfer from research to operations necessitating a change in responsibil­ity from one agency to another during the project; or

• There are other compelling reasons to pursue collaboration, for example, a desire to build capacity at one of the cooperating agencies.

Even when the total project cost may increase, parties may still find collaboration attractive if their share of a mission is more affordable than funding it alone. In these cases, alternatives to interdependent reliance on another government agency should be considered. For example, agencies may find that buying services from another agency or pursuing interagency coordination of spaceflight data collection is preferable to fully interdependent cooperation.

Important lessons for national interagency collaboration efforts may also be learned from experiences with international collaboration (i.e., more than one country working together). In particular, the committee found that the U.S. experience in international collaborative projects is instructive with regard to the degree of upfront planning involved to define clear roles, responsibilities, and interfaces consistent with each entity’s strategic plans.

Experience has shown that collaborative projects almost invariably lead to increased costs. When additional participants join a project, the basic costs remain, but the costs of duplicating management systems and of managing interactions must be added. It is also important to recognize that even though the overall cost of the program may increase, the cost to each partner is often decreased, thus making a program more affordable to each partner. With international cooperation, the cost of a program to the U.S. government can be decreased, since a foreign government is absorbing some of the basic costs. With interagency cooperation, the cost to the government inevitably rises, because the basic cost plus the additional costs must all be absorbed by the participating U.S. agencies.

A prerequisite for a successful international collaboration is that all parties believe the collaboration is of mutual benefit. Proposals for interagency collaboration within the United States should receive similar serious attention as part of each agency’s strategic decision-making process prior to proceeding with technical commitments and procurements. As with international agreements, interagency agreements should not be entered into lightly and should be undertaken only with a full assessment of the inherent complexities and risks.

Impediments to interagency collaboration can result from sources both internal and external to the agencies themselves. Internal sources can include conflicts that result from differing agency goals, ambitions, cultures, and stakeholders, as well as agency-unique technical standards and processes. External sources can include the differing budget cycles for agencies—especially for the National Oceanic and Atmospheric Administration (NOAA), which must first submit its budget to the Department of Commerce—each of which has different congressional authorization and appropriation subcommittees, budget instability, and changes in policy direction from the administration and Congress. These impediments manifest themselves as impacts on mission success and as changes in cost, schedule, performance, and associated risks.

The most serious impediments to collaboration are external to the agencies. They are typically symptoms of conflicting policies that are often not made explicit at the beginning of proposed cooperative efforts. Such impediments manifest themselves as different budget priorities by agencies, the Office of Management and Budget (OMB), and the Congress toward the same collaborative activity. While there may be acknowledgment of the value of collaboration at a national level, at the implementation level decision makers can be unwilling to prioritize collaboration above other agency mission assignments and constraints.

As detailed in Chapter 3 of this report, many of the impediments to interagency collaboration, both internal and external, manifest themselves as impediments to good systems engineering. Good systems engineering and project management techniques² are important in any space mission, but especially when multiple organizations are involved. The inevitable creation of seams (i.e., divisions of responsibility and/or accountability between participants for planning, funding, decision making, and project execution) as a result of interagency collaboration is a source of technical and programmatic risks. Such risks could include failure to meet agreed-upon technical performance requirements, compromised system reliability, unacceptable schedule delays, or cost overruns, and mitigating such shortfalls requires proactive management and attention.

The committee identified a number of impediments that should be considered and addressed prior to the start of collaboration, and it outlines below a number of best practices to mitigate risk at various stages of mission devel-

opment. From its consideration of numerous case studies (see Appendix C), the committee found that interagency collaboration based on working-level collaborations among the agencies’ technical staff is preferred to top-down direction to pursue collaboration (e.g., via policy edict), because top-down direction may be burdened from the beginning with a lack of working-level buy-in. Successful collaboration was also found to be more likely when each agency considers the partnership one of its highest priorities; such an understanding should be codified in signed agreements that also document the terms of the collaboration’s management and operations.

To facilitate interagency collaborations, there is a need for coordinated oversight by the executive and legislative branches. Because the current roles of OMB and the Office of Science and Technology Policy (OSTP) are not suited to this kind of day-to-day operational oversight, some other governance mechanism may be needed to facilitate accountable decision making across multiple agencies while providing senior administration and congressional support for those decisions.

The committee recommends that if OSTP, OMB, or the Congress wishes to encourage a particular interagency research collaboration, then specific incentives and support for the interagency project should be provided. Such incentives and support could include facilitating cross-cutting budget submissions; protecting funding for interagency projects; providing freedom to move needed funds across appropriation accounts after approval of a cross-cutting budget; multiyear authorizations; lump-sum appropriations for validated independent cost estimates; minimization of external reviews that are not part of the project’s approved implementation plans; and unified reporting to Congress and OMB, as opposed to separate agency submissions.

The committee also investigated the particular problems associated with NASA-NOAA collaboration in support of climate research. Ensuring the continuity of measurements of particular climate variables, sustaining measurements of the climate system, and developing and maintaining climate data records are long-standing problems rooted in the mismatch of agency charters and budgets. As noted in the 2007 National Research Council decadal survey, Earth Science and Applications from Space,³ the nation’s civil space institutions, including NASA and NOAA, have responsibilities that are in many cases mismatched with their authorities and resources: institutional mandates are inconsistent with agency charters, budgets are not well matched to emerging needs, and shared responsibilities are supported inconsistently by mechanisms for cooperation. This committee concurs with the decadal survey committee, which concluded that solutions to these issues will require action at a level of the federal government above that of the agencies.

Successful interagency collaborations (i.e., those that have achieved their mission objectives and satisfied sponsor goals) share many common characteristics that are, in turn, the result of realistic assessment of agency self-interests and capabilities before and during the collaboration, and involve a disciplined attention to systems engineering and project management best practices.⁴ The committee recommends that the following key ele­ments be incorporated in every interagency Earth and space science collaboration agreement:

• A small and achievable list of priorities. Projects address a sharply focused set of priorities and have clear goals. Agreement is based on specific projects rather than general programs.

• A clear process to make decisions and settle disputes. Project decision making is driven by an intense focus on mission success. This is facilitated by formal agreement at the outset on explicitly defined agency roles and responsibilities and should involve agreed processes for making management decisions, single points of account­-

ability (i.e., not committees), and defined escalation paths to resolve disputes. Long-term planning, including the identification of exit strategies, is undertaken at the outset of the project and includes consideration of events that might trigger a reduction-in-scope or cancellation review and associated fallback options if there are unexpected technical difficulties or large cost overruns that make the collaboration untenable.

• Clear lines of authority and responsibility for the project. Technical and organizational interfaces are simple and aligned with the roles, responsibilities, and relative priorities of each collaborating entity. Project roles and responsibilities are consistent with agency strengths and capabilities. Expert and stable project management has both the time and the resources available to manage the collaboration. Specific points of contact for each agency are identified. Agency and project leadership provides firm resistance to changes in scope. When possible, one of the collaborating agencies should be designated as the lead agency with ultimate responsibility and accountability for executing the mission within the agreed set of roles and responsibilities, command structure, and dispute resolution process defined in a memorandum of understanding. In some cases the lead agency might change as a function of time, as for missions in which the lead agency differs between the implementation and operations phases.

• Well-understood participation incentives for each agency and its primary stakeholders. All parties share a common commitment to mission success and are confident in and rely on the relevant capabilities of each collaborating agency. Each agency understands how it benefits from the cooperation and recognizes that collaborative agreements may need to be revisited at regular intervals in response to budgetary and political changes. There is buy-in from political leadership (e.g., senior administration, Congress, and agency-level administrators), which can help projects move past the inevitable rough spots. There is a general spirit of intellectual and technical commitment from the agency workforce and contractors to help projects mitigate the disruptive effects of technical and programmatic problems that are likely to occur. Early and frequent stakeholder involvement throughout the mission keeps all stakeholders informed, manages expectations, and provides appropriate external input.

• Single acquisition, funding, cost control, and review processes. There is a single agency with acquisition authority, and each participating entity accepts financial responsibility for its own contributions to joint projects. Reliance on multiple appropriation committees for funding is avoided or reduced to the greatest possible extent. Cost control is ideally the responsibility of a single stakeholder or institution, because without a single point of cost accountability, shared costs tend to grow until the project is in crisis. Single, independent technical and management reviews occur at major milestones, including independent cost reviews at several stages in the project life cycle.

• Adequate funding and stakeholder support to complete the task. Funding adequacy is based on technically credible cost estimates with explicitly stated confidence levels.

In summary, engaging in collaboration carries significant cost and schedule risks that need to be actively mitigated. Agencies are especially likely to seek collaborators for complex missions so that expected costs can be shared. However, as the committee observed from historical experience and interviews, inefficiencies arise when collaborating agencies’ goals, authorities, and responsibilities are not aligned. Thus, collaborations require higher levels of coordination, additional management layers, and greater attention to mechanisms for conflict resolution.