Interim Review of the Subsonic Assessment Project: Management, Science, and Goals (1997)

The SASS project is confronted with issues that involve a large number of questions, a variety of possibilities, a relatively data-poor research area, and the need to establish a strategy that can bound a complex problem in a cost-effective way. Without documentation of hypotheses and preliminary assessments, and of the analyses used to determine priorities, PAEAN cannot properly evaluate what is driving the current SASS project and how its current funding of research operations has evolved. We strongly urge NASA to correct this weakness. The problem might be considered as though it were a third category in Table 1–2 of the first SASS report (Thompson et al., 1996), entitled "Project Management Topics." A first entry could be the following:

"Question:

• What topics do preliminary assessments suggest are most important, given their hypothesized effects, the consequences thereof, and their probabilities of occurrence?

"Program Response:

• Perform initial cost/benefit analyses to determine the broad division of effort among the project's major elements.
• Perform uncertainty and sensitivity analyses to discover how, within those elements, uncertainties can best be reduced to a common level.
• Draw up an initial research strategy for SASS, with provisions for modifying it as new information is received."

In the absence of large-scale-model results that could help formulate a strategy, this process would yield a defensible project plan for SASS. Such an approach might involve some risk, but on the whole the panel considers identifying (and investing in addressing) a few critical issues well to be more useful than trying to cover many issues sketchily.

The discussion so far has focused on the objectives of the program, the scientific questions that dominate programmatic issues, and project management from the standpoint of integrated planning, priority-setting, and resource allocation. PAEAN believes that one other important measure must be taken to implement all of these: Continuous, strong scientific leadership must be provided. The panel understands that organizational shifts at NASA (which have continued during the writing of this report) have affected the administration and management of AEAP. The recent assignment of a dedicated on-site AEAP manager is a positive step, but for optimum scientific progress PAEAN recommends that the responsibility and authority for relating and integrating all scientific and technological aspects of SASS be vested in a permanently assigned, experienced person, who can ensure that the disposition of research resources properly reflects program priorities. The panel has suggested that some of the field operations have not been thoroughly planned or properly focused, although closer control and oversight during the last couple of years have alleviated that problem to some degree. A SASS-dedicated (rather than rotating) scientist with broad experience would be in a better position to make hard decisions (and perhaps suggest new approaches) in a time of potential reductions in research funding, and would be more apt to see and effect appropriate links to other national and international programs (such as the WCRP) where collaboration could cost-effectively accomplish SASS's objectives. We therefore urge NASA to consider redefining the current management structure to include such a senior scientist who will stay with the project not for a year or two, but until its anticipated end.

The following, then, are the two recommendations that PAEAN concludes should receive the highest possible priority from NASA/AEAP management.

Some of the elements of a SASS research strategy and plan are contained in the various documents and briefing materials provided by NASA, but they are scattered, hard to compare, and far from complete. A detailed and unambiguous research plan is needed. The current over-general strategy handicaps SASS in several ways. Among them are:

• Unclear priorities. It is extremely difficult to compare either current or optimum priorities, let alone determine the important gaps, if specific goals are

• not outlined and a strategy laid out. In case of significant changes in funding levels, or unexpected research successes or failures, how will priorities be altered? All program components are treated as though they were relatively equal in importance, and their interdependencies are not clearly described.
• Reduced leveraging. When the strategy and priorities are not clear, the potential is diminished for leveraging of other agencies' activities, the interests of researchers outside the SASS community, and international programs. Given the current funding situation, it behooves AEAP to draw on information available from other parts of NASA, from other nations, or from international groups such as the WCRP or the International Geosphere-Biosphere Programme (IGBP). Without an explanation of exactly how the project aim of understanding the impacts of aircraft operations on climate is to be achieved, the applicability of others' work may be overlooked.
• Evaluation problems. When quantifiable links between project components are not clearly laid out, real progress toward the overall goal is difficult to judge. Review of the parts will not yield a consensus on the success of the whole.

The panel understands that efforts are being made to remedy the situation. PAEAN therefore recommends that concerted attention be given to constructing a research strategy and a program plan of such clarity that no researcher or agency program manager need wait for a call for proposals to know what SASS needs and where their own skills or resources may contribute. Such a plan is likely to need modification as research progresses, but is essential for wise reassessment of goals, priorities, and schedules.

Determining the atmospheric effects of aviation is extremely difficult and controversial, from the standpoint of both the science involved and the finite resources available to address a scientific and technological problem that is virtually unbounded. Demands on AEAP are great, since its scientific results will provide a foundation for the formulation of a U.S. policy position regarding the international operation of current and future subsonic aircraft fleets: Such international protocols could be formulated as early as 1998 under the auspices of the International Civil Aviation Organization (ICAO).

Because the stakes are high, the time is short, and budget trends suggest reductions, the subsonic-aircraft portion of AEAP must have focused, uninterrupted scientific leadership and management in the future. The experiments and investigators must be selected via a methodology that ensures a balanced, integrated, cohesive approach to the overall task. Accountability for the planning and execution of all phases of the program must be specific, clearly understood, and not fragmented, so as to permit effective program evaluation.

AEAP is a scientific project with technological and economic implications. It needs leadership by a person who (i) has a technical reputation that is recognized in the scientific community at large, (ii) has the broad perspective needed for setting appropriate priorities, (iii) is experienced in the management of large scientific or technological programs, and (iv) is assigned for the duration of SASS/AEAP. Like the Project Manager, he or she should be in the direct chain of command. A long-term assignment would enable that person to have a lasting impact on project effectiveness and productivity, while personally experiencing the effects of project success or failure. Although AEAP has been very fortunate in its Project Scientists, the current arrangement of a two-year tour for a relatively young scientist gives the job the appearance of a senior internship. Someone with the qualifications outlined above could dramatically affect the project through sharing with the Project Manager significant authority, responsibility, and accountability in program planning, resource allocation, overall execution, and measurable results. The position would of necessity distance its holder somewhat from active research, but successful management of other large, complex, critically important programs has been a significant stepping stone in a career path to positions of even greater scientific and programmatic responsibility.