2
Innovation Facilitators and Accelerators for Aeronautics

The lack of clarity about the purpose and priority of the NASA aeronautics program has made it difficult for the committee to comply with our charge—to recommend practical measures to enhance the implementation of NASA-developed technology in the Aeronautics Research Mission Directorate (ARMD). Obviously, the advice would not be the same for projects designed to yield fundamental knowledge of aerodynamics or materials or human factors and projects undertaken for clearly identified customers leading to prototype technologies, for example for fuel-efficient commercial aircraft engines or advanced air traffic control systems. If the former were to constitute the core of the NASA program, then our focus should be on how well fundamental knowledge is disseminated to all potential users, for example, via peer-reviewed publication, the participation of investigators in scientific and technical meetings, and training of entrants into the professional workforce. We focused instead on NASA’s efforts to develop solutions targeted to specific users’ needs and the efforts made to get the solutions adopted. Our focus on innovation in this sense led us to examine the management of the R&D process and the hand-off of resulting technologies.

In our view, refocusing the NASA aeronautics program exclusively on fundamental research is neither a likely nor a very desirable result of the policy deliberations so clearly needed. The public good areas of NASA R&D work in which the argument for government involvement is strongest—safe, efficient air traffic management and environmentally benign



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 41
Aeronautics Innovation: NASA’s Challenges and Opportunities 2 Innovation Facilitators and Accelerators for Aeronautics The lack of clarity about the purpose and priority of the NASA aeronautics program has made it difficult for the committee to comply with our charge—to recommend practical measures to enhance the implementation of NASA-developed technology in the Aeronautics Research Mission Directorate (ARMD). Obviously, the advice would not be the same for projects designed to yield fundamental knowledge of aerodynamics or materials or human factors and projects undertaken for clearly identified customers leading to prototype technologies, for example for fuel-efficient commercial aircraft engines or advanced air traffic control systems. If the former were to constitute the core of the NASA program, then our focus should be on how well fundamental knowledge is disseminated to all potential users, for example, via peer-reviewed publication, the participation of investigators in scientific and technical meetings, and training of entrants into the professional workforce. We focused instead on NASA’s efforts to develop solutions targeted to specific users’ needs and the efforts made to get the solutions adopted. Our focus on innovation in this sense led us to examine the management of the R&D process and the hand-off of resulting technologies. In our view, refocusing the NASA aeronautics program exclusively on fundamental research is neither a likely nor a very desirable result of the policy deliberations so clearly needed. The public good areas of NASA R&D work in which the argument for government involvement is strongest—safe, efficient air traffic management and environmentally benign

OCR for page 41
Aeronautics Innovation: NASA’s Challenges and Opportunities aviation operations—are arguably the areas in which users need fairly well-proven technologies to be delivered and in which NASA’s technical capabilities are in some respects superior. In all likelihood, ARMD will continue to have a portfolio quite diversified in terms of the stage of technology development being pursued. If it does not, the program could rather quickly lose its relevance and much of its support. That, in any case, is our premise. We further assume progress in articulating a mission reflecting financial realities, stakeholder needs, and NASA personnel and contractor capabilities and research infrastructure. In this chapter we consider a variety of decision-making processes, tools, and incentive structures that will aid the process and enhance the prospects of innovation in the remaining portfolio. These include cohesive portfolio planning, engagement of stakeholders in the prioritization process, preidentifying the stages of and criteria for resource allocation and project continuation or termination decisions (“decision gates”), and planning for technology transitioning. In addition, we outline a number of personnel and financial management practices that can contribute to innovation. Those tools might broadly be conceived as process discipline. Fundamental to keeping an organization on a path of relevant accomplishment is a set of tools that accelerate decision making. Quite the opposite of constraining an organization in bureaucracy, process tools and discipline help accelerate results and aid in decision making by clarifying expectations among customers, leadership, and development teams. These tools provide an expectation that mechanisms and metrics need to be developed to keep innovation relevant in terms of the values it can provide. These tools also help clarify schedules and timelines. Notions that innovation cannot be scheduled, that invention has to happen on its own pace, contribute to ignoring customer needs and, on the part of the innovator, diminished expectations of creating value. In recommending these tools, the committee recognizes that there are important differences between public agencies and private firms, for example in their ability to focus resources narrowly, to reallocate funds, and to change or transfer personnel. We do not thoughtlessly recommend practices that are appropriate solely for private firms but are inappropriate and impossible for ARMD to implement. In fact, a number of the practices that we think NASA should consider are ones that derive from public-sector experience, including that of NASA. At the same time, the composition of our committee does not reflect sufficiently broad NASA experience to anticipate all of the challenges that

OCR for page 41
Aeronautics Innovation: NASA’s Challenges and Opportunities might arise in implementing our recommendations. We do recognize that objectives requiring negotiation with the Office of Management and Budget (OMB) or congressional authorizing and appropriating committees (or both) are likely to be harder to achieve and require more accommodation than measures within NASA’s current authority, but even in the latter case, some of our proposals may be at odds with traditional practices that are difficult to change. The recommendations are not intended to represent a package that must be accepted as a whole. PORTFOLIO APPROACH AND COHERENT ALIGNMENT WITH MISSION AND CUSTOMERS Although the strategic focus discussed in Chapter 1 is the single leading principle of best-practice R&D management, a close second is to Recommendation 3-A: Conceive of R&D activities as a cohesive and strategically balanced portfolio of projects and competencies closely aligned with mission and stakeholder needs. Individual R&D activities should not operate independent of an overall understanding and agreement of how they contribute to and fit within the portfolio.1 Key dimensions of the portfolio include balance across goals, timeframe, level of risk and potential value, and skill sets. Another key dimension that should be explicit in developing the portfolio is the national additive value, that is, the degree to which ARMD is uniquely suited to pursue the R&D “as only NASA can.” Easy to say, yet difficult to identify. ARMD should focus on where it is not competing or 1   Philips, for example, one of the world’s leading consumer electronics firms, calls its portfolio of R&D activities a “program haystack,” with cross-portfolio analysis of each program or research competency’s horizontal and vertical contributions to other programs or competency areas. Vertical research programs, such as health care systems, directly target specific customers and product areas. Competency areas, such as devices and microsystems, encompass broadly applicable technology components that support across the program silos horizontally. This allows Philips to view different R&D investments and make decisions across and among the different silos. See D. Busher et al., Management of Technology in Europe 2003: Comparing Strategies and Tools in 17 High Technology Organizations, T. A. Watkins, contributing ed. (Minneapolis: National Technological University, May 2003), p. 16. Available at http://www.lehigh.edu/~taw4/eumot03.pdf.

OCR for page 41
Aeronautics Innovation: NASA’s Challenges and Opportunities duplicating what is or could be done in industry, universities, the Department of Defense (DOD), or other agencies. In pruning the portfolio, this should be a primary guiding principle. Many useful portfolio assessment and planning tools exist (graphical representations like risk-reward bubble diagrams, technology roadmapping and milestones, future scenario visioning, stages and gates reviews, strengths-weaknesses analysis, cost-benefit-risk assessment, etc.), developed by a growing industry of consultants, textbooks, and how-to primers.2 Our committee’s collective experience suggests that Recommendation 3-B: Graphical illustrations of the portfolio are particularly useful tools for fostering communication and discussion and identifying and resolving disagreements, both internally among managers and in engaging external stakeholders and customers. We emphasize that what is important is not the specific tools employed—organizational idiosyncrasies suggest that no single set of tools will work in all contexts—but that the decision-making system is transparent, designed and understood by those who will implement it. The process should not be overly complex or burdensome; straightforward tools exist. The hard but most valuable part is not the tools or information gathering associated with them but the quality and depth of the conversations they can facilitate. Best practice also means rigorous pruning of portfolio elements found to be yielding limited value. Hence, ARMD should Recommendation 3-C: Use decision processes, sometimes referred to as decision gate processes, at predetermined points to establish common expectations among customers, leaders, and the technical team throughout the development process, to clarify goals, schedules, 2   Some leading books include P. K. S. Rousel and T. Erickson, Third Generation R&D: Managing the Link to Corporate Strategy (Boston: Harvard Business School Press, 1991); and R. G. Cooper, S. J. Edgett, and E. J. Kleinschmidt, Portfolio Management for New Products, 2nd ed. (Reading, MA: Perseus Books, 2001). Shorter articles include N. Danila, “Strategic Evaluation and Selection of R&D Projects,” R&D Management 19(1, 1989), pp. 47-62; P. Groenveld, “Roadmapping Integrates Business and Technology,” Research Technology Management (September 1997), pp. 48-55; D. L. Hall and A. Nauda, “An Interactive Approach for Selecting IR&D Projects,” IEEE Transactions on Engineering Management 37(May 1990).

OCR for page 41
Aeronautics Innovation: NASA’s Challenges and Opportunities deliverables, concrete target performance metrics, and review templates and to set decision criteria and force accountability of all constituents involved. In the committee’s second workshop, David Whelan, a Boeing and former senior manager at the Defense Advanced Research Projects Agency (DARPA), described the notion as midterm exams for projects, deciding what should be required to pass. Decision gates and specific targets set criteria for hand-off from one phase to the next, including the hand-off to the user. Best practice also assesses and ensures that the technology readiness needed by the customer is understood and met by the developers. Key elements include sunset provisions and criteria for retiring projects. Terminating projects that fail midterms also increases economic flexibility to more rapidly pursue new opportunities. The process requires knowledgeable, disciplined leaders to operate effectively. We heard repeatedly in our interviews and at our workshops that there are several impediments to successful R&D portfolio management at ARMD apart from lack of mission clarity around which to build a portfolio. First, ARMD and NASA research activities more generally have been “projectized” and decision making done largely top-down in silos isolated to a degree that we think runs counter to R&D portfolio best practices. Each project manager and the upper layers of administration should understand how each project fits within the broader portfolio and how it contributes to the overall focused strategy and to external stakeholder needs. We concluded that ARMD’s managerial approach does not fully meet this test. One NASA project manager, speaking about silos in a single NASA center, described it this way: “I look out the window here and see all these ostriches in separate sandboxes, not looking up to know what’s going on around in other sandboxes, or understanding why they are doing what, or how their activities connect with the customer.” Similarly, John Klineberg, chair of a National Academies study assessing NASA’s aeronautics technology programs, speaking before Congress in March 2005, testified that “subproject and task-level plans, funding, goals, metrics, staffing, and responsibility are often difficult to define or cannot be clearly traced back to a plan or vision for the program as a whole.”3 3   Statement of Dr. John M. Klineberg, Chair, Committee to Review NASA’s Aeronautics Technology Program Aeronautics, and Space Engineering Board Division on Engineering and Physical Sciences, National Research Council, the National Academies, before the

OCR for page 41
Aeronautics Innovation: NASA’s Challenges and Opportunities The organizational and geographic separation of the three major ARMD facilities magnifies the silo problem. For example, our interviews with technology managers at the Ames Research Center led us to believe that work on air traffic management there is not closely linked to related work at the Langley Research Center. The groups appeared not to be thoroughly familiar with each others’ work or how their activities relate to one another. This is a clear sign that portfolio planning is not well established in ARMD. That said, one positive sign is that at the time of our visits the two groups were planning to meet in the near future to identify ways to leverage each others’ activities. Second, best practice suggests there should be more coherence and organizational agreement about the balance across various dimensions of the portfolio. In our interviews, some ARMD mangers reported that they perceive themselves under great pressure, mostly from OMB, toward shorter term, nearer payoff development projects—“we need successes to justify our budgets.” And “long-term kinds of things seem consistently difficult to keep,” as they are “always the first thing to go when there are budget issues at almost all levels.” Some blue-ribbon external review committees agree that ARMD sometimes does not take its technologies far enough toward implementation. In contrast, other managers believe and some external reviews4 and political pressure against perceived corporate welfare suggest the opposite, that government-funded laboratories should focus more on long-term fundamental science and high-risk, high-payoff breakthroughs. This kind of disagreement, this lack of coherence among the views of various managers in the organization as to what the research organization is or should be doing, is to us a signal that technology management best practices are not well established. The individual projectized parts do not add up to a cohesive whole nor do they have a common understanding of their collective purpose. R&D portfolio management best practice is to avoid exclusive focus one way or the other but rather achieve a balance across long-, medium-, and short-term R&D. Along these lines, a 2003 national steering commit-     Committee on Science Subcommittee on Space and Aeronautics, U.S. House of Representatives, March 16, 2005. Available at http://www.house.gov/science/hearings/space05/Mar16/Klineberg.pdf. 4   E.g., National Research Council, Review of NASA’s Aerospace Technology Enterprise: An Assessment of NASA’s Aeronautics Technology Programs (Washington, DC: The National Academies Press, 2004).

OCR for page 41
Aeronautics Innovation: NASA’s Challenges and Opportunities tee on aeronautics and aviation technologies, organized by the Office of Science and Technology Policy and sponsored by the American Society of Mechanical Engineers, suggested and we agree that NASA aeronautics should Recommendation 3-D: Pursue a portfolio “balanced between near term needs, driven by market forces, and longer-term investments required to achieve transformational national capabilities.”5 Criteria for including or eliminating R&D activities should be driven by the focused mission and key stakeholder needs. We discuss the importance of engaging stakeholders in more detail below. A potential bonus of a balanced approach would be political: near-term successes could help defend longer term programs’ budget lines. The perceived public value of ARMD research would be clearer than with entirely long-term breakthrough programs. We also heard multiple reports of a third significant impediment to R&D portfolio best practices, a reluctance to terminate projects Indeed, the incentive structure works strongly against it. Terminating projects does not quickly save resources because legislation makes it difficult or impossible for ARMD managers independently to move resources or reduce civil service staff quickly. This structural inability limits incentives to prune and to make midcourse corrections. We address staffing flexibility in more detail below. One indication of the prevalence of this tendency is that it has an internal nickname: “slip and dip.” This refers to the pressure to first oversell a project’s potential to attract funding in the annual political cycles and then to stretch goals and timelines as budgets allow. One former ARMD manager put it bluntly: “Aeronautics has to make promises it knows it can’t meet in order to get funding…. A lot of times we stretch ourselves more than we think we should, to sell the program. Otherwise, we won’t have anything.” A second manager referred to “the hollowing out of milestones…. [I]t’s not that clear to me that there’s a penalty for not delivering.” He explained that project managers put most milestones in September, just before the end of the fiscal year. “You deliver something less but like what 5   American Society of Mechanical Engineers, Aerospace Division, Persistent and Critical Issues in the Nation’s Aviation and Aeronautics Enterprise, (Washington, DC, November 2003).

OCR for page 41
Aeronautics Innovation: NASA’s Challenges and Opportunities you promised, and unless you’ve wasted money or done something stupid, they give you another crack at it.” John Klineberg similarly noted, in the context of artificial five-year sunset provisions on research programs, that some longer term research had been disguised as a series of five-year plans under different names and different organizational structures.6 Such artificial timelines are budget driven, rather than technology and challenge driven. Unfortunately for innovation management, one- and five-year timelines do not fit all technologies. The result is that the time horizons of ARMD technology problems are not in line with pressures of external bodies and contingencies well beyond ARMD management’s control. This makes efficiently planning and managing the resources and gauging technical progress remarkably difficult. An associated tendency we noted among ARMD managers is to see all projects as worthy. Clearly, the vast majority of ARMD activities do have value. Indeed, recent NRC reviews found few obvious weak projects from a technical point of view.7 But the relevant managerial criterion cannot be whether individual projects have absolute value but rather prioritizing their value relative to each other in the context of severely constrained and shrinking resources. Pursuing large numbers of hollow, isolated projects aimed exclusively at short-term results is characteristic of worst practice, not best. This tendency continues even under the refocused new FY 2006 budget proposals. Of the 11 projects in the proposed FY 2006-FY 2010 Airspace Systems Program schedule, 9 have milestone slips of at least a year, including several that also “descope” (the dip). A tenth dips without extending the milestone. Only the eleventh is scheduled for cancellation. Organization-wide application of portfolio assessment and uniform decision gate processes would foster the conversations needed to enable cross-project evaluation. A fourth major impediment to R&D portfolio planning at ARMD is the growth in congressional directly funded projects. At NASA as a whole, these projects increased from $74 million for six items in FY 1997 to $426 million for 167 items in FY 2005.8 This 28-fold increase in the number of 6   Statement of Dr. John M. Klineberg, Chair, Committee to Review NASA’s Aeronautics Technology Program Aeronautics, and Space Engineering Board Division on Engineering and Physical Sciences, National Research Council, the National Academies, before the Committee on Science Subcommittee on Space and Aeronautics, U.S. House of Representatives, March 16, 2005. 7   Review of NASA’s Aerospace Technology Enterprise. 8   NASA FY05 Initial Operating Plan. Available at http://www.nasa.gov/pdf/107781main_FY_05_op_plan.pdf.

OCR for page 41
Aeronautics Innovation: NASA’s Challenges and Opportunities projects and 5-fold increase in costs had to be funded by offsetting reductions in ongoing NASA programs. The FY 2005 NASA earmarks in aeronautics amounted to $92 million. Compare this to the entire budget for air traffic management research, the Airspace Systems Program for that same year: $152 million. Indeed, fully 14 percent of the ASP budget was congressionally earmarked. Given fixed facility infrastructure costs and civil service employment constraints, this means that a significant fraction of ARMD’s portfolio is largely beyond managerial control. To make matters more difficult, NASA is prohibited by Congress from charging administrative expense overhead to these projects, in contrast to the full cost accounting principle applied to other programs. Earmarks can reflect a congressional perception that NASA officials are neglecting an important component of their program. For example, funds were increased for rotorcraft development following NASA’s elimination of this program. However, as is frequently the case, the rotorcraft funding mandate came without a corresponding increase in the aeronautics budget and forced a reduction in some other programs, playing havoc with the budget planning process. Increased stakeholder participation in portfolio planning and budget balancing can help contain earmarking motivated by disagreement with NASA’s priorities. But in some instances, earmarks are indicative of a philosophical conflict over whether a market failure exists to justify government intervention to support R&D. Earmarks are also used to appeal to local constituent interests. No budget planning process can eliminate earmarks in these circumstances. Although this practice is unlikely to cease or even significantly decline, there are steps that ARMD can take to limit its disruptive effects. One constructive action along these lines was a suggestion by former NASA administrator Sean O’Keefe to the Senate Appropriations Subcommittee that NASA would begin to subject earmarks to selection criteria applied to all nonsolicited, noncompetitive proposals. These criteria include “relevance to NASA mission, intrinsic merit and cost realism.”9 NASA management is well aware of the problem and the technical disruptions they cause. We discuss below financial management options for handling externally mandated projects. There are some promising examples of the use of portfolio planning tools in various parts of ARMD and evidence that these tools can in fact be 9   NASA FY05 Initial Operating Plan. Available at http://www.nasa.gov/pdf/107781main_FY_05_op_plan.pdf.

OCR for page 41
Aeronautics Innovation: NASA’s Challenges and Opportunities successfully implemented. We note, for example, that NASA participated in technology roadmapping working with the Federal Aviation Administration (FAA) in developing the FAA Operational Evolutionary Plan.10 We also understand that a NASA-wide core competency review and prioritization was under way in 2005. We noted earlier a problem with core competency understanding in the Vehicle Systems Program (VSP). These are good signs but appear to us as ad hoc, rather than parts of a systematic organization-wide practice of portfolio analysis and planning. For example, we are troubled that managers at Langley perceived that the review primarily focused on supporting space exploration, not aeronautics. Recommendation 3-E: NASA should continue to undertake core competency reviews and explicitly include aeronautics among the highest priority core competencies. Within aeronautics, the ranking of competencies should take into account world leadership in technology, public additive value, and skills enabling partnerships and transitioning processes. In this context, we also encourage expanded NASA-wide use of skills assessment tools, such as information technology systems, to collect and sort the status of all education, experience, and skills throughout the organization, so that the right people can be flexibly assigned high-priority tasks anywhere in the organization. This can be especially valuable in accelerating schedules in early innovation phases. ARMD has also succeeded in some pruning in response to falling resources. External reviewers suggest that the result has been a reasonably internally balanced portfolio-like outcome. A 2004 RAND Corporation study of wind tunnel and propulsion-test facilities concluded that “currently, redundancy is minimal across NASA. Facility closures in the past decade have eliminated almost a third of the agency’s test facilities in the categories under review in this study. In nearly all test categories, NASA has a single facility that serves the general- or special-purpose testing needs, although some primary facilities also provide secondary capabilities in other test categories.”11 For the overall portfolio, it found “the test complex 10   NASA, The NASA Aeronautics Blueprint: Toward A Bold New Era of Aviation. NP-2002-04-283-HQ. (Washington, DC, 2002). 11   P. S. Anton et al., Wind Tunnel and Propulsion Test Facilities: An Assessment of NASA’s

OCR for page 41
Aeronautics Innovation: NASA’s Challenges and Opportunities within NASA is mostly ‘right sized’ to the range of national aeronautic engineering needs.” Nevertheless, portfolio planning can be more fully internalized and regularized and external stakeholders more regularly engaged in the process. The RAND study concluded that closer coordination and planning across DOD’s Engineering Development Center and NASA could further identify national infrastructure overlap and reduce expenses on redundant facilities. The reviewers were troubled that “NASA’s recent unilateral decision to close two facilities at Ames without high-level DOD review shows that progress has been spotty.” Another positive sign is increasing recognition among ARMD managers of the need for balance between short and long term, although disagreement remains about what mix is appropriate. One manager we interviewed tries in an ad hoc way to spend 20 percent of his research money on “high promise breakthrough kinds of things” that “you’re not sure are going to work,” an investment he described as “minuscule.” But he admitted that most of those resources are contained in related project budgets. However, because all budgets are projectized, this less than transparent approach to portfolio balancing defeats the best-practice possibilities for strategic-level conversations and healthy debate. The next steps should be to make the need for a balanced portfolio uniformly understood organization-wide and to bring the planning and debate more into the open. The strategy of balance should be explicit as it is at DARPA, for example, which aims for breakthroughs and focuses on high-change-potential projects, yet also explicitly maintains a portfolio across relatively near, medium, and longer term R&D. The committee also supports an initiative in ARMD’s FY 2006 budget to create a central pool of funds for exploratory research. The associate administrator indicated to Congress that “a level of funding will be reserved for ‘seed corn’ research.”12 This would bring longer term exploratory thinking out from hiding and into the open as an explicit management tool.     Capabilities to Serve the National Needs. MG-178 RAND (Santa Monica, CA: National Defense Research Institute, 2004), pp. xviii-xxi. 12   Dr. J. Victor Lebacqz, associate administrator for aeronautics research, NASA, “Appropriations Subcommittee Staff Briefing,” March 8, 2005.

OCR for page 41
Aeronautics Innovation: NASA’s Challenges and Opportunities ativity and innovation.” Researchers may submit a proposal to spend a part of their time on projects of their own conception. This is a more formal arrangement than we envision, but it nevertheless conveys a strong signal that individual imagination and initiative are valued. Believing that time rather than money is the more severe constraint on creativity, we encourage ARMD to institute more such programs for in-house investigators. FINANCIAL MANAGEMENT TO MINIMIZE THE DISRUPTIVE EFFECTS OF EXTERNAL DEMANDS Our last set of recommendations for fostering aeronautics innovation through NASA deals with the structuring of financial management at ARMD. To a significant degree, best-practice approaches to financial management aim to send clear signals internally and externally about the value of resources to help managers make efficient choices about how to allocate those resources within and across programs. When signals are not aligned with priorities, resource misallocation and inefficiency result. This is especially important to correct in an era of significantly declining resources. In FY 2004 NASA completed implementation of an agency-wide full-cost accounting system, which had been in planning and pilot-testing since 1995.41 The purpose of full-cost accounting is to give mangers more complete information about the real costs of their activities, including the costs of personnel and facilities. Historically, program managers were not responsible for certain significant costs associated with their activities, including the actual cost of civil service personnel. As a result, agency administrators believed that the cost implications of program decisions were not well understood and appreciated. Although we strongly support the objective of achieving greater financial transparency, we think that attempting to achieve full-cost recovery pricing for both civil service and facilities use in NASA has had unintended negative consequences for aeronautics R&D activities. Recommendation 6-A: NASA should modify full-cost pricing for ARMD facilities use, with charges more closely aligned with marginal costs. 41   See the 2004 NASA Cost Estimation Handbook, available at http://ceh.nasa.gov/webhelpfiles/Cost_Estimating_Handbook_NASA_2004.htm

OCR for page 41
Aeronautics Innovation: NASA’s Challenges and Opportunities Many ARMD research facilities have two characteristics that make full-cost recovery problematic. First, the facilities have significant long-term value from the standpoint of national security and economic competitiveness; this value should be reflected in public support rather than private user charges. Second, significant fractions of the total costs are essentially independent of short-term facility usage levels. For example, in our interviews at Langley we were told by administrators that the annual cost of operating Langley’s transonic wind tunnel, in which virtually every U.S. aircraft has been tested, is mostly a fixed cost independent of how many tests are run in it; this may also be true of NASA’s other operational wind tunnels. Under NASA’s full-cost accounting principles, however, prices are based on short-term (i.e., annual) facility usage levels and thus are sensitive to how many tests are run, even in places where operating costs may not vary in that manner. As a result, prices do not reflect the real impact of individual managerial decisions on costs, skewing the incentive signals. We see in this practice significant risk to the long-term financial viability of critical national aeronautics research infrastructure. Full-cost pricing for ARMD facilities entails charging users the direct operating costs of their activities (e.g., materials, test components, support personnel, power) plus some prorated fraction indirect expenses (e.g., general maintenance, facilities upgrades, technician training, general administrative overhead). The latter is based on the fraction represented by the user in the total hours that the facility is used that year. When facilities run near capacity and have many users, each user appropriately absorbs a small fraction of the fixed overhead, maintenance, and equipment upgrade expenses. However, for a facility that in a particular year is used only occasionally, users who might account for only small fractions of total available capacity but large fractions of actual use in that year must absorb essentially all the costs for unused capacity. This can lead to less utilization as fixed costs are spread over fewer and fewer users, as has been the case with NASA’s wind tunnels—in short, a “death spiral.” Reportedly, fees increased on average 30-35 percent from 2003 to 2005, and, in one particular case, “utilization hours for the 20-Foot Vertical Spin Tunnel dropped 71 percent between 2003 and 2004, from 855 hours to 248 hours.”42 42   D. Schleck, “NASA Windtunnel Feed Under Review,” Hampton Roads Daily Press, June 12, 2005.

OCR for page 41
Aeronautics Innovation: NASA’s Challenges and Opportunities At our workshops some aerospace industry representatives corroborated their increasing reluctance to use NASA facilities. Gulfstream qualified all four of its most recent aircraft in either France or the United Kingdom, not in the United States,43 despite the fact that the federal government is the company’s largest customer. Similarly, Boeing is going to Toulouse for Dreamliner 787 testing. Both firms report that NASA facilities are not competitive under full-cost charging. The result is that U.S. firms are supporting European infrastructure while reducing facility usage rates in the United States. This raises charges to other users, contributing to a further drop in utilization. This pricing policy applies equally to internal NASA users, leading to somewhat arbitrary cross-program subsidization. In ARMD this is particularly burdensome for air traffic management research, which tends not to be fixed capital intensive but rather relies on people and on rapidly advancing information technologies. In some of our interviews, project managers suggested that ATM projects end up paying high overhead to support facilities used mainly for non-ATM research. To make matters worse, Congress prohibits NASA from charging administrative overhead expenses on directly funded earmarked projects,44 a growing fraction of ARMD discretionary budgets, shrinking the base on which overhead expenses might be spread. This, in turn, has the effect of encouraging project managers to use contractor facilities and staff rather than civil service personnel whenever possible. A former NASA official pointed to DOD’s experience with full-cost recovery. He referred to a 1969-1972 failed experiment by the Air Force Arnold Engineering and Development Center. For that period DOD charged users full average costs, including all overhead and equipment capacity, while DOD funded none itself. This led to an unsustainable steep decline in revenue,45 leading DOD to reverse the policy. Since then, DOD has funded more than 50 percent of AEDC’s total annual costs, sharing the burden with users in order to retain an important national strategic asset and insulate it from short-term variations in usage. Not only does full-cost pricing endanger particular facilities, but it also risks undermining relationships with external partners and internal research 43   Testimony of Dick Johnson, Gulfstream, to the committee January 18, 2005. 44   See NASA’s FY05 Initial Operating Plan, p. 3. Available at http://www.nasa.gov/pdf/107781main_FY_05_op_plan.pdf. 45   For more details see P. S. Anton et al., 2004, p. 61.

OCR for page 41
Aeronautics Innovation: NASA’s Challenges and Opportunities competencies. Our interviews with ARMD program managers suggest that some cooperative programs have been “one of the victims” of full-cost pricing, with repercussions for the competence of NASA employees. Because a customer has to put up all the funding to use a facility, substantive research collaboration with external partners potentially suffers: “government people become data generators and technicians for operating facilities to a greater extent and experts in the field to a lesser extent.” This could result in the hollowing out of internal leading edge research competencies, with ARMD centers becoming simply a for-hire infrastructure with a high fixed cost. Other external reviewers have expressed similar reservations about NASA’s approach. The 2004 RAND study on NASA’s wind tunnel and propulsion test facilities concluded that the full-cost pricing approach was “creating real risks to the United States’ competitive aeronautics advantage”46 by undermining the financial health of those facilities already underutilized—about one-third of the facilities in all. RAND found that “with declining usage and full cost recovery accounting, these facilities run the risk of financial collapse.”47 As examples, the report cited two Ames facilities that “are unique and needed in the United States [but] have already been mothballed and slated for closure as a result.”48 The National Academies’ Review of NASA’s Aerospace Technology Enterprise also expressed concern about “unintended consequences” of full cost pricing—disincentives to use facilities to demonstrate new technologies, underutilization, and eventual closing of critical infrastructure. The first task of NASA administrators, the administration, and Congress is to decide which aeronautics research facilities have unique, long-term national strategic and economic value. Once this is done, prices can be set to make optimal use of this capital investment. Marginal cost pricing is likely to be appropriate up to the point that a test facility is fully utilized. Anything that covers marginal costs produces revenue to help defray fixed costs without discouraging use of the facility. Full-cost pricing prices to restrict use. This is not an appropriate policy when facilities are underutilized. 46   Anton et al., 2004, p. xiii. 47   Anton et al., 2004, p. xx. 48   Anton et al., 2004, p. xxii.

OCR for page 41
Aeronautics Innovation: NASA’s Challenges and Opportunities Recommendation 6-B: AMRD should work with OMB and Congress to establish separate centrally funded budget lines for national infrastructure and facilities maintenance. The RAND study pointed to this solution: “[Wind tunnel and propulsion test] facility operations are not funded directly by specific line items in the NASA budget…. [W]hen a needed facility is closed because of a lack of funding, there is a disconnect between current funding and prudent engineering need, indicating that the commercial and federal budget processes may be out of step with the full cost associated with research and design of a particular vehicle class and indicating a lack of addressing long-term costs and benefits.”49 Without changes in accounting practices, much of the nation’s aeronautics research infrastructure is in jeopardy. Indeed, NASA’s current budget projections anticipate closing many of these facilities. We think NASA has erred in equating full-cost accounting with full-cost pricing. The two concepts are conceptually and practically distinct. Cost accounting information may be used not only for fee setting but also for accountability and performance measurement, budgeting, and managerial control. Average cost–based pricing is not considered best practice in industry50 and is particularly problematic in circumstances of large fixed costs and high public value. NASA should centrally bear the fixed overhead costs incurred to maintain strategically important facilities. Users can be expected to bear the additional costs associated with their incremental use of facilities, but not full costs. In a shared-cost model, users should not pay for unused capacity. It appears to us that NASA is too narrowly interpreting the legislative requirements regarding full-cost accounting.51 Federal standards do allow flexibility in implementation. In the case of aeronautics R&D, there are broad 49   Anton et al., 2004, p. xvi. 50   See, for example, E. Mansfield et al., “Pricing Techniques,” in Managerial Economics, 5th ed. (New York: Norton, 2002). On the shift away from cost-based prices, see R. Tang, “Transfer Pricing in the 1990s,” Management Accounting 73(8), pp. 22-26. 51   The most important related federal standards are the Statement of Federal Financial Accounting Standards (SFFAS) No. 4, Managerial Cost Accounting Concepts and Standards for the Federal Government. Available at http://www.fasab.gov/pdffiles/sffas-4.pdf. SFFAS No. 6, Accounting for Property, Plant, and Equipment. Available at http://www.fasab.gov/pdffiles/sffas-6.pdf.

OCR for page 41
Aeronautics Innovation: NASA’s Challenges and Opportunities benefits to the nation above and beyond the benefits to specific users, and sharing costs for such public purposes is even within NASA’s own standards for full-cost recovery.52 These require full-cost charges only when special benefits accrue to users, not when there is general public value. We encourage a more liberal interpretation of these full-cost recovery requirements. There are some recent hopeful signs that ARMD administrators are aware of the potential problems of full-cost recovery. The ARMD associate administrator’s briefing53 on the FY 2006 budget defends the full-cost initiative but acknowledges the need for flexibility: “Full cost accounting is necessary to understand the return on taxpayer investment … [but] NASA is developing innovative ways to maintain flexibility in human resources and institutions…. One component of this new management approach may be a direct ARMD investment in key facilities to ensure longer-term facility sustainability.” In another initiative, NASA’s new administrator, Michael Griffin, has directed a group of headquarters officials to study how to “better manage NASA research facilities in a full-cost environment.”54 We hope that these deliberations embrace the principle of central funding of shared fixed costs and incremental pricing for internal and external users. Another candidate for centralized budgeting is contingency funds, outside specific projects, enabling more flexible responses to unforeseen research contingencies. Rigid project silos with inflexible milestones that do not tolerate failure or changes of direction are a recipe for narrow, short-term research agendas. Recommendation 6-C: Because midstream changes are the nature of leading edge R&D, ARMD should achieve greater budget and milestone flexibility through centrally funded pools and contingency accounts. ARMD project managers told the committee they have no official contingency budgets, centrally funded or otherwise. Some report that they 52   NASA, Review, Approval, and Imposition of User Charges, Policy Directive NPD 9080.1F, October 14, 2004. Available at http://nodis3.gsfc.nasa.gov. 53   Dr. J. Victor Lebacqz, associate administrator for aeronautics research, National Aeronautics and Space Administration, “Appropriations Subcommittee Staff Briefing,” March 8, 2005. 54   Schleck, 2005.

OCR for page 41
Aeronautics Innovation: NASA’s Challenges and Opportunities occasionally manage to create ad hoc contingency accounts, but that this is dependent on individual managers and does not enable cross-program conversations about relative priorities. Explicit contingency funds to which project managers could apply would make these decisions more transparent and more likely to be in alignment with the overall mission. Another option is an agency-wide central pool to carry civil servants whose projects are cancelled. This would not yield short-term resource savings overall, but it would increase flexibility and better align managerial incentives at the project level. Two principal challenges in dealing with the inevitable uncertainties in leading edge research are the rigidity of the annual appropriations process and the constraints imposed by overreliance on project budgets. The short-term planning needed to accommodate annual budget cycles and the associated fluctuations in priorities are especially challenging for long-term research. Neither project managers nor top NASA administrators can change major project milestones without OMB approval. The perception expressed to us by ARMD management at Langley, for example, was that anything they defined as a contingency would get cut by OMB. Moreover, civil service regulations severely restrict midstream staffing changes. In the past, NASA aeronautics had a systems technology program and a base research and technology program. The former was composed of projects conceived, funded, and operated as projects, with funding terminated in some cases. The basic R&D work was longer term and continuously supported. One center official observed that today all activities are funded in five-year chunks with a beginning and an end, making it difficult to take a long-range point of view. “Now that there’s no more R&T base, there’s a bias in favor of [finite outcomes] and therefore against experiment and innovation.” One approach begun as a small pilot program is the Working Capital Fund (WCF). The legislative authority for this new formal structure enables more budget flexibility for capital and personnel, not tied to direct annual appropriations.55 NASA is able to establish WCFs for internal business-like entities with customers for products and services. Funds received from customers can then be expended as needed, without regard to fiscal year limitation.56 NASA began a pilot WCF in FY 2005 with an informa- 55   See NASA, WCF, available at http://www.nasa.gov/lb/offices/ocfo/references/ocfo_WCF_detail.html. 56   See http://www.nasa.gov/pdf/107225main_FY03%20approps%20working%20cap%20langOriginalFINAL.pdf.

OCR for page 41
Aeronautics Innovation: NASA’s Challenges and Opportunities tion technology procurement group, called Science and Engineering Workstation Procurement, that acquires computers and related equipment on a transfer fee basis for programs throughout NASA. Because the WCF legislation extends to services, consideration should be given to extending the idea to aeronautics wind tunnel facilities and to R&D services more broadly. Annual budget cycles would apply to the procurement projects, but management of the provision of services would have considerably more discretion and enable longer term planning. Recommendation 6-D: ARMD should explore establishing WCF structures for wind tunnels and aeronautics R&D services. Earlier we described the increasing incidence of congressionally directed projects, most of which are unfunded, that is, they are mandated with the expectation that NASA will perform the tasks within the agency’s current budget. The managers we spoke to complained not about their value—“most are good things to do”—but about their disruptive effective on planning. We suggest that every effort be made to align these activities with established programs. This may be most feasible with projects that reflect congressional concern that some important public good objective is being neglected in NASA’s planned activities. However, some earmarked projects bear little relationship to NASA’s mission. In those cases, a separate budget account should be created for managing them. Recommendation 6-E: ARMD should negotiate with congressional sponsors and earmark recipients to align mandated activities better with established programs and should assign the projects to a separate budget account and management area. The immediate effect of a separate budget for congressionally directed projects would be to reduce the apparent size of the balance of the ARMD budget and seemingly narrow the discretion of associated program managers. However, real discretion over the balance of the program would increase. In 1995 approximately one-quarter of DARPA’s $2.5 billion budget was earmarked.57 The director ceded control and responsibility for the ear- 57   Comments by David Whelan, Boeing Skunk Works, at the committee’s workshop, January 18, 2005.

OCR for page 41
Aeronautics Innovation: NASA’s Challenges and Opportunities marked projects to the military services, reducing DARPA’s budget to $1.9 billion. But the transfer significantly improved budget flexibility and stability, resulting in a healthier technical management environment. NASA should consider following this model. ORGANIZATION OF NASA AERONAUTICS R&D As noted in the preface to this report, the administration’s policy preference is to shrink ARMD’s resources and portfolio on the assumption that a prominent government role in aeronautics R&D is no longer justified. The majority view in the technical, industrial, and academic communities appears to be the opposite: national technology needs in aeronautics are broad, compelling, and inadequately served by ARMD’s declining resources. If the first course prevails, ARMD’s subordinate role in NASA is appropriate. Its job will be to conduct fewer projects more efficiently while managing the contraction of three research centers. Eventually, lacking unique robust technical capabilities, it will go out of business. However, in the event that stakeholders mobilize effectively in support of an expansionist program, other forms of organization may be worth considering. The President’s 2004 Commission on Implementation of United States Space Exploration Policy (the Aldridge Commission), which among other things recommended a restructuring of NASA’s research centers, considered the option of removing the aeronautics R&D program from NASA altogether. The principal reason the commission gave for rejecting that alternative was ARMD’s involvement in addressing planetary atmospheric transportation as an eventual component of space exploration. In other words, space program needs dictated the conclusion, not the direct needs of aeronautics, even though an independent organization might be able to contract with NASA to support the Mars mission. Another way to elevate the importance of the aeronautics portfolio and provide some protection from the demands of the space program is an agency-within-an-agency arrangement. In this case, too, NASA’s space program could contract with NASA’s aeronautics program for planetary aircraft work, but it would be more difficult to divert aeronautics resources to space activities. The Defense Department and the military services could similarly directly contract for aeronautics R&D services. DARPA has operated along these lines since its creation in 1958, reporting directly to the secretary of defense and operating independently of the other military R&D establishments. In a fee-for-service manner, DARPA subcontracts for most

OCR for page 41
Aeronautics Innovation: NASA’s Challenges and Opportunities support services and infrastructure. Intelsat is a related example, in which bonds issued against user-fee revenue streams help pay for long-term technology and infrastructure. Another precedent, closer in time and related in function, is the Air Traffic Organization (ATO), established within the FAA in February 2004 with its own chief operating officer and 36,000 employees.58 ATO organizationally combines responsibility for air traffic operations, equipment acquisition, and research, separate from FAA’s regulatory role. The committee is not recommending either reorganization. That would be premature as well as beyond our mandate and competence. Rather we are underscoring our belief that the implications of the current policy divide are far-reaching—for NASA, for innovation, and for the nation’s aviation sector. Until the divide is bridged, our management advice, although we hope useful, is a secondary priority. 58   See the May 20, 2005 FAA organizational chart, available at http://www.faa.gov/aba/html_pm/mi/files_doc/HQ-ORG.DOC.

OCR for page 41
Aeronautics Innovation: NASA’s Challenges and Opportunities This page intentionally left blank.