Click for next page ( 2


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 1
THE NATIONAL ACADEMIES Advisers to the Nation on Sciente, Engineering, and Medicine Division on Engineering and Physical Sciences Aeronautics and Space Engineering Board Dr. Richarc! Wlezien Program Director Vehicle Systems Program National Aeronautics and Space Administration Washington, DC 20546 Dear Dr. Wlezien: 500 Fifth Street, NW Keck 1 002-A Washington, DC 20001 Phone: 202 334 2668 Fax: 202 334 2482 Email: aseb@nas.edu www.national-academies.org/deps January 5' 2004 This letter report summarizes the findings ant! recommendations of the National Research Council's (NRC's) Pane! on the Vehicle Systems Program (VSP) on the progress made by the VSP over the past 9 months in formulating ant! directing its program. The panel's review centers on the technical merit of the projects and tasks in the VSP as well as the planned program structure and rationale for meeting top-lever goals. This letter report follows the full NRC report An Assessment of NASA 's Aeronautics Technology Programs, which was authored by the Committee for the Review of NASA's Revolutionize Aviation Program ant! was releaser! to the public on November ~ 8, 2003. ~ The VSP pane] was one of three panels reporting to that committee. The VSP pane! began its review of NASA's VSP on March 17, 2003, in Washington, D.C. The review continues! with numerous subpane] visits to the NASA centers where a large amount of VSP work was taking place. The pane! issued guidance to the full committee on the state of the VSP as it was ~ief~ned in March 2003, and the committee aclopted the panel's findings ant! recommendations, inclucling them in its report. The assessments macle by the VSP pane! at the site visits to NASA centers were cletailect in nature and focuses! down to the task level. Thus, the top-teve! program assessment was consistent with the timing of these reviews, which were conducted in March 2003. For complete cietaiis of center visits and detailed Anglings ant! recommendations, please see the full committee report, issued on November ~ 8. NASA requested that the VSP panel reconvene after the committee's report had been released and the main review completed. The purpose of the two-clay meeting, held on November ~ 9-20, 2003, was to review ant! comment on progress and changes that hac} occurred in the structure and management of the VSP subsequent to March 2003. The results of the new review are contained herein. The recommendations contained in this letter report, in almost all instances, amplify ant] reinforce recommendations and observations contained in the full committee report. In selected cases, some further observations have been made based on more recent data, which supersede data provided in the earlier full committee report. ~ The published report is forthcoming from National Academies Press, Washington, D.C., in January 2004. NATIONAL ACADEMY OF SCIENCES NATIONAL ACADEMY OF ENGINEERING INSTITVIE OF MEDICINE NATIONAL RESEARCH COUNCIL

OCR for page 1
ACKNOWLEDGMENT OF REVIEWERS This letter report has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise, in accordance with procedures approved by the National Research Council's Report Review Committee. The purpose of this independent review was to provide candid and critical comments that will assist the institution in making its publisher! report as sound as possible and to ensure that the report meets institutional stanclarcis for objectivity, evidence, and responsiveness to the stucly charge. The review comments and draft manuscript remain confidential to protect the integrity of the cleliberative process. We wish to thank the following individuals for their review of this report: Ear] H. Dowell (NAE), Duke University; Gordon McKinzie, Uniter! Airlines, retired; Richarc! Petersen (NAE), NASA, retired; ant! William Agnew, General Motors Corporation, retired. Although the reviewers listed above have provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations, nor clid they see the final ciraft of the report before its release. The review of this report was overseen by Louis Lanzerotti, Lucent Technologies. Appointec! by the National Research Council, he was responsible for making certain that an inclepenclent examination of this report was carried out in accordance with institutional procedures and that all review comments were carefully consicierect. Responsibility for the fine] content of this report rests entirely with the authoring panel and the institution. UPDATED ASSESSMENT OF NASA'S VEHICLE SYSTEMS PROGRAM Since the beginning of the review in March 2003 for the full committee report, the VSP has undergone significant change and restructuring. Seven existing programs, each with a bottom-up content formulation, were brought uncler the umbrella of the VSP, and NASA management had the daunting task of bringing cohesion to a diverse array of research tasks. This letter report provides the VSP panel's assessment of the current state of the VSP research program restructuring. Because there was so little time between the release of the full committee report and the November ~ 9-20 review, the pane! ctid not determine how well NASA had responclect to the guidance in the full report. The panel did, however, use the top-level recommendations contained in Chapter I, "Top-Level Assessment," of the full report to structure the assessment here. The pane! has farther explained those recommendations baser! on the current status of the VSP. Of the 12 top-level recommendations in the full report, recommendations ~ ant! ~ O are not included here as they are not as directly relevant to the VSP ant! this new review. Major Findings and Key Issues in aligning VSP goals and program organization structure. The team has begun the process of prioritizing and aligning projects uncler the VSP. The pane! commencis the VSP for its openness during this review ant! for taking the initiative to solicit further comments from the pane! ant! other outside sources. The NASA The VSP team has made significant progress 2

OCR for page 1
presenters were refreshingly candid in their presentations and assessments of the work being conducted. VSP management explicitly requested that the pane] answer three questions: What is being done well within VSP? What needs further attention and improvement? What has been overlooked and needs to be added to the program? In answering these questions, the pane} identified four key issues that are discussed in detail in the 10 top-level recommendations below. These four issues are summarized as follows: . The top-down planning and program formulation process being put in place by the VSP appears sound, and implementation has begun. However, the reporting, accountability, and other management tools necessary for organizations of this type and size appear to be disjointed. The Finds management accountability and responsibility/accountability links between the program and NASA centers are dysfunctional and seriously jeopardize VSP's ability to successfully transform and focus NASA's work on vehicle systems. For many years NASA has had a problem with maintaining aeronautics facilities owing to declining use and budgets. Although the required paring down of fixed assets to meet future visions and needs is being implemented, it does not appear to be progressing in a strategic manner but instead in a reactive way, driven by near-term budget needs and accounting rules. While full-cost accounting has compounded this situation by heightening the visibility of NASA's ability to maintain aeronautics facilities, the reality is that NASA has yet to aclequately address this critical issue at the headquarters level. The pane! appreciates that the VSP is not responsible for making the determination of which test capabilities wild be retained and which lost. However, the pane} points out that decisions on infrastructure may adversely impact the VSP's ability to successfully demonstrate and transition technologies to an end- user, a higher technology readiness level (TRL) program, or the commercial marketplace. Centers often assign more staff to projects than is required by programs, given the often limited budgets of the programs. Little if any paring down of tasks has occurred to date, but the plans shared with the panel call for poorly linked or lower-priority projects to be closed out over the next 3 to 6 months. However, it is not clear how reduction in efforts can be done efficiently given the existing limitations on hiring and firing personnel because, under the new full-cost-accounting regime, programs must pay for staff out of limited and declining budgets. The pane! strongly urges the VSP and NASA to do everything within its power to correct this situation. If a solution cannot be found within a short period of time, NASA will have to consider the last alternative - initiating a reduction in force (RIF). Something must be done soon or the VSP may never be successfully transformed into a new, efficient and effective entity that will allow NASA to regain or retain a position of leadership in selected capability areas. And, even though NASA faces severe problems in overstaffing, it is important for the agency to simultaneously implement hiring practices that ensure a steady flow of new talent.

OCR for page 1
. While some of the full report's programmatic recommendations have been implemented, it was not clear if projects had truly been recluced in terms of content or if centers ant] lower-level managers have simply consoliciated projects and tasks without significantly reducing their content to match limiter! and declining budgets. In any case, resources for high-priority programs have not been increaser! as a result of this consolidation. This gives the impression that some work is being perpetuates} based on the interests of specific researchers or centers, rather than the needs of the nation. Discussion Top-Leve} Recommendation 2 NASA shop provide Boric! leadership in aeronautics research ant! alevelopment The VSP has creates! a process for planning projects and tasks that links NASA's national science and technology policy to the objectives of four aeronautics themes: Protect the environment, Increase mobility, Explore new aerospace missions, ant! Partner for national security. These, in turn, flow clown to six long-term strategic technology focus areas. To iclentiiPy specific projects ant! tasks that should be pursued, the VSP has developer! six vehicle concepts comprising multiple vehicle sectors from supersonic to extreme short takeoff to highlight potential technology applications that align with the technology focus areas. This program planning activity has producecI a vision that should allow NASA to establish worIct-ciass leadership in: L;nvironment-friendly, clean-burning engines, New energy sources for aircraft ant] power management, Quiet aircraft for community-friencIly service, Aeroclynarnic performance for fuel efficiency, Aircraft weight reduction, and Smart aircraft and autonomous control. Six of the V:SP's seven projects are closely related to these technology focus areas. The seventh project provides flight and system demonstration services. The acictition of the vehicle integration strategy technical analysis (VISTA) team near the top of VSP management provides the important cross-program integration ant! requirements assessment that were missing when the review started in March 2003. VISTA is an important capability for prioritizing program activities, assessing return on investment, and monitoring progress in terms of key performance objectives. Although considerable talent aIreacly exists to support VISTA, the addition of some senior staff with experience in system integration and 4

OCR for page 1
performance would help the VSP to fully exploit the VISTA concept. It is important to have people capable of creating requirements that can flow down to the technical researcher level in a way that is meaningful and traceable. The pane] recommends that the VISTA team also assess the viability of the markets for the end products of the technologies being developed. Simply put, technology should be developed only if it is marketable and has an internal NASA customer or external government or commercial sector end-user. In order to increase the possibility for technology transfer, VISTA may want to consult with industry marketing managers who are forecasting new product development. Top-Leve! Recommendation 3 NASA has many excellent technicalpersonne! ancifacitities to achieve its aeronautics technology objectives but should improve its processes for program management. Significant progress has been made in recognizing the need for a VSP-wicie approach to program management that is consistent at all NASA centers. The pane! commends the VSP on its reorganization, which it appears will result in better top-down management. However, the pane] has substantial concerns that NASA's management processes decouple responsibility to meet goals and accountability for the end results from the authority to manage fiends. The VSP manager cannot implement the changes needed to make the VSP efficient and effective unless drastic changes are made to the financial, personnel, and operational management structure of the NASA centersan interrelationship referred to in this report as the "Center-Program Relationship." The funds management accountability and responsibility/accountability links between the VSP and NASA centers are dysfunctional and seriously jeopardize the ability of the VSP to successfully transform and focus NASA's vehicle systems work. Currently NASA headquarters program managers have responsibility for achieving goals but do not have the authority to manage the allocation of all funds. This system is not consistent with standard business practices, which link responsibility for meeting goals with total authority ~ over managing finances. The VSP program manager also has no control over staffing. Centers may allocate more staff to projects than are required by programs. Since programs are now required to pay for unneeded staff out of limited and declining budgets, the situation seriously affects the amount of research and development that can be completed by the VSP. In addition, there is no process to ensure a steady flow of new talent into the program. The new VSP planning process shared with pane! members shows that poorly linked or lower-priority projects and tasks will be terminated over the next 3 to 6 months. However, the pane! was informed that the centers will not decrease the personnel associated with terminated projects or tasks. The pane! strongly urges the VSP and NASA to do everything within its power to correct this situation. If a solution cannot be found within a short period of time, NASA must consider the last alternative, initiating a reduction in force (RIF). Something must be done soon or the VSP may never be transformed to a new, efficient and effective entity. And, even though NASA faces severe problems in overstaffing, it is important for the agency to implement hiring practices that ensure a steady flow of new talent. s

OCR for page 1
Finally, effective program management requires transparent finances anti schedules and clear means of measuring progress against key performance parameters. The pane! noted that there are no effective procedures for enforcing accountability or for providing transparent reporting on all tasks anti all finances from the centers to the VSP office. The pane] recommencis that NASA follow industry's lead and construct an operating mode] that links responsibility for meeting goals, and accountability for the ens! results, with the authority to manage functs. The goal is to give the program manager the authority to direct and account for funds ant! personnel effectively and transparently. Top-Leve! Recommendation 4 MESA should eliminate arbitrary time constraints on program completion and schedule key milestones baser! on task complexity and technology maturity. The recently restructurer! VSP plans shares! with the pane! predict that measurable progress will be macle in 3- to 5-year windows. As such, the plans have effectively acldressect the panel's earlier concern that the sunset manciatec} by the Office of Management ant! Budget wouIc! preclude clevelopment of high-risk, high-payoff technologies that might span more than the 5- year sunset Winslow. A remaining concern is that program plans at the task level often have no clear exit criteria that specify when research is complete or really for transition to industry or other agencies, yet such a specification is absolutely necessary for cleaning decision gateways. While a milestone tracks progress, that is, gives a measure of plannecl technical success against a specific schedule, a gateway should be a precleterminec! point at which research is assessed ant! a decision is made about a project's or task's continuation, cancellation, or redirection. It was not clear to the pane! that all the NASA managers know the distinction between milestones ant! gateways. The VSP has acknowledged the neec! for exit criteria and decision gateways and has oracle some attempts to establish them. However, they are neither uniformly established across all projects nor effectively implemented. The pane} recommencis that individual PIs set exit criteria and decision gateways for their tasks, as long as the criteria ant! gateways define clear levels of achievement ant! are approved by upper-level program managers. It further recommends that these decision gateways be incorporated in the restructured VSP program plans and that VSP management clefine a common gateway ant} milestone template for use across all projects under the VSP. Top-Leve! Recommendation 5 NASA shouic! reduce the number of tasks in its aeronautics technology portfolio. Strategic technology focus areas have been clefinect and are described under recommendation 2, above. However, as states! in the full report on page ~ 3, the VSP "has not yet reclucecl the [number of tasks] in the face of changing market needs and reduced budgets for vehicle systems throughout the ~ 990s and early 2000s. Rather, NASA has left the same broad set of capabilities in place, with each portion of VSP research forced to operate on ever smaller budgets. As a consequence, some (not alI) of the current VSP projects ant! tasks find themselves on budgetary 'life support."' 1 J - 6

OCR for page 1
The pane] again recommends that the VSP create a rank-orclerect set of core competency areas, as describer! in detail under "Key Issue I: Core Competencies" in Chapter 2 of the full report. There are indications that the NASA centers are addressing the neec! to recluce the number of projects and tasks. However, it was not clear if projects hac! truly been recluced in terms of content or if centers and lower level managers have simply consoliciated projects and tasks without significantly reducing their content. In any case, resources for high-priority programs have not been increaser! as a result of this consolidation. For example, the funds used for the pulse detonation engine project (also known as the constant volume combustion engine) shouic! be redirected! to other, more important efforts. The pane! acknowleciges that even given the short time between release of the full report and this review, the VSP die! an excellent job of responding to the panel's concerns about engine noise in connection with the pulse detonation engine project. However, the cletailect response raised additional serious technical issues that recluce the likelihood of success for this high-risk technology project. The panel reiterates its recommendation that the project be cancelled. One newly apparent technical issue arising cluring the panel's November meeting is that turbine inlet temperature will rise by ~ 000 degrees as a result of constant volume combustion. No materials exist today that can function successfully in this environment, and the likelihood that they will appear in the future is small. In addition, pulsing the gas stream will greatly increase the number of fatigue cycles accumulates! by the blades, adversely affecting blacle durability and reliability. The consequences of unsteacly-flow physics for performance, cooling neecis, and cooling forcing function appear to be overwhelmingly necative. so research in this area should be discontinued. c, , ~ . , The VSP managers have not yet effectively reducer! the content of the VSP overall, ant! it was not clear to the pane! that this step will be taken even given the careful planning from the program management level at NASA headquarters. Incleed, the pane! does not believe that reducing the VSP's overall content can be accomplished unless the problems with the Center- Program Relationship cliscussec! above under recommendation 3 are corrected. The pane} appreciates that the VSP is still in the midst of transforming itself. VSP management inclicatec! that the number of VSP tasks will be reduced over the next 3 to 6 months as the remainder of the new program planning process is put in place. The choices and changes aheac! will be difficult, but they are essential if the VSP program is to do the best it can for the national good with the Angling it has at its disposal. The pane! recommencis that NASA make these hard choices. Without them, money will continue to be spread across too many programs, and NASA will become a follower rather than a leacler in many of the areas it today considers to be its core competencies. Top-Leve! Recommendations 6 and 7 NASA shouldpursue more high-risk, high-payoff technologies. NASH should reconstitute a long-term base research program, separate from the other aeronautics technology programs and projects 7

OCR for page 1
VSP management expects to allocate at least 20 percent of VSP funding as seed corn for basic research, from which large-scale activity wild spring. VSP currently plans to have the basic research fending and funding decisions resicle at the project level. To improve the basic research that takes place under the VSP, the pane] recommencts that the 20 percent be viewed as a separate long-term investment portfolio for the VSP as a whole, in orcler to create synergies across the program. VSP top-level management should use this Angling as an incentive for the projects to ensure that the research is consistent with NASA's national science and technology policy and is directed toward high-risk, core research areas. The basic research portfolios can be managed within each of the six technology project portfolios but must be measurable and manageable from the top level of the VSP. If the funding for basic research is centralizer! at the VSP manager level, the overall goals and objectives of the VSP will be automatically built into the basic research program. The panel also notes that university research is an important part of a high-risk, high- payoff investment strategy and should therefore continue to receive appropriate funding. The pane! is concernec! that although current VSP planning specifies a 20 percent target level for basic research, there is no way to know what is actually being spent in these areas given the Center-Program Relationship. Unless the NASA centers are given a top-clown process that requires bottom-up reporting, the 20 percent objective may never be met. The VSP's current approach integrating the basic research program into the strategic focus areas is not responsive to Recommendation 7. The approach suggested here by the panel, which will still allocate at least 20 percent of each area's budget to acivanced technology projects, is a different method of meeting recommendation 7, but this approach will require discipline on the part of both center and program office personnel to ensure the intent is met. While perhaps a fine distinction to some, the VSP's current program management plan, as presented to pane} members, uses strategic focus technologies as a surrogate for basing the technology development plan on core competencies. The panel acknowledges that this substitution may accomplish the clesirec! effect of embedding worIcI-cIass expertise at NASA. However, there is concern that the division of responsibilities between the NASA centers and the VSP office may prevent the {lesirecl core competencies from being established within NASA. A key example of a NASA core competency is the agency's ability to generate highly valued analytical tools that can be applied to a broad spectrum of disciplines, including materials, aerodynamics, ant! structures. These tools have traditionally been made available to, ant! are wiclely used by, researchers and industry. Top-Leve! Recommendation ~ NASA 's aeronautics technology infrastructure exceeds its current needs, anc! the agency shouic! continue to dispose of unclerutilizec! assets anciffacilities. For many years NASA has hack a problem with maintaining aeronautics facilities, including wind tunnels, engine turbine facilities, ant! large test rigs, owing to declining use and budgets The aeronautics budget was cut by more than 33 percent in the past 5 years, and is no longer sufficient to sustain all of NASA's aeronautics facilities. Full-cost accounting has heightened the visibility of the problem and has shifted the majority of the facilities' maintenance costs from the NASA centers to the projects that fund tasks using the facilities. However, regardless of full-cost accounting issues, NASA has yet to aclequately aciciress this 8

OCR for page 1
critical problem. The pane! appreciates that the VSP is not responsible for making the determination of which test capabilities will be retained and which lost. However, the pane! points out that decisions on infrastructure may adversely impact the VSP's ability to successfully demonstrate and transition technologies to the marketplace. Aeronautics infrastructure reduction activities to ciate indicate that, although some action is being taken to pare clown fixed assets, the infrastructure reduction is apparently being implementer! in a reactive way, in response to budgets ant! accounting rules, rather than in accord with strategic planning baser! on long-term national needs. For instance, there is a possibility that the clual-spoo] turbine facility at NASA Glenn, which was identified in the full report as a national asset, may be in jeopardy clue to budget considerations. The pane! recommencis that reduction in facilities be aciciressed on a strategic basis so as to keep ant! maintain required national assets, rather than by an indirect process in reaction to annual budgets ant] near-term program plans. Top-Leve} Recommendation 9 NASA should implement full-cost accounting in a way that avoids unintenclec! consequences harmful to the long-term health of the aeronautics program. In the full report, the pane] expressed concern that under a full-cost accounting scheme the cost of flight testing, wine! tunnel testing, or the use of full-scale test rigs wouict be prohibitively expensive for projects. As a result, projects would be tailored towards less ambitious, lower-level TRL pursuits for test analysis or computational investigative methods so as to avoid paying the full cost of maintaining the facilities that enable tasks and technologies to reach a higher TRE. Another unintended consequence of full-cost accounting is to disturb the critical balance between empirical and numerical methods that reacts to technical maturity. The end result is that the project cleliverables will be less attractive to industry and less likely to be implemented into actual flight platforms, threatening the successful completion of VSP investments in a variety of technologies. The pane! reiterates Top-Leve] Recommendation 9 from the full report and recommends that the NASA associate administrator for aerospace technology put policies into effect that acIdress the recommendation. Top-Leve} Recommendation Il NASA shoulal seek better feedback from senior management in industry and other government organizations. VSP managers spoke of how they have started interfacing with top-level executives at U.S. aeronautics companies. This interaction is commendable, and the pane} believes it is required and should continue. By the same token, VSP management should not be responsible for initiating this interaction on their own; rather, the associate administrator for aerospace technology should create mechanisms at the highest level to ensure that this interaction is taking place. The pane] therefore recommencts that the NASA associate administrator for aerospace technology aciciress the issue by reestablishing a tiered} advisory system that gives NASA input on aeronautics issues from both technical ant} business managers. This issue is ac3dressect in cletai! in the full report. 9

OCR for page 1
Top-Leve} Recommendation 12 NASA shouic! conduct research in selective areas relevant to rotorcraft. The pane! commencts the VSP for taking positive action in rotorcraft research. The current plan inclucles rotorcraft research that leverages investments by the Army and by other, relater! NASA programs. The pane] urges the VSP to tailor these tasks so that they are consistent with the NASA theme objectives and strategic technology focus areas as presented by VSP managers to the panel. On behalf of the NRC's Pane} on the Vehicle Systems Program, T wouic} like to thank you and your team for the hard work you put into these reviews and for your open and frank responses during our many question-and-answer sessions. Working with you and all of the VSP personnel these past several months has been a pleasure. ~ hope that our work is helping you to achieve your objectives. Sincerely, A_ 4 ~ Thomas L. Williams, Chair Panel on the Review of NASA's Vehicle Systems Program cc: Victor Lebacqz Terry Hertz Peter Blair Elizabeth Panos Janice Mehier George Kevin Douglas Bennett Maureen Mellocly Attachments: 1. Pane] Member Biographies 2. Final Review Agenda 10