National Academies Press: OpenBook
« Previous: 1 Top-Level Assessment
Suggested Citation:"2 Assessment of the Vehicle Systems Program." National Research Council. 2004. Review of NASA's Aerospace Technology Enterprise: An Assessment of NASA's Aeronautics Technology Programs. Washington, DC: The National Academies Press. doi: 10.17226/10861.
×
Page 9
Suggested Citation:"2 Assessment of the Vehicle Systems Program." National Research Council. 2004. Review of NASA's Aerospace Technology Enterprise: An Assessment of NASA's Aeronautics Technology Programs. Washington, DC: The National Academies Press. doi: 10.17226/10861.
×
Page 10
Suggested Citation:"2 Assessment of the Vehicle Systems Program." National Research Council. 2004. Review of NASA's Aerospace Technology Enterprise: An Assessment of NASA's Aeronautics Technology Programs. Washington, DC: The National Academies Press. doi: 10.17226/10861.
×
Page 11
Suggested Citation:"2 Assessment of the Vehicle Systems Program." National Research Council. 2004. Review of NASA's Aerospace Technology Enterprise: An Assessment of NASA's Aeronautics Technology Programs. Washington, DC: The National Academies Press. doi: 10.17226/10861.
×
Page 12
Suggested Citation:"2 Assessment of the Vehicle Systems Program." National Research Council. 2004. Review of NASA's Aerospace Technology Enterprise: An Assessment of NASA's Aeronautics Technology Programs. Washington, DC: The National Academies Press. doi: 10.17226/10861.
×
Page 13
Suggested Citation:"2 Assessment of the Vehicle Systems Program." National Research Council. 2004. Review of NASA's Aerospace Technology Enterprise: An Assessment of NASA's Aeronautics Technology Programs. Washington, DC: The National Academies Press. doi: 10.17226/10861.
×
Page 14
Suggested Citation:"2 Assessment of the Vehicle Systems Program." National Research Council. 2004. Review of NASA's Aerospace Technology Enterprise: An Assessment of NASA's Aeronautics Technology Programs. Washington, DC: The National Academies Press. doi: 10.17226/10861.
×
Page 15
Suggested Citation:"2 Assessment of the Vehicle Systems Program." National Research Council. 2004. Review of NASA's Aerospace Technology Enterprise: An Assessment of NASA's Aeronautics Technology Programs. Washington, DC: The National Academies Press. doi: 10.17226/10861.
×
Page 16
Suggested Citation:"2 Assessment of the Vehicle Systems Program." National Research Council. 2004. Review of NASA's Aerospace Technology Enterprise: An Assessment of NASA's Aeronautics Technology Programs. Washington, DC: The National Academies Press. doi: 10.17226/10861.
×
Page 17
Suggested Citation:"2 Assessment of the Vehicle Systems Program." National Research Council. 2004. Review of NASA's Aerospace Technology Enterprise: An Assessment of NASA's Aeronautics Technology Programs. Washington, DC: The National Academies Press. doi: 10.17226/10861.
×
Page 18
Suggested Citation:"2 Assessment of the Vehicle Systems Program." National Research Council. 2004. Review of NASA's Aerospace Technology Enterprise: An Assessment of NASA's Aeronautics Technology Programs. Washington, DC: The National Academies Press. doi: 10.17226/10861.
×
Page 19
Suggested Citation:"2 Assessment of the Vehicle Systems Program." National Research Council. 2004. Review of NASA's Aerospace Technology Enterprise: An Assessment of NASA's Aeronautics Technology Programs. Washington, DC: The National Academies Press. doi: 10.17226/10861.
×
Page 20
Suggested Citation:"2 Assessment of the Vehicle Systems Program." National Research Council. 2004. Review of NASA's Aerospace Technology Enterprise: An Assessment of NASA's Aeronautics Technology Programs. Washington, DC: The National Academies Press. doi: 10.17226/10861.
×
Page 21
Suggested Citation:"2 Assessment of the Vehicle Systems Program." National Research Council. 2004. Review of NASA's Aerospace Technology Enterprise: An Assessment of NASA's Aeronautics Technology Programs. Washington, DC: The National Academies Press. doi: 10.17226/10861.
×
Page 22
Suggested Citation:"2 Assessment of the Vehicle Systems Program." National Research Council. 2004. Review of NASA's Aerospace Technology Enterprise: An Assessment of NASA's Aeronautics Technology Programs. Washington, DC: The National Academies Press. doi: 10.17226/10861.
×
Page 23
Suggested Citation:"2 Assessment of the Vehicle Systems Program." National Research Council. 2004. Review of NASA's Aerospace Technology Enterprise: An Assessment of NASA's Aeronautics Technology Programs. Washington, DC: The National Academies Press. doi: 10.17226/10861.
×
Page 24
Suggested Citation:"2 Assessment of the Vehicle Systems Program." National Research Council. 2004. Review of NASA's Aerospace Technology Enterprise: An Assessment of NASA's Aeronautics Technology Programs. Washington, DC: The National Academies Press. doi: 10.17226/10861.
×
Page 25
Suggested Citation:"2 Assessment of the Vehicle Systems Program." National Research Council. 2004. Review of NASA's Aerospace Technology Enterprise: An Assessment of NASA's Aeronautics Technology Programs. Washington, DC: The National Academies Press. doi: 10.17226/10861.
×
Page 26
Suggested Citation:"2 Assessment of the Vehicle Systems Program." National Research Council. 2004. Review of NASA's Aerospace Technology Enterprise: An Assessment of NASA's Aeronautics Technology Programs. Washington, DC: The National Academies Press. doi: 10.17226/10861.
×
Page 27
Suggested Citation:"2 Assessment of the Vehicle Systems Program." National Research Council. 2004. Review of NASA's Aerospace Technology Enterprise: An Assessment of NASA's Aeronautics Technology Programs. Washington, DC: The National Academies Press. doi: 10.17226/10861.
×
Page 28
Suggested Citation:"2 Assessment of the Vehicle Systems Program." National Research Council. 2004. Review of NASA's Aerospace Technology Enterprise: An Assessment of NASA's Aeronautics Technology Programs. Washington, DC: The National Academies Press. doi: 10.17226/10861.
×
Page 29
Suggested Citation:"2 Assessment of the Vehicle Systems Program." National Research Council. 2004. Review of NASA's Aerospace Technology Enterprise: An Assessment of NASA's Aeronautics Technology Programs. Washington, DC: The National Academies Press. doi: 10.17226/10861.
×
Page 30
Suggested Citation:"2 Assessment of the Vehicle Systems Program." National Research Council. 2004. Review of NASA's Aerospace Technology Enterprise: An Assessment of NASA's Aeronautics Technology Programs. Washington, DC: The National Academies Press. doi: 10.17226/10861.
×
Page 31
Suggested Citation:"2 Assessment of the Vehicle Systems Program." National Research Council. 2004. Review of NASA's Aerospace Technology Enterprise: An Assessment of NASA's Aeronautics Technology Programs. Washington, DC: The National Academies Press. doi: 10.17226/10861.
×
Page 32
Suggested Citation:"2 Assessment of the Vehicle Systems Program." National Research Council. 2004. Review of NASA's Aerospace Technology Enterprise: An Assessment of NASA's Aeronautics Technology Programs. Washington, DC: The National Academies Press. doi: 10.17226/10861.
×
Page 33
Suggested Citation:"2 Assessment of the Vehicle Systems Program." National Research Council. 2004. Review of NASA's Aerospace Technology Enterprise: An Assessment of NASA's Aeronautics Technology Programs. Washington, DC: The National Academies Press. doi: 10.17226/10861.
×
Page 34
Suggested Citation:"2 Assessment of the Vehicle Systems Program." National Research Council. 2004. Review of NASA's Aerospace Technology Enterprise: An Assessment of NASA's Aeronautics Technology Programs. Washington, DC: The National Academies Press. doi: 10.17226/10861.
×
Page 35
Suggested Citation:"2 Assessment of the Vehicle Systems Program." National Research Council. 2004. Review of NASA's Aerospace Technology Enterprise: An Assessment of NASA's Aeronautics Technology Programs. Washington, DC: The National Academies Press. doi: 10.17226/10861.
×
Page 36
Suggested Citation:"2 Assessment of the Vehicle Systems Program." National Research Council. 2004. Review of NASA's Aerospace Technology Enterprise: An Assessment of NASA's Aeronautics Technology Programs. Washington, DC: The National Academies Press. doi: 10.17226/10861.
×
Page 37
Suggested Citation:"2 Assessment of the Vehicle Systems Program." National Research Council. 2004. Review of NASA's Aerospace Technology Enterprise: An Assessment of NASA's Aeronautics Technology Programs. Washington, DC: The National Academies Press. doi: 10.17226/10861.
×
Page 38
Suggested Citation:"2 Assessment of the Vehicle Systems Program." National Research Council. 2004. Review of NASA's Aerospace Technology Enterprise: An Assessment of NASA's Aeronautics Technology Programs. Washington, DC: The National Academies Press. doi: 10.17226/10861.
×
Page 39
Suggested Citation:"2 Assessment of the Vehicle Systems Program." National Research Council. 2004. Review of NASA's Aerospace Technology Enterprise: An Assessment of NASA's Aeronautics Technology Programs. Washington, DC: The National Academies Press. doi: 10.17226/10861.
×
Page 40
Suggested Citation:"2 Assessment of the Vehicle Systems Program." National Research Council. 2004. Review of NASA's Aerospace Technology Enterprise: An Assessment of NASA's Aeronautics Technology Programs. Washington, DC: The National Academies Press. doi: 10.17226/10861.
×
Page 41
Suggested Citation:"2 Assessment of the Vehicle Systems Program." National Research Council. 2004. Review of NASA's Aerospace Technology Enterprise: An Assessment of NASA's Aeronautics Technology Programs. Washington, DC: The National Academies Press. doi: 10.17226/10861.
×
Page 42

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

2 Assessment of the Vehicle Systems Program . BACKGROUND Program Information The Vehicle Systems-Program (VSP) is divided into seven projects that contain 172 tasks. Table 2-1 lists the VSP budget for FY03 and FY04. The values are listed in full-cost accounting, where the cost of civil servant salaries and all support infrastructure is in- cluded in the budgets of individual projects, as dis- cussed below. Figure 2-1 shows a program organiza- tion chart for the VSP. The VSP contains seven projects: · Breakthrough Vehicle Technologies. Develops high-risk, high-payoff technologies that will dramatically and substantially improve vehicle As- . . . ettlclency ant emissions. Quiet Aircraft Technology. Discovers, devel- ops, and verifies, in the laboratory, technolo- gies that improve the quality of life by reducing society's exposure to aircraft noise. Twenty-first Century Aircraft Technology. Develops and validates, through ground- . . TABLE 2-1 Net Budget for the Vehicle Systems Program Budget (million $) NASA No. and Project Name FY03 FY04 Vehicle Systems 604.6 573.5 1.0 Breakthrough Vehicle Technologies 124.2 115.3 2.0 Quiet Aircraft Technology 41.4 60.2 3.0 Twenty-first Century Aircraft Technology 46.0 42.0 4.0 Advanced Vehicle Concepts 72.5 41.0 5.0 Flight Research 91.4 85.4 6.0 Ultra-Efficient Engine Technology 87.8 90.0 7.0 Propulsion and Power 141.3 139.6 SOURCE: Information provided by R. Wlezien, VSP Project Manager, NASA Headquarters. 9

:t 10 AN ASSESSMENT OF NASA 'S AERONA UTICS TECHNOLOGY PROGRAMS Vehicle Systems Program 1.0 Breakthrough Vehicle Technologies (BVT) 2.0 Quiet Aircraft Technology (QAT) 3.0 21 st Century Aircraft Technology (TCAT) 4.0 Advanced Vehicle Concepts (AVC) 5.0 Flight Research 6.0 Ultra-Efficient Engine Technology (UEET) 1 1 1 ~ 1 1 1 1 1 1 1 1.1 Morphing it, 7.0 Propulsion and Power 2.1 Airframe System Noise Reduction (ASNR) 3.2 Efficient Aerodynamic Shapes and Integration (EASI) 1.3 Super Lightweight Multifunctional Systems Technologies 2.3 Engine System Noise Reduction (ESNR) 1 , - 11 1 1 1 1 .4 Advances Through Cooperative Efforts (ACE) 1 .5 Aerospace Systems Analysis Project (ASAP) 1.6 Robust Aerospace Systems 4.2 Revolutionary Aircraft Flight Validation (RAFV) 3.3 Integrated Tailored Structures (ITS) 3.4 Green Efficient Aircraft Power (GEAP) 4.3 Hyper-X (X-43A) 3.1 Technology Integration and Assessment (TIA) 4.1 Revolutionary Aircraft Concepts Research (RACR) 5.1 Flight Research Productivity (FRP) 6.1 Propulsion Systems Integration and Assessment Revolutionary Aeropropulsion Concepts (RAC) ,,. , 1 ~ ' 1 ' ~ 1 ~ 1 ~ 1 ~ 1 1 .2 2.2 Aerospace Community Systems Concept Noise Impact to Test Reduction (ASCoT) (ON I R) [! ~ t! , ~~ , I! , ~ . ~ , 1 1 1 . ' 1 1 - 1 ' 1 1 ' 1 5.2 Advanced Systems Concepts (ASC) 5.3 Integrated Transport and Testbed Experiment (ITTE) 5.4 Western Aeronautical Test Range (WATR) 5.5 Environmental Research Aircraft and Sensor Technology (ERAST) 6.2 Emissions Reduction 6.3 Highly Loaded Turbomachinery (HLT) 6.4 Materials and Structures for High Performance (MSHP) 7.2 Propulsion Fundamentals Research (PFR) 7.3 Aeropropulsion and Power University Research and Engineering Technology Institute (URETI) 1 7.4 Smart Efficient Components (SEC) 1 ' , ' 11 , I 6.5 Propulsion- Airframe Integration (PAI) 6.6 Integrated Component Technology (ICT) 1 1 _ I ' I: 6.7 Intelligent Propulsion Controls (IPC) 7.5 Oil-Free Turbine Engine Technology (OFTET) 1 7.6 Higher Operating Temperature Propulsion Components 1 1 7.7 Ultra-Safe Propulsion (USP) 1 11 1 1 7.8 Pulse Detonation Engine Technology (PDET) FIGURE 2-1 Vehicle Systems Program organization chart showing VSP projects and subprojects as of March 2003. At the completion of the study, major reorga~zations by product family (commercial, unmanned air vehicles, etc.) were under way but had not been finalized.

ASSESSMENT OF THE VEHICLE SYSTEMS PROGRAM , .1, . , ~ based experiments, the aerodynamic, struc- tural, and electric power technologies that will reduce by 20 percent the fuel burn and carbon dioxide emissions from future sub- sonic transport aircraft. Advanced Vehicle Concepts. Develops ad- vanced vehicle concepts and configurations to reduce travel time, expand commerce, and open new markets. Flight Research. Focuses primarily on testing and validating, in a realistic flight environment, technologies and tools developed by NASA. Ultra-Efficient Engine Technology. Focuses on identifying, developing, and validating high- payoff turbine engine technologies to reduce . . emlsslons. Propulsion and Power. Researches revolution- ary turbine engine technologies, propulsion concepts, and fundamental propulsion and power technologies to decrease emissions and increase mobility. Review Process The panel on the Vehicle Systems Program con- ducted a series of reviews over a 3-month period to assess the quality and relevancy of the research and technology development efforts being conducted across NASA's VSP. The panel surveyed 172 tasks organized within the seven projects and 36 subprojects that made up the VSP at the start of this review. As NASA already had efforts under way to reorganize the VSP before the start of this review, the program and supporting task structure in place at that time was used as the baseline for all evaluations in this report. To help focus the VSP panel's review, the broad guidelines contained in the statement of task in Appendix B were reformulated into a set of concise questions: Is NASA conducting research and development in appropriate areas that are clearly aligned with its vision and mission? Are there projects that should be discontinued because they have completed their work or are not performing well? Is the mix of research about right? Is the balance of near-term and far-term tech- nology development tasks about right? Does the program have a balanced portfolio of near-term and far-term projects, along with fundamental and more mature research and development? Does NASA have a good research plan that sets forth specific goals, identifies the right people/ skill sets, and specifies an appropriate level of funding to achieve the goals as outlined? Is the work done poorly or well? Is it world-class? Is the research making good progress? - Is NASA successfully transitioning the tech- nologies being developed to the user commu- nity and the technical community at large? Prior to the first meeting of the VSP panel, the N1<C asked each principal investigator at NASA to complete a short questionnaire with 12 questions about the re- search and development goals, products, roadblocks, user connectivity, and technical outcomes of their work. A blank questionnaire is shown in Appendix D. The completed questionnaires were distributed to the VSP panel. Reviewing the questionnaires allowed panel members to become familiar with the program and individual projects. The questionnaire proved to be a valuable tool for the panel's use in assessing the pro- gram. It allowed each panel member to assign an initial ranking of the perceived quality of each project and task. At the first meeting of the VSP panel on March 17- 19, 2003, in Washington, D.C., VSP managers gave technical briefings on the overall program to the 16- member panel. Technical briefings were also given for the specific projects and the individual tasks in the VSP. The panel updated the initial ratings of all projects and tasks at the meeting using a weighted evaluation matrix. They then agreed on the assessments. The re- view process is shown in Figure 2-2. After the first meeting, panel members and a con- sultant participated in site visits to each of the relevant NASA facilities (Langley Research Center, Glenn Re- search Center, and Dryden Flight Research Center). The purpose of these visits was to clarify the goals, methods, and performance of specific projects and tasks; to speak directly with the principal investigators of some of the projects; and to inspect the products, facilities, and operations in the program firsthand. Because of the large number of VSP tasks, the panel prioritized how to spend its time investigating the different projects. The pane] used its initial assess- ment of the responses to the task questionnaires and the briefings received at the March 2003 meeting as its start-

12 AN ASSESSMENT OF NASA 'S AERONA UTICS TECHNOLOGY PROGRAMS Preliminary assessment of tasks using questionnaires 1 ! 1 First Panel Meeting Update (validate) assessment of tasks Assess program, projects, and subprojects Document preliminary recommendations and findings Concurrent data-gathering activities '---l Task info follow-up r ~ Site visits All information gathering complete Second Panel Meeting Finalize findings and recommendations Generate report Deliver report to main committee 1 FIGURE 2-2 VSP panel review process. ing point. Using that information, combined with their own expertise, panel members divided the tasks into two groups: those that were adequately understood and for which additional information was not necessary, and those about which the pane] was concerned or for which the panel did not have enough information. For the latter category, the pane} determined what additional informa- tion it needed (site visits, answers to written questions, or some other communication with NASA) and pro- ceeded accordingly. A detailed list of all the panel site visits can be found in Appendix C. As a necessary con- sequence of this approach, some tasks received more attention from the panel and committee than others, and some tasks have a more cletailed assessment of their strengths and weaknesses than others. At the conclusion of the site surveys, the VSP panel was reconvened in Los Angeles on May 27-29, 2003. The goal of the meeting was to finalize the panel's ear- lier findings and generate a set of consolidated recom- mendations. Using the criteria established by the panel and the weighted evaluation matrix, the 172 tasks in the VSP were placed in four categories:

ASSESSMENT OF THE VEHICLE SYSTEMS PROGRAM World-class. Outstanding work that is on a par with the best work anywhere in the world. Good. Solid, meaningful tasks that should be continued but that have some opportunity for improvement. Marginal. Solid, meaningful tasks that have substantial room for improvement. Poor. Tasks that have systemic issues requiring major reevaluation or restructuring, or even cancellation. KEY FINDINGS AND RECOMMENDATIONS The committee generated numerous findings arid recommendations for the VSP, which can be found throughout this chapter. It then identified three key ar- eas of concern and made a number of general observa- tions about the VSP. Key Issue 1: Core Competencies The competencies developed by NASA during the 1960s, 1970s, and 1980s enabled the U.S. aerospace in- dustry to take a dominant position in both the military and commercial marketplaces worldwide. NASA has not reduced the scope of those core competencies or research focus areas even in the face of changing market needs and reduced budgets for vehicle systems throughout the l990s and early 2000s. Rather, NASA has left the same broad set of capabilities in place, with each portion of VSP research forced to operate on ever smaller budgets. As a consequence, some (not all) of the current VSP projects and tasks find themselves on budgetary "life support." These projects are unable to produce technolo- gies that transition to, and significantly impact, the aero- nautics marketplace. In other areas of the VSP, industry state of the art has overtaken NASA capabilities, which raises the question of whether NASA should continue to pursue those competencies. NASA's Office of Aerospace Technology and the VSP have a top-level aeronautics vision that was being finalized at the time of this review. The committee has confidence that this vision will be realized and yield positive results, because the program has already dem- onstrated that it has clearly defined product areas. How- ever, the committee is concerned that NASA has not defined the core competency areas that it will need to support those product areas. While NASA's core com- petencies were clearly defined in the 1960s, 1970s, and 1980s, NASA no longer has a clear set of core compe- 13 tencies and technologies. VSP should create a rank-or-- dered set of core competency areas to help guide invest- ment decisions. It will then be able to leverage those core competencies to ensure that proposed projects cul- tivate new opportunities rather than just competing with what is already being pursued by others. It will also be able to ensure that, for the highest-ranked priorities, NASA is recognized as a world leader and has the po- tential to revolutionize aviation in these areas. The committee assumes that the technologies cho- sen as core competencies will have a higher risk of unsuccessful completion (high risk/high payoff) than the technologies industry would be willing to accept. This philosophy is in alignment with NASA' s vision of "doing what only NASA can." The future VSP invest- ment portfolio should also take into account and look to rectify the problem that over the past two decades industry has reduced its investment in basic research, which serves as the seed corn for future technology opportunities. Finding: Core Competencies. NASA and the Vehicle Systems Program have a clear mission statement with a set of fully linked goals and products; how- ever, NASA lacks a good understanding of the core competencies (in order of importance) required to meet these goals. Finding: Investment Strategy for the Vehicle Sys- tems Program. The VSP appears to have an ad hoc investment strategy, with too many unprioritized projects and tasks and no apparent methodology to determine which research areas will provide the greatest benefit to the U.S. gross domestic product and do the most public good., satisfying the needs of industry, the user marketplace, and other govern- ment agencies. This situation is compounded by ever-decreasing budgets. Program Recommendation: Investment Strategy for the Vehicle Systems Program. The VSP should identify and prioritize technologies (core compe- tency areas) that have the greatest potential to revo- lutionize the future of aviation and impact the gross domestic product of the United States. Key Issue 2: Full-Cost Accounting Before FY03, NASA's program budgets reflected only the cost of the actual hands-on development of the

: 14 particular technology. All civil servant salaries and in- frastructure costs were allocated separately. This ac- counting practice allowed researchers at NASA to have access to fairly expensive test facilities, which their small research budgets would not have been able to support. The advantage of this system is that it allows individuals to innovate without having to justify the need for large capital investments. The drawback of this system, however, is that the real costs of research are not always apparent, and there is the potential for financial waste. In FY03 NASA introduced full-cost accounting, which requires each budget line and task to account for all civil servant salaries as well as the infrastructure that it uses. The advantage of this system is that it will give NASA improved insight into the cost and utiliza- tion of its facilities and infrastructure and make the true cost of research readily apparent. The committee's concern, based on the past expe- rience of some members in transitioning to full-cost budgets, is that researchers may no longer take tech- nologies to large- or full-scale testing. Researchers faced with using available dollars to pay for both hu- man capital and costly full-scale testing may elect to significantly reduce their level of concept validation testing. Although this testing is expensive, it has his- torically been the benchmark by which industry and the user community determine if technologies are ma- ture enough to transition to a marketplace, public or private. If concept validation testing is reduced, NASA could be faced with little justification for certain test facilities and infrastructure that are critical national as- sets. The committee encourages the VSP to learn from industry experience when moving to full-cost account- ing. It is vital for NASA to avoid the unintended atro- phy of NASA's validation and verification test capa- bilities, because without sufficient final testing, transitioning research and development to practice is nearly impossible. NASA may need to have an overhead charge ap- plied to all tasks to cover the core costs for certain test facilities. A core cost overhead budget should be used to retain and maintain a test facility or asset when it is not in use. These budgets should include labor associ- ated with basic maintenance of a facility or test asset. Labor and operating costs above the core maintenance level should be charged directly to the project or task that requests the test service. AN ASSESSMENT OF NASA 'S AERONAUTICS TECHNOLOGY PROGRAMS Finding: Full-Cost Accounting. NASA's transition to full-cost accounting will present challenges to preserving the ability to conduct final, full-scale validation and verification tests. Program Recommendation: Full-Cost Accounting. The Vehicle Systems Program should create an overhead charge to cover the core cost of test facili- ties and assets. Core costs are the costs of retaining a test facility or asset when not in use, including the cost of labor for basic maintenance. Key Issue 3: External Advisory Groups The committee noted that the VSP has various ap- proaches to the staffing and use of advisory groups. In some cases the committee found these advisory groups (as NASA assets) are not as effective as they could be because industry was not involved at the appropriate level namely, chief operating officers. Finding: Advisory Groups. NASA's industry advi- sory panels do not seem to have sufficient participa- tion from top-level industry management to assure buy-in to projects. Program Recommendation: Advisory Groups. The Vehicle Systems Program should reevaluate the composition of its industry advisory panels to en- sure that the appropriate participants are in- volved namely, those who are responsible for turning technologies into marketable products in their respective companies and those who can implement recommended changes. Other General Observations The committee encourages NASA to take a close look at the fixed costs incurred by the VSP, such as the cost of the facilities that NASA now supports. The com- mittee believes that NASA should work to identify those test facilities that are truly unique, while looking for op- portunities to cut costs through consolidation. Such con- solidations might require one-time investments, but over the long term, fixed costs would be reduced. For example, significant resources have been in- vested in computational fluid dynamics (CFD) model- ing to reduce the need for extensive physical modeling ant] wind tunnel testing required in the past and to make

ASSESSMENT OF THE VEHICLE SYSTEMS PROGRAM . i ,, better use of current laboratory experiments. As vali- dated computational tools reduce or even eliminate the need for particular experimental facilities, some of these costly units should be consolidated or deactivated. The committee also found that the sunset provision mandated by the Office of Management and Budget (OMB) forcing all projects to end in 5 years regardless of status or progress of the technology made its assess- ment difficult. Such a provision means that projects must be reorganized periodically so that they appear to be newly formulated, making their history and progress difficult to assess. The committee believes that the sun- set provision is appropriate for some technology projects, but not all. It urges OMB and NASA to devise a new method for ensuring that the nation's funds are spent efficiently. Finding: Office of Management and Budget Sunset Requirements. The OMB sunset provision, which requires all projects to end in 5 years regardless of status or progress, often necessitates reorganization and can damage the continuity of legacy programs. Although such a provision may be appropriate for some projects, many research projects have a time horizon from basic research to mature technology of more than 5 years. Program Recommendation: Sunset Requirements. Managers of the Vehicle Systems Program should actively work to remove the sunset requirement for research programs as necessary. PORTFOLIO The VSP research portfolio ranges from projects and tasks that are pursuing long-term high-risk/high- payoff technologies to near-term initiatives that are closely aligned with industry and that will come to market over the next 5 to 10 years. One of NASA's strengths has always been its ability to work on high- risk concepts with long-term payoffs, which industry often cannot do. The committee found, however, that NASA is not always taking advantage of its ability to do this high-risk work. This may be partly due to the sunset requirements noted above, which cause NASA to focus on 5-year horizons. Finding: Program Balance. The Vehicle Systems Program appears to have become involved in many 15 near-term activities, sometimes at the expense of the revolutionary high-risk/high-payoff activities that are needed to keep NASA's core competencies and leadership role alive. Program Recommendation: Program Balance. The Vehicle Systems Program should increase its pro- portion of revolutionary projects and tasks relative to projects and tasks with near-term results in or- der to keep NASA's core competencies alive and preserve NASA's leadership role in aeronautics re- search and development. The committee also found that VSP is simply con- ducting too many tasks for the amount of funding avail- able. Since it is unlikely in the current fiscal climate that additional funds will become available, the com- mittee believes that NASA should look for ways to re- duce costs by eliminating tasks or projects, as needed, as well as by creatively seeking to leverage money from industry and other government agencies. For instance, a small number of the tasks identified in this report are catching up or competing with industry. These tasks are not providing any skills or technologies that are NASA-unique and are good candidates for cancella- tion. Finding: Portfolio Breadth. The Vehicle Systems Program is pursuing too many tasks for the funds available. Program Recommendation: Portfolio Breadth. The Vehicle Systems Program should try to reduce its overall research portfolio in order to concentrate on projects that make use of capabilities unique to NASA and that strengthen NASA's core compe- tency in aeronautics. NASA requested that the panel identify any criti- cal missing technologies or technology areas that the Office of Aerospace Technology should be pursuing. The committee identified two such areas that fell into this category: (1) technologies for the advancement of rotorcraft and (2) research in flight controls and han- dling qualities. Although there are technology elements applicable to both areas, there is no focused program or project set that advances them. NASA led many of the revolutions in rotorcraft design that we now find in the commercial and military sectors. Unfortunately, how-

16 ever, the NASA plans reviewed by the panel had no focused rotorcraft activities. If the U.S. rotorcraft in- dustry is to remain competitive in the international marketplace, NASA leadership and innovation will be required to respond to the European and Asian prod- ucts now entering the market. NASA's past work in flight controls and handling qualities provided the reference standard for today's system designs. However, as we move toward un- manned systems, the existing standards, which are for manned systems, may be too restrictive. Further evo- lution of the base work done by NASA to include un- manned systems is essential to creating a competitive advantage for U.S. products as this market becomes . . more price-c riven. ~ -:, · ~ AN ASSESSMENT OF NASA 'S AERONAUTICS TECHNOLOGY PROGRAMS Finding: Use of Milestones by the Vehicle Systems Program. NASA does not always use milestones as decision points for continuing a project or task, re- evaluating a project/task plan or test procedures, or canceling a project or task outright. Finding: Flight Controls and Handling Qualities and Rotorcraft. The committee identified two tech- nology areas missing from the Vehicle Systems Pro- gram research portfolio: (1) flight controls and han- dling qualities and (2) rotorcraft research. Program Recommendation: Flight Controls and Handling Qualities and Rotorcraft. NASA should pursue additional efforts in (1) flight controls and handling qualities and (2) rotorcraft. PROGRAM PLAN The committee found that research plans were good, and managers and researchers were making good progress on projects that were appropriately funded. This solid progress was especially true for projects or tasks with a long, clearly defined history- another ar- gument for removing or revising the OMB sunset re- quirement discussed above. The exemplary Hyper-X subproject is discussed below. The committee believes that the VSP would improve overall if other projects were to model their management activities on the Hyper-X. Many of the projects had gateway milestones (mea- sures of technical success). It was not clear to the com- mittee, however, what happened to a task or project when it failed to meet those milestones. NASA seldom used linkage to other tasks (where one technology de- velopment project is critical for another task's comple- tion) or task or project interdependency as a factor in establishing decision gateways for project or task con- tinuation. Program Recommendation: Use of Milestones by the Vehicle Systems Program. VSP should make ef- fective use of milestone gateways for program man- agement decisions and to guide program exit strat- egies and cancellation decisions. Finally, the committee initially had difficulty logi- cally grouping the projects placed under the VSP. The committee believes that this is due to a lack of defined, prioritized core competencies, as discussed earlier. NASA is aware of this problem and appears to be tak- ing appropriate steps to remedy the situation. TECHNICAL PERFORMANCE The committee found that 51 of the tasks reviewed were world-class, 91 were good, and 6 were marginal. Finally, 24 of the 172 tasks were found to be poor. Table 2-2 summarizes all of the tasks that were viewed as world-class. These tasks were well aligned with the visions and goals of NASA and VSP, well organized and managed, and performing cuttin`~-edge research. Tasks categorized as good are not discussed in detail in this report as they are not in urgent need of attention. Twenty-four tasks were at the other end of the per- formance spectrum. These tasks were in need of either major restructuring or realignment or they were candi- dates for cancellation. Table 2-3 lists the six tasks that are marginal and need improvement. Table 2-4 lists tasks that are recommended for reevaluation to deter- m~ne if they should be restructured or canceled. Table 2-5 identifies tasks that the committee believes should be canceled. During the consensus meeting in Los Angeles, the VSP pane! developed observations that cut across dif- ferent projects and tasks within VSP. The panel reached consensus on the findings and recommendations and submitted them to the full review committee for its consideration. The great majority of VSP tasks were either excep- tional or good. The committee found that overall the VSP employs an extremely qualified and capable staff

ASSESSMENT OF THE VEHICLE SYSTEMS PROGRAM TABLE 2-2 Fifty-one VSP Tasks That Are World-Class 17 Task No. Task Name Task No. Task Name 1.1.1 Micro-Adaptive Control 6.2.17 Lean Direct-Injection, Low-NOx Combustor 1.1.3 Adaptive Structural Morphing Concepts 1.2.1 Physics-Based Flow Modeling 6.3.6 Average Passage Modeling 1.2.2 Fast, Adaptive Aerospace Tools 6.3.7 Dual Spool Turbine Facility 1.2.5 Computational Aeroelasticity, Modeling, and 6.4.1a Materials and Structures Turbine Airfoil System/ Scaling Low Conductivity 1.3.1 Biomimetics/Nanotechnology 6.4.1b Materials and Structures Turbine Airfoil Systeml 1.4.3 Tire Mechanics/Dynamics Advanced Airfoil Alloy Development 1.4.4 NASAJDoD Collaborative Activities 6.4.3b Computational Materials Science—Ceramic 1.6.2 Robust Avionic Architectures 6.5.1 Active Flow Control 1.6.3 Control of Complex Air Vehicles 6.7.1 Rotating Machinery Clearance Management 1.6.6 Ageless Structural Systems Technology 7.1.2 Hot/Smart Materials for Aeropropulsion 2.2.1 Impact Modeling 7.1.3 Morphing Structures for Self-adaptive 2.2.3 Low Noise Flight Procedures Aeropropulsion 2.3.5 Engine Systems and Advanced Concepts 7.1.5 Miniature Autonomous Sensors and Actuators for 3.2.1 High-Speed Slotted Wing Smart Propulsion Systems 3.2.3 Ground-to-Flight Scaling 7.1.7 High Power Motor Control Inverter for 3.3.1 Tailored Structures Aeropropulsion 4.1.4 Active Vibration Suppression 7.1.13 Interstage Turbine Burner 4.2.1 Intelligent Flight Control System: C-17 7.2.2 Nanotechnology 4.2.2 Intelligent Flight Control System: NF-15 7.4.1 Aspirating Flow Control 4.3.1 Flight 2/Return to Flight 7.4.2 Compressor Flow Control 5.1.1 Flight Research Productivity 7.4.3 Intelligent Flutter Control 5.2.1 Active Aeroelastic Wing 7.4.5 Combustor Technologies 5.2.2 Autonomous Aerial Refueling 7.4.9 Active Combustion Control 5.5.1 Hellos 7.5.1 Foil Bearing Development/Testing/Analysis 6.2.14 Benchmark Test with Liquid Spray Injector 7.6.3 Metallics 6.2.15 Combustor Code 7.6.4 Instrumentation 6.2.16 Large Eddy Simulation of a Gas-Turbine Model 7.7.2 Crack-Resistant Materials Combustor TABLE 2-3 VSP Marginal Tasks That Need Improvement Task No. Task Name Task No. Task Name 1.1.4 Biologically Inspired Flight and Control Systems 2.3.1 Fan Noise Reduction 2.1.2 Propulsion Airframe Aeroacoustics 2.3.4 Liner Technologies 2.1.3 Passenger/Crew Environment 3.4.3 Configuration and Performance Evaluation TABLE 2-4 VSP Tasks That Should Be Reevaluated for Restructuring or Cancellation Task No. Task Name Task No. Task Name 1.5.1 Aviation Assessments 6.4.3a Computational Materials Science Metallic 3.1.1 Technology Benefits Assessments 6.4.4a 3000°F Ceramic Matrix Composite System 3.3.2 Tailored Materials/Processing Technology 6.4.5 Ultra-High-Temperature Ceramics 4.1.5 Vehicle Concept Teams 7.3.3 Intelligent Engine Systems 6.3.1 Fan Trailing Edge Ejection

18 TABLE 2-5 VSP Tasks Recommended for Cancellation AN ASSESSMENT OF NASA 'S AERONA UTICS TECHNOLOGY PROGRAMS Task No. Task Name Task No. Task Name .3.2 1.3.3 3.4.1 3.4.2 6.6.4 7.8.1 7.8.2 and that the program has more than adequate infrastruc- ture to support the initiatives being pursued. USER CONNECTIONS The committee believes it is essential to have strong connectivity to the user community and the technical community at large to ensure that the technology being developed by NASA is being used for the public good. The committee found that many projects very success- fully leveraged industry participation, small business innovation research awards, and academic research to achieve many objectives. However, at the user buy-in level, the committee did not see evidence of top-level industry connections. The committee emphasizes that there is a difference between industry advice and indus- try buy-in. It would like NASA to review not only the composition of its industry advisory committees, but also the positions the advisory committee members hold within their respective companies. Although the com- m~ttee understands that industry advisory committees depend on voluntary participation, NASA should seek to reconfigure projects if necessary to ensure participa- tion from the appropriate top-level industry people who can take action within their companies, including cost sharing and commitment to the process. The committee found this to be a critical issue in the VSP (see Key Issue 3: External Advisory Groups earlier in this chapter). ASSESSMENT BY PROJECT Breakthrough Vehicle Technologies Project (~.O) Background The goal of the Breakthrough Vehicle Technolo- gies (BVT) project is to enable a more efficient and Revolutionary Metallic Materials and Structures Lightweight Multifunctional Structures Hydrocarbon Fuels Processing and Fuel Characterization Power Management and Distribution Testbed Mechanical Components Cycle Analysis Materials and Structures 7.8.3 7.8.4 7.8.s 7.8.6 7.8.7 7.8.8 7.8.9 Instrumentation and Control Combustion/Pulse Detonation Engine Testbed Inlets Nozzles Combined Cycles/Ejectors Hybrids Acoustics environmentally friendly air transportation system. The project plan is to achieve this goal through the discov- ery and creation of technological breakthroughs. It is divided into five subprojects with specific technolo- gies (morphing, lightweight technologies) and' high- level concepts (systems analysis, systems testing, and cooperative efforts). This is a high-risk, high-payoff, exploratory endeavor designed to create "disruptive" technologies that will dramatically and substantially improve vehicle performance. This effort was funded at $41.4 million in FY03 and is budgeted at $60.2 m~l- lion in FY04, under the full-cost accounting scheme. - Portfolio The portfolio of the Breakthrough Vehicle Tech- nologies project (1.0) represents a good mix of tech- nologies programs that address both near- and far-term needs. Activities of two subprojects the nanotech- nology work in Super Lightweight Multifunctional Systems Technology (SLMFST) (1.3) and the robust avionics work in Robust Aerospace Systems (1.6)- are developing revolutionary technologies and have the potential to significantly impact future aerospace prod- ucts. The tools work being done in the Computational Aeroelasticity, Modeling, and Scaling task (1.2.5) and the Robust Aerospace Systems subproject (1.6) are linked to the successful development of many of these revolutionary technologies. The Advances Through Cooperative Efforts sub- project (1.4) is effectively developing near-term prod- ucts such as runway friction parameters and tire me- chanics, while leveraging the unique NASA facilities and skills to support key DoD program initiatives. The committee had difficulty understanding the rationale for the logical grouping of these efforts under the Breakthrough Vehicle Technologies project head-

:' ASSESSMENT OF THE VEHICLE SYSTEMS PROGRAM ing. In some areas, the committee had difficulty find- ing program decision linkages among subprojects 1.1 through 1.6. For example, the Control of Complex Air Vehicles task (1.6.3) has a lengthy development pro- gram that assumes there will be no hardware execution obstacles in implementing the control concepts. In- stead, there should be some cross-task interdependency so that the control theories can be initially validated when the hardware is being validated. The final devel- opment of the control algorithms should only occur after both the control theories and the hardware have been validated. Several elements of the portfolio are poorly linked to NASA goals and objectives and warrant reexamina- tion. These include the Aerospace Systems Analysis subproject (1.5) and two of the three tasks under SLMFST (1.3~. Also, the Biologically Inspired Flight and Controls task (1.1.4) is considered marginal. De- tails of these items are provided below. Program Plan The Breakthrough Vehicle Technologies project, like many research projects, is dependent on the suc- cessful demonstration of core technologies under de- velopment. The technology demonstrations often de- pend on research outcomes from other projects. For this reason, the Breakthrough Vehicle Technologies project gateways (technical goals that indicate success) and off-ramps (the transition of successful tasks or the cancellation of unsuccessful tasks) should include de- cision points from related projects, as discussed previ- ously in this chapter. Such improved integration of gateways and off-ramps will strengthen the project. Technica/ Performance The committee notes that excellent work is being done in many locations within this project, particularly in the work on intelligent controls ant! methods. This work is just what NASA should be doing and does well. The project staff recently demonstrated high-quality work in the Abrupt Wing Stall Research task (1.2.4), in which NASA successfully resolved the F/A-18E/F abrupt wing stall problem. User Connections There is good collaboration with government, in- dustry, and academia across the Breakthrough Vehicle 19 Technologies project portfolio. This collaboration is evidenced in the NASA/DoD Cooperative Programs task (1.4.4), where the name of the task shows that NASA's work is closely tied to DoD's work. This task leverages NASA resources to service DoD and indus- try needs. While the committee commends NASA for its co- operative efforts with the DoD, it cautions NASA not to use its expectation of future work with the DoD to determine the number of facilities to be retained or how often those facilities will be used. Specifically, the committee urges NASA to maintain only those facili- ties that are needed to meet NASA-specific needs at NASA Langley. Assessment by Subyroject Although the overall portfolio of the Breakthrough Vehicle Technologies project was strong, some refo- cusing of subprojects and tasks would strengthen the project. Specifically, the committee identified tasks within the Morphing (1.1), SLMFST (1.3), and Aero- space Systems Analysis (1.5) subprojects that should be reexamined in order to strengthen the overall project. The committee believes the Aerospace Systems Analysis subproject (1.5) is an essential tool for select- ing, evaluating, and tracking the value of technologies in the research portfolio. However, the committee be- lieves that the efficiency of this initiative can be im- proved by reexamining staffing and cost. Even though this effort is essential, NASA should keep the operat- ing costs to a minimum since the effort yields no tech- nology product. The committee offers the following comments on specific subprojects and tasks within the Breakthrough Vehicle Technologies project for NASA's consider- ation. Morphing Subproject (1.1) Micro-Adaptive Control Task (1.1. I J This technology-oriented task has developed strong collaborations with a diverse range of academia and industry players. It has shown some gains that can be leveraged in the Twenty-first Century Aircraft Tech- nology project. The work is strongly linked with that of the Smart Technologies task (1.1.2), where NASA first tests a concept and then transitions it to flight testing.

20 mittee. Smart Technologies Task (1.1.2) This task was assessed to be adequate by the com- Adaptive Structural Morphing Task (1.1.3) Industry is well connected to the Adaptive Struc- tural Morphing task (1.1.3), and the committee ap- plauds such collaboration. However, the committee is also concerned that the technologies being pursued will not easily scale to a full-sized aircraft. It encourages NASA to make certain that the technologies it is pursu- ing will be widely applicable to a range of aircraft. . . . AN ASSESSMENT OF NASA 'S AERONAUTICS TECHNOLOGY PROGRAMS others. Work currently being conducted or planned in this area, which includes low-cost autonomy (hardware and software) and flexible structure/wing warping tech- nology, does offer potential advantages to larger classes of vehicles. Biologically Inspired Flight and Control Systems Task '(1.1.4J This task comprises a host of diverse programs, some of which are doing groundbreaking work and oth- ers of which, while interesting, are not aligned with NASA's objectives of bringing unique capabilities to the aeronautics field. The committee notes that there is some very positive work being conducted. For instance, the piezoelectric pumped jets for control subtask is an example of good, cutting-edge wing technology that has the potential to significantly influence how future low-observable and non-low-observable vehicles are designed. The hyperbolic wing designs inspired by nature clearly offer advantages over planar elliptical wings, but the basis of comparison should not be planar ellip- tical wings but, rather, optimized wingless. The com- m~ttee is concerned that although this work will yield results comparable to wingless, manufacturing a hy- perbolic wing poses significant cost and design issues that may discourage its production. Unless it can be proven that hyperbolic wing designs are significantly better than optimized wingless, the committee recom- mends that this work be terminated. The micro air vehicles and control algorithms subtask is very interesting. However, the committee believes that this niche market segment is not one that NASA should try to influence. The commercial arena is already competing for this market. This activity seems poorly aligned with NASA's top-level goals of impacting large markets and ultimately benefiting the gross domestic product of the United States. Further- more, the committee could not identify anything unique that NASA is contributing that could not be done by Finding: Wing Design. The Vehicle System Pro- gram uses planar elliptical wings as the baseline against which to evaluate the advantages of hyper- bolic wing designs inspired by nature. However, a more valid comparison would be with optimized wingless. Recommendation: Wing Design. The hyperbolic wing design work in the Biologically Inspired Flight and Control Systems task (1.1.4) should demon- strate a major improvement over optimized wingless to justify continuation. Finding: Micro Air Vehicles. The Vehicle System Program's work on micro air vehicles and control algorithms is quite interesting but seems to cater to a niche market already captured by competitive market forces. Recommendation: Micro Air Vehicles. For the mi- cro air vehicle subtask to have a broader impact it should focus on low-cost autonomy (hardware and software) and flexible structure/wing warping tech- nology; otherwise, the effort should be discontinued. Also, within the Biologically Inspired Flight and Control Systems task (1.1.4), NASA has also dem- onstrated that fuel savings can be realized' by flying in the wake of an adjacent aircraft in formation fly- ing. Additional research activities proposed to opti- mize this approach and further develop the control algorithms that would be used by formation aircraft appear premature, however. NASA must discuss this research with the military services, the FAA, and/or other government agencies to determine a concept of operations. This concept of operations should rely on formation flown aircraft and must be accepted by a user community (or, at a minimum, be within the realm of acceptability). Without advance involve- ment of a potential user community, NASA will have difficulty transitioning the formation flying work to the user community. The committee there- fore recommends that activities associated with for- mation flying focus on gaining user acceptance.

ASSESSMENT OF THE VEHICLE SYSTEMS PROGRAM Finding: Formation Flying. At present, NASA's work in formation flying and associated control al- gorithms does not adequately involve the military services or the FAA. Recommendation: Formation Flying. The research on formation flying under the Biologically Inspired Flight and Control Systems task (1.1.4) should con- tinue only if the FAA, the military services, or some other government agency outside NASA agrees to a concept of operations that calls for formation flying. Aerospace Systems Concept to Test Subproject (1.2) This Subproject (tasks 1.2.1 to 1.2.6) as a whole has excellent integration of advanced concepts into practical applications. The NASA staff are working well with a diverse range of industry, academic, and government staff. The committee encourages the re- searchers to expand their scope by including a central effort to integrate CFD experiments and the work by industry into their research. The European researchers have been successful in integrating CFD experiments into their designs, and the committee strongly believes that NASA could become the agency that coordinates CFD integration. Super Lightweight Multifunctional Systems Technologies Subproject (1.3) Biomimetics and Nanotechnology Task (1.3.1J NASA is conducting world-class nanotechnology research under this task and the technology being pro- duced, if successful, could be a significant enabler of future technological progress. However, because a number of other agencies have made large investments in nanotechnology initiatives, the committee is con- cerned about the potential impact of NASA's efforts in the Biomimetics and Nanotechnology task. Revolutionary Metallic Materials and Structures Task (1.3.2) and Lightweight Multifunctional Structures Task (1.3.3) The committee is concerned that the friction stir- ring work conducted under the Lightweight Multifunc- tional Structures task (1.3.3) is simply repeating what is already being done by industry. The committee does 21 not believe that this is an area to which NASA brings unique value. Additionally, the parts being developed in the electron beam initiative under the Revolutionary Metallic Materials and Structures task (1.3.2) may yield only very small weight reductions that do not justify the effort. Finding: Task Relevance. The committee is con- cerned that friction stirring work in the Lightweight Multifunctional Structures task (1.3.3) is already being done by industry, while the electron beam welding work in the Revolutionary Metallic Mate- rials and Structures task (1.3.2) produces parts not leading to recognizable weight reductions. Thus, these tasks do not have clear relevance to the pro- gram-level goals of enhancing aircraft performance. Recommendation: Task Relevance. Due to the ad- vanced state of industry practice in the same areas, the committee recommends that NASA discontinue the Revolutionary Metallic Materials and Struc- tures task (1.3.2) and the Lightweight Multifunc- tional Structures task (1.3.3~. Advances Through Cooperative Efforts Subproject (1.4) This Subproject was assessed to be adequate by the committee. Aerospace Systems Analysis Subproject (1.5) Aviation Assessments Task (1.5.1) This task provides overall analyses activity within the program mainly as a service for project managers, allowing them to assess technical progress and the rel- evance of various programs to the objectives of NASA's Aeronautics Technology Theme—Protect the Environment, Increase Mobility, Explore New Aero- space Missions, and Support National Security. The team has a large staff (28 technical members), who should be able to perform the complex system analyses using a variety of specialized tool sets. How- ever, the committee found that the analysts were not using the best planning and analysis tools currently on the market and some of their results were difficult to interpret. Specifically, the team should help the end users interpret and understand the results, not simply present results and assume the end users will under- stand them.

22 ,' . . AN ASSESSMENT OF NASA 'S AERONA UTICS TECHNOLOGY PROGRAMS Also, specific metrics are used and applied via plat- form models in numerous categories to assess the vi- ability of a given technology development program. The technique tends to be very accurate in the higher technology readiness levels (TRLs) but falls off as smaller, lower TRL tasks are introduced (see Figure 2-3~. Given the number of lower TRL programs in VSP, the need for this technique is still very great as it provides a tool to prioritize programs in a resource- limited environment. The committee understands that coordination with the project and subproject management teams is re- quired to perform this task. However, the team should report directly to the Vehicle Systems Program man- ager without oversight from the project managers, as this could pose a conflict of interest. Reporting directly to the program manager will assure proper reporting of progress. Finding: Assessment Reporting Structure. The Aviation Assessments task (1.5.1) team currently reports to a project manager, which could mean a potential conflict of interest. Recommendation: Assessment Reporting Struc- ture. In order to improve efficiency and effective- ness in the Aviation Assessments task (1.5.1), the team should be set up as an independent, project- level entity (or its equivalent) so that it reports di- rectly to the Vehicle Systems Program manager. In addition, the team needs to be in close touch with actual platforms through the airframe community in or- der to assure that metrics and benefits assessments are realistic. The committee noted that there was a lack of Coordination between team members and their aviation peers in spite of the efforts of the Aerospace Technology Advisory Committee (ATAC). This situation could be resolved by using actual platforms in some instances and working with industry benefits analyses teams. Industry acceptance of the goals, objectives, technical challenges, and approaches roadmaps—its buy-in is needed to succeed in technology transition. Robust Aerospace Systems Subproject (1.6) The tasks in this subproject were assessed to be adequate by the committee. Quiet Aircraft Technology Pro lest (2.0) Background The Quiet Aircraft Technology (QAT) project seeks to discover, develop, and verify, in the labora- tory, technologies that improve quality of life by re- ducing society's exposure to aircraft noise. The adverse environmental by-products of aviation- primarily noise and emissions—are major constraints on the growth of the aviation industry. Public concerns over the environmental effects of aircraft and airport opera- tions, as well as the increasingly stricter requirements embodied in laws and regulations, can severely con- strain the ability of the aviation system to meet the world's future needs for mobility, increased trade/mar- ket access, and sustained economic growth. In 1995, the National Science Technology Council said that "en- vironmental issues are likely to impose the fundamen- tal limitation on air transportation growth in the 21st century.") Portfo/io At the project level, the committee found the QAT portfolio to be well balanced. The portfolio addresses major concerns, with the notable exception of sonic booms. The QAT project comprises three subprojects: Airframe System Noise Reduction (2.13; Community Noise Impact Reduction (2.2~; and Engine System Noise Reduction (2.3~. The distribution of funding be- tween the engine and airframe subprojects appears commensurate with their relative contribution to noise. The committee commends NASA for also addressing community noise impact through operational proce- dures, as endorsed by the International Civil Aviation Organization. Successfully mitigating the impacts of aircraft noise is dependent on a balanced approach that considers advanced technology, operational proce- dures, controls, and land use practices. The committee found a gap in the portfolio in the area of sonic boom mitigation. The Defense Advanced Research Project Agency (DARPA) recently conducted a very success- ful test of technology for reducing sonic boom. NASA iNational Science Technology Council. 1995. Goals for a Na- tional Partnership in Aeronautics Research and Technology. Wash- ington, D.C.: White House Office of Science and Technology Policy. Available online at <htip://www.ostp.gov/htmllaero/cv- ind.html>.

ASSESSMENT OF THE VEHICLE SYSTEMS PROGRAM ;, 'I' ! ~ , ~ 23 9. Operational flight-proven. 8. Actual system completed and flight qualified. 7. System prototype demonstrated in flight. 6. System/subsystem (configuration) model or prototype demonstration/validation in a relevant environment. 5. Component (or breadboard) verification in a relevant environment. 4. Component and/or breadboard test in a relevant environment. 3. Analytical and experimental critical function, or characteristic proof-of-concept, or completed design. 2. Technology concept and/or application formulated (candidate selected). 1. Basic principles observed and reported. FIGURE 2-3 NASA technology readiness levels. The TRIM of a NASA research product is specified according to the above criteria. SOURCE: NASA (2000~. . Is mentioned as a supporter of this program, but the committee saw no research activity in this area during its review. The committee believes NASA should con- sider following the DARPA success with a project in this area. Noise concerns include takeoff and landing noise, taxi and engine run-up noise, noise from flyovers at cruise altitude over areas of low ambient noise, and sonic booms associated with supersonic flight. The committee believes that noise source abatement tech- nologies are an important area for NASA investment. This is consistent with a study by the National Acafd- emies Transportation Research Board, which noted that the great advances made in the 20th century to reduce the impact of aircraft noise were due primarily to ad- vanced technology. Finding: Noise Source Abatement. Developing noise source abatement technology is a critical area for the air transport system and is consistent with NASA's mission. Recommendation: Noise Source Abatement. NASA must continue to identify tasks and conduct

24 research to advance technology for noise source abatement. The committee specifically commends the Com- munity Noise Impact Reduction subproject (2.2~. It commends the excellent linkage between modeling and full-scale data acquisition/flight procedure verification programs, such as the Low-Noise Flight Procedures task (2.2.31. The committee also commends the excel- lent use of teaming, including a relevant airport (Lou- isville), a manufacturer (Boeing), academia (Massa- chusetts Institute of Technology), NASA (both Langley and Ames), an operator (United Parcel Service), and both controllers and policy makers at the FAA. In this case, NASA took the technology to a TRL 6 by doing a real-life demonstration. Although it is arguably not a model that should be followed by all NASA programs, it is very appropriate for some technologies that re- quire a full community demonstration. The committee did have some concerns about the portfolio content of the other subprojects. For example, the committee questioned NASA research in cabin noise abatement given manufacturers' ongoing invest- ments in this area and the limited resources NASA has to pursue the very ambitious goals of the task. AN ASSESSMENT OF NASA 'S AERONA UTICS TECHNOLOGY PROGRAMS Program Plan At the project level, Quiet Aircraft Technology (2.0) has clearly defined goals. These goals include reducing the perceived noise levels of future aircraft by one half (10 dB) from 1997 subsonic aircraft within 10 years and by three quarters (20 dB) within 25 years. The NASA presenters understood and articulated the user benefits well. The 10-year goal enables containing 65 Day-Night Level (DNL) noise within an airport' s physical bound- ary. The 25-year goal ambitiously seeks to contain noise within airport boundaries at 55 DNL. Manufacturers, operators of the airlines and airports, and the FAA en- dorse these goals, as does this committee. The committee believes that the program plan for the Community Noise Impact Reduction subproject (2.2) is exemplary. The plan has an excellent mix of modeling, simulation, flight validation, and laboratory experimentation. The NASA project team showed ex- cellent qualifications, and the NASA Ames simulation facilities are world-class. NASA also does an excellent job of getting relevant stakeholders to participate on the team. The committee had concerns about the program plans for the Airframe System Noise Reduction (2.1) and Engine System Noise Reduction (2.3) subprojects. Although the committee believes the work is impor- tant, it noted weaknesses in that the subprojects did not always focus on key areas with the highest payoff. The committee recommends that NASA select the highest- priority research through consultation with relevant stakeholders. Before initiating tasks, managers should clearly define goals and the milestones along the way that signal research success, redirection, or failure. For example, the committee believes that NASA should examine the process for measuring success and for cre- ating appropriate technology off-ramps in the Revolu- tionary Aeropropulsion Concepts subproject (7.1 ) and use it as a model. Finding: Focusing on High-Payoff Research. The work in the Airframe Systems Noise Reduction (2.1) and Engine Systems Noise Recluction (2.3) sub- projects is critical to the air transport system. How- ever, NASA has not identified key areas with the highest payoffs in these subprojects and focused re- sources accordingly. Recommendation: Focusing on High-Payoff Re- search. The Vehicle Systems Program should exam- ine the use of goals and milestones in the Revolu- tionary Aeropropulsion Concepts subproject (7.1) and use similar methodology in the Airframe Sys- tems Noise Reduction (2.1) and Engine Systems Noise Reduction (2.3) subprojects to focus resources on key areas that offer the highest payouts. Finding: Assessing System Penalties. In some in- stances, NASA is developing noise abatement tech- nology that has an unacceptable overall system pen- alty (weight, performance). Recommendation: Assessing System Penalties. NASA should use its systems analysis capabilities to assess tasks from a system impact standpoint and not continue to pursue those tasks that are not justi- fied when overall system penalties exceed expected gains. The committee commends NASA for seeking to discern the physics governing aircraft noise. However, the committee is concerned about the transitioning of models to manufacturers. The committee did not see evidence of this transition and believes NASA must

ASSESSMENT OF THE VEHICLE SYSTEMS PROGRAM generalize its models for them to become useful pre- diction tools. Finding: Transitioning to Industry. The committee did not see evidence of Transitioning NASA models of the physics of noise in aircraft to industry. Recommendation: Transitioning tot Industry. NASA should work with industry to identify re- quirements and produce generalized fundamental physics models to make useful noise prediction tools. Finally, the committee commends NASA's plan to leverage QAT funding by seeking greater industry con- tributions in future plans to take selected technologies to TRL 6. However, the committee is concerned with the quality of industry's contribution. In the Advanced Subsonic Technology Noise Reduction Program (the QAT predecessor), industry's contributions were gen- erally in-kind, which is acknowledged to be less satis- factory. NASA should seek to enhance both the amount and quality of the industry contribution. Technica/ Performance The QAT project has shown evidence of good per- formance and achievements. It builds on a strong base from past national noise reduction efforts. The project generally has a solid, successful technical team in place and very good industry, university, and FAA participa- tion. Examples of exemplary recent achievements in- clude the development of chevrons and swept stators along with the identification and verification of con- tinuous descent approach procedures in the Commu- nity Noise Impact Reduction subproject (2.2~. In all cases, the recipe for success included having teams made up of the relevant stakeholders and advancing the technologies to TRL 6. The committee believes that technical challenges get more difficult with each decibel of reduction and believes that future performance and achievements may be hindered by resource limitations. In the past, items taken to TRL 6 had clear payoffs. The committee urges NASA to continue to advance QAT technologies to TRL 6. The committee commends NASA's recent aug- mentation of QAT funding and believes NASA should focus these funds in advancing technologies. Finding: Industry Cost Share. Industry cost share 25 in some Quiet Aircraft Technology tasks was mini- mal, and when industry did provide cost share, the quality of the cost share tended to be low (i.e., it offered in-kind support). In cases where there was good collaboration with relevant stakeholders in a task and the technology was taken to technology readiness level 6, there was significant payoff:. Recommendation: Industry Cost Share. NASA should work with the industry advisory groups to enhance both the amount and quality of the indus- try contribution through direct and innovative cost sharing practices and to assure that promising tech- nologies are carried to the aunrouriate technology readiness level. User Connections ~ ~ ~ rev The QAT project does exhibit good user connec- tivity at the technical level. As already stated, the goals are relevant and endorsed by the community at large. The committee particularly commends QAT on includ- ing airlines and airport personnel on its advisory board and encourages other projects to follow suit. The Com- munity Noise Impact Reduction subproject (2.2) is an excellent example of integrating stakeholders' interests and capabilities to produce positive results. Although the other subprojects benefit from the same group of advisors, the committee believes that more of their tasks should have the involvement of industry. At the policy and user buy-in level, the committee did not see evidence of top-level industry connections. The committee emphasizes that there is a difference between industry advice and industry buy-in. The com- mittee believes that NASA should review not only the composition of its industry advisory committees, but also the positions advisory committee members hold in their respective companies. Although the committee understands that industry advisory committees depend on voluntary participation, NASA should seek to reconfigure projects if necessary to ensure participa- tion from the right people that is, the people who can take action on cost sharing and commit their company to the process. Finding: Outside Stakeholders. Outside stakehold- ers are not adequately involved in all Quiet Aircraft Technology tasks. In particular, NASA does not suf- ficiently use contracts, grants, or SBIR partnerships to leverage outside expertise.

26 Recommendation: Outside Stakeholders. NASA should involve outside stakeholders while maintain- ing in-house expertise. Innovation through the in- volvement of small businesses and universities is encouraged. The committee highlights, as an ex- ample, the Propulsion and Power project (7.0), which innovatively funded some high-risk sub- projects and used technology incubation in its Revo- lutionary Aeropropulsion Concepts Subproject (7.~. , ~ ,; AN ASSESSMENT OF NASA 'S AERONA UTICS TECHNOLOGY PROGRAMS ment task (2.1.3), which concentrates on cabin noise reduction, was excellent, it was unconnected to the overall QAT goal of reducing community noise expo- sure. Furthermore, the committee questioned the ne- cessity of NASA investment in cabin noise abatement efforts given similar work in industry. The committee believes that the Airframe Noise Reduction task (2.1 .1 ) needs to be more tightly coupled with full-scale experiments to provide guidelines for model development. Specifically, in the Propulsion Finding: Prioritizing Quiet Aircraft Technology Research. Funding constraints appear to be limit- ing the program content in QAT, confining the work to low technology readiness levels and thereby in- hibiting technology transition. Recommendation: Prioritizing Quiet Aircraft Tech- nology Research. NASA should work with its advi- sory committee to focus its limited QAT resources. NASA should also carefully consider mitigating the risk of task failure by carrying technologies to a higher technology readiness level before down- selecting to a single technology or contractor. Recommendation: Use of Milestones and Reviews. The committee encourages NASA to put in place milestones and review processes involving senior management or outside advisory groups in the as- sessment of progress and the decision to continue, redirect, or cancel a task. Assessment by Subproject Airframe System Noise Reduction Subproject (2.1) The Airframe System Noise Reduction Subproject has a sound goal, and the industry steering committee appears to represent the stakeholders well. The com- m~ttee also notes that large-scale demonstrations are planned to advance technologies to a higher TRL. To meet the subproject's very ambitious goals, NASA has explored high-r~sk concepts. The committee found that the Subproject lacked focus and was too broad in its scope. This weakness, coupled with limited resources, results in an inability to ensure industry involvement and commitment. The committee felt, for instance, that although the quality of the efforts in the Passenger/Crew Environ- Airframe Aeroacoustics task (2.1.2), the work in pre- dictive tools for installation effects appears to overlap with work under the Airframe Noise Reduction task (2.1.14. NASA should integrate or merge these tasks. Finding: Airframe Noise. The committee found that the Airframe System Noise Reduction Subproject has sound, critical goals. However, some of the sub- projects lack focus and their scope was too broad for the available resources. Recommendation: Airframe Noise. NASA should assess whether the Passenger/Crew Environment task (2.1.3) cabin noise reduction research is consis- tent with overall goals of the Quiet Aircraft Tech- nology project and evaluate the potential payoff of this NASA investment given similar efforts by in- dustry. Recommendation: Landing Gear Noise. The Air- frame Noise Reduction task (2.1.1) needs to be linked to activities with full-scale experiments to provide guidelines for model development. Recommendation: Landing Gear Noise. The predic- tive tools for installation effects in the Propulsion Airframe Aeroacoustics task (2.1.2) appear to over- lap with the Airframe Noise Reduction task (2.1.1~. NASA should explore integrating or merging these tasks. Community Noise Impact Reduction Subproject (2.2) Subproject 2.2 is the strongest one in the Quiet Aircraft Technology project. It demonstrates excellent connectivity to major constituents, which is leading to near-term return on investment. The work with Boeing has produced useful applied models, and the industry steering committee represents all the stakeholders well.

ASSESSMENT OF THE VEHICLE SYSTEMS PROGRAM The committee urges NASA to develop innovative ways of expanding the project, even in the face of fund- ing limitations. Finding: Community Noise. Research by the Com- munity Noise Impact Reduction subproject (2.2) on continuous descent approach noise abatement pro- ceilures was exemplary and offered excellent oppor- tunity for near-term return on investment. Recommendation: Community Noise. NASA should develop innovative ways of expanding continuous de- scent approach research to other geographic areas. Engine System Noise Reduction Subproject (2.3) The goals of this subproject are also sound, and the 9 ft x 15 ft wind tunnel test facility is a significant national asset. The subproject is looking at some inno- vative concepts, such as microelectromechanical sys- tems (MEMS), and the industry steering committee appears to represent stakeholders well. In view of the limited resources available for the subproject, NASA should refocus it by giving it fewer tasks in order to achieve realistic goals. The rescoping of the subproject should be done with industry involve- ment so as to leverage industry funds. While there is evidence of good-quality work in this subproject, the committee encourages NASA to reexamine some of the tasks in light of top-level design concept issues that may be insurmountable. For in- stance, in the Fan Noise Reduction task (2.3.1), the committee believes that any technology developed un- der this task might result in unacceptable system penal- ties owing to weight penalties and the need for high- speed airflow. NASA should assess this and all other technology development tasks by taking into account their impact on overall system performance. For example, in the Liner Technologies task (2.3.4), it is not evident that the potential benefits of integrating piezoelectric devices in liners for jet en- gines outweigh the negatives of their complex opera- tional requirements. Further, the committee did not see evidence that the scaling limitations of such devices have been adequately addressed at the system level. Finding: Engine Noise. The Engine System Noise Reduction subproject has sound, critical goals. However, in some instances the subproject lacks 27 focus and the scope is too broad for the available resources. Recommendation: Engine Noise. NASA should en- gage relevant stakeholders to help focus the project on fewer tasks in order to achieve realistic goals. Finding: Assessing System Penalties. In some in- stances, NASA did not consider whether the poten- tial benefits would outweigh the penalties resulting from various engine noise abatement approaches. Recommendation: Assessing System Penalties. NASA should assess the benefits and penalties of noise abatement technology development for the Fan Noise Reduction (2.3.1) and Liner Technolo- gies (2.3.4) tasks and terminate or refocus them if there is low payoff. Twenly-first Century Aircraft Technology Pro lest (3.0) Background The Twenty-first Century Aircraft Technology (TCAT) project intends to develop and validate, through ground-based experiments, the aerodynamic, structural, and electric power technologies that will re- duce by 20 percent the fuel burn and carbon dioxide emissions from future subsonic transport aircraft. The project is motivated by a desire to reduce the impact of aircraft on global climate change and to reduce smog surrounding airports. The project combines research in aerodynamics, structures, and electric power to achieve its objectives. The effort was funded at $46 million in FY03 and $42 million in Fyo4.2 Polffo/io The Twenty-first Century Aircraft Technology project plans to accomplish its goal through the devel- opment of the following: · Aerodynamic technologies to reduce drag, · Structures and materials technologies to reduce weight, and 2JL Pitiman and D. Hahne, NASA Langley, "Twenty-first Cen- tury Aircraft Technology Project," presentation to the Vehicle Sys- tems Panel on March 18, 2003.

28 . Secondary electric power for zero COD and NO . . emissions. ~ x The committee believes that the project has a good portfolio and that NASA has properly assembled the right mix of aerodynamic and structural design tech- nology tasks to target drag and weight reduction, which will result in reduced CO2 emissions. Program Plan The program plans of the project presented by NASA support development of the key technologies required to meet the program-level goals of the Ve- hicle Systems Program. The program plan presented for the Efficient Aerodynamic Shapes and Integration subproject (3.2) was exemplary, with well-defined milestones and a good mix of mature and developing technologies. It was clear to the committee how these technologies produced direct benefits that will enable NASA to meet the stated goals. The program plan for the Integrated Tailored Struc- tures subproject (3.3) was not as well defined. The sub- project would specifically benefit from the creation of interim milestones. It was also not clear why industry and university involvement in these tasks had not been integrated earlier in the task life cycle. While aeroelas- tic tailoring clearly benefits the project goals, the weight benefits derived from the structures and materi- als technologies were not as evident. Technica/ Performance The committee found performance across the project to be good, with some excellent derived benefits in the High-Speed Slotted Wing (3.2.1) and Simplified High- Lift Systems (3.2.2) tasks. The experimental data pre- sented in the latter showed a clear improvement in ve- hicle performance using high-lift technologies. These tasks make excellent use of the wind tunnel resources at NASA to validate the computational predictive methods with experimental data. The Integrated Tailored Struc- tures subproject (3.3), in its buckling stability and dam- age tolerance tasks, also validates its predictive models using NASA test resources. User Connections The Twenty-first Century Aircraft Technology project has made good progress toward the goal of maximizing cost sharing with public and private funds, AN ASSESSMENT OF NASA 'S AERONAUTICS TECHNOLOGY PROGRAMS as well as making use of interagency partnerships. The project has also been effective at incorporating univer- sity researchers, although (as mentioned above) the committee believes NASA should include university researchers earlier in the development of the sub- projects. NASA has been able to increase the TRL of technologies and enhance technology transfer because of its cost-sharing with public and private partners in the Efficient Aerodynamic Shapes and Integration sub- project (3.23. Although these partnerships provide large benefits, they are not without risks. Reliance on partner assets creates resource uncertainties and may jeopardize project commitments. The committee encourages NASA to broaden its scope of involvement as much as possible. For instance, in the Integrated Tailored Struc- tures subproject (3.3), the wing structural design ac- tivities for both aeroelastic tailoring and structural con- cept development should be more closely tied to commercial transport industry teams. Finding: Ties to Industry. For certain advanced wing development activities, ties to the current com- mercial end user were not readily apparent. Recommendation: Ties to Industry. In order to broaden the scope of the work and to ensure long- term viability, both the aeroelastic tailoring and structural concept development tasks in the Inte- grated Tailored Structures subproject (3.3) should work more closely with the nation's commercial transport manufacturers. Assessment by Subyroject While many of the tasks in the Twenty-first Cen- tury Aircraft Technology project are solid and the com- mittee does not have recommendations for change, sev- eral others could benefit from redirection. For example, NASA should acquire better tools and develop appro- priate metrics in the Technology Integration and As- sessment subproject (3.1) and evaluate alternative pro- duction processes in the Tailored Materials and Processing Technologies task (3.3.2~. The committee presents details of subprojects and tasks below. Technology Integration and Assessment Subproject (3.1) The purpose of this subproject is to quantitatively measure the impact of TCAT on the emission reduc- tion goal of NASA's Aeronautics Technology Pro-

ASSESSMENT OF THE VEHICLE SYSTEMS PROGRAM grams. The subproject attempts to identify unfunded, high-payoff technologies. In this sense, the subproject is not actually a research and development effort, but a management tool. The committee acknowledges that there is certainly a benefit to having the capability to assess technology payoff. Finding: Technology Assessment Tools. The tools being used by the subproject appear to be insuff~- cient to perform the detailed systems analyses and assessments required to measure the results of re- search by the Twenty-first Century Aircraft Tech- nology project against the goal of 20 percent CO2 reduction. In the write-ups provided to the commit- tee, NASA does recognize this shortfall, but no plan or investment to resolve these deficiencies was ap- parent. Finding: Technology Assessment Metrics. The po- tential benefits of certain technologies are not easily quantifiable (e.g., new computational tools) and measuring them may require either a detailed sys- tems analysis or the development of an unconven- tional baseline aircraft (e.g., a blended wing body). Metrics for establishing priorities within the sub- project do not exist. Recommendation: Technology Assessment Tools and Metrics. In order to project the technology pay- off for the Technology Integration and Assessment subproject (3.1), NASA should establish an assess- ment plan with appropriate resources and clear metrics. Efficient Aerodynamic Shapes and Integration Sub- project (3.2) Efforts in the Efficient Aerodynamic Shapes and Integration subproject (3.2) are exemplary. The goals of the subproject are to develop and demonstrate a suite of innovative, vehicle-integrated aerodynamic tech- nologies. These technologies would reduce vehicle drag, increase transonic speed, or allow for extremely short takeoff and landing distances for subsonic trans- port (commercial, business jet) and tanker missions. Finding: Aerodynamic Shapes and Integration. The project plans and milestones for the Efficient Aero- dynamic Shapes and Integration subproject (3.2) clearly support the project goals, and NASA has established solid relationships with both industry 29 and universities. The computational fluid dynamics methods and experiments, with emphasis on the transonic regime, have been well linked. Integrated Tailored Structures Subproject (3.3) This subproject contains a good mix of the basic technology development required to meet subproject goals. The goals of a 35 percent weight reduction and a 25 percent increase in aspect ratio strongly support the overall VSP goals. To achieve these goals, however, NASA must aggressively pursue highly efficient aircraft structures and materials to meet strength and stiffness requirements. Specifically, in order to meet the aggres- sive vehicle weight goals, NASA must expand its port- folio to include highly efficient structural concepts. The methods development work in the various tasks is excellent. The hardware development activi- ties are appropriate for the higher TRL tasks and they also serve to validate the analytical methods using em- pirical methods. The structural development tasks should expand the scope of stiffening approaches to include sandwich structures. While it is clear that progress is being made in the Tailored Materials/Processing Technology task (3.3.2) with vacuum-assisted resin transfer molding (VARTM) processing, this production method is already very ma- ture in the industrial sector and it is questionable whether this approach has any performance benefits. NASA should be the leader in technology development efforts and should not be playing catch-up with industry. Tailored Structures Task (3.3.1 J NASA may realize its project weight and aspect ratio goals if it succeeds in developing the aeroelastic tailoring of a wing. NASA is taking full advantage of the capabilities for composite materials by building in a bend-twist coupling for the wing structure. The NASA team appears to be competent in use of the tools required to perform this task. However, the team should utilize multidisciplinary optimization tech- niques that couple the unsteady aerodynamics (flut- ter) response of the wing to the more basic structural optimization. Finding: Resin-Film-Infused Wing Baseline. NASA appears to have selected a stitched resin-film-in- fused wing baseline because of funding limitations for executing the research program.

t 30 Recommendation: Resin-Film-Infused Wing Baseline. NASA should reevaluate this baseline structural con- cept, developed for low-cost manufacturing in the Advanced Composites Technology program, which is not likely to produce the highest performance wing structure. Unitape preimpregnated composite sand- wich wing skin concepts should be included in the initial trade study candidates for the aeroelastically tailored wing. Strong ties to the Boeing commercial wing development program in Seattle should be established to attract the interest of an exter- nal customer who might make use of the wing box. l ) AN ASSESSMENT OF NASA 'S AERONAUTICS TECHNOLOGY PROGRAMS to capture emerging and highly efficient structural shell concepts. These concepts are consistent with industry recommendations for both the NASA Advanced Com- posites Technology and NASA High Speed Research fuselage structures. Finding: Design Technology. The validation of analysis methods using structural test shell elements is well done. Recommendation: Design Technology. The valida- tion of analysis methods using structural test shell elements should be continued in a modular test plan. NASA should establish closer ties to the end users that might eventually use this technology for their commercial fuselage structures. Tailored Materials and Processing Technology Task (3.3.2) The relevance of this task to VSP goals is minimal. While the VARTM process has the potential to reduce cost, the structural properties of VARTM-produced composites are typically poorer than those of standard uptake preimpregnated laminates. The VARTM process described by NASA is mature in the aerospace industry and available from several subcontractors. Tailored laminates with tight process control are available from the industry using standard unitape preimpregnated ma- tenals with custom fiber orientations. Residual Strength/Damage Tolerance Task (3.3.4J Like the preceding task, this task supports project goals by developing technologies that have the poten- tial to reduce airframe weight. The methods presented for high-fidelity residual strength analysis of compos- ites also improve future airframe reliability. The analy- sis method validation using structural test elements was well done and should be continued. The activity was well planned. The committee believes that this task should continue per the current task plan. Green Efficient Aircraft Power Subproject (3.4) Finding: High-Performance Laminates. The vac- The long-range objective of the Green Efficient uum-assisted resin transfer molding process is ma- Aircraft Power suborolect (3.41 is to use fuel cells to ture and used by industry. Recommendation: High-Performance Laminates. NASA should continue to investigate high-perfor- mance laminates; however, current funds should be redirected to address emerging, nonautoclave cur- ing methods, such as ultrasonic tape lamination or e-beam cure, to achieve program goals. Design Technology for Tailored Structures Task (3.3.3 J The current work in this task supports project goals, and the technologies have the potential to reduce air- frame weight. NASA should expand the methods for evaluating the buckling stability of stiffened shells, in- cluding skin-stringer stiffened shells, syntactic sand- wich shells, and honeycomb sandwich stiffened shells, ~ ~ ~- ~~-r--J--- I-- ~ create an all-electric, zero-emission aviation propulsion system. This objective requires a paradigm-shifting approach consistent with NASA's role as a high-risk, high-payoff technology incubator, and the committee applauds NASA for having this vision. Since the initial review, NASA has decided to ter- minate the subproject as it was originally constructed. The committee agrees with this action, because the sub- project is no longer focused on its long-term objective of creating a zero-emission propulsion system and will not bring about the paradigm shift to all-electric air- craft. Most of the ongoing projects did not logically support the technologies that will be required for the long-term vision. Specifically, much of the effort was directed toward the development of a Jet-A reformer/ solid oxide fuel cell auxiliary power unit that would be used at airports.

; -: ~ :< ~ ' r - . ~ . ASSESSMENT OF THE VEHICLE SYSTEMS PROGRAM Neither of the two most directly relevant tasks, Hydrocarbon Fuels Processing and Fuel Characteriza- tion (3.4.1 ) and Configuration and Performance Evalu- ation (3.4.3), were developing technologies relevant to the long-term all-electric vision. Furthermore, much of the content of these two tasks and the content in the Power Management and Testbed task (3.4.2) is redun- dant with work being done elsewhere. On a positive note, NASA directed a small element of the Configuration and Performance Evaluation task (3.4.3) toward a fuel-cell-powered small plane. This element was well leveraged, cost effective, and in sync with the long-range goals. In summary, the committee applauds the vision behind the Green Efficient Aircraft Power subproject (3.4) and encourages future endeavors in this area. NASA is to be commended for recognizing the issues with the current formulation of the subproject and mak- ing the appropriate decision to cancel the effort. Finding: Fuel Cells. The use of fuel cell technology to create an all-electric, zero-emission aviation propulsion system is a paradigm-shifting approach consistent with NASA's mission. However, much of the ongoing work in Green Efficient Aircraft Power (3.4) did not support this vision, and the committee was informed that NASA is canceling this subproject. Recommendation: Fuel Cells. The committee en- dorses the decision to cancel the Green Efficient Aircraft Power subproject (3.4) as previously con- stituted but urges NASA to pursue future work in this area, which leads to the long-range goal of a zero-emissions propulsion system. Advanced Vehicle Concepts Pro ject (4.0) Background The Advanced Vehicle Concepts project (4.0) is charged with developing advanced vehicle concepts and configurations to reduce travel time, expand com- merce, and open new markets. This is accomplished by examining a variety of systems to identify new high- payoff aircraft design configurations, performing ground-based tests, and conducting analyses and flight validation. This project Will be complete and no longer exist after FY 2004. It has had some successes, discussed 31 below, that will be transitioned to other, higher-TRL programs for continued development. Portfolio The committee found the portfolio of the Advanced Vehicle Concepts project (4.0) to be comprehensive and appropriate to NASA's and the program's mis- sions. The technology being developed is state of the art, and the work is relevant and aligned with top-level goals. The committee believes this project should be continued. As a whole, it should be noted that the Advanced Vehicle Concepts project (4.0) represents a well-man- aged and well-conducted mixture of technologies and . . applications. Program Plan In general, the plans for most programs are ad- equate. The Hyper-X subproject (4.3) is extremely well managed and stands out as exceptional. This project was logically derived from it research prede- cessors and has a clear path forward with appropriate planning off-ramps; the committee believes these fac- tors contribute significantly to the success of this project. NASA personnel account for the fact that the project is high risk and manage the project accord- ingly. The committee encourages NASA to look closely at this project and use the management prac- tices developed to serve as an example for other projects. In the Vehicle Concept Teams task (4.1.5), how- ever, only preliminary plans appear to be in place. A1- though the task aims are commendable, the committee believes that baseline data for other than the subsonic transport will be needed (e.g., for the joined and strut- braced wing) in order to meet the subproject goals. Technica/ Performance The committee believes the overall project is well run and contains much merit. As an example, it finds the controls work in the Revolutionary Aircraft Flight Validation subproject (4.2) outstanding. Finding: Project Quality. The subprojects within the Advanced Vehicle Concepts project (4.0) are appropriate, and NASA should pursue them ag- gressively.

An Assessment of NASA 's Aeronautics Technology Programs - Prepublication Copy Finding: Noise Source Abatement. Developing noise source abatement technology is a critical area for the air transport system and is consistent with NASA's mission. Recommendation: Noise Source Abatement. NASA must continue to identify tasks anct conduct research to advance technology for noise source abatement. The committee specifically commencis the Community Noise Irnp act Reduction subproject (2.2~. It commends the excellent linkage between modeling and full-scale data acquisition/flight procedure verification programs, such as the Low-Noise Flight Procedures task (2.2.3~. The committee also commends the excellent use of teaming, including a relevant airport (Louisville), a manufacturer (Boeing), academia (Massachusetts Institute of Technology), NASA (both Langley and Ames), an operator (United Parcel Service), and both controllers and policy makers at the FAA. In this case, NASA took the technology to a TRE 6 by doing a real-life demonstration. Although it is arguably not a mode} that should be followed by all NASA programs, it is very appropriate for some technologies that require a full community demonstration. The committee did have some concerns about the portfolio content of the other subprojects. For example, the committee questioned NASA research in cabin noise abatement given manufacturers' ongoing investments in this area and the limited resources NASA has to pursue the very ambitious goals of the task. Program Plan At the project level, Quiet Aircraft Technology (2.0) has clearly defined goals. These goals include reducing the perceived noise levels of future aircraft by one half (10 dB) iTom 1997 subsonic aircraft within ~ O years and by three quarters (20 dB) within 25 years. The NASA presenters understood and articulated the user benefits well. The TO-year goal enables containing 65 Day-Night Level (Deaf) noise within an airport's physical boundary. The 25-year goal ambitiously seeks to contain noise within airport boundaries at 55 DNL. Manufacturers, operators of the airlines and airports, and the FAA endorse these goals, as does this committee. The committee believes that the program plan for the Community Noise hnp act Reduction subproject (2.2) is exemplary. The plan has an excellent mix of modeling, simulation, flight validation, and laboratory experimentation. The NASA project team showed excellent qualifications and the NASA Ames simulation facilities are world class. NASA also does an excellent job of getting relevant stakeholders to participate on the team. The committee had concerns about the program plans for the Airframe System Noise Reduction (2.~) and Engine System Noise Reduction (2.3) subprojects. Although the committee believes the work is important, it noted weaknesses in that the subprojects did not always focus on key areas with the highest payoff. The committee recommends that NASA select the highest- priority research through consultation with relevant stakeholders. Before initiating tasks managers should clearly define A, , goals and the milestones along the way that signal research success, redirection, or failure. For example, the committee believes that NASA should examine 32

ASSESSMENT OF THE VEHICLE SYSTEMS PROGRAM into civilian transport because this technology has great promise for flight controls transparency in the presence of system component failures. This subproject represents the type of work NASA should be doing that is, bringing technologies to a higher TRL. It has taken many interdisciplinary com- ponents and has properly applied NASA resources to the problem. The subproject draws upon the superior expertise in flight systems, controls, and simulation of researchers at Ames, Langley, Dryden, and the aca- demo community. The subproject has evolved from academic work to piloted simulation. Ultimately, NASA will perform flight testing, which nicely fits NASA capabilities. The subproject is well planned and takes technologies that can be demonstrated with real hardware in a multistage process. This is a clear-cut example of what NASA is uniquely qualified to do in a step-by-step process that ends in flight test. The committee commends NASA for its innovation in acquiring assets to conduct the test- ing. The combination of these entities under the NASA rubric is world-class. Hyper-X Subproject (4.3) The Hyper-X subproject shows some of the best planning seen across all the programs reviewed by the committee. The NASA planning reflects the high-risk aspect of this task by providing for three vehicles and anticipating possible loss. The first flight test was not successful because a rocket booster failed, demonstrat- ing the wisdom of the contingency aspects of this plan. The subproject is well connected programmatically to its antecedents, another of its notable features. In- deed, many of the detailed aspects to be investigated are directed at answering key questions surrounding hypersonic flight. By virtue of careful consideration of this background and good planning, the goals of the subproject are realistic and the risk associated with it has been mitigated. The ultimate goal is to demonstrate positive net thrust of the scramjet; this is a laudable, though difficult, goal that the committee hopes can be achieved. Flight Research Project {5.0) Background The VSP Flight Research project (5.0) is conducted at NASA Dryden. The project is focused primarily on 33 testing and validating, in a realistic flight environment, technologies and tools developed at NASA's other re- search centers. Consequently, the committee focused its review of the project not on technologies but on the people, assets, and infrastructure in place at NASA Dryden to support these test requirements. The 5.0 project activities are broken into five subprojects: Flight Research Productivity (5.1), Advanced Systems Concepts (5.2), Integrated Transport and Testbed Experiments (5.3), Western Aeronautical Test Range (5.4), and Environmental Research Aircraft and Sensor Technology (5.51. Each subproject was reviewed in detail, culminat- ing in an on-site review at NASA Dryden. Overall, the facilities, people, and resources at Dryden are outstand- ing. Dryden plays a key role in providing the user and aeronautics market with the confidence that technolo- gies are indeed ready for transition. Because the content of this project does not fit the template for the other Vehicle System Project areas, the committee provides its summary in a slightly dif- ferent format. Portfolio Simulator facilities, laboratories, aircraft hangar, and storage space have been consciously pared down over time to be in alignment with projected future test- ing levels. Facilities such as the flight simulation cen- ter are limited in scope but allow for ground mission rehearsals and preflight validation of operating flight program software for piloted and nonpiloted test pro- grams. This simulation capability is vital to reducing flight test costs. Appreciating that the NASA Dryden and its assets represent a large fixed cost for the VSP, the committee attempted to identify where NASA could reduce costs. Overall, the committee found little opportunity to reduce infrastructure or test-related assets, with two possible exceptions. The first opportunity is with the F/A-18 fleet and the second in the electronics proto- typing laboratory. The committee believes it may be feasible to re- duce the number of F/A-18 test aircraft. There are rela- tively few programs that utilize these aircraft, and NASA has been allowing the Air Force to use its air-

34 craft on occasion, a sign that the F/A-18 test fleet is not used to its fullest capacity by NASA. Finding: Fleet Size. The total number of F/A-1X test aircraft appears to be greater than NASA requires. Recommendation: Fleet Reduction. The Vehicle Systems Program should examine the future needs of the F/A-~8 aircraft test fleet at NASA Dryden as a possible way of reducing fixed costs. The commit- tee estimates that four or five flyable F/A-18 air- craft will be required to meet future needs. AN ASSESSMENT OF NASA 'S AERONA UTICS TECHNOLOGY PROGRAMS maintainers, laboratory technicians, engineering staff are also commendable in quality, experience, and breadth of knowledge. As with most flight test or- ganizations, Dryden has become skilled in adapting to the constantly changing support needs and priorities of different customers. For the level of testing currently being done there and planned through FY05, the facil- ity is the right size. One concern the committee has is that as NASA implements its full-cost accounting system, many re- searchers might elect to stop their technology develop- ment at TRL 5 or 6, because pushing their technologies forward to full-scale test might become prohibitively expensive. NASA should take positive steps to ensure that full-cost accounting is implemented in a manner that does not unintentionally reduce the willingness of developers to conduct full-scale testing and, conse- quently, the willingness of the user community and market to adopt these technologies. In the prototype electronics lab, which supports unique and short-turnaround circuit board manufacture, the manufacturing assets in place are being used, by NASA's estimate, at 20 percent of capacity to populate boards. The committee recommends that standing con- tractual arrangements for populating printed circuit boards be pursued with outside contractors as a means to eliminate the cost of maintaining and housing these assets. The committee expects that if the above arrange- ments are made, the support infrastructure associated with these assets would also be reduced. The electron- ics packaging and design capability should, however, be retained in-house. Finding: Circuit Board Manufacturing. A portion of the current practice of internal circuit board manufacture at NASA Dryden appears to be ineff~- cient and costly. Recommendation: Circuit Board Manufacturing. To save costs, NASA Dryden should establish external contract agreements to provide printed circuit board assembly services (circuit board population) as it cur- rently does with circuit board manufacturing. Technica/ Performance NASA Dryden provides NASA with the unique ability to conduct research from concept to full valida- tion in a realistic flight environment. An on-site review of the facilities, control rooms, labs, and hangars showed that the center has assets and skills in place to meet the broad range of test needs brought to it by other NASA research centers. Safety practices, operational procedures, and facility mainte- nance practices are of the highest quality. Similarly, the talents at NASA Dryden- pilots, Finding: Full-Scale Flight Testing. Full-scale flight test capability at NASA Dryden is an important catalyst in getting industry to embrace new tech- nologies and to move technologies into the market- place. If this last step in the test and validation pro- cess becomes unaffordable, industry will be unwilling to take new technologies beyond technol- ogy readiness level 5 or 6. Recommendation: Full-Scale Flight Testing. NASA should work diligently to ensure that full-cost ac- counting is implemented in a manner that does not reduce the effectiveness of research by inhibiting the use of full-scale flight testing at NASA Dryden. User Connections By their very nature, all activities conducted at NASA Dryden have strong involvement from a user community. The committee noted one aspect of the Hellos task (5.5.1) that requires special consideration. The technical results of this task have been outstand- ing, as demonstrated by overnight flights of this all-elec- tric, high-altitucle vehicle. The committee fully expects that the Hellos vehicle will yield significant results for the earth sciences portion of NASA, its primary cus- tomer. The committee further applauds NASA for inno- vative thinking in identifying other possible uses and other possible markets for the aircraft, such as serving as a low-cost, high-altitude (relatively) stationary telecom-

ASSESSMENT OF THE VEHICLE SYSTEMS PROGRAM munications platform. Despite the best efforts of the part- ner company and aircraft manufacturer, AeroVironment, to attract interest from the U.S. industry, only Japanese telecommunications researchers have tested their equip- ment on the Helios platform. The committee acknowl- edges that if this telecommunications strategy pays off, NASA will have helped establish a small overseas niche market for future Hellos aircraft. Ultra-Efficient Engine Technology Pro feet {6.0) Background The combination of the Ultra-Efficient Engine Technology project (6.0), the Propulsion and Power project (7.0), and the Green Efficient Aircraft Power subproject (3.4) incorporates most of the engine and propulsion elements of the aeronautical program at NASA. The range of projects extends from low TRL- very high-risk, high-payoff tasks—to projects with relatively high TRL values that in some cases have appeared in flight vehicles. Because the Ultra-Efficient Engine Technology (6.0) and Propulsion and Power (7.0) projects are so intertwined, their background, portfolio, program plans, technical performance, and user connections are discussed together here. The Propulsion and Power tasks are discussed in the next section. The Propulsion and Power project (7.0) discovers, develops, and verifies in the laboratory advanced tech- nologies that improve the quality of life by reducing exposure to aircraft emissions and increasing mobility. NASA accomplishes this by investing in new turbine engine technologies, new propulsion concepts, and foundational propulsion and power technologies em- phasi~ing high-risk, high-payoff concepts and tech- nologies. Portfolio The Ultra-Efficient Engine Technology (UEET) project (6.0) is a relatively tightly structured array of subprojects and tasks clearly aimed at improving en- gine performance, using either efficiency or emissions as the metric of success. Much of the work is at a rela- tively high TRL level and is done jointly with industry. Consequently, the paths forward and the placement and interrelationships of the various tasks and subprojects within the project are straightforward. The Integrated Component Technology subproject . 35 (6.6) contains a notable array of tasks. It consists of a series of "other transaction" agreements, which permit NASA to have creative partnerships with industry and advance technology readiness. However, NASA should not use these agreements exclusively, as the committee was concerned about overinvesting in technologies that would not contribute to the general knowledge base and overall public good because of intellectual prop- erty restrictions. The other project, Propulsion and Power (7.0), has a much more diverse mix of subprojects and tasks. It tends to emphasize the research and low-TRL side of the technology maturation process more than does the 6.0 project. The two projects are complementary in this regard. Of particular note in the 7.0 portfolio is the Revo- lutionary Aeropropulsion Concepts subproject (7.11. This is a commendable subproject as it offers an excel- lent approach for achieving a key portion of NASA's mission: the pursuit of high-risk, high-payoff work that otherwise would not be performed by the community. Another commendable subproject is Oil-Free Turbine Engine Technology (7.5~. This subproject targets an area that could be a significant gain for small gas tur- bine engines and that might have long-term applicabil- ity to large commercial engines as well. The Propul- sion and Power (7.0) portfolio contains a good balance of modeling and experimental tasks. One of the ele- ments involved a resourceful approach to obtaining long-term engine data by using a commercial turbo- generator system. The committee was concerned about some tasks in the general engine and propulsion portfolio. mainly in the Green Efficient Aircraft Power (3.4) and Pulse Detonation Engine Technology (7.8) subprojects. The committee notes that the Green Efficient Air- craft Power subproject has been canceled and agrees with this decision. However, it also believes strongly that the vision and goals of the subproject offer a para- digm-shifting approach that is clearly consistent with NASA's role as a high-risk, high-payoff technology incubator and that NASA should pursue these visions and goals. This subproject is discussed in more detail above. The committee believes NASA should also closely reexamine the need for the Pulse Detonation Engine Technology subproject (7.8), because much of the ef- fort directed at military objectives is redundant and that directed at civil aviation is unlikely to be useful. There are other areas within the propulsion area

36 that NASA should reexamine and possibly reconfigure, refocusing the work to reflect available resources. In the Materials and Structures for High Performance sub- project (6.4), some milestones are too ambitious and there are no realistic plans to satisfy them. This is the case in the 3000°F Ceramic Matrix Composite System (6.4.4) and Ultra-High-Temperature Ceramics (6.4.5) tasks. Furthermore, the goals of the Computational Ma- terials Science-Metallic task (6.4.3) do not appear to be realizable with the time and funds available. The con- tractors are working at temperatures where microstruc- ture is not stable with time and plastic deformation is continuously occurring. This set of conditions presents a considerable challenge, although ultimately signifi- cant progress could be made. However, the task cur- rently lacks sufficient progress metrics, decision points, and off-ramps. It should be reconsidered and refocused, with reasonable milestones against which its progress can be compared. ., ~ ' .! ' .i AN ASSESSMENT OF NASA 'S AERONA UTICS TECHNOLOGY PROGRAMS Program Plan Plans for the majority of projects and subprojects are appropriate, with meaningful milestones and re- views. The Materials and Structures Turbine Airfoil System task (6.4.1) is an example of excellent plan- ning. The Revolutionary Aeropropulsion Concepts subproject (7.1) also had an excellent approach for as- sessing progress at the appropriate milestone and for managing off-ramps if needed. The committee did have some concerns. For in- stance, the process used by NASA to formulate emis- sions goals was unclear to the committee: it appears to link the goals to regulatory processes, which may not be appropriate. Aviation regulatory emissions goals are not generally designed to push technology, and the goals could become subject to political issues unrelated to technology. The committee strongly believes that NASA should be at the forefront of setting emissions reduction goals that look beyond what regulators are doing today, leading the way in addressing new issues that push the boundaries of current technology. The committee believes that it is imperative for NASA to address the interrelationships between noise and emissions. NASA should leverage the work tradi- tionally being done in UEET and QAT, using common demonstrators where appropriate. The high cost associated with the large number of technicians involved in facilities such as the combustor facility is problematic. This burden should be shared among a broader segment of the related projects; other- wise a facility could unintentionally become unafford- able for all projects. An example of this is the Emis- sions Reduction subproject (6.2~. NASA should review this situation carefully so it does not negatively impact other projects. Technica/ Performance Many of these subprojects are extensions of previ- ous work and they have recently been replanned. Since many are still in the early stages of development, the technical accomplishments are limited at this juncture. The committee believes that a strong example of achievement is the Compressor Flow Control task (7.4.2), which was able to transition from a fundamen- tal research idea at the Massachusetts Institute of Tech- nology to a proposed test in an Army engine (T-700) in 2004. Performance and achievement of many tasks are hindered by resource limitations and concern about technology clownselection, as happened to combustor contractors in the Emissions Reduction subproject (6.2~. Premature downselection in this subproject may limit the degree of technology exploration, and if there is a single failure, the overall subproject might fail. The committee had several areas of concern. For instance, in the Fan Trailing Edge Ejection task (6.3.1), despite additional dialogue with NASA, the committee questions the connection between the benefits assess- ment and the overall vehicle systems benefit. Specifi- cally, NASA should consider tracle-offs between noise reduction and system penalties such as weight, specific fuel consumption, and emissions. User Connections Owing to the large number and varied types of sub- projects and tasks within the projects, there is naturally a wide range of user involvement. Particular examples of excellent user connectivity include the Ultra-Safe Propulsion subproject (7.7). This is a user-driven re- search project addressing real-worId problems with close industrial collaborations. The University Research and Engineering Tech- nology Institute (URETI) subproject (7.3) is another example of very good user connectivity. The URETI advisory board enables connections with many enti- ties, which ensures good leveraging of resources. How-

ASSESSMENT OF THE VEHICLE SYSTEMS PROGRAM ever, the URETI directorate needs to empower the advisory board to terminate and redirect tasks as ap- propriate to ensure progress toward project goals. The committee is also concerned that, while the URETI in- volves multiple universities, it excludes many others, with substantial investment going to a small subset of potential participants. Hence by default some poten- tially useful contributors are not engaged, possibly lim- iting opportunities and reducing the diversity of views. This is offset by the worthy objective of attaining criti- cal mass by involving the URETI centers. Another noteworthy example of user connectivity is the Highly Loaded Turbomachinery subproject (6.31. This subproject has extensive involvement with engine companies and DoD in various tasks, as well as in- volvement with the Integrated Component Technology subproject (6.64. This latter array of tasks uses flexible contracting mechanisms to provide industry with a stakeholder role in the efforts, which further enhances technology acceleration and transitions. As with URETI, though, there could be a weakness if this tool is overusecl, as the focus on a single contractor could diminish the overall benefit to the community. The committee believes that extensive interaction with industry review panels is essential to ensure that NASA is effectively using its limited resources. Ac- cordingly, it believes that NASA should critically evaluate the current composition of its industry review panels. For example, the inclusion of the airline and airport industry is highly recommended. ah , .;.... ~~ ~ Hi-\ ., . ~ 37 Assessment by Subproject Propulsion Systems Integration and Assessment Sub- project (6.1) The strengths of this subproject include stake- holder interest in high-fidelity system simulations. The NASA research team and available facilities are of high quality and are appropriate for the stated tasks. The overall subproject is well structured and has well- defined milestones. The committee found weaknesses in this subproject in that the work relies heavily on the team at the Geor- gia Institute of Technology and its probabilistic metric assessment. The committee notes that this work is gen- erally sound but also believes that NASA should con- sider additional and alternative methods of evaluation. Another concern the committee had is the apparent lack of NASA participation in projects related to interna- tional atmospheric environmental data. As a general observation, the committee had con- cerns that NASA's approach to integrating discrete technologies is not consistent with accepted industry practice for systems integration. Emissions Reduction Subproject (6.2J The strengths of this subtask include unique fa- cilities such as high-pressure combustor rigs at NASA Glenn. The committee believes that the goal of reduc- ing emissions is sound and that NASA uses good milestones for advancing the TRL of the technologies in the subproject. Industry partnership agreements also enhance connectivity with the stakeholders. The modeling in the subproject is predicated on industry- accepted codes such as large eddy simulation and na- tional combustion codes. This work is considered world-class. The committee considered the transfer of more basic work from the Smart Efficient Compo- nents subproject (7.4) to this subproject to be a posi- . . tine, evolutionary move. The committee believes that this subproject should separate its goals from regulatory processes, which are generally conservative and potentially fraught with political issues. These regulatory processes do not al- ways consider technological implications nor do they address new environmental issues such as the reduc- tion of particulate matter and air toxins, which NASA should rightly address. NASA should also consider trade-offs. NASA may be inhibited by resource limitations from working with the broad industry base required for transition, which might result in missed opportunities. The high cost associated with the large number of technicians involved in the combustor facility is an- other weakness, mentioned above. A broader segment of related subprojects should share this burden. NASA should review facility burden carefully so it does not negatively impact the subprojects. The current plan to downselect to a single contrac- tor for each of two engine types concerns the commit- tee because it might limit the degree of technology ex- ploration. Moreover, should the one selected option fail, the overall project would also fail. The committee recommends that NASA carefully consider mitigating this risk of project failure by carrying the projects to a higher TRL before downselecting. The committee ac-

38 ~ . AN ASSESSMENT OF NASA 'S AERONAUTICS TECHNOLOGY PROGRAMS knowledges the reality of funding constraints. NASA should seek innovative ways to maintain the project, perhaps through industry cost-sharing. Finding: Downselecting. As a consequence of fund- ing limits, NASA's current plan for the Emissions Reduction subproject (6.2) is to downselect to a single contractor for emission reduction technology work at a relatively low technology readiness level. Recommendation: Downselecting and Contractors. NASA should replan the Emissions Reduction sub- project (6.2) and plan future projects to carry ac- tivities to an appropriate technology readiness level before Downselecting to a single concept or contrac- tor. This process will mitigate the risk of losing valu- able technology. Recommendation: Downselecting and Technology Readiness Levele NASA should carefully consider what technology readiness level is appropriate for use in downselect decisions points in future program planning to avoid the loss of valuable concepts and technology. Highly Loaded Turbomachinery Subproject (6.3) The strengths of this subproject include high-risk, high-payoff tasks that take technology from TRL 1 to 4. This is a sound plan and one that NASA should con- tinue. The goal of reducing carbon dioxide by 8 to 15 percent through a reduction in fuel burn is a valid one. There is also good involvement from engine manufac- turers and DoD components in the subproject. The Dual Spool Turbine Facility task (6.3.7) is a valuable re- source and a national asset. The committee had concerns about the Fan Trail- ing Edge Ejection task (6.3.11. Despite additional dia- logue with NASA, the committee questions the con- nectivity of the benefits assessment to the overall vehicle systems benefit. Specifically, the task should consider trade-offs between noise reduction and emis- sions reductions and the impact on overall system per- formance. The committee questions the justification for the activities taking place under this task in light of the system-level trade-offs. As a general rule, if a task can- not be justified in terms of system-level gains (noise gains versus weight and fuel burn penalties), then it should be replanned or canceled. The committee rec- ommends that NASA reexamine task 6.3.1 in that light. Finding: Assessing System Penalties. The Fan Trail- ing Edge Ejection task (6.3.1) is an innovative concept with the potential to significantly reduce fan noise. This task, however, is also currently projected to in- cur significant performance and weight penalties. Recommendation: Assessing System Penalties. NASA should review projects and subprojects on a timely basis, including the Fan Trailing Edge Ejection task (6.3.1), and cancel tasks and/or sub- projects when gains do not outweigh overall sys- tem penalties. Materials and Structures for High Performance Sub- project (6.4) The strengths of this subproject include well-con- ceived and -defined tasks and realistic goals. The Ma- terials and Structures Turbine Airfoil System (6.4.1) is an excellent example and has a good chance to achieve those goals. The subproject goals meet the needs of both commercial and military engines and include a major engine company in the fabrication process. Weaknesses include testing that was using unreal- istic test conditions. The committee notes NASA is . . · . correcting this situation. In some cases, the committee found that milestones in some tasks were too ambitious and there was no re- alistic plan to reach those milestones. This situation occurs in the 3000°F Ceramic Matrix Composite Sys- tem task (6.4.4, part a) and the Ultra-High-Tempera- ture Ceramics task (6.4.54. In the Materials and Structures Turbine Airfoil System task (6.4.1), the fourth-generation turbine blade alloy may not be acceptable to airlines owing to a lack of oxidation resistance in the base metal under the coat- ing. Also, the task is using only a single mechanical property, stress-rupture, as an exploratory metric, which the committee feels is not sufficient. The com- mittee believes that NASA's choice of a civilian cus- tomer may be inappropriate because of the problems stated above. It encourages NASA to define and iden- tify military customers, or to consider blade oxidation resistance in the task work. Since the United States is currently conducting little or no nickel-based research, the committee believes this task is important and should continue. The Computational Materials Science-Metallic task (6.4.3, part a) appears to lack sufficient internal capabilities in this technical area, and the contract goals

ASSESSMENT OF THE VEHICLE SYSTEMS PROGRAM appear to not be realistic or realizable. Overall, the committee questions the unique value of this particular work and suggests that NASA reassess the task. There are two tasks working toward 3000°F ce- ramic matrix composites: the 3000°F Ceramic Matrix Composites System task (6.4.4, part a), and the Ultra- High-Temperature Ceramics task (6.4.59. The commit- tee did not observe innovation in task 6.4.4 part a. The committee also had concerns about task 6.4.5 since it has been in place for 10 years but has made little progress. NASA should address the necessity of hav- ing two programs with nearly identical goals. In addi- tion, the goal temperature of these programs may not be realizable. NASA should reassess the subproject goals and, if they cannot be justified, cancel the effort. Finding: Use of Milestones and Reviews. Goals for some of the tasks (6.4.3, 6.4.4, and 6.4.5) in the Ma- terials and Structures for High Performance sub- project were set extremely high, and the plans for achievement are overoptimistic. Recommendation: Use of Milestones and Reviews. NASA should structure projects and subprojects with milestones and review processes using senior management or outside advisory groups to assess progress and determine if NASA should continue, redirect, or cancel tasks or subprojects on a timely basis. Propulsion-Airframe Integration Subproject (6.5) The strengths of this subproject are that it uses an appropriate mix of system studies, aerodynamic mod- eling, and wind tunnel tests to identify and evaluate advanced integrated systems. The subproject involves relevant industry team members and universities. It has strong researchers and facilities, such as the Langley National Transonic Facility, that are essential for these types of tasks. In addition, the management of the sub- project had the courage to make the tough decision to cancel a task when industry interest was no longer there. The Active Flow Control task (6.5.1) is an example of a strong performer. This is needed, innovative re- search for low-observable aircraft and S-shaped ducts · ~ In 1n .ets. One area of weakness the committee identified is in the Advanced Configurations task (6.5.3~. This task focuses on limited airframe concepts, placing all em- 39 phasis on the blended wing body concept. While this effort has merit, the committee believes there are too few milestones for a 4-year effort and that NASA should learn from past industry work and from ongo- ing activities in similar configurations. The committee saw no indication that NASA had consulted with in- dustry on similar configurations. There was another area of concern in the Propul- sion-Airframe Integration subproject (6.51. NASA said that owing to limited funding, it is not looking at issues such as crosswind and angle-of-attack factors (distorted inlet flow) in the inlet testing and analysis. The com- mittee believes that if NASA does not examine these issues, which it can do even in the face of a limited budget, it may never find a practical solution. Integrated Component Technology Subproject (6.6) This subproject benefits from having manufactur- ers with a stake in the process through the use of flex- ible contracting mechanisms. This situation shortens the time for technology development and transition. However, there is an accompanying weakness if flex- ible contracting mechanisms are used too extensively. Flexible contracting mechanisms focus on one contrac- tor, so the technology does not always benefit the com- munity as a whole and may not meet the goal of greater public good. Given these criteria, the Aspirating Seal Demon- stration task (6.6.3) was well regarded by the commit- tee as having possible application to multiple engine types. The committee has also determined that the su- personic 10 It x 10 ft wind tunnel used in the Nozzle/ Inlet Components for High Speed Flight task (6.6.5) is a national asset. The inlet work is critical for continu- ing advancement. This task is well integrated and le- verages Versatile Affordable Advanced Turbine En- gine (VAATE) and Long Range Strike work conducted with the military. It also benefits supersonic business jet programs. One overall weakness of the subproject is that it does not appear to be well constructed and does not have a clear focus and prioritization of goals. The com- mittee was also concerned that the Mechanical Com- ponents task (6.6.4) is aimed at developing geared fan systems, which are supported by only one contractor in the user community. Finding: Supporting the User Community. Task 6.6.4 does not appear to support the engine commu-

40 nity at large, nor does it appear to have broad sup- port from the airline community. Recommendation: Supporting the User Commu- nity. Since the Mechanical Components task (6.6.4) does not support a broad range of community us- ers, the committee recommends that NASA replan or cancel this task. Intelligent Propulsion Controls Subproject (6. 7) A strength of this Subproject is the NASA Glenn Class-100 silicon carbide work, particularly the clean room at NASA Glenn, which is a unique facility. A number of programs rely heavily on this facility, which the committee believes shows its uniqueness. Another strength is the high-temperature semicon- ductor work, which has the potential for developing wireless sensors that would reduce weight, fuel flow, and emissions. Such sensors would also enhance affordability by requiring less maintenance. This work is applicable to supersonic technology and low observables and might be exploited for high-tempera- ture power electronic-based drive systems. The com- mittee identified one weakness: The linkage between this work and that of the Higher Operating Tempera- ture Propulsion Components Subproject (7.6) is not clear. Finding: In-House Collaboration. There is no clear linkage between the Intelligent Propulsion Controls (6.7) and the Higher Operating Temperature Pro- puIsion Components (7.6) subprojects. Propulsion and Power Pro jest (7.0) Revolutionary Aeropropulsion Concepts Subproject (7.1) The overall strengths of this Subproject are these: . . AN ASSESSMENT OF NASA 'S AERONA UTICS TECHNOLOGY PROGRAMS The committee is concerned that the number of projects being pursued is too great and inconsistent with current funding levels. Also, the external connec- tivity is predominantly with universities and small companies. A more appropriate connectivity for this type of work would be with larger manufacturers, which do not currently play a role in the Subproject Concentrating on universities and small companies may be a programmatic necessity, but NASA should give thought to engaging larger groups or corporations. Propulsion Fundamentals Research Subproject (7.2J The overall strengths of this Subproject are that many of the tasks address very early basic research work that industry would not take on, such as the Fun- damental Noise task (7.2.4~. The Nanotechnology task (7.2.2), which involves single-crystal silicon carbide nanotube systems, is innovative. NASA is a world- class leader in this work at higher temperatures. The committee was concerned that some of the fa- cilities appear to duplicate those at the Arnold Engineer- ing Development Center and the Air Force Research Laboratory. Also, connectivity to the national nanotech- nology program, the National Narlotechnology Initiative, is unclear. This raises concerns on the part of the com- mittee that perhaps not all of the tasks are firmly inte- grated into the broader community. Aeropropulsion and Power University Research and Engineering Technology Institute Subproject (7.3) The overall strengths of this Subproject are the fol- lowing: NASA is conducting research in an area it is uniquely qualified to evaluate and execute. The experimental and analytical work is consistent with theme objectives for vehicle systems. The long-term vision in task selection ad- equately balances risk with gain. NASA Glenn is making good use of both its own facilities and facilities external to NASA. The URETI concept is creative and provides a critical mass of researchers and facilities. The URETI advisory board brings connectivity that ensures resources are well leveraged. The URETI principal investigators and director are doing a good job of monitoring relevant in- ternational work. The committee noted that the experimental capa- bility of the compressor research in the Intelligent En- gine Systems task (7.3.3) was not clear. Finding: University Research and Engineering Technology Institute. The Aeropropulsion and

ASSESSMENT OF THE VEHICLE SYSTEMS PROGRAM Power URET! subproject (7.3) is innovative but contains some weaknesses, including these: . The URETI advisory board does not have the power to terminate or redirect tasks as appropriate, to ensure good progress toward goals. While the URETI comprises multiple uni- versities, it also excludes many other quali- fied ones. NASA does not have a mechanism to ensure continuity of the program in- situations where critical principal investigators change universities and where principal investiga- tors at other universities can be a significant asset if added. There is a conflict of interest in having an advisory board that includes individuals who conduct the research funded under the URETI. Recommendation: University Research and Engi- neering Technology Institute. NASA should review the URETI operating guidelines and make appro- priate changes to assure that the goals of the pro- gram are achieved. Smart Efficient Components Subproject (7.4) The adaptive flow control is a good example of transition from fundamental concepts at the Massachu- setts Institute of Technology to full-scale testing of an Army engine (T-700) in 2004. The committee believes the lean direct injection combustion research of this subproject is pioneenng. Finally, the facilities such as the large, low-speed, multistage axial compressor and the transonic oscillating cascade facility are unique for flow control and unsteady aerodynamics. The committee had concerns that the connectivity of the URETI research in the Compressor Flow Con- trol task (7.4.2) with NASA Glenn is not evident. Glenn is conducting solid research in compressor flow con- trol but it is not collaborating with the URETI program. Finally, NASA is accomplishing significant levels of research in-house, but leveraging the university com- munity would also benefit research progress. Finding: In-House Collaboration. The Compressor Flow Control task (7.4.2) does not appear to be col- 41 laborating with the solid research at NASA Glenn in compressor flow control. Oil-Free Turbine Engine Technology Subproject (7.5) This subproject targets an area of significant po- tential gain for small gas turbine engines and has a good balance of modeling and experimental work, including a creative approach for acquiring long-term engine data through a turbogenerator system. There is also good university involvement in developing a structural model for planned verification tests. The success of some of this work is evident from the collaboration between NASA and industry on air bearing designs applied to business jets, such as the Eclipse. The committee had two concerns about this sub- project. First, the committee encourages NASA to ad- dress drive issues such as power takeoff requirements for engine accessories and utilities. To help in this, the committee suggests coordination with, and leveraging of, the Air Force Research Laboratory's Versatile Af- fordable Advanced Turbine Engine (VAATE) pro- gram. Secondly, the subproject has not addressed the benefits of reducing drag from standard bearings. Finding: Oil-Free Turbine Engine Technology. NASA does not address concerns about drive issues such as power takeoff for engine accessories and utili- ties. NASA also does not currently address the ben- efits of reducing the drag from standard bearings. Recommendation: Oil-Free Turbine Engine Tech- nology. To make the subproject more effective, NASA should make contact with the Air Force Re- search Laboratory's Versatile, Affordable Ad- vanced Turbine Engine program in order to help leverage the Oil-Free Turbine Engine Technology subproject (7.5~. In addition, the subproject should address benefits of reducing drag from standard bearings. Higher Operating Temperature Propulsion Components Subproject (7.6) In the Ceramics task (7.6.1), the publication of ASTM standards for fracture toughness testing and bi- axial strength of ceramics was exemplary, as was the task's involvement with user-driven, high-quality re- search that addressed real-world problems. Also, the

42 ..: AN ASSESSMENT OF NASA IS AERONAUTICS TECHNOLOGY PROGRAMS Metallics task (7.6.3) is an example of scientists oper- ating outside the mainstream community on potentially high-payoff research, such as research methods that are computationally less intensive than classical methods. The present investigators understand that their ap- proach to computational alloy development is some- what outside the mainstream, but they cite their early successes as sufficient reason to continue their effort. It is impossible to determine at this stage if these inves- tigators have developed a suitable approach that will yield answers of acceptable quality while being much less computationally expensive than the classical meth- ods or if their techniques have limited scope and will not be able to produce acceptable results in a wide range of situations. Sometimes such work leads to breakthroughs and new paths for further development. This work should be continued until these questions can be answered. l NASA Glenn' s Class-100 silicon carbide clean room, which is heavily used in the High-Temperature Instru- mentation task (7.6.4) and in the Intelligent Propulsion Controls subproject (6.7), is a national research facility with many uses, although it is relatively inexpensive. The committee had the following concerns for this subproject: Traditional ceramics processing methods (hot pressing and slip casting) may be difficult to apply to complex configurations. Adherence of environmentally protective (life- extending) coatings on silicon nitride has not been adequately addressed. For instance in the Metallics task (7.6.3), the coating technique may not be adequate for two-phase materials. There is a high degree of reliance on computer- based predictions that have not been verified and may not be reliable. Ultra-Safe Propulsion Subproject (7. 7) An overall strength of the Ultra-Safe Propulsion subproject is its connection with its customers through user-driven research addressing real-world problems, which appropriately involves collaboration with indus- try. This subproject effectively leverages work of oth- ers in the field while achieving significant advances. The committee urges those in NASA involved in this task to review the recommendations in Chapter 4 of this report related to propulsion safety technology. Specifically, there needs to be more fundamental mate- rials work in this area. Safety considerations should be present in all research related to improving propulsion component performance in terms of higher turbine in- let temperatures, lower emissions, and less noise. Pulse Detonation Engine Technology Subproject (7.8) The committee acknowledges that increasing cycle efficiency by 10 to 15 percent is an admirable goal. However, it believes that pulse detonation technology is unlikely to help achieve this goal because of its many drawbacks. There appears to be no appreciation for the concerns of commercial customers (e.g., airlines) about noise. The committee believes there is a pressing need for a system analysis to show the potential for a pulse detonation engine to overcome apparent limitations and achieve the stated goals. Finally, NASA's unique con- tribution to pulse detonation engines is not apparent. NASA should reevaluate whether it should continue investing in pulse detonation engine research or lever- age DoD research for applications in the commercial sector. Finding: Pulse Detonation Technology. Much of the eiTort of the Pulse Detonation Engine Technology subproject (7.~), while having potential military application, is unlikely to serve civil aviation needs. Recommendation: Pulse Detonation Technology. To bring tasks more in line with NASA capabilities and goals, the committee recommends that the Pulse Detonation Engine Technology subproject (7.X) be canceled.

Next: 3 Assessment of the Airspace Systems Program »
Review of NASA's Aerospace Technology Enterprise: An Assessment of NASA's Aeronautics Technology Programs Get This Book
×
Buy Paperback | $48.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The National Research Council (NRC) of the National Academies was asked by NASA and the Office of Management and Budget to perform an assessment of NASA's Aerospace Technology Enterprise. The first such review, which began in early 2002, examined Pioneering Revolutionary Technology (now known as Mission and Science Measurement Technology). The assessment presented here, of the Aeronautics Technology Programs, began in early 2003 and is the second in the review series.

The Aeronautics Technology Programs has three components: the Vehicle Systems Program, the Airspace Systems Program, and the Aviation Safety Program. To conduct this review, the NRC established three panels, one for each of the component programs. The NRC also established a parent committee, consisting of the chairman and a subset of members from each panel. The committee and panels comprised a cross-section of experts from industry, academia, and government and included senior-level managers and researchers in the aeronautics field. Biographical information on the committee and panel members is found in Appendix A.

Review of NASA's Aerospace Technology Enterprise: An Assessment of NASA's Aeronautics Technology Programs contains the committee's assessment of the Aeronautics Technology Programs. Chapter 1 presents a top-level assessment, and Chapters 2 through 4 provide the assessments of the Vehicle Systems Program, the Airspace Systems Program, and the Aviation Safety Program, respectively.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!