4
Key Findings and Recommendations

Advancing the state of aviation safety is central to the mission of the National Aeronautics and Space Administration (NASA) and its Aeronautics Research Mission Directorate (ARMD). In keeping with direction from the National Aeronautics Research and Development Policy1 (referred to as the “National Policy”) and the National Plan for Aeronautics Research and Development and Related Infrastructure)2 (referred to as the “National Plan”), ARMD’s emphasis is on pursuing what it describes as “long-term, cutting-edge research that expands the boundaries of aeronautical knowledge for the benefit of the broad aeronautics community.” In particular, ARMD is charged with undertaking “foundational” research that consists of basic research and developing a strong aeronautics knowledge and technology base to overcome barriers to technological progress in aviation.

Much of ARMD’s safety-related research is aimed at predicting and preventing safety hazards in an increasingly diverse and complex Next Generation Air Transportation System (NextGen). Underlying this interest is concern that as NextGen capabilities are introduced and the national airspace becomes more heavily used by a diversity of aircraft, new safety hazards will arise that require innovative solutions through research. Yet, not only must these future safety hazards be predicted and prevented with NASA’s help, but the agency also will continue to be called on to find solutions to safety problems arising in the current system. While this system performs at very high levels of safety, there is an expectation that safety advancements will continue, both in the near and longer terms.

Congress requested this review of NASA’s aviation safety-related research programs. This chapter presents the main findings of the review as they relate to the four questions in the congressional request. Some of the findings go beyond these specific queries. For instance, in assessing NASA’s safety research coordination and transitioning activities, consideration is given to the connections established with industry, academia, and the aviation community generally as well as those forged with the Federal Aviation Administration (FAA) and other federal agencies. Because NASA’s aeronautics research constitutes a key component of the nation’s aviation research enterprise, its safety-related activities warrant consideration within this larger context.

1

National Science and Technology Council, National Aeronautics Research and Development Policy, Office of Science and Technology Policy, Executive Office of the President, Washington, D.C., December 2006, available at http://www.aeronautics.nasa.gov/releases/national_aeronautics_rd_policy_dec_2006.pdf.

2

National Science and Technology Council, National Plan for Aeronautics Research and Development and Related Infrastructure, Office of Science and Technology Policy, Executive Office of the President, Washington, D.C., December 2007, available at http://www.aeronautics.nasa.gov/releases/aero_rd_plan_final_21_dec_2007.pdf.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 49
4 Key Findings and Recommendations Advancing the state of aviation safety is central to the mission of the National Aeronautics and Space Admin - istration (NASA) and its Aeronautics Research Mission Directorate (ARMD). In keeping with direction from the National Aeronautics Research and Development Policy1 (referred to as the “National Policy”) and the National Plan for Aeronautics Research and Development and Related Infrastructure )2 (referred to as the “National Plan”), ARMD’s emphasis is on pursuing what it describes as “long-term, cutting-edge research that expands the boundar- ies of aeronautical knowledge for the benefit of the broad aeronautics community.” In particular, ARMD is charged with undertaking “foundational” research that consists of basic research and developing a strong aeronautics knowledge and technology base to overcome barriers to technological progress in aviation. Much of ARMD’s safety-related research is aimed at predicting and preventing safety hazards in an increas - ingly diverse and complex Next Generation Air Transportation System (NextGen). Underlying this interest is concern that as NextGen capabilities are introduced and the national airspace becomes more heavily used by a diversity of aircraft, new safety hazards will arise that require innovative solutions through research. yet, not only must these future safety hazards be predicted and prevented with NASA’s help, but the agency also will continue to be called on to find solutions to safety problems arising in the current system. While this system performs at very high levels of safety, there is an expectation that safety advancements will continue, both in the near and longer terms. Congress requested this review of NASA’s aviation safety-related research programs. This chapter presents the main findings of the review as they relate to the four questions in the congressional request. Some of the findings go beyond these specific queries. For instance, in assessing NASA’s safety research coordination and transitioning activities, consideration is given to the connections established with industry, academia, and the aviation commu - nity generally as well as those forged with the Federal Aviation Administration (FAA) and other federal agencies. Because NASA’s aeronautics research constitutes a key component of the nation’s aviation research enterprise, its safety-related activities warrant consideration within this larger context. 1 National Science and Technology Council, National Aeronautics Research and Development Policy, Office of Science and Technology Policy, Executive Office of the President, Washington, D.C., December 2006, available at http://www.aeronautics.nasa.gov/releases/ national_aeronautics_rd_policy_dec_2006.pdf. 2 National Science and Technology Council, National Plan for Aeronautics Research and Development and Related Infrastructure, Office of Science and Technology Policy, Executive Office of the President, Washington, D.C., December 2007, available at http://www.aeronautics. nasa.gov/releases/aero_rd_plan_final_21_dec_2007.pdf. 9

OCR for page 49
0 ADVANCING AERONAUTICAL SAFETY The findings indicate that NASA’s aeronautics research enterprise has made, and continues to make, valuable contributions to aviation system safety, but it is falling short in some key respects. To address these shortcomings, the committee offers several recommendations at the conclusion of this chapter. KEY FINDINgS Do NASA’s Safety-Related Research Programs Have Well-Defined, Prioritized, and Appropriate Research Objectives? The National Plan identifies a series of fundamental research challenges that are top national priorities and are intended to provide strategic guidance for all federal civil aeronautics research and development (R&D). In accordance with the plan’s challenges for safety research, ARMD’s Aviation Safety Program has established research projects to address the following concerns: • New Operations, • Flight In or Around Hazardous Conditions, • Loss of Control, • Durable Aircraft Structures and Systems, • On-Board System Failures and Faults, and • Analyzing Complex Systems for Safety. As guideposts for programming safety research, these appear to be worthwhile objectives, consistent with the challenges identified in the National Plan and reflective of current and emerging safety issues. Whether the research being undertaken by ARMD to further these six objectives is sufficiently defined, prioritized, and appro - priate, however, is a separate matter. Because there are many aspects to each of the six research objectives, ARMD could choose to pursue any number of potential research topics commensurate with each. To make the best use of its resources requires that ARMD carefully examine the research needs associated with each safety objective and determine where it can contribute the most to meeting these needs in light of its own research competencies and work being undertaken elsewhere. The committee, therefore, expected to find a research prioritization process that is deliberate and well informedsupported by empirical analyses and advice from expertsand accompanied by a well-documented rationale for the program content. In not finding a prioritization process resembling this ideal, the committee struggled in understanding why ARMD safety research focuses on certain issues and topics while paying little, if any, attention to others. The committee observed, for instance, that much of the research addressing “loss of control” centers on automation by seeking to advance the state of the art of adaptive control systems for aircraft. The basis for this research interest—as opposed to research on other topics relevant to loss of control, such as human interactions with automation—was neither obvious nor well explained. More generally, the committee observed an emphasis throughout the research programs on safety issues that apply to commercial air transport but relatively little work having direct pertinence to unmanned aircraft or general aviation, even though the former represents a potential vehicle configuration in NextGen and the latter has long accounted for the majority of fatal aviation crashes. By not having a defensible, analytically based process for prioritizing its safety research, ARMD could not justify, in a convincing manner, much of the content of its research programs. Thus, in not having access to such an independent assessment of safety research needs, the committee could not determine whether ARMD’s safety research programs are prioritized to make the best use of available resources and neither can ARMD. Finding 1: NASA needs a more objective process for prioritizing safety research. While the objec- tives of ARMD’s Aviation Safety Program are worthy guideposts for safety research, ARMD lacks a well-founded process for prioritizing the research needs associated with each objective, and thus for ensuring that its research is well aligned with meeting critical national aviation safety needs.

OCR for page 49
 KEY FINDINGS AND RECOMMENDATIONS Although not possessing the qualities defined above, ARMD does employ a process for prioritizing and programming its safety research in accordance with its safety objectives. Based on the committee’s review of the research content and ARMD’s own description of its programming methods, this process appears to be driven largely by ARMD’s interest in employing existing personnel and assets at the NASA research centers. Those safety objectives that map well with ongoing research activities and with these internal interests are generally given priority in the programming of research and allocation of resources. One example (discussed in Chapter 3) that illustrates this observation is the selection of research challenges in the Aircraft Aging and Durability Project that align with “NASA’s core capabilities.”3 This internally driven process is undoubtedly a manifestation of a number of constraints and demands on ARMD, which the committee could not fully explore. NASA’s aeronautics research enterprise has experienced budgetary retraction for more than a decade, during which time it has undergone multiple reorganizations and changes in leadership, mission, and goals. These factors, among others, may have contributed to a climate that emphasizes the protection and preservation of existing research activities, personnel, and assets. Whether this method of research programming leads to a marginally or fundamentally different safety research portfolio than would have emerged using a more objective process cannot be readily determined. It is self-evident, however, that any prioritization process driven by an organization’s existing interests will risk neglecting new and emerging safety needs that are not well aligned with the status quo. Finding 2: Internal interests are overemphasized in the programming of safety research. ARMD gives undue weight to research that aligns well with its existing activities, personnel, and assets rather than the results of critical evaluations of current and emerging aviation safety needs. Have Resources Been Allocated Appropriately to Research Objectives? ARMD currently has research competencies that are recognized nationally and internationally in critical safety areas, such as icing research. While these existing nodes of expertise need to be recognized and their critical mass sustained, they risk being neglected as research funding is spread widely to preserve all of ARMD’s research capabilities, including those that are no longer unique to NASA or of high safety relevance. Indeed, the committee suspects ARMD has already lost this critical mass in the important safety area of human factors. yet, even as ARMD seeks to retain and strengthen its core safety research competencies, it must give sufficient attention to investing in the research expertise that will be needed to address new and emerging safety issues. ARMD recognizes the importance of such forward-looking investments, as evidenced by its efforts to expand and strengthen its expertise in software verification and validation (V&V). In relying too much on program budgetary increases to fund such investments, however, ARMD may be seriously hampered in its ability to make such criti - cal investments in its workforce and facilities. To acquire the needed expertise going forward, ARMD will almost certainly need to make many difficult decisions about the allocation of resources among its existing facilities and program areas, requiring that some capabilities be eliminated and others substantially scaled back. Not having in place an objective and well-informed means of assessing safety needs and priorities makes building the case for such resource investments and realignments even more difficult. Finding 3: Too few resources are devoted to sustaining and acquiring critical safety research capa - bilities. Continued emphasis on preserving existing research expertise and assets risks degradation of ARMD’s core safety research strengths and the prolonged neglect of competencies required to address new and emerging safety issues. In light of ARMD’s emphasis on advanced and long-range research, the committee is surprised to find few programmatic means and resources set aside for fostering exploratory and innovative thinking on ways to solve 3 NASA, Aviation Safety Program Aircraft Aging and Durability Project Technical Plan Summary, Washington, D.C., available at http://www. aeronautics.nasa.gov/nra_pdf/aad_technical_plan_c1.pdf, p. 2.

OCR for page 49
2 ADVANCING AERONAUTICAL SAFETY safety problems. Having mechanisms to invite such innovation would seem to be an important aspect of a research enterprise intent on engaging in “cutting-edge research that expands the boundaries of aeronautical knowledge.” Even though funding such research may involve greater uncertainty about expected payoffs, taking such calculated risks may be warranted in cases where research investments to solve safety problems have reached the point of diminishing returns and for which breakthrough insights and technologies are needed. Most of ARMD’s aviation safety work is performed by agency personnel and contractors at NASA research centers. Less than 15 percent of the funding for the Aviation Safety Program goes to outside researchers through the NASA Research Announcements (NRA). For the most part, ARMD uses the NRA program to sponsor external research that is closely aligned with its existing internal projects, and thus to make incremental contributions to them. Used in this way, NRA’s provide little incentive or latitude for researchers from academia, industry, and independent research institutesas well as from within NASAto propose and explore new and innovative ideas to enhance safety. Moreover, ARMD does not have any other formal mechanisms in place to support such exploratory research and innovation. Finding 4: Too few resources and programs are devoted to stimulating innovation. ARMD lacks the structure to elicit, explore, and develop innovative ideas to advance aviation safety. Are the Programs Properly Coordinated with the Safety Research Programs of FAA and Other Relevant Federal Agencies? At the agency level, NASA, FAA, and other federal agencies collaborate in developing the National Plan, which lays out the fundamental safety challenges that are a priority for the entire federal civil aviation research enterprise. At the program level, ARMD and its Aviation Safety Program also have strong connections to the Joint Planning and Development Office (JPDO), enabling a stronger understanding of research needs associated with NextGen and NASA’s role in meeting them. At the project level, NASA and FAA coordinate during the conduct of safety research, often quite effectively and with many positive interactions as evidenced by several of the projects reviewed by the committee. Indeed, an outstanding example of this collaboration is the work addressing the issue of high ice water content (HIWC) hazards. Such varied means of coordination across federal agencies are commendable and critical. yet, while much of NASA’s safety-related research is relevant to the needs of the FAA, much of it also pertains to the aviation community more generally. In this regard, NASA’s connections must extend beyond the FAA to include industry and academia. The interconnectedness and complexity of the aviation system demands that NASA seek the broadest possible cooperation and coordination of all parties involved. Indeed, the committee finds that ARMD has continuing involvement and interactions with a broad array of safety-related industry-led groups, such as the Commercial Aviation Safety Team (CAST), the Aviation Safety Information Analysis and Sharing system (ASIAS), and many discipline- and problem-oriented groups, such as the Joint Aircraft Survivability Program and Joint Council on Aging Aircraft. While it is difficult to know the intensity of these interactions based on NASA briefings and committee reviews of planning documents, there seem to be many avenues for NASA interaction and coordination across the aviation community. One area where greater inclusion is desirable is in seeking external reviews of ARMD safety research activi - ties. Aviation Safety Program activities are currently reviewed on a periodic basis by experts from the FAA and other federal agencies. Such agency reviews and consultations are vital to ensuring the research is compatible with the work going on elsewhere in the federal government and is relevant to the operational and regulatory needs of federal agencies. Limiting these external reviews to experts from federal agencies, however, may cause ARMD to miss out on opportunities to gain additional perspective from industry and academia and to inform outside percep - tions of the program’s capabilities and priorities. There is a general recognition within the research community that external reviews foster higher-quality research. yet, it is important that the reviews be conducted not only at the end stages of the research but on a continuing basis, from the beginning of the work to its completion and transitioning. Seeking such advice early in the research process is important because this is when research managers have the greatest latitude to reshape

OCR for page 49
 KEY FINDINGS AND RECOMMENDATIONS the program. Interim reviews as the work progresses are helpful for ensuring that the work remains of high qual - ity and retains safety relevance. If the research holds sufficient promise, some of the reviewers themselves may wish to partner in the work; after the work is completed, the reviewers may become early users and champions of the results. There are a variety of mechanisms for obtaining such continuous input, such as special project review com - mittees, advisory committees, ad hoc review teams, and individual reviews by recognized experts. Because a good portion of ARMD’s research is not intended to be near term and highly applied in nature, it would be important to ensure that external advisors recognize this and consist of some individuals with an understanding of broader safety-related trends and technology development processes. ARMD sponsors many activities aimed at information dissemination and exchange. The Aviation Safety Program convenes an annual Safety Technical Conference and hosts other outreach activities, such as the Air- space Systems Program’s Technical Exchange Meeting and NextGen Workshops. These forums provide valuable opportunities for researchers to present results and progress and for the exchange of research ideas and technical information. While the kind of external reviews described above would have a different purpose than these outreach activities, they would be highly complementary. Finding 5: Connections with the FAA, other federal agencies, and the aviation community are varied but not deep. NASA and the FAA coordinate in the planning and conduct of safety research, and many mechanisms exist for interacting and exchanging information with other federal agencies, academia, and industry. These connections could be deepened through more inclusive and sustained reviews of NASA safety research by such outside experts. Safety research is the main mission of ARMD’s Aviation Safety Program. While having a program that is dedicated to aviation safety is important, a single program cannot be expected to cover the full gamut of aviation safety issues. In the large, evolving, and complex aviation system, safety concerns can arise from interactions among system elements that may be considered safe on their own but not in combination, such as interactions among aircraft and airspace technologies and procedures. Under these circumstances, one would expect to find a significant amount of safety-related research being planned and undertaken in the other ARMD programs, often in collaboration with the Aviation Safety Program. With a few exceptions, however, ARMD’s safety research programs were presented to the committee as discrete activities of the Aviation Safety Program, the Fundamental Aeronautics Program, and the Airspace Systems Program. yet in reviewing each of these research programs, it proved difficult to identify the full array of safety-related research in the Fundamental Aeronautics Program and the Airspace Systems Program. One reason for this difficulty is that the research in these programs is often characterized as focusing on other objectives, such as increasing operational efficiency and developing innovative aircraft designs, structures, and materials. In such cases, it appears as though safety is treated more like a constraint than an opportunity—that is, safety performance is viewed as something the research cannot diminish, rather than something that it can improve. Given the complexity and breadth of the aviation safety challenge, it is troubling to find such “stove-piping,” or compartmentalization, within ARMD. Even within the Aviation Safety Program itself, however, the committee observes this phenomenon, such as the relatively few collaborative activities occurring across the related objective areas of New Operations and Loss of Control and across the Aircraft Aging and Durability and Integrated Vehicle Health Management projects. Finding 6: Internal coordination of and collaboration on safety research need improvement . Within ARMD, there is stove-piping of research that risks system-level safety solutions not being explored and safety hazards not being addressed that arise from interactions among aviation system elements.

OCR for page 49
 ADVANCING AERONAUTICAL SAFETY Do Suitable Mechanisms Exist for Transitioning the Research Results from the Programs into Operational Technologies and Procedures and Certification Activities in a Timely Manner? Exploiting the opportunities described above for coordination and collaboration with the FAA and industry can be critical to the successful transitioning of research results to follow-on stages of development or application. The aforementioned HIWC activity exemplifies how such coordination and collaboration can foster the transfer of research results to the field. In reviewing the Aviation Safety Program’s research portfolios, the committee observed many instances of researchers partnering with industry through use of Space Act agreements and other mechanisms designed to build such connections with end-users and ensure the research is relevant and remains on a productive course. At the same time, however, there are many practical challenges to building and maintaining such connections in light of ARMD’s emphasis on longer-range research and the demands of the FAA and industry for relatively mature and well-tested products and procedures. Because much of ARMD’s safety work consists of fundamental and long-term research, a high degree of operational applicability cannot be a metric for judging whether the research should be undertaken, nor can there be a strong emphasis on definitive end products and transition planning. yet, for some of the safety-relevant research being pursued by ARMD, the resulting technologies and process are intended to support FAA operational needs and have application, in which case transition planning is important. A prominent example is ARMD’s work on air traffic management (ATM) technologies and procedures, such as trajectory-based operations and weather avoid - ance. The utility of NASA’s work in this area has long been questioned because of the high levels of safety assur- ance demanded by the FAA when making changes to ATM technologies and procedures. The committee observes, however, that NASA and the FAA recognize these challenges and are actively seeking to overcome them, by for instance, coordinating special research transition teams intent on identifying and addressing safety implications much earlier in the research process. Even for research that is longer-term in nature, NASA must have some understanding of eventual implementa - tion challenges in order to judge the merits of the work. The Aviation Safety Program’s efforts on advancing the state of the art of adaptive systems for loss of control is a clear example of this need. This is because achieving an accepted level of safety assurance required for certification of the non-deterministic software used in these systems promises to be a difficult and prolonged undertaking. In this case, the implementation challenge itself may be more complicated and daunting than the development of the control technologies themselves. The committee believes the kind of external reviews espoused above are likely to bring such implementa - tion challenges to the forefront, compelling early attention by those programming and engaging in the research. By applying foresight in mapping out the implementation challenges, researchers can better assess the long-term practicality of the research in relation to other research options addressing the same safety objective. Doing so will not only yield insight into these implementation challenges, but may suggest areas where ARMD research can be helpful in overcoming them, such as in developing improved safety analysis and approval methods. Finding 7: Demands for safety-assured technologies and procedures can conflict with NASA’s empha - sis on long-range, foundational research. ARMD exploits many mechanisms to assist in further- ing the technologies and procedures developed through its research; however, safety assurance and approval requirements can present vexing implementation challenges. In light of these challenges, some of ARMD’s safety-related research would appear to have very limited prospects for even - tual implementationa risk that deserves more explicit consideration when ARMD programs its research. RECOMMENDATIONS Recommendation 1: The Aeronautics Research Mission Directorate (ARMD) should adopt a more fully informed, empirical, and documented process for identifying and prioritizing safety research needs for use in guiding its aeronautics research and development programming and investments in research expertise and capacity. A central

OCR for page 49
 KEY FINDINGS AND RECOMMENDATIONS element of this process should be the development of comprehensive aviation safety needs assessments. These assessments should be: • Made objectively, independent of ARMD’s existing research expertise, assets, and resource requirements and constraints. • Undertaken in close coordination with the Federal Aviation Administration and other relevant federal agen - cies and in consultation with industry, academia, other safety-related organizations, and considering the relevant aviation safety data and literature. • Cognizant of the safety needs being researched elsewhere in government, industry, and academia to know where critical gaps in research coverage may exist. Recommendation 2: The Aeronautics Research Mission Directorate should establish programmatic means for encouraging more exploratory research on innovative ideas to improve aviation safety. The program should elicit and develop the promising ideas of researchers from industry, academia, other government agencies, and NASA. Recommendation 3: The Aeronautics Research Mission Directorate’s safety-related research activities should be subject to regular reviews by outside experts from the Federal Aviation Administration and other government agencies, industry, independent research institutes, and universities. These reviews, which will help in ensuring continued safety relevance, quality, implementation challenges, and successful transitioning, should be undertaken during the formative stages of the research, interim phases, and as the work is being completed. The reviews should have a prominent role in informing research programming decisions. Recommendation 4: The Aeronautics Research Mission Directorate should develop and implement processes that will lead to more coordination and collaboration in the planning and conduct of safety research both within the Aviation Safety Program and across all its aeronautics research programs.

OCR for page 49