Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Executive Summary The Air Force's newest combat aircraft, the F-22, is designed principally for offensive counter-air missions. Advanced technologies, including stealth, are intended to enable it to penetrate deeply into hostile airspace and shoot down threatening aircraft before they can detect the F-22. If detected, the F-22 has the countermeasures, speed, and maneuverability that should minimize the likelihood of its being hit. The National Defense Authorization Act for fiscal year 1995 directed the Secretary of Defense to request the National Research Council to study the desirability of waiving the live fire tests that are required by law for the F-22. The Committee on the Study of Live Fire Survivability Testing of the F-22 Aircraft was formed by the National Research Council to conduct the study. The committee began its work in December 1994. Several data gathering meetings were held in Washington, D.C., one was at Wright-Patterson Air Force Base, Dayton, Ohio, which is the location of the F-22 System Program Of lice and the Wright Laboratory, where many Air Force live fire tests are conducted, and one was at the Naval Air Warfare Center located at China Lake, California, where the Navy conducts similar tests. These meetings exposed the committee to the filet spectrum of views involving live fire testing of fighter aircraft. The work of a previous National Research Council committee, reported in the 1993 publication Vulnerability Assessment of Aircraff: A Review of the Department of Defense Live Fire Test and Evaluation Program, was invaluable to the present committee. The previous committee confirmed that, unless waived by the Secretary of Defense because of urlreasonable expense and impracticality, live . ~ According to the Air Combat Command, which is the operational user of the F-22, there are two kinds of counter-air missions. The first, known as offensive counter air, is to "penetrate deep into heavily defended hostile airspace and destroy threat capability." The second, defensive counter air, is to "detect, identify, intercept, and destroy Meat aircraft penetrating friendly airspace." (W.S. Hinton, "Threat, Mission, and Operational Requirements for the F-22," presentation to the Committee on the Study of Live Fire Survivability Testing ofthe F-22 Aircraft, Washington, D.C., December 21, 1994.) For the purpose of this report the committee uses the definition of offensive counter air that involves only air-to-air engagements.
2 . Live Fire Testing of the F-22 fire tests known as full-up, full-scale tests are required for major weapon systems like the F-22. Full-up, full-scare tests would subject an F-22 to live fire in its combat configuration, including on-board ordnance and fuel. These tests would use munitions likely to be encountered by the F-22 in combat. There is currently no doubt within the Department of Defense about this requirement or the need for a waiver if such tests are not to be performed. If the tests are waived, the law permits the Secretary of Defense to allow Instead live fire tests against components, subsystems, and subassemblies, together with design analyses, modeling and simulation, and analysis of combat data. Along with any waiver, the Secretary must report how system survivability will be evaluated and assess possible alternatives. Circumstances associated with a waiver for the F-22 are unique in one respect. Normally, if a waiver is to be granted, the Secretary of Defense must act before a weapon system enters the phase of acquisition known as engineering and manufacturing development. The decision point for this phase is called Milestone Il. Milestone I} for the F-22 occurred in 1991 without a waiver being requested. In 1993, the Department of Defense, along with developing an alternative test plan, requested a change to the law that would permit the F-22 to be granted a retroactive waiver. That request prompted members of Confess to call for this study. A, The remainder of this summary contains (a) an overview of the principal findings that resulted from this study, followed by (b) a more detailed presentation of the committee's major conclusions arid recommendations. PRINCIPAL FINDINGS The committee's principal findings concern three topics: (1) practicality and cost-benefit issues related to live fire testing of the F-22, (2) sufficiency of the current vulnerability assessment program for the F-22, and (3) vulnerability assessment tools. Practicality and Cost-BeneE~t The committee believes that live fire testing can be conducted at four levels of realism for the intended mission.2 From lowest to highest, these four levels are (~) hardware simulations or mock-ups of production systems; (2) full-scale 2 In the case of the F-22, the primary mission is offensive counter air, in which the Air Force expects very few encounters to result in enemy missiles or guns being fired at the aircraft.
Executive Summary 3 components, subsystems, or major subassemblies representative of production items; (3) a complete production aircraft not loaded with live ordnance or fuel and (4) a full-up, full-scale ground test. ~&1_ _ 1_- _1_ _ _` 1 __ _1 ~_ ~ ~ _ ~ Even one Agnes level Does not adequately simulate actual flight conditions, and testing the virtually infinite number of threat-target interactions is out of the question. Destructive testing (as opposed to nondestructive testing) of a complete aircraft at the full-scare level yields extremely low confidence factors due to the small number of trials possible. The opinion of most experts with whom the committee met, and the committee's opinion, is that full-scale testing is much less likely to provide useful information than are component, subsystem, and subassembly tests. The committee carefully weighed arguments on both sides of the issue involving tests of a complete, fiull-scaTe aircraft (e.g., the chances that such full- scale tests will reveal "unknown unknowns"~. The committee was persuaded that, for a system like the F-22, a well conceived, incremental build-up of tests that proceed from the component level to the subassembly or large assembly levels made the most sense. The combination of lack of realism in test conditions, the difficulty of obtaining a sufficient number of teals, and expert opinion all support the conclusion that completely realistic, destructive, full-up, fi~-scale testing of the F-22 is not practical and offers low benefits for the costs. These views formed the basis of the committee's position regarding the waiver requested for the F-22. The committee's recommendation on the waiver appears below. Sufficiency The committee evaluated the current vulnerability assessment program for the F-22. The Air Force and its contractors have incorporated many features in the ~ ^^ 11 ~ `1 _ ~ -11 ~ ~1 · id. ~ ~ ~ ·~ -. r r-~z Design mat Wll1 reduce me a~rcrarc-s vulneranlllty te.g., a largely multiple load path structural design, filet tank inerting, and much subsystem redundancy). The design is complemented by a strong vulnerability analysis and live fire test program. However, the committee has some concerns about the program. For example, no live fire tests are currently planned to assess damage done by direct hits on important aft structural members; the current vulnerability specifications do not include the effects of on-board ordnance; and the analysis does not properly account for the flammable properties of the hydraulic and cooling fluids. The committee believes that several additional tests and analyses are needed to strengthen the program. Specific recommended actions are discussed later. Vulnerability assessment is a complicated matter. As part of its evaluation, the committee carefi~ly considered the need for continued vulnerability testing of
4 Live Fire Testing of the F-22 the F-22 beyond the currently planned tests that support engineering and manufacturing development and initial production. The committee held detailed discussions with the vulnerability and lethality assessment communities. In particular, the committee considered the Navy's assessment methodology for fighters. Large assembly testing of variants of the Navy's F-] ~ has been conducted and is planned, even though the Navy vulnerability assessment team does not expect to discover unanticipated outcomes. It is possible that additional data could be obtained from similar live fire testing of the major parts of an F-22 and result in revisions to the aircraft design to reduce furler its vulnerability. The committee recognizes arid accepts that test assets may not become available Critic after production begins and are not likely to influence the production configuration of the counter-air version of the F-22. The Navy's rationale for larger scale testing was persuasive: something is always learned, vulnerability assessment tools are evaluated and improved, and the test base cannot be allowed to wither. After much deliberation, the committee agreed that the expeditious conduct of tests against large assemblies made sense for the F-22. This fighter will be in the inventory for decades and Carl be expected to evolve, to include other missions (e.g., air-to-surface) and new configurations. Testing of its large assemblies could also verify techniques for repairing battle damage to the F-22's new composite materials and systems. Vulnerability Assessment Tools To be successful, vulnerability assessment requires much mutual support between documentation, data bases, models, and testing. The committee's review of the F-22 vulnerability assessment program indicated that significant improvements are needed in several of the tools (i.e., documentation, data bases, and models) that complement live fire testing. Specific conclusions and recommendations relating to these tools are provided below. CONCLUSIONS AND RECOMMENDATIONS The committee's major conclusions and recommendations follow. They are directed at the specific matters in the legislation that requested this study. The committee's principal recommendation, which appears immediately below, requires action by Congress. The numbered recommendations that follow require action by the Department of Defense. Specific authorization arid appropriation by Congress may be necessary to implement some of the numbered recommendations.
Executive Summary Desirability of Waiver for the F-22 Tests Principal Recommendation. Permit a waiver of the fi~-up, fi~-scale, live fire tests required by law for the F-22. The committee believes that such tests are impractical and offer low benefits for the costs. Recommendation I. Interpret a waiver as reinforcing the need to conduct robust live fire tests of the F-22 that build incrementally from the component level to the subassembly or large assembly levels. (Recommendations to strengthen the current test program appear below.) Changed Circumstances Since Milestone I} 5 Nothing specific to the F-22 program appears to have changed since Milestone IT in 1991 that would have prompted the Air Force to request a waiver. However, there appears to be a changed view of what is required to fulfill the intent of the live fire test law as it was interpreted within the Department of Defense. Although there was some consideration of a request for a waiver in 1991, it was not until after the 1993 report by the National Research Council that the Air Force appeared to accept filthy the fact that a waiver was necessary. The Air Force then instituted the current live fire test program, which does not include a filll-up, full-scale test, and the waiver request followed. Affordability and Cost-Benefit The committee's conclusions relative to the impracticality and Tow benefits for the costs of fillI-up, full-scale, live fire testing were provided above. The committee was also asked to address affordability. The committee defines an "affordable" activity as one within budgetary constraints or attainable budgets. A full-up, fulI-scale test of the F-22 would cost approximately $250 million (then-year dollars) above the currently planned program. Thus, to perform the test, either more funding would need to be authorized, or the test would have to displace $250 million of currently funded activities in the program. There is an argument that even $250 million is only a small percentage (less than 0.5 percent) of total F-22 program costs, and therefore cost should not be a determining factor. The co~runittee was not able to consider fully or challenge the prioritization of items within the current F-22 program budget. However, the committee's judgment (discussed earlier) is that the benefits of full-up, fi~-scaTe
6 Live Fire Testing of the F-22 testing of the F-22 are Tow relative to the costs. Even if $250 million were provided for additional vulnerability assessment of the F-22, the committee would not support using the funds for full-up, full-scale testing. The co~x~nittee concludes that the affordability of live fire tests is not the matter of foremost relevance; cost-benefit is most relevant. Affordability only becomes relevant if the benefits relative to the costs of whatever tests are being considered are commensurate with the benefits relative to the costs of other alternatives. The committee notes that its judgments regarding the costs and benefits of fi~-up, full-scale testing were reached in the absence of a mature methodology for assessing benefits relative to costs. The committee reviewed cost-benefit methodologies related to incremental live fire testing, conclusions and recommendations concerning these methodologies are covered next. Despite recommendations made by the previous National Research Council committee, this committee regrets the lack of more progress in developing a cost-benefit methodology for determining the return on investment of successive levels of live fire tests. The methodologies briefed to the committee during the study are immature and appear to address suboptunal measures of benefit. The committee is leery of reliance on methodologies that use an overly simple construct of the F-22's fixture to make judgments about how far to go with live fire testing. A broader analytical framework could elevate the importance of reduced F-22 vulnerability over the long term and might enhance the benefits relative to the costs of given levels of testing. Recommendation 2. Continue Department of Defense efforts to develop viable cost-benefit methodologies for planning the extent of live fire testing. Pursue methodologies to examine cost-benefit issues in the light of frameworks that take a broad view of how the future may develop for weapon systems like the F-22. Sufficiency of Tests Planned for the F-22 The committee's evaluation of the current vulnerability assessment program considered threat-related assumptions as well as the attendant ~ruInerabilities and analyses and tests associated with the F-22's major subsystems (i.e., structure and integral fuel tanks, fuel system and dry bays, flight control and auxiliary systems, weapons bay and on-board ordnance, engines, flight crew, and fire protection systems). The major results of the committee's evaluation are summarized below. The committee accepts the threat environment defined for the current mission of the F-22. Threat replication in the test program is reasonable. However, for some classes of anti-air missile warheads (e.g., annular or focused blast
Executive Summary 7 fragmentation), the kill mechanism that involves dense multiple fragment impacts may be important for the F-22. Recommendation 3. Consider, in future analyses and tests, the kill mechanism that involves dense multiple fragment impacts. Overall, the committee believes that the vulnerability assessment program for the F-22, given its current counter-air mission, is sufficiently realistic to support the requested waiver. The following specific actions are recommended by the committee to strengthen the program as the F-22 proceeds with engineering and manufacturing development and initial production. Recommendation 4a. Conduct additional live fire testing to determine the damage that can be expected from a hit in the Frame 6 aft boom attachment area. Determine the most critical shot lines for this testing. Recommendation 4b. Expand analyses to predict damage sizes and residual strengths of the aft boom, Frame 6, and horizontal tall pivot shafts after being hit by 30mm high-explosive incendiary rounds. Also, determine the risk of aircraft loss should it be found that loss of a horizontal tad] is possible. Recommendation 4c. Conduct further analysis of the aft fuel tank (A-~) prior to the conduct of Test 4D.3 Focus this analysis on dete~-~ining the adequacy of the test specimen, with particular emphasis on its ability to simulate accurately the reaction of the entire Ink. Recommendation 4~. Make the operational community fully aware that a fuel ingestion risk to the aircraft exists at a fuel state higher than 60 percent. (This risk arises because the file! tanks next to the engine inlets are not empty at fuel states above 60 percent; thus, a puncture could lead to fuel ingestion by an engine and potential engine failure.) Recommendation 4e. Conduct the tests and analyses, proposed by the F-22 System Program Office, on the flammability of coolant and other fluids and the attendant vulnerability of the aircraft. 3 Test 4D was planned for August 1995, when a section of the fuel tank between Frames 5 and 6 was to be filled with water, externally loaded, and hit with a 30mm high-explosive incendiary round.
8 Live Fire Testing of the F-22 Recommendation 4f. Undertake the analysis and test, proposed by the System Program Office, of ablative materials in the weapons bay. Also, conduct further analysis of the tradeoffs associated with additional ordnance protection or defensive measures. Recommendation 4g. Fund the Joint Technical Coordinating Group on Aircraft Survivability and the Joint Live Fire Test Program to assure the completeness of data on the vuInerabilities of on-board ordnance. Recommendation 4h. Fund the proposed Joint Live Fire testing of FIl9 engine components to alleviate the paucity of testing against those components. Recommendation 4i. Emphasize continuing efforts by the F-22 System Program Office and the Joint Technical Coordinating Group on Aircraft Survivability to develop improved methodologies for reducing flight crew vulnerability. Recommendation 4j. Use the prototype air vehicle fuselage in Test 6A in lieu of a mock-up. (The Air Force has considered using this fuselage for Test 6A, which will examine the synergistic effects of pressurized coolant lines and cooled avionics modules and the adjacent powered electrical wiring in the F-22 forward fuselage lower avionics bays.) In addition, the committee recommends that the Air Force begin planning for expeditious vulnerability assessment testing of the F-22 similar to that being conducted or planned for variants of the Navy's F-]8. Recommendation 5a. Use large subassemblies from production- representative hardware (e.g., a damaged aircraft or other source) in these tests. Recommendation Sb. Provide these assets, as soon as they become available, to the vulnerability assessment community for the conduct of live fire tests. Recommendation 5c. Direct the tests at (a) verifying predictions from the current F-22 live fire test program and the models used, and (b) testing the effects on overall F-22 vulnerability assessment brought about by configuration and mission changes. Also, use the tests to
Executive Summary verify techniques for repairing battle damage to the F-22's new composite materials and systems. Other Recommendations VuinerabiZity Requirements 9 Given the F-22's counter-a~r mission, the Air Force placed a premium on achieving high survivability through reduced aircraft susceptibility. Reduction of vulnerability in the design, although important, was deemed less significant. The user (Air Combat Command) did not specify quantitative vulnerability requirements for the F-22. Instead, contract specifications for vulnerability were developed bv the Svstem Program Office in coordination with the Alar to e~t~hli~h · . ~ .. ~ ~ ~ ~ . 1 · r ~ · , · · , · ~ . ~ a basis tor design opum~z~at~on and assess contractor performance. The existing vulnerability specifications and the live fire test program do not address fixture missions that the F-22 might be required to perform. The System Program Office (the developer) is currently planning for an air-to-surface mission using ordnance delivered from relatively high altitude. At some point in the fixture, lower altitude missions could be required. Recommendation 6. Reexamine expeditiously, for future F-22 missions (e.g., air-to-surface), the balance of requirements among susceptibility, vulnerability, and related performance parameters. Recommendation 7. Include in operational requirements for any new missions user validation of quantitative vulnerability requirements, and plan new live fire tests as necessary in response to those requirements. Vulnerability Assessment Tools The committee reviewed the documentation, data bases, and models that complement live fire testing for the F-22 arid other aircraft programs. Most of the documents (e.g., standards for vulnerability design) were found to be ~ O or more years old arid in need of updating and improvement. Recommendation 8. Update and improve expeditiously the various standards, handbooks, md design guides that are important to the aircraft vulnerability community.
10 Live Fire Testing of the F-22 Data bases play a distinct role in vulnerability assessment in that they form the institutional memory that bridges specific systems, prevents repetitious testing, arid avoids the mistakes of the past. Insufficient component tests have been accomplished to produce confidence in the data bases for existing components and materials; arid new composite materials, engines, stealth techniques, and other advances wait require additional testing. The committee believes Were is considerable need for expanded efforts to improve the data bases. · ~ · . · ~, . . ~- . . . ~ Recommendation 9. Direct the loins Technical Coordinating Group on Aircraft Survivability to define and plan a Joint Live Fire Test Program that will, over the next several years, produce sound vulnerability data bases; apply aggressive fielding to implement this program. Modeling is essential because the virtually infinite possibilities for threat-target interactions Remarry ways to extend the limited number of tests that can be conducted. Experimentation has been the main approach for obtaining data to empirically fit the models. The advanced modeling techniques used by the designers of nuclear weapons and the aerospace and automotive industries have not been exploited by the vulnerability community. For the F-22, the committee judges that a relatively large uncertainty is the ability to mode! the response of its composite materials. Also, valid large-scale models could provide an efficient means of making sound judgments without the need for expensive and repetitive live fire tests on large subassemblies. Recommendation lOa. Validate and accredit formally, by the Joint Technical Coordinating Group on Aircraft Survivability and the Joint Technical Coordinating Group on Munitions Effectiveness, the wInerability assessment models used by the Air Force and over services. Recommendation lOb. Improve the vulnerability models of the vulnerability community, and adopt these improvements for the F-22. Recommendation lOc. Explore the application of advanced methodologies currently being used by nuclear weapons designers and other industries. Recommendation lOd. Focus on ways to understand filthy the response of F-22 composite materials to ballistic damage, and develo and exercise analysis tools that can handle large-scale damage effects.
