National Academies Press: OpenBook

Live Fire Testing of the F-22 (1995)

Chapter: 2 ORIGIN OF TESTING REQUIREMENTS

« Previous: 1 INTRODUCTION
Suggested Citation:"2 ORIGIN OF TESTING REQUIREMENTS." National Research Council. 1995. Live Fire Testing of the F-22. Washington, DC: The National Academies Press. doi: 10.17226/4971.
×
Page 16
Suggested Citation:"2 ORIGIN OF TESTING REQUIREMENTS." National Research Council. 1995. Live Fire Testing of the F-22. Washington, DC: The National Academies Press. doi: 10.17226/4971.
×
Page 17
Suggested Citation:"2 ORIGIN OF TESTING REQUIREMENTS." National Research Council. 1995. Live Fire Testing of the F-22. Washington, DC: The National Academies Press. doi: 10.17226/4971.
×
Page 18
Suggested Citation:"2 ORIGIN OF TESTING REQUIREMENTS." National Research Council. 1995. Live Fire Testing of the F-22. Washington, DC: The National Academies Press. doi: 10.17226/4971.
×
Page 19
Suggested Citation:"2 ORIGIN OF TESTING REQUIREMENTS." National Research Council. 1995. Live Fire Testing of the F-22. Washington, DC: The National Academies Press. doi: 10.17226/4971.
×
Page 20
Suggested Citation:"2 ORIGIN OF TESTING REQUIREMENTS." National Research Council. 1995. Live Fire Testing of the F-22. Washington, DC: The National Academies Press. doi: 10.17226/4971.
×
Page 21
Suggested Citation:"2 ORIGIN OF TESTING REQUIREMENTS." National Research Council. 1995. Live Fire Testing of the F-22. Washington, DC: The National Academies Press. doi: 10.17226/4971.
×
Page 22
Suggested Citation:"2 ORIGIN OF TESTING REQUIREMENTS." National Research Council. 1995. Live Fire Testing of the F-22. Washington, DC: The National Academies Press. doi: 10.17226/4971.
×
Page 23
Suggested Citation:"2 ORIGIN OF TESTING REQUIREMENTS." National Research Council. 1995. Live Fire Testing of the F-22. Washington, DC: The National Academies Press. doi: 10.17226/4971.
×
Page 24
Suggested Citation:"2 ORIGIN OF TESTING REQUIREMENTS." National Research Council. 1995. Live Fire Testing of the F-22. Washington, DC: The National Academies Press. doi: 10.17226/4971.
×
Page 25
Suggested Citation:"2 ORIGIN OF TESTING REQUIREMENTS." National Research Council. 1995. Live Fire Testing of the F-22. Washington, DC: The National Academies Press. doi: 10.17226/4971.
×
Page 26
Suggested Citation:"2 ORIGIN OF TESTING REQUIREMENTS." National Research Council. 1995. Live Fire Testing of the F-22. Washington, DC: The National Academies Press. doi: 10.17226/4971.
×
Page 27
Suggested Citation:"2 ORIGIN OF TESTING REQUIREMENTS." National Research Council. 1995. Live Fire Testing of the F-22. Washington, DC: The National Academies Press. doi: 10.17226/4971.
×
Page 28
Suggested Citation:"2 ORIGIN OF TESTING REQUIREMENTS." National Research Council. 1995. Live Fire Testing of the F-22. Washington, DC: The National Academies Press. doi: 10.17226/4971.
×
Page 29
Suggested Citation:"2 ORIGIN OF TESTING REQUIREMENTS." National Research Council. 1995. Live Fire Testing of the F-22. Washington, DC: The National Academies Press. doi: 10.17226/4971.
×
Page 30

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

2 Origin of Testing Requirements This chapter discusses the legal requirements for live fire testing, the interpretations of those requirements within the Department of Defense (DoD), and the circumstances that led to the request for a retroactive waiver for the F-22. The chapter also addresses F-22 design requirements related to vulnerability. Finally, the committee offers some observations about the disposition of reco~runendations made by the previous National Research Council (NRC) committee that studied vulnerability assessment of aircraft (NRC, 19931. FRAMEWORK The ultimate objective of any combat system is to destroy or disable the enemy before being destroyed or disabled by that enemy. In meeting that objective, one consideration is for Me combat system to have some level of inherent battle damage tolerance. Thus, aircraft designers must give serious attention to reducing vulnerability as part of the overall aircraft design. Once a system enters the formal acquisition process, company designers receive contractual specifications from the system program of lice of the designated service. The system program office (the developer) bases its specifications on system requirements defined by the user. In establishing F-22 survivability requirements, the system Operational Requirements Document (ORD) was Me operative source (TAF, 1991) . The F-22 ORD reflects the Systems Threat Assessment Report (STAR) (FASTC, 1992), which defines the threat the F-22 is expected to face based on its planned mission. The user's emphasis in the ORD was on susceptibility rather Man vulnerability. Experience in the 1980s with Me Army's Bradley Fighting Vehicle led marry to conclude that survivability of equipment and personnel had not been adequately considered and tested. As a consequence, in fiscal year 1987 Congress amended Title 10 of the U.S. Code by adding the original live fire test (LFT) law (10 U.S.C. 2366, 1986), which mandated certain procedures in vulnerability testing. Authority that permits the Secretary of Defense to waive the application of the tests under certain defined circumstances was included in the law. (The current version of the EFT law is provided in Appendix B.) 16

Origin of Testing Requirements 1 17 An overview of the guidelines, opinions, and amendments pertinent to the original law appears below. This overview sets He context for considering the F-22 waiver currently requested by the DoD. F-22 LIVE FIRE TESTING REQUIREMENTS This section addresses the many documents that influence the F-22 live fire test program. Because the early documents were reviewed in detail by the previous NRC committee (NRC, ~ 993), much of the information below is taken from that committee's report. The Live Fire Test Law Requirements and Historical Interpretations The following historical information addresses the law's intent: The intent of the EFT law is to determine the inherent strengths and weaknesses of adversary, U.S., and allied weapon systems sufficientiv ~. . ~ ~ ~ ¢ --- A J early in one program to allow any design deficiency to be corrected. According to the FYI988-1989 DoD Authorization Act Conference Report, Congress intended that the Secretary of Defense implement the EFT law "in a mariner which encourages the conduct of fi~-up vulnerability and lethality tests trader realistic combat conditions, first at the sub-scale level as they are developed, arid later at the full-scale level mandated in the legislation" (U.S. Congress, 19881. (NRC, 1993) For purposes of this report, the present committee adopts the previous NRC committee's definitions of full-up and full-scare tests: . . law: A full-up test is defined as "a test conducted on a complete system or a partial system, with the full complement of fuel, ammunition, and hydraulic fluid carried by the system into combat." A full-scale or complete system test is defined as "a test conducted on the complete or total system, with or without the filll complement of fuel, ammunition, arid hydraulic fluid carried into combat." The previous committee left no doubt about the requirements of the EFT

18 Live Fire Testing of the F-22 Based upon the evidence gathered by the committee and its study of the law, the committee is unanimous in the opinion that the EFT law requires a full-scale, full-up aircraft to be tested, regardless of the outcome of the sub-scale tests, unless a waiver is granted. This committee agrees with the above opinion. In retrospect, it would appear that the EFT law should have been clear to everyone with little chance for misinterpretation. However, in the discussion that follows, it becomes obvious that serious misinterpretations alias occur, which were not fully corrected until approximately 1993. Without going into Me nuances of past interpretations and misinterpretations of the law resulting from guidance issued by the DoD, it is instructive to review the prior NRC committee's statements: tThe committee] believes that the definitions in the 1988 Guidelines and the guidance given in the 1989 Planning Guide are not sufficiently clear as to the law's requirement that full-scale, Let-up testing must be conducted. As a consequence of this misunderstanding, the Services have proceeded with sub-scale Live Fire Test programs on several weapon systems without making a provision for testing a complete system and without asking for a waiver because of the belief that no filll-scale, full-up testing was required if early tests on sub-scale targets showed no design weaknesses. The F-22 EFT program, among others, was covered in briefings to the prior NRC committee. Live fire testing of a ~-up, full-scale F-22 was not planned (NRC, 19931. The F-22 passed Milestone I] in mid-1991 and entered engineering and manufacturing development. No waiver from the law's requirements was granted by the Secretary of Defense before the F-22 entered this phase. Yet there is evidence to suggest that some in the Air Force knew, in 1991, that a waiver was required. In October 1991, Brigadier General James A. Fain, who was then Director of the F-22 System Program Office (SPO), sent a memorandum to the Office of the Secretary of the Air Force outlining F-22 live fire test alternatives. With respect to full-up testing of the system configured for combat, General Fain recommended "requesting a waiver to this requirement" (Fain, 1991; see below). In August 1992, James O'Bryon, the Deputy Director of Test arid Evaluation for Live Fire Testing in the Office of the Secretary of Defense, sent a memorandum to the Air Force Director of Test and Evaluation. With respect to the planned F-22 Live Fire Test and Evaluation Vulnerability Program, Mr. O'Bryon stated:

Origin of Testing Requirements Although in many respects [the ballistic] part of the proposed plan appears to be comprehensive and well thought out, in one respect at least it falls seriously short of the requirements of the law. . . . In June of last year, ~ held a series of meetings with then BG Fain, Director of the F-22 SPO. ~ left those meetings with the clear understanding that full-up testing would be included in the Live Fire Test Program . . . . ~ am concerned that, after more than a year, this most important inadequacy, the lack of full-up testing, has not been addressed. (O'Bryon, 1992) 19 The committee notes that Fain's memorandum to the Of lice of the Secretary of the Air Force preceded O'Bryon's memorandum by nearly a year, indicating that O'Bryon was unaware of Fain's waiver reco~runendation. The SPO's position is discussed further in a later section of this chapter. With respect to waivers, the previous NRC committee (NRC, ~ 993) recommended that: the Director, Test and Evaluation, formalize the waiver process by developing a r~sk-benefit assessment methodology that can be used uniformly to determine whether a full-scale, full-up test program for any particular aircraft is "unreasonably expensive and impractical." The methodology must also be applicable to the evaluation of the alternate Live Fire Test program for the sub-scale targets. the Secretary of Defense take measures to ensure (a) that the EFT&E tLive Fire Test & Evaluation] Guidelines are properly enforced by requiring either that covered systems be subjected to full- scale, fi~-up testing or that a waiver be obtained; (b) that any waiver be fully justified, (c) that the waiver process be uniformly applied, and (~) that no stigma be attached to the use of the waiver process. Recent Live Fire Test Guidelines and Interpretations A new set of DoD guidelines, issued January 27, 1994, superseded all previous editions. Revised definitions of full-up test and fi~-up system-level test are presented in those guidelines, as follows (Longuemare, 19941: Full-up Test. A vulnerability test conducted on a complete or partial system loaded or equipped with all dangerous materials (including flammables and explosives) that would normally be on board in

20 Live Fire Testing of the F-22 combat (configured for combat). All critical subsystems which could contribute to the test outcome must be operating (e.g., hydraulic and electrical power) under realistic conditions. . . . This testing alone may not satisfy 10 USC, Section 2366. FuR-up, System-Leve! Test. A EFT&E Strategy for a covered system, major munition program, or missile program, or covered product improvement program will include Full-up, System-level tests. The term "Full-up, System-leve! Test" is that testing that fully satisfies the statutory requirement for "realistic survivability testing" or "realistic lethality testing" as defined in Section 2366, Title 10, USC. On July 25, 1994, the Air Force published Instruction 99-105 to provide "guidance and procedures for the live fire test and evaluation of Air Force systems" (USAF, 1994~. The document states that it is "in compliance with EFT&E legislation." It also states: EFT&E is a sub-set of developmental test and evaluation (DT&E), the Air Force accomplishes it through a balanced program of test and analysis. Analysis must be an integral part of Me I,FT&E process because it is unreasonably expensive and impractical to test all possible combinations of threats, aircraft configurations, and environments. The Air Force must initiate the EFT&E program sufficiently early to allow the results to impact system design prior to fulI-rate production or major modification. The primary emphasis is on testing and analysis to identify and correct potential design flaws. The EFT&E program should provide an assessment of a system's vulnerability or lethality characteristics relative to the expected spectrum of battlefield threats. Regarding LFT&E waivers, the instruction states: The Secretary of Defense may waive EFT&E legislation requirements for covered systems, major munitions programs, and product improvements (that significantly affect vulnerability or lethality) to covered systems and major munitions programs where EFT&E would be unreasonably expensive and impractical. Request, process, and have a granted waiver before Milestone Il. The Air Force cannot grant EFT&E waivers after Milestone Il. except through legislative relief. An approved EFT&E waiver exempts a system, program, or product improvement from full-up, system-level testing (full scale, complete system configured for combat, equipped with all fluids, materials, and

Origin of Testing Requirements explosives). However, you must have an alternate plan to evaluate system vulnerability or lethality (such as combinations of analysis, component, sub-system, or sub-assembly testing, etc.) to fillfill the intent of the EFT&E legislation. 21 The committee believes that, with the exception of the requirement for full-up, full-scale testing, the requirements for waived EFT&E programs are no less stringent than for nonwaived programs. Recent Amendment to Waiver Provision of the Live Fire Test Law The waiver provision of the EFT law was amended in 1994 by addition of the following (10 U.S.C. 2366; see Appendix B for full text of the waiver · - provision : [T]he Secretary may waive the application of the survivability and lethality tests of this section to such a system or program and instead allow testing of the system or program in combat by firing munitions likely to be encountered in combat at components, subsystems, and subassemblies, together with performing design analyses, modeling and simulation, and analysis of combat data. In making this change, Me congressional conference committee stated that the amendment (Congressional Record, 1994~: would make it clear that the certification which must be provided to (~nn~re~ millet he ~libmitt^A heifer" the Or ^ - tore cuff;_ - _' _~^ _v~ · ~ · ^~ - in_ v~ V - EVA ~ ma_ ~] OL - ~l wAlL - l ~ ~11~1- peering and manufacturing development. The effect would be to maintain realistic survivability and lethality testing through testing of components, subsystems, and subassemblies in cases where the Secretary waives requirements for fills up testing . REQUEST FOR A RETROACTIVE F-22 TEST WAIVER The committee's third formal task (see Preface for Statement of Task) was to discuss what happened after Milestone I] to cause the F-22 program manager to request a waiver. The short answer is an improved understanding within DoD of Me legal requirement. Milestone I] for Me F-22 program occurred in June 1991 (Raggio, 19941; the provision of the law that permits a waiver dates to 1987; and a waiver for the

22 Live Fire Testing of the F-22 F-22 program was requested in 1993. The waiver request is discussed below, followed by the committee's assessment of what happened. The DoD General Counsel submitted draft legislation to Congress on October 8, 1993, "[too authorize a retroactive waiver of the survivability and lethality testing procedures that apply to the F-22 program" (Gorelick, 1993a, ~ 993b). The draft legislation was accompanied by a plan for alternative assessment of the vulnerability of the F-22 aircraft. The General Counsel's letter (as addressed to the President of the Senate) and its attachments are reproduced in Appendix C of this report. Briefly, DoD justifies its proposal as follows (Gorelick, 1993a, 1993b): Because of the cost of an F-22 aircraft, such [live fire] testing is both unreasonably expensive and impractical. Since the F-22 has already entered full-scale engineering development, legislation is needed to allow the Secretary of Defense to grant a waiver. . . . tT]he Air Force developed the revised live fire test program that . . . includes detailed analyses, review of historical test data, and incremental build-up testing that includes material characterization tests and live fire testing of selected components and subassemblies. Information from the results of these tests will be taken into account in the F-22's design. The SPO identified two areas for potential aircraft loss: (~) fire/explosion within the dry bay and/or filet tank ullage (i.e., the volume of the tank above the finely areas, and (2) hydraulic-ram-induced structural failure of the fuel cells (see Chapter 4~. The Air Force cites the conduct of limited ballistic tests, directed energy tests, and vulnerability mode! enhancements as demonstrating its efforts in vulnerability reduction methods. There has been no technical change in the program to account for the change in the Air Force's position. The history of what happened is best understood in terms of four points: the Air Force position on vulnerability; the Air Force position on filll-up, filll-scale live fire testing, confusion over the interpretation of the law; and hope within DoD and the Air Force that the previous NRC committee would help mitigate the requirements of the law. Each of these points is discussed below.

Origin of Testing Requirements 23 Position on Vulnerability Air Force briefings to this committee expressed the view that F-22 design emphasis should be placed on reducing susceptibility as opposed to reducing vulnerability (Hinton, 1994; Leaf et al., 1994~. Briefers also expressed the opinion that system weight was of overriding importance and should not be Increased significantly to achieve vulnerability reductions. The Air Force position, at least that voiced to the committee, is that (a) F-22 qualities like speed, maneuverability, and ability to kill the enemy first are more important to survivability than vulnerability (Hinton, ~ 994), and (b) further major improvements in vulnerability reduction would likely be won only by more serious losses in these other qualities (Ogg, 1995~. Position on Full-Up, Full-Scale Testing To the committee's knowledge, the Air Force has never planned to conduct full-up, full-scale tests of the F-22, believing that they would be too expensive and result in the loss of aircraft needed for other test purposes. The Air Force estimates that full-up, full-scale live fire testing of the F-22 would cost approximately $250 million, an amount that has not been budgeted (Raggio, ~ 994~. Chapter 3 discusses this matter in more detail. Confusion Over Interpretation of the Law Confusion over DoD interpretations of the law has already been discussed. During the previous NRC committee's deliberations in 1991 and 1992, various service representatives and contractors expressed their understanding that then current guidelines indicated that fil11-up, full-scale, live fire testing was not really required (NRC, 19931. However, someone in the legal office of the Office of the Secretary of Defense (OSD) had other ideas, since the SPO (i.e., General Fain, see above) wrote to the Office of the Secretary of the Air Force in October 1991 (Fain, 19911: We have been informed that OSD's legal council tsic] determined that full-up "system configured for combat" testing . . . is required to meet the intent of Chapter 139 of Title 10, United States Code, Section 2366, major systems and munitions programs: survivability and lethality testing; operational testing. We recommend requesting a waiver to this requirement.

24 Live Fire Testing of the F-22 The events surrounding this correspondence from General Fain pose something of a puzzle: . . According to James O'Bryon (O'Bryon, 1992), he and General Fain met in June ~ 991, near the time of Milestone Il. Mr. O'Bryon then had the clear understanding that fi~-up testing would be included in the F-22 live fire test program. According to Charles "Pete" Adolph, General Fain never mentioned live fire testing when he briefed the Defense Acquisition Board for Milestone I} in June 1991.~ General Faiths correspondence seems to indicate a clear appreciation by the SPO in October 1991 of the requirements of the law. Finally, while General Fain's correspondence in October of 1991 was prepared after Milestone IT in June 1991, a long time elapsed between that recommendation to request a waiver and the waiver activity in 1993. At least part of the explanation for this sequence of events seems related to expectations of help from the previous NRC committee. Help From the Previous Committee Between mid-1991 and early 1993, when the previous NRC committee was conducting its study, the Air Force and some DoD people apparently hoped that the previous committee would help mitigate the requirements of the law by recommending to Congress that the requirement for full-up, fi~-scale, live fire testing be reduced (Hawley, 19941.2 Publication of the NRC's January 1993 report, which upheld the need for a waiver, was a critical turning point. Several Air Force briefings to this committee referred to the recornrnendations of the previous NRC committee arid the subsequent DoD guidelines (Hawley, 1994; Ogg, 1995; arid Raggio, 19941. Others, who were OSD officials in 1991, expressed disappointment that the previous NRC committee did not take issue with the need for filll-up, full-scare, live fire testing of aircraft (Reed, 19951: ~ As indicated in a conversation on February 8, 1995, among Mike Clarke and Harry Reed of the Committee on the Study of Live Fire Survivability Testing for the F-22 Aircraft and Charles "Pete" Adolph, Larry Stanford, and Al Rainis, all of whom were OSD officials involved in test and evaluation at the time of Milestone II. 2 The February 8, 1995, conversation with Mr. Adolph substantiated this point. See note 1.

Origin of Testing Requirements They [Adolph, Stanford, and Rainis] were of the general opinion that the legislation is not appropriate for most of the Air Force's aircraft, which are not expected to be highly able to survive close encounters with bullets and warheads. Although they all felt that the legislation had added some value to the design process for aircraft, they expressed disappointment that the NRC EFT Committee had not challenged portions of the law Mat bear on the amount of knowledge acquired versus the cost of full-up testing as opposed to what was learned through the component and sub-assembly testing. A Lingering Question 25 In spite of all of these circumstances, there is the lingering question of why the SPO did not request a waiver before Milestone I} (when the Secretary of Defense could have granted it) to be on the safe side. The answer to this question may lie in the services' reluctance to submit waiver requests, which the previous NRC committee detected, leading it to recommend "that no stigma be attached to the use of the waiver process" (NRC, 1993~. F-22 DESIGN REQUIREMENTS FOR VULNERABILITY Design requirements that relate to F-22 vulnerability are based on its counter-air missions principally, the conduct of offensive courter-air operations (see Chapter I). The offensive counter-air mission placed a premium on achieving high survivability through reduced aircraft susceptibility. Reduction of vulnerability, although important, was deemed less sign~ficar~t. There is, however, an explicit statement in the 1994 Test and Evaluation Master Plan that indicates one air-to-surface mission has already been added.3 Even with this mission, the F-22 is to use internal carriage of weapons arid fly at 3 The plan (TEMP, 1994) states: The F-22 SPO has been directed to integrate the 1000 pound Joint Direct Attack Munition (JDAM) on the aircraft in support of the F-22's Air to Ground capability. The F-22 SPO is currently working with ACC [Air Combat Command] and the F-22 contractor team in further defining the actual program plan. When this planning is complete and placed on contract, program plans which will include test planning requirements will be available and incorporated into filture releases of the TEMP [Test and Evaluation Master Plan]. Therefore, the January 1994 TEMP includes no reference to the JDAM integration.

26 Live fire Testing of the F-22 a relatively high altitude. Nonetheless, the EFT law requires weapon systems with major modifications to undergo appropriate live fire testing or request a waiver. Presumably, a significant change in the F-22's mission (e.g., to an air-to-surface mission, which could increase the aircraft's exposure to surface-to-air defenses) would be considered a major modification that would necessitate additional vulnerability assessments. The ORD (Operational Requirements Document) identifies two general vulnerability requirements (TAF, 19911. The first is that flame-retardant hydraulic fluid and fuel-tart/c inerting be employed. The second is that avionics oower. and hv~raulics be redundant. fault tolerant and damage tolerant 7 ~ __~_ In addition, quantitative design requirements with respect to vulnerability are specified in the contract for the F-22. These contract specifications are needed to optimize aircraft design by addressing various tradeoffs and to assess contractor performance (Stewart and Shipman, 1995~. The F-22 SPO did art engineering assessment of the state-of-the-art of vulnerability reduction technology and the desired attributes of the aircraft. The SPO then specified vulnerability design requirements in two ways. First, general design requirements (e.g., minimize filet dumping into the engine inlet ducts) were specified. Second, the SPO specified that the total vulnerable area of the F-22 should not exceed certain levels, expressed in square feet, against various threats.4 The resulting wInerable areas were based on analyses of the F-22 design and were accepted since they compared favorably to the vulnerable areas of the F- ~ 5 against the same threats (Stewart and Shipmate, 1995~. These vulnerable areas were coordinated with and accepted by the user and established as the contractual design values. The values in these specifications do not include the effects of on-board ordnance. This assumption is addressed again in Chapter 4. Specifications for vulnerability to high-energy microwave devices (stated in frequencies and maximum power levels) are also included in the F-22 contract (GiorIando, 1995~. There are no specifications for protection against laser devices other than for the F-22's missile systems and the pilot's vision. The SPO considers that pilot safety is satisfied by the current visor. In summary, Me user chose to put more emphasis on reducing susceptibility, the probability of being hit in the first place, than on reducing vulnerability, the probability of being killed if hit. The users' choices were reflected in the specifications developed by the SPO. For future F-22 missions (e.g., air-to- surface), the Air Force should reexamine the balance of requirements among susceptibility, vulnerability, arid related performance parameters. ~. Vulnerable areas are defined as the product of the probability of kill given a random hit (by a fragment or a bullet) and the presented area of the target aircraft (NRC' 1993).

Origin of Testing Requirements DISPOSITION OF PRIOR COMMITTEE'S RECOMMENDATIONS 27 The recommendations of the previous NRC committee fall within three categories (NRC, 1993~: (~) clarifying and enforcing the waiver process, (2) improving the vulnerability data base and methodology, to include the development of cost-benefit analyses; and (3) orgaruzational arid administrative changes in the way OSD approaches the L`FT&E program and vulnerability assessment in general. This committee is pleased that OSD has in fact clarified the waiver process. The committee regrets that a cost-benefit analysis process has not been developed, although the Air Force is making some progress in this regard (Griffis, ~ 9951. The committee also strongly advocates that several areas of vulnerability technology receive considerably more emphasis: the development of an empirical data base, development of models, and experimental validation of these models. Finally, the committee does not wish to take up the organizational and administrative issues-partly because that would take the committee far from its stated mission and partly because some of the reorganizations within OSD would make the recommendations moot. SUMMARY Requirements Background The events that led to the late (post-Milestone IT) request for a waiver of the F-22 from full-up, fi~-scale live fire tests are complex. The original EFT law was passed in fiscal year 1987, since then there has been inconsistent interpretation of what live fire testing included. The most stringent interpretation was that a fully configured (with armament, fuel, etc.) combat system must be used. A less strict interpretation was that various sub-scale tests and models could be used to simulate the full-up, full-scale test and thereby provide information on system vulnerability. This committee (in agreement with the previous NRC committee) believes that the most stringent interpretation of the law is correct. The currently accepted interpretation, as contained in the ~ 994 DoD guidelines, is quite clear on the requirements for full-up, filll-scale, live fire testing or for a waiver from such testing prior to Milestone Il. These requirements were subsequently reflected clearly in Air Force Instruction 99-105. If a waiver is granted, the law instead permits live fire testing of components, subsystems, and subassemblies to maintain realistic survivability testing.

28 Live Fire Testing of the F-22 What Changed to Cause Request for Waiver Nothing specific to the aircraft program appears to have changed since Milestone I] in 1991 that would have prompted the Air Force to request a waiver. However, there appears to be a changed view of what is required to fulfill the intent of the EFT law as it is interpreted within the DoD. The committee's interpretation of the events is as follows. To the committee's knowledge, the Air Force never planned to conduct full-up, full-scare, live fire testing on the F-22. Prior to the NRC report in 1993, the Air Force believed that the component and subassembly testing protocol met the intent of the law. The Air Force initially based its belief on the DoD guidelines and then apparently hoped that the previous NRC committee would help mitigate the full-up, full-scale requirement (allowing instead the component and subassembly test methodology). Although there was some consideration of a request for a waiver in 1991, it was not until after the NRC report in 1993 that the Air Force appeared to accept fully the fact that a waiver was necessary. By then Milestone I] had Tong since passed (in mid-1991) for the F-22. F-22 Design Requirements for Vulnerability The F-22 was engineered principally on the basis of its offensive counter-air mission, which emphasized reduced susceptibility over vulnerability reductions. The user did not specify quantitative vulnerability requirements. Instead, contract specifications for vulnerability were established by the SPO and accepted by the user. . The existing vulnerability specifications arid the live fire test program do not address fixture missions (e.g., air-to-surface) Mat the F-22 might be required to perform. For fixture F-22 missions, the Air Force should reexamine the balance of requirements parameters. among susceptibility, vulnerability, and related performance Disposition of Previous Recommendations The committee is pleased that the waiver process has been clarified but regrets the lack of more progress in developing a cost-benefit methodology. Vulnerability technology still needs considerably more emphasis on the development of data bases and the development arid validation of models.

Origin of Testing Requirements REFERENCES Congressional Record. 1994. 103d Cong., 2d Sess., 140~1201: H8932. 29 Fain, I.A. 1991. Correspondence to Office of the Secretary of the Air Force, Acquisition, October 21. F-22 System Program Office. FASTC (Foreign Aerospace Science and Technology Center1. 1992. F-22 Svstem Threat Assessment Report (STAR), November 23. (S). GiorIando, I. 1995. High Power Microwave. Presentation to the Committee on the Study of Live Fire Survivability Testing of the F-22 Aircraft, F-22 System Program Office, Wright-Patterson Air Force Base, Ohio, January 19. Gorelick, I.S. 1993a. Letter to the Honorable Thomas Foley, Speaker of the House of Representatives, October 8. Washington, D.C.: Office of the General Counsel of the U.S. Department of Defense. Gorelick, ].S. 1993b. Letter to the Honorable Al Gore, President of the Senate, October 8. Washington, D.C.: Office of the General Counsel of the U.S. Department of Defense. Griffis, H. ~ 995. Cost Benefit Analysis Methodology. Presentation to the Committee on the Study of Live Fire Survivability Testing of the F-22 Aircraft, Dayton, Ohio, January 20. Hawley, R.E. 1994. Discussion of Waiver of Live Fire Tests. Presentation to the Committee on the Study of Live Fire Survivability Testing of the F-22 Aircraft, National Academy of Sciences, Washington, D.C., December 21. Hinton, W.S. 1994. Threat, Mission, and Operational Requirements for the F-22. Presentation to the Committee on the Study of Live Fire Survivability Testing of the F-22 Aircraft, National Academy of Sciences, Washington, D.C., December 21. Leaf, H.W., R. dazzle, and I. Ogg. 1994. Air Force Test and Evaluation Plans for F-22 and Other Applicable Air Force Aircraft. Presentation to the Committee on the Study of Live Fire Survivability Testing of the F-22 Aircraft, National Academy of Sciences, Washington, D.C., December 22. Longuemare, R.N. 1994. Live Fire Test and Evaluation Guidelines. Washington, D.C.: U.S. Department of Defense, Acquisition arid Technology.

30 Live Fire Testing of the F-22 NRC (National Research Council). 1993. Vulnerability Assessment of Aircraft: A Review of the Department of Defense Live Fire Test and Evaluation Program. Air Force Studies Board, NRC. Washington, D.C.: National Academy Press. O'Bryon, I. 1992. Memorandum to Director, Air Force Test and Evaluation, August 17. F-22 Live Fire Test and Evaluation (LFT&E) Vulnerability Program. Washington, D.C.: Office of the Secretary of Defense. Ogg, I. 1995. Vulnerability Program Overview. Presentation to the Committee on the Study of Live Fire Survivability Testing of the F-22 Aircraft, at the F-22 System Program Of lice, Wright-Patterson Air Force Base, Ohio, January 19. Raggio, R.F. ~ 994. Overview of the F-22 Program. Presentation to the Committee on the Study of Live Fire Survivability Testing of the F-22 Aircraft, National Academy of Sciences, Washington, D.C., December 21. Reeds H. 1995. Personal correspondence to Mike Clarke, Al Rains, and Larry Stanford, February ~ 5, on a discussion that took place February 8, ~ 995, at Science Applications International Corporation, Tysons Corner, Va., between Reed, Clarke, Adolph, Stanford, and Rains. Stewart, M., and I. Shipman. 1995. Ballistic Vulnerability Analysis. Presentation to the Committee on the Study of Live Fire Survivability Testing of the F-22 Aircraft, F-22 System Program Office, Wright-Patterson Air Force Base, Ohio, January 19. TAP (Tactical Air Force). 1991. Advanced Tactical Fighter (ATF) System Operational Requirements Document (SORD): TAP 304-3-/-A. March 1. (S/NF). TEMP. 1994. F-22 Test and Evaluation Master Plarl. Wright-Patterson Air Force Base, Ohio: F-22 System Program Office. January. USAF (U.S. Air Force). 1994. Interpretation of Live Fire Test Law. Air Force Instruction 99-105. Washington, D.C. U.S. Congress. 1988. (Cited in NRC, 1993.) FY 88-89 DoD Authorization Act Conference Report, Live-Fire Testing (Section 8021.

Next: 3 PRACTICALITY, AFFORDABILITY, AND COST-BENEFIT »
Live Fire Testing of the F-22 Get This Book
×
Buy Paperback | $48.00 Buy Ebook | $38.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The Live Fire Test Law mandates realistic survivability and lethality testing of covered systems or programs. A provision of the law permits the Secretary of Defense to waive tests if live fire testing would be "unreasonably expensive and impractical." Though no waiver was requested before the F-22 program entered engineering and manufacturing development, the Defense Department later asked that Congress enact legislation to permit a waiver to be granted retroactively. Rather than enact such legislation, Congress requested a study to explore the pros and cons of full-scale, full-up testing for the F-22 aircraft program. The book discusses the origin of testing requirements, evaluates the practicality, affordability, and cost-benefit of live fire tests, and examines the role of testing, modeling, and data bases in vulnerability assessment.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!