4
Live Fire Test Programs of the Three Services, and Views of the Test Community and Industry

The Army, Navy, and the Air Force have developed general Live Fire Test and Evaluation (LFT&E) policies in response to the passage of the 1987 Live Fire Test (LFT) law, and these policies are reflected in the LFT&E programs for the RAH-66 (Army), the V-22 and A-12 (Navy), and the F-22 and C-17 (Air Force). These policies and the individual aircraft LFT&E programs were presented to the committee on September 26–27, 1991, and are summarized below. Also contained in this chapter are the summaries of the presentations given to the committee on July 25, 1991 by the test community from each Service. The presenters represented the offices of U.S. Air Force, Office of the Director, Test & Evaluation; U.S. Army, Office of the Director, U.S. Army Test and Evaluation Management Agency; U.S. Navy, Office of Test and Evaluation and Technology Requirements. The highlights of all of these presentations are given below.

The committee believes it is important to point out before proceeding to the presentations that, except for the RAH-66, all of the programs described below were well under way when the law was passed and the C-17 development had proceeded beyond the deadline for the application of a waiver. Congress had given no transitional guidance for these programs. Furthermore, because the Services had only recently begun to test full-scale, full-up aircraft in the Joint Live Fire (JLF) program, they lacked adequate test facilities for conducting some of the tests; and they did not have any long-term experience in preparing the test plans for, and conducting, such tests.1 Finally, because of the confusion over the requirements of the law, no waivers were requested for any program because of the belief within the Services that the law was satisfied with less-than-full-scale, full-up tests and that a stigma would be attached to any waiver. This situation has complicated the preparation of a thorough LFT&E program that satisfies the letter of the law as well as its intent.

The Army LFT&E Programs

The Army gave two presentations on its Live Fire Testing program. One, given by a representative from the U.S. Army Test and Evaluation Management Agency, was entitled “Army Philosophy and Policy on Live Fire Testing.” The other, entitled “RAH-66 COMANCHE Ballistic Vulnerability DT&E Program Briefing,” was given by a representative from the COMANCHE Program Manager’s office. The presentations are summarized below.


Army Philosophy and Policy on Live Fire Testing. According to the presenter, the objectives of the Army Live

1

The 1987 General Accounting Office study presented in Appendix B addressed the issues of the status of the JLF test program, the methodological quality of the testing and evaluation process, and how to improve live fire testing.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 42
typesetting files. Page breaks are true to the original; line lengths, word breaks, heading styles, and other typesetting-specific formatting, however, cannot be retained, About this PDF file: This new digital representation of the original work has been recomposed from XML files created from the original paper book, not from the original Live Fire Test Programs of the and some typographic errors may have been accidentally inserted. Please use the print version of this publication as the authoritative version for attribution. Three Services, and Views of the Test Community and Industry 4 The Army, Navy, and the Air Force have developed general over the requirements of the law, no waivers were requested Live Fire Test and Evaluation (LFT&E) policies in response to for any program because of the belief within the Services that the passage of the 1987 Live Fire Test (LFT) law, and these the law was satisfied with less-than-full-scale, full-up tests policies are reflected in the LFT&E programs for the RAH-66 and that a stigma would be attached to any waiver. This (Army), the V-22 and A-12 (Navy), and the F-22 and C-17 (Air situation has complicated the preparation of a thorough Force). These policies and the individual aircraft LFT&E LFT&E program that satisfies the letter of the law as well as programs were presented to the committee on September 26– its intent. 27, 1991, and are summarized below. Also contained in this chapter are the summaries of the presentations given to the committee on July 25, 1991 by the test community from each The Army LFT&E Programs Service. The presenters represented the offices of U.S. Air Force, Office of the Director, Test & Evaluation; U.S. Army, The Army gave two presentations on its Live Fire Testing Office of the Director, U.S. Army Test and Evaluation program. One, given by a representative from the U.S. Army Management Agency; U.S. Navy, Office of Test and Test and Evaluation Management Agency, was entitled “Army Evaluation and Technology Requirements. The highlights of Philosophy and Policy on Live Fire Testing.” The other, all of these presentations are given below. entitled “RAH-66 COMANCHE Ballistic Vulnerability The committee believes it is important to point out before DT&E Program Briefing,” was given by a representative from proceeding to the presentations that, except for the RAH-66, the COMANCHE Program Manager’s office. The all of the programs described below were well under way presentations are summarized below. when the law was passed and the C-17 development had proceeded beyond the deadline for the application of a Army Philosophy and Policy on Live Fire Testing. waiver. Congress had given no transitional guidance for According to the presenter, the objectives of the Army Live these programs. Furthermore, because the Services had only recently begun to test full-scale, full-up aircraft in the Joint Live Fire (JLF) program, they lacked adequate test facilities 1 The 1987 General Accounting Office study presented in Appendix B for conducting some of the tests; and they did not have any addressed the issues of the status of the JLF test program, the long-term experience in preparing the test plans for, and methodological quality of the testing and evaluation process, and how to conducting, such tests.1 Finally, because of the confusion improve live fire testing. 42

OCR for page 42
LIVE FIRE TEST PROGRAMS 43 Fire test and evaluation (T&E) program are to demonstrate the critical subsystems, and the third deals with the mission typesetting files. Page breaks are true to the original; line lengths, word breaks, heading styles, and other typesetting-specific formatting, however, cannot be retained, About this PDF file: This new digital representation of the original work has been recomposed from XML files created from the original paper book, not from the original ability of a system to provide survivability, to provide insight essential subsystems. Accordingly, the objectives of the into the principal damage mechanisms and kill modes, and to development test and evaluation (DT&E) program for provide insight into techniques for reducing personnel ballistic vulnerability are to verify adequate aircrew and casualties and enhancing system survivability. The Army aircraft protection, to reduce the vulnerability of the flight and some typographic errors may have been accidentally inserted. Please use the print version of this publication as the authoritative version for attribution. policy reiterates the legislative language. A draft policy critical components, and to reduce the vulnerability of the document AR 73-XX, “Test and Evaluation Policy,” was mission essential components. All subsystems are to be released for interim use in December 1990. According to that evaluated in the step-by-step test program, starting from document, LFT&E will be integrated into the overall T&E section/ coupon testing, proceeding through static/dynamic program strategy. The Live Fire Tests are part of technical component testing and full-up component and subsystem testing, and the scope of the program will build on early testing, and finishing with full-up complete system Live Fire testing of components and on modeling. The LFT&E strategy Testing. The early tests are design support tests, and the later will be contained in the Test and Evaluation Master Plan tests are to establish specification compliance. Two examples (TEMP). A draft document Department of the Army Pamphlet of design support tests using coupons or engineering (DA PAM) 73-XX, “Test and Evaluation Procedures mockups are the hydraulic ram tests on fuel tank panels and Guidelines,” is being prepared. This document will the ballistic penetration tests on composite panels. Two of the incorporate the document “An Army Guide to Live Fire Test test articles to be examined for specification compliance are and Evaluation,” August 1990. the flight control linkage and the anti-torque system drive According to the presenter, the Army LFT&E strategy shaft. Most of the 26 tests scheduled will use the 12.7- involves a complementary testing and modeling effort. millimeter API; five tests are scheduled for a larger-caliber Modeling is an essential part of the strategy for determining projectile. The Milestone II exit criteria tests are to system vulnerability since it is impractical to test the total demonstrate the damage tolerance potential of selected spectrum of weapon/aircraft interactions, and testing is critical components to the specified design threats. Some of required to provide the necessary data to develop the the test articles are the main rotor hub, main rotor blade, fantail vulnerability model algorithms. The LFT&E strategy follows assembly, and the T800 engine. The focus of these tests is to a “building-block” approach consisting of component verify the ability of the helicopter to fly for 30 minutes after a testing; full-up subsystem testing; and full-scale, full-up ballistic impact. system testing. The emphasis of the testing program is on According to the presenter from the COMANCHE Program front-end testing of the components and subsystems to ensure Office, the objective of the dedicated Live Fire Test program is that the performance of these tested articles is understood. to evaluate the vulnerability of the full-up production Limited full-scale, full-up testing is conducted to confirm this configuration components, subsystems, and if necessary, a understanding. representative COMANCHE air vehicle. The focus of these tests will be on postdamage tolerance and the evaluation of Live Fire Testing of the RAH-66 COMANCHE Helicop ter. synergistic effects. According to the presenter, the LFT&E The Program/Ballistics Detailed Schedule for the program uses the approved threats; schedules the tests in a COMANCHE (presented to the committee in September sequential “building-block” approach; tests full-up 1991) is shown in Figure 4-1.2 The ballistic assessment tests components and subsystems; will test a full-scale, full-up began in April 1991 and ended in November 1991. A three- system, if necessary; and is in compliance with the law. and-half year period of ballistic verification and demonstration tests and Live Fire Tests begins in March 1995 Committee Comments. Several aspects of the COMANCHE and ends in September 1998. LFT&E program are of concern to the committee. First, the The presenter identified three major ballistic vulnerability schedule shows a Live Fire Test program being completed issues that could result from expected threat encounters. The just before going into full-rate production but after 72 first issue deals with the hazards to the COMANCHE aircrew, aircraft have been bought in low-rate production contract the second issue deals with the vulnerabilities of the flight awards, which is approximately 15% of the planned buy. It may not be possible to retrofit the changes required to correct any design deficiencies discovered in the LFT&E program into these aircraft. Furthermore, if approval is given 2 In early 1992, the Army restructured the COMANCHE program, stretching out the prototype phase by two years through the summer of to go into full production, another 96 aircraft will be 1997. Plans for completing the development and going into production produced in the next year while the design changes are being after 1997 have been dropped, and the survivability and live fire tests prepared. This is contrary to the LFT law’s requirement that have been deferred. Apparently, this is part of the Defense Department’s plans to emphasize prototype research and development of new aircraft testing shall be carried out sufficiently early in the (Bond, 1992).

OCR for page 42
44 VULNERABILITY ASSESSMENT OF AIRCRAFT typesetting files. Page breaks are true to the original; line lengths, word breaks, heading styles, and other typesetting-specific formatting, however, cannot be retained, About this PDF file: This new digital representation of the original work has been recomposed from XML files created from the original paper book, not from the original and some typographic errors may have been accidentally inserted. Please use the print version of this publication as the authoritative version for attribution. FIGURE 4-1 Program/ballistics detailed schedule for the COMANCHE. development phase of the system to allow any observed vulnerable area of the aircraft. There was little discussion of design deficiency to be corrected before proceeding beyond what the internal stowage of ordnance might do to low-rate initial production (LRIP). There are two ways this vulnerability, whether this would be addressed with some situation could be improved. One is that the demonstration kind of protection. The committee believes that the and validation (DEM/VAL) prototypes will have been flying consideration of ordnance in the assessment of vulnerability, for several years before the first of the low-rate aircraft will both in modeling and in testing, is essential.3 have been contracted for, and one of these could be used for It is the committee’s opinion that the policy level of the Live Fire Testing. The other way is to slow down the Army supports limited full-scale, full-up testing, but that this production rate. support has not been fully recognized at the program level. Second, the briefer was vague on the actual testing of a full- scale, full-up aircraft and, when questioned, said a full-scale aircraft was not expected to be needed. On the other hand, his The Navy LFT&E Programs summary says the program will operate in compliance with the current Live Fire Test law. The presenter from the Army Test The Navy Live Fire Test philosophy and the program for the and Evaluation Agency left no doubt of the Army’s position; it V-22 were presented to the committee by a representative of will do full-scale, full-up testing. the Survivability Branch, Naval Air Systems Command. The COMANCHE briefer mentioned the on-board munitions, but did not dwell on its contribution to the 3 This aspect of vulnerability is examined in Chapter 2.

OCR for page 42
LIVE FIRE TEST PROGRAMS 45 typesetting files. Page breaks are true to the original; line lengths, word breaks, heading styles, and other typesetting-specific formatting, however, cannot be retained, About this PDF file: This new digital representation of the original work has been recomposed from XML files created from the original paper book, not from the original and some typographic errors may have been accidentally inserted. Please use the print version of this publication as the authoritative version for attribution. FIGURE 4-2 Live Fire Test and Evaluation of Navy aircraft. The Deputy Director of the Navy’s Office of Test & Evaluation simulators and surrogates provides information on the and Technology Requirements was present at the briefing. design’s compliance with the specifications. Preliminary information on the LFT&E program for the A-12 According to the Navy presenter, the benefits identified for aircraft was provided during several committee meetings. The full-scale, full-up testing are the possible discovery of Navy’s presentation is summarized below. previously unknown reactions to the weapons and synergistic effects. The disadvantages include the fact that the results are The Navy Live Fire Test and Evaluation Policy. The Navy’s very limited statistically; the full-scale aircraft is at risk on view of the Live Fire Test law is that it encourages full-up each test; the tests are very costly; and the full-scale, full-up vulnerability testing, under realistic combat conditions, first tests do not eliminate the need for all of the earlier sub-scale at the sub-scale level and later at the full-scale level. Live fire tests. Furthermore, testing the full-scale aircraft can be testing is part of developmental testing and is a continuing accomplished only at a very late stage in the development, process that contributes to the development and engineering and it may not be possible or practical to correct any design of Navy aircraft. The Navy policy is to comply with the intent vulnerabilities discovered in the tests. This is contrary to the of the law by establishing realistic operational requirements requirement in the law that the testing shall be carried out and combat threat scenarios, establishing explicit sufficiently early to allow any design deficiency Survivability and vulnerability design requirements, and demonstrated by the testing to be corrected before proceeding requiring Live Fire Testing and Evaluation as part of the beyond low-rate initial production. In summary, Live Fire development process using a shoot-fix-shootagain approach. Testing is a process that begins in the D/V phase; it is not a The Navy LFT&E program integrates Live Fire Testing with single pass/fail event. It involves both analyses and testing, the development of the aircraft, as shown in Figure 4-2. The with results available sufficiently early to aid in design. integration of the tests on simulators and sub-scale hardware early in the development provides design information in time Live Fire Testing of the V-22 Aircraft. A comprehensive plan to be useful. Later Live Fire Verification Testing using for the Live Fire Testing of the V-22 was made available

OCR for page 42
46 VULNERABILITY ASSESSMENT OF AIRCRAFT to the committee, and though it is somewhat dated it was a of the LFT Office was read into the program because of the fact typesetting files. Page breaks are true to the original; line lengths, word breaks, heading styles, and other typesetting-specific formatting, however, cannot be retained, About this PDF file: This new digital representation of the original work has been recomposed from XML files created from the original paper book, not from the original useful input. The program is old enough to be grandfathered that the aircraft would become a candidate for LFT when it no for Live Fire Testing even though the summary page says the longer was a black program. According to one of the briefers, current program meets the Live Fire Test law. the A-12 live fire test program for components and subsystems The V-22 aircraft had specific design requirements on was one of the best programs to date. Although the proposed and some typographic errors may have been accidentally inserted. Please use the print version of this publication as the authoritative version for attribution. the allowable vulnerable area for several specific threat LFT strategy did not include testing on the full-scale, full-up weapons, excluding hydraulic ram. A number of alternative aircraft, it appeared to be reasonable according to the briefer. vulnerability reduction designs were identified, and the Apparently, informal approval was given to the proposed LFT reduction in vulnerable area and increase in weight were strategy, and this strategy was in effect when the program was determined for each alternative. The most promising canceled. Two major criticisms of the program were the alternatives were tested. Some of the test articles used in the omission of testing of the on-board ordnance and the omission stage I ballistic tests included a wing structure, a fuselage of consideration of threats other than projectiles and missiles, structure, a pylon actuator, and some of the flight controls. such as directed energy weapons. Ballistic verification tests have been completed for phase I and phase II wing tank hydraulic ram and the propellar Committee Comments on the A-12. It is the committee’s rotor gearbox components. A series of dry bay fire opinion that the policy levels of the Navy and the Program suppression tests has been conducted using a Offices appear to have no intention of doing any full-scale, representative three-bay simulator. Four different full-up test on any aircraft before proceeding beyond low- suppression techniques were tested; a Halon system, power rate initial production. The Navy considers full-scale, full- panels, foam, and an on-board inert gas generating system up testing of aircraft to be unreasonably expensive and (OBIGGS). The threats were several ballistic projectiles. In impractical. Apparently as a consequence of this belief and the baseline tests without any protection, large, consistent the interpretation of the LFT requirements given in the fires with heavy fixture damage were observed. The results LFT&E Guidelines, no full-scale, full-up LFT tests were using the different suppression techniques ranged from proposed for the A-12. The committee’s opinion is that any several fires to no fires. approval of an LFT strategy for the A-12 that did not include tests on the full-scale, full-up aircraft and did not request a Committee Comments on the V-22. The V-22 aircraft has been waiver from these tests would have been in error. The under development for several years and has been under a committee is concerned that future black programs will face cloud for many of them. The cloud is related to the need for the the same problems with respect to the LFT law that occurred system rather than anything related to its vulnerability. The with the A-12. The Navy has initiated efforts on a cancellation of the program by the Secretary of Defense and replacement program for the A-12, presently called the AX. the reinstatement of the program by Congress tend to make Because this also is a black program, it is not a covered systematic planning for various phases of Live Fire Testing system under the LFT law. However, it too will eventually difficult. become an unclassified program and at that time will be The purpose of the aircraft is to take troops and light subject to the law’s provisions. This will probably occur after material into the vicinity of danger, and for that mission the the program enters into full-scale engineering development, Joint Services Operational Requirement for ballistic the deadline for the application for a waiver. If the tolerance seems inadequate. Furthermore, there is no requirements in the LFT law are ignored until the program discussion of the vulnerable area associated with on-board comes out of the black, there most likely will be a ordnance. The aircraft will be hauling U.S. Marines into confrontation with the LFT Office. combat, and they will put all of the weapons and ordnance they can load aboard. An analysis that shows how the load could be distributed, or how some light armoring could The Air Force LFT&E Programs change the vulnerability of the aircraft to ground fire, would give more credence to the claim that the ballistic protection The Air Force Live Fire Test Policy. The Air Force policy on had been achieved. LFT&E was presented by a representative from Test & The fate of the LFT&E program is dependent upon a Evaluation, Air Force, and was entitled “Air Force Policy decision to enter into low-rate production. Considerations for Live Fire Test & Evaluation.” According to the presenter, the objective of Air Force LFT&E is to provide Live Fire Testing of the A-12 Aircraft. The A-12 was a highly a timely and thorough assessment of the vulnerability and classified program, and therefore, it was not a covered system lethality of a system as it progresses through its according to the LFT law. Nevertheless, a member development. This is accomplished by a balanced program

OCR for page 42
LIVE FIRE TEST PROGRAMS 47 of analysis and test. It is a systems approach, similar to weaknesses and to identify vulnerability reduction typesetting files. Page breaks are true to the original; line lengths, word breaks, heading styles, and other typesetting-specific formatting, however, cannot be retained, About this PDF file: This new digital representation of the original work has been recomposed from XML files created from the original paper book, not from the original development testing used in other areas. Because it is candidates. Vulnerability testing of materials, components, prohibitive to test all possible combinations of threat/aircraft/ and subassemblies is used to verify the vulnerability conditions, analysis must be an integral part of the LFT&E reduction candidates and to identify issues that necessitate process. A general sequence of design-analyze-test is followed Live Fire Testing. The results from the analyses and the lower- and some typographic errors may have been accidentally inserted. Please use the print version of this publication as the authoritative version for attribution. and repeated if necessary. The analytical models are those level tests will determine the need for, and extent of, further accepted by the Joint Technical Coordinating Group on validation tests. According to the presenter, the LFT&E Aircraft Survivability (JTCG/AS). The LFT&E program is program is a sound development and demonstration program initiated sufficiently early to allow the results to impact the that represents a balance of the technical merits, cost, and system design. The benefits of LFT&E can be maximized only schedule. The methodology used in the program is shown in when the test results are used to aid in the design. The LFT&E Figure 4-3, and the vulnerability program schedule and strategy, which is included in the TEMP, ensures that all issues funding are given in Figure 4-4. According to the presenter, are addressed and integrated with the other elements of the the budget and schedule supports the conduct of prudent Live system program. According to the presenter, the Air Force Fire Testing. Some of the assessments to be conducted to policy is in compliance with the reporting requirements of the verify the design include fuel tank inciting using a rig test, LFT&E Office. The threat used for the Live Fire Tests is the redundancy and separation studies using analyses and threat(s) defined by the Operational Requirements Document inspection, verification of the fragmentation resistance of (ORD). The ORD is the official source of user requirements, pressure vessels using ballistic tests, and a static test of and the ORD process places the user in the role of integrating structure. the System Threat Assessment Report (STAR), aircraft characteristics, employment concepts, etc., to derive the Committee Comments on the F-22. In the F-22 presenter’s expected threat. discussion of vulnerability modeling for the engineering, According to the presenter, the test hardware will be of manufacturing and development (EMD) study he stated that sufficient size and quantity so that realistic test results, internal armament was treated. However, a chart of including synergism and secondary effects, will be obtained. vulnerability requirements included a column for armaments Experience indicates that components, subsystems, and that discussed vulnerable area as to be determined (TBD) and subassemblies are generally sufficient and more desirable fire inciting, fire protection, and shielding as not applicable than a complete system. Full consideration will be given to (N/A). The EMD vulnerability program shown in Figure 4-5 the use of actual hardware, replicas, and surrogates. However, has a category for stored munitions, engine, and aircraft use of these items must depend on the technical payoff, availability, but it is under concurrent JLF. When members of availability, and cost. The requirements for additional live fire the committee questioned the possibility that this was the test facilities are being identified in the Air Force’s Test prototype aircraft, the briefer seemed unaware that a Live Fire Investment Planning Process. In summary, according to the Air Test would be done on a full-scale, full-up aircraft, and that the Force, its LFT&E program is integral to the system design, prototypes were so different from the production model that development, test, and evaluation process. The Air Force they would not be representative. believes it endorses an intelligent approach to LFT&E that considers all variables affecting aircraft effectiveness. It The Live Fire Test Program for the C-17 Aircraft. The program believes the Air Force T&E approach employs the most for the C-17 LFT&E was given by a representative from the C- prudent combination of analyses and tests, is viable, and is 17 System Program Office in a presentation entitled “C-17, Live producing positive results. Fire Test (LFT) Program.” According to the presenter, the C-17 is expected to be deployed to within 20 to 40 km of the Forward The Live Fire Test Program for the F-22 Aircraft. The Live Edge of the Battle Area FEB A). The vulnerability requirement Fire Test program was briefed to the committee by a for the selected design threat in the FEBA is met by using representative from the system program office in a redundancy with separation, damage tolerant components, and presentation entitled “F-22 Vulnerability Program.” The fire and explosion suppression. A number of design changes approach to aircraft survivability used by the F-22 is a were made to meet the requirement as the result of analytical combination of low susceptibility and reduced vulnerability. assessments. These included the relocation of the oxygen Directed energy weapons are considered, as well as the converter, rerouting of the hydraulic lines, and changes to the conventional gun and missile threats. The reduction in pitch trim actuator and aileron hinge fitting. The test vulnerability is accomplished through incorporation of methodology included design, analysis, test, redesign, and redundant subsystems and damage tolerant features. retest. The test articles included components, such as crew Vulnerability analyses will be used to establish any system armor, pressure vessels, the upper wing skin, and a flap

OCR for page 42
About this PDF file: This new digital representation of the original work has been recomposed from XML files created from the original paper book, not from the original typesetting files. Page breaks are true to the original; line lengths, word breaks, heading styles, and other typesetting-specific formatting, however, cannot be retained, and some typographic errors may have been accidentally inserted. Please use the print version of this publication as the authoritative version for attribution. 48 FIGURE 4-3 Vulnerability assessment methodology for the F-22. VULNERABILITY ASSESSMENT OF AIRCRAFT

OCR for page 42
LIVE FIRE TEST PROGRAMS 49 typesetting files. Page breaks are true to the original; line lengths, word breaks, heading styles, and other typesetting-specific formatting, however, cannot be retained, About this PDF file: This new digital representation of the original work has been recomposed from XML files created from the original paper book, not from the original and some typographic errors may have been accidentally inserted. Please use the print version of this publication as the authoritative version for attribution. FIGURE 4-4 Vulnerability program for the F-22. hinge fitting; and a functional 8-foot section of the wing Test candidate and expects to see Live Fire Testing leading edge. The leading edge section is to be Live Fire conducted on the C-17 aircraft (Bennett, 1991). The Tested to determine if there is a leading edge fire problem. The contention by Congress is that testing of components from a test article is designed from production drawings from the military vehicle in an essentially inert condition is no Douglas Aircraft Company and will be built by the 4950th substitute for the firing of threat ammunition at that vehicle Test Wing Modification Branch. The Live Fire Tests will be loaded for combat with the intended fuels, fluids, and conducted by the Wright Laboratory’s Flight Dynamics ordnance on board and in place. Directorate. In the case of cargo aircraft such as the C-17, if the aircraft contains ordnance when configured for combat, the ordnance and its reaction to ballistic impact must be treated in the Committee Comments on the C-17. The C-17 represents one assessment. A sub-scale, but full-up, Live Fire Test of the C-17 of the most interesting challenges to the Live Fire Test law of would logically consist of testing a mock-up of an actual 1987. It was in full scale development when the law was cargo compartment, with ammunition loaded in pallets on passed and hence could not legally request a waiver. It is a dunnage. If the shot created significant damage due to a very large and expensive aircraft, although not the largest to reaction of the ammunition, shielding or armoring could be enter the inventory. The largest, the C-5A, was delivered undertaken to bring the damage expected to an acceptable during the 1960s and was not subject to any live fire testing. level. Apparently, Congress considers the C-17 to be a Live Fire During a temporary shifting around of personnel in the

OCR for page 42
About this PDF file: This new digital representation of the original work has been recomposed from XML files created from the original paper book, not from the original typesetting files. Page breaks are true to the original; line lengths, word breaks, heading styles, and other typesetting-specific formatting, however, cannot be retained, and some typographic errors may have been accidentally inserted. Please use the print version of this publication as the authoritative version for attribution. 50 FIGURE 4-5 Vulnerability requirements for the F-22. VULNERABILITY ASSESSMENT OF AIRCRAFT

OCR for page 42
LIVE FIRE TEST PROGRAMS 51 Office of the Deputy Director, Defense Research and an aircraft are known and can be anticipated; they state that they typesetting files. Page breaks are true to the original; line lengths, word breaks, heading styles, and other typesetting-specific formatting, however, cannot be retained, About this PDF file: This new digital representation of the original work has been recomposed from XML files created from the original paper book, not from the original Engineering (Test & Evaluation), the Air Force’s proposal to very seldom see a test result that was unanticipated. The test only a portion of the leading edge of the wing was magnitude of the response to the hit may be larger or smaller than accepted as being in compliance with the LFT law. This expected, but the response was anticipated.5 enforced the belief that the conduct of full-scale, full-up Live and some typographic errors may have been accidentally inserted. Please use the print version of this publication as the authoritative version for attribution. Fire Testing could be avoided by the acquisition arm of the Air Committee Comments. The committee notes that not Force. The Air Force user should have been put more of a everyone who has observed live fire tests on sub-scale and decision-making position on this issue. The size of the C-17 is full-scale test articles shares the views held by these testers. going to make it an attractive target for various potential They believe that there have been unanticipated results from enemies who might hold fire from smaller targets in order to these tests. Furthermore, even when the response is as concentrate fire on the larger aircraft. If an aircraft is to go into expected, the difference in the expected magnitude of the combat, and if combat damage is expected to be encountered, response and the observed magnitude often is too large to be the aircraft is going to have to be tolerant of the damage or a acceptable. The testers have also overlooked the fact that price will be paid in manpower and material losses. Supporters information from the full-scale tests is an input to acquisition of the Live Fire Test law believe that if the full-scale C-17 is decision makers at milestone reviews. The committee also too expensive to be subjected to live fire in a full-up believes that the test community has not given proper condition, the aircraft is too expensive to risk losing by consideration to the on-board ordnance problem. delivering cargo to a forward military zone. If that is the case, the upper limit on the cost a Service may be willing to put into an aircraft may have been reached. A reasonable question to The Industry View of Live Fire Testing ask is, would the aircraft be bought if it was to be used only to deliver military cargo to rear areas? If that answer is no, the The U.S. aircraft industry does not play a major role in LFT&E. importance of testing the aircraft against the weapons it is However, the committee solicited its opinion because these expected to encounter is paramount. are the users of the results of the Live Fire Tests. The four industry representatives were unanimous in their belief that the LFT law as described in the LFT Guidelines and Planning The Test Community’s View of Live Fire Testing Guide does not require a full-scale, full-up aircraft to be tested. Furthermore, they do not believe it should. One of the major The live fire test community of the three Services appears to be in reasons they gave for not recommending full-scale testing of a general agreement about the efficacy of the Live Fire Test law as “production” aircraft is that the design is usually frozen by the it applies to aircraft. Members of that community do not consider time the results become available.6 However, they were in full it necessary to do full-scale, full-up tests in order to determine agreement with the test community on the necessity of sub- most of the design vulnerabilities. However, they do consider it scale testing throughout the program development. essential to conduct many sub-scale tests on components and sub-systems, both inert and full-up, in the development cycle of an aircraft. The sub-scale test results are invaluable sources of Controversy Regarding Which Munitions to information that can be used initially to aid the design and later to Use in the Live Fire Testing Program verify the design. The test community believes that the full-scale, full-up tests are conducted too late in the development cycle to The Live Fire Test law stipulates that “munitions likely to be be of much value to the designer and that the amount of encountered in combat (or munitions with a capability information obtained from the tests is very limited.4 Every time a similar to such munitions)” are to be used in the “realistic test is conducted a portion of the test article is damaged, and it survivability testing” The specific munitions to be used in the may be impossible to restore the article to its original condition. Live Fire Testing of a particular aircraft are selected by the Furthermore, these tests require very large budgets that could go aircraft Program Office as part of its LFT&E program plan. toward the expansion of test facilities and capabilities and the Typically, the threats selected by the Program Office for Live conduct of many more sub-scale tests. The test community does Fire Testing are the threats the aircraft was designed to recognize the possibility of an unanticipated reaction, cascading withstand, such as a single hit by an armor-piercing (API) damage, or synergism occurring in the full-scale aircraft. However, its members believe that nearly all of the kill modes of 5 This attitude appears to be in contrast to the experience of the testers of ground vehicles, where the particular response to a hit was often 4 The test community is not against all testing of full-scale, full-up unanticipated. 6 aircraft; it fully supports this type of testing on existing aircraft in programs The committee notes that industry shares the same reluctance as the such as JLF. Services to change a design late in the development cycle.

OCR for page 42
52 VULNERABILITY ASSESSMENT OF AIRCRAFT or small-caliber high explosive with incendiaries (HEI). The is called threat projection. The results of the threat projection typesetting files. Page breaks are true to the original; line lengths, word breaks, heading styles, and other typesetting-specific formatting, however, cannot be retained, About this PDF file: This new digital representation of the original work has been recomposed from XML files created from the original paper book, not from the original assumption is made that the more lethal overmatching threats, appear in STAR. Historically, the threat community has been such as the larger anti-aircraft artillery (AAA) and the surface- reluctant to commit to a firm projection of the specific threats to-air missiles (SAMs), will be avoided and hence should not a particular system will encounter 10 to 20 years in the future be a part of the LFT&E program. For example, the Navy and for several reasons. Consequently, a long list of threats that the and some typographic errors may have been accidentally inserted. Please use the print version of this publication as the authoritative version for attribution. the Air Force are currently developing stealth aircraft whose system may encounter in combat is usually prepared, perhaps survivability is strongly related to not getting hit. They with some prioritization. However flawed the threat projection believe that some “credit” should be given for this aspect of process may be, the user of the proposed system must select a survivability. Aircraft that do not get hit as often do not have design threat for the system. The developer then designs a to rely as much on their reduced vulnerability to survive. system that will survive the threat described in the user’s requirements document.8 Furthermore, if these overmatching threats are included in the LFT&E program, they could destroy the only test article(s) Note that the user is the organization that examines the available, as well as the test equipment, and perhaps some of threat projection and selects the design threat. The design the test facilities. threat selected may or may not be the most likely threat that The Live Fire Test Office has interpreted the phrase will be encountered; usually it is but one of many threats that “munitions likely to be encountered in combat” to mean that are likely to be encountered. The Live Fire Test Office does those munitions the aircraft may encounter, including the latest not consider the design threat selected by the user as the only directed energy weapons, should be included in the LFT&E threat likely to be encountered and strongly encourages the program, regardless of the design threat for the aircraft. This Program Office to test the system against the other threats that interpretation of the phrase “likely to be encountered in exist, or will exist, in the operating environment. The combat” made by the LFT Office conflicts with the argument given by the LFT Office is that the threat selected for interpretation made by the Program Managers, and this design is usually one that can be defeated without significant difference in interpretations has been the source of considerable increases in weight and cost. Thus, given that it is important to controversy. The LFT Office argues that the user Service’s verify the design’s ability to withstand the design threat, it is intention to avoid the most lethal threats in combat through also important to determine, and correct if possible, any susceptibility reduction may not always be achievable. Given weaknesses in the design when subjected to the other “likely that the intent of the LFT law is not to intentionally destroy to be encountered” threats before the system goes beyond aircraft, the LFT Office believes that a test involving a LRIP. An argument against testing against the other, possibly “potentially” overmatching weapon can be designed that will overmatching, threats is that even though the system was not provide information on any design weaknesses with little required to defeat these weapons, any negative results from likelihood of destroying the aircraft. For example, suppose the such tests may jeopardize the program, as well as siphon off weapon is a guided missile with a large HE warhead and a much needed funding. proximity fuze. The lethality of this weapon depends upon the The C-17 LFT&E program is an example of the conflict location of the warhead with respect to the aircraft at the time of between testing only against the design threat and testing detonation. If the detonation occurs next to the aircraft, the against other threats that may be encountered in combat. aircraft will most likely be killed. On the other hand, if the The Operational Requirements Document for the C-17 detonation occurs far from the aircraft, the aircraft will most stipulates a design that will withstand a single hit by a likely not be killed. Consequently, the LFT Office believes that certain ballistic projectile. According to the C-17 presenter, if information on any design weaknesses can be obtained from a this projectile may be encountered by the C-17 in the region distant detonation with little probability of destroying the of 20 to 40 km from the Forward Edge of the Battle Area. The aircraft, the test should be considered. The Services counter this Live Fire Test Office, on the other hand, believes that C-17s argument with the argument that any information obtained in the forward area of the battlefield and at austere landing from the limited number of tests is not statistically meaningful. sites near combat zones can expect threats that are This is, in turn, countered by the argument that one purpose of the full-scale tests is to discover vulnerabilities, not to quantify the expected threat, particularly on the vulnerability aspect of survivability. vulnerability. Perhaps the first aircraft to have a vulnerability requirement on the design were the U.S. Army’s UTTAS and AAH, now the UH-60 Black Committee Comments. Today, military systems are designed Hawk and AH-64 Apache. Both aircraft had to be designed to survive a single hit by a non-explosive ballistic projectile anywhere on the aircraft to survive the threat they can expect to find in the field at the and fly for 30 minutes after the hit. time they are fielded.7 The process of defining the future threat 8 As a consequence of this procedure, systems are typically designed against today’s threats—and sometimes even against yesterday’s threats. Both practices result in systems being designed to survive yesterday’s 7 threats when they finally appear in the field. Aircraft have not always had specific design requirements to survive

OCR for page 42
LIVE FIRE TEST PROGRAMS 53 more much lethal than the design threat selected for the that they, the Services, are directly responsible for the fate of typesetting files. Page breaks are true to the original; line lengths, word breaks, heading styles, and other typesetting-specific formatting, however, cannot be retained, About this PDF file: This new digital representation of the original work has been recomposed from XML files created from the original paper book, not from the original C-17.9 the military personnel who use this equipment in combat. They are very apprehensive about any outside organization that can dilute their ability to define the necessary equipment A Program Manager’s View of the Live Fire testing and the procedures required to accomplish this testing. and some typographic errors may have been accidentally inserted. Please use the print version of this publication as the authoritative version for attribution. Test Law They appear to further believe that the Live Fire Test law gives to others not directly responsible for the delivered product The committee, after listening to all of the presentations, believes inordinate control without any accompanying responsibility the following description of a Program Manager’s (PM’s) view for the quality of the product or its cost. of the LFT law may be representative of current opinion: In the committee’s opinion, the Office of the Secretary of The full-scale, full-up testing mandated by the LFT law is Defense (OSD) may believe that there have been a sufficiently an unquantifiable, but potentially catastrophic, risk to his large number of prior experiences in the area of live fire testing program. LFT has no quantitative contractual specification or to indicate that pressure by a Service for successful and rapid acceptance criteria at program initiation. No quantitative certification of its products under development can lead to criteria for acceptable or unacceptable damage are included in inadequate live fire testing and to subsequent unnecessary the requirements process, milestone commitments, or combat vulnerabilities. The OSD, therefore, has chosen to contractor performance documents. LFT of the full-scale exercise close control over the Live Fire Test programs and aircraft occurs late in the development phase of the program, assumes the ultimate authority for approval of the equipment and there may be neither adequate time nor money to conduct based on the program results. the tests or to make any changes required as a result of the In the committee’s opinion, the Congress may believe that tests. Further, the PM may believe that neither the definition of there are sufficient numbers of proven instances of the tests, nor the conduct of the tests, nor the interpretation of unnecessary combat vulnerability in the Department of the test results is totally under his control. The perceived Defense (DoD) equipment previously delivered to the field to jeopardy to his program created by LFT is exacerbated by the warrant legislative direction of DoD test certification to severe requirement to fit the program into a somewhat include live fire testing of full-scale, full-up systems using inflexible overall resource schedule, both in time and in munitions likely to be encountered in combat. Congress dollars. In summary, LFT represents a considerable source of further believes that it has the ultimate responsibility for the problems to the PM, in the form of an uncontrollable, programs it authorizes and therefore has the obligation to potentially catastrophic uncertainty, as he attempts to exercise that legislative direction. successfully complete the development of his system, and should be avoided if at all possible. Conclusion Issues Relating to Distrust Among the The committee believes the common view of all parties Participants in Live Fire Testing knowledgeable in this business is that the current assessment procedure, including both analysis and Live Fire Testing, The committee is aware of the strong differences of opinion does not guarantee that the U.S. armed forces will field cost- held by various individuals and organizations concerning the effective systems designed for reduced vulnerability. The efficacy of the Live Fire Test law and of the level of mutual intent of the LFT law to contribute to the creation of less distrust that has evolved as a result of these opinions. This vulnerable aircraft designs is valid; its execution to achieve distrust between the various participants of each other’s this intent has been flawed in several ways as identified in motives and actions is probably responsible for the ever- Chapters 2, 3, and 4. The committee believes that the crux of increasing tensions within the current Live Fire Test program. the problems with the LFT law is (1) the ambiguity of the The attitudes of the major participants concerning the Live law; (2) the lack of a clear and binding LFT policy directive; Fire Test law and its place in the acquisition process appear to (3) the reluctance by the Services to—for fear of a stigma10— be those described below. ask for a waiver from full-scale, full-up LFT for those In the committee’s opinion, the Services may believe programs which they believe LFT to be unreasonably that, as system developers and users, they know what is expensive and impractical; and (4) the absence of a formal needed in the equipment they will take into the field, and waiver process that includes a procedure for identifying when full-scale, full-up testing is or is not unreasonably 9 Military airlifters have flown in hostile environments in the past. Examination of the combat data from the Southeast Asia conflict reveals 10 damage and losses to C-7s, C-123s, and C-130s from small arms, AAA, There is also the possibility that the Services do not consider it and shoulder-fired SAMs, and 14 airlifters were damaged by small arms in appropriate for Congress to become involved in this area of program only 400 sorties in 1989 in Operation Just Cause (Ropelewski, 1990). management and acquisition.

OCR for page 42
54 VULNERABILITY ASSESSMENT OF AIRCRAFT • Bond, D.F., 1992. U.S. Army Revamps COMANCHE Plans, expensive and impractical which would eliminate any stigma typesetting files. Page breaks are true to the original; line lengths, word breaks, heading styles, and other typesetting-specific formatting, however, cannot be retained, Considers Next Bid for More Funds, Aviation Week & Space About this PDF file: This new digital representation of the original work has been recomposed from XML files created from the original paper book, not from the original associated with the waiver. Technology, pp. 20–21, March 16. • Ropelewski, R.R., 1990. Planning, Precision, and Surprise Led to Panama Successes, Armed Forces Journal International, February, and some typographic errors may have been accidentally inserted. Please use the print version of this publication as the authoritative version for attribution. References 26–32. • Bennett, Charles E., 1991. Vice Chairman of the Armed Services Committee, Letter to the Honorable Charles E.Adolph, January 2.