3
The OSD Live Fire Test and Evaluation Program

The Live Fire Test law passed in 1987 stipulates that realistic survivability testing be conducted on covered systems. Realistic survivability testing is defined as “testing for vulnerability of the system in combat by firing munitions likely to be encountered in combat (or munitions with a capability similar to such munitions) at the system configured for combat, with the primary emphasis on testing vulnerability with respect to potential user casualties and taking into equal consideration the susceptibility to attack and combat performance of the system” (U.S. Congress, 1986–1989). Systems covered by the law are vehicles, weapon platforms, or conventional weapon systems that include features designed to provide some degree of protection to users in combat and are major systems within the meaning of that term in section 2303(5) of title 10, i.e., Acquisition Category I and II systems. The Secretary of Defense may waive the requirement for Live Fire Testing of candidate systems if, before the system enters full-scale development, the Secretary “certifies to the Congress that such testing would be unreasonably expensive and impractical” (U.S. Congress, 1986–1989). The request for a waiver will be prepared by the Service and submitted to the Secretary through the appropriate chain of command. If the waiver is granted, the Secretary shall include with any such certification a report explaining how the Secretary plans to evaluate the survivability or the lethality of the system or program and assessing possible alternatives to realistic survivability testing of the system or program (U.S. Congress, 1986–1989).

Personnel from the Live Fire Test Office and from the Deputy Director, Defense Research and Engineering (Test and Evaluation/Tactical Warfare Program) [DDDR&E(T&E)/TWP] presented the Office of the Secretary of Defense (OSD) Live Fire Test and Evaluation (LFT&E) program to the committee on July 24, 1991, and the Director of the Live Fire Office made a second presentation to the committee on January 15, 1992. Personnel from the Institute for Defense Analyses gave presentations to the committee on September 26, 1991.


Oversight and Published Guidance for LFT&E. When the Live Fire Test law was passed in 1987, the position of Director, Live Fire Testing, was established under the Office of the Director of Defense Research and Engineering (Test and Evaluation) and was given the responsibility for implementing the Live Fire Test (LFT) legislation, developing and issuing guidelines for test plan preparation, reviewing and approving Service-prepared plans, performing independent assessments of the test results, providing liaison with Congress on the test results, and working with the Services and the Department of Defense (DoD) in improving LFT methodology and instrumentation, and in general, ensuring



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 38
typesetting files. Page breaks are true to the original; line lengths, word breaks, heading styles, and other typesetting-specific formatting, however, cannot be retained, About this PDF file: This new digital representation of the original work has been recomposed from XML files created from the original paper book, not from the original and some typographic errors may have been accidentally inserted. Please use the print version of this publication as the authoritative version for attribution. The OSD Live Fire Test and Evaluation Program 3 The Live Fire Test law passed in 1987 stipulates that realistic program and assessing possible alternatives to realistic survivability testing be conducted on covered systems. survivability testing of the system or program (U.S. Congress, Realistic survivability testing is defined as “testing for 1986–1989). vulnerability of the system in combat by firing munitions Personnel from the Live Fire Test Office and from the likely to be encountered in combat (or munitions with a Deputy Director, Defense Research and Engineering (Test capability similar to such munitions) at the system and Evaluation/Tactical Warfare Program) [DDDR&E(T&E)/ configured for combat, with the primary emphasis on testing TWP] presented the Office of the Secretary of Defense (OSD) vulnerability with respect to potential user casualties and Live Fire Test and Evaluation (LFT&E) program to the taking into equal consideration the susceptibility to attack committee on July 24, 1991, and the Director of the Live Fire and combat performance of the system” (U.S. Congress, Office made a second presentation to the committee on 1986–1989). Systems covered by the law are vehicles, January 15, 1992. Personnel from the Institute for Defense weapon platforms, or conventional weapon systems that Analyses gave presentations to the committee on September include features designed to provide some degree of 26, 1991. protection to users in combat and are major systems within the meaning of that term in section 2303(5) of title 10, i.e., Oversight and Published Guidance for LFT&E. When the Acquisition Category I and II systems. The Secretary of Live Fire Test law was passed in 1987, the position of Director, Defense may waive the requirement for Live Fire Testing of Live Fire Testing, was established under the Office of the candidate systems if, before the system enters full-scale Director of Defense Research and Engineering (Test and development, the Secretary “certifies to the Congress that Evaluation) and was given the responsibility for implementing such testing would be unreasonably expensive and the Live Fire Test (LFT) legislation, developing and issuing impractical” (U.S. Congress, 1986–1989). The request for a guidelines for test plan preparation, reviewing and approving waiver will be prepared by the Service and submitted to the Service-prepared plans, performing independent assessments Secretary through the appropriate chain of command. If the of the test results, providing liaison with Congress on the test waiver is granted, the Secretary shall include with any such results, and working with the Services and the Department of certification a report explaining how the Secretary plans to Defense (DoD) in improving LFT methodology and evaluate the survivability or the lethality of the system or instrumentation, and in general, ensuring 38

OCR for page 38
OSD LIVE FIRE TEST AND EVALUATION PROGRAM 39 that conduct of Live Fire Testing is within the spirit of on LFT&E provided by the LFT office to the Services and typesetting files. Page breaks are true to the original; line lengths, word breaks, heading styles, and other typesetting-specific formatting, however, cannot be retained, About this PDF file: This new digital representation of the original work has been recomposed from XML files created from the original paper book, not from the original congressional intent (O’Bryon, 1987). A series of written industry. The Planning Guide was intended to provide a good documents has been prepared by OSD to provide guidance foundation for understanding the LFT&E program, and to be regarding details not specifically contained in the law. useful to system program offices and test agencies responsible The first set of guidelines intended to implement the for the testing and evaluation of systems identified as Live and some typographic errors may have been accidentally inserted. Please use the print version of this publication as the authoritative version for attribution. congressionally mandated Live Fire Test program were issued Fire candidates; nothing in the Planning Guide was intended on May 7, 1987 (DDDR&E, 1987). These guidelines to be inconsistent with the Live Fire legislation or any related supplemented DoD Directive 5000.3-M-1, “Test and DoD directives. The Planning Guide contains the TEC LFT&E Evaluation Master Plan Guidelines,” October 1986, in areas guidelines of 1988; the Live Fire Testing legislation; and pertaining to Live Fire Testing. They also defined the LFT&E guidance concerning the definition of critical issues for Live plan requirements. According to these guidelines, “the essential Fire testing, the development of a strategy for LFT&E, the feature of Live Fire Testing is that threat munitions are fired preparation of a detailed LFT&E plan, and the integration against a major U.S. system configured for combat to test its process involving contracting, budgeting, and scheduling. vulnerability, and/or that a major U.S. major munition or missile According to the Planning Guide, the intent of LFT&E (with is fired against a threat target configured for combat to test the regard to vulnerability) is “to assure that battle damage lethality of the munition or missile. Each (L)ive (F)ire (T)est tolerance and damage control of our crew-carrying combat plan must include testing of complete systems. However, a systems to actual threat weapons is known and acceptable…”. limited set of live fire tests may involve production components configured as a subsystem prior to full up testing. In such a case the components must be tested in the context of the complete The LFT Program Approach system in that the test issues must be specific to both a threat system and a target system. In addition, at least a preliminary In concert, the LFT&E Guidelines of 1988 and the LFT&E decision shall have been made to configure the production Planning Guide of 1989 require the services to take an ordered system with the components tested. Thus, (L)ive (F)ire (T)esting approach to LFT&E on all covered aircraft systems. This is not synonymous with traditional research and development approach includes the following steps: vulnerability/lethality testing” (DDDR&E, 1987). 1. identification of the critical issues; The 1988 Test and Evaluation Committee (LFT&E) Guidelines. 2. development of an LFT&E strategy, which must be an The 1987 LFT&E Guidelines were superseded by the Live integral part of the Test and Evaluation Master Plan (TEMP) Fire Test and Evaluation Guidelines issued June 1, 1988, by and is subject to OSD review and approval; the Test and Evaluation Committee (TEC), Office of the 3. integration of this strategy into the covered system Secretary of Defense, in a memorandum to the Assistant program plan, budget, and schedule; Secretary of the Army (Research Development and 4. development of a detailed LFT&E test plan that fully Acquisition), the Assistant Secretary of the Navy (Research describes the tests to be conducted and criteria for measuring Development and Acquisition), and the Assistant Secretary of test results and is subject to OSD review and comment; the Air Force (Acquisition) (OSD, 1988). The TEC 5. conduct of the tests; and memorandum stated that the enclosed guidelines implemented 6. generation of a detailed test report subject to OSD review the congressionally-mandated Live Fire Test program within for submission to the Secretary of Defense and the congressional the DOD. The guidelines had been reviewed and coordinated committees. by the individual Service deputies responsible for test and evaluation. According to the memorandum, “the LFT program The identification of the critical issues is required to form a is a highly visible and sometimes misunderstood program. foundation upon which a strategy for the Live Fire Test These guidelines should eliminate much of that program can be developed for each system. To fully misunderstanding” (OSD, 1988). The guidelines describe the understand the system and the threat, information that fully objectives of LFT&E, the scope of the guidelines, the describes the requirements for the system, the environment in implementation of the program, and the responsibilities of the which it must operate, and concepts for operation of the DoD staff and the Services relative to LFT&E. system must be gathered. Vulnerability analyses must then be conducted to identify potential weaknesses in the system and The 1989 LFT&E Planning Guide. A Live Fire Test and to obtain first-order assessments of the ability of the system to Evaluation (LFT&E) Planning Guide was issued in June 1989 meet its operational requirements. Given this information, by the Director, Live Fire Testing (DDDR&E, 1989). This vital vulnerability concerns can be identified and distilled Planning Guide is the current primary source of information into the critical Live Fire Test

OCR for page 38
40 VULNERABILITY ASSESSMENT OF AIRCRAFT issues. Depending upon the system, there are several sources Controversy Regarding the Definition of typesetting files. Page breaks are true to the original; line lengths, word breaks, heading styles, and other typesetting-specific formatting, however, cannot be retained, About this PDF file: This new digital representation of the original work has been recomposed from XML files created from the original paper book, not from the original for Live Fire Test issues. For example, if the system is designed “Realistic Survivability Testing” to replace an existing system, a Live Fire Test issue would be, How does the vulnerability of the new system compare with The committee is concerned that the written guidance does that of the existing system? and some typographic errors may have been accidentally inserted. Please use the print version of this publication as the authoritative version for attribution. not provide sufficient detail, particularly with respect to the The next step in the planning process is the development of full-scale tests, to ensure that the Program Manager can satisfy a strategy for conduct of the Live Fire Test program. This the requirements of the OSD policy and the law and also strategy shall include the establishment of measures of design a cost-effective test plan that will ensure system evaluation, procedures for the evaluation of the results, the requirements are satisfied. Furthermore, the committee is data requirements, and the test objectives. This strategy shall concerned about the official status, or lack thereof, of the be a part of the Test and Evaluation Master Plan. LFT&E Guidelines and Planning Guide.1 Provision must next be made for the integration of the Live In addition to the questions regarding the level of detail Fire Test strategy into the program plan for the development of and the current status of the guidance, there is a question the system. The Program Manager will ensure that the overall regarding the interpretation of specific mandates contained in program plan, budget, and schedule provide adequate the Live Fire Test law. A major issue apparently exists between resources to assure a successful Live Fire Test program. the Services and OSD, and indeed within OSD itself, regarding A detailed Live Fire Test plan is required that fully the definition and meaning of “realistic survivability testing” describes the tests to be conducted and the criteria for given in the law. The definition has been interpreted by the evaluating test results. This test plan shall be in sufficient 1988 Live Fire Test and Evaluation Guidelines as follows: detail to ensure that the tests will satisfy the strategy a. Live Fire Test: A test event within an overall LFT&E program objectives. The test plan is subject to review and comment by which involves the firing of actual munitions at target components, the Director, Live Fire Test. target subsystems, target subassemblies, and/or full-scale targets The conduct of the tests is to be witnessed by the Live Fire to examine personnel casualty, vulnerability, and/or lethality issues. Test Office, which does an independent evaluation of the b. Full-up Testing: Firings against a full-scale target results. The Services prepare a test report that is reviewed by containing all the dangerous materials (e.g., ammunition, fuel, hydraulic fluids, etc.), system parts (e.g., electrical lines with the Live Fire Test Office. The test report is then forwarded to operating voltages and currents applied, hydraulic lines the Secretary of Defense with the independent OSD containing appropriate fluids at operating pressures, etc.), and evaluation of the results, and from there it is sent to the stowage items normally found on that target when operating in interested congressional committees. combat. Full-up testing includes firings against fullup The 1988 LFT&E Guidelines and the 1989 LFT&E components, full-up subsystems, full-up subassemblies, or full- up systems. The term “Full-up Testing” is synonymous with Planning Guide are the current written guidance provided to “realistic survivability testing” or “realistic lethality testing” the Services and their Program Managers to assist them in the as defined in the legislation covering Live Fire Testing. planning and conduct of a Live Fire Test program that is guaranteed to satisfy the requirements of the OSD and, The phrasing of these definitions has been interpreted by through the law, the Congress. In addition to this written some to imply that full-scale, full-up Live Fire Tests do not material, the staff of the Live Fire Test office is available for have to be conducted (i.e., OSD policy is satisfied by Live Fire regular consultation with the program offices in the Tests only on sub-scale targets, such as major portions or preparation of the strategy for the TEMP and of the detailed subassemblies of an aircraft). The essence of the problem is the test plan. Other methods used by the LFT Office to phrase in the law that states that realistic survivability testing communicate the LFT program to the testing and evaluation means “testing for vulnerability of the system by.” What is the (T&E) and acquisition communities include full membership “system?” Does testing only one major subassembly for the LFT Director on the Defense Acquisition Board constitute testing for vulnerability of the system? If it does, as Committees, periodic one-on-one discussions with the some believe, then no waiver from the mandated system Live Services’ Live Fire Test gate keepers, introduction of the Fire Testing is necessary. program into the curriculum of the Defense System Management College, preparation of a video containing an overview of the program, public testimony before congressional defense subcommittees, sponsorship of six Live Fire Lessons Learned Workshops, participation in more than 25 T&E symposia, more than 40 on-site overview 1 If action is taken to assign official status to these documents, they should first be modified to ensure that there is no ambiguity in the question-and-answer presentations to the major defense requirements for full-scale testing (see discussion below). One briefer contractors, and publication in the open literature of more indicated to the committee that guidelines were good for only one year than 30 articles on Live Fire Testing (O’Bryon, 1991). and, beyond that, the “official status” would not be valid.

OCR for page 38
OSD LIVE FIRE TEST AND EVALUATION PROGRAM 41 Although these definitions in the LFT&E Guidelines are tests of this section to a covered system, munitions program, or typesetting files. Page breaks are true to the original; line lengths, word breaks, heading styles, and other typesetting-specific formatting, however, cannot be retained, About this PDF file: This new digital representation of the original work has been recomposed from XML files created from the original paper book, not from the original apparently the official guidance on realistic survivability missile program if the Secretary, before the system enters full- testing provided to the Services’ Program Managers, the Live scale engineering development, certifies to Congress that Fire Test Office does not fully accept them. It believes the live-fire testing of such system or program would be Guidelines make too liberal an interpretation of the intent of unreasonably expensive and impractical” (U.S. Congress, and some typographic errors may have been accidentally inserted. Please use the print version of this publication as the authoritative version for attribution. the Congress. It believes that the law requires the testing of 1986–1989). If a waiver is granted, the law states that “the complete systems, not just sub-systems or mock-ups, fully Secretary shall include with any such certification a report configured for combat. However, it appears to recognize that explaining how the Secretary plans to evaluate the this can be unreasonably expensive and impractical, and that survivability or the lethality of the system or program and circumstances exist in which the testing of only subscale assessing possible alternatives to realistic survivability targets is justified. In those situations, it believes a waiver testing of the system or program” (U.S. Congress, 1986– should be requested. 1989). Thus, requesting and receiving a waiver from the The committee believes, as stated in Chapter 1, that the requirement for a full-scale, full-up test program do not Live Fire Test law requires that full-scale, full-up tests be eliminate the requirement for vulnerability assessment; only conducted on covered systems, unless a waiver is granted. for the full-scale, full-up testing portion of the assessment. Furthermore, it believes that the definitions in the 1988 Guidelines and the guidance given in the 1989 Planning Guide are not sufficiently clear as to the law’s requirement that References full-scale, full-up testing must be conducted. As a • Deputy Director, Defense Research and Engineering (Test & consequence of this misunderstanding, the Services have Evaluation), 1987. Live Fire Test and Evaluation (LFT&E proceeded with sub-scale Live Fire Test programs on several Guidelines), Live Fire Testing Office. weapon systems without making a provision for testing a • Deputy Director, Defense Research and Engineering (Test & complete system and without asking for a waiver because of Evaluation), 1989. Live Fire Test and Evaluation Planning Guide, the belief that no full-scale, full-up testing was required if Live Fire Testing Office. early tests on sub-scale targets showed no design weaknesses.2 • O’Bryon, James F., 1987. Live Fire Testing: the Legislation and Its Impact, Army Research, Development, & Acquisition Bulletin, May- June. • O’Bryon, James F., 1991. Presentation made to the Committee on Conclusion Weapons Effects on Airborne Systems, July 24. • Office of the Secretary of Defense (OSD), 1988. Live Fire Test and The committee believes that the Services, when they consider Evaluation Guidelines, Test and Evaluation Committee, Memorandum that full-scale, full-up Live Fire Tests are unreasonably for Assistant Secretary of the Army (Research Development and expensive and impractical, should ask for a waiver. There Acquisition), Assistant Secretary of the Navy (Research should be no stigma attached to a waiver if a strong case can be Development and Acquisition), and Assistant Secretary of the Air made for one. According to the law, “the Secretary of Defense Force (Acquisition). may waive the application of the survivability and lethality • U.S. Congress, 1986–1989. Survivability and Lethality Testing of Major Systems, DoD Authorization Acts, FY86—Sec. 123, FY87— Sec. 910, FY88–89—Sec. 802. 2 These programs are examined in Chapter 4.