Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
5 Vulnerability Assessment Tools Live fire testing can produce data that, when used in conjunction with models having predictive capabilities, will be useful in extending vulnerability assessment to a much greater range of conditions than can practically be tested. Providing data (i.e., input) for predictive analyses is essential because such analyses must be relied on to account for the probabilistic nature of the threat and the marry operating conditions during which a combat aircraft may be hit. Considering the importance of vulnerability assessment tools, in particular the extent to which they complement live fire testing, a review of the array of tools (e.g., documents, data bases, and models) is mandatory for a complete _ _ 1 ~To ~ ^ 1 · ~- , , · ~ ~ ,4 ~ eV~UallOn OI ~ - ~ ~ live sire testing. bUCh a review is the objective of this chapter. ROLE OF TESTING, MODELING, AND DATA BASES IN VULNERABILITY ASSESSMENT The objectives of vulnerability assessment are to identify both mission and vehicle kill mechanisms and to estimate quantitatively the robustness ofthe aircraft to hits from relevant threats. This quantitative information is then used by the designer to produce an optimized vehicle design that takes into account the full range of performance requirements as well as metrics like vulnerability and susceptibility. Cheap kills would be eliminated whenever possible. Accurate vulnerability assessment requires a balance of testing and modeling, aided by information in established data bases. Modeling is integral to quantitative vulnerability assessment since it is only in this manner that large numbers of threat-target interactions can be examined. The term "modeling" is used widely here to encompass analysis arid numerical simulation based on mathematical approximation of structural and fluid mecharucs, combustion, detonation, and other pertinent phenomena, as well as statistical bookkeeping. Accurate vulnerability assessment of complex aircraft requires models at many levels: models of basic physical processes (e.g., fuel spray ignition, composite damage, and hydraulic ram), models of subsystem behavior (e.g., fault trees, response of hydraulics to a severed line, and wing 83
84 Currently, the models used by the vulnerability assessment commun- ity depend on approximations rooted in empirical observations. Thus, mod- els may be improved both by more representative mathematical approx- imation and by more accurate and complete data. Models must be exten- sively tested to establish accuracy and limits to applicability. Testing is essential to aircraft vulnerability assessment. Testing is required for several reasons: (a) to establish the essential relations used in modeling (e.g., fuel flammability and composite failure criteria), (b) to validate models; (c) to verify subsystem response to damage (e.g., hydraulic system response to battle damage, ordnance response to projectile penetration, fuel tank response to internal detonation); and (d) to assess the response of major subassemblies or a complete vehicle to threat damage (e.g., to audit the modeling, investigate hard-to-mode} interactions, and identify failures not predicted by the modeling). Only the last level is controversial, mainly owing to the difficulty of adequate ground simulation of complex flight conditions and the expense it entails. In the committee's opinion, perhaps the most important use of large-scare testing is verification. Current vulnerability modeling is, at best, an art of estimation. Therefore, testing of complete subassemblies is needed to assess the fidelity of the modeling (i.e., to verify that We models produce acceptable results). This testing must be done in a very judicious manner because, unlike an armored vehicle, an aircraft is a relatively fragile structure, easily destroyed in testing. Vulnerability test results cannot stand alone. They must be interpreted through modeling to assess the quantitative impact of the test results on overall aircraft vulnerability. Models synthesize the results of discrete tests to make predictions. If the models that are used to replicate the test results are faulty, then the predictions made by Me models may be incorrect. Verification, validation, and authentication of models are important steps in the vulnerability assessment process. Empirical models are only as good as the test ciata on which Hey are based. Data bases play a distinct role in wInerability assessment in that they form the institutional memory that bridges specific systems, prevents repetitious testing, and avoids the mistakes of the past. As with modeling and testing, vulnerability data bases exist on several levels: data bases of constitutive properties (for use in Live Fire Testing of the F-22 Lee;. . r`;~n a. zc'rz~r. My . I. c;'y. . ..................................................... -.- - - .. . . . ........... ... ... . .. ..... ..... ....... .. ............. ......... ... .. . . . . . .. . . ......... ~ .33 g~ves a e a e ::::::::::::::::::: ::::::. ::::::::::?::::::::::::::: :::: :::::::::: :::::::::: ::::::.::. discussion of Be steps In e ro ss ..... .............. .. ... . ............ ... . . :~erab~ assessme3~. .~ce.~at. Is-. . . ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ..... . . . . ... . . . . . . . . . . . . . .. ... ....... .. . ; .. ....................... i ., . . . . ......... ... .. ... . ..slo:r~. re~esents..~e..generat..~proach..thlten ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . : b. ~Ic be ~ ~ ~ ~ ~ exceeds ~m :~t am: : . .,,, . . : . : .: : . ..rep=~uc.ed.ln App~n~..~to.~:r.ovlde..~r.-. .~ .................. . . . . ester rowers witch aiddlttonal ~oimat~o~i.: : ........................................................................................ - --. ......................................................................................................... .. . .. response to a damaged spar), and bookkeeping models to account for aircraft system response to component damage. Definitions of various types of models appear later in this chapter.
Vuirzerability Assessment Tools 85 models), data bases of engineering practice (for military specifications), and data bases of battle damage (for lessons learned). This information provides both specific data and general guidance for the vulnerability engineer. Accurate vulnerability assessment requires a careful balance of testing and modeling. Neither is sufficient in itself for a modern weapon system. The following sections address the assessment tools currently available to the F-22 community. DOCUMENTATION Many documents have been produced by organizations within the DoD, including the Joint Technical Coordinating Group on Aircraft Survivability (J]CG/AS) and others, that discuss the design of aircraft to reduce their vulnerability to various types of threats. Prime examples include those addressing the use of inerting gas systems in empty fuel tanks and ullage areas, the design of damage-tolerant structures by incorporating dual load paws, and the separation of critical components. Some of these documents were reviewed by the committee; they are listed below to provide the reader with an impression of the type of information that is available. . . Military Standard, Survivability, Aeronautical Systems (For Combat Mission Effectiveness) (DoD, 1986); Military Standard, Aircraft Nonnuclear Survivability Terms (DoD, 1981a); Military Standard, Requirements for Aircraft Nonnuclear Survivability Program (DoD, 198Ib), Military Handbook, Survivability, Aircraft, Nonnuclear, General Criteria Volume ~ (DoD, 1982); Military Handbook, Survivabilitv Aircraft Nonnuclear Airframe_ Volume 2 (DoD, 1983a) , , , ~, , ~ Military Handbook, Survivability, Aircraft Nonnilc~le~r F.n~in~.- Volume 3 (DoD, 1983b), arid Aircraft Fuel System Fire and Explosion Suppression Design Guide (Mowrer et al., 1990). , . ~_ _ , .. _ The committee noted that, although there are numerous military standards, handbooks, and design guides available, many of these documents are 1 0 or more years old and have not been updated since they were developed and promulgated. The committee believes that many of these documents need to be updated and improved.
86 Live Fire Testing of the F-22 DATA BASES Designers are aided by vulnerability engineers who use vulnerability assessment models to identify and address system vuInerabilities. Vulnerability engineers and live fire test planners use vulnerability assessment models to help identify areas of the aircraft that require live fire testing. Both processes depend on confidence in the data bases that support the models. The more that tests are conducted on a certain component, the better the understanding of what that component's damage and failure modes watt be. If a statistically significant number of tests are run, the designers, engineers, and testers watt have greater confidence in the resulting data base. This example extends to all the components on the aircraft and their data bases. The Joint Live Fire Test Program managed by ~TCG/AS has done some of this testing in the past, and the results have been very beneficial. Unfortunately, the program has been significantly underfunded. This situation has resulted in insufficient component tests being accomplished to allow confidence in the data bases for existing components and materials. In addition, there are many new composite materials, engines, stealth techniques, weapons, and other advances that wait require significant additional testing to ensure that the damage and failure modes are understood and confidence in the data bases is warranted. A building block vulnerability testing approach (i.e., building up from materials to components to subsystems to major subassemblies), with enough tests to produce statistically meaningful results, is necessary to develop the knowledge base to a point where designers, engineers, and testers will have high confidence in the data bases. This effort would generate results that could be useful for all aircraft programs. The committee believes that this approach could provide significant savings in the live fire test program for individual aircraft. Live fire testing would still have to be accomplished for each new aircraft because each design is different. However, less testing would be necessary for each new aircraft because the confidence level in the supporting data bases would be higher. MODELS Modeling is essential for the analysis and assessment of vulnerability. The virtually infinite possibilities for threat-target interactions (e.g., types of threat, intercept geometries, and target conditions) demand a modeling capability to extend the limited number of experiments arid tests that can be conducted. Likewise, the design for reduced vulnerability requires an understanding of the processes involved that is reflected in the ability to model.
Vulnerability Assessment Tools 87 .... : ~he ~inol~ ~0 n d } I in to ai i s ..~llowin~. et o e s i e e . . ~. ~. , ~ ..... , .......... ........ . ~.............. ...... ..... ............. . ~0ae~ . ~mame~ca cow-~ ~a : ~ 365 ~ ~ ~ySt¢~} pi ess or ~ptex some ce . ::::::- . :::::::: ::::r:: ::::::i :~: :::. ::. ::::::~::::~::::::::;:::;:::;:;:::::::::;::::::: :::::::::::: ::::::::: ::::::::::::: ::: ~ oseararm Ago- ~ mooe t At involves a met ~1= equation toees.~n ~ he nn=o · . :~.~.r.~mst~c .mo~:::~oae~. .~at.pro~uces ~ denn~teresult ~= .~.~lon amber. .: : : pr.~tle estimate (~ O-'s law.), :::: :::::: :::::: :: .:::::::::::::::::::::::::: ::::::::::::::: .::: ~ ~ . :':::::::: :-:::: :-:::::::::: I::::::: :-::::::::::::::::::::::: : . ~i"~. ~. ~A mod l .:1 si a .- . . .: . . : ; : , . ~.bl ~=s w ~.~hem~.to.des:~be ~.ac`~.~:~calm harems ~ 1' 'd : ............ e e e e. Acorn .. . . . . , , , .,.,., , ,.,.,., ,., ; - - - - - - : ~: ~u~ri~ - s~ ~. ~ ~ ~ e n a . s ~ s e g .. .. .. ......... .. . . - ~n~r~rusnu~-u~ ;l~f~u~. ~ lIli3~l or ~ pl}~slca] prUce5s rn~:.ls su00~mate ED ~ co~iD:lete .. .... ... . . . - .. . . .. .......... ......... ~t wb~blilW ~a. ~t ~e .~ne=~n c~ab~lu~ ct ~ buDeb . . ~obabi~ Mo~:. del s s es : s i i :: : . : ,. ,,,. , ,:. ...................... o c - ~e, m~ v~ ~ ~ (~., ~ u - , ej c81n g~s nb~ds' b~ j~e 0~. ... ~.h~le M ~prob~h modes ~ u~s re etlt1 ~ . al~ .~: wi~ ~m ~ hn ............ .~. ~ c~., , r~.n~ ~~ ~ ~ ~ .~.~.I.~ ~: .::: .: .::::::::: :~::::::::::::::::::: :::::.:::: :a c~::w .~L,l::~::iV1V ;~L~::~ :l\:J::ll:~Ll~;l:l.::::::::::::::::: ~: lt ~s ~mportant to remember that the modeling process involves continual iteration between experimentation and calculation. As in other scientific endeavors, the mode] represents the hypothesis to be tested by the experiment, and the experiment represents real data upon which to build or modify the model. Models used for vulnerability assessment are developed both by joint organ~zations such as the ~TCG/AS and by the services. For example, ~e ITCG/AS is developing a physics-based mode! for use in predicting fires in dry bays adjacent to fuel tanks. There are plans to expand the mode] (e.g., to encompass flammable fluids other than fi~e! and to account for ulIage explosions) as fi~nds are made available (I~at~z~e, 19951. The Air Force has also developed some models for its specific vuInerability assessment of the F-22 (see discussion below). Phenomenolog~cal Models It is no exaggeration to say that the validity of an encounter mode! is only as good as the phenomenological models that it employs. These models can be in closed form or can be elaborate numerical-analysis models.
88 Live Fire Testing of file F-22 Since the processes involved tend to be quite complicated physically, the closed-form models are often empirical fits to experimental data, and many times they must be expressed in probabilistic terms. Extensive experimentation is implied, both to cover the range of parameters involved and to develop a · . . . · . . - . - . . - . - . ~ ~ . . . - representat~on that has statistical validity. experimentation nas neen the main approach for obtaining penetration and component-damage data. In the committee's judgment, the vulnerability community has unfortunately not taken full advantage of advances in finite-element and hydrocode analysis even though these tools would allow a more complete analysis and a considerable reduction in testing. There seem to be two reasons: A lack of basic data on the properties of materials that are subjected to extremely high pressures and high rates of loading. The large computational times required to calculate individual cases for the deformation of complex structures. The committee believes that Me vulnerability community could do much more to use these advanced tools by (a) developing the necessary data on the materials (especially composite materials), (b) making We lengthy calculations for important instances of damage to structures, and (c) using the results to calibrate and validate simpler empirical models. There is also a need to develop codes using these tools so that the combination of stress, hydrodynamic, and thermal effects can be considered. The notion of using finite-element analysis to solve complex and combined stress, fluid, and thermal problems is not a remote future option. This type of modeling is already being used by the designers of nuclear weapons and the aerospace and automotive industries. While no one would argue that the science is mature and fully developed, the tools are available to gain significant Insight into problems such as the vulnerability of aircraft. The models that to date have most fully combined these analyses concern nuclear weapons in abnormal environments (e.g., fires and crashes). The F-22 and ITCG/AS commuruties would be well advised to explore using the methodologies already perfected and still undergoing refinement In We U.S. nuclear weapon design laboratories. The committee judges that a relatively large modeling uncertainty related to the F-22 is the inability to replicate the response of its new composite materials, this has already led to one surprise regarding the effects of hydraulic ram (see Chapter 4~. While that particular problem has been fixed, there is always the possibility of more surprises. · . . - ~. . . ~.
Vulnerability Assessment Tools 89 Encounter Models Almost by definition, encounter models are probabilistic. Initial conditions are quite variable, phenomena such as fragmentation are random, and the results are expressed as probabilities. While the events portrayed are probabilistic, the models usually calculate the probabilities in a deterministic way (i.e., by accepted probability rules). However, the Army is working with stochastic models (Deitz, 995) toward achieving better interfacing with the results of live fire tests. Models Used by the F-22 System Program Office The F-22 SPO uses the following models in its work: . FASTGEN 3 projects parallel rays through the target and describes intersections with aircraft components (Cramer and Hilbrand, 1985~. FASTGEN 4 is similar to FASTGEN 3 but can use target information from a finite-element structural analysis mode] used in design. It was developed by members of the F-22 SPO (Griffis and Lentz, 1994~. COVART 4.0 (Computation of Vulnerable Area and Repair Time) determines-vuInerable areas for single kinetic-energy penetrators (or fragments) and high-explosive rounds. It was developed by the F-22 SPO (Bionetics Corporation, 1995~. This extension of COVART 3.0 allows the incorporation of effects of small high-explosive rounds and includes special penetration equations for high-speed fragment impacts. Component defeat probabilities are calculated for each shot line and combined for target defeat probabilities for the individual fragment or round. The Blast Overpressure Analysis model is used to determine aircraft vulnerability to blast overpressure from conventional missile warheads. It is based on nuclear blast methodology that has been adapted to simulate the effects of smaller conventional warheads (Smith and Stewart, 1986).2 The SHAZAM computer program is used to evaluate the effectiveness of art air-intercept missile by describing the terminal phase of the encounter. The program determines missile fizzing arid detonation positions arid calculates target damage sequentially from prioritized kill 2 Regarding this model, the committee notes that the documentation it received was infonnal and of relatively poor quality, and the methodology is not used in the vulnerability community at large.
9o Live Fire Testing of the F-22 mechanisms (e.g., direct hit, blast overpressuxe, and fragment impacts) (Moore et al., 1994). The ESAMS (Enhanced Surface-to-Air Missile Simulator) simulates an encounter between Me target and a radar-guided surface-to-air missile. It provides a one-on-one framework in which to evaluate air vehicle survivability and optimization of tactics (BDM International, 1991). With the possible exception of ESAMS, the committee understands that none of these models has been formally validated or accredited by either the ITCG/AS or the Joint Technical Coordinating Group on Munitions Effectiveness. Although the models represent the current state of the art of vulnerability analysis, it appears that much of their development has been accomplished by the services without the participation of the joint groups. Large-Scale Effects A major argument given for the need to perform tests on large assemblies or a foul-up, f~l-scale aircraft is the inability of the vulnerability models to analyze adequately large-scale effects.3 The encounter models merely reflect the capabilities of the phenomenological models, and the phenomenological models that are currently available tend to represent localized effects. This state of affairs reflects the empirical nature of these models and the impracticality of conducting an adequate experimental program to allow them to define completely large-scale damage. As was discussed in Chapters 3 arid 4, this committee believes that testing of ~ fi~-un fi~-scale aircraft does not at least in the case of a very expensive ¢, , aircraft like the F-ZZ, provide benefits judged worthy ot the costs and wade My not yield useful data, however, testing at the subassembly level is worthwhile. Yet tests at the large subassembly level are more expensive than tests at Me subsystem arid component levels arid less able to be repeated marry times to yield statistically significant results. Valid large-scare vulnerability models could provide art efficient mear~s of making sound judgments about large-scale effects without the need for expensive and repetitive live fire tests on large subassemblies. Therefore, if models could be 3 Large-scale effects could include the following: the effects of stress propagation in large portions of the target, interaction of damage with flight loads on the aircraft, propagation of fire and ignition, and response of loaded weapon bays to damage. Of course, very large effects that would simply destroy the aircraft do not need to be modeled at this level of detail.
Vulnerability Assessment Tools 91 developed that would lead to correct conclusions regarding large-scale effects, this committee would filthy endorse that effort. CONCLUSIONS To be successful, vulnerability assessment requires much mutual support between documentation, data bases, models, and testing. This mutual support is as necessary for the F-22 program as for arty other weapon system. The committee's review of the F-22 vulnerability assessment program indicated that significant improvements are needed in several of the tools that complement live fire testing. With respect to documentation, most of the documents reviewed by the committee need to be updated and improved. I~runediate attention to this matter by the DoD is warranted. There is considerable need for expanded efforts over the next several years to improve the data bases used in models for conducting vulnerability assessments arid planning live fire tests. Aggressive funding of joint live fire tests would enhance the data bases and could provide a major payoff for both the vulnerability reduction design and the live fire test of individual aircraft programs like the F-22. Formal validation and accreditation, by the ~TCG/AS and the Joint Technical Coordinating Group on Munitions Effectiveness, of models used by the Air Force and other services is warranted. The vulnerability community could make much better use of advanced analytical tools (e.g., finite-element analysis), especially in connection with understanding the response of F-22 composite materials to ballistic damage. Also, there is great merit to the development and exercise of numerical-analysis tools that watt provide a better understanding of large-scale effects.
92 Live Fire Testing of the F-22 REFERENCES BDM International. 1991. ESAMS Version 2.5 Users Manual, BDM/ABQ- 89-0587-TR-R2. Albuquerque, N.M. April 12. Bionetics Corporation. 1995. COVART 4.0 User Manual (Draft). Wright-Patterson Air Force Base, Ohio: F-22 System Program Office. Cramer, R.E., and Hilbrand, R. 1985. FASTGEN 3 User's Manual. Wright- Patterson Air Force Base, Ohio: Aeronautical Systems Division, Air Force Systems Command. Deitz, P. ~ 995. Army Vulnerability Testing Results Methodology and Modeling. Presentation to the Committee on the Study of Live Fire Survivability Testing of the F-22 Aircraft, National Academy of Sciences, Washington, D.C., February 16. DoD (U.S. Department of Defense). 1 98 la. Military Standard. Aircraft Nonnuclear Survivability Terms. MIL-STD-2089. Washington, D.C.: DoD. DoD. 198Ib. Military Standard. Requirements for Aircraft Nonnuclear Survivability Program. MIL-STD-2069. Washington, D.C.: DoD. DoD. 1982. Military Handbook. Survivability, Aircraft, Nonnuclear, General Criteria Volume i. MIL-HDBK-336-~. Washington, D.C.: DoD. DoD. 1983a. Military Handbook. Survivability, Aircraft, Nonnuclear, Airframe- Vol~ne 2. MIL-HDBK-336-2. Washington, D.C.: DoD. DoD. 1983b. Military Handbook. Survivability, Aircraft, Nonr~uclear, Engine- Volume 3. MIL-HDBK-336-3. Washington, D.C.: DoD. DoD. 1986. Military Standard. Survivability, Aeronautical Systems (For Combat Mission Effectiveness). MIL-STD-1799. Washington, D.C.: DoD. Griffis, H., and Lentz, M. 1994. FASTGEN 4.] User's Manual. XRESV Tech Note 94-01. Wright-Patterson Air Force Base, Ohio: Headquarters Aero- nautical Systems Center. La~'z~e, R. 1995. Personal communication to Dale Atkinson, June 30.
Vulnerability Assessment Tools 93 Moore, C., D. Roberts, I. Martin, and H. Griffins. 1994. SHAZAM 2.0 User's Manual. XRESV Tech Note 94-02. Wright-Patterson Air Force Base, Ohio: Headquarters Aeronautical Systems Center. Mowrer, D.W., R.G. Bernier, W. Enoch, R.E. Lake, and W.S. Vikestad. 1990. Aircraft Fuel System Fire and Explosion Suppression Design Guide. Aberdeen, Md.: Survice Engineering Co. NRC (National Research Cour~cil). 1993. Vulnerability Assessment of Aircraft: A Review of the Department of Defense Live Fire Test and Evaluation Program. Air Force Studies Board, NRC. Washington, D.C.: National Academy Press. Smith, R.D., and W.M. Stewart. 1986. Documentation for the Blast Overpressure Analysis Model. Ft. Worth, Tex.: Lockheed Corp.
