|
||||||||||||||||
|
||||||||||||||||
|
||||||||||||||||
|
| ||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 1
Executive Summary
The Air Force's newest combat aircraft, the F-22, is designed principally for
offensive counter-air missions. Advanced technologies, including stealth, are
intended to enable it to penetrate deeply into hostile airspace and shoot down
threatening aircraft before they can detect the F-22. If detected, the F-22 has the
countermeasures, speed, and maneuverability that should minimize the likelihood
of its being hit.
The National Defense Authorization Act for fiscal year 1995 directed the
Secretary of Defense to request the National Research Council to study the
desirability of waiving the live fire tests that are required by law for the F-22. The
Committee on the Study of Live Fire Survivability Testing of the F-22 Aircraft
was formed by the National Research Council to conduct the study.
The committee began its work in December 1994. Several data gathering
meetings were held in Washington, D.C., one was at Wright-Patterson Air Force
Base, Dayton, Ohio, which is the location of the F-22 System Program Of lice and
the Wright Laboratory, where many Air Force live fire tests are conducted, and
one was at the Naval Air Warfare Center located at China Lake, California, where
the Navy conducts similar tests. These meetings exposed the committee to the filet
spectrum of views involving live fire testing of fighter aircraft.
The work of a previous National Research Council committee, reported in
the 1993 publication Vulnerability Assessment of Aircraff: A Review of the
Department of Defense Live Fire Test and Evaluation Program, was invaluable to
the present committee. The previous committee confirmed that, unless waived by
the Secretary of Defense because of urlreasonable expense and impracticality, live
.
~ According to the Air Combat Command, which is the operational user of the F-22, there are
two kinds of counter-air missions. The first, known as offensive counter air, is to "penetrate deep
into heavily defended hostile airspace and destroy threat capability." The second, defensive counter
air, is to "detect, identify, intercept, and destroy Meat aircraft penetrating friendly airspace."
(W.S. Hinton, "Threat, Mission, and Operational Requirements for the F-22," presentation to the
Committee on the Study of Live Fire Survivability Testing ofthe F-22 Aircraft, Washington, D.C.,
December 21, 1994.) For the purpose of this report the committee uses the definition of offensive
counter air that involves only air-to-air engagements.
OCR for page 2
2
.
Live Fire Testing of the F-22
fire tests known as full-up, full-scale tests are required for major weapon systems
like the F-22.
Full-up, full-scare tests would subject an F-22 to live fire in its combat
configuration, including on-board ordnance and fuel. These tests would use
munitions likely to be encountered by the F-22 in combat. There is currently no
doubt within the Department of Defense about this requirement or the need for a
waiver if such tests are not to be performed.
If the tests are waived, the law permits the Secretary of Defense to allow
Instead live fire tests against components, subsystems, and subassemblies, together
with design analyses, modeling and simulation, and analysis of combat data. Along
with any waiver, the Secretary must report how system survivability will be
evaluated and assess possible alternatives.
Circumstances associated with a waiver for the F-22 are unique in one
respect. Normally, if a waiver is to be granted, the Secretary of Defense must act
before a weapon system enters the phase of acquisition known as engineering and
manufacturing development. The decision point for this phase is called Milestone
Il. Milestone I} for the F-22 occurred in 1991 without a waiver being requested.
In 1993, the Department of Defense, along with developing an alternative test
plan, requested a change to the law that would permit the F-22 to be granted a
retroactive waiver. That request prompted members of Confess to call for this
study.
A,
The remainder of this summary contains (a) an overview of the principal
findings that resulted from this study, followed by (b) a more detailed presentation
of the committee's major conclusions arid recommendations.
PRINCIPAL FINDINGS
The committee's principal findings concern three topics: (1) practicality and
cost-benefit issues related to live fire testing of the F-22, (2) sufficiency of the
current vulnerability assessment program for the F-22, and (3) vulnerability
assessment tools.
Practicality and Cost-BeneE~t
The committee believes that live fire testing can be conducted at four levels
of realism for the intended mission.2 From lowest to highest, these four levels are
(~) hardware simulations or mock-ups of production systems; (2) full-scale
2 In the case of the F-22, the primary mission is offensive counter air, in which the Air Force
expects very few encounters to result in enemy missiles or guns being fired at the aircraft.
OCR for page 3
Executive Summary
3
components, subsystems, or major subassemblies representative of production
items; (3) a complete production aircraft not loaded with live ordnance or fuel
and (4) a full-up, full-scale ground test.
~&1_ _ 1_- _1_ _ _` 1 __ _1 ~_ ~
~ _ ~
Even one Agnes level Does not adequately simulate actual flight conditions,
and testing the virtually infinite number of threat-target interactions is out of the
question. Destructive testing (as opposed to nondestructive testing) of a complete
aircraft at the full-scare level yields extremely low confidence factors due to the
small number of trials possible. The opinion of most experts with whom the
committee met, and the committee's opinion, is that full-scale testing is much less
likely to provide useful information than are component, subsystem, and
subassembly tests.
The committee carefully weighed arguments on both sides of the issue
involving tests of a complete, fiull-scaTe aircraft (e.g., the chances that such full-
scale tests will reveal "unknown unknowns"~. The committee was persuaded that,
for a system like the F-22, a well conceived, incremental build-up of tests that
proceed from the component level to the subassembly or large assembly levels
made the most sense. The combination of lack of realism in test conditions, the
difficulty of obtaining a sufficient number of teals, and expert opinion all support
the conclusion that completely realistic, destructive, full-up, fi~-scale testing of
the F-22 is not practical and offers low benefits for the costs.
These views formed the basis of the committee's position regarding the
waiver requested for the F-22. The committee's recommendation on the waiver
appears below.
Sufficiency
The committee evaluated the current vulnerability assessment program for
the F-22. The Air Force and its contractors have incorporated many features in the
~ ^^ 11 ~ `1 _ ~ -11 ~ ~1 · id. ~ ~ ~ ·~ -. r
r-~z Design mat Wll1 reduce me a~rcrarc-s vulneranlllty te.g., a largely multiple
load path structural design, filet tank inerting, and much subsystem redundancy).
The design is complemented by a strong vulnerability analysis and live fire test
program.
However, the committee has some concerns about the program. For example,
no live fire tests are currently planned to assess damage done by direct hits on
important aft structural members; the current vulnerability specifications do not
include the effects of on-board ordnance; and the analysis does not properly
account for the flammable properties of the hydraulic and cooling fluids. The
committee believes that several additional tests and analyses are needed to
strengthen the program. Specific recommended actions are discussed later.
Vulnerability assessment is a complicated matter. As part of its evaluation,
the committee carefi~ly considered the need for continued vulnerability testing of
OCR for page 4
4
Live Fire Testing of the F-22
the F-22 beyond the currently planned tests that support engineering and
manufacturing development and initial production. The committee held detailed
discussions with the vulnerability and lethality assessment communities. In
particular, the committee considered the Navy's assessment methodology for
fighters.
Large assembly testing of variants of the Navy's F-] ~ has been conducted
and is planned, even though the Navy vulnerability assessment team does not
expect to discover unanticipated outcomes. It is possible that additional data could
be obtained from similar live fire testing of the major parts of an F-22 and result
in revisions to the aircraft design to reduce furler its vulnerability. The committee
recognizes arid accepts that test assets may not become available Critic after
production begins and are not likely to influence the production configuration of
the counter-air version of the F-22.
The Navy's rationale for larger scale testing was persuasive: something is
always learned, vulnerability assessment tools are evaluated and improved, and the
test base cannot be allowed to wither. After much deliberation, the committee
agreed that the expeditious conduct of tests against large assemblies made sense
for the F-22. This fighter will be in the inventory for decades and Carl be expected
to evolve, to include other missions (e.g., air-to-surface) and new configurations.
Testing of its large assemblies could also verify techniques for repairing battle
damage to the F-22's new composite materials and systems.
Vulnerability Assessment Tools
To be successful, vulnerability assessment requires much mutual support
between documentation, data bases, models, and testing. The committee's review
of the F-22 vulnerability assessment program indicated that significant
improvements are needed in several of the tools (i.e., documentation, data bases,
and models) that complement live fire testing. Specific conclusions and
recommendations relating to these tools are provided below.
CONCLUSIONS AND RECOMMENDATIONS
The committee's major conclusions and recommendations follow. They are
directed at the specific matters in the legislation that requested this study.
The committee's principal recommendation, which appears immediately
below, requires action by Congress. The numbered recommendations that follow
require action by the Department of Defense. Specific authorization arid
appropriation by Congress may be necessary to implement some of the numbered
recommendations.
OCR for page 5
Executive Summary
Desirability of Waiver for the F-22 Tests
Principal Recommendation. Permit a waiver of the fi~-up, fi~-scale,
live fire tests required by law for the F-22. The committee believes
that such tests are impractical and offer low benefits for the costs.
Recommendation I. Interpret a waiver as reinforcing the need to
conduct robust live fire tests of the F-22 that build incrementally from
the component level to the subassembly or large assembly levels.
(Recommendations to strengthen the current test program appear
below.)
Changed Circumstances Since Milestone I}
5
Nothing specific to the F-22 program appears to have changed since
Milestone IT in 1991 that would have prompted the Air Force to request a waiver.
However, there appears to be a changed view of what is required to fulfill the
intent of the live fire test law as it was interpreted within the Department of
Defense. Although there was some consideration of a request for a waiver in 1991,
it was not until after the 1993 report by the National Research Council that the Air
Force appeared to accept filthy the fact that a waiver was necessary. The Air Force
then instituted the current live fire test program, which does not include a filll-up,
full-scale test, and the waiver request followed.
Affordability and Cost-Benefit
The committee's conclusions relative to the impracticality and Tow benefits
for the costs of fillI-up, full-scale, live fire testing were provided above. The
committee was also asked to address affordability.
The committee defines an "affordable" activity as one within budgetary
constraints or attainable budgets. A full-up, fulI-scale test of the F-22 would cost
approximately $250 million (then-year dollars) above the currently planned
program. Thus, to perform the test, either more funding would need to be
authorized, or the test would have to displace $250 million of currently funded
activities in the program.
There is an argument that even $250 million is only a small percentage (less
than 0.5 percent) of total F-22 program costs, and therefore cost should not be a
determining factor. The co~runittee was not able to consider fully or challenge the
prioritization of items within the current F-22 program budget. However, the
committee's judgment (discussed earlier) is that the benefits of full-up, fi~-scaTe
OCR for page 6
6
Live Fire Testing of the F-22
testing of the F-22 are Tow relative to the costs. Even if $250 million were
provided for additional vulnerability assessment of the F-22, the committee would
not support using the funds for full-up, full-scale testing.
The co~x~nittee concludes that the affordability of live fire tests is not the
matter of foremost relevance; cost-benefit is most relevant. Affordability only
becomes relevant if the benefits relative to the costs of whatever tests are being
considered are commensurate with the benefits relative to the costs of other
alternatives.
The committee notes that its judgments regarding the costs and benefits of
fi~-up, full-scale testing were reached in the absence of a mature methodology for
assessing benefits relative to costs. The committee reviewed cost-benefit
methodologies related to incremental
live fire testing, conclusions and
recommendations concerning these methodologies are covered next.
Despite recommendations made by the previous National Research Council
committee, this committee regrets the lack of more progress in developing a
cost-benefit methodology for determining the return on investment of successive
levels of live fire tests. The methodologies briefed to the committee during the
study are immature and appear to address suboptunal measures of benefit. The
committee is leery of reliance on methodologies that use an overly simple
construct of the F-22's fixture to make judgments about how far to go with live
fire testing. A broader analytical framework could elevate the importance of
reduced F-22 vulnerability over the long term and might enhance the benefits
relative to the costs of given levels of testing.
Recommendation 2. Continue Department of Defense efforts to
develop viable cost-benefit methodologies for planning the extent of
live fire testing. Pursue methodologies to examine cost-benefit issues
in the light of frameworks that take a broad view of how the future
may develop for weapon systems like the F-22.
Sufficiency of Tests Planned for the F-22
The committee's evaluation of the current vulnerability assessment program
considered threat-related assumptions as well as the attendant ~ruInerabilities and
analyses and tests associated with the F-22's major subsystems (i.e., structure and
integral fuel tanks, fuel system and dry bays, flight control and auxiliary systems,
weapons bay and on-board ordnance, engines, flight crew, and fire protection
systems). The major results of the committee's evaluation are summarized below.
The committee accepts the threat environment defined for the current mission
of the F-22. Threat replication in the test program is reasonable. However, for
some classes of anti-air missile warheads (e.g., annular or focused blast
OCR for page 7
Executive Summary
7
fragmentation), the kill mechanism that involves dense multiple fragment impacts
may be important for the F-22.
Recommendation 3. Consider, in future analyses and tests, the kill
mechanism that involves dense multiple fragment impacts.
Overall, the committee believes that the vulnerability assessment program for
the F-22, given its current counter-air mission, is sufficiently realistic to support
the requested waiver. The following specific actions are recommended by the
committee to strengthen the program as the F-22 proceeds with engineering and
manufacturing development and initial production.
Recommendation 4a. Conduct additional live fire testing to determine
the damage that can be expected from a hit in the Frame 6 aft boom
attachment area. Determine the most critical shot lines for this testing.
Recommendation 4b. Expand analyses to predict damage sizes and
residual strengths of the aft boom, Frame 6, and horizontal tall pivot
shafts after being hit by 30mm high-explosive incendiary rounds. Also,
determine the risk of aircraft loss should it be found that loss of a
horizontal tad] is possible.
Recommendation 4c. Conduct further analysis of the aft fuel tank
(A-~) prior to the conduct of Test 4D.3 Focus this analysis on
dete~-~ining the adequacy of the test specimen, with particular
emphasis on its ability to simulate accurately the reaction of the entire
Ink.
Recommendation 4~. Make the operational community fully aware
that a fuel ingestion risk to the aircraft exists at a fuel state higher than
60 percent. (This risk arises because the file! tanks next to the engine
inlets are not empty at fuel states above 60 percent; thus, a puncture
could lead to fuel ingestion by an engine and potential engine failure.)
Recommendation 4e. Conduct the tests and analyses, proposed by the
F-22 System Program Office, on the flammability of coolant and other
fluids and the attendant vulnerability of the aircraft.
3 Test 4D was planned for August 1995, when a section of the fuel tank between Frames 5
and 6 was to be filled with water, externally loaded, and hit with a 30mm high-explosive
incendiary round.
OCR for page 8
8
Live Fire Testing of the F-22
Recommendation 4f. Undertake the analysis and test, proposed by the
System Program Office, of ablative materials in the weapons bay.
Also, conduct further analysis of the tradeoffs associated with
additional ordnance protection or defensive measures.
Recommendation 4g. Fund the Joint Technical Coordinating Group
on Aircraft Survivability and the Joint Live Fire Test Program to
assure the completeness of data on the vuInerabilities of on-board
ordnance.
Recommendation 4h. Fund the proposed Joint Live Fire testing of
FIl9 engine components to alleviate the paucity of testing against
those components.
Recommendation 4i. Emphasize continuing efforts by the F-22
System Program Office and the Joint Technical Coordinating Group
on Aircraft Survivability to develop improved methodologies for
reducing flight crew vulnerability.
Recommendation 4j. Use the prototype air vehicle fuselage in
Test 6A in lieu of a mock-up. (The Air Force has considered using this
fuselage for Test 6A, which will examine the synergistic effects of
pressurized coolant lines and cooled avionics modules and the adjacent
powered electrical wiring in the F-22 forward fuselage lower avionics
bays.)
In addition, the committee recommends that the Air Force begin planning for
expeditious vulnerability assessment testing of the F-22 similar to that being
conducted or planned for variants of the Navy's F-]8.
Recommendation 5a. Use large subassemblies from production-
representative hardware (e.g., a damaged aircraft or other source) in
these tests.
Recommendation Sb. Provide these assets, as soon as they become
available, to the vulnerability assessment community for the conduct
of live fire tests.
Recommendation 5c. Direct the tests at (a) verifying predictions from
the current F-22 live fire test program and the models used, and (b)
testing the effects on overall F-22 vulnerability assessment brought
about by configuration and mission changes. Also, use the tests to
OCR for page 9
Executive Summary
verify techniques for repairing battle damage to the F-22's new
composite materials and systems.
Other Recommendations
VuinerabiZity Requirements
9
Given the F-22's counter-a~r mission, the Air Force placed a premium on
achieving high survivability through reduced aircraft susceptibility. Reduction of
vulnerability in the design, although important, was deemed less significant. The
user (Air Combat Command) did not specify quantitative vulnerability
requirements for the F-22. Instead, contract specifications for vulnerability were
developed bv the Svstem Program Office in coordination with the Alar to e~t~hli~h
· . ~ .. ~ ~ ~ ~ .
1 · r ~ · , · · , · ~ . ~
a basis tor design opum~z~at~on and assess contractor performance.
The existing vulnerability specifications and the live fire test program do not
address fixture missions that the F-22 might be required to perform. The System
Program Office (the developer) is currently planning for an air-to-surface mission
using ordnance delivered from relatively high altitude. At some point in the fixture,
lower altitude missions could be required.
Recommendation 6. Reexamine expeditiously, for future F-22
missions (e.g., air-to-surface), the balance of requirements among
susceptibility, vulnerability, and related performance parameters.
Recommendation 7. Include in operational requirements for any new
missions user validation of quantitative vulnerability requirements, and
plan new live fire tests as necessary in response to those requirements.
Vulnerability Assessment Tools
The committee reviewed the documentation, data bases, and models that
complement live fire testing for the F-22 arid other aircraft programs.
Most of the documents (e.g., standards for vulnerability design) were found
to be ~ O or more years old arid in need of updating and improvement.
Recommendation 8. Update and improve expeditiously the various
standards, handbooks, md design guides that are important to the
aircraft vulnerability community.
OCR for page 10
10
Live Fire Testing of the F-22
Data bases play a distinct role in vulnerability assessment in that they form
the institutional memory that bridges specific systems, prevents repetitious testing,
arid avoids the mistakes of the past. Insufficient component tests have been
accomplished to produce confidence in the data bases for existing components and
materials; arid new composite materials, engines, stealth techniques, and other
advances wait require additional testing. The committee believes Were is
considerable need for expanded efforts to improve the data bases.
· ~ · . · ~, . . ~- . . . ~
Recommendation 9. Direct the loins Technical Coordinating Group
on Aircraft Survivability to define and plan a Joint Live Fire Test
Program that will, over the next several years, produce sound
vulnerability data bases; apply aggressive fielding to implement this
program.
Modeling is essential because the virtually infinite possibilities for
threat-target interactions Remarry ways to extend the limited number of tests that
can be conducted. Experimentation has been the main approach for obtaining data
to empirically fit the models. The advanced modeling techniques used by the
designers of nuclear weapons and the aerospace and automotive industries have not
been exploited by the vulnerability community. For the F-22, the committee judges
that a relatively large uncertainty is the ability to mode! the response of its
composite materials. Also, valid large-scale models could provide an efficient
means of making sound judgments without the need for expensive and repetitive
live fire tests on large subassemblies.
Recommendation lOa. Validate and accredit formally, by the Joint
Technical Coordinating Group on Aircraft Survivability and the Joint
Technical Coordinating Group on Munitions Effectiveness, the
wInerability assessment models used by the Air Force and over
services.
Recommendation lOb. Improve the vulnerability models of the
vulnerability community, and adopt these improvements for the F-22.
Recommendation lOc. Explore the application of advanced
methodologies currently being used by nuclear weapons designers and
other industries.
Recommendation lOd. Focus on ways to understand filthy the
response of F-22 composite materials to ballistic damage, and develo
and exercise analysis tools that can handle large-scale damage effects.
Representative terms from entire chapter:
vulnerability assessment
