|
Items in cart [0] |
Questions? Call 888-624-8373 |
| Copyright © 2009. National Academy of Sciences. All rights reserved. Terms of Use and Privacy Statement |
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 125
APPENDIX E
AN IMPROVED CRITICAL ITEM RISK ASSESSMENT PROCEDURE
FOR THE
NATIONAL SPACE TRANSPORTATION SYSTEM
(With an Example of Application to the 51-L Field Joints)
1. INTRODUCTION
]
On May 28, 1987, a NASA representative made
a presentation to the Committee on Shuttle Criti-
cality Review ant! Hazard Analysis Auclit entitled,
`'Critical Items List (CIL) Prioritization." The methoc!
discusser! was subsequently issued in moclifiec! form
as NSTS Instruction 2249 1, Reference F34. This
Instruction for the preparation of Critical Item Risk
Assessments (CTRA) provides a methoc! for prior-
itizing the failure modes in the CIL. It contains
many excellent ideas and is a significant step
forward. However, the Committee has some con-
cerns and some related suggestions on how to
simplify ant! clarify the method.
This Appendix also contains in Section 5 an
example of the application of trend analysis and
Probabilistic Risk Assessment (PRA) to the pre-
Challenger O-rings. This application, include(l here
only as an example of some applicable analysis
techniques, makes heavy use of moclern statistical
science ant] Bayesian ideas.
2. CONCERNS WITH THE CURRENT
METHOD
The Committee's concerns with the CIRA method,
as currently formulated, can be summarized as
follows:
I. In Table ~ of Reference t31 (shown here in
Attachment I) the column labeled "SEVER-
ITY" DEFINITIONS really contains worst-
case ciamage states.
2. In Table I, the columns labeler! SUCCESS
PATHS and STATUS CODE FOR REDUN-
DANCY/BACKUP are really descriptions of
system or subsystem architectures. They affect
risk by affecting the probabilities in the last
Failure
Mode
1
2
Severity
Definition
(A) Loss
of Life
(A) Loss
of Life
Success
Paths
o
o
Redundancy/
Backup
(a) None
(a) None
125
two columns. However, the relevant informa-
tion is in the probabilities themselves not in
the architecture. Any guidelines written on
how to assess the probabilities, either empir-
ically or subjectively, should contain much
discussion on how success paths, redundancy
structure, ant] periodic checking strategy af-
fects the probabilities in columns 4 and 5.
3. The probabilities in the last two columns of
Table ~ are qualitative ant] open to interpre-
tation as to what the terms "Very Likely,"
"Likely," "Unlikely," ant! "Very Unlikely,"
mean. The two columns, which have the same
qualitative scale, appear to have different
quantitative scales associates] with them. In
column 4, '~Very Unlikely" appears to mean
something like c ~ 0 - 6 and "Very I=ikely"
means something like 10- i. In column 5, the
scale depends on whether or not there is
reclundancy . If there is no reclundancy, then
"Very Unlikely" means something like 10-2
and "Very Likely" means something like
greater than .95. But if there is redundancy,
then "Very Unlikely" may mean ]0-6. With
the qualitative definitions of probability, it is
quite possible that two engineers working on
two failure modes with the same severities
and probabilities would assign them to dif-
ferent probability categories and therefore
produce inconsistent priorities. It is very im-
portant that the probabilities have opera-
tional clefinitions. Terms like "Unlikely" are
not operational definitions.
4. There is no way to produce a unique priority.
Suppose there are two failure modes, and
Table 1 is filled out as follows:
Design
Conf idence
Likelihood of
Worst Case
(iv) Unlikely
(ii) Likely
OCR for page 126
Which one should have the highest priority?
Suppose that the last two columr~s were
replaced by the following structure:
Failure
Mode
2
Probability of
Failure
Likely= 01
Unlikely = 00001
Probability of
Worst Case
Given Failure
.
Unlikely= 01
! ikely = 5
Probability of
Worst Case
0001
000005
Now it is clear that failure mode ~ presents
a higher risk.
3. PROPOSED IMPROVEMENTS
As an improvement to Reference f3I, the Com-
mittee proposes the procedure described in Tahie
E-1 below:
All failure modes with the same Worst Damage
State Given Lack of Redundancy or Reclunclancy
Failure wouic] be ranked by column Z.
The probabilities shown in Tahie E-l are for
illustration only anc! do not reflect any specific
example. In actual application, it wouIc! he highly
desirable for the analyst to include confidence limits
(or the equivalent) for each of the probabilities
listed in the tahies produced through the CTRA.
The Committee recommends strongly that such
probabilities be documented by a rationale. Many
of the facts mentioned in the current CIL "Rationale
for Retention" would be cites! in the probability
rationale but in the quantitative manner illus-
trated by the example in Section 5. In addition,
facts that imply higher probabilities wouIc! also be
analyzecl. For example, the long-run frequency of
catastrophic failure for solid rocket motors of a
TABLE E-1 Improved Risk Assessment Procedure
Failure
Mode
.
2
3
u
Criticality
OR
.
OR
V
Probability of
Primary Failure
During Mission
-
00
00
w
Probability of
Redundancy Failure
Given
Primary Failure
001 999
1-
0505
01
1
126
mature design is I/50; ant! therefore I/25 for two
solic! rocket motors. A dis-aggregation of this
frequency by ~ aiTure mode wouic! be a useful
baseline for an analysis. How are our clesign and
failure modes different from history? For example,
the field joint is similar to Titan ITI, but also
different. The redundant O-ring points to a smaller
probability, but the insulation geometry points to
a higher probability.
In Table E-l, failure mode 3 has the most risk,
even though it is only a Criticality JR item. For
this case, the computation of column W uses the
following estimates:
(i) There is one success path remaining after
the primary failure.
(ii) The availability of the backup is not readily
detectable ant] is checked every thirc! flight;
and the estimated availability is .99.
(iii) The probability of a secondary failure is
.05.
The formula for column W is
W= PrlBackup Available) x PrlSeconclary Failure)
+ PrlBackup not Available)
= (.99) (.05) + (.01)
= .0595
For failure mode I, there is no backup; but, it
is a relatively rare (probability = .001~ failure
mode and infrequently (probability = .01) causes
the worst damage state.
Failure mode 2 is much less risky. The compu-
tation of column W uses the following estimates:
(i) There is one success path remaining after
the first failure.
x
Worst
Damage State
Given Lack of
Redundancy or
Redundancy Failure
(A)—Loss of Life
and/or Vehicle
(A) Loss of Life
n~l/~r V~hi~.l~
(A) Loss of Life
and/or Vehicle
1
y
Probability of
Worst
Damage State,
Given Lack of
Redundancy or
Redundancy Failure
01
.1
Z = (V)(W)(Y)
.-
Probability of
Worst
Damage State
Event
.
00001
0000001 999
000595
OCR for page 127
(ii) The backup is readily detectable and fixed
when failed and the availability of the backup
is .999.
(iii) Given the backup, the probability of sec-
ondary failure is .001—the same as the
primary.
Use of equation (~) in this case yields
W= (.9991~.001) + (.001)
= .001999
4. RELATIONSHIP BETWEEN IMPROVED
PROCEDURE AND TABLE E-1
There is a strong relationship between the im-
provements described in Section 3 and NASA's
Table ~ (Attachment 1 here). From the "SEVER-
ITY" DEFINITIONS in column ~ of Table I, we
can deduce the following Worst Damage States:
A. Loss of Life and/or Vehicle
B. Mission is Aborted
C. Degraded Operational Capability or Early
Mission Termination or Damage to a Vehicle
System
D. Loss of Some Operational Capability of Ve-
hicle, but Full Mission Duration.
E. No Operational Effect
The probability scales couic! be set up as categories
with the definitions given in Table E-2.
The Committee urges the use of quantitative
definitions of probability. Even though for some
failure modes the probabilities will be assessed
subjectively, it is very important that the analyst
have art operational clefinition. To reiterate, terms
like "Unlikely" are not operational definitions. In
TABLE E-2 Probability Scales For Improved Risk Assessment Procedure
aciclition, use of a quantitative probability scale
will augment the pure engineering judgment ap-
proach.
The factors in Reference F3], Section 3.4, are
very relevant to assessing the Probability of Primary
Failure During Mission in Table E-1. Other factors
include:
Product design certification test results
Manufacturing process qualification test re-
sults
· Engineering analytical models
· Related industry data
· Etc.
The number of SUCCESS PATHS ant] the
REDUNDANCY/BACKUP scenarios given in
NASA's Table 1 (Attachment 1 to this appendix)
are very relevant to assessing the Probability of
Redundancy Failure Given Primary Failure in Table
E-1.
The factors relevant to assessing the Probability
of Worst Damage State Event in Table E-1 are very
similar to those listed in Reference f3], Section 3.5.
As part of the exercise of assessing this probability,
one could list all the events subsequent to redun-
dancy failure that do not lead to the worst damage
state.
5. APPI ICATION TO THE D-RINGS
Only as an example to illustrate the foregoing
proposal, consicler the field joint O-rings prior to
the Challenger flight 51-L at a joint temperature
of 31°F, which was predicted for the Challenger
flight. It is based only on a limited knowledge of
the subject derived from References A] and t2],
Center Point of Ranges of Probability Values
1 1
Description
Very Likely
Likely
Possible
Unlikely
Very Unlikely
Probability of
Primary Failure
During Mission
10 1
10 2
10 3
10 5
10 7
Probability of
Redundancy Failure
Given
Primary Failure
10 1
10 2
10 3
10 5
10 7
127
Probability of
Worst
Damage State
Given Lack of
Redundancy
or Redundancy Failure
0
5
0-~
-2
0-3
OCR for page 128
and thus must be viewed ONLY AS AN ILLUS-
TRATION OF A PROCESS.
To keep things simple only one failure scenario is
considerecI. In the language of Table E-1 we have:
TABLE E-3 Application of Table E-1 to the SAM
Field Joint
Language of Table E-1
Primary failure
during mission
Redundancy failure given
primary failure
Worst damage state
Application to Field Joint
Erosion and blowhy
of the primary O-ring
Failure of the secondary
O-ring given erosion and
blowhy of the primary O-ring
Loss of life and vehicle
The reason for considering this scenario is that
ciata are readily available. Also in Reference FI]
p. 135 it is stated that bypass erosion or blowby
was considered much more serious than just im-
. .
plngement erosion.
The data set used in this analysis (see Attachment
2) is taken from pages 129-131 of Reference FI].
The subset of these data user! here involves only
the actual flights and only the field ant! nozzle
joints. A useful organization of this subset is shown
in Attachment 3. In the columns labeled erosion
blowby and erosion or blowby the blanks
mean that the event (lid not occur. In the column where
labelect blowhy given erosion the blank means
there was no erosion and the zero means that there
was erosion but no blowby. Most of the data are
for the primary O-rings; but the data with an
asterisk are for the secondary O-rings.
5.1 Primary Failure
For primary O-ring failures we consider the
scenario of erosion ant] blowby. The primary failure
probability is:
PrlPrimary Failure) = Pr{Primary Erosion)
Pr mary Primary
x Pr{Blowhy Erosions. (2)
The vertical bar in the probability expression (2)
reacls conditional on. So for example
Pr{Blowhy ~ Erosion)
would read probability of the event Blowhy
conditional on the event Erosion occurring. For
two events A ant! B a funciamental law of prob-
ability is
PrlA and B) = PrlA) x Pr{B ~ A) .
5.1.1 Primary Erosion
A plot of the incidents of field joint primary O-
rings with erosion is shown in Attachment 4. For
example flight 51-C in January 1985 had two
field] joints with primary O-ring erosion; this mis-
sion experienced a joint temperature of 53° F and
a leak check pressure of 200 psi. The fitter! curves
are derived from a statistical model which allows
for possible joint temperature ant! leak check pres-
sure effects.
Flight 5 l-C experienced both erosion and blowhy
of the field joint. At a subsequent Flight Readiness
Review where 51-C was discusser! there was a
conclucling statement I=o`v temperature enhancer!
probability of blow-by (Reference tI] p. 1471.
On page H-73 of Reference f21 it is statec! that
Frequency of O-ring damage has increased since
the incorporation of . . . higher stabilization pres-
sures in leak test procedures ... . So it is of interest
to statistically mocle! the effect of temperature ant!
leak check pressure on O-ring anomalies.
pit, s') = Probability of erosion per field joint
primary O-ring,
t= Joint temperature
s = Leak check pressure.
The assumptions for this statistical model are:
I. The model for pit, s) is:
{t /J(~'s)}
This is caller! a Logistic Regression moclel. The
variables a,F,~ are unknown parameters to be
estimated from the data. Different values of these
parameters represent different relationships be-
tween erosion probability and (temperature,
pressure). For example, if ~ < 0, then probability
(lecreases with temperature; but if ,(3 ~ O. then
probability increases with temperature. We will
let the data determine which of these is most
likely.
2. Given pit, s'), the field joints are statistically
independent.
128
OCR for page 129
Let
aft, s) = Number of field joint primary O-rings
with erosion for a launch with joint
temperature t ant] leak check pressure s.
Uncler these assumptions, the probability distri-
bution of aft, s) given pit, s) is binomial with
parameters n = 6 (i.e., 6 fielc! joints) and p = pit,
s). So for k = 0, I, . . ., or 6,
Pr {X(t,s) = k j ptt,sy}
= {6) Spit s)]k;] — pit S)46-k
Let the subscript i represent the ith launch in
Attachment 3. So i = I, 2, . . ., 23. Let
xi = Number of field joint primary
O-rings with erosion
ti= loins temperature
si = Leak check pressure
Pi = P(ti, si)
Also let
t
X = (x I, X,, . . ., X, 3)
= (t,, t,, . . ., ten)
S = (S 1, S., . . ., S. 3).
The likelihood function, A, given the data x, is
defined as the probability of observing x conclitional
on t, s, and (a,Q,~). The variables t and s are
regarded as known variables (in standard] regression
analysis they are callec! inclepenclent variables); ant!
(`x,Q,~) are the unknown parameters. The likeli-
hood function is regarcled as a function of (`x,h,~)
and is
Li a ~ A) = ~ ( 6 ) pxi ( } _ p )6 -xi
Recall that Pi is a function of (`x,,B,~y).
The maximum likelihood estimates of the (a, [3,~y)
are those values that maximize the likelihood!
function. In effect, they are the values of (`x,h,~)
that make the observer] value of x the most probable
under our model.
There is a close relationship between maximum
likelihood] estimation ant! least squares. The least
squares estimates of (a,,(3,A) are those values that
. . .
mlnlmlze
where 6pi is the expected value of xi under our
moclel. If the xi's had a Gaussian (normal) distri-
bution with common variance, then the maximum
likelihood estimates and the least squares estimates
would be the same. This is because the Gaussian
probability density would then be monotonically
related to the sum of squares above. However, the
probability densities of the xi's in our problem are
binomial ant] not Gaussian. Ant! it is a well
established fact in statistical science that maximum
likelihooc! estimation is usually more efficient (closer
to the truth) than least squares; so we use maximum
likelihoocI.
The results of a maximum likelihood analysis of
these ciata under the above mociel yields the values
in Table E-4.
TABLE E-4 Maximum Likelihood Analysis of the SRM
Field Joint Primary O-Rino Erosion Data
Parameter
cY
Maximum Likelihood
Estimate
78
-.17
0024
_-
— 90% Confidence
Interval
[ -.1, 15 7]
[-.28 - 06]
[ - .01 2, 01 6]
The 90% Confidence Interval reveals the fact
that from our data we cannot learn the "true"
value of ((x,,B,A) with great precision. For example,
a Bayes interpretation of the interval [-.28, -.06]
for the temperature effect, A, is that given our data,
there is a .9 probability that the "true" value of
lies in the interval L-.28, -.064. Note that this
interval does not include the value ~ = 0 (i.e., no
effect). This means that the temperature effect is
"statistically significant;" or that there is only a
very small probability that the true value of ,l3 is
greater than or equal to zero.
Also note that there is no statistically significant
pressure effect on field joint erosion. That is because
most of the variation is explained by temperature
variation. This is curious, because in Reference L1],
blow-holes caused by high pressure were cited as
a cause of erosion.
Plugging the maximum likelihood estimates into
equation (3) yields
A
(Xi- 6pi)2,
i= 1
In[1 p(ti200) ] = 7.8 - ( 17)t + ( 0024)(200)
= 8.3 - (.17)t
129
OCR for page 130
~1 ~
1 nls 1rnplles
elf ~ (. 17)tl
p(t,200) = 1 + elf ~-('7)tl (4)
The curve for 200 psi (plotted in Attachments 4
and S) is (6)p(t,200), because there are 6 field
1olnts.
The predicted probability per joint of primary
O-ring erosion at 31° ~ joint temperature and 200
psi leak check pressure is
p(31,200~=.95 [probability of
- Primary Erosion
The 90 percent confidence interval for the "prob-
ability of primary O-ring erosion" is shown in
Attachment S and is t.S, 1.04. This shows that the
extrapolation to 3-~° E; introduces considerable
uncertainty in the estimate. The propagation of
this uncertainty to the final result will be discussed
in Section 5.5.
S.1.7 Pri~7zar~' Blc''~by Gin Priorly Erosions
The frequencies per primary O-ring of blowby
given erosion were extracted from Attachment 3
and are given in Table E-S. An analysis of the
blowby given erosion data shows no statistically
significant effects of joint type, joint temperature,
or leak check pressure. So we use the estimate
p ~ Primary Blowby ~ Primary Erosion
r ~ for Field Joint I for Field Joint J
t primary Blowby Primary Erosion]
= Pr] for Field or for Field or
t Nozzle Joint Nozzle loins J
= .292
TABLE E-5 Frequency per Primary O-Ring of
Blowhy Given Erosion
1 1--
Joint .
Field
Plugging (5) and (6) into (2) yields
Pr{Primary Failure} = (.95) (.292)
= .277
It is revealing to look at the frequency of primary
O-ring blowby, given no erosion, in Table E-6.
TABLE E-6 Frequency per Primary O-Ring of
Blowhy Given No Erosion
1 1
Joint
-
Field
Nozzle
Frequency
Joint per O-Ring
2
Field 7= 286
Nozzle ~ 7= 294
I, Field plus ! 7 - .292
I Nozzle 1 24
Frequency
per O-Ring
2= 50
1= 20
5
Field plus 7 = 286
Comparison with Table E-S shows that there is
a strong statistical dependence between primary
O-ring erosion and blowby particularly for the
field joint. For the field joint, blowby was rare
(frequency = .015) when there was no erosion,
but not rare (frequency = .286) when there was
erosion. no
PrlBlowby ~ Erosion) >> PrlBlowby ~ No Erosionl,
which implies strong statistical dependence. If blowhy
and erosion were statistically indepenclent, then
these two conditional probabilities wouic! be the
same.
The strong statistical dependence shown above
suggests that erosion might be a causal factor for
blowhy. This idea is born out by field data ant!
various experiments. Experiments (reference L21, p.
H-82) showed that an O-ring will fad! to seal with
an erosion depth of 0.15 inches. In flights 51-C
(6) an`] 5~-B, there was both erosion and blowby of
the field primary O-ring, and a heat effect or erosion
of the secondary O-ring. In both cases, the erosion
of the primary O-ring was among the worst ero-
sions experienced (reference F2], p. H-7l, H-72) as
measured by cross-sectioned depths of 0.038 and
0.171 inches, cross-sectionecl perimeters of 130°
and 360°, and a top view of affected lengths of
58.75 anc! 12 inches. This implies that blowby can
be caused by excessive erosion. So our model that
the higher the probability of primary O-ring ero-
sion, the higher the probability of primary O-ring
blowby, is plausible.
130
OCR for page 131
5.2 Probability of Secondary Failure
Next we consicler the Probability of Redundancy
Failure Given Primary Failure in Table E-~. This
would be failure of the secondary O-ring. Our
n~ode! of secondary failure is secondary erosion
and failure given primary erosion and blowhy.
Therefore
Pr ~ Secondary ~ Primary Erosion
Failure I end Blowby J
_ p ~ Secondary ~ Primary Erosion
r ~ Erosion I and Blowhy
x Pr{SeCondary | Secondaryl
Failure ~ Erosion i. (7)
A statistical analysis of secondary erosion given
primary erosion and blowby shows no statistically
significant effects of joint type joint temperature
or leak check pressure. So we use the estimate from
Table E-7 below:
~ . Primary Erosion and]
Pr] Secondary Erosion Blowhy ~ 2
t for Field joint for Field Joint
Secondary Erosion Primary Erosion and]
= Pr] for Field or Blowhy for Field
t Nozzle loins or Nozzle Joint J
Joint
Field
(8)
TABLE E-7 Frequency per SRM Joint of
Secondary O-Ring Erosion Given Erosion and
Blowhy of the Primary O-Ring in 23 Flights Prior to
Challenger 51-L
1
Secondary Erosion
Given Primary Erosion and Blowbv
..._ ~
Nozzle i
Field plus 2 = 286 1 of Secondary O-Ring
Time After ignition
Ignition Transient:
O to 170 ms
1 70 to 330 ms
330 to 600 ms
The estimation of
Pr | Secondary | Secondary | Steady State:
Failure | Erosion J 60 ms to 2 min
131
in equation (7) presents some difficulties because
there were no secondary failures before 51-~. So
we shall express the solutions parametrically in
terms of the parameter
A4 = PrISecondary Failure~Secondary Erosion) (9)
The state of knowledge curve (described in Appen-
dix D) for A4 could be determined on the basis of
engineering information. Examples of relevant en-
gineering information which was available before
51-L are:
loins rotation created doubt about the ability
of the secondary O-ring to seal. In fact the
O-ring failure mode was considered Critical-
ity 1, not Criticality 1R. So, officially, the
FMEA did not recognize the secondary 0-
rings as providing redundancy. However, ac-
cording to Reference f 1 l, p. 126, NASA
management and Thiokof still considered the
joint to be a redundant seal because there
were flights where the primary O-ring failed
and the secondary O-ring sealed in accord-
ance with its design intent.
In July 1985, a ThiokoT engineer, in light of
the 51-B nozzle joint secondary O-ring ero-
sion, expressed his concern that if the same
scenario should occur in a field joint (and he
believed it could), then it would be a "jump
ball" as to the success or failure of the joint
because the secondary O-ring could not re-
spond to the clevis opening rate and might
not be capable of pressurization (i.e., in the
51-E design, which has been changed in the
redesigned joint). (See Reference F1], p. 139.)
3. The qualitative assessment (Reference t2], p.
H-84, Chart 166) of the probability that the
field joint secondary O-ring will fait given
erosion penetration of the primary O-ring
seal is listed in Table E-8.
TABLE E-8 Qualitative Probability of SRM
Secondarv O-Rino Failure Given Erosion Penetration
Qualitative Probability of
Secondary O-Ring Failure
low
medium
high
high
OCR for page 132
There were only two incidents of secondary
O-ring erosion in a field] joint. So there was
no solid statistical evidence that the secondary
O-ring would work given primary O-ring
failure; i.e. nothing like ~ 000 successes with-
out a failure. Also as seen in Table E-8 the
probability of secondary O-ring failure cle-
F'ends on time after ignition.
r. The night before the Challenger launch a
chart provicled to NASA by a Thioko! engi-
neer about the possible temperature effect on
the O-rings (Reference tI] p. 89 Chart 2-2)
included concerns that: (i) lower temperature
of the O-rings would result in a change in
their sealing timing function which would
result in higher O-ring pressure actuation
time; (ii) if the actuation time increases
threshold of secondary seal pressurization
capability is approached; (iii) if threshold is
reached then secondary seal may not be
capable of being pressurized.
Plugging (8) and (9) into (7) yields
Pr ( Secondary
Failure J
= (.286 ~ As
Probability of ) ~ ~ 0)
Secondary Failure
5.3 Probability of Worst Damage State Given
Redundancy Failure
If the Felt! joint seal were to fad! there is some
possibility that the crew anal vehicle wouIcl survive.
For example the seal might fad! right before the
solid rocket motors completed their burn. How-
ever the chances are very high that such a failure
shouic! it occur would be earlier in the flight. This
suggests a value approaching ~ for the probability
of Toss of life and vehicle given total seal failure.
Thus the closest probability value of ~ from Table
E-2 column Probability of Worst Damage State,
is selectee! in this example.
5.4 Probability of Worst Damage State Event
Using the estimates derived above the value for
column Z in Table E-l is
Z= (.277~.286jA4 l'Probabilityper~oint:
~ of Worst Damage J A4 =
= (.0792)A4 .
5.5 Probability of At Least One Field Joint Failure
The estimated probability in Section 5.4 is for
only one field joint. The estimated probability of
held joint failure for the mission is
Pry Mission Field
~ joint Failure J
= ~ _ prtNO Field
= ~—t!—(.0792) A446
(Probability of Failure) (12)
It is clear from the statistical analyses that there
is uncertainty in the estimates of the probabilities
used. For example the 90 percent confidence in-
tervals in Table E-4 show that the parameter
estimates are uncertain. Also the .286 estimate in
equation (8) was based on two failures out of
seven and is therefore uncertain. The uncertainty
associated with equation (12) is quantified in At-
tachment 6. The two almost linear curves form a
90 percent confidence interval for the "probability
of mission fielcl joint failure," conditional on the
value of As. So if the value of A4 is .25, for example,
then the conclitional 90 percent confidence interval
is t0.010,.1184.
A subject matter expert could analyze the rele-
vant engineering information and assess a state of
knowledge curve for 4. If this curve were centered
on A4 = .25 with a considerable variance, then the
unconditional 90 percent confidence interval for
the "probability of mission field joint failure,"
would be much wider than the F.010, .118] interval
cited above.
The 90 percent confidence intervals in Attach-
ment 6 were derived by a Bayesian analysis (see
Appendix D for more discussion). For the ST-L
environment (e.g., 31° F), we define the following
Tong run "true" frequency probabilities:
H
132
= Probability of mission field joint failure
per mission; and for a given field joint,
~ = Probability of failure
A I = Probability of primary O-ring erosion
A, = Probability of primary O-ring blowby
given primary O-ring erosion
Probability of secondary O-ring erosion
given primary O-ring erosion and
blowhy
Probability of secondary O-ring failure
given secondary O-ring erosion.
OCR for page 133
Our mocie! is that ~ = ~ - ~ ~ - l)6
4
d)= 11 A.
i = I
Let /i =
A,A,A~
then D= 1 - [1—AA4]6
/1 ~
(14)
.
(15)
(16)
In the Bayesian analysis we assume that, condi-
tional on our data, Al, A,, and As are statistically
independent. This is reasonable because the Ai's
are successive conditional frequencies. The state of
knowledge curves for the inclividual Ai's were
derived from Bayesian analyses assuming "flat" a
priori state of knowledge curves. This means that
we die] not use much information external to the
ciata in Attachment 3. For example, we macie no
attempt to use the engineering models described
in, e.g., Reference F2], p. H-60. This may have
been possible by modeling the uncertainties in the
variables of the engineering models. This idea was
curves for the Ai's through equation (151. This was
done by a discrete probability approximation tech-
nique. The implied 90 percent confidence interval
for ~ is t.007, .0824.
The upper and lower curves in Attachment 6 are
clerived from equation (16) and are
6~(A4) = 1—f 1 - (.082) A4] 6
6~(A4 ) = 1—f 1 - (.007) A4] 6
REFERENCES
(17)
t1] Report of the Presiclential Commission on the
Space Shuttle Challenger Accident, Volume 1,
June 6, 1986, Washington, D.C.
t2l Report of the Presiclential Commission on the
Space Shuttle Challenger Accident, Volume 2,
June 6, 1986, Washington, D.C.
suggested by Feynman (Reference t2l, Appendix
F). The uncertainties in the engineering mociels are
a. l . . . f3l National Space Transportation System, "In-
a possible explanation as to why the models old . . r ,~ . c,- . A. .
not predict very well.
Finally, the state of knowledge curve for A was
clerive(l by propagating the state of knowledge
6
struct~ons tor Preparation ot critical item Mask
Assessment (CIRA)," NSTS 2249 1, June 19,
1 987.
OCR for page 134
ATTACHMENT 1 NASA's Proposed CIRA Technique.
0 uJ u
<: ~ ~ ~ `,, ,,, ,x, ~ ·r~
`1` O Off LL ~ L1~ ~ Us U ~ ~ <~ m ~ ~ us
Up O _' ~ U ~ > ~ U ~ ~ ~ Lo ~ ~ U ~ ~ ~ U Ud
~ ~ ~ . , ~ ~ .
LL .2 := := .>
I ~ he Z
~ Z I ~
Z ~ I 3 ~ Y
X ~ Z ~ ~ m us Z
_ Y O Z
> ~ ~ >
car 1 -
car ~
Z up ~ ~ up up
uJ ~ ~ ~ ~ C)
Z ~ — — ~ ,x, ,~, LL
O LL Z Z Z Z Z
Z O O O O 0-
O ~ U U U U'
~J L1J ~ L1J UJ L~
Z Z ~ ~ C) ~ ~ ~ O
1 ~ ~ ~ ~ ~ ~ ~ ~ ~ ~
L1J ~ o ~ ~ ~ ~ :) v~ ~) I
,= ,~ _ u~ ~ L., ~
, L~ ~ O L~ O ~ J
~ . ~ ' .
LLI . = _
U
| O ~ ~, m ~ Z ~ ~ Z ~ - z ,~ Z ,~ C ~ c, ~ - U O
1: L~ O Z ~ ^ ~ I UJ ~ 0 z ~ ~ L~ ,= z ~ ^ ~ ~ ~ ~ ~ ~ Z
U C ~ ~ ~ Z ~ m ~ ~ ~ ~ ~ ~ = ~ ~ ~ ~ ~ O z ~ C ~
O ~ _ ~ ~ ~ O O ~ O ~ `~ ~ ~ O O ~ u~ ~ ~ c O ~ C) ~ uJ 2 I _ ~ C
Z ~ 0 m ~ m ~ u ~ ~ m ~ m Lu 1l u m ~ ~ ~ c u ~ 1l m c~ ,
, . .
Q
~ ~ V~ ~ ~
m ~ z u~
I ~ ~ _ c~
V ~ ~ V L~ ~( ~= 3
~ ~ O
U
— O 3 0 ~ ~ ' ~ · r . z I
<( ~ O ~ O < 0 z O 5 ~ O
v~ v~ I v~ ~ O ~ <~ ~ ~ ~' ~ <~ ~ ~ ~ ,= ~ U ~ O ~ ~ u ~ U ~I
L~ ~ O Lu C~ ~C > Z ~ ~ 4< LL ~ Z O L~ t~~ ~ t1~ Ud C~ O ~ ~ 11
-m ~ ~ ~ _ ~ ~ Z ~ ll ~ ~ ~ ~ ~ O c~ ~ ·= O O
,
m
134
OCR for page 135
ATTACHMENT 2 0-Ring Anomall" Compared with Jolnt Temperatur" and Leak Check Pressurc
Flight (Solid Prossurc Jolnt
or Rockot JolnV (In pal) Tomp.
Motor Date Booster) O-Rlaa Fleld Nozzlc Eroslon Blowbv °E
DM-1 07/18m
DM-2 01/18n8
DM-3 10/19n8
DM-4 OV17/79
QM-1 07/13/79
-
NA NA - - 84
NA NA - - 49
NA NA - - 61
NA NA - - 40
NA NA - - 83
OM-2 09/27179 - - NA NA - - 67
QM-3 OV13/80 - - NA NA - - 45
STS-1 04/1V81 - - 50 50 - - 66
STS-2 11/1V81 (Right) AM FloldlP~ma~ 50 50 X - 70
STS-3 03m/82 - - 50 50 - - 69
STS-4
DM-5
STS-5
QM-4
STS-6
STS-7 06/18/83
STS-8 08/30/83
STS-9 11/28/83
STS 41 -B OV03/84
06t27/82 unknown: hardware lost at "a
10t21/82
11/11/82
03/21/83
04/04/83
(RIS]ht)
(Len)
(Right)
(Len)
STS 41 -C 04J06/84 (Right)
(LoR)
(Right)
(Right)
STS 41 -D 08t30/84
STS 41 -G 10/05/84
DM-6 10125/84
STS 51-A 11/08/84
STS 51 -C 01/24/85
-
Nozzle/Prlmary
Nozzle/Prlmary
Nozzle/Prlmary
-
(LeR)
(Right)
-
(Rlght)
(Right)
(Right)
(LeR)
(Leh)
Nozzle/Prlmary
Forward Fleld/
Prlmary
Nozzle/Prlma~y
Att FleldfPrlmary
IgnNorfPrlmary
FonNard
Fleld/Prlmary
NozziolPrlmary
l~nNerfPrlma~y
Inncr GaskcV
Prlmary
Center Fleldt
Prlmary
Ccntcr Fleldt
Secondaly
Nozzle/Prlmary
Forward Fleld/
Prlmary
Nozzis/Prlmary
Dash (-) denotes no anomaly; NA denotes not appilcable.
See cnd of attachment for footnotes.
50 50
NA NA
50 50
NA NA
50 50
50 W
50 50
100 50
100(2) 100
200 100
NA NA
X
(1)
(1)
_ _
X
200 100 X
200 100 X
200 100 (3)
NA NA -
200
200
NA
200
NA
200
200
100 X
100 X
NA
100
NA
100
100
200 100
200 100
200
200
135
100
100
-
80
58
68
60
67
67
72
73
70
57
-
-
_ _
57
63
63
X 63
70
X 70
- X 70
78
X X
X X
(4) —
_ X
X X
X
52
67
53
53
53
53
53
OCR for page 136
ATTACHMENT 2 (continued)
Flight (Solid Pressure Jolnt
or Rocket JolnV (In p~l) Temp.
Motor Dato Booster) O~R~ Fleld Nozzle Eroslon Blowbv °£
STS 51-D 04/1 V85 (Right) Nozzle/Prln~ary 200 200 X - 67
(RIght) l~niter/Prlmary NA NA - X 67
(Loffl Nozzh/Prlmary 200 200 X - 67
(Lett) l~niterIPrlmary NA NA - X 67
STS 51-B 04/29/85 (RIght) Nozzh/Prlmary 200 100 X - 75
(L.n) Nozzle/Prlmary 200 100 X X 75
(Len) Nozzl~Socondary 200 100 X - 75
DM-7 05/09/85 Nozzle/Prlmary NA NA X - 61
STS 51-G 06/17/85 (RIght) Nozzle/Prlmary 200 200 X (5) X 70
(Left) Nozzle/Prlmary 200 200 X X 70
(Lett) Igniter/Prlmary NA NA - X 70
STS 51-F 07/29/85 (RIght) Nozzle/Prlmary 200 200 (6) - 81
STS 51-1 08127/85 (Leff) Nozzle/Prlmary 200 200 X (7) - 76
STS 51-J 10/03/85 - 200 200 - - 79
STS 61 -A 10/30/85 (RIght) Nozzle/Prlmary 200 200 X - 75
(Len) ARFlel~P~ma~ =0 ~ - X 75
(Lett) Center Fleld/
Prlmary 200 200 - X 75
STS 61-B 11126/85 (Right) No~l~Prlma~ 2 - 200 X _ 76
(Len) No~l~Prlma~ 200 200 X X 76
STS 61-C 01/1V86 (Right) Nozzlc/Prlmary 200 200 X - 58
(Len) AM FleldJPrlmary 200 200 X _ 58
(LMt) NozzlelPrlmary 200 200 _ X 58
STS 51-L 01/28/86 200 200 31
(1) On STS-6, both nozzl" had a hot gas path detected In thc putty with an Indlcatlon of heat on thc
primary O-rlng.
(2) On STS-9, onc ot thc right Solid Rocket Boostcr fleld ~olnts was pressurized at 200 psl after a
destack.
(3) On STS 41-C, btt aft flold had a hot gas path detected In thc putty wIth an Indlcatlon ot heat on
thc primary O-rlng.
(4) On a center fleld Jolnt of STS 51-C, soot was blown by thc prlmaty and there was a heat effect on
the secondary.
(5) On STS 51-G, right nozzle had aroslon In ~o places on thc primary O-rlng.
(6) On STS 51-F, right nozzle had hot gas path detected In putty with an Indlcatlon of heat on thc
primary O-rlng.
(7) On STS 51-l, Ictt nozzle had croslon In two plac" on thc primary O-rlng.
136
OCR for page 137
rat
oS88 8888 8
~ ~ ~ ~ ~ _ _ _
o o o o o o o o o o 8 o o o 8 o 8 g 8 8 8 g o 8
~ ~ ~ ~ In ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~
~ 3~$~` ~~G ~~` ~~-~ ~0~
81 ~ ~ ~ ~ ~ 0
=
~ _
o
4.
o
._
._
_
C o
._
o
~ 1-
_
_
Io, o
. .
g In ~
o
o o
.
.
o
.
I. 8
i,
-
~ 6'sm~
o ~ o
_ ._ .
o
8 o o
~ _
1
o
._
8
~ ~ ~ CY ret rut rat ~ ~ ~ ~ ~ ~ ~ ~ us ~ us us us Ut us ~0 %0
0 0 co a~ oo oo a) ca 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
t~J ~J N ~ ~ 0 0 0 ~1 ~ O 1^ 0 ~ ~J ~ ~ ~ ~ ~ O ~ ~ 0
~ ~ ~ t~J ~ O ~ ~ I%J O O ~ O O ~J ~ ~J _ - 1 ~J O - 1 ~ ~ ~J
C) O ~ O ~ O O O ~ O O O ~ ~ O O O O O O ~ ~ ~ O O
4~
CJI
-
_ ~ ~ ~ ~
o
O O 0 ~i
137
OCR for page 138
ATTACHMENT 4 Occurrence of Field Joint Primary O-rings with Erosion.
3.0
2.5 _
In
z 2.0
LL
At
id 1.5
a
a:
m
1.0
0.5
0~0
.
.
-\~e $
.
\
\
\ ~ ~
\
\ ~ ~
_ \ ~
\ ~ ~
.
v.
50 55
ATTACHMENT 5 Maximum Likelihood Estimate and 90%
Confidence Interval for the Number of Field Joint Primary
O-rings with Erosion at 200 psi.
.,
5
4
Q
3
3
at
.
1
o
·.
·.
30 40 50
Pressure
— Data
50
100 ~
200 #
\~#` # #
.
.
\ ..
N."
me.
·.
a'
,~
60 65 70 75 80 85
TEMPERATURE
ATTACHMENT 6 90 Percent Confidence Interval for the
"Probability of Mission Field Joint Failure," as a Function
of A4.
.
6 _............
it\ : -
\
J _ \ ..
2 ~ . i.
. \
....
.25
.20
-
-
o
o.
In
~ .10
o
-
Q
o
.15 ~
/
90% Confidence
Interval
.05
70
Temperature
80
138
~4
Probability of Secondary O-ring Failure
Given Secondary O-ring Erosion
Representative terms from entire chapter:
worst damage
