Tools for Risk Assessment
Models of the Meteoroid and Orbital Debris Environment
NASA uses ORDEM96 as an input to its assessment of the risk to the shuttle orbiter from debris. ORDEM96 predicts the impact flux and velocity distribution of debris in a prescribed orbit (Kessler et al., 1996). ORDEM96 is the latest in a series of NASA models of the orbital debris environment developed since 1989 as user-friendly tools for spacecraft designers. ORDEM96 is not a conservative model. It attempts to describe the debris environment as accurately as possible (Johnson, 1997).
ORDEM96, although still semi-empirical, is significantly more complex than previous models. It divides the debris population into six source components (intact objects, large fragments, small fragments, sodium/potassium droplets, paint flakes, and aluminum oxide particles). These objects are divided into six inclination bands and two eccentricity families. The population of each source component in each representative orbit varies with altitude according to a formula based on both data and analysis.
The data used to develop and validate ORDEM96 were gathered from a wide variety of sources, including the United States Space Command Satellite Catalog, radar sampling of the LEO environment by the Haystack and Goldstone radars, and samples of materials returned from space. Analytic derivations are used to estimate populations of debris that are difficult to measure. For example, NASA’s EVOLVE model (Reynolds, 1991) is used to predict the population of smaller fragments from breakups.
The ORDEM96 model relies largely on internal NASA verification and validation to ensure that it operates as intended. Because it is primarily an empirical curve fit of data, NASA compares new data with ORDEM96 predictions. If the two correlate well, no changes are made. If sufficient new data indicate that the model is incorrectly predicting the environment, the model may be modified. ORDEM96 underwent an international peer review before it was released but has not been subject to formal verification and validation (NASA, 1996).
Because ORDEM96 is empirically derived, it can be modified whenever a significant breakup occurs. Before each shuttle mission, an evaluation is made of the effects of recent breakups. The effects of breakups that might affect the orbiter’s environment are added to the output of ORDEM96 for predictions of the debris environment for the mission. The breakup of a Pegasus rocket upper stage in June 1996, which produced several hundred fragments detectable by the SSN, is an example (Johnson, 1997).
The meteoroid model used by the shuttle program consists of a flux model (Grün et al., 1985) and a velocity model (Erickson, 1968; Kessler, 1969). Both are well accepted and widely used. The effects of normal, annual meteor showers are incorporated into the model, but rare meteor storms that occur when the Earth passes through a particularly dense portion of a comet dust trail are not. NASA, however, evaluates threats from meteor showers and storms before every shuttle mission and has delayed two missions to avoid potential hazards from meteor showers. NASA does not plan to fly the shuttle during future meteor storms. NASA is currently developing a new meteoroid model that includes the background environment as well as the effects of meteor showers and meteor storms.
The primary tool for preflight risk assessment and damage prediction from meteoroids and orbital debris is the BUMPER computer code. This code has been used since 1990 to assess the risks to the orbiter from meteoroids and orbital debris. BUMPER’s configuration is controlled by the Space Shuttle Requirements Control Board. The NASA Johnson Space Center Space and Life Sciences Directorate maintains the model and determines when updates are warranted (Christiansen, 1997). Recent updates have included new failure criteria and the incorporation of ORDEM96 (Zhang and Prior, 1996).
BUMPER employs a finite element model to represent the geometry of the orbiter and various mission components. This model contains more than 25,000 elements and includes the effects of shadowing some orbiter elements by others. On average, each element in the model measures 25 cm on a side. The size of the elements varies with location on the orbiter: the areas most vulnerable to critical penetrations are modeled using the smallest elements. The model divides the orbiter into 57 different regions (excluding payloads) to describe different materials, configurations, and failure criteria. BUMPER’S finite element model library
also contains models for a variety of payloads, including single and double Spacehabs, Spacelab, and the extended-duration orbiter pallet.
For each shuttle mission, BUMPER calculates two quantities. The first is the probability of impact, which is based on the expected meteoroid and orbital debris environment, the spacecraft configuration, and the mission profile. The second calculation is the probability of critical penetration and failure, given a particular impact on the orbiter. This is based on the geometry of the orbiter and its critical subsystems, empirically-derived equations governing damage levels and ballistic limits for various orbiter components and materials, and quantified, impact-based, failure criteria for the orbiter systems and components. Because BUMPER cannot evaluate the damage to orbiter components and systems caused by a given penetration, conservative assessments of a penetration causing critical damage are used unless a detailed study (such as the ones described in Chapter 3) of a particular area has been conducted, in which case the results of the study are used to set failure criteria.
High speed impact tests that simulate the impact of orbital debris have been used in the development of the damage predictor and ballistic limit equations in BUMPER for various orbiter materials and component configurations. Equations for materials and configurations not tested are pieced together from empirically-based equations. BUMPER calculations assume that the impactors are aluminum spheres, and all tests are performed with aluminum impactors. Using a single type of impactor simplifies the interpretation of test results and makes assessing the effects of different impact conditions easier.
ANALYSIS AND FINDINGS
NASA has been, and continues to be, a leader in the development of models of the space environment. A substantial portion of the analyses of the meteoroid and debris environment performed over the last 30 years has been conducted by NASA scientists. The product of the analyses, ORDEM96, is generally considered to be one of the best, if not the best, current model of the debris environment. A 1996 peer review of ORDEM96 revealed minimal dissent (Johnson, 1997). Peer review appears to be an appropriate approach to verifying the relatively simple empirical ORDEM96 model, and a formal independent verification and validation of the model does not appear to be necessary at this time.
The ORDEM96 model, however, is based on limited data and analyses, and its predictions include a high level of uncertainty. The only debris population that is well understood is the tracked population—objects larger than about 10 to 30 cm in diameter. All other population estimates are based on in-situ data gathered intermittently and supplemented by analysis.
Areas where uncertainty compromises the accuracy in ORDEM96 include
the conversions of measurements (either remote observations or returned samples) into population characteristics, orbital lifetimes (which are dependent on ballistic coefficients, solar activity, solar-lunar perturbations, and solar radiation pressure), and gaps—temporal, spatial, and object size—in the data (such as the almost complete lack of data on the population of debris with diameters from 0.5 to 2 mm). The use of models to estimate the amount of debris from breakups and releases of small debris also increase uncertainty. Estimates of the future rate of debris production obviously add to the uncertainty. No attempt has been made to quantify the effects of all of the uncertainties on ORDEM96’s predictions.
After each shuttle mission, the number of impacts found on shuttle surfaces is compared to pre-flight predictions (which use ORDEM96 flux data as an input). To date, pre-flight predictions of required window replacements appear to be fairly close to the observed results, but the very small amount of data gathered to date on millimeter-sized impactors suggests that the model may be underpredicting the flux in that size range (Levin et al., 1997). Because of the high level of uncertainty in the model and the dynamic nature of the debris environment, these results are still preliminary. Continued comparison of pre-flight predictions to post-flight damage assessments, however, appear to be warranted.
Finding. The ORDEM96 model is arguably the best available model of the debris environment. However, the model is based upon limited data and numerous assumptions. The magnitude of uncertainty in the model’s predictions is not known.
Population Variability over Time
Atmospheric drag and other factors cause debris—particularly smaller debris—to rain through the orbiter’s altitude regime (300 to 450 km) for relatively short periods of time. Figure 4–1 shows estimates of particle lifetimes (by diameter) as a function of solar activity. The figure shows that millimeter-sized particles will stay in the 300 to 450 km range for a few months at most and that particles 0.04 mm in diameter will stay in orbiter altitudes only for days or weeks.
Basic sampling theory dictates that an accurate description of the dynamics of a certain size of debris in a certain altitude regime requires that the sampling rate be at least twice as fast as the debris will be cleansed from that region. For example, during periods of high solar activity, sampling would have to be done about once a week to produce an accurate assessment of the population of millimeter-sized debris. Although this is currently fiscally infeasible, it does indicate what would be required for an accurate representation of the natural variability in population estimates.
ORDEM96 averages all of its predictions over at least one year and, therefore, does not account for natural variability in the debris environment although it makes some adjustments for changes in solar activity. Thus, for short duration
missions, the actual flux may vary greatly from the ORDEM96 predictions. (This is not to imply a deficiency in ORDEM96 but to sound a cautionary note about interpreting its results.)
Finding. Because of limited data and the natural variability in the population of debris smaller than about 5 mm in diameter, ORDEM96 predictions of debris fluxes for individual shuttle missions may be highly inaccurate.
To predict the hazard from debris accurately throughout a shuttle mission, NASA needs either to develop spatially and temporally dependent analytic models for debris smaller than 5 mm in diameter or to greatly increase its ability to gather data about this population. Both of these approaches are promising, but the benefits of each must still be weighed against the costs.
The Haystack radar measurement (Stansbery et al., 1996; Settecerri et al., 1997) that investigated the presence of space-borne sodium potassium droplets is an example of the sampling quality and rate that would improve the understanding of the production and removal of millimeter-sized debris. By increasing the sensitivity of ground-based radars, NASA may be able to sample the population
down to 2 mm in diameter, although the cost could be prohibitive. With a sufficient sampling rate and rapid data analysis, short-term predictions of the flux of particles in this size range could be made.
At the same time, or as a less expensive alternative, NASA could focus on improving the understanding of sources of debris less than 5 mm in diameter at orbiter altitudes. One of these sources is solid rocket motor firings, which project a variety of aluminum oxide particles into elliptical orbits as they boost spacecraft into higher orbits. Because of their elliptical orbits and high ballistic coefficients, the movement of these particles is difficult to model.
A second major source of debris less than 5 mm in diameter are debris wakes, which are even more difficult to model. Debris wakes are created by the release of debris from a spacecraft by nonenergetic means (typically as external surfaces deteriorate over time and slough off debris). Deterioration is caused by a number of mechanisms, including atomic oxygen erosion of surface materials, thermal cycling, ultraviolet radiation that causes embrittlement, particulate impacts that produce ejecta, and spacecraft operations that release foreign material. All of these release mechanisms create debris wakes that vary with the size, age, and orbit of the object producing the wake, as well as the construction and composition of its external surfaces (in particular, the type of paint).
Finding. Predicting the short-term hazard to individual shuttle missions from orbital debris more accurately would require a greatly improved capability to sample the population of sub-5 mm debris and/or improved models of the sources and orbital behavior of sub-5 mm debris.
ORDEM96 currently does not include information about debris shape or composition although the amount of damage caused by a collision is strongly affected by the shape and composition of the impactor. But debris shape and density are difficult to model for most types of debris. Given the large uncertainties inherent in both NASA’s environment and penetration models (Johnson, 1997), it is not clear that shape or density information would significantly improve current damage predictions. However, NASA will need more information about the shape and composition of debris for more accurate, end-to-end meteoroid and orbital debris risk assessments for the shuttle.
Although BUMPER has not undergone formal, external, independent validation and verification, it has been reviewed by many of its users since 1989. Problems in the code or output are forwarded by users to NASA for review and possible action. BUMPER predictions have been shown to compare well with those of ESABASE (the European Space Agency equivalent of BUMPER), SURVIVE (Lockheed Martin), and SD-SURF (Lockheed Martin). BUMPER predictions are
also regularly compared with inspections of the surfaces of returned spacecraft, including the shuttle orbiter (NASA, 1997).
Impactor Size, Shape, and Velocity
BUMPER uses solid spherical aluminum projectiles to simulate the impacts of orbital debris. However, testing and analyses in the last 30 years have shown that the damage caused by the impact of non-aluminum and nonspherical projectiles is decidedly different from the damage caused by spherical aluminum projectiles with the same impact energy. Hence, BUMPER’S predictions may dramatically overestimate or underestimate damage to the shuttle orbiter from on-orbit impacts by orbital debris particles, which are usually not spherical and are usually not composed of aluminum. For example, NASA estimates that if 30 percent of debris were plastic, 30 percent were steel, and 40 percent were aluminum, the penetration risk would increase by 80 percent (Christiansen, 1997).
The impact velocities of orbital debris are expected to average about 10 km/s, but routine testing using light gas guns cannot exceed impact speeds of approximately 7 km/s. Therefore, all of the equations developed for and used by BUMPER are strictly valid only up to 7 km/s. However, a complete risk assessment requires obtaining the impact responses of materials and configurations at impact velocities of more than 7 km/s. NASA believes its analytical model for extending the ballistic limit curves above 7 km/s is conservative. Although this conservatism may be appropriate for safety reasons, it may unduly restrict operational flexibility (see discussion in Chapter 3).
NASA has performed some initial impact tests at 11 km/s using an inhibited shaped charge launcher (ISCL), and additional tests are planned for 1998. The ISCL can generate impact test results at velocities higher than conventional light gas guns, but the projectile in an ISCL test (a hollow cylinder) differs from the solid sphere typically used in light gas gun testing. NASA has performed an initial study to try to correlate the 11 km/s hollow cylinder data and the 7 km/s or less solid sphere data (Christiansen, 1997). Preliminary results suggest that BUMPER may indeed be conservative at high velocities, but the results are, at best, very limited and inconclusive.
Finding. BUMPER is probably the best available model for orbital debris risk assessment and damage prediction for the orbiter. However, it incorporates a number of simplifying assumptions, as well as a limited set of empirically-derived ballistic limit equations, and the magnitude of uncertainty in its predictions has not been well characterized.
NASA could take two approaches to making the BUMPER model more useful for shuttle mission planners and program managers. First, more extensive data
collection and analyses could reduce uncertainties in BUMPER predictions. The effects of non-aluminum and nonspherical projectiles on orbiter materials and configurations, for example, could be systematically characterized and the equations in BUMPER revised accordingly. NASA could also try to improve its understanding of impacts at velocities above 7 km/s, perhaps by better establishing the relationship between its sub-7 km/s tests and its 11 km/s hollow cylinder tests, or by working with other organizations that conduct high-velocity impact tests and simulations.
A second approach would be to characterize the uncertainties in BUMPER predictions more accurately. BUMPER currently does not provide users with error bars or confidence intervals. Such information, however, could be invaluable to those making decisions based upon BUMPER predictions. A sensitivity analysis of BUMPER results to various input parameters could help NASA determine which parts of the model to refine in order to reduce the uncertainties of BUMPER calculations of mission risk most effectively. A rigorous peer review process, including some form of independent validation and verification, could also increase user confidence in the model’s results.
Recommendation 4. NASA should increase its data gathering and modeling efforts to improve understanding of the sources (e.g., solid rocket motors and debris wakes) of the sub-5 mm debris population in the shuttle’s orbital regime.
Recommendation 5. NASA should try to refine BUMPER and ORDEM96 so that their results include appropriate error bars and associated confidence levels. To begin with, NASA should analyze the sensitivity of the output of both models to changes in input parameters.
Christiansen, E. 1997. BUMPER Model. Presentation to the Committee on Space Shuttle Meteoroid/ Debris Risk Management, Houston, Texas, June 17, 1997.
Erickson, J.E. 1968. Velocity distribution of sporadic photographic meteors. Journal of Geophysical Research 73:3721–3726.
Grün, E., H.A.Zook, H.Fechtig, and R.H.Giese. 1985. Collisional balance of the meteoritic complex. Icarus 62:244–272.
Johnson, N. 1997. Risk assessment process, tools, and databases. Presentation to the NRC Committee on Space Shuttle Meteoroid/Debris Risk Management, Houston, Texas, June 17, 1997.
Kessler, D.J. 1969. Average relative velocity of sporadic meteoroids in interplanetary space. American Institute of Aeronautics and Astronautics Journal 7:2337–2338.
Kessler, D.J., J.Zhang, M.J.Matney, P.Eichler, R.C.Reynolds, P.D.Anz-Meador, and E.G. Stansbery. 1996. A Computer-Based Orbital Debris Environment Model for Spacecraft Design and Observation in Low Earth Orbit. NASA Technical Memorandum. Houston: NASA.
Levin, G., E.Christiansen, J.Loftus, and R.Bernhard. 1997. Recent results from space shuttle meteoroid/orbital debris pre-flight risk and post-flight damage assessments. IAA-97-III.6.4.01. Presented at the 48th International Astronautical Congress, October 6–10, 1997, Turin, Italy. Paris: International Astronautical Federation.
NASA. 1996. Invitation to Participate in the Testing and Verification Phase of the Computer-Based Orbital Debris Environment Model for Spacecraft Design and Observations in Low Earth Orbit. SN#-96–033. February 23, 1996. Houston: National Aeronautics and Space Administration.
NASA. 1997. BUMPER verification/validation. Viewgraph presented to the Committee on Space Shuttle Meteoroid/Debris Risk Management, Houston, Texas, June 17, 1997.
Reynolds, R.C. 1991. Documentation of program EVOLVE: a numerical model to compare projections of the man-made orbital debris environment. OD91–002-U-CSP. Arlington, Va.: System Planning Corporation.
Settecerri, T.J., E.G.Stansbery, J.N.Opiela, and R.Henderson. 1997. Haystack Radar Measurements of the Orbital Debris Environment, 1994–1996. JSC-27842. Houston: NASA Johnson Space Center.
Stansbery, E.G., M.J.Matney, J.Zhang, and R.C.Reynolds. 1996. Haystack Radar Measurements of the Orbital Debris Environment, 1990–1994. JSC-27436. Houston: NASA Johnson Space Center.
Zhang, J.C., and T.Prior, 1996. Verification of the 1996 environment model implementation into BUMPER-II. Memorandum to E. Christiansen, NASA Meteoroid/Debris Shielding and Survivability Analyst. October 21. Houston: NASA.