National Academies Press: OpenBook

Environmental Cleanup at Navy Facilities: Risk-Based Methods (1999)

Chapter: 3 Strengths and Weaknesses of Risk-Based Methodologies

« Previous: 2 Review of Risk-Based Methodologies
Suggested Citation:"3 Strengths and Weaknesses of Risk-Based Methodologies." National Research Council. 1999. Environmental Cleanup at Navy Facilities: Risk-Based Methods. Washington, DC: The National Academies Press. doi: 10.17226/6330.
×

3
Strengths and Weaknesses of Risk-Based Methodologies

Certain advantages of risk-based methodologies (over resource conservation or technology-based approaches) make them attractive for environmental cleanup. However, important disadvantages of risk-based methodologies cannot be ignored. This chapter includes a description of the strengths and weaknesses common to risk-based methodologies. The major characteristics of the ASTM RBCA methodology and the CERCLA process (including the EPA's Risk Assessment Guidance for Superfund and Soil Screening Guidance) are compared in an illustrative matrix that reveals their similarities and differences. The chapter then discusses how the strengths and weaknesses described for any risk-based methodology are manifested in the ASTM and EPA processes. Finally, other strengths and weaknesses of ASTM RBCA and CERCLA(RAGS/SSG),1 beyond those characteristic of risk-based methodologies in general, are discussed. These strengths and weaknesses have important implications for the use of these methodologies at naval facilities.

Strengths And Weaknesses of A Generic Risk-Based Approach

Strengths

Risk-based methodologies rely on a systematic process for characterizing contaminated sites and the risks they pose. This feature allows for easier imple-

1  

The committee recognizes that CERCLA is a federal law written by Congress, while RAGS and SSG are policy documents produced by the EPA for use during the CERCLA process and other cleanup efforts. To simplify terminology in this chapter, the term CERCLA(RAGS/SSG) will be used to refer to the entire CERCLA process, including those steps that use RAGS and the SSG.

Suggested Citation:"3 Strengths and Weaknesses of Risk-Based Methodologies." National Research Council. 1999. Environmental Cleanup at Navy Facilities: Risk-Based Methods. Washington, DC: The National Academies Press. doi: 10.17226/6330.
×

mentation of the cleanup process at a large number of contaminated sites. The systematic nature of risk-based methodologies can be the result of a tiered approach, as in the ASTM RBCA standard guides and the EPA Soil Screening Guidance, or some other organizing framework.

Because a quantitative determination of risk is one of the main goals of any risk-based approach, significant amounts of high quality data about the contaminated sites must be collected. Thus, risk-based methodologies require extensive site characterization activities. These activities increase the user's understanding of the nature of the contamination and can identify the immediate need for response actions to reduce the risk. Many risk-based methodologies result in less stringent cleanup goals when significant site characterization efforts are made.

Risk-based methodologies are founded on scientific descriptions of contaminant fate and transport and exposure of human and ecological receptors, and they tend to be scientifically defensible. The best methodologies can incorporate new scientific information as it becomes available.

Associated with the systematic nature of risk-based methodologies is the ability to help prioritize sites. Generally, there are early opportunities during a risk-based process to make an initial determination of the risk posed by a contaminated site. Risks from multiple sites can be compared to rank sites in terms of the hazards posed. Such priority setting can be based on the relevant concerns of the responsible party (e.g., which sites should be targeted for cleanup first, which pose the greatest immediate risk, or which will be easiest to close).

An important benefit of the ranking of sites in a risk-based approach is the potential for the efficient allocation of resources. Depending on the goals of the user, a risk-based approach can help determine how to achieve the greatest risk reduction per dollar spent.

Weaknesses

Many of the weaknesses of risk-based methodologies are ramifications of the fact that they are more likely to leave contamination in place than resource conservation or technology-based approaches. In keeping with recent interagency guidance, the committee defines "contamination left in place" as "hazardous substances, pollutants, or contaminants remaining at the site above levels that allow for unlimited use and unrestricted exposure" (Air Force/Army/Navy/EPA, 1998). Risk-based methodologies are more likely to leave contamination in place because they rely on engineering controls and institutional controls as alternatives to treatment. Institutional controls are often very attractive in the abstract, but in reality they can be difficult to implement and enforce, especially over the long run. The committee has observed that (1) property law makes it difficult to enforce so-called deed restrictions against subsequent owners and (2) local jurisdictions responsible for land use planning rarely coordinate with environmental regulators. Institutional controls, therefore, do not necessarily control exposure, and in some cases, they simply postpone a necessary cleanup.

Suggested Citation:"3 Strengths and Weaknesses of Risk-Based Methodologies." National Research Council. 1999. Environmental Cleanup at Navy Facilities: Risk-Based Methods. Washington, DC: The National Academies Press. doi: 10.17226/6330.
×

As stated in Chapter 1, one of the main reasons that potentially responsible parties have sought risk-based cleanup approaches is the lack of proven, affordable technologies for removing source contamination. For many reasons, the development of innovative technologies for removing source contamination has declined over the past ten years (NRC, 1997). It is possible that the adoption of risk-based methodologies will further slow this development, because there will be less demand for such technologies. This would be an unfortunate consequence of the adoption of risk-based methodologies, especially if source removal technologies could be developed that would rival containment technologies in cost and provide permanent risk-reduction.

Finally, leaving contamination in place makes it more likely that unidentified and potentially harmful toxicants will remain on site than if a significant amount of source removal is accomplished. The contents of contaminated sites may be largely unknown, especially at landfills and other sites where multiple waste types exist. These compounds could have fate and transport properties and toxicological properties that could pose significant unanticipated risk to receptors. An example of how the discovery of a previously overlooked compound, MTBE, derailed the implementation of a risk-based approach for cleaning up petroleum UST sites in California is presented in the case study in Box 3-1.

Risk-based methodologies suffer from one other major weakness: uncertainty. Unless all sources of contamination are removed, there will always be some amount of uncertainty regarding the effectiveness of a risk-based approach. (It should be noted that all approaches that leave some contamination in place, including technology-based approaches, are characterized by uncertainty.) Uncertainty affects the risk assessment calculations, as well as the efficacy of treatment technologies, engineering controls, and institutional controls. Risk-based approaches are generally scientifically defensible because they use mathematical descriptions of contaminant fate and transport, exposure pathways, and dose-response relationships; however, they are also inherently limited by the quality of the site data and the accuracy of those models.

The degree of uncertainty associated with the use of a risk-based methodology increases greatly for sites contaminated with non-petroleum compounds. The chemical and biological characteristics of these compounds in ground-water systems, as well as their mobility and toxicity, are much more poorly known than for petroleum hydrocarbons. The behavior of these contaminants can change dramatically in and between sites for the same compound. For example, perchloroethene is biodegradable in anaerobic environments, but it is persistent under aerobic conditions. The application of a risk-based approach to sites contaminated with metals is especially problematic, because the uncertainties in metal behavior are substantially greater than for petroleum hydrocarbons. Unlike many organic contaminants, metals cannot be eliminated from the site by a chemical or biological transformation. Depending on the pH and redox conditions at the site, metals can exist in different chemical forms, each of which may have a different toxicity

Suggested Citation:"3 Strengths and Weaknesses of Risk-Based Methodologies." National Research Council. 1999. Environmental Cleanup at Navy Facilities: Risk-Based Methods. Washington, DC: The National Academies Press. doi: 10.17226/6330.
×

and mobility. Thus, the uncertainties that accompany contamination by non-petroleum compounds may be substantial. Risk-based methodologies that do not acknowledge the existence of uncertainty, or make assumptions to reduce uncertainty that may or may not be valid, imply a level of certainty that in fact does not exist.

Comparing ASTM RBCA and CERCLA(RAGS/SSG)

The ASTM RBCA standard guides and the CERCLA framework (including RAGS and the Soil Screening Guidance) are both types of risk-based methodologies. Before describing how the previously mentioned strengths and weaknesses manifest themselves in these two methodologies, the basic characteristics of the two methodologies are compared.

As shown in Table 3-1, there are many more similarities than differences between ASTM RBCA and CERCLA (RAGS/SSG). The Soil Screening Guidance and the ASTM RBCA standard guides are both tiered approaches in which generic cleanup levels are replaced with site-specific cleanup levels as more data become available. Both methodologies allow for removal actions throughout the duration of the process. Both mention the use of certain fate-and-transport and dose-response models, although the models given as examples in RBCA tend to be more advanced than those required by the EPA. CERCLA and chemical RBCA contain almost identical criteria for choosing the remedial option, and point to long-term monitoring as an important way of ensuring that the remedial option (including treatment technologies, engineering controls, and institutional controls) works over the long-term.

A number of procedural differences between the two methodologies affect their implementation. First, the tiered approach for evaluating increasingly complex sites is available under RBCA for all types of contamination, while under CERCLA the tiered approach is not universal. An explicit tiered approach is given only for soil contamination (the Soil Screening Guidance), although some states have developed generic screening levels for both soil and ground water (see Chapter 2). A comparison of the ASTM and EPA tiered approaches reveals that RBCA has slightly more management options available to the user than the Soil Screening Guidance. That is, under a RBCA tier 1 or tier 2 evaluation, if the site conditions exceed target levels (RBSLs or SSTLs), the user may clean up the site to target levels or proceed to a higher tier evaluation. Under the Soil Screening Guidance, if site conditions exceed generic or site-specific SSLs, the user must proceed to the next tier of evaluation. Thus, RBCA allows management options to be taken earlier in the cleanup process than the Soil Screening Guidance.

Other important differences between ASTM RBCA and CERCLA(RAGS/ SSG) relate to how the methodologies are perceived. The ASTM documents are brief, simple, and generally accessible, while EPA documentation tends to be lengthy and scattered. This has resulted in the general perception that the ASTM

Suggested Citation:"3 Strengths and Weaknesses of Risk-Based Methodologies." National Research Council. 1999. Environmental Cleanup at Navy Facilities: Risk-Based Methods. Washington, DC: The National Academies Press. doi: 10.17226/6330.
×

BOX 3-1 Case Study: LLNL/UC Recommendations for Leaking Underground Storage Tanks

This case study illustrates an attempt to implement a risk-based approach to cleanup that ultimately failed because of the presence of a previously unidentified compound at the contaminated sites. It also highlights the importance of scientific peer review during the design of risk-based methodologies.

The Lawrence Livermore National Laboratory (LLNL) and the University of California (UC) have been involved in a variety of activities that assess leaking underground fuel tanks (LUFT) for the state of California. Recently, several projects have been undertaken to (1) determine to what extent California's ground-water resources are affected by LUFT, (2) determine what factors affect the length and mass of fuel hydrocarbon (FHC) plumes, and (3) assess whether FHC plumes behave in a predictable fashion (McNab et al., 1997a, b; Rice et al., 1995a, b; Rice et al., 1997; Rice and Kavanaugh, 1997).

In 1995, LLNL/UC analyzed 271 sites that did not have fractured rock hydrogeology, had fairly uniform water chemistry and hydrogeologic characteristics, and often had shallow depth to ground water (< 15 feet). Since gasoline is the major fuel released from LUFTs, and because benzene is considered to be the most toxic of the gasoline components, it was selected as the chemical for study and generalization. The focus was ground water contamination; soil-only cases were not included and exposure via drinking water wells was the only pathway considered. In general, the sites appeared to be characterized by conditions necessary for intrinsic bioremediation.

The conclusions of the initial study (given its stated limitations) were that (1) 90% of the benzene plumes with >10 ppb were less than about 260 feet in length; (2) benzene plumes were relatively stable or shrinking; (3) benzene concentrations decreased with time without active remediation; and (4) LUFTs do not have a significant impact on California's ground-water resources or on public or ecological health. The panel recommended that California immediately modify the ASTM RBCA framework based on the LUFT case data and apply this modified ASTM RBCA framework as soon as possible to LUFT cases. During a second phase of activity, the LLNL/UC panel developed such a modified RBCA framework and applied it to ten DOD demonstration sites. For these case studies, sites were assessed and a site conceptual model describing sources, concentrations of most chemicals of interest, and potential pathways and receptors was developed. Further, an as

Suggested Citation:"3 Strengths and Weaknesses of Risk-Based Methodologies." National Research Council. 1999. Environmental Cleanup at Navy Facilities: Risk-Based Methods. Washington, DC: The National Academies Press. doi: 10.17226/6330.
×

sessment was made concerning future land and resource (e.g., ground water) use. Unlike the more formal ASTM RBCA methodology, the major emphasis of this approach was to collect enough site characterization data to allow estimation of plume stability and length (using benzene as the chemical of concern), and to determine if intrinsic bioremediation was occurring. If it could be demonstrated that the plume was stable, that intrinsic bioremediation was occurring, and that potential receptors were located far enough from the plume, then no further cleanup action would be warranted. These were the conclusions for the two reports made available to this committee. Removal of tanks, pipes, and free product was the only remedial action recommended (if it had not already been done) and monitoring requirements were minimal (either for 2 years or 5 years, with at least 2 monitoring wells).

Although this approach may seem relatively simple compared to the ASTM RBCA methodology, it is not a trivial matter to estimate plume length and stability and determine whether intrinsic bioremediation is occurring. Extensive monitoring and fairly complex modeling are required. In fact, one might argue that this type of modeling is more complex than the fate-and-transport models recommended for a tier 2 ASTM RBCA evaluation.

There was an indication that California policies may be changed as a result of the LLNL/UC reports (Pettit, 1995). A simple variation of the risk-based methodology for LUFT cleanup used at the ten DOD sites was even proposed (California Water Resources Control Board, 1997). In the end, however, the new methodology and the recommendations of the LLNL/UC panel were not endorsed for two main reasons (Farr et al., 1996). First, the recent discovery of widespread MTBE contamination at LUFT sites shifted attention from benzene to MTBE as the chemical of concern. MTBE is more mobile than benzene, far more resistant to biodegradation by indigenous organisms, and its toxicity is not well documented. Several of the sites studied by the LLNL/UC panel were found to contain MTBE plumes that were larger and more mobile than the benzene plumes present at the site. Had a cleanup policy based on average conditions for benzene plumes been developed for LUFT sites, it would have incorrectly accounted for the risks of MTBE. The second factor that hindered the adoption of a risk-based approach in California was the lack of scientific peer review of the LLNL/UC reports and the methodology developed from the report recommendations (Giannopoulos, 1998). This situation highlights the need for broad, independent, and ongoing scientific peer review of risk-based methodologies.

Suggested Citation:"3 Strengths and Weaknesses of Risk-Based Methodologies." National Research Council. 1999. Environmental Cleanup at Navy Facilities: Risk-Based Methods. Washington, DC: The National Academies Press. doi: 10.17226/6330.
×

TABLE 3-1 Comparison of EPA and ASTM Risk-Based Approaches

Characteristics/Issues

CERCLA(RAGS/SSG)

ASTM pRBCA & cRBCAa

Tiered approach

Soil Screening Guidance only

Yes

Applies to both petroleum and non-petroleum compounds

RCRA-UST - petroleum USTs CERCLA or RCRA - all compounds

Petroleum RBCA - petroleum Chemical RBCA - all compounds

Data requirements

Generally large

Generally large; depends on tier used

Acceptable cancer risk level

10-4 – 10-6

10-4 – 10-6

Allows for either generic or site-specific cleanup standards

Only under the Soil Screening Guidance

Tier 1 is generic Tiers 2 and 3 are site-specific

Uses a prescribed set of models

Provides default models and assumptions with deviation on approval

Models suggested in the appendix and the literature; appendix models are often used

Removal actions allowed

Yes

Yes

Quantification of uncertainty

Not well described

Not well described

Remedial options

Preference for source removal rather than containment and institutional controls

All remedial options available, including institutional controls

Takes into account the 9 NCP criteria

By definition

Not mentioned - pRBCA Yes - cRBCA

Provides options to revisit sites over the long-term

Yes, but seldom practiced

Yes - cRBCA

Underwent scientific peer review

Yes

ASTM peer review only

Public involvement during design

Public comment period

Limited

General perception of the methodology

Costly and time-consuming

Inexpensive and easy

Size of the document(s)

Large

Small

a pRBCA = petroleum RBCA, cRBCA = chemical RBCA

Suggested Citation:"3 Strengths and Weaknesses of Risk-Based Methodologies." National Research Council. 1999. Environmental Cleanup at Navy Facilities: Risk-Based Methods. Washington, DC: The National Academies Press. doi: 10.17226/6330.
×

RBCA standard guides are easy to use and will result in faster site closeout than CERCLA.

Finally, the ASTM RBCA standard guides did not undergo external scientific peer review prior to their release and were designed by a limited group of stakeholders. EPA methodologies, on the other hand, are designed in an open forum in which the public is allowed months to comment, and they are subject to extensive external peer review.

Strengths and Weaknesses of ASTM RBCA

How the Strengths of a Risk-Based Approach Are Manifested in ASTM RBCA

The ASTM RBCA methodology is a streamlined and systematic approach to evaluating contamination, mainly as a result of its tiered structure. The tiered structure enables conservative cleanup goals to be replaced by site-specific cleanup goals, when appropriate. The nature of the problem and the quality of data characterizing the site help determine which tier is chosen. Generally, moving from tier 1 to tiers 2 and 3 requires more site-specific data. The tiered approach also makes a large number of diverse sites more manageable by providing a consistent framework for application at all sites and for all contaminants.

Like other risk-based methodologies, ASTM RBCA requires site characterization to evaluate immediate risks, prioritize sites, and define the need for remedial action. Chemical RBCA, in particular, emphasizes the importance of setting data quality objectives. There is little discussion in the standard guides about the quality of data needed for a tier 1 assessment. Some feel that the needs for high quality data are minimized by ASTM's tier 1 evaluation, while others feel that tier 1 data needs are more extensive than is currently required at sites not using ASTM RBCA. Nonetheless, a consistent requirement for some site-specific data is a strength of ASTM RBCA.

The ASTM RBCA methodology is scientifically defensible in its handling of individual risk assessment calculations. The human exposure and health risk assessment models recommended by ASTM are accepted by the regulatory community. Many of the chemical fate and transport concepts recommended for use in the pathway analysis involve current scientific and engineering understanding.

The ASTM RBCA methodology facilitates the prioritization of sites for cleanup. Table 1 in petroleum RBCA explains the initial classification of sites based on immediate, short-term, long-term, or no demonstrable risk to human health and the environment and prescribes response actions for each site category. This information can be used to rank sites for a variety of purposes, including the allocation of financial resources. Experience in Oklahoma, which is using a modified ASTM RBCA methodology for underground storage tank

Suggested Citation:"3 Strengths and Weaknesses of Risk-Based Methodologies." National Research Council. 1999. Environmental Cleanup at Navy Facilities: Risk-Based Methods. Washington, DC: The National Academies Press. doi: 10.17226/6330.
×

cleanup, has clearly demonstrated the effectiveness of ASTM RBCA in prioritizing sites.

The ASTM RBCA methodology has other unique strengths that account for much of the attention given to the standard guides. For three reasons, the RBCA process may accelerate the cleanup process and expedite site closure: (1) as mentioned above, management options, including remedial actions are available at each tier of the RBCA process (including tier 1); (2) the RBCA methodology makes greater use of engineering and institutional controls, which may take the place of cleanup efforts that are perceived to be time-consuming; and (3) the assessment procedures are standardized for all types of contamination due to the systematic tiered nature of the methodology.

The standardized RBCA process is also beneficial to voluntary cleanup programs. Because it is a well-defined set of rules, the RBCA methodology can be easily used by certified independent risk assessors, and responsible parties will be more likely to implement voluntary cleanup activities. This should allow for a greater number of contaminated sites to be considered under voluntary programs.

Petroleum RBCA's streamlined documentation has facilitated its comprehension and adoption both by states and industry professionals. Though only a qualitative measure, its extensive statewide use (see Chapter 2) supports the notion that petroleum RBCA is an improvement over other available methodologies. Many states have used the ASTM guidelines to develop programs for cleaning up leaking underground storage tanks, and many are considering similar strategies for other chemicals (Begley, 1996). However, even with all the interest in and application of the ASTM RBCA approach, it is still too soon to evaluate comprehensively the success of the methodologies, especially chemical RBCA.

How the Weaknesses of a Risk-Based Approach Are Manifested in ASTM RBCA

Like other risk-based methodologies that leave contamination in place, the ASTM RBCA standard guides suggest remedial options that include combinations of treatment technologies, engineering controls, and institutional controls. All of the negative consequences of remedies that rely on institutional controls will pertain to cleanups done under RBCA. Although the ASTM documents contain information on types on institutional controls, they do not discuss the drawbacks of their use as part of a remedial option or discuss mechanisms for enforcing them.

The potential threat of unidentified compounds left in place is not addressed in the ASTM RBCA methodology. The petroleum RBCA standard guide contains weak language on revisiting sites over the long-term. There is no guidance on how long monitoring (or other measures) should be conducted to ensure protection of public health, or how one might reassess the risk of a hazardous waste site at a later date. If the concentrations of CoCs are less than target levels, and

Suggested Citation:"3 Strengths and Weaknesses of Risk-Based Methodologies." National Research Council. 1999. Environmental Cleanup at Navy Facilities: Risk-Based Methods. Washington, DC: The National Academies Press. doi: 10.17226/6330.
×

the user is confident that sufficient data exist to support the conclusion that concentrations will not exceed target levels in the future, the site can be closed. Chemical RBCA is greatly improved in this respect. It describes more specifically what the purpose of long-term monitoring should be after remedial options are complete. However, chemical RBCA also contains a preliminary step during a tier 1 evaluation that assesses the completeness of the pathways connecting sources of contamination to potential receptors (see Figure 2-5). If it can be shown that relevant exposure pathways are incomplete, sites may be closed prior to comparison of site data with any target cleanup levels. The ASTM has left the interpretation of this figure to the discretion of the state regulatory agencies (Waldorf, 1998). This figure constitutes a significant weakness of the RBCA methodology if it is interpreted to mean that sites can be closed with no further action, rather than being monitored for some period of time to document that relevant exposure pathways remain incomplete.

The potential for risk-based methodologies to discourage development of innovative technologies for source removal has been widely discussed since publication of the ASTM petroleum RBCA standard guide (Hazardous Waste News, 1998; Thompson, 1997). Because the methodology advocates engineering and institutional controls to eliminate receptors and pathways, rather than relying solely on remediation of contaminant sources, there is less incentive to develop new technologies that could remediate a site to unrestricted uses. Critics of the ASTM standard guides suggest that the term ''standard" to describe RBCA may be slowing the development of innovative technologies because this term implies that RBCA is a universal standard.

Uncertainty is not dealt with effectively in the ASTM RBCA standard guides. The guides lack explicit considerations of uncertainty factors throughout. Both ASTM RBCAs confront the issue of uncertainty in risk assessment by assuming that RBSLs and, to a certain degree, SSTLs are conservative numbers that take uncertainty into account. Problems with the use of such conservative cleanup goals are discussed in detail in Chapter 4. Little consideration is given to the variability and uncertainty in the concentrations of the chemicals of concern. Appreciation of sampling errors is lacking; it is assumed that all data collected are accurate and precise. Although use of Monte Carlo techniques is suggested as a part of a tier 3 analysis, it is not mentioned for earlier tiers that use site-specific data. ASTM RBCA makes absolutely no mention of the uncertainties in (1) the toxicity of exposure to multiple chemicals, (2) ecological risk assessment, (3) engineering controls, or (4) institutional controls.

The committee identified other weaknesses in the ASTM RBCA methodology beyond those common to all risk-based approaches. As other parties have noted (Cal/EPA, 1994, 1995; Cooper, 1998), the RBCA standard guides give insufficient consideration to cumulative risk. Cumulative risk assessment is completely absent from petroleum RBCA and appears minimally in chemical RBCA during tiers 2 and 3. To be a valid alternative to source removal or technology-

Suggested Citation:"3 Strengths and Weaknesses of Risk-Based Methodologies." National Research Council. 1999. Environmental Cleanup at Navy Facilities: Risk-Based Methods. Washington, DC: The National Academies Press. doi: 10.17226/6330.
×

based approaches, risk-based approaches must assess the combined risks emanating from multiple contaminants or from multiple pathways on a target receptor.

The RBCA framework is subject to misuse and misinterpretation. For example, the look-up tables and equations presented in the appendixes may be used at sites for which they are inappropriate. Misinterpretation of the document can result if users assume that ASTM RBCA includes all exposure pathways. Unlike other ASTM publications, the RBCA standard guides are not comprehensive tools that require minimal customization on the part of the user. Instead, the methodology was designed so that users could modify the framework to include almost any facet deemed important. Potential misuse has been recognized as a problem by those involved in creation of the RBCA standard guides (Rocco, 1998). ASTM has made efforts to prevent such misinterpretations of the methodology by listing (in section 4.5 of petroleum RBCA and section 4.4 of chemical RBCA) what should be done to apply the frameworks successfully (ASTM, 1998).

The lack of public involvement called for during RBCA implementation is a weakness of the ASTM standard guides, particularly petroleum RBCA. Because the environmental movement is generally wary of risk assessment and risk-based approaches, this lack of public participation can be damaging (Tal, 1997). Chemical RBCA improves upon petroleum RBCA by including a recommendation that the public be involved early in the cleanup process. Two EPA manuals are cited

The excavation of two inactive underground storage tanks is one of thousands taking place at Navy facilities across the country. Courtesy of the U.S. Navy.

Suggested Citation:"3 Strengths and Weaknesses of Risk-Based Methodologies." National Research Council. 1999. Environmental Cleanup at Navy Facilities: Risk-Based Methods. Washington, DC: The National Academies Press. doi: 10.17226/6330.
×

as resources for identifying and communicating with interested stakeholders (EPA, 1992a, 1996). Public involvement in the application of risk-based approaches is critical if the goal is to move away from complete remediation toward a level of environmental contamination that does not exceed acceptable health risks.

Finally, a drawback of both RBCA standard guides that cannot be altered now is their design by a limited group of stakeholders. The participants were not necessarily representative of a diverse range of viewpoints, especially those of environmental advocacy organizations. In addition, the RBCA methodology did not undergo documented external, independent scientific peer review prior to being released. An external peer review would not only have added credibility to the ASTM RBCA approach but it would have helped to remove any perceived bias toward the petroleum and chemical industries. (It should be noted that because ASTM standard guides are voluntary, they are not typically drafted by a broad group of stakeholders, nor do they generally undergo external scientific peer review.)

Strengths and Weaknesses of CERCLA

One of the interesting features of the ASTM RBCA standard guides is their embrace of concepts that are not significantly different from EPA guidance on the same topic. Given all the similarities between the two (as described in Table 3-1), it is not surprising that many strengths and weaknesses of RBCA are shared by CERCLA, RAGS, and the Soil Screening Guidance. There are, however, some important differences between the two. In general, the strengths and weaknesses of a risk-based methodology are not manifested in CERCLA to the same extent as they are in ASTM RBCA.

How the Strengths of a Risk-Based Approach are Manifested in CERCLA

Site characterization is heavily stressed under the CERCLA framework, taking place during both the preliminary assessment/site inspection and the remedial investigation. The equations used during the RAGS risk assessment calculations are also scientifically defensible. Like other risk-based methodologies, CERCLA is systematic. The RI/FS process, which is used nationwide at thousands of hazardous waste sites (including almost 3,000 Navy sites) is well organized and logical. However, unlike RBCA, which uses the tiered approach for all types of contaminated sites, only the Soil Screening Guidance is tiered. The EPA has not developed a similar tiered approach for ground water because remedial options under CERCLA must comply with ARARs, which often specify maximum contaminant levels for ground water. It is true that many states have devised generic screening levels for soil and ground water contamination that are similar to RBCA

Suggested Citation:"3 Strengths and Weaknesses of Risk-Based Methodologies." National Research Council. 1999. Environmental Cleanup at Navy Facilities: Risk-Based Methods. Washington, DC: The National Academies Press. doi: 10.17226/6330.
×

tier 1 RBSLs. However, because of the enormous variability in these screening levels from state to state, their existence does not impart greater consistency to the cleanup process. Thus, the different methods for considering ground water and soil contamination, and the variability between state programs that may or may not use generic screening levels, complicate the CERCLA process relative to ASTM RBCA, and make CERCLA less streamlined and systematic.

Another general strength of risk-based methodologies not clearly apparent in the CERCLA process is the prioritization of sites. The Hazard Ranking System, which is used early in the CERCLA process to determine whether facilities should be placed on the National Priorities List, does not allow for a ranking of sites according to relative risk. RAGS was not designed to be a priority-setting tool, as it requires too much data and occurs too late in the process to generate an initial ranking of sites. The military has created its own strategy for requesting and allocating resources among sites (see Chapter 1). However, this evaluation, which involves a cursory estimation of potential sources, pathways, and receptors, does not rank sites for remedial action.

There are important, unique strengths of CERCLA. It is a well-documented methodology that has a history of use at federal facilities. Many Navy remedial project managers expressed confidence in CERCLA over newer methodologies and disliked the notion of retraining personnel to learn a new methodology. They also were skeptical that local regulators would accept newer methodologies, especially at facilities that were being transferred to the private sector. The public is comfortable with the CERCLA framework, largely because it requires clear community relations strategies and offers legal avenues for appeal. Opportunities for the public to become involved in the cleanup occur primarily during the presentation of the proposed remedy. Finally, because CERCLA was generated through a legislative process, it is less likely to be tied to any special interest other than environmental protection. Thus, the public may perceive CERCLA as providing the best assurance of follow-up commitments (e.g., maintenance of long-term monitoring and institutional controls). Whether these commitments are in fact being met at CERCLA sites, however, is under debate.

How the Weaknesses of a Risk-Based Approach Are Manifested in CERCLA

As with the general strengths, the general weaknesses of risk-based methodologies are also less apparent in the CERCLA process. The EPA prefers remedial options that treat sources of contamination, so all weaknesses related to leaving contamination in place are somewhat lessened. Regarding uncertainty, the EPA has acknowledged its importance in the cleanup process (Browner, 1995; EPA, 1992b), but it has yet to provide concrete guidance on how to quantify and reduce uncertainty. Thus, it is not clear that the CERCLA process includes a more thor-

Suggested Citation:"3 Strengths and Weaknesses of Risk-Based Methodologies." National Research Council. 1999. Environmental Cleanup at Navy Facilities: Risk-Based Methods. Washington, DC: The National Academies Press. doi: 10.17226/6330.
×

ough consideration of uncertainty than the ASTM RBCA methodology. Like ASTM RBCA, there is no indication that the EPA has considered uncertainties inherent in potential remedial options.

Because CERCLA is somewhat less systematic than ASTM RBCA, the process is likely to be slower. An explicit tiered approach, which can make a large number of diverse sites more manageable, is only available for soil contamination, and even then it can be applied only to sites that fit the criteria outlined in the Soil Screening Guidance. Generic screening levels for soil and ground water are available only in some states and tend to be highly variable.

Finally, the National Contingency Plan clearly states that institutional controls are a last choice for remedial options under CERCLA and should be considered only in combination with containment strategies. In the committee's opinion, the use of institutional controls speeds up the remediation process compared to remedies requiring treatment technologies, because the latter take longer to design and implement. Thus, minimal reliance on institutional controls also contributes to the perception that the CERCLA process is slow.

Conclusions

This chapter has illustrated that the ASTM RBCA and CERCLA processes share many similarities. However, the ASTM RBCA methodology exhibits more of the characteristics of a typical risk-based methodology, while the CERCLA process focuses more on source removal and unrestricted use of the resource. The potential speed of the cleanup process and the general perception of the methodologies seem to constitute the greatest differences between the two approaches. Neither CERCLA nor ASTM RBCA deals quantitatively with uncertainty. Because uncertainty is the major weakness of risk-based approaches over source removal, it deserves more recognition than it is currently afforded. The next chapter reviews the sources of uncertainty in both the risk assessment and risk management processes and suggests ways for either reducing or quantifying these uncertainties. The committee considers such activities vital to the successful implementation of a risk-based approach for hazardous waste site cleanup.

References

Air Force/Army/Navy/EPA Working Group. 1998. The Road to Site Closeout (working draft). Washington, D.C.


Begley, R., 1996. Risk-based Remediation Guidelines Take Hold. Environ. Sci. & Technol. 30:438A–441A.

Browner, C. M. 1995. Policy for Risk Characterization at the U.S. Environmental Protection Agency. Washington, D.C.


California Environmental Protection Agency (Cal/EPA). 1994. Memorandum dated 4/4/94 from G. Michael Schum, Office of Scientific Affairs, to Dennis Rounds, chairman of ASTM E50.01 Subcommittee.

Suggested Citation:"3 Strengths and Weaknesses of Risk-Based Methodologies." National Research Council. 1999. Environmental Cleanup at Navy Facilities: Risk-Based Methods. Washington, DC: The National Academies Press. doi: 10.17226/6330.
×

Cal/EPA. 1995. Analysis of RBCA. Memorandum dated 12/1/95 from Ned Butler, staff toxicologist, to Ted Rauh, deputy director, Hazardous Waste Management Program.

California State Water Resources Control Board. 1997. Resolution #01-21-97. Sacramento, Calif.

Cooper, D. 1998. Environmental Protection Agency. Personal communication to Helen Waldorf, chairman of the ASTM chemical RBCA task force.

Environmental Protection Agency (EPA). 1992a. Community Relations in Superfund: A Handbook. EPA540/R-92/009. Washington, D.C.

EPA. 1992b. Understanding Superfund Risk Assessment. USEPA 9285.7-06FS. Washington, D.C.

EPA. 1996. RCRA Public Participation Manual. EPA/530/R-96/007. Washington, D.C.


Farr, J., G. Apostolakis, M. Collins, R. C. Crounch, G. Fogg, M. Reinhard, and K. Scow. 1996. Senate Bill 1764 Advisory Committee's Recommendations Report Regarding California's Underground Storage Tank Program. Submitted to the State Water Resources Control Board pursuant to the requirements of section 25299.38 of the California Health and Safety Code. Sacramento, Calif.


Giannopoulos, J. 1998. California State Water Resources Control Board. Personal communication.


Hazardous Waste News. 1998. ASTM Debate Continues. Hazard. Waste News 20(9):65.


McNab, W. W., B. P. Dooher, D. W. Rice, M. C. Kavanaugh, P. C. Johnson, S. J. Cullen, L. G. Everett, W. E. Kastenberg, and M. C. Small. 1997a. Assessment of Appropriate Fuel Hydrocarbon Risk-Management Strategies for George Air Force Base, Victorville, California, Using a Risk-Based Approach. Lawrence Livermore National Laboratory and the University of California.

McNab, W. W., B. P. Dooher, D. W. Rice, M. C. Kavanaugh, P. C. Johnson, S. J. Cullen, L. G. Everett, W. E. Kastenberg, and M. C. Small. 1997b. Assessment of Appropriate Fuel Hydrocarbon Cleanup Strategies for Travis Air Force Base, Fairfield, California, Using a Risk-Based Approach. Lawrence Livermore National Laboratory and the University of California.


National Research Council (NRC). 1997. Innovations in Ground Water and Soil Cleanup: From Concept to Commercialization. National Academy Press, Washington, D.C.


Pettit, W. 1995. Lawrence Livermore National Laboratory (LLNL) Report on Leaking Underground Storage Tank (UST) Cleanup. Memorandum dated 12/8/95 to all regional water board chairpersons, all regional water board executive officers, and all LOP agency directors.


Rice, D. W., B. P. Dooher, S. J. Cullen, L. G. Everett, W. E. Kastenberg, R. D. Grose, and M. A. Marino. 1995a. Recommendations to Improve the Cleanup Process for California's Leaking Underground Fuel Tanks (LUFTs). Lawrence Livermore National Laboratory and the University of California.

Rice, D. W., J. C. Michaelson, B. P. Dooher, D. H. MacQueen, S. J. Cullen, L. G. Everett, W. E. Kastenberg, R. D. Grose, and M. A. Marino. 1995b. California's Leaking Underground Fuel Tank (LUFT) Historical Case Analyses. Lawrence Livermore National Laboratory and the University of California.

Rice, D. W., B. P. Dooher, S. J. Cullen, L. G. Everett, W. E. Kastenberg, and R. C. Ragaini. 1997. Response to U.S. EPA Comments on the LLNL/UC LUFT Cleanup Recommendations and California Historical Case Analysis. Lawrence Livermore National Laboratory and the University of California.

Rice, D. W., and M. Kavanaugh. 1997. Site Assessment Review to Apply Risk-Based Corrective Action at Site 390 Marine Corps Air Station El Toro, California. Lawrence Livermore National Laboratory.

Rocco, J. 1998. British Petroleum Oil. Personal communication.


Tal, A. 1997. Assessing the Environmental Movement's Attitudes Toward Risk Assessment. Environ. Sci. Technol. 31:470A–476A.

Thompson, D. 1997. ASTM - A Good Thing Gone Astray. Civil Engineering. September, 1997.


Waldorf, H. 1998. Massachusetts Department of Environmental Protection. Personal communication.

Suggested Citation:"3 Strengths and Weaknesses of Risk-Based Methodologies." National Research Council. 1999. Environmental Cleanup at Navy Facilities: Risk-Based Methods. Washington, DC: The National Academies Press. doi: 10.17226/6330.
×
Page 68
Suggested Citation:"3 Strengths and Weaknesses of Risk-Based Methodologies." National Research Council. 1999. Environmental Cleanup at Navy Facilities: Risk-Based Methods. Washington, DC: The National Academies Press. doi: 10.17226/6330.
×
Page 69
Suggested Citation:"3 Strengths and Weaknesses of Risk-Based Methodologies." National Research Council. 1999. Environmental Cleanup at Navy Facilities: Risk-Based Methods. Washington, DC: The National Academies Press. doi: 10.17226/6330.
×
Page 70
Suggested Citation:"3 Strengths and Weaknesses of Risk-Based Methodologies." National Research Council. 1999. Environmental Cleanup at Navy Facilities: Risk-Based Methods. Washington, DC: The National Academies Press. doi: 10.17226/6330.
×
Page 71
Suggested Citation:"3 Strengths and Weaknesses of Risk-Based Methodologies." National Research Council. 1999. Environmental Cleanup at Navy Facilities: Risk-Based Methods. Washington, DC: The National Academies Press. doi: 10.17226/6330.
×
Page 72
Suggested Citation:"3 Strengths and Weaknesses of Risk-Based Methodologies." National Research Council. 1999. Environmental Cleanup at Navy Facilities: Risk-Based Methods. Washington, DC: The National Academies Press. doi: 10.17226/6330.
×
Page 73
Suggested Citation:"3 Strengths and Weaknesses of Risk-Based Methodologies." National Research Council. 1999. Environmental Cleanup at Navy Facilities: Risk-Based Methods. Washington, DC: The National Academies Press. doi: 10.17226/6330.
×
Page 74
Suggested Citation:"3 Strengths and Weaknesses of Risk-Based Methodologies." National Research Council. 1999. Environmental Cleanup at Navy Facilities: Risk-Based Methods. Washington, DC: The National Academies Press. doi: 10.17226/6330.
×
Page 75
Suggested Citation:"3 Strengths and Weaknesses of Risk-Based Methodologies." National Research Council. 1999. Environmental Cleanup at Navy Facilities: Risk-Based Methods. Washington, DC: The National Academies Press. doi: 10.17226/6330.
×
Page 76
Suggested Citation:"3 Strengths and Weaknesses of Risk-Based Methodologies." National Research Council. 1999. Environmental Cleanup at Navy Facilities: Risk-Based Methods. Washington, DC: The National Academies Press. doi: 10.17226/6330.
×
Page 77
Suggested Citation:"3 Strengths and Weaknesses of Risk-Based Methodologies." National Research Council. 1999. Environmental Cleanup at Navy Facilities: Risk-Based Methods. Washington, DC: The National Academies Press. doi: 10.17226/6330.
×
Page 78
Suggested Citation:"3 Strengths and Weaknesses of Risk-Based Methodologies." National Research Council. 1999. Environmental Cleanup at Navy Facilities: Risk-Based Methods. Washington, DC: The National Academies Press. doi: 10.17226/6330.
×
Page 79
Suggested Citation:"3 Strengths and Weaknesses of Risk-Based Methodologies." National Research Council. 1999. Environmental Cleanup at Navy Facilities: Risk-Based Methods. Washington, DC: The National Academies Press. doi: 10.17226/6330.
×
Page 80
Suggested Citation:"3 Strengths and Weaknesses of Risk-Based Methodologies." National Research Council. 1999. Environmental Cleanup at Navy Facilities: Risk-Based Methods. Washington, DC: The National Academies Press. doi: 10.17226/6330.
×
Page 81
Suggested Citation:"3 Strengths and Weaknesses of Risk-Based Methodologies." National Research Council. 1999. Environmental Cleanup at Navy Facilities: Risk-Based Methods. Washington, DC: The National Academies Press. doi: 10.17226/6330.
×
Page 82
Next: 4 Uncertainty in Risk-Based Methodologies »
Environmental Cleanup at Navy Facilities: Risk-Based Methods Get This Book
×
Buy Paperback | $68.00 Buy Ebook | $54.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The fiscal and technological limitations associated with cleaning up hazardous waste sites to background conditions have prompted responsible parties to turn to risk-based methods for environmental rememdiation.

Environmental Cleanup at Navy Facilities reviews and critiques risk-based methods, including those developed by the U.S. Environmental Protection Agency and the American Society of Testing and Materials. These critiques lead to the identification of eleven criteria that must be part of any risk-based methodology adopted by the Navy, a responsible party with a large number of complex and heavily contaminated waste sites. January

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!