Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 211
6 Issues in the Development and Use of Models INTRODUCTION In the United States, two agencies, the U.S. Nuclear Regulatory Commission (USNRC) and the Environmental Protection Agency (EPA), are particularly concerned with ground water modeling to support many of their regulatory activities. Their experience with the uses of models has been completely different. The USNRC, while placing considerable emphasis on developing guidance for the selection and use of models, has never really employed them for regulatory purposes. The USNRC's low-level waste (ELM) program has yet to be tested, because no applications for licenses have been received by the USNRC. In any case, the USNRC is likely to receive fewer than 10 applications for disposal sites. The high-level radioactive waste program is also untested. License applications for high-level waste repositories have not been received, and none are expected before 1995. The EPA's experience in using models is documented to a much greater extent because of the number of active sites under its ju- risdiction. Models play an important role in EPA-related activities; however, many problems related to the use of models have emerged. For example, prior reviews of the Superfund cleanup process have concluded the following: 211

OCR for page 211
212 GROUND WATER MODELS ~ all analytical methodologies suffer from a lack of knowledge on the fundamental process underlying observed phenomena (National Research Council, 1988~; ~ models do not account for all the processes affecting the fate and impact of the contaminants (National Research Council, 1988~; ~ models lack accuracy when confronted with a high degree of heterogeneity (complex hydrogeology, multiple contaminants, two- phase flow, and variable susceptibility in populations) (National Re- search Council, 1988~; ~ there is no clear guidance provided by agencies concerning when to use and how to select models (International Ground Water Modeling Center, 1986; Office of Technology Assessment, 1982~; ~ the decision concerning when to use a mode! and which code to use is often left to the discretion of the contractor who was hired by EPA or a potentially responsible party (International Ground Water Modeling Center, 1986~; ~ there is limited understanding among EPA staff concerning which models are available (International Ground Water Modeling Center, 1986~; , there is inadequate expertise within federal and state regula- tory agencies to apply such models (Office of Technology Assessment, 1982~; the validity of some codes for the problem to which they are applied has not been established (Office of Technology Assessment, 1982~; ~ EPA enforcement offices strongly discourage the use of propri- etary models (International Ground Water Modeling Center, 1986; Office of Technology Assessment, 1982~; ~ there is inadequate quality assurance, quality control, and peer review (Office of Technology Assessment, 1982~; and ~ there is a reluctance to use models if their use would be considered controversial (Office of Technology Assessment, 1982~. The committee's review confirmed most of these findings. The problem is not a lack of appropriate documents to guide the modeling process. One can see from the list that the basic problems concern the lack of training and experience in the people who are choosing and using models, deficiencies or limitations in the codes themselves, and scientific barriers that determine to what extent models are able to incorporate relevant processes. The committee addresses these issues in this chapter.

OCR for page 211
DEVELOPMENT AND USE OF MODELS THE PEOPLE PROBLEM 213 It should be apparent from earlier chapters outlining the state of the science that modeling ground water flow and contaminant transport is not a trivial exercise. Ideally, a modeler should have a broad background in earth sciences with particular strengths in hydrogeology, low-temperature geochemistry, and analytical and nu- merical mathematics. This background will have developed through graduate and undergraduate studies and will have been tempered by relevant experience. A significant problem in dealing with regulatory agencies is the lack of individuals who are trained at an appropriate level to understand and use models. For example, EPA's ground water and contaminant transport modeling needs currently outpace its actual use of models in virtu- ally all program areas (Office of Technology Assessment, 1982~. EPA currently has an insufficient number of qualified and experienced hy- drogeologists and other professionals knowledgeable in contaminant transport modeling (Office of Technology Assessment, 1982~. Super- fund hydrogeologists are quitting their jobs at a rate 6 times higher than the average for other federal government employees (General Accounting Office, 1987~. The more experienced hydrogeologists are leaving EPA at a higher rate than the younger professionals, and the situation is likely to become worse. Most states possess even more limited capabilities (Council of State Governments, 1985; Environ- mental Protection Agency, 1987; General Accounting Office, 1987; International Ground Water Modeling Center, 1986~. The substan- tial increase in the need for site-specific regulatory decisions in all the EPA programs concerned with regulating ground water can only exacerbate the breadth and depth of these shortages and critical neecis. Contaminant transport models simply cannot be used unless people who are experts in ground water processes and models are available to select, apply, and peer-review such models. As a result, EPA's system for selecting and applying models is guaranteed to result in misuses of such models. The solution to this problem will require (~) recruiting and re- taining more qualified and experienced personnel; (2) establishing specific guidelines Ed criteria for the use of contaminant flow mod- els; (3) instituting peer review techniques; and (4) providing technical assistance and additional training. The lack of qualified individuals in the regulatory agencies at all levels has had some significant ramifications. For example, some

OCR for page 211
214 GROUND WATER MODELS of the methods chosen to expedite hazardous waste cleanups are contrary to good science. EPA's policy of performing remedial inves- tigations in less than 6 months often provides insufficient data for a complete characterization of the site. Given the seasonal character of ground water flow, it would be prudent to measure water levels over a longer time frame. The rush to judgment on Superfund remedies risks more than a "bad" scientific decision or an economically wasteful cleanup. Deci- sions based on inadequate data may aggravate a problem or, at least, prolong its eventual remedy. The prudent, if not necessary, course of action in such cases is to proceed in orderly phases, such as in the S-Area case study. The committee recognizes the desirability of, and public mandate for, expediting hazardous waste cleanups. Those components of the remedial action for a site that reasonably can be taken with limited data- for example, interception and treatment of the ground water plume should be implemented immediately. Other components of a site remecliation plan can be implemented at a more measured pace once the primary potential source of exposure is eliminated. Another problem generated through inexperience is an overre- liance on the results of a modeling exercise. Computer models have a unique capacity to appear more certain, more precise, and more authoritative than they really are. As a result, assumptions, even wholly unrealistic ones, can be stated with deceptive precision and seeming accuracy by being included in a computer model. Special care therefore must be taken in presenting the results of such modeling. Decisionmakers (whether they be heads of agencies, judges, or juries) must understand the distinction between scientific fact and science policy. If policy is relied on to make a decision, the policy rationale should be explicitly identified. Faced with the problem of an overall lack of qualified staff to use models and interpret results, regulatory agencies have a natural tendency toward simplification through the use of standard models and worst-case assumptions, as is done in the hazardous waste delist- ing program (Environmental Protection Agency, 1987; International Ground Water Modeling Center, 1986~. This decision is motivated by a concern about the lack of adequate resources and a preference for using overprotective assumptions. There is an inherent conflict between using more complex, site-specific models and using simpler models; i.e., "isitandardization may increase consistency, but tends

OCR for page 211
DEVELOPMENT AND USE OF MODELS 215 to trade off accuracy, producing answers that are not always appro- priate in all situations" (Environmental Protection Agency, 1987~. A site-specific decision should be based on the actual conditions existing at a site. More certainty should be required if the output of the mode! is used directly to trigger additional regulatory action than if the mode! is used as an interpretative too! to better understand how contaminants migrate near the site. The committee recognizes the need to follow the mandate of the enabling statutes, use health-protective assumptions, and consider the practical limitations on agency resources. The committee, how- ever, believes that the use of standard models at specific sites lacks a scientific basis. The use of overly simplistic models, such as the vertical-horizontal spread (VHS) model, at Superfund sites or other hazardous waste sites (1) would be an arbitrary distortion of the remedial selection process, (2) could reduce protection of the pub- lic health by misallocating finite cleanup resources, and (3) would result in the imposition of substantial costs with no commensurate environmental or public health benefit. The Environmental Protection Agency's choice of remedies can also be affected by the choice of mode} and the assumptions used in such a model. For example, EPA may use an advection-dispersion contaminant transport mode} to predict the future concentrations of chemicals at local drinking water wells to derive the on-site soil cleanup levels (i.e., soil levels that would not result in off-site ground water concentrations above health-based ground water cleanup lev- els) (Record of Decision, July 1985, McKin Site, Maine, RO1-85-009) or to estimate the time that it will take to achieve various cleanup levels by alternative remedial actions (Record of Decision, August 1985, Old Mill, Ohio, RO5-85-018; Record of Decision, September 1987a, Su~ern Well Field, New York, RO2-87-042~. Such a mode! will not take into account dilution, adsorption, volatilization, or biodegradation and other more realistic features (Record of Deci- sion, September 1987b, Rose Township, Michigan, RO5-87-052~. For example, an advection-dispersion mode! generally will overestimate the concentration and underestimate the travel time for the contam- inants, thus making the problem appear much more serious than it is in reality. Extreme worst-case assumptions can drive the remecly selection process toward draconian and extremely costly remedies. The se- lection of these assumptions as input to models is also prone to misuse (Pesticide and Toxic Chemical News, 1987~. The difference

OCR for page 211
216 GROUND WATER MODELS between worst-case assumptions and levels predicted by contaminant transport modeling (no less real) can be substantial. The benefit of using extreme worst-case assumptions is often simply administrative convenience to the agency; i.e., using such as- sumptions eliminates the need to obtain additional data and make difficult expert judgments. This benefit must be weighed against the additional cost or the possibility that the assumption will sig- nificantly underestimate the risk. Worst-case assumptions should never be preferred over actual data. Some assumptions may be so unrealistic that their use is inappropriate. UNCERTAINTY AND RELIABILITY Modeling can be defined as the art and science of collecting a set of discrete observations (our incomplete knowledge of the real worId) and producing predictions of the behavior of a system. Such predictions will be necessarily uncertain, as will be our knowledge of the true behavior of the system. The goal of this section is to identify and discuss the scientific, technical, and practical issues that arise in applying models to particular sites, and to develop proce- dures and guidelines to help assure that these issues are addressed during the mode! application process. A convenient framework for organizing a discussion of uncertainty and reliability in modeling is presented by Figure 6.1. What is shown is one possible representation of the process of applying a mode] to a regulatory (or other) deci- sionmaking problem. This representation rests on the assumption that the ultimate goal of a modeling exercise is a prediction of the behavior of the real world. That is, there is a "true" system, made up of the geologic environment (the soils and/or aquifers), climatic stresses (precipitation and evaporation), subsurface flora and fauna, and human-induced stresses (e.g., irrigation welis). The success of a modeling exercise will depend on the degree to which the mode! pre- diction agrees with the behavior of this true system. Therefore the reference in discussing and/or assessing the accuracy of the modeling process is this real system, indicated by the top path of Figure 6.1. The state and characteristics of the real world may be described by a set of information termed the inputs to the system, such as the spatial distribution of soil and aquifer properties, or the time histories of system stresses. These inputs are often highly variable in time and/or space. Some may be inherently uncertain, such as future time series of rainfall infiltration and subsequent recharge.

OCR for page 211
DEVELOPMENT AND USE OF MODELS Reference inputs Sampling errors Known I inputs I - REFERENCE SYSTEM (Distributed) SAMPLING PROCESS Measurements , I Sampling strategy INPUT ESTIMATION PROCESS Prior Estimation information strategy 217 Reference output l Prediction Modeling error strategy _ _: _, Estimated inputs MODEL (Discretized) 1 Predicted output FIGURE 6.1 Conceptual framework for ground water model accuracy analysis. SOURCE: McLaughlin and Wood, 1988a. The processes at work in the real system, including those induced by proposed management actions, act on these inputs to yield the true, or real, outputs that characterize the behavior of the system. Such outputs might be contaminant concentration distributions in space and time, travel times, mass losses, or exposure levels and durations at selected locations. These true outputs are, of course, themselves often variable and uncertain. Even though ground water flow and transport systems tend to smooth out the variability of inputs, much variability and uncertainty remain in the true outputs. The following sections use this conceptual mode} to describe the major sources of uncertainty in the modeling process. The Sampling Process One can observe the real world only via a sampling process. We make a finite number of observations, choosing what parameters to measure, how to measure them (what instrument to use), where to measure them, and when to measure them. In other words, a sam- pling scheme is designed and implemented. For example, one might

OCR for page 211
218 GROUND WA1'ER MODELS collect a set of cores during well drilling and measure the permeabil- ity of subsamples of each core in the laboratory using a permeameter. Alternatively, one might collect a set of water samples from wells and analyze each in the laboratory for contaminant concentration. Such a sampling scheme typically provides a set of discrete quantitative observations of one or more parameters of interest, or sometimes more continuous, qualitative information about the system (e.g., the geologic sedimentary environment). A sampling process introduces uncertainty. First, the measure- ment process itself introduces uncertainty in the form of instrument errors. Every measurement device has associated with it a mea- surement error. Such errors usually contain a random (uncertain) component (and are often biased). Second, the sampling process introduces uncertainty because of incomplete information. The sys- tem can be observed only at a small set of points, and conditions between sampling points are not known with certainty, whether in space or time or both. This uncertainty is obviously most critical for systems characterized by significant spatial and temporal variability. Thus the real system is an uncertain one because of (1) its inherent randomness, (2) measurement error, and perhaps most important, (3) limited sampling of the highly variable physical, chemical, and biological properties of ground water systems. This uncertainty ap- plies to both the inputs and the outputs of the system. All modeling is conducted without certain knowledge of the true state of a ground water environment. The magnitude of our uncertainty is a func- tion of the spatial heterogeneity and temporal variability of aquifer properties, boundary conditions, dependent variables, the density of observation points relative to the scale of the variability, and the measurement techniques. With these general concepts in mind, we can address more specific issues concerned with field sampling and data collection. Field sampling, experimental design, and related data analysis issues are topics that have not traditionally received much attention from ground water modelers. While most modelers appreciate the need for good field data, they have often had to depend on others for the data used in their models. Published field data have typi- cally been taken at face value and have been freely extrapolated and generalized beyond their original purpose. This situation has begun to change, partly as a result of the demanding requirements of haz- ardous waste studies and partly because modelers are beginning to

OCR for page 211
DEVELOPMENT AND USE OF MODELS 219 take a broader view of the modeling process, which recognizes that data issues need to be taken seriously. As mentioned earlier in this report, ground water systems are difficult to observe and describe, not only because they are hidden from view, but also because they are three-dimensional and often very heterogeneous. Hydrogeological properties observed at one lo- cation may give relatively little information about conditions only a few meters away. Soil strata or rock fractures only a few centimeters thick may greatly influence the movement of water and contaminants but pass undetected in a typical field survey. Such heterogeneities limit our ability to generalize from laboratory measurements to field conditions or from one site to another. The ground water sampling problem is complicated further by the expense of well drilling, which is still the primary method used to gain information about subsurface flow and transport. Drilling is time-consuming and labor intensive, and requires specialized equipment. Moreover, the drilling process disturbs the subsurface environment and, as a result, compromises the accuracy of pump tests and contaminant data collected from observation wells. Although alternative sampling methods based on geophysical or remote sensing technology have been applied success- fully in some situations, they are generally even less reliable than well samples. The expense, difficulty, and inaccuracy of field sam- pling all tent] to have an adverse impact on ground water modeling. Most modeling studies must make do with a very limited amount of unreliable data, which at best give only a rough picture of actual subsurface conditions. This basic fact needs to be recognized in any realistic assessment of the prediction capabilities of ground water models. Generally speaking, the field data used to estimate the inputs and check the predictions of ground water flow models are compiled from historical hydrogeologic surveys that were not planned with modeling in mind. Examples include periodic status reports issued by irrigation districts and state water agencies (primarily in the western United States), U.S. Geological Survey (USGS) water supply papers and open file reports, and water resource atlases compiled by a number of different governmental agencies. Until recently, many of the data included in these surveys were collected by local well drillers and geologists concerned primarily with water supply. These data tend to cover regions that are larger than those of interest in ground water contamination studies and therefore rarely deal with local geologic or hydrologic anomalies that may control transport

OCR for page 211
220 GROUND WATER MODELS in the vicinity of a hazardous waste site. In most hazardous waste studies, these traditional data sources are useful only for defining the boundary conditions of a site-specific flow model. The field data used in contaminant transport models typically have a very different history from those used in flow models. Most contaminant concentration measurements are collected at or near a contaminate site after an indication that some problem exists (e.g., observations of unusual taste or odor in well water). These mea- surements are usually limited and scattered, reflecting the locations of existing water supply wells rather than the geometry of the con- taminant plume (or plumes). Furthermore, contaminant data may be even more difficult to interpret than hydrogeologic data because the compounds observed and their physical state depend on chemical and biological conditions in the subsurface environment (see Chapter 2~. These comments suggest that there will be a need for a special- ized problem-oriented sampling program at most hazardous waste sites. Because sampling resources are nearly always quite limited, the objectives of the sampling program need to be spelled out care- fully so that a systematic and cost-effective field strategy can be developed. This strategy needs to be flexible enough to be able to deal with unanticipated results and unforeseen logistic problems but specific enough to provide guidance to drilling crews and managers responsible for approving budget expenditures. The dichotomy of flexibility and specificity is one that arises time and again in practi- cal sample programs. A site-specific hazardous waste field sampling program may have many different objectives, which can exert conflicting demands on limited resources. Some frequently encountered objectives include the following: ~ assessment of the severity of a newly discovered contamina- tion problem (i.e., a reconnaissance study); ~ monitoring of a known but more or less controlled hazardous waste site (e.g., for enforcement of a consent decree); monitoring of the performance of a remediation strategy (e.g., a pumping, treatment, and reinfection system); and ~ acquisition of data needed to develop or test a predictive model. Because this report is primarily concerned with ground water mod- eling, the focus is on the last of these objectives. It should be noted,

OCR for page 211
DEVELOPMENT AND USE OF MODELS 221 however, that modelers may need to reconcile their needs with those of other data users competing for limited resources and, in the pro- cess, may be forced to make compromises and adjustments in their approach. Recently, there has been a significant increase in research con the design of model-oriented ground water monitoring programs (Chu et al., 1987; Graham and McI,aughlin, 1989a,b; Knopma~ and Voss, 1987, 1988; McLaughlin and Wood, 198Ba,b). Although the specific methods proposed differ considerably, they generally view data col- lection an a way to reduce uncertainty. If it is possible to relate a specific data collection strategy to the uncertainty inherent in mod- eling, then it is possible to compare different strategies and select the one that is, in some sense, the best. Field sampling studies can, at [east in principle, help to reduce the major types of uncertainty including (1) lack of knowledge about the processes that control contaminant transport and transformation at a particular site and (2) incomplete knowledge of the spatially and temporally variable environmental factors that influence these processes. In fact, it is useful to divide a model-oriented sampling program into two phases: the first (less structured) phase attempts to identify relevant transport processes, whereas the second (more specific) phase attempts to quantify heterogeneous hydrogeologic and biochemical properties. Each of these is briefly discussed below. Process and Parameter Identification There is no truly systematic way to identify the physical, chem- ical, and biological processes at work at a particular contaminated site. This is a difficult scientific and engineering problem that re- quires creativity and experience as well as a good ability to identify inconsistencies and suspicious anomalies in a limited set of observa- tions. Nevertheless, it is possible to state three general principles that may help structure the field studies needed to support a sit~specific mode} development effort. I. A site-specific description of contaminant transport is strongly dependent on the quality of the flow mode} used to develop estimates of subsurface water velocities. Considerable care should be used in developing the inputs to the flow model, particularly in reference to the following:

OCR for page 211
238 GROUND WATER MODELS models is generally limited to extended verifications, using existing analytical solutions, and to code comparisons. In the comparison of codes, a newly developed mode} is compared with established models designed to solve the same type of problems. If the results from the new code do not deviate significantly from those obtained tenth the existing code, a relative or comparative validity is established. However, if significant differences occur, in- depth analysis of the results and codes is required. If code comparison was used to evaluate a new code, all the involved models should again be validated as soon as adequate data sets become available. Various approaches to field validation of a mode! are viable. Therefore the validation process should start with defining validation scenarios. Field validation should include the following steps (Hern et al., 1985~: . Define data needs for validation and select an available data set or arrange for a site. Assess the quality of data in terms of accuracy (measurement errors), precision, and completeness. Define performance or acceptance criteria of the model. Develop strategy for analysis of sensitivity. ~ Perform validation runs and compare performance of the mode} with established acceptance criteria. Document the validation exercise in detail. Recordl~eeping Quality assurance for development and maintenance of codes should include complete recor~keeping of the development, modifi- cations, and phase validation of the code. The paper trail for QA in the development phase consists of reports and files on the develop- ment and testing of the model. Software Documentation Software documentation explains all pertinent aspects of the sys- tem represented in the software, including purposes, methods, logic, relationships, capabilities, and limitations (Gass, 1979~. Complete documentation consists of information recorded during the design, development, and maintenance of computer applications. It is the principal instrument used by those involved in a modeling effort,

OCR for page 211
DEVELOPMENT AND USE OF MODELS 239 such as authors, programmers, users, and system operators, to com- municate efficiently regarding all aspects of the software. Good documentation includes a complete treatment of the equa- tions on which the mode} is based, the underlying assumptions, the boundary conditions that can be incorporated in the model, the method used to solve the equations, and any limitations related to the particular method of solution. The documentation must also include instructions for operating the code and preparing data files, example problems complete with input and output, programmer's instructions, operator's instructions, and a report of the verification. The importance of clear documentation cannot be overemphasized. Improper documentation will prevent a code from being adequately reviewed and could propagate errors in code use. Documentation should commence at the very beginning of a software development project. Scientific and Ethnical Review Generally, the complete scientific and technical review process is qualitative in nature and comprises examination of mode! concepts, governing equations, and algorithms chosen, as well as evaluation of documentation and general ease of use, inspection of the structure of the program and the logic, handling of errors, and examination of the coding (Bryant and Wilburn, 1987; van der Beige et al., 1985b). If verification or validation runs have been made, the review process should inclucle evaluation of these processes. To facilitate thorough review of the model, detailed documenta- tion of the mode} and its developmental history is required, as is the availability of the source code for inspection. In addition, to ensure independent evaluation of the reproducibility of the results of verifi- cation and validation, the computer code should be available for use by the reviewer, together with files containing the original test data used in the verification and validation. MODEL APPLICATION Quality assurance in mode! application studies includes review of the selection of data, data analysis procedures, methodology of modeling, and administrative procedures and auditing. To a large extent, the quality of a modeling study is determined by the expertise of the team involved in the modeling and quality assessment.

OCR for page 211
240 GROUND WATER MODELS In many cases, the person developing the mode! may never have seen or visited the field area. This can easily lead to fatal flaws in the mode! design and parameter estimation, because significant (and perhaps obvious) hydrogeologic features are not recognized and in- corporated into the model. This problem may be exacerbated if the mode] designer or user has expertise with mathematics, numerical analysis, and/or computer simulation methods, but little field expe- rience. In larger organizations, it is common to have such a division of labor, particularly when projects have relatively short deadlines. Where efforts are so divided, modeling may be performed by an en- tirely different and separate group of specialists. Unfortunately, this may produce a tendency for the mode} to become an end unto itself, rather than a means (or one of many tools) for analysis and problem solving. There may also be a tendency for such projects to not fully recognize or accommodate the need for the mode} developer to have familiarity with the field area or allow time for analysts to benefit from feedback between the mode! analysis and the field investiga- tion. On the other hand, if the same person or team of analysts is performing data analysis, data collection, and modeling, it is more likely that the mode! will include realistic and appropriate boundary conditions, system properties, and discretization. It is too easy to calibrate (validate) a mode} while being unaware of major springs, pumping wells, and surface drains or ditches (or other features) that may be controlling ground water levels and gradients in an area. Ignorance of one feature is compensated by errors in values specified for other parameters. A locally steep hydraulic gradient that exists because of a drainage ditch may be interpreted as indicating a low- transm~ssivity zone. Such ignorance of the field area can lead to a mode} that matches historical data but fails in a predictive mode. (Prediction, of course, is purportedly one of the primary values and incentives for using deterministic models. If the goal were merely to achieve a best fit to observed data, then a purely statistical model, such as a multiple regression equation, would most easily meet that objective.) Thus a well-calibrated and validated mode! is not nec- essarily an accurate or a reliable one. This fact is supported by Freyberg (1988), who reports on a numerical experiment in which nine groups of analysts used the same numerical mode! and identical sets of observed data to calibrate the mode! and predict the response to a specified change in a boundary condition. Success in predic- tion was unrelated to success in matching observed heads, and good calibration alone did not lead to good prediction.

OCR for page 211
DEVELOPMENT AND USE OF MODELS 241 In summary, a ground water mode} (or any scientific mode} or theory, for that matter) can never be proven, verified, or validated in the strictest sense of the terms by agreement ninth a specific set of observations. Rather, a mode} can only be invalidated by disagreement with observations. Agreement should serve only to increase confidence in the theory or model. Quality assurance in code application should cover all facets of the modeling process. It should address issues such as the following: project description and objectives; · correct and clear formulation of problems to be solved: type of modeling approach to the project; . conceptualization of system and processes, including hydro- geologic framework, boundary conditions, stresses, and controb; . detailed description of assumptions and simplifications, both explicit and implicit (to be subject to critical peer review); . data acquisition and interpretation; mode} selection or justification for choosing to develop a new model; . mode! preparation (parameter selection, data entry, or refor- matting, "ridding); . the validity of the parameter values used in the mode! appli- cation; . protocols for estimating values of controlling parameters and for steps to be followed in calibrating a model; · level of information in computer output (variables and pa- rameters displayed, formats, layout); . identification of calibration goals and evaluation of how well they have been met; . role of sensitivity analysis; · postsimulation analysis (including verification of reasonabil- ity of results, interpretation of results, uncertainty analysis, and the use of manual or automatic data processing techniques, as for contouring); . establishment of appropriate performance targets (e.g., a 6- ft head error should be compared with a 20-ft head gradient or drawdown, not with the Waft aquifer thickness) that recognize the limits of the data; presentation and documentation of results; and . evaluation of how closely the modeling results answer the questions raised by management.

OCR for page 211
242 GROUND WATER MODELS In exceptional circumstances, it may be possible to conduct what has come to be referred to as a postaudit. A postaudit compares mode! predictions to the actual outcome in field conditions. Al- though postaudits are used primarily to determine the success rate of a mode} application, positive results of a well-executed postaudit analysis contribute to the acceptability of the mode} itself. To use a postaudit successfully in conceptualization, assumptions, and sys- tem parameters and stresses, it should be evaluated and, if necessary, updated and the mode} rerun to facilitate comparison of predictions with recent, observed system responses. The importance of postaudit studies has been outlined by Konikow (1986), Lewis and Goldstein (1982), and Person and Konikow (1986~. An example that illustrates the importance of postaudits is the Snake River plain case study described in Chapter 5. An increasing number of costly decisions are made in part on the basis of the outcome of modeling studies. In the light of major differences noted in comparative studies on mode} application (e.g., Freyberg, 1988; McLaughlin and Johnson, 1987), it should not come as a complete surprise that several groups modeling the same problem may obtain different results. While this is not a QA issue, provisions might have to be made to resolve the inconsistencies in the modeling effort. A third team or a pane] can be created to review and compare the results of both modeling efforts and to assess the importance and nature of differences present. Quality assurance is the responsibility of both the project team and the contracting or supervising organization. It should not drive or manage the direction of a project, nor is it intended to be an after-the-fact filing of technical data. Although the need for QA programs ~ apparent, the extent to which they are being applied in practice can be variable. For example, EPA rarely uses peer review for models applied in the Superfund and Resource Compensation and Recovery Act (RCRA) programs. Only recently has EPA provided a checklist of steps that a modeler must take to assure that a mode} is valid. When carefu! peer review and oversight of the development and application of contaminant transport models have been performed, the quality of the modeling has been good (see the S-Area case study). The application of contaminant models can be greatly improved by the use of peer review experts. Every mode} used by or relied on by EPA, including those in the Superfund program, should go through peer review. (Various groups have endorsed peer review

OCR for page 211
DEVELOPMENT AND USE OF MODELS 243 in the regulatory system, e.g., Administrative Conference of the United States, Recommendation No. 82-5, Advisory Panels No. 1, 1 CFR 305.82.5, 1988.) This review could involve the mathematical code, the hydrogeological/chemical/ biological conceptualization, the adequacy of the data, and the application of the mode] to the site- specific data. Additionally, the peer review should consider whether the prediction being called for exceeds the scientific validity of the model, e.g., the prediction of a concentration over a 10,000-yr period with a mode! validated over a 10-yr period. BIBLIOGRAPHY Adrion, W. R., M. A. Branstad, and J. C. Cherniasky. 1981. Validation, Verification, and Testing of Computer Software. NBS Special Publication 500-75, Institute for Computer Science and Technology, National Bureau of Standards, U.S. Department of Commerce, Washington, D.C. Adrion, W. R., M. A. Branstad, and J. C. Cherniasky. 1982. Validation, verification, and testing of computer software. ACM Computing Surveys 14~2), 159-192. American Society for Testing and Materials. 1984. Standard Practices for Evaluating Environmental Fate Models of Chemicals. Annual Book of ASTM Standards, E 978-84, Philadelphia. Beljin, M. S., and P. K. M. van der Heijde. 1987. Testing, Verification, and Validation of Two-Dimensional Solute Transport Models. In Groundwater Contamination: Use of Models in Decision-making, Y. Y. Haimes and J. Bear, eds. D. Reidel, Dordrecht, The Netherlands. Boutwell, S. H., S. M. Brown, B. R. Roberts, and D. F. Atwood. 1985. Modeling Remedial Actions at Uncontrolled Hazardous Waste Sites. EPA/540/2- 85/001, Environmental Protection Agency, OSWER/ORD, Washington, D.C. Brandstetter, A., and B. E. Buxton. 1987. The role of geostatistical, sensi- tivity and uncertainty analysis in performance assessment. In Proceedings of the 1987 DOE/AECL Conference on Geostatistical, Sensitivity and Uncertainty Methods for Groundwater Flow and Radionuclide Transport Modeling, September 15-17. U.S. Department of Energy, Washington, D.C. Bryant, J. L., and N. P. Wilburn. 1987. Handbook of Software Quality Assur- ance Techniques Applicable to the Nuclear Industry. NUREG/CR-4640, Office of Nuclear Reactor Regulation, U.S. Nuclear Regulatory Commis- sion, Washington, D.C. Carrera, J., and S. P. Neuman. 1986. Estimation of aquifer parameters under transient and steady state conditions, 1. Maximum likelihood method incorporating prior information. Water Resources Research 22~2), 199- 210. Chu, W., E. W. Strecker, and D. P. Lettenmaier. 1987. An evaluation of data requirements for groundwater contaminant transport modeling. Water Resources Research 23~3), 408-424. Cochran, W. G., and G. M. Cox. 1957. Experimental Designs. Wiley, New York.

OCR for page 211
244 GROUND WATER MODELS Codell, R., and S. Silling. 1984. Draft Quality Assurance Plan for Opera- tional Software. Memorandum 3110/DC/84/09/20/0, Division of Waste Management, U.S. Nuclear Regulatory Commission, Washington, D.C. Council of State Governments. 1985. Risk Management and the Hazardous Waste Problems in State Governments. Prepared for the National Science Foundation. Cox, D. R. 1958. Planning of Experiments. Wiley, New York. Dagan, G. 1984. Solute transport in heterogeneous porous formations. Journal of Fluid Mechanics 145, 151-177. Dagan, G. 1986. Statistical theory of groundwater Bow and transport: Pore to laboratory, laboratory to formation, and formation to regional scale. Water Resources Research 22~9), 120s-134s. Delhomme, J. P. 1979. Spatial variability and uncertainty in groundwater flow parameters: A geostatistical approach. Water Resources Research 15~2), 251-256. Environmental Protection Agency. 1986. Information on Preparing Quality Assurance Narrative Statements. RSKERL-QA-2, Robert S. Kerr, Envi- ronment Research Laboratory, Ada, Okla. Environmental Protection Agency. 1987. Evaluation of Implementation of Risk- Based Decisionmaking in RCR-A, Annotated Briefing. Program Evaluation Division, Office of Policy, Planning and Evaluation, pp. 8, 9, 12, 17, 25. Federal Computer Performance Evaluation and Simulation Center. 1981. Com- puter Model Documentation Guide. NBS Special Publication 500-73, Insti- tute for Computer Science and Technology, National Bureau of Standards, Department of Commerce, Washington, D.C. Federov, V. V. 1972. Theory of Optimal Experiments. Academic Press, New York. E`reyberg, D. L. 1988. An exercise in ground-water model calibration and prediction. Ground Water 26~3), 350-360. Gass, S. I. 1979. Computer Model Documentation: A Review and an Approach. NBS Special Publication 500-39, Institute for Computer Science and Tech- nology, National Bureau of Standards, U.S. Department of Commerce, Washington, D.C. Gelhar, L. W. 1986. Stochastic subsurface hydrology from theory to application. Water Resources Research 22~9), 135S-145S. Gelhar, L. W., and C. L. Axness. 1983. Three dimensional stochastic analysis of macrodispersion in aquifers. Water Resources Research 19~1), 161-180. General Accounting Office. 1987. Report to Congress: Superfund Improvements Needed in Work Force Management. GAO/RCED-88-1, pp. 2, 3. Graham, W., and D. McLaughlin. 1989a. Stochastic analysis of nonstationary subsurface solute transport, 1. Unconditional moments. Water Resources Research 25~2), 215-232. Graham, W., and D. McLaughlin. 1989b. Stochastic analysis of nonstationary subsurface solute transport, 2. Conditional moments. Water Resources Research, in press. Gupta, S. K., C. R. Cole, F. W. Bond, and A. M. Monti. 1984. Finite-Element Three-Dimensional Ground-Water (FE3DGW) Flow Model: Formulation, Computer Source Listings, and User's Manual. ONWI-548, Battelle Memo- rial In~titute, Columbus, Ohio.

OCR for page 211
DEVELOPMENT AND USE OF MODELS 245 Harlan, C. P., and G. F. Wilkinson. 1988. High-Level Waste Management Code Maintenance and Quality Assurance. SAND87-2254/UC-70. Sandia National Laboratories, Albuquerque, N. Mex. Hern, S. C., S. M. Melancon, and J. E. Pollard. 1985. Generic steps in the field validation of Vadose Zone fate and transport models. Pp. 61-80 in Vadose Zone Modeling of Organic Pollutants, S. C. Hern and S. M. Melancon, eds. Lewis Publishers, Chelsea, Mich. Hoeksema, R. J., and P. Kitanidis. 1984. An application of the geostatistical approach to the inverse problem in two-dimensional groundwater modeling. Water Resources Research 20~7), 1003-1020. Hoffman, F. O., and R. H. Gardner. 1983. Evaluation of uncertainties in radio- logical assessment models. In Radiological Assessment: A Text on Environ- mental Dose Analysis, J. E. Till and H. R. Meyer, eds. NUREG/CR-3332, ORNL-5968, U.S. Nuclear Regulatory Commission, Washington, D.C. Huyakorn, P. S., A. G. Kretschek, R. W. Broome, J. W. Mercer, and B. H. Lester. 1984. Testing and Validation of Models for Simulating Solute Transport in Groundwater: Development, Evaluation, and Comparison of Benchmark Techniques. GWMI 84-13, International Ground Water Mod- eling Center, Holcomb Research Institute, Butler University, Indianapolis, Ind. Institute of Electrical and Electronic Engineers, Inc. 1984. Standard for Software Quality Assurance Plane. IEEE Standard 730-1984, New York. Intera Environmental Consultants, Inc. 1983. A Proposed Approach to Un ,: ~ ,~ certainty Analysis. ONWI-488, Battelle Memorial Institute, Columbus, Ohio. International Association of Hydrological Sciences. 1986. Integrated Design of Hydrological Networks, Proceedinge of the Budapest Symposium. IAHS Publication 158. International Ground Water Modeling Center. 1986. U.S. EPA Ground-Water Modeling Policy Study Group: Report of Findings and Discussion of Selected Ground-Water Modeling Issues. Holcomb Research Institute, Butler University, Indianapolis, Ind., pp. 2, 5, 15, 19, 27. Journel, A. G., and Ch. I. Hinjbregts. 1978. Mining Geostatistics. Academic Press, San Diego, Calif. Kiefer, J., and J. Wolfowitz. 1959. Optimum designs in regression problems. Annals of Mathematical Statistics 30, 271-294. Kincaid, C. T., J. R. Morrey, and J. E. Rogers. 1984. Geohydrological Models for Solute Migration, Vol. 1, Process Description and Computer Code for Selection. EA 3417.1, Electric Power Research Institute, Palo Alto, Calif. Knopman, D. S., and C. I. Voss. 1987. Behavior of sensitivities in the one-dimensional advection-dispersion equation: Implications for parameter estimation and sampling design. Water Resources Research 23~2), 253-272. Knopman, D. S., and C. I. Voss. 1988. Further comments on sensitivities, parameter estimation, and sampling design in one-dimensional analysis of solute transport in porous media. Water Resources Research 24~2), 225-238. Konikow, L. F. 1986. Predictive accuracy of a ground-water model-Lessons from a post-audit. Ground Water 24~2), 677-690.

OCR for page 211
246 GROUND WATER MODELS Lewis, B. D., and F. J. Goldstein. 1982. Evaluation of a Predictive Ground- Water Solute Transport Model at the Idaho National Engineering Labora- tory, Idaho. Water Resources Investigation 82-25, U.S. Geological Survey, Reston, Va. McLaughlin, D., and W. K. Johnson. 1987. Comparison of three groundwa- ter modeling studies. ASCE Journal of Water Resources Planning and Management 113~3), 405-421. McLaughlin. D.. and E. F. Wood. 1988a. A distributed parameter approach ,7 _ . . . . .. . _ for evaluating the accuracy of groundwater predictions, 1. Theory. Water Resources Research 24~7), 1037-1047. McLaughlin, D., and E. F. Wood. 1988b. A distributed parameter approach for evaluating the accuracy of groundwater predictions, 2. Application to groundwater flow. Water Resources Research 24~7), 1048-1060. Mercer, J. W., and C. R. Faust. 1981. Ground Water Modeling. National Water Well Association, Dublin, Ohio. National Bureau of Standards. 1976. Guidelines for Documentation of Com- puter Programs and Automated Data Systems. FIPS 38, Federal In- formation Processing Standards Publications, Department of Commerce, Washington, D.C. National Research Council. 1988. Hazardous Waste Site Management: Water Quality Issues. Report on a colloquium sponsored by the Water Science and Technology Board. National Academy Press, Washington, D.C., p. 9. Neuman, S. P., C. L. Winter, and C. M. Newman. 1987. Stochastic theory of field-scale Fickian dispersion in anisotropic porous media. Water Resources Research 23~3), 453-466. Nicholson, T. J., T. J. McCartin, P. A. Davis, and W. Beyeler. 1987. NRC experiences in HYDROCOIN: An international project for studying ground- water Bow modeling strategies. In Proceedings, GEOVAL 87, April 4-9, Stockholm. Swedish Nuclear Power Inspectorate, Stockholm. Office of Technology Assessment. 1982. Use of Models for Water Resources Management, Planning, and Policy. Pp. 9, 18-19,20,22,24,102-104. Person, M., and L. F. Konikow. 1986. Recalibration and predictive reliability of a solute-transport model of an irrigated stream-aquifer system. Journal of Hydrology 87,145-165. Pesticide and Toxic Chemical News (Food Chemical News). 1987. Exposure estimates seen as weak link in EPA risk assessments. Vol. 16, p. 5 (December 23, 1987~. Rao, P. S. C., R. E. Jessup, and A. C. Hornsby. 1981. Simulation of nitrogen in agro-ecosystems: Criteria for model selection and use. Pp. 1-16 in Nitrogen Cycling in Ecosystems of Latin America and the Caribbean, Proceedings of the International Workshop, Call, Colombia, March 16-21. Silling, S. A. 1983. Final Technical Position on Documentation of Computer Codes for High-Level Waste Management. NUREG/CR-0856-F, Office of Nuclear Material Safety and Safeguards, U.S. Nuclear Regulatory Com- mission, Washington, D.C. Simmons, C. S., and C. R. Cole. 1985. Guidelines for Selecting Codes for Groundwater Transport Modeling of Low-Level Waste Burial Sites, Vol. 1, Guideline Approach. PNL-4980 Vol. 1. Battelle Pacific NW Laboratory, Richland, Wash. Smith, L., and F. W. Schwartz. 1980. Mass transport, 1. A stochastic analysis of macroscopic dispersion. Water Resources Research 16~2), 303-313.

OCR for page 211
DEVELOPMENT AND USE OF MODELS 247 Smith, L. and F. W. Schwartz. 1981a. Mass transport, 2. Analysis of uncertainty in predictions. Water Resources Research 17~2), 351-369. Smith, L. and F. W. Schwartz. 1981b. Mass transport, 3. Role of hydraulic conductivity. Water Resources Research 17~5), 1463-1479. Sykes, J. F., S. B. Pahwa, D. S. Ward, and R. B. Lantz. 1983. The validation of SWENT, a geosphere transport model. In Scientific Computing, R. Stepleman et al., eds. IMACS/North-Holland Publishing Company, New York. Taylor, J. K. 1985. What is quality assurance? Pp. 5-11 in Quality Assurance for Environmental Measurements, J. K. Taylor and T. W. Stanley, eds. ASTM Special Technical Publication 867, American Society for Testing and Materials, Philadelphia. van der Heijde, P. K. M. 1984. Availability and applicability of numerical mod- els for groundwater resources management. In Practical Applications of Ground Water Models, Proceedings NWWA/IGWMC Conference, Colum- bus, Ohio, Aug. 15-17. National Water Well Association, Dublin, Ohio. van der Heijde, P. K. M. 1987. Quality assurance in computer simulations of groundwater contamination. Environmental Software 2~1), 19-28. van der Heijde, P. K. M., and M. S. Beljin. 1988. Model Assessment for Delineat- ing Wellhead Protection Areas. EPA 440/6-88-002, Office of Ground-Water Protection, Environmental Protection Agency, Washington, D.C. van der Heiide, P. K. M., and R. A. Park. 1986. U.S. EPA Groundwa- ter Modeling Policy Study Group, Report of Findings and Discussion of Selected Groundwater Modeling Issues. International Ground Water Mod- eling Center, Holcomb Research Institute, Butler University, Indianapolis, Ind. van der Heijde, P. K. M., Y. Bachmat, J. Bredehoeft, B. And rews, D. Holtz, and S. Sebastian. 1985a. Ground-Water Management: The Use of Numerical Models, 2nd ed. Water Resources Monograph 5, American Geophysical Union, Washington, D.C. van der Heijde, P. K. M., P. S. Huyakorn, and J. W. Mercer. 1985b. Testing and validation of ground water models. In Proceedings, NWWA/IGWMC Conference on Practical Applications of Groundwater Models, Columbus, Ohio, Aug. 19-20. National Water Well Association, Dublin, Ohio. van der Heijde, P. K. M., A. I. El-Kadi, S. A. Williams, and D. L. Cave. 1988. Groundwater Modeling: An Overview. GWMI 88-10. International Ground Water Modeling Center, Holcomb Renearch Institute, Butler Uni- versity, Indianapolis, Ind. van Tassel, D. 1978. Program Style, Design, Efficiency, Debugging, and Testing, 2nd ed. Prentice-Hall, Englewood Cliffs, N.J. Ward, D. S., M. Reeves, and L. E. Duda. 1984. Verification and Field Comparison of the Sandia Waste-Isolation Flow and Transport Model (SWIFT). NUREG/CR-3316, Office of Nuclear Safety and Safeguards, U.S. Nuclear Regulatory Commission, Washington, D.C. Wilkinson, G. F., and G. E. Runkle. 1986. Quality ~surance (QA) Plan for Computer Software Supporting the U.S. Nuclear Regulatory Commission's High-Level Waste Management Program. NUREG/CR-4369, OHice of Nuclear Safety and Safeguards, U.S. Nuclear Regulatory Commission, Washington, D.C.

OCR for page 211
248 GROUND WATER MODELS Yeh, W. W-G. 1986. Review of parameter identification procedures in ground- water hydrology: The inverse problem. Water Resources Research 22~2), 95-108. Yourdon, E., and L. L. Constantine. 1979. Structured Design: Fundamentals and Systems Design. Prentice-Hall, Englewood Cliffs, N.J.