National Academies Press: OpenBook

Ground Water Models: Scientific and Regulatory Applications (1990)

Chapter: 6 ISSUES IN THE DEVELOPMENT AND USE OF MODELS

« Previous: 5 EXPERIENCE WITH CONTAMINANT FLOW MODELS IN THE REGULATORY SYSTEM
Suggested Citation:"6 ISSUES IN THE DEVELOPMENT AND USE OF MODELS." National Research Council. 1990. Ground Water Models: Scientific and Regulatory Applications. Washington, DC: The National Academies Press. doi: 10.17226/1219.
×
Page 211
Suggested Citation:"6 ISSUES IN THE DEVELOPMENT AND USE OF MODELS." National Research Council. 1990. Ground Water Models: Scientific and Regulatory Applications. Washington, DC: The National Academies Press. doi: 10.17226/1219.
×
Page 212
Suggested Citation:"6 ISSUES IN THE DEVELOPMENT AND USE OF MODELS." National Research Council. 1990. Ground Water Models: Scientific and Regulatory Applications. Washington, DC: The National Academies Press. doi: 10.17226/1219.
×
Page 213
Suggested Citation:"6 ISSUES IN THE DEVELOPMENT AND USE OF MODELS." National Research Council. 1990. Ground Water Models: Scientific and Regulatory Applications. Washington, DC: The National Academies Press. doi: 10.17226/1219.
×
Page 214
Suggested Citation:"6 ISSUES IN THE DEVELOPMENT AND USE OF MODELS." National Research Council. 1990. Ground Water Models: Scientific and Regulatory Applications. Washington, DC: The National Academies Press. doi: 10.17226/1219.
×
Page 215
Suggested Citation:"6 ISSUES IN THE DEVELOPMENT AND USE OF MODELS." National Research Council. 1990. Ground Water Models: Scientific and Regulatory Applications. Washington, DC: The National Academies Press. doi: 10.17226/1219.
×
Page 216
Suggested Citation:"6 ISSUES IN THE DEVELOPMENT AND USE OF MODELS." National Research Council. 1990. Ground Water Models: Scientific and Regulatory Applications. Washington, DC: The National Academies Press. doi: 10.17226/1219.
×
Page 217
Suggested Citation:"6 ISSUES IN THE DEVELOPMENT AND USE OF MODELS." National Research Council. 1990. Ground Water Models: Scientific and Regulatory Applications. Washington, DC: The National Academies Press. doi: 10.17226/1219.
×
Page 218
Suggested Citation:"6 ISSUES IN THE DEVELOPMENT AND USE OF MODELS." National Research Council. 1990. Ground Water Models: Scientific and Regulatory Applications. Washington, DC: The National Academies Press. doi: 10.17226/1219.
×
Page 219
Suggested Citation:"6 ISSUES IN THE DEVELOPMENT AND USE OF MODELS." National Research Council. 1990. Ground Water Models: Scientific and Regulatory Applications. Washington, DC: The National Academies Press. doi: 10.17226/1219.
×
Page 220
Suggested Citation:"6 ISSUES IN THE DEVELOPMENT AND USE OF MODELS." National Research Council. 1990. Ground Water Models: Scientific and Regulatory Applications. Washington, DC: The National Academies Press. doi: 10.17226/1219.
×
Page 221
Suggested Citation:"6 ISSUES IN THE DEVELOPMENT AND USE OF MODELS." National Research Council. 1990. Ground Water Models: Scientific and Regulatory Applications. Washington, DC: The National Academies Press. doi: 10.17226/1219.
×
Page 222
Suggested Citation:"6 ISSUES IN THE DEVELOPMENT AND USE OF MODELS." National Research Council. 1990. Ground Water Models: Scientific and Regulatory Applications. Washington, DC: The National Academies Press. doi: 10.17226/1219.
×
Page 223
Suggested Citation:"6 ISSUES IN THE DEVELOPMENT AND USE OF MODELS." National Research Council. 1990. Ground Water Models: Scientific and Regulatory Applications. Washington, DC: The National Academies Press. doi: 10.17226/1219.
×
Page 224
Suggested Citation:"6 ISSUES IN THE DEVELOPMENT AND USE OF MODELS." National Research Council. 1990. Ground Water Models: Scientific and Regulatory Applications. Washington, DC: The National Academies Press. doi: 10.17226/1219.
×
Page 225
Suggested Citation:"6 ISSUES IN THE DEVELOPMENT AND USE OF MODELS." National Research Council. 1990. Ground Water Models: Scientific and Regulatory Applications. Washington, DC: The National Academies Press. doi: 10.17226/1219.
×
Page 226
Suggested Citation:"6 ISSUES IN THE DEVELOPMENT AND USE OF MODELS." National Research Council. 1990. Ground Water Models: Scientific and Regulatory Applications. Washington, DC: The National Academies Press. doi: 10.17226/1219.
×
Page 227
Suggested Citation:"6 ISSUES IN THE DEVELOPMENT AND USE OF MODELS." National Research Council. 1990. Ground Water Models: Scientific and Regulatory Applications. Washington, DC: The National Academies Press. doi: 10.17226/1219.
×
Page 228
Suggested Citation:"6 ISSUES IN THE DEVELOPMENT AND USE OF MODELS." National Research Council. 1990. Ground Water Models: Scientific and Regulatory Applications. Washington, DC: The National Academies Press. doi: 10.17226/1219.
×
Page 229
Suggested Citation:"6 ISSUES IN THE DEVELOPMENT AND USE OF MODELS." National Research Council. 1990. Ground Water Models: Scientific and Regulatory Applications. Washington, DC: The National Academies Press. doi: 10.17226/1219.
×
Page 230
Suggested Citation:"6 ISSUES IN THE DEVELOPMENT AND USE OF MODELS." National Research Council. 1990. Ground Water Models: Scientific and Regulatory Applications. Washington, DC: The National Academies Press. doi: 10.17226/1219.
×
Page 231
Suggested Citation:"6 ISSUES IN THE DEVELOPMENT AND USE OF MODELS." National Research Council. 1990. Ground Water Models: Scientific and Regulatory Applications. Washington, DC: The National Academies Press. doi: 10.17226/1219.
×
Page 232
Suggested Citation:"6 ISSUES IN THE DEVELOPMENT AND USE OF MODELS." National Research Council. 1990. Ground Water Models: Scientific and Regulatory Applications. Washington, DC: The National Academies Press. doi: 10.17226/1219.
×
Page 233
Suggested Citation:"6 ISSUES IN THE DEVELOPMENT AND USE OF MODELS." National Research Council. 1990. Ground Water Models: Scientific and Regulatory Applications. Washington, DC: The National Academies Press. doi: 10.17226/1219.
×
Page 234
Suggested Citation:"6 ISSUES IN THE DEVELOPMENT AND USE OF MODELS." National Research Council. 1990. Ground Water Models: Scientific and Regulatory Applications. Washington, DC: The National Academies Press. doi: 10.17226/1219.
×
Page 235
Suggested Citation:"6 ISSUES IN THE DEVELOPMENT AND USE OF MODELS." National Research Council. 1990. Ground Water Models: Scientific and Regulatory Applications. Washington, DC: The National Academies Press. doi: 10.17226/1219.
×
Page 236
Suggested Citation:"6 ISSUES IN THE DEVELOPMENT AND USE OF MODELS." National Research Council. 1990. Ground Water Models: Scientific and Regulatory Applications. Washington, DC: The National Academies Press. doi: 10.17226/1219.
×
Page 237
Suggested Citation:"6 ISSUES IN THE DEVELOPMENT AND USE OF MODELS." National Research Council. 1990. Ground Water Models: Scientific and Regulatory Applications. Washington, DC: The National Academies Press. doi: 10.17226/1219.
×
Page 238
Suggested Citation:"6 ISSUES IN THE DEVELOPMENT AND USE OF MODELS." National Research Council. 1990. Ground Water Models: Scientific and Regulatory Applications. Washington, DC: The National Academies Press. doi: 10.17226/1219.
×
Page 239
Suggested Citation:"6 ISSUES IN THE DEVELOPMENT AND USE OF MODELS." National Research Council. 1990. Ground Water Models: Scientific and Regulatory Applications. Washington, DC: The National Academies Press. doi: 10.17226/1219.
×
Page 240
Suggested Citation:"6 ISSUES IN THE DEVELOPMENT AND USE OF MODELS." National Research Council. 1990. Ground Water Models: Scientific and Regulatory Applications. Washington, DC: The National Academies Press. doi: 10.17226/1219.
×
Page 241
Suggested Citation:"6 ISSUES IN THE DEVELOPMENT AND USE OF MODELS." National Research Council. 1990. Ground Water Models: Scientific and Regulatory Applications. Washington, DC: The National Academies Press. doi: 10.17226/1219.
×
Page 242
Suggested Citation:"6 ISSUES IN THE DEVELOPMENT AND USE OF MODELS." National Research Council. 1990. Ground Water Models: Scientific and Regulatory Applications. Washington, DC: The National Academies Press. doi: 10.17226/1219.
×
Page 243
Suggested Citation:"6 ISSUES IN THE DEVELOPMENT AND USE OF MODELS." National Research Council. 1990. Ground Water Models: Scientific and Regulatory Applications. Washington, DC: The National Academies Press. doi: 10.17226/1219.
×
Page 244
Suggested Citation:"6 ISSUES IN THE DEVELOPMENT AND USE OF MODELS." National Research Council. 1990. Ground Water Models: Scientific and Regulatory Applications. Washington, DC: The National Academies Press. doi: 10.17226/1219.
×
Page 245
Suggested Citation:"6 ISSUES IN THE DEVELOPMENT AND USE OF MODELS." National Research Council. 1990. Ground Water Models: Scientific and Regulatory Applications. Washington, DC: The National Academies Press. doi: 10.17226/1219.
×
Page 246
Suggested Citation:"6 ISSUES IN THE DEVELOPMENT AND USE OF MODELS." National Research Council. 1990. Ground Water Models: Scientific and Regulatory Applications. Washington, DC: The National Academies Press. doi: 10.17226/1219.
×
Page 247
Suggested Citation:"6 ISSUES IN THE DEVELOPMENT AND USE OF MODELS." National Research Council. 1990. Ground Water Models: Scientific and Regulatory Applications. Washington, DC: The National Academies Press. doi: 10.17226/1219.
×
Page 248

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

6 Issues in the Development and Use of Models INTRODUCTION In the United States, two agencies, the U.S. Nuclear Regulatory Commission (USNRC) and the Environmental Protection Agency (EPA), are particularly concerned with ground water modeling to support many of their regulatory activities. Their experience with the uses of models has been completely different. The USNRC, while placing considerable emphasis on developing guidance for the selection and use of models, has never really employed them for regulatory purposes. The USNRC's low-level waste (ELM) program has yet to be tested, because no applications for licenses have been received by the USNRC. In any case, the USNRC is likely to receive fewer than 10 applications for disposal sites. The high-level radioactive waste program is also untested. License applications for high-level waste repositories have not been received, and none are expected before 1995. The EPA's experience in using models is documented to a much greater extent because of the number of active sites under its ju- risdiction. Models play an important role in EPA-related activities; however, many problems related to the use of models have emerged. For example, prior reviews of the Superfund cleanup process have concluded the following: 211

212 GROUND WATER MODELS ~ all analytical methodologies suffer from a lack of knowledge on the fundamental process underlying observed phenomena (National Research Council, 1988~; ~ models do not account for all the processes affecting the fate and impact of the contaminants (National Research Council, 1988~; ~ models lack accuracy when confronted with a high degree of heterogeneity (complex hydrogeology, multiple contaminants, two- phase flow, and variable susceptibility in populations) (National Re- search Council, 1988~; ~ there is no clear guidance provided by agencies concerning when to use and how to select models (International Ground Water Modeling Center, 1986; Office of Technology Assessment, 1982~; ~ the decision concerning when to use a mode! and which code to use is often left to the discretion of the contractor who was hired by EPA or a potentially responsible party (International Ground Water Modeling Center, 1986~; ~ there is limited understanding among EPA staff concerning which models are available (International Ground Water Modeling Center, 1986~; , there is inadequate expertise within federal and state regula- tory agencies to apply such models (Office of Technology Assessment, 1982~; the validity of some codes for the problem to which they are applied has not been established (Office of Technology Assessment, 1982~; ~ EPA enforcement offices strongly discourage the use of propri- etary models (International Ground Water Modeling Center, 1986; Office of Technology Assessment, 1982~; ~ there is inadequate quality assurance, quality control, and peer review (Office of Technology Assessment, 1982~; and ~ there is a reluctance to use models if their use would be considered controversial (Office of Technology Assessment, 1982~. The committee's review confirmed most of these findings. The problem is not a lack of appropriate documents to guide the modeling process. One can see from the list that the basic problems concern the lack of training and experience in the people who are choosing and using models, deficiencies or limitations in the codes themselves, and scientific barriers that determine to what extent models are able to incorporate relevant processes. The committee addresses these issues in this chapter.

DEVELOPMENT AND USE OF MODELS THE PEOPLE PROBLEM 213 It should be apparent from earlier chapters outlining the state of the science that modeling ground water flow and contaminant transport is not a trivial exercise. Ideally, a modeler should have a broad background in earth sciences with particular strengths in hydrogeology, low-temperature geochemistry, and analytical and nu- merical mathematics. This background will have developed through graduate and undergraduate studies and will have been tempered by relevant experience. A significant problem in dealing with regulatory agencies is the lack of individuals who are trained at an appropriate level to understand and use models. For example, EPA's ground water and contaminant transport modeling needs currently outpace its actual use of models in virtu- ally all program areas (Office of Technology Assessment, 1982~. EPA currently has an insufficient number of qualified and experienced hy- drogeologists and other professionals knowledgeable in contaminant transport modeling (Office of Technology Assessment, 1982~. Super- fund hydrogeologists are quitting their jobs at a rate 6 times higher than the average for other federal government employees (General Accounting Office, 1987~. The more experienced hydrogeologists are leaving EPA at a higher rate than the younger professionals, and the situation is likely to become worse. Most states possess even more limited capabilities (Council of State Governments, 1985; Environ- mental Protection Agency, 1987; General Accounting Office, 1987; International Ground Water Modeling Center, 1986~. The substan- tial increase in the need for site-specific regulatory decisions in all the EPA programs concerned with regulating ground water can only exacerbate the breadth and depth of these shortages and critical neecis. Contaminant transport models simply cannot be used unless people who are experts in ground water processes and models are available to select, apply, and peer-review such models. As a result, EPA's system for selecting and applying models is guaranteed to result in misuses of such models. The solution to this problem will require (~) recruiting and re- taining more qualified and experienced personnel; (2) establishing specific guidelines Ed criteria for the use of contaminant flow mod- els; (3) instituting peer review techniques; and (4) providing technical assistance and additional training. The lack of qualified individuals in the regulatory agencies at all levels has had some significant ramifications. For example, some

214 GROUND WATER MODELS of the methods chosen to expedite hazardous waste cleanups are contrary to good science. EPA's policy of performing remedial inves- tigations in less than 6 months often provides insufficient data for a complete characterization of the site. Given the seasonal character of ground water flow, it would be prudent to measure water levels over a longer time frame. The rush to judgment on Superfund remedies risks more than a "bad" scientific decision or an economically wasteful cleanup. Deci- sions based on inadequate data may aggravate a problem or, at least, prolong its eventual remedy. The prudent, if not necessary, course of action in such cases is to proceed in orderly phases, such as in the S-Area case study. The committee recognizes the desirability of, and public mandate for, expediting hazardous waste cleanups. Those components of the remedial action for a site that reasonably can be taken with limited data- for example, interception and treatment of the ground water plume should be implemented immediately. Other components of a site remecliation plan can be implemented at a more measured pace once the primary potential source of exposure is eliminated. Another problem generated through inexperience is an overre- liance on the results of a modeling exercise. Computer models have a unique capacity to appear more certain, more precise, and more authoritative than they really are. As a result, assumptions, even wholly unrealistic ones, can be stated with deceptive precision and seeming accuracy by being included in a computer model. Special care therefore must be taken in presenting the results of such modeling. Decisionmakers (whether they be heads of agencies, judges, or juries) must understand the distinction between scientific fact and science policy. If policy is relied on to make a decision, the policy rationale should be explicitly identified. Faced with the problem of an overall lack of qualified staff to use models and interpret results, regulatory agencies have a natural tendency toward simplification through the use of standard models and worst-case assumptions, as is done in the hazardous waste delist- ing program (Environmental Protection Agency, 1987; International Ground Water Modeling Center, 1986~. This decision is motivated by a concern about the lack of adequate resources and a preference for using overprotective assumptions. There is an inherent conflict between using more complex, site-specific models and using simpler models; i.e., "isitandardization may increase consistency, but tends

DEVELOPMENT AND USE OF MODELS 215 to trade off accuracy, producing answers that are not always appro- priate in all situations" (Environmental Protection Agency, 1987~. A site-specific decision should be based on the actual conditions existing at a site. More certainty should be required if the output of the mode! is used directly to trigger additional regulatory action than if the mode! is used as an interpretative too! to better understand how contaminants migrate near the site. The committee recognizes the need to follow the mandate of the enabling statutes, use health-protective assumptions, and consider the practical limitations on agency resources. The committee, how- ever, believes that the use of standard models at specific sites lacks a scientific basis. The use of overly simplistic models, such as the vertical-horizontal spread (VHS) model, at Superfund sites or other hazardous waste sites (1) would be an arbitrary distortion of the remedial selection process, (2) could reduce protection of the pub- lic health by misallocating finite cleanup resources, and (3) would result in the imposition of substantial costs with no commensurate environmental or public health benefit. The Environmental Protection Agency's choice of remedies can also be affected by the choice of mode} and the assumptions used in such a model. For example, EPA may use an advection-dispersion contaminant transport mode} to predict the future concentrations of chemicals at local drinking water wells to derive the on-site soil cleanup levels (i.e., soil levels that would not result in off-site ground water concentrations above health-based ground water cleanup lev- els) (Record of Decision, July 1985, McKin Site, Maine, RO1-85-009) or to estimate the time that it will take to achieve various cleanup levels by alternative remedial actions (Record of Decision, August 1985, Old Mill, Ohio, RO5-85-018; Record of Decision, September 1987a, Su~ern Well Field, New York, RO2-87-042~. Such a mode! will not take into account dilution, adsorption, volatilization, or biodegradation and other more realistic features (Record of Deci- sion, September 1987b, Rose Township, Michigan, RO5-87-052~. For example, an advection-dispersion mode! generally will overestimate the concentration and underestimate the travel time for the contam- inants, thus making the problem appear much more serious than it is in reality. Extreme worst-case assumptions can drive the remecly selection process toward draconian and extremely costly remedies. The se- lection of these assumptions as input to models is also prone to misuse (Pesticide and Toxic Chemical News, 1987~. The difference

216 GROUND WATER MODELS between worst-case assumptions and levels predicted by contaminant transport modeling (no less real) can be substantial. The benefit of using extreme worst-case assumptions is often simply administrative convenience to the agency; i.e., using such as- sumptions eliminates the need to obtain additional data and make difficult expert judgments. This benefit must be weighed against the additional cost or the possibility that the assumption will sig- nificantly underestimate the risk. Worst-case assumptions should never be preferred over actual data. Some assumptions may be so unrealistic that their use is inappropriate. UNCERTAINTY AND RELIABILITY Modeling can be defined as the art and science of collecting a set of discrete observations (our incomplete knowledge of the real worId) and producing predictions of the behavior of a system. Such predictions will be necessarily uncertain, as will be our knowledge of the true behavior of the system. The goal of this section is to identify and discuss the scientific, technical, and practical issues that arise in applying models to particular sites, and to develop proce- dures and guidelines to help assure that these issues are addressed during the mode! application process. A convenient framework for organizing a discussion of uncertainty and reliability in modeling is presented by Figure 6.1. What is shown is one possible representation of the process of applying a mode] to a regulatory (or other) deci- sionmaking problem. This representation rests on the assumption that the ultimate goal of a modeling exercise is a prediction of the behavior of the real world. That is, there is a "true" system, made up of the geologic environment (the soils and/or aquifers), climatic stresses (precipitation and evaporation), subsurface flora and fauna, and human-induced stresses (e.g., irrigation welis). The success of a modeling exercise will depend on the degree to which the mode! pre- diction agrees with the behavior of this true system. Therefore the reference in discussing and/or assessing the accuracy of the modeling process is this real system, indicated by the top path of Figure 6.1. The state and characteristics of the real world may be described by a set of information termed the inputs to the system, such as the spatial distribution of soil and aquifer properties, or the time histories of system stresses. These inputs are often highly variable in time and/or space. Some may be inherently uncertain, such as future time series of rainfall infiltration and subsequent recharge.

DEVELOPMENT AND USE OF MODELS Reference inputs Sampling errors Known I inputs I - REFERENCE SYSTEM (Distributed) SAMPLING PROCESS Measurements , I Sampling strategy INPUT ESTIMATION PROCESS Prior Estimation information strategy 217 Reference output l Prediction Modeling error strategy _ _: _, Estimated inputs MODEL (Discretized) 1 Predicted output FIGURE 6.1 Conceptual framework for ground water model accuracy analysis. SOURCE: McLaughlin and Wood, 1988a. The processes at work in the real system, including those induced by proposed management actions, act on these inputs to yield the true, or real, outputs that characterize the behavior of the system. Such outputs might be contaminant concentration distributions in space and time, travel times, mass losses, or exposure levels and durations at selected locations. These true outputs are, of course, themselves often variable and uncertain. Even though ground water flow and transport systems tend to smooth out the variability of inputs, much variability and uncertainty remain in the true outputs. The following sections use this conceptual mode} to describe the major sources of uncertainty in the modeling process. The Sampling Process One can observe the real world only via a sampling process. We make a finite number of observations, choosing what parameters to measure, how to measure them (what instrument to use), where to measure them, and when to measure them. In other words, a sam- pling scheme is designed and implemented. For example, one might

218 GROUND WA1'ER MODELS collect a set of cores during well drilling and measure the permeabil- ity of subsamples of each core in the laboratory using a permeameter. Alternatively, one might collect a set of water samples from wells and analyze each in the laboratory for contaminant concentration. Such a sampling scheme typically provides a set of discrete quantitative observations of one or more parameters of interest, or sometimes more continuous, qualitative information about the system (e.g., the geologic sedimentary environment). A sampling process introduces uncertainty. First, the measure- ment process itself introduces uncertainty in the form of instrument errors. Every measurement device has associated with it a mea- surement error. Such errors usually contain a random (uncertain) component (and are often biased). Second, the sampling process introduces uncertainty because of incomplete information. The sys- tem can be observed only at a small set of points, and conditions between sampling points are not known with certainty, whether in space or time or both. This uncertainty is obviously most critical for systems characterized by significant spatial and temporal variability. Thus the real system is an uncertain one because of (1) its inherent randomness, (2) measurement error, and perhaps most important, (3) limited sampling of the highly variable physical, chemical, and biological properties of ground water systems. This uncertainty ap- plies to both the inputs and the outputs of the system. All modeling is conducted without certain knowledge of the true state of a ground water environment. The magnitude of our uncertainty is a func- tion of the spatial heterogeneity and temporal variability of aquifer properties, boundary conditions, dependent variables, the density of observation points relative to the scale of the variability, and the measurement techniques. With these general concepts in mind, we can address more specific issues concerned with field sampling and data collection. Field sampling, experimental design, and related data analysis issues are topics that have not traditionally received much attention from ground water modelers. While most modelers appreciate the need for good field data, they have often had to depend on others for the data used in their models. Published field data have typi- cally been taken at face value and have been freely extrapolated and generalized beyond their original purpose. This situation has begun to change, partly as a result of the demanding requirements of haz- ardous waste studies and partly because modelers are beginning to

DEVELOPMENT AND USE OF MODELS 219 take a broader view of the modeling process, which recognizes that data issues need to be taken seriously. As mentioned earlier in this report, ground water systems are difficult to observe and describe, not only because they are hidden from view, but also because they are three-dimensional and often very heterogeneous. Hydrogeological properties observed at one lo- cation may give relatively little information about conditions only a few meters away. Soil strata or rock fractures only a few centimeters thick may greatly influence the movement of water and contaminants but pass undetected in a typical field survey. Such heterogeneities limit our ability to generalize from laboratory measurements to field conditions or from one site to another. The ground water sampling problem is complicated further by the expense of well drilling, which is still the primary method used to gain information about subsurface flow and transport. Drilling is time-consuming and labor intensive, and requires specialized equipment. Moreover, the drilling process disturbs the subsurface environment and, as a result, compromises the accuracy of pump tests and contaminant data collected from observation wells. Although alternative sampling methods based on geophysical or remote sensing technology have been applied success- fully in some situations, they are generally even less reliable than well samples. The expense, difficulty, and inaccuracy of field sam- pling all tent] to have an adverse impact on ground water modeling. Most modeling studies must make do with a very limited amount of unreliable data, which at best give only a rough picture of actual subsurface conditions. This basic fact needs to be recognized in any realistic assessment of the prediction capabilities of ground water models. Generally speaking, the field data used to estimate the inputs and check the predictions of ground water flow models are compiled from historical hydrogeologic surveys that were not planned with modeling in mind. Examples include periodic status reports issued by irrigation districts and state water agencies (primarily in the western United States), U.S. Geological Survey (USGS) water supply papers and open file reports, and water resource atlases compiled by a number of different governmental agencies. Until recently, many of the data included in these surveys were collected by local well drillers and geologists concerned primarily with water supply. These data tend to cover regions that are larger than those of interest in ground water contamination studies and therefore rarely deal with local geologic or hydrologic anomalies that may control transport

220 GROUND WATER MODELS in the vicinity of a hazardous waste site. In most hazardous waste studies, these traditional data sources are useful only for defining the boundary conditions of a site-specific flow model. The field data used in contaminant transport models typically have a very different history from those used in flow models. Most contaminant concentration measurements are collected at or near a contaminate site after an indication that some problem exists (e.g., observations of unusual taste or odor in well water). These mea- surements are usually limited and scattered, reflecting the locations of existing water supply wells rather than the geometry of the con- taminant plume (or plumes). Furthermore, contaminant data may be even more difficult to interpret than hydrogeologic data because the compounds observed and their physical state depend on chemical and biological conditions in the subsurface environment (see Chapter 2~. These comments suggest that there will be a need for a special- ized problem-oriented sampling program at most hazardous waste sites. Because sampling resources are nearly always quite limited, the objectives of the sampling program need to be spelled out care- fully so that a systematic and cost-effective field strategy can be developed. This strategy needs to be flexible enough to be able to deal with unanticipated results and unforeseen logistic problems but specific enough to provide guidance to drilling crews and managers responsible for approving budget expenditures. The dichotomy of flexibility and specificity is one that arises time and again in practi- cal sample programs. A site-specific hazardous waste field sampling program may have many different objectives, which can exert conflicting demands on limited resources. Some frequently encountered objectives include the following: ~ assessment of the severity of a newly discovered contamina- tion problem (i.e., a reconnaissance study); ~ monitoring of a known but more or less controlled hazardous waste site (e.g., for enforcement of a consent decree); monitoring of the performance of a remediation strategy (e.g., a pumping, treatment, and reinfection system); and ~ acquisition of data needed to develop or test a predictive model. Because this report is primarily concerned with ground water mod- eling, the focus is on the last of these objectives. It should be noted,

DEVELOPMENT AND USE OF MODELS 221 however, that modelers may need to reconcile their needs with those of other data users competing for limited resources and, in the pro- cess, may be forced to make compromises and adjustments in their approach. Recently, there has been a significant increase in research con the design of model-oriented ground water monitoring programs (Chu et al., 1987; Graham and McI,aughlin, 1989a,b; Knopma~ and Voss, 1987, 1988; McLaughlin and Wood, 198Ba,b). Although the specific methods proposed differ considerably, they generally view data col- lection an a way to reduce uncertainty. If it is possible to relate a specific data collection strategy to the uncertainty inherent in mod- eling, then it is possible to compare different strategies and select the one that is, in some sense, the best. Field sampling studies can, at [east in principle, help to reduce the major types of uncertainty including (1) lack of knowledge about the processes that control contaminant transport and transformation at a particular site and (2) incomplete knowledge of the spatially and temporally variable environmental factors that influence these processes. In fact, it is useful to divide a model-oriented sampling program into two phases: the first (less structured) phase attempts to identify relevant transport processes, whereas the second (more specific) phase attempts to quantify heterogeneous hydrogeologic and biochemical properties. Each of these is briefly discussed below. Process and Parameter Identification There is no truly systematic way to identify the physical, chem- ical, and biological processes at work at a particular contaminated site. This is a difficult scientific and engineering problem that re- quires creativity and experience as well as a good ability to identify inconsistencies and suspicious anomalies in a limited set of observa- tions. Nevertheless, it is possible to state three general principles that may help structure the field studies needed to support a sit~specific mode} development effort. I. A site-specific description of contaminant transport is strongly dependent on the quality of the flow mode} used to develop estimates of subsurface water velocities. Considerable care should be used in developing the inputs to the flow model, particularly in reference to the following:

222 GROUND WATER MODELS Well logs and water level data should be examined to determine the importance of three-dimensional (vertical) effects related to geological stratification, density differ- ences, buried sources, and so on. Vertical homogeneity should not be assumed without supporting documenta- tion. Borings and surficial geological information should be used to identify the primary hydrogeological features of the site including, as much as possible, local anomalies that may influence contaminant migration. Boundary conditions should be used to match the local flow field with known regional patterns and to account for interactions with surface features such as lakes or stream. Some sampling resources should be reserved for gathering information about flow boundary conditions, including recharge from the surface, if this information is not already available. The average value and likely range of soil properties such as hydraulic conductivity, porosity, and specific storage should be estimated from pump and piezometer tests, grain-size analyses, and if possible, permeameter tests of soil samples. A range of tests should be used so that variations at different scales can be assessed. If flow in the unsaturated zone or through highly con- ductive fractures is important, special care should be taken to assess, at least in a qualitative way, the role of these features. It is risky to assume that such effects are unimportant just because they are inconvenient. 2. The sampling program should recognize that contaminant dispersion is largely a manifestation of unknown hydrogeological heterogeneities (see Chapter 2~. These heterogeneities produce a more tortuous (heterogeneous) subsurface flow field than would be obtained under uniform conditions. Although intentional or after- the-fact tracer analyses can be used to estimate macroscopic disper- sivities, it is also possible to derive these macrodispersivities from theoretical analyses that recognize the variable nature of the small- scale flow field (Dagan, 1984; Gelhar and Axness, 1983; Neuman et al., 1987~. This is an important and somewhat controversial issue, which at least deserves consideration when designing a field sampling program.

DEVELOPMENT AND USE OF MODELS 223 3. The sampling program should attempt to either verify or rule out the various chemical and biological processes that may play a role in the transport and transformation of contaminants at the site. Because the responsible mechanisms depend largely on the chemical composition of the contaminants, the field program should provide a waste inventory, if one is not already available. The role of processes such as sorption, precipitation and colloidal transport, biodegradation, and multiphase transport and volatilization should be assessed before any detailed modeling is undertaken. This is a difficult task that is not readily codified but can benefit greatly from experience and from familiarity with the scientific literature on the transformation of subsurface contaminants. Particular care should be taken to ensure that sampling procedures and analytical techniques do not, by their very nature, automatically rule out observation of a potentially important transformation process. These general guidelines suggest that a significant portion of a model-oriented sampling effort should be devoted to a somewhat un- structured exploratory study that identifies the dominant processes to be included in subsequent modeling efforts. Many of the data collected in this exploratory phase can later be used to estimate the value of key mode! inputs. Input estimation and validation, the second phase of a model- oriented field sampling program, presume that the processes included in the mode} are, in fact, the ones that control contaminant behavior at the site of interest. The sampling program should provide the data needed to obtain the most accurate mode} inputs and predictions possible, subject to resource constraints. This is, in fact, a statement of the traditional sampling problem addressed by classical statistics (see, for example, Cochran and Cox, 1957; Cox, 1958; Fe~lerov, 1972; Kiefer and Wolfowitz, 1959~. Much of the literature dealing with this problem is based on sunple regression models and is oriented toward controlled field experiments (e.g., agricultural evaluations of hybrid crop types). Useful variants on the traditional approach are found in the extensive literature on the design of rain and stream gage networks (International Association of Hydrological Sciences, 1986) and in the geostatistical literature, which is largely concerned with mapping heterogeneous soil properties (Delhomme, 1979; Journe! and Hinjbregts, 1978~. Sampling design for contaminant transport applications is a new and active research area that has yet to produce practical techniques for designing site-specific monitoring programs.

224 GROUND WATER MODELS Nevertheless, it is possible to state some general principles that are beginning to emerge: ~ If a sampling program is intended to provide data for estimat- ing mode! inputs, it should be designed to minimize an appropriate measure of estimation uncertainty. In practice, this measure is often the mean-squared estimation error, although many other measures have been proposed. Because the estimation error depends on the structure of the mode! (e.g., the computational grid used to define mode} inputs) and on the inDut estimation Procedure. sampling de _ _ - - -A: - J ~& ~ ~ ~ ~ sign should be viewed as an Integral part ot the modeling process. In particular, the structure (e.g., resolution) of the mode! should re- flect field sampling constraints, and the sampling program should be designed to serve the model. This is a simple but frequently ignored principle that needs to be given more attention in practical modeling applications. ~ It is probably neither realistic nor desirable to seek a unique "optimal" sampling design, i.e., one that is unequivocally better than all competitors. This is because formalized optimization cannot consider all the factors that influence the selection (and evolution) of a given design. Such factors include logistic and legal constraints (e.g., access), conflicting objectives, unanticipated interruptions and delays, uncertainty about the relative importance of various natural processes, and the ever-present possibility of a totally unexpected discovery partway through the sampling process. Instead of seeking an optimum, an attempt should be made, at any stage of sampling, to identify the best among a set of reasonable alternatives. ~ The unpredictable nature of field sampling (which, after all, is most informative when it yields the least predictable results) suggests that practical sampling programs should evolve sequentially, with resources committed over a series of stages rather than all at once. Thus the results of each stage of sampling are used to update the models that form the basis for the sampling design. In fact, if field data suggest that a particular mode] is inappropriate, it may be discarded altogether and replaced by a more appropriate one before additional resources are committed. Sampling designs are highly dependent on the technical ca- pabilities ant} cost of the sampling devices used to collect data in the field and on the methods used to obtain and preserve samples for later analysis in the laboratory. Sampling technology is changing rapidly; therefore, a range of alternatives should be carefully considered be- fore extensive resources are committed to specialized equipment. It is

DEVELOPMENT AND USE OF MODELS 225 probably best to use a mix of several different approaches, some well- established and some more experimental (depending on the scope and objectives of the field effort). Remote sensing techniques may, for ex- ample, provide useful qualitative information about regional water level trends but not give a good match to observations obtained from more localized (and more expensive) piezometer measurements. A judicious combination of both techniques might be the best approach at some sites. These guidelines, like the ones stated earlier, confirm that the design of model-oriented field sampling programs is still a largely ad hoc endeavor that requires a good understanding of subsurface physical, chemical, and biological processes, of mode! and input estimation algorithms, and of sampling technology. It is rare for any one individual to be capable in all of these areas, which makes sampling design a truly multidisciplinary effort that typically requires the active participation of several specialists. This situation is likely to continue for the foreseeable future. Input Estanation In order to apply a chosen mode} formulation to a particular site, certain mode! inputs are required. These include coefficients, such as hydraulic conductivity, specific storage, porosity, and thermody- namic constants; boundary conditions, such as aquifer geometry and piezometric head, contaminant concentration, and mass flux along or across the aquifer boundary; and initial conditions, such as head and concentration distribution at a particular point in time. The most appropriate values of these inputs depend not only on the true physical, chemical, and biological state of the real world, but also on the amount of aggregation or averaging in the mode! formulation (e.g., the size of grid elements) and the mode} structure. The process of selecting appropriate input values is termed "input estimation." Many of the coefficients and input variables included in ground water models must be estimated on a case-by-case basis, usually from a relatively limited number of field observations of related quantities. Input estimation is one of the most difficult, and often most frustrat- ing, aspects of ground water modeling. Engineers and decisionmakers who use models need to understand the difficulties inherent in the estimation process if they are to make informed judgments about the desirability of modeling and the accuracy of mode! predictions.

226 GROUND WATER MODELS It is useful to distinguish at the outset two types of mode! inputs, which are treated somewhat differently in practice: 1. Constitutive Coefficients and Parameters. When a ground water mode} is formulated from basic principles (such as conserva- tion of mass or conservation of energy), quasi-empirical "laws" are often used to relate certain mode] variables. hnport ant examples include Darcy's law, which relates specific discharge to the hydraulic local head gradient, and Fick's law, which relates dispersive flux to the local concentration gradient (see Chapter 2~. Such empiri- cal laws introduce various so-called constitutive parameters that are generally not directly observable, but, rather, must be inferred from observations of other mode! variables. These include parameters such as hydraulic conductivity, dispersion coefficients, and partition coefficients. Field studies indicate that many of these parameters vary dramatically over space and, possibly, over time. Saturated hydraulic conductivity variations can easily vary 3 or 4 orders of magnitude over the scale of a typical contaminant site (Dagan, 1986; Gelhar, 1986~. Unsaturated conductivities vary even more, reflecting their dependence on moisture content. Theoretical and experimen- tal analyses indicate that field-scale dispersivity coefficients can vary over time and with the scale of the experiment, sometimes by orders of magnitude. The nonobservability and variability of constitutive parameters make them difficult to estimate, particularly when field measurements are limited. 2. Forcing Terms and Auxiliary Conditions. Most ground wa- ter models include forcing terms, which account for sources and sinks of water or dissolved contaminants. Flow models typically include pumping and recharge terms, whereas transport moclels typically include terms that describe where and when contaminants are intro- duced into the subsurface environment. In some cases, such forcing terms are measured directly. In other cases, they are inferred from measurements of more accessible variables, or they are simply pos- tulated (as, for example, when the effects of a proposed cleanup strategy are being investigated). Forcing terms generally act in the interior of a simulated region, at wells or disposal sites. Ground water and associated contaminants can also enter or leave the region across boundaries. The boundary conditions imposed on a model's solution can have an important impact on predicted flow and trans- port behavior. Parameters included in these boundary conditions (such as specified heads, concentrations, and fluxes) can sometimes

DEVELOPMENT AND USE OF MODELS 227 be inferred from field observations. They are more often simply pos- tulated. Similar remarks apply to initial conditions, which can be important in transient simulations. The traditional approach to ground water input estimation, de- veloped largely in water resource investigations of large aquifers, focuses on constitutive parameters such as hydraulic conductivity and dispersivity. In aquifer-scale applications, it is often feasible to select mode} boundaries and simulation periods so that auxiliary conditions can be readily specified. This is why, for example, bound- aries are often drawn along flow divides (yielding Influx boundary conditions) and simulations are often initialized when the ground water system is at steady state (enabling the initial conditions to be computed rather than measured). The traditional approach may not always work in hazardous waste applications, where the scale of the modeling problem is often much smaller (e.g., hundreds of meters rather than tens of kilometers) and where background contaminant concentrations may be highly uncertain. In such cases, it is more realistic to view boundary and initial conditions as inputs that need to be estimated from field measurements. Methods for estimating ground water inputs vary greatly, de- pending on the application and the resources available to the modeler. Input estimation is often posed as a so-called inverse problem. That is, mode! inputs are estunated from measurements of the model's outputs (the "inverse" of the direct modeling problem that computes outputs from specified inputs). The concept of "mode! calibration" is a variant on inverse estimation. Calibration is the process of ad- justing mode! inputs until the resulting predictions give a reasonably good fit to observed data. This process, which sounds reasonable enough on the surface, has the disadvantage of being "ill posed" in most ground water applications. The ill-posedness arises from the fact that an infinite number of input combinations can generally pro- vide acceptable fits to historical measurements. These combinations of parameters may differ greatly and may give significantly differ- ent results when used to predict future conditions. TIl-posedness (or nonuniqueness) has been studied by a number of researchers in vari- ous areas of science and engineering and is a problem that is familiar to most modelers. The primary practical solution to ill-posedness problems in ground water mode! calibration is to use "prior information" to guide or constrain parameter adjustments. Such information includes data obtained from soil samples, well and piezometer tests, laboratory

228 GROUND WATER MODELS experiments, and tracer tests, as well as sound engineering and ge- ological judgment based on experience with similar sites. If prior information is available, a variety of automated procedures may be used to carry out the inverse estimation process. Good reviews are provided by Carrera and Neuman (1986) and Yeh (1986~. Generally speaking, such procedures are used mostly by researchers, although they are beginning to be applied more frequently by the USGS and by some consulting firms. In the future, it is likely that automated inverse estimation algorithms will be included as part of the modeling packages distributed for general use by practicing hydrogeologists. The measurements used to estimate both constitutive parame- ters and auxiliary conditions are typically obtained at discrete times and locations (usually at monitoring welis). These local mea~ure- ments need to be extrapolated over larger regions if they are to be used for modeling purposes. There are many important examples. Well observations of hydraulic head (water level) and solute concen- trations constitute the primary source of data for mode! calibration. These observations need to be contoured to provide a synoptic pic- ture of the desired mode! response. Head and concentration contour maps are also needed to define source terms and auxiliary condi- tions that may be difficult to estunate with inverse techniques. Maps of hydraulic conductivities and porosities deduced from soil samples, piezometer tests, or geophysical measurements provide a good source of prior information for use in mode} calibration efforts. A number of methods are available for estimating regional dis- tributions of ground water mode} inputs from scattered well observa- tions. One of the most popular is a least-squares procedure known as kriging (Delhomme, 1979; Journe! and HinJbregts, 1978~. This pro- cedure, which originated in the mining industry, provides estimates of the accuracy of the contours it generates. Estimation accuracy depends, as might be expected, on the distance from observation points and on the heterogeneity of the contoured variable. Some care must be taken in using the procedure, however. It can give particu- larly deceptive pictures of contaminant plumes if used in its standard form, which assumes that contaminant concentration is in isotropic (nondirectional) and stationary (homogeneous) random fields. In reality, contaminant concentrations are highly anisotropic and non- stationary, particularly near sources. Generally speaking, traditional kriging packages are useful for contouring soil properties and other smoothly varying quantities but should not be used to extrapolate contaminant concentrations beyond spindle locations.

DEVELOPMENT AND USE OF MODELS 229 . Recently, several researchers have attempted to combine aspects of traditional inverse estimation with aspects of kriging, to provide a more integrated approach to input estunation (see, for example, Hoeksema and Kitanidis, 1984~. While this approach is a worthwhile endeavor, most applied hydrogeologists will continue, at least for the near future, to estimate mode! inputs in a more or less ad hoc way, using trial-and-error adjustments based on contoured data and intuition, with occasional help from an automated package. Whether the estimation procedure is manual or automated, the final results will depend greatly on how well the modeler understands the factors that relate predictions to the model's structure and input values. In particular, four issues should be kept in mind: 1. Mode! Formulation and Structure. The success of any pa- rameter estimation effort is critically dependent on the validity of the underlying mode! formulation. If the model's structure ignores important sources, geological heterogeneities, physical processes, or chemical reactions, parameter estimation will be reduced to a fitting exercise that forces available inputs to compensate (usually inade- quately) for an improper formulation. 2. Past Versus Future Performance. A good fit to historical data does not guarantee good predictions, particularly if the histori- cal fit is based on a small amount of data or if it does not test mode} capabilities that are required for making predictions. It is dangerous to "overfit" historical measurements by adjusting parameters beyond reasonable ranges. Although historical fits can reveal important in- formation about mode] behavior, they should be related to other relevant factors, including qualitative geological observations. 3. Sensitivity Analysis. Sensitivity analysis provides a useful (although not perfect) way to identify the mode} inputs that have the most influence on mode! predictions, at least over a specified range. Although a detailed sensitivity analysis can be laborious and time-consum~ng, it is usually feasible to carry out a small-scale exploratory analysis that focuses on a few critical inputs identified, most likely, by informed intuition. The sensitivity analysis should guide the selection of inputs included in the estimation process (see item 4~. 4. Choice of Estimated Inputs. The results of a ground water input estimation depend greatly on which inputs are based on field data and which are assumed to be well known. If, for example, a velocity used in a transport mode! points in the wrong direction, it

230 GROUND WATER MODELS will not be possible to obtain correct predictions by adjusting dis- persion coefficients or retardation rates, no matter how sophisticated the estimation algorithm is. When in doubt, all important inputs, e.g., source locations, source magnitudes, and auxiliary conditions, should be included in the estimation process. Underlying all of these points is the theme of mode} accuracy introcluced earlier in this chapter. Input estimation is one of several interrelated factors that influence the accuracy of a model's predic- tions. If the subsurface environment is very heterogeneous, measure- ments are very limited, or the mode} is improperly formulated, it is unlikely that the estimation process will be able, by itself, to ensure accurate predictions. The effort devoted to input estimation, and the sophistication of the estimation procedure, should be judged in a larger context that includes data collection and mode} formulation. Mode] Validation and Accuracy Aseesement The output of a mode! application exercise is a set of data representing the predicted behavior of the ground water system in response to one or more proposed management actions. These pre- dictions are determined by the particular combination of sampling process, mode} formulation, input estimation, and solution technique employed. They depend on decisions made in each step of the mode! application process. The accuracy (or validity) of a particular mode! application should logically be measured by the magnitude of the prediction er- rors, i.e., by some measure of the difference between the response of the real world and the response of the simulated system to man- agement actions. Such a comparison is complicated by the fact that prediction errors are uncertain because of sampling error. Also, the scale and/or level of aggregation of both the real and the modeled system response must be consistent if a valid comparison is to be made. It is particularly difficult to develop a priori assessments of modeling accuracy. Traditional methods of accuracy assessment focus on comparisons of predictions to historical measurements, eval- Hating goodness-of-fit after the fact. Although important indicators of mode! performance, such methods do not truly measure prediction errors. Mode! validation is a term that means different things to differ- ent people, largely because it is rarely defined with any precision. This general concept has both technical and policy origins. From

DEVELOPMENT AND USE OF MODELS 231 a technical viewpoint, modelers fee} a need to confirm or verify the hypotheses used in their models. This is generally accomplished by comparing predictions to observations, preferably under controlled conditions that can clearly reveal the sources of any discrepancies. From a policy viewpoint, regulatory agencies, courts, and public of- ficials fee! a need for standards that can be used to certify the results of a modeling effort or, more narrowly, to certify the use of a partic- ular computer program. In this case, the implicit goal seems to be to reduce the risk that a mode! will leac! to inappropriate decisions. Although this risk clearly depends on the model's accuracy, it also depends on how the mode! is used, i.e., on how the decision is made. Checking a model's validity by comparing its predictions with measurements is an important part of classical statistics. There are many statistical tests for evaluating models and related hypothe- ses. Traditional statistical methods are not particularly useful in ground water modeling studies, however, for several reasons. First, there are rarely enough measurements in ground water applications to provide a statistically rigorous test of a model's explanatory ca- pabilities. These measurements are typically available at scattered well locations, which are spaced further apart than characteristic scales of variability. Second, the conditions prevailing when the measurements were collected may not reflect those that the mode! is designed to simulate. Finally, most classical statistical tests are based on assumptions that are not necessarily met in complex sub- surface environments. These tests typically assume that the model's structure is perfect, and they are based solely on an analysis of the effects of measurement error. In reality, natural heterogeneity and deficiencies in mode! structure are likely to be far more important than measurement error. Since rigorous statistical validation tests are generally not ap- propriate in ground water applications, mode! validation is typically an ad hoc exercise that does not have a firm scientific foundation. Instead, mode! parameters are adjusted until a "reasonable" fit is oW tained and the result is presented as a "validated model." Modelers practically never declare their models to be "invalidated," primarily because ground water models nearly always have enough adjustable parameters to fit a limited set of field observations. This leads us to ask how we can distinguish a good fit that is based on artificial manipulation of an overparameterized mode} from a good fit that is based on an accurate description of the processes that control contaminant transport.

232 GROUND WATER MODELS One way to respond to the question posed above is to extend or generalize the concept of mode! validation. Instead of focusing on whether a mode! ~ valid, one can focus on evaluating its accu- racy. That is, one can attempt to quantify the probability that the model's predictions deviate from reality by more (or less) than a specified amount at any given time or location. Accuracy can be conveniently expressed in terms of confidence limits or, given appro- priate assumptions, in terms of the risk associated with a particular decision based on mode} predictions. Such information is ultimately both more useful and more realistic than a certification that a mode! is or is not validated. Although a quantitative assessment of mode} accuracy would undoubtedly be useful, it is not clear how such an assessment can be developed when the data needed to test mode! performance are very limitecI. One approach to this dilemma is to carry out a mode! "error analysis." Prediction errors can ultimately be traced to three basic sources: 1. natural heterogeneity that cannot be completely described with a limited number of field samples, 2. 3. mode! used to represent it. measurement errors, and structural differences between the real-worId system and the A structurally perfect mode} that uses inputs estimated from perfect measurements of a homogeneous real world will produce perfect predictions. Departures from this ideal situation can be attributed to one or more of the above error sources. Once the fundamental sources of mode! error are identified, prediction accuracy may be investigated by analyzing the model's sensitivity to changes in appropriate error source variables. Monte CarIo simulation provides a particularly convenient method of anal- ysis (Chu et al., 1987; Graham and McLaughlin, 1989a; Smith and Schwartz, 1980, 198la,b). A Monte Cario analysis teased on Figure 4.1 essentially repeats the entire modeling process sanding, input estimation, and prediction- many times. Each of these hypothetical modeling studies (or replicates) is baked on a different synthetically generated real-worId description and a different synthetically gener- ated set of measurement errors. The prediction error obtained from a given replicate is simply the difference between the mode} prediction and the corresponding real-worId value. Prediction confidence inter- vals and other related statistics can be readily computed from the

DEVELOPMENT AND USE OF MODELS 233 complete ensemble of prediction error replicates. This generalization of traditional mode! sensitivity analysis includes field sampling and input estimation as well as the mode! proper. An application to ground water transport is described in detail in Chu et al. (1987~. It might be argued that Monte CarIo validation methods are too complex and time-consuming to be practical in most hazardous waste modeling applications. Although this viewpoint may be true, it should be remembered that much of the controversy surrounding the use of models ultimately stems from differing assessments of mode} accuracy. Typically, a judge, jury, or administrator is asked to decide between two differing conclusions, both based on ostensibly competent modeling studies. If the confidence intervals associated with the mode! predictions are greater than the difference between the predictions, this difference cannot be considered meaningful, at least in a statistical sense. There is no way to know whether such a situation has occurred without carrying out a serious investigation of mode! accuracy. It seems likely that courts, regulatory agencies, and other users of mode! results will press for more rigorous approaches to mode! validation as dependence on mode! results becomes more common. This is clearly the motivation behind recent proposals to establish common modeling standards and quality assurance criteria (see the sections on mode! quality assurance below). In the future, modelers will probably have to devote more effort to variegation and related error analysis if their models are to have any credibility in public or legal forums. ASSURING THE QUALITY OF MODELS A successful application of a mode! requires knowledge of sci- entific principles, mathematical methods, and site characterization, paired with expert insight into the modeling process. Previous chap- ters discuss the types of processes that can be modeled and the formulation of models. The issues discussed are certainly complex and require a concerted effort by the user of the model. A practitioner approaching a modeling study is presented with two challenges: (1) formulating the model, including the development of boundary con- ditions and input parameters, and (2) the more mundane task of documenting and checking the modeling process. Most modelers en- joy the modeling process but find less satisfaction in the process of documentation and quality assurance (QA). However, both aspects

234 GROUND WATER MODELS are equally important to a successful application of a model. Docu- mentation of all aspects of the modeling is important to ensure that the study would be reliably repeated. Quality assurance is defined as the procedural and operational framework used by an organization managing the modeling study to assure technically and scientifically adequate execution of all tasks included in the study and to assure that all modeling-based analysis is reproducible and defensible (Taylor, 1985~. This definition will be used in this report. In ground water modeling, QA is crucial to both development and application of the mode} and should be an integral part of planning, applied to all phases of the modeling process. Adequate documentation and other forms of QA are becoming increasingly important as applications of models become part of regulatory submittals and are used to judge regulatory compliance. Both the regulators and the regulated community have an obligation to provide a complete picture of any modeling study. The section below on "Quality Assurance Procedures for Code Development" identifies some of the issues that may arise during modeling and provides some direction on documenting the modeling process. The information contained in this chapter is necessarily limited. The reader is directed to the reference list for additional information on various aspects of quality assurance. Certain negative elements may be associated with poorly con- ceived and implemented QA plans. For example, there is the risk that a QA checklist will serve only to instill false confidence in mode! results. Another problem can be that the time and the cost of follow- ing bureaucratically imposed QA procedures may be so great that funds available for data collection and hydrogeologic analysis in any given problem could be significantly depleted. Quality assurance procedures are not generally embraced by the modeling community. There is a perception that it has not been convincingly demonstrated that QA programs improve the quality of models. Some individuals believe that if a mode! is developed by qualified people who understand both the processes being simulated and the properties and boundaries of the area being represented, then it is very unlikely that QA would yield a better mode! (and if QA costs are high, QA may actually yield a worse model). further, if unqualified or unprepared people are doing the work, then it is unclear that a formal QA procedure would yield a better model. (It may, however, yield unwarranted confidence in the mode! because a QA procedure was followed.)

DEVELOPMENT AND USE OF MODELS 235 The committee believes in the need for QA procedures as a key element in contemporary modeling studies. The important benefits of well-designed QA procedures need to be highlighted and demon- strated through programs of applied research. QA concepts will make a positive contribution to the way models are constructed and used; however, there is certainly a need to refine and extend existing approaches. QUALITY ASSURANCE PROCEDURES FOR CODE DEVELOPMENT The most important QA procedures in code development and maintenance applicable to ground water models are (van der Heij~e, 1987) as follows: verification of structure and coding; validation of theoretical basis (mode! validation); documentation of code development and testing (recor~kee~ in"); documentation of characteristics, capabilities, and use of code (software documentation); and scientific and technical reviews. If any modifications are made to the mode! coding for a spe- cific problem, the code should be tested again; all QA procedures for mode! development should again be applied, including accurate recor~keeping and reporting. All new input and output files should be saved for inspection and possible reuse. The following subsections briefly describe the various elements of QA procedures. Verification of Program Structure and Code The objective of the code verification process is twofold: (1) to demonstrate that the computational algorithms can accurately solve the governing equations and (2) to assure that the computer code is fully operational. To check the code for correct coding of theoretical principles, for code logic, and for major programrn~ng errors ("bugs"), the code is run with specially designed problems. The computational algorithms embedded in the code are often tested using problems for which an analytical solution exists. This stage of code testing is also used to evaluate the sensitivity of the code to the design of the grid, to

236 GROUND WATER MODELS various dominant processes, and to a wide selection of parameter values (Gupta et al., 1984; Huyakorn et al., 1984~. Although testing numerical computer codes by comparing results for simplified situations with those of analytical models does not guarantee a fully debugged code, a well-selected set of problems ensures that the main program and most of the subroutines are used in the testing. The effectiveness of a verification exercise can be further enhanced by using a so-called walk-through, that is, astep- by-step analysis of the program operation using the data from the test cases. A major problem with testing numerical computer codes is that analytical models are available only for simple flow or mass transport problems. The situations that numerical models are built to deal with (e.g., heterogeneous system properties and irregular boundaries) cannot be evaluated. In effect, it often has to be assumed that because the mode! is accurate for simple problems it will be accurate for complex conditions. Therefore, as part of the verification process, hypothetical problems might be used to test special computational features that are not represented in simple, analytical models, as in testing for irregular boundaries, varying boundary conditions, or certain heterogeneous and anisotropic aquifer properties. These hypothetical problems can be simulated by independent codes and the results compared. Mode! developers are in the best position to produce a comprehensive set of verification tests for their models because they are most familiar with the structure of the coding. Mode] Validation The objective of mode} validation is to determine how well the mathematical representation of the processes describes the actual system behavior in terms of the degree of correlation between mode} calculations ant} actual measured data. Ideally, results should be compared to the results of a well-defined field experiment or a welI- conditioned laboratory experiment. Validation of the predictive capabilities of the mode} is accom- plished through comparison with experimental data by using inde- pendent estimates of the parameters. In principle, this is the ideal approach to validation. However, unavailability and inaccuracy of field data often prevent the application of such a rigid validation approach to actual field systems. Typically, parts of the field data are designated as calibration data, and a calibrated site mode! is

DEVELOPMENT AND USE OF MODELS 237 obtained through reasonable adjustment of parameter values. Other parts of the field data are designated as validation data; the cali- brated site mode} is used in a predictive mode} to simulate similar data for comparison. Although this procedure will not allow com- plete validation of a modeling process, it will provide some insight into potential problems in mode! use. This approach is limited be- cause the splitting of the ground water data set into two components (to be used for two purposes) does not yield two independent sets of data. Two independent sets of data do not occur because of the slowness of responses in ground water systems and the production of persistence in memory over time, so that the calibration data and validation data are related. Mode} validation is, of course, supposed to yield a valid model. However, a valid mode} is an unattainable goal of mode} validation. Several different groups in ground water modeling have in the past defined and used the terms validate and verify to mean different things. In fact, there is no consensus among ground water hydrolo- gists, either on the definition of these two terms or on how to achieve validation and verification. These terms and their underlying concepts and implications are among the contributing factors that have led to some abuses and misuses of models. Some people may fee} that once a mode} has been designated as being validated, it does not require further questioning or testing. They may then apply the mode! to make predictions for conditions in which stresses lie outside their historically observed range (or outside the range of the calibration data) and produce an unreliable prediction (or an invalid prediction from a supposedly valid model). Conversely, do uncalibrated (or unvalidated or unverified) mod- els have any value? Definitely. Deterministic models can be used to gain insight into ground water processes that may be controlling responses in a given area, even if too few data are available from that area to yield a satisfactory calibration. Such preliminary models can help the analysts improve their understanding of the problem, for- mulate hypotheses to be tested, and prioritize data collection efforts. Absolute validity of a mode! is never determined. Establishing absolute validity requires testing over the full range of conditions for which the mode! is designed, an exercise that is often not possible or practical. For many types of models this is due to the lack of adequate, high-quality field data. Thus testing of ground water

238 GROUND WATER MODELS models is generally limited to extended verifications, using existing analytical solutions, and to code comparisons. In the comparison of codes, a newly developed mode} is compared with established models designed to solve the same type of problems. If the results from the new code do not deviate significantly from those obtained tenth the existing code, a relative or comparative validity is established. However, if significant differences occur, in- depth analysis of the results and codes is required. If code comparison was used to evaluate a new code, all the involved models should again be validated as soon as adequate data sets become available. Various approaches to field validation of a mode! are viable. Therefore the validation process should start with defining validation scenarios. Field validation should include the following steps (Hern et al., 1985~: . Define data needs for validation and select an available data set or arrange for a site. Assess the quality of data in terms of accuracy (measurement errors), precision, and completeness. Define performance or acceptance criteria of the model. Develop strategy for analysis of sensitivity. ~ Perform validation runs and compare performance of the mode} with established acceptance criteria. Document the validation exercise in detail. Recordl~eeping Quality assurance for development and maintenance of codes should include complete recor~keeping of the development, modifi- cations, and phase validation of the code. The paper trail for QA in the development phase consists of reports and files on the develop- ment and testing of the model. Software Documentation Software documentation explains all pertinent aspects of the sys- tem represented in the software, including purposes, methods, logic, relationships, capabilities, and limitations (Gass, 1979~. Complete documentation consists of information recorded during the design, development, and maintenance of computer applications. It is the principal instrument used by those involved in a modeling effort,

DEVELOPMENT AND USE OF MODELS 239 such as authors, programmers, users, and system operators, to com- municate efficiently regarding all aspects of the software. Good documentation includes a complete treatment of the equa- tions on which the mode} is based, the underlying assumptions, the boundary conditions that can be incorporated in the model, the method used to solve the equations, and any limitations related to the particular method of solution. The documentation must also include instructions for operating the code and preparing data files, example problems complete with input and output, programmer's instructions, operator's instructions, and a report of the verification. The importance of clear documentation cannot be overemphasized. Improper documentation will prevent a code from being adequately reviewed and could propagate errors in code use. Documentation should commence at the very beginning of a software development project. Scientific and Ethnical Review Generally, the complete scientific and technical review process is qualitative in nature and comprises examination of mode! concepts, governing equations, and algorithms chosen, as well as evaluation of documentation and general ease of use, inspection of the structure of the program and the logic, handling of errors, and examination of the coding (Bryant and Wilburn, 1987; van der Beige et al., 1985b). If verification or validation runs have been made, the review process should inclucle evaluation of these processes. To facilitate thorough review of the model, detailed documenta- tion of the mode} and its developmental history is required, as is the availability of the source code for inspection. In addition, to ensure independent evaluation of the reproducibility of the results of verifi- cation and validation, the computer code should be available for use by the reviewer, together with files containing the original test data used in the verification and validation. MODEL APPLICATION Quality assurance in mode! application studies includes review of the selection of data, data analysis procedures, methodology of modeling, and administrative procedures and auditing. To a large extent, the quality of a modeling study is determined by the expertise of the team involved in the modeling and quality assessment.

240 GROUND WATER MODELS In many cases, the person developing the mode! may never have seen or visited the field area. This can easily lead to fatal flaws in the mode! design and parameter estimation, because significant (and perhaps obvious) hydrogeologic features are not recognized and in- corporated into the model. This problem may be exacerbated if the mode] designer or user has expertise with mathematics, numerical analysis, and/or computer simulation methods, but little field expe- rience. In larger organizations, it is common to have such a division of labor, particularly when projects have relatively short deadlines. Where efforts are so divided, modeling may be performed by an en- tirely different and separate group of specialists. Unfortunately, this may produce a tendency for the mode} to become an end unto itself, rather than a means (or one of many tools) for analysis and problem solving. There may also be a tendency for such projects to not fully recognize or accommodate the need for the mode} developer to have familiarity with the field area or allow time for analysts to benefit from feedback between the mode! analysis and the field investiga- tion. On the other hand, if the same person or team of analysts is performing data analysis, data collection, and modeling, it is more likely that the mode! will include realistic and appropriate boundary conditions, system properties, and discretization. It is too easy to calibrate (validate) a mode} while being unaware of major springs, pumping wells, and surface drains or ditches (or other features) that may be controlling ground water levels and gradients in an area. Ignorance of one feature is compensated by errors in values specified for other parameters. A locally steep hydraulic gradient that exists because of a drainage ditch may be interpreted as indicating a low- transm~ssivity zone. Such ignorance of the field area can lead to a mode} that matches historical data but fails in a predictive mode. (Prediction, of course, is purportedly one of the primary values and incentives for using deterministic models. If the goal were merely to achieve a best fit to observed data, then a purely statistical model, such as a multiple regression equation, would most easily meet that objective.) Thus a well-calibrated and validated mode! is not nec- essarily an accurate or a reliable one. This fact is supported by Freyberg (1988), who reports on a numerical experiment in which nine groups of analysts used the same numerical mode! and identical sets of observed data to calibrate the mode! and predict the response to a specified change in a boundary condition. Success in predic- tion was unrelated to success in matching observed heads, and good calibration alone did not lead to good prediction.

DEVELOPMENT AND USE OF MODELS 241 In summary, a ground water mode} (or any scientific mode} or theory, for that matter) can never be proven, verified, or validated in the strictest sense of the terms by agreement ninth a specific set of observations. Rather, a mode} can only be invalidated by disagreement with observations. Agreement should serve only to increase confidence in the theory or model. Quality assurance in code application should cover all facets of the modeling process. It should address issues such as the following: project description and objectives; · correct and clear formulation of problems to be solved: type of modeling approach to the project; . conceptualization of system and processes, including hydro- geologic framework, boundary conditions, stresses, and controb; . detailed description of assumptions and simplifications, both explicit and implicit (to be subject to critical peer review); . data acquisition and interpretation; mode} selection or justification for choosing to develop a new model; . mode! preparation (parameter selection, data entry, or refor- matting, "ridding); . the validity of the parameter values used in the mode! appli- cation; . protocols for estimating values of controlling parameters and for steps to be followed in calibrating a model; · level of information in computer output (variables and pa- rameters displayed, formats, layout); . identification of calibration goals and evaluation of how well they have been met; . role of sensitivity analysis; · postsimulation analysis (including verification of reasonabil- ity of results, interpretation of results, uncertainty analysis, and the use of manual or automatic data processing techniques, as for contouring); . establishment of appropriate performance targets (e.g., a 6- ft head error should be compared with a 20-ft head gradient or drawdown, not with the Waft aquifer thickness) that recognize the limits of the data; presentation and documentation of results; and . evaluation of how closely the modeling results answer the questions raised by management.

242 GROUND WATER MODELS In exceptional circumstances, it may be possible to conduct what has come to be referred to as a postaudit. A postaudit compares mode! predictions to the actual outcome in field conditions. Al- though postaudits are used primarily to determine the success rate of a mode} application, positive results of a well-executed postaudit analysis contribute to the acceptability of the mode} itself. To use a postaudit successfully in conceptualization, assumptions, and sys- tem parameters and stresses, it should be evaluated and, if necessary, updated and the mode} rerun to facilitate comparison of predictions with recent, observed system responses. The importance of postaudit studies has been outlined by Konikow (1986), Lewis and Goldstein (1982), and Person and Konikow (1986~. An example that illustrates the importance of postaudits is the Snake River plain case study described in Chapter 5. An increasing number of costly decisions are made in part on the basis of the outcome of modeling studies. In the light of major differences noted in comparative studies on mode} application (e.g., Freyberg, 1988; McLaughlin and Johnson, 1987), it should not come as a complete surprise that several groups modeling the same problem may obtain different results. While this is not a QA issue, provisions might have to be made to resolve the inconsistencies in the modeling effort. A third team or a pane] can be created to review and compare the results of both modeling efforts and to assess the importance and nature of differences present. Quality assurance is the responsibility of both the project team and the contracting or supervising organization. It should not drive or manage the direction of a project, nor is it intended to be an after-the-fact filing of technical data. Although the need for QA programs ~ apparent, the extent to which they are being applied in practice can be variable. For example, EPA rarely uses peer review for models applied in the Superfund and Resource Compensation and Recovery Act (RCRA) programs. Only recently has EPA provided a checklist of steps that a modeler must take to assure that a mode} is valid. When carefu! peer review and oversight of the development and application of contaminant transport models have been performed, the quality of the modeling has been good (see the S-Area case study). The application of contaminant models can be greatly improved by the use of peer review experts. Every mode} used by or relied on by EPA, including those in the Superfund program, should go through peer review. (Various groups have endorsed peer review

DEVELOPMENT AND USE OF MODELS 243 in the regulatory system, e.g., Administrative Conference of the United States, Recommendation No. 82-5, Advisory Panels No. 1, 1 CFR 305.82.5, 1988.) This review could involve the mathematical code, the hydrogeological/chemical/ biological conceptualization, the adequacy of the data, and the application of the mode] to the site- specific data. Additionally, the peer review should consider whether the prediction being called for exceeds the scientific validity of the model, e.g., the prediction of a concentration over a 10,000-yr period with a mode! validated over a 10-yr period. BIBLIOGRAPHY Adrion, W. R., M. A. Branstad, and J. C. Cherniasky. 1981. Validation, Verification, and Testing of Computer Software. NBS Special Publication 500-75, Institute for Computer Science and Technology, National Bureau of Standards, U.S. Department of Commerce, Washington, D.C. Adrion, W. R., M. A. Branstad, and J. C. Cherniasky. 1982. Validation, verification, and testing of computer software. ACM Computing Surveys 14~2), 159-192. American Society for Testing and Materials. 1984. Standard Practices for Evaluating Environmental Fate Models of Chemicals. Annual Book of ASTM Standards, E 978-84, Philadelphia. Beljin, M. S., and P. K. M. van der Heijde. 1987. Testing, Verification, and Validation of Two-Dimensional Solute Transport Models. In Groundwater Contamination: Use of Models in Decision-making, Y. Y. Haimes and J. Bear, eds. D. Reidel, Dordrecht, The Netherlands. Boutwell, S. H., S. M. Brown, B. R. Roberts, and D. F. Atwood. 1985. Modeling Remedial Actions at Uncontrolled Hazardous Waste Sites. EPA/540/2- 85/001, Environmental Protection Agency, OSWER/ORD, Washington, D.C. Brandstetter, A., and B. E. Buxton. 1987. The role of geostatistical, sensi- tivity and uncertainty analysis in performance assessment. In Proceedings of the 1987 DOE/AECL Conference on Geostatistical, Sensitivity and Uncertainty Methods for Groundwater Flow and Radionuclide Transport Modeling, September 15-17. U.S. Department of Energy, Washington, D.C. Bryant, J. L., and N. P. Wilburn. 1987. Handbook of Software Quality Assur- ance Techniques Applicable to the Nuclear Industry. NUREG/CR-4640, Office of Nuclear Reactor Regulation, U.S. Nuclear Regulatory Commis- sion, Washington, D.C. Carrera, J., and S. P. Neuman. 1986. Estimation of aquifer parameters under transient and steady state conditions, 1. Maximum likelihood method incorporating prior information. Water Resources Research 22~2), 199- 210. Chu, W., E. W. Strecker, and D. P. Lettenmaier. 1987. An evaluation of data requirements for groundwater contaminant transport modeling. Water Resources Research 23~3), 408-424. Cochran, W. G., and G. M. Cox. 1957. Experimental Designs. Wiley, New York.

244 GROUND WATER MODELS Codell, R., and S. Silling. 1984. Draft Quality Assurance Plan for Opera- tional Software. Memorandum 3110/DC/84/09/20/0, Division of Waste Management, U.S. Nuclear Regulatory Commission, Washington, D.C. Council of State Governments. 1985. Risk Management and the Hazardous Waste Problems in State Governments. Prepared for the National Science Foundation. Cox, D. R. 1958. Planning of Experiments. Wiley, New York. Dagan, G. 1984. Solute transport in heterogeneous porous formations. Journal of Fluid Mechanics 145, 151-177. Dagan, G. 1986. Statistical theory of groundwater Bow and transport: Pore to laboratory, laboratory to formation, and formation to regional scale. Water Resources Research 22~9), 120s-134s. Delhomme, J. P. 1979. Spatial variability and uncertainty in groundwater flow parameters: A geostatistical approach. Water Resources Research 15~2), 251-256. Environmental Protection Agency. 1986. Information on Preparing Quality Assurance Narrative Statements. RSKERL-QA-2, Robert S. Kerr, Envi- ronment Research Laboratory, Ada, Okla. Environmental Protection Agency. 1987. Evaluation of Implementation of Risk- Based Decisionmaking in RCR-A, Annotated Briefing. Program Evaluation Division, Office of Policy, Planning and Evaluation, pp. 8, 9, 12, 17, 25. Federal Computer Performance Evaluation and Simulation Center. 1981. Com- puter Model Documentation Guide. NBS Special Publication 500-73, Insti- tute for Computer Science and Technology, National Bureau of Standards, Department of Commerce, Washington, D.C. Federov, V. V. 1972. Theory of Optimal Experiments. Academic Press, New York. E`reyberg, D. L. 1988. An exercise in ground-water model calibration and prediction. Ground Water 26~3), 350-360. Gass, S. I. 1979. Computer Model Documentation: A Review and an Approach. NBS Special Publication 500-39, Institute for Computer Science and Tech- nology, National Bureau of Standards, U.S. Department of Commerce, Washington, D.C. Gelhar, L. W. 1986. Stochastic subsurface hydrology from theory to application. Water Resources Research 22~9), 135S-145S. Gelhar, L. W., and C. L. Axness. 1983. Three dimensional stochastic analysis of macrodispersion in aquifers. Water Resources Research 19~1), 161-180. General Accounting Office. 1987. Report to Congress: Superfund Improvements Needed in Work Force Management. GAO/RCED-88-1, pp. 2, 3. Graham, W., and D. McLaughlin. 1989a. Stochastic analysis of nonstationary subsurface solute transport, 1. Unconditional moments. Water Resources Research 25~2), 215-232. Graham, W., and D. McLaughlin. 1989b. Stochastic analysis of nonstationary subsurface solute transport, 2. Conditional moments. Water Resources Research, in press. Gupta, S. K., C. R. Cole, F. W. Bond, and A. M. Monti. 1984. Finite-Element Three-Dimensional Ground-Water (FE3DGW) Flow Model: Formulation, Computer Source Listings, and User's Manual. ONWI-548, Battelle Memo- rial In~titute, Columbus, Ohio.

DEVELOPMENT AND USE OF MODELS 245 Harlan, C. P., and G. F. Wilkinson. 1988. High-Level Waste Management Code Maintenance and Quality Assurance. SAND87-2254/UC-70. Sandia National Laboratories, Albuquerque, N. Mex. Hern, S. C., S. M. Melancon, and J. E. Pollard. 1985. Generic steps in the field validation of Vadose Zone fate and transport models. Pp. 61-80 in Vadose Zone Modeling of Organic Pollutants, S. C. Hern and S. M. Melancon, eds. Lewis Publishers, Chelsea, Mich. Hoeksema, R. J., and P. Kitanidis. 1984. An application of the geostatistical approach to the inverse problem in two-dimensional groundwater modeling. Water Resources Research 20~7), 1003-1020. Hoffman, F. O., and R. H. Gardner. 1983. Evaluation of uncertainties in radio- logical assessment models. In Radiological Assessment: A Text on Environ- mental Dose Analysis, J. E. Till and H. R. Meyer, eds. NUREG/CR-3332, ORNL-5968, U.S. Nuclear Regulatory Commission, Washington, D.C. Huyakorn, P. S., A. G. Kretschek, R. W. Broome, J. W. Mercer, and B. H. Lester. 1984. Testing and Validation of Models for Simulating Solute Transport in Groundwater: Development, Evaluation, and Comparison of Benchmark Techniques. GWMI 84-13, International Ground Water Mod- eling Center, Holcomb Research Institute, Butler University, Indianapolis, Ind. Institute of Electrical and Electronic Engineers, Inc. 1984. Standard for Software Quality Assurance Plane. IEEE Standard 730-1984, New York. Intera Environmental Consultants, Inc. 1983. A Proposed Approach to Un ,: ~ ,~ certainty Analysis. ONWI-488, Battelle Memorial Institute, Columbus, Ohio. International Association of Hydrological Sciences. 1986. Integrated Design of Hydrological Networks, Proceedinge of the Budapest Symposium. IAHS Publication 158. International Ground Water Modeling Center. 1986. U.S. EPA Ground-Water Modeling Policy Study Group: Report of Findings and Discussion of Selected Ground-Water Modeling Issues. Holcomb Research Institute, Butler University, Indianapolis, Ind., pp. 2, 5, 15, 19, 27. Journel, A. G., and Ch. I. Hinjbregts. 1978. Mining Geostatistics. Academic Press, San Diego, Calif. Kiefer, J., and J. Wolfowitz. 1959. Optimum designs in regression problems. Annals of Mathematical Statistics 30, 271-294. Kincaid, C. T., J. R. Morrey, and J. E. Rogers. 1984. Geohydrological Models for Solute Migration, Vol. 1, Process Description and Computer Code for Selection. EA 3417.1, Electric Power Research Institute, Palo Alto, Calif. Knopman, D. S., and C. I. Voss. 1987. Behavior of sensitivities in the one-dimensional advection-dispersion equation: Implications for parameter estimation and sampling design. Water Resources Research 23~2), 253-272. Knopman, D. S., and C. I. Voss. 1988. Further comments on sensitivities, parameter estimation, and sampling design in one-dimensional analysis of solute transport in porous media. Water Resources Research 24~2), 225-238. Konikow, L. F. 1986. Predictive accuracy of a ground-water model-Lessons from a post-audit. Ground Water 24~2), 677-690.

246 GROUND WATER MODELS Lewis, B. D., and F. J. Goldstein. 1982. Evaluation of a Predictive Ground- Water Solute Transport Model at the Idaho National Engineering Labora- tory, Idaho. Water Resources Investigation 82-25, U.S. Geological Survey, Reston, Va. McLaughlin, D., and W. K. Johnson. 1987. Comparison of three groundwa- ter modeling studies. ASCE Journal of Water Resources Planning and Management 113~3), 405-421. McLaughlin. D.. and E. F. Wood. 1988a. A distributed parameter approach ,7 _ . . . . .. . _ for evaluating the accuracy of groundwater predictions, 1. Theory. Water Resources Research 24~7), 1037-1047. McLaughlin, D., and E. F. Wood. 1988b. A distributed parameter approach for evaluating the accuracy of groundwater predictions, 2. Application to groundwater flow. Water Resources Research 24~7), 1048-1060. Mercer, J. W., and C. R. Faust. 1981. Ground Water Modeling. National Water Well Association, Dublin, Ohio. National Bureau of Standards. 1976. Guidelines for Documentation of Com- puter Programs and Automated Data Systems. FIPS 38, Federal In- formation Processing Standards Publications, Department of Commerce, Washington, D.C. National Research Council. 1988. Hazardous Waste Site Management: Water Quality Issues. Report on a colloquium sponsored by the Water Science and Technology Board. National Academy Press, Washington, D.C., p. 9. Neuman, S. P., C. L. Winter, and C. M. Newman. 1987. Stochastic theory of field-scale Fickian dispersion in anisotropic porous media. Water Resources Research 23~3), 453-466. Nicholson, T. J., T. J. McCartin, P. A. Davis, and W. Beyeler. 1987. NRC experiences in HYDROCOIN: An international project for studying ground- water Bow modeling strategies. In Proceedings, GEOVAL 87, April 4-9, Stockholm. Swedish Nuclear Power Inspectorate, Stockholm. Office of Technology Assessment. 1982. Use of Models for Water Resources Management, Planning, and Policy. Pp. 9, 18-19,20,22,24,102-104. Person, M., and L. F. Konikow. 1986. Recalibration and predictive reliability of a solute-transport model of an irrigated stream-aquifer system. Journal of Hydrology 87,145-165. Pesticide and Toxic Chemical News (Food Chemical News). 1987. Exposure estimates seen as weak link in EPA risk assessments. Vol. 16, p. 5 (December 23, 1987~. Rao, P. S. C., R. E. Jessup, and A. C. Hornsby. 1981. Simulation of nitrogen in agro-ecosystems: Criteria for model selection and use. Pp. 1-16 in Nitrogen Cycling in Ecosystems of Latin America and the Caribbean, Proceedings of the International Workshop, Call, Colombia, March 16-21. Silling, S. A. 1983. Final Technical Position on Documentation of Computer Codes for High-Level Waste Management. NUREG/CR-0856-F, Office of Nuclear Material Safety and Safeguards, U.S. Nuclear Regulatory Com- mission, Washington, D.C. Simmons, C. S., and C. R. Cole. 1985. Guidelines for Selecting Codes for Groundwater Transport Modeling of Low-Level Waste Burial Sites, Vol. 1, Guideline Approach. PNL-4980 Vol. 1. Battelle Pacific NW Laboratory, Richland, Wash. Smith, L., and F. W. Schwartz. 1980. Mass transport, 1. A stochastic analysis of macroscopic dispersion. Water Resources Research 16~2), 303-313.

DEVELOPMENT AND USE OF MODELS 247 Smith, L. and F. W. Schwartz. 1981a. Mass transport, 2. Analysis of uncertainty in predictions. Water Resources Research 17~2), 351-369. Smith, L. and F. W. Schwartz. 1981b. Mass transport, 3. Role of hydraulic conductivity. Water Resources Research 17~5), 1463-1479. Sykes, J. F., S. B. Pahwa, D. S. Ward, and R. B. Lantz. 1983. The validation of SWENT, a geosphere transport model. In Scientific Computing, R. Stepleman et al., eds. IMACS/North-Holland Publishing Company, New York. Taylor, J. K. 1985. What is quality assurance? Pp. 5-11 in Quality Assurance for Environmental Measurements, J. K. Taylor and T. W. Stanley, eds. ASTM Special Technical Publication 867, American Society for Testing and Materials, Philadelphia. van der Heijde, P. K. M. 1984. Availability and applicability of numerical mod- els for groundwater resources management. In Practical Applications of Ground Water Models, Proceedings NWWA/IGWMC Conference, Colum- bus, Ohio, Aug. 15-17. National Water Well Association, Dublin, Ohio. van der Heijde, P. K. M. 1987. Quality assurance in computer simulations of groundwater contamination. Environmental Software 2~1), 19-28. van der Heijde, P. K. M., and M. S. Beljin. 1988. Model Assessment for Delineat- ing Wellhead Protection Areas. EPA 440/6-88-002, Office of Ground-Water Protection, Environmental Protection Agency, Washington, D.C. van der Heiide, P. K. M., and R. A. Park. 1986. U.S. EPA Groundwa- ter Modeling Policy Study Group, Report of Findings and Discussion of Selected Groundwater Modeling Issues. International Ground Water Mod- eling Center, Holcomb Research Institute, Butler University, Indianapolis, Ind. van der Heijde, P. K. M., Y. Bachmat, J. Bredehoeft, B. And rews, D. Holtz, and S. Sebastian. 1985a. Ground-Water Management: The Use of Numerical Models, 2nd ed. Water Resources Monograph 5, American Geophysical Union, Washington, D.C. van der Heijde, P. K. M., P. S. Huyakorn, and J. W. Mercer. 1985b. Testing and validation of ground water models. In Proceedings, NWWA/IGWMC Conference on Practical Applications of Groundwater Models, Columbus, Ohio, Aug. 19-20. National Water Well Association, Dublin, Ohio. van der Heijde, P. K. M., A. I. El-Kadi, S. A. Williams, and D. L. Cave. 1988. Groundwater Modeling: An Overview. GWMI 88-10. International Ground Water Modeling Center, Holcomb Renearch Institute, Butler Uni- versity, Indianapolis, Ind. van Tassel, D. 1978. Program Style, Design, Efficiency, Debugging, and Testing, 2nd ed. Prentice-Hall, Englewood Cliffs, N.J. Ward, D. S., M. Reeves, and L. E. Duda. 1984. Verification and Field Comparison of the Sandia Waste-Isolation Flow and Transport Model (SWIFT). NUREG/CR-3316, Office of Nuclear Safety and Safeguards, U.S. Nuclear Regulatory Commission, Washington, D.C. Wilkinson, G. F., and G. E. Runkle. 1986. Quality ~surance (QA) Plan for Computer Software Supporting the U.S. Nuclear Regulatory Commission's High-Level Waste Management Program. NUREG/CR-4369, OHice of Nuclear Safety and Safeguards, U.S. Nuclear Regulatory Commission, Washington, D.C.

248 GROUND WATER MODELS Yeh, W. W-G. 1986. Review of parameter identification procedures in ground- water hydrology: The inverse problem. Water Resources Research 22~2), 95-108. Yourdon, E., and L. L. Constantine. 1979. Structured Design: Fundamentals and Systems Design. Prentice-Hall, Englewood Cliffs, N.J.

Next: 7 RESEARCH NEEDS »
Ground Water Models: Scientific and Regulatory Applications Get This Book
×
Buy Paperback | $85.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The discovery of toxic pollution at Love Canal brought ground water contamination to the forefront of public attention. Since then, ground water science and modeling have become increasingly important in evaluating contamination, setting regulations, and resolving liability issues in court.

A clearly written explanation of ground water processes and modeling, Ground Water Models focuses on the practical aspects of model application. It:

  • examines the role of models in regulation, litigation, and policy development;
  • explains ground water processes and describes specific applications for models;
  • presents emerging technologies; and
  • offers specific recommendations for better use of ground water science in policy formation.
  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!