3
Model Development

INTRODUCTION

Models are a simplification of reality that can be compared to maps. Road maps indicate certain aspects of reality (for example, roads of a certain size) and not others (for example, sewer lines, power lines, and buildings). No one map can include all aspects of reality and, similarly, all models, no matter how complex, are constrained by basic assumptions, structure, and uncertainties. Model development involves the definition of model objectives, conceptualization of the problem, translation into a computational model, and model testing, revision, and application. Although almost all model development follows these general steps, models designed for regulatory purposes are subject to constraints in addition to those for models developed strictly for research. This chapter focuses on how model development might best proceed toward regulatory objectives, although there is no one route for successful model development. Our objective is not to provide a treatise on model development. Many other references in an array of disciplines provide comprehensive descriptions of model development for various types of models (Starfield and Bleloch 1991; Clemen 1995; Mesterton-Gibbons 1995; Beck 2002a; Bassetti and Woodward 2005; Ramaswami et al. 2005). This chapter discusses the major steps in regulatory model development focusing on the main lessons learned from previous efforts in EPA and other organizations. It is intended to discuss some of the literature on model development and provide a general framework for EPA as it goes about its business. The wide range of environmental model types makes



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 83
Models in Environmental Regulatory Decision Making 3 Model Development INTRODUCTION Models are a simplification of reality that can be compared to maps. Road maps indicate certain aspects of reality (for example, roads of a certain size) and not others (for example, sewer lines, power lines, and buildings). No one map can include all aspects of reality and, similarly, all models, no matter how complex, are constrained by basic assumptions, structure, and uncertainties. Model development involves the definition of model objectives, conceptualization of the problem, translation into a computational model, and model testing, revision, and application. Although almost all model development follows these general steps, models designed for regulatory purposes are subject to constraints in addition to those for models developed strictly for research. This chapter focuses on how model development might best proceed toward regulatory objectives, although there is no one route for successful model development. Our objective is not to provide a treatise on model development. Many other references in an array of disciplines provide comprehensive descriptions of model development for various types of models (Starfield and Bleloch 1991; Clemen 1995; Mesterton-Gibbons 1995; Beck 2002a; Bassetti and Woodward 2005; Ramaswami et al. 2005). This chapter discusses the major steps in regulatory model development focusing on the main lessons learned from previous efforts in EPA and other organizations. It is intended to discuss some of the literature on model development and provide a general framework for EPA as it goes about its business. The wide range of environmental model types makes

OCR for page 83
Models in Environmental Regulatory Decision Making our effort prone to both overgeneralization and oversimplification. To reduce such difficulties, we will often refer to examples of regulatory model development, especially those from air quality modeling. Box 1-1 in Chapter 1 contains a brief history of EPA’s effort to model tropospheric ozone. ALTERNATIVE MODEL DEVELOPMENT PATHWAYS Some regulatory models arise from those developed as general research tools. Others were developed specifically for addressing regulatory issues. They have been developed by EPA scientists, academia, national laboratories, or the private sector. Some of the most complex models have benefited from contributions by almost all of the above. For example, complex regional chemical transport models for simulating air quality usually include components contributed by multiple parties. The urban airshed model (UAM), heavily used for the design of ozone control strategies in the 1980s and 1990s, was developed by a private company (Systems Applications International) relying on contributions of academia and on support from public and private organizations. The major air quality model developed for use in-house by EPA is the community multiscale air quality (CMAQ) model (EPA 1999b). EPA and NOAA scientists developed the most recent CMAQ model in partnership with a nonprofit organization (Microelectronics Center of North Carolina) and contributions by academia funded by EPA, the National Science Foundation and state authorities, most prominently California authorities (CMAS 2006). A variation of CMAQ, called CMAQ-MADRID, has been developed by a private company (Atmospheric and Environmental Research, Inc) using the CMAQ model as a starting point and adding components developed by academic researchers or by company scientists (Zhang et al. 2004). A private organization, Electric Power Research Institute, provided funding for the CMAQ-MADRID development. All the above codes are in the public domain. Under the Toxic Substances Control Act, EPA must make individual pre-manufacturing decisions on 2,000 new chemicals per year before a new chemical can enter the market. Because of the large number of decisions, the agency has had to rely on screening tools that predict properties from chemical structure. EPA uses EPI (Estimation Programs Interface) Suite, which consists of several quantitative structure-activity relationships (QSARs) models that are available in the public domain.

OCR for page 83
Models in Environmental Regulatory Decision Making However, the set of models began as a few proprietary models developed by Syracuse Research Corporation. Later some of the models were developed in collaboration with EPA, and then all the models were sold to EPA. QSARs are able to take complex chemical structures and predict physical properties, behavior in the environment, and toxicity (Jaworska et al. 2003; Tunkel et al. 2005). The EPI Suite is used by EPA’s Office of Pollution Prevention and Toxics to predict physical-chemical properties, environmental fate and transport, and aquatic toxicity for regulatory decisions on new chemicals when data are not available. The models are also used by industry for pollution prevention and by many government agencies for identifying persistent, bioaccumulative, and toxic (PBT) chemicals (Jaworska et al. 2003). Alhough the development paths of models may be different, many end up having long lives in the regulatory process. Table 3-1 shows the life history of the MOBILE model, which is used to estimate atmospheric emissions from vehicles. This table indicates the periodic revisions that necessarily accompany a model that has been in use for almost 30 years. In the case of MOBILE, such revisions are often major overhauls and updates of the model, resulting in emissions estimates being much different from these in previous versions (NRC 2000; Holmes and Russell 2001). Along with the UAM discussed above, the QUAL2 water quality model is an additional example of a regulatory model that has seen multiple versions and major scientific modifications and extensions in over 2 decades of existence (Barnwell et al. 2004). OVERVIEW OF MODEL DEVELOPMENT Jakeman et al. (2006) separates model development and evaluation into the 10 steps shown in Figure 3-1. The committee agrees with the concept shown in Figure 3-1 that model development is typically an iterative process, especially for long-lived models used over several decades. However, for the purposes of this chapter, the model development process is compressed into the six phases shown in Box 3-1. Documentation occurs at each step of the process, as do certain aspects of evaluation. Chapter 4 describes in detail the evaluation process that occurs throughout the model’s life cycle, compressing the model lifecycle further into 4 steps (problem identification, conceptual

OCR for page 83
Models in Environmental Regulatory Decision Making TABLE 3-1 MOBILE Model RevisionsVersion Version Release Date Model Revisions MOBILE1 1978 Included modeling of exhaust emissions rates as functions of vehicle age and mileage (zero-mile levelsand deterioration rates). MOBILE2 1981 Updated with substantial data (available for the first time) on emission-controlled vehicles (catalyticconverters, model years 1975 and later) at higher ages and mileages. Provided additional model user control of input options. MOBILE3 1984 Updated with substantial new in-use data. Elimination of California vehicle emissions rates (continued to model low- and high-altitudeemissions). Added tampering (rates and associated emissions impacts) and antitampering program benefits. In-use emissions-factor estimates for nonexhaust emissions adjusted for real-world fuel volatility as measured by Reid vapor pressure (RVP). MOBILE4 1989 Updated with new in-use data. Added running losses as distinct emissions source from gasoline-powered vehicles. Modeled fuel volatility (RVP) effects on exhaust emissions rates. Continued expansion of user-controlled options for input data. MOBILE4.1 1991 Updated with new in-use data. Added numerous features allowing user control of more parameters affecting in-use emissions levels, including more inspection and maintenance (I/M) program designs. Included effects of various new emissions standards and related regulatory changes (for example, test procedures). Included impact of oxygenated fuels (for example, gasohol) on CO emissions. MOBILE5 1993 Updated with new in-use data, including basing new basic emissions-rate equations on much larger database derived from state-implemented IM240 test programs. Included effects of new evaporative emissions test procedure (impact on in-use nonexhaust emissions levels).

OCR for page 83
Models in Environmental Regulatory Decision Making MOBILE5 1993 Included effects of reformulated gasoline (RFG). Included effects of new NOx standard of 4.0 g/bhp-hr for heavy-duty engines. Included impact of oxygenated fuels on VOC emissions. Included Tier 1 emissions standards under 1990 Clean Air Act Amendments. Added July 1 evaluation option. Included impact of low-emission vehicle (LEV) programs patterned after California regulations. Revised speed corrections used to model emissions factors over range of traffic speeds. MOBILE5a 1993 Corrected a number of minor errors in MOBILE5. MOBILE5b 1996 Included final on-board vapor-recovery regulations. Included final reformulated gasoline regulations. Added more user options for I/M programs. MOBILE6 2002 Added the effects of Tier 2 and new heavy-duty engine and diesel fuel rules. Updated with new and improved data in many areas, including in-use deterioration of 1981 and newer vehicles, light-duty speed effects, gasoline sulfur effects, and evaporative emissions. Revised I/M benefits algorithm; removed calculation of purge test benefit. Revised algorithms for air conditioning and high acceleration driving. Expanded number of vehicle subclasses from 8 to 28. Added hourly calculation of emissions and emission estimates by roadway type. Separated start and running exhaust emissions. MOBILE6.2 2004 Added ability to model emission factors for particulate matter and six air toxics; added ability tomodel additional air toxics with user-supplied emission factors. Updated carbon monoxide emission factors. Source: EPA 1999c, 2006j.

OCR for page 83
Models in Environmental Regulatory Decision Making FIGURE 3-1 Iterative steps in model development proposed by Jakeman et al. (2006). Source: Jakeman et al. 2006. Reprinted with permission; copyright 2006, Environmental Modelling & Software. model development, computational model development, model use) to make the evaluation process more tractable to the reader. A general issue concerns the uses for which a model is being constructed. This report includes models that are used for two main purposes: those used before regulations are developed to strategically plan and assess priorities and design, evaluate, and propose regulatory approaches (hereafter referred to as pre-regulatory planning models) and those used to implement regulatory programs, including programs that have been delegated to states and local governments (hereafter referred to as post-regulatory implementation and compliance models). These are two quite separate uses, although some models can be used for both purposes as discussed in Chapter 2. However, there might be some important differences. Pre-regulatory planning models require a more general

OCR for page 83
Models in Environmental Regulatory Decision Making BOX 3-1 Basic Steps in Modeling Development Process Model Development Step Modeling Issues Definition of Model Purpose Goal Decisions to be supported Predictions to be made Specification of Modeling Context Scale (spatial and temporal) Application domain User community Required inputs Desired output Evaluation criteria Conceptual Model Formulation Assumptions (dynamic, static, stochastic, deterministic) State variables represented Level of process detail necessary Scientific foundations Computational Model Development Algorithms Mathematical/computational methods Inputs Hardware platforms and software infrastructure User interface Calibration/parameter determination Documentation Model Testing and Revision Theoretical corroboration Model components verification Corroboration (independent data) Sensitivity analysis Uncertainty analysis Robustness determination Comparison to evaluation criteria set during formulation Model Use Analysis of Scenarios Predictions evaluation Regulations assessment Policy analysis and evaluation framework, allowing alternative policy initiatives to be analyzed, perhaps by varying basic model assumptions. This analysis may be done at the national scale and by EPA. Post-regulatory implementation and compli-

OCR for page 83
Models in Environmental Regulatory Decision Making ance models will typically be more closely tied to site-specific observational data, producing a plan for implementing a regulation or assessment of compliance for a given location or substance. Besides being used by EPA, modeling in the post-regulatory process may also be done by state and local governments and their consultants. INTERDEPENDENCE OF MODELS AND DATA FROM MEASUREMENTS Developing and evaluating models typically requires dependence on measurements. In some cases, there are plenty of measurement data for developing model parameters, boundary conditions, and other inputs. Often, however, data are missing, which is an inherent factor in the need for models. Optimally, measurements and models develop iteratively, each informing the other. Box 3-2 describes some examples where measurement data have influenced model development. Although there are trade-offs about whether it is preferable to invest in more data or in better models, the committee does not conclude that the problem of resource allocation for data versus models can be viewed simply as an optimization problem. The difficulty in attempting to formulate such a problem is how to define the optimization criteria and objective function—how does one define and represent the benefit from additional data one does not have relative to investing in additional modeling one does not have. Further, data are typically collected to fulfill multiple objectives, including determining compliance with environmental regulations, further complicating attempts to formulate the data versus model issue into an optimization context. MODEL DEVELOPMENT PHASES Definition of Model Purpose The first step involves defining the major purpose or purposes for which the model is developed. As discussed in Chapter 4, this occurs at the problem identification stage when decision makers, model developers, and other analysts must consider regulatory needs and whether

OCR for page 83
Models in Environmental Regulatory Decision Making BOX 3-2 Interdependence of Models and Data from Measurements Models are developed and evaluated using a wide range of data, theories, and assumptions and are revised in the process. The wisdom of iteration of measurements and modeling is illustrated by three examples. Persistent organic pollutants (POPs) in the Arctic. POPs are chemical substances that persist in the environment, bioaccumulate through the food web, and pose a risk of causing adverse effects to human health and the environment. The first evidence for long-range transport of these substances came about when measurements in animals and the environment of the Arctic revealed the presence of POPs that were never produced there. The lack of reliable emissions data led to a number of modeling efforts used to explore hypotheses regarding the atmospheric transport of and deposition of POPs in the Arctic. For example, Wania and Mackay (1995, 1999) introduced multimedia global distribution models for persistent organic chemicals with a focus on transport and deposition to the Arctic. Then Scheringer (1996, 1997) developed evaluative models to assess global persistence and spatial range as end points in screening level assessments. These models and their results provided key insight both to international agencies, such as the United Nations, and to innovative scientists working independently to measure how POP concentrations vary with latitude. These new measurements provided important feedback that made it possible to develop the next generation of models by merging results from both the first generation of models and the new measurements. Pharmacokinetic modeling. Andersen et al. (2005) describe examples of how integrated measurements and modeling have advanced risk assessment modeling by providing more insight on how intake of chemicals by humans relates to tissue dose and metabolism. Early pharmacokinetic models of the time course of absorption, distribution, metabolism, and excretion of chemicals relied on concepts buttressed by rudimentary information. By the 1950s, data on tissue volume, blood flow, and metabolic pathways were emerging, resulting in early physiologically based pharmacokinetic (PBPK) models. These models, in turn, led to the identification of key input variables (for example, blood flow through various tissues and metabolic parameters), the measurement of which would advance the models. The first use of a PBPK model in a formal risk assessment was for dichloromethane in 1987. Advances were made in assessment methods (for example, EPA’s reference concentration method) as well as in PBPK models of specific chemicals (acrylic acid, vinyl chloride, and dioxin). This iterative process continues today to inform risk assessments that can be used in regulation. It also provides a platform for more novel computational and biological systems approaches of the future (Anderson et al. 2005). Comprehensive Everglades Restoration Plan. The planned restoration of the Florida Everglades is the largest ecosystem restoration effort ever undertaken in terms of it geographical extent and number of individual components. The NRC Committee on Restoration of the Greater Everglades Ecosystem, which was charged with providing scientific advice on this effort, describes the role that modeling and measurements should play in implementing an adaptive approach to restoration (NRC 2003). Under the committee’s vision, monitoring of

OCR for page 83
Models in Environmental Regulatory Decision Making hydrological and ecological performance measures should be integrated with mechanistic modeling and experimentation to better understand how the Everglades functions and how the system will respond to management practices and external stresses. Because the individual components of the restoration plan will be staggered in time, the early components can be used as experiments to provide scientific feedback to guide and refine implementation of later components of the plan. modeling could contribute to the regulatory process. If there is sufficient need for computational modeling, modelers must work with decision makers to define the goal of the model, the decisions it supports, and the groups that might use the model. Addressing these questions is important for setting the direction of the model. As described in Chapter 2, legislative, regulatory, or policy mandates will often drive model development and implementation. For example, a legislative or policy mandate may require that EPA protect the most exposed individual, the most vulnerable individual, or reasonably highly exposed individual and that the agency consider long-term average exposure, the highest one-day exposure, the most exposed subpopulation, or the location of highest concentration. Indeed, legislative mandates may sometimes force development of new models or require major modifications to existing ones. This initial stage sets the direction for the conceptual model and the computational model development. The sidebar from Alice in Wonderland illustrates this message. If you do not know where you want to go, it may appear to others that the direction you take is not particularly important. The key goal of this initial phase is to identify whether modeling would be an effective tool for the problem at hand. Potential uses of an environmental model include the following (Jakeman et al. 2006): Long-term prediction (both extrapolating from the past and answering “what if” questions). Short-term forecasting. Interpolation (estimating variables that have not or cannot be measured directly). Concise summarizing of data. Data assessment (coverage, limitations, inconsistencies, and gaps).

OCR for page 83
Models in Environmental Regulatory Decision Making Control system design (monitoring, diagnosis, decision making, and action taking). Regulatory models are also used to do the following: Help determine compliance with a particular regulation. Evaluate a variety of alternative regulations. Provide a general framework to assess compliance with multiple regulations. Summarize available knowledge needed for regulatory decisions. Insight from Alice’s Adventures in Wonderland by Lewis Carroll Alice speaking to the Cheshire cat: “Would you tell me, please, which way I ought to go from here?” Cheshire cat: “That depends a good deal on where you want to get to.” Alice: I don’t much care where.” Cheshire cat: “Then it doesn’t matter which way you go.” Even if defining the model purpose appears to be a straightforward and easy step, it is often difficult to be clear about the purposes of an environmental model and its application domain. For scientists, the major objective is often to describe the processes dominating the behavior of the system, and for a decision maker, the objective might be to provide clear assessments of policy options. These motives are not mutually exclusive, but neither do they overlap completely. In addition, policy makers may want results for policy variables not directly represented in the model, or they may want results at scales not easily represented by a model. In any case, it is important to establish clearly the purpose and priorities of the specific model. Specification of Modeling Context After determining the purposes of the model, the modeler must develop specifications for the model context. This task involves addressing such questions as

OCR for page 83
Models in Environmental Regulatory Decision Making At what temporal and spatial scales is the model to be applied? This question involves the grain (resolution in time and space) and the extent (spatial and temporal domain) at which the model is to be focused. Who will be the major model users and what constraints does that imply for model application once developed? What is the level of expertise of the proposed users? What type of input data must the model users provide? How can these data be obtained (from other models and measurements)? What sources of data are available to support model evaluation? What are the basic outputs needed and must they be constrained by a deterministic approach or is a probabilistic approach allowable? What additional outputs, although not strictly required, might be useful to enhance model transparency (for example, ability to explain it to various stakeholders and users) and flexibility (for example, capacity for the model to be modified and applied to situations for which it was not constructed)? What level of reliability is required? What evaluation criteria should be applied to determine the applicability of the model or of particular model components? For example, when developing a cancer health assessment of a chemical, considerations include whether to use a linear or a nonlinear model. If the latter is chosen, the model specifications will need to be based on interpretation of the mode of action. A second example is the assessment of human exposure to mobile-source emissions of particulate matter. Here the model developers must work with others to determine whether the objective is to estimate cumulative exposure, time history of exposure, peak exposure, or another measure of exposure. Model developers and others must consider whether they need to consider particle mass, particle number, and particle volume as a metric of exposure. They must also consider the spatial and temporal resolution in the data and parameters that probably will be available for the model. Finally, if the goal is to create linkages to broader health assessments of particulate matter, they must make decisions on whether to consider mobile-source contributions only or mobile sources as a component of all sources of airborne particulate matter. The mismatch between data needed by the model and data available to the model often results in failure of the model exercise, even if the

OCR for page 83
Models in Environmental Regulatory Decision Making model itself may be an accurate representation of the science governing the behavior of the specific system. For example, air quality models require two major types of input: weather fields and emissions inventories. This input can be large (gigabytes of information) and impossible to obtain exclusively from measurements, so meteorological and emissions models estimate the input data and prepare the corresponding input files. Ideally, the input to the meteorological and emissions models would be based on actual measurements (for example, wind speed and direction in specific locations, vertical profiles of atmospheric properties, vehicle activity patterns, and emissions factors). Often, however, these models must use default inputs (for example, those based on emission factors from other parts of the country or even the world without taking into account the local conditions). Improving model inputs with measurements can be costly, and especially for emissions, measurement costs may overwhelm the actual modeling cost. Dealing with the issue of how to obtain the required inputs before developing the computational model and before building bridges with the measurement communities can make a substantial difference in the success of the modeling effort. All the above questions apply to both pre- and post-regulatory models. Some specific questions probably will arise for each model, including the following: For pre-regulatory planning models: What range of plans and scenarios must be considered? What array of impacts is to be included in the assessments of alternatives? For post-regulatory planning and compliance models: Is the required decision a “bright line” compliant/noncompliant one or is a broader view (for example risk of noncompliance) allowable? What constraints are there on computational complexity? Will users insist on rapid assessments from the model (for example, does this need to be available in field situations) that preclude more advanced computational equipment? Conceptual Model Formulation A conceptual model formulates the basic model organization,

OCR for page 83
Models in Environmental Regulatory Decision Making sometimes expressed graphically without the details of individual model components or assumptions. As discussed in Chapter 1, such an abstract representation provides the general structure of a system and the relationships within the system that are known or hypothesized to be important. Figure 3-2 provides an example of a conceptual model for assessing eutrophication. This conceptual model can be viewed as a map summarizing the structure of a model, the inputs, the state variables and outputs, and possibly the domain of applicability. Indeed, one of the critical roles of the conceptual model is to provide a visual description to decision makers, stakeholders, and interested parties of the model, including the fundamental relationships within the model and how inputs lead to outputs. Determination of an appropriate conceptual model relies first upon the problem formulation decisions discussed above as well as the decisions on the following: What basic scientific principles are involved in the model (for example, areas of physics, chemistry, and biology that need to be considered on the basis of the objectives)? Is there agreement about these principles or does their inclusion potentially result in controversy (in which case, allowing for alternative assumptions might be necessary)? Is an appropriate model formulation already extant? What level of aggregation is appropriate to the model objectives? This question applies to the scales for the model (for example, spatial and temporal averaging may be needed) and the structure of model components (for example, including demographic structure or putting all ages into a single class). What are the variables for the model (for example, what will it explicitly track in characterizing the system) and how are they related (often characterized by a box and arrow diagram, flow chart, influence diagram, or similar graphic)? What are the means by which the variables will be expressed? Are they deterministic (discrete, continuous, nominal), stochastic (discrete, continuous, nominal), and spatially or temporally dependent or static? What level of mechanistic detail is needed (for example, processes operating at what level should be included: cell, tissue, individual organism, population, and so forth)? Is a purely empirical approach (for example, a data-driven model, including many statistical ones) appropriate? Is a mixture of these necessary?

OCR for page 83
Models in Environmental Regulatory Decision Making FIGURE 3-2 Conceptual model for assessing eutrophication in the European seas linking nutrient enrichments and its direct and indirect effects in the ecosystem. Source: EC 2004. Reprinted with permission; copyright 2004, the European Commission’s Joint Research Centre. What are the model inputs and the scales at which the inputs will be provided? There are distinct trade-offs in model development that should be addressed at the time of the conceptual model formulation. No one model can do everything. Development of a more comprehensive model will not necessarily resolve or even reduce all uncertainties in understanding and in predicting how a system will react. It is at this stage of model development that constraints, assumptions, and acceptability criteria should be established. Given financial or effort limitations, it is appropriate to set “stopping” criteria for when to decide that a model is sufficiently useful to be applied, even while acknowledging its limitations.

OCR for page 83
Models in Environmental Regulatory Decision Making All the above trade-offs apply to both pre- and post-regulatory models. Some specific questions probably will arise for each model, including the following: For pre-regulatory planning models: How are alternative plans formulated or specified? Do they arise from modifications of a single plan (say, by varying constraints on structures allowable to be built) or are they chosen from a broad array of options? What metrics are applied to compare and contrast alternative plans? For post-regulatory implementation and compliance models: What criteria determine compliance versus noncompliance and how do they relate to the model state variables? What level of detail is needed in applications involving regulatory implementation? Computational Model Development This stage requires formulating the model explicitly by translating the model assumptions from the previous step into a mathematical formulation, by determining the detailed structure of the model, and by encoding the resulting model. This stage requires decisions about the following: Model equations that determine the relationships between variables (rules, statements, equations, statistics) and account for the mathematical structure of the model (for example, static, dynamic, discrete, continuous). Parameter estimations (from either data or underlying scientific assumptions) to determine input model parameters or distribution of such parameters in the case of a stochastic formulation. Appropriate software design and engineering tools to encode and/or solve the model, appropriate computational algorithms, and appropriate model interface to ensure applicability by the user community. Methods for analysis of model results, including graphic outputs and the capability to conduct sensitivity and uncertainty analysis.

OCR for page 83
Models in Environmental Regulatory Decision Making Flexibility to modify model structure and inputs in the future as new data arise, alternative objectives are specified, or different regulations are assessed. Documentation to allow for transparency of the model based on the needs of the user community and the potential for future modification. Such documentation maintains the history of major revisions of the model. One critical issue is whether to revise an existing model or to develop a new one. “Model recycling” can save a huge development effort by applying a tested model to a purpose that is different from its original one. Furthermore, modelers often face the difficult decision between the development of one model that describes everything (the “swiss army knife” of models) and can be used for a variety of purposes and the development of multiple smaller models that have a common core but are developed separately for different purposes. For example, the main purpose of the CMAQ model is to simulate concentrations of fine particulate matter and ozone in the lower atmosphere and to assist the analysis of the corresponding regulations. As a complex model, it describes the concentrations of more than 100 air pollutants in space and time. It has become a family of models (for example, CMAQ-MADRID and CMAQ-Hg) addressing a range of different air quality problems, including visibility reduction and acid deposition. Given that the different versions of CMAQ take advantage of the core of the model (atmospheric transport, gas-phase chemistry, and so forth) without violating any of the major assumptions of CMAQ, the strategy is a good one. For example, CMAQ has been extended (after some nontrivial modifications in its code) to address mercury (CMAQ-Hg). Although there are many gaps in our scientific understanding of the corresponding problem, CMAQ is an appropriate platform for such an extension. On the other hand, the model would require major redevelopment to address potential regulations of ultrafine particles (diameter less than 100 nanometers [nm]) due to numerical issues with its description of the particle-size distribution. Thus, there are limitations to the degree that CMAQ can be adapted. Another example is the MOBILE model (Table 3-1), which has evolved from a tool for estimating regional motor-vehicle emissions inventories to a model used for estimating emissions on individual highway segments where instantaneous operating conditions of individual vehicles may be critical but that are not represented in the

OCR for page 83
Models in Environmental Regulatory Decision Making model. As concluded by the NRC (2000), the farther MOBILE’s applications deviate from its original purpose of estimating aggregate regional emissions, the more difficult it becomes to verify the accuracy of its predictions. Because of the difficulty in developing a single motor-vehicle emissions model appropriate for all applications, the NRC (2000) recommended that EPA develop a toolkit of models based on a consistent data set and model interface. Such a toolkit would include an aggregated regional emissions component, a smaller scale model for simulating emissions along major highway corridors, and a microscale instantaneous emissions-modeling component for more transient and localized traffic conditions. For a toolkit approach, the type of motor-vehicle emissions model applied could better meet the characteristics of the problem while being consistent from one problem scale to another. Tierney (2004) described how EPA’s new mobile-source emissions model, known as the MOVES model, will address this and other issues raised by the NRC. Additional considerations probably will arise for pre- and post-regulatory models. For pre-regulatory planning models: Provide methods to compare and contrast the implications of alternative plans, perhaps requiring the capability for exploratory analysis by the users of model outputs. Provide methods either to vary the constraints on plans-scenarios or to vary the metrics to evaluate each plan. Provide automated optimization methods to specify the highest ranked plan from a given set based on chosen criteria. For post-regulatory implementation and compliance models: Provide methods to modify inputs to determine how readily a noncompliant case might become a compliant one and vice versa. Provide methods to ascertain the impacts of additional data on model results and assist users in determining the most effective methods to obtain such data (for example, methods to choose optimal locations for new data collection). Understand whether models used for implementing regulation would be used widely by state and local governments and consulting firms that help those entities to develop implementation plans.

OCR for page 83
Models in Environmental Regulatory Decision Making Modular Approaches for Environmental Model Development The code of environmental models often can be written in a modular form. A module is an independent piece of code that forms a part of one or more models. Often, each module describes one process. For example, CMAQ includes modules for the description of horizontal and vertical advection, horizontal and vertical dispersion, gas-phase chemistry, aqueous-phase chemistry, aerosol thermodynamics and dynamics, plume chemistry effects, dry and wet deposition, and process analysis. This modular approach facilitates testing of the model (one can test the individual pieces separately) and reuse of the relevant modules in separate modeling projects. The parts of the model can be replaced with others without changing the overall structure of the model. There are also choices of modules for the same task. CMAQ allows its user to choose among three gas-phase chemistry mechanisms, depending on the specifics of the problem being modeled. The modular approach to CMAQ allows the level of complexity in the application to be aligned with the needs of the regulatory decisions. For example, the use of the mercury chemistry simulation capability of CMAQ is not necessary for ozone or particulate matter applications. A major advantage of the modular model development approach is the ability to easily add or remove parts of the model, thus creating models of different complexity. For example, the full range of available modules (describing all potentially relevant processes) can be used, and then after quantifying the importance of each one of them for the specific application, the model can be simplified and used by removing the parts that have little or no effect on the results. For example, one can remove the cloud chemistry module from an application focusing on ozone episodes. The rest of the analysis can be done with the simplified model. Therefore, the modular approach and the resulting models of different degrees of complexity allow the user to satisfy the scientific requirements about quantifying the influence of the different processes and to avoid unnecessary complexity in the model used for the regulation. This approach allows modelers to defend their choices of excluding parts of the system from the analysis by allowing modelers to demonstrate the impacts of including or excluding various processes from the model. This approach also allows models to be updated more easily.

OCR for page 83
Models in Environmental Regulatory Decision Making RECOMMENDATIONS The committee offers several recommendations based on the discussion in this chapter. They deal with the interdependence of models and measurements, the model extrapolation, and the need for model parsimony. The Interdependence of Models and Measurements The interdependence of models and measurements is complex and iterative for several reasons. Measurements help to provide the conceptual basis of a model and inform model development, including parameter estimation. Measurements are also a critical tool for corroborating model results. Once developed, models can drive priorities for measurements that ultimately get used in modifying existing models or in developing new ones. Measurement and model activities are often conducted in isolation. For example, modelers often add details to models without sufficient measurements to justify or confirm the importance of these changes. Likewise, field and laboratory scientists might expand their compilation of samples without understanding the utility of such information for modeling. Although environmental data systems serve a range of purposes, including compliance assessment, monitoring of trends in indicators, and basic research performance, the importance of models in the regulatory process requires measurements and models to be better integrated. Adaptive strategies that rely on iterations of measurements and modeling, such as those discussed in the 2003 NRC report titled Adaptive Monitoring and Assessment for the Comprehensive Everglades Restoration Plan, provide examples of how improved coordination might be achieved. Recommendations Using adaptive strategies to coordinate data collection and modeling should be a priority of decision makers and those responsible for regulatory model development and application. The interdependence of measurements and modeling needs to be fully considered as early as the conceptual model development phase. Developing adaptive strategies

OCR for page 83
Models in Environmental Regulatory Decision Making will benefit from the contributions of modelers, measurement experts, decision makers, and resource managers. Model Parsimony Models are always incomplete, and efforts to make them more complete can be problematic. As features and capabilities are added to a model, the cumulative effect on model performance needs to be evaluated carefully. Increasing the complexity of models without adequate consideration can introduce more model parameters with uncertain values, and decrease the potential for a model to be transparent and accessible to users and reviewers. It is often preferable to omit capabilities that do not improve model performance substantially. Even more problematic are models that accrue substantial uncertainties because they contain more parameters than can be estimated or calibrated with available observations. Recommendations Models used in the regulatory process should be no more complicated than is necessary to inform regulatory decisions. In the process of evaluating whether a model is suitable for its given application, there should be a critical evaluation of whether the model has been made unreasonably complicated. This evaluation should include how model developers and those that select a model for a particular application have addressed the trade-offs between the need for a given model application to be an accurate representation of the system of interest and the need for it to be reproducible, transparent, and useful for the regulatory decision at hand.