National Academies Press: OpenBook
« Previous: 3 The Life Histories of the Fishes
Suggested Citation:"4 Use of Models." National Research Council. 2010. A Scientific Assessment of Alternatives for Reducing Water Management Effects on Threatened and Endangered Fishes in California's Bay-Delta. Washington, DC: The National Academies Press. doi: 10.17226/12881.
×

4
Use of Models

MODELING SCENARIOS

Modeling of baselines and future project actions is a standard practice of evaluating impacts. Both biological opinions relied on the use of modeling scenarios (known as Studies) provided by the Operations Criteria and Plan (OCAP) biological assessment (BA) (http://www.usbr.gov/mp/cvo/ocap_page.html), although the extent to which such results were used in each biological opinion and in the formulation of RPAs varied significantly. The “proposed action” with reference to ESA is the continued operation of the CVP and SWP with additional operational and structural changes (USBR, 2008, Table 2-1) to the system. The U.S. Bureau of Reclamation (USBR) and the California Department of Water Resources (DWR) provided the results of the modeling conducted for simulating baseline conditions, future system components, operational strategies, and the water supply demands. In addition to simulating the water-supply deliveries of the project, the modeling also attempted to mimic the project operations associated with the regulatory environments described in operating criteria described in D-1485, D-1641,CVPIA Section 3406 (b)(2) and the Environmental Water Account (EWA) (USBR, 2008). A major difference in the current and future scenarios is the extent to which EWA is used. The purpose of EWA was to enable diversion of water by the SWP and CVP from the delta to be reduced at times to benefit fish species while minimizing uncompensated loss of water to SWP and CVP contractors (USBR, 2008, Chapter 2). The EWA is intended to replace the water loss due to pumping curtailments by purchasing surface water and groundwater from willing sellers and through increasing the flexibility of operations. The simulations include both a “full EWA” characterizing the full use of EWA assets as well as a “limited EWA” focusing only on a limited number of assets. The EWA is currently under review to determine its future (FWS, 2008, p. 34) and the RPA actions were not based on it.

Another factor that changed from current to future conditions is the way water demand by CVP/SWP users is simulated. Demands have been pre-

Suggested Citation:"4 Use of Models." National Research Council. 2010. A Scientific Assessment of Alternatives for Reducing Water Management Effects on Threatened and Endangered Fishes in California's Bay-Delta. Washington, DC: The National Academies Press. doi: 10.17226/12881.
×

processed using either contractual amounts and/or level of development (existing versus future). Some demands were assumed to be fixed at contractual amounts whereas in other cases they varied according to the hydrologic conditions. This topic will be considered in the committee’s second report.

While several study scenarios were developed for the OCAP biological assessment (USBR, 2008), the use of modeling results in the biological opinions was largely limited to a smaller set of scenarios (Table 4-1).

Study 7.0 describes the existing condition (circa 2005), whereas Study 7.1 presents the existing condition demands with near future facilities as well as the projected modification to EWA. Study 8 describes the future condition corresponding to the year 2030 (USBR, 2008, pp. 9-33, 9-53, 9-54). Study series 9 constitutes a future condition representing modified hydrology (warm and warmer, dry and wet) along with a projected sea level rise of one foot.

CENTRAL ISSUES CONCERNING MODEL USE IN THE BIOLOGICAL OPINIONS

The USFWS and NMFS supplemented the modeling results provided by USBR and DWR with their own modeling efforts and available science on the implications of management actions on species. The primary suite of models provided to FWS and NMFS include (USBR, 2008, Chapter 9):

  1. Operations and hydrodynamic models: CalSim-II, CalLite, the Delta Simulation Model II (DSM2), including particle-tracking models (PTMs, which also are considered as surrogates for biological models)

TABLE 4-1 Key scenarios used for biological opinions of FWS and NMFS

Study

Level of Development (Year)

Environmental Water Account (EWA)

Future Project Facilities1

Climate and Sea Level Rise

7.0

2005

Full EWA

No

No

7.1

2005

Limited EWA

Yes

No

8.0

2030

Limited EWA

Yes

No

9.0-9.5

2030

Same as in Study 8.02

Yes

Yes

1 Future project features include South Delta Improvement Program (Stage 1), Freeport Regional Water Project, California Aqueduct and Delta-Mendota Canal intertie

2 According to the OCAP BA (USBR, 2008), Study suite 9 is identical to Study 8.0 except for climate change and sea-level rise

Suggested Citation:"4 Use of Models." National Research Council. 2010. A Scientific Assessment of Alternatives for Reducing Water Management Effects on Threatened and Endangered Fishes in California's Bay-Delta. Washington, DC: The National Academies Press. doi: 10.17226/12881.
×
  1. Temperature models: Reclamation Temperature, SRWQM, and Feather River Mode

  2. Biological models: Reclamation Mortality, and SALMOD

The modeling framework used by the agencies is diagrammed in Figure 4-1.

The USFWS, in its biological opinion, used available results from a combination of tools and data sources, including CalSim-II, DSM2-PTM, DAYFLOW historical flows, and statistical models based on observational data and particle-tracking simulations (FWS, 2008, p. 204). NMFS analyses included results from coupled CalSim-II simulations with various water-quality and biological models for a few of the life stages (NMFS, 2009, p. 64).

The CalSim-II model, the primary tool used to evaluate the water-resources implication of the proposed actions, was developed by the DWR and the USBR to simulate water storage and supply, streamflows, and delta export capability for the Central Valley Project (CVP) and the State Water Project (SWP). CalSim-II simulates water deliveries and the regulatory environment associated with the water-resources system north of the delta and south of the delta using a

FIGURE 4-1 Modeling framework used in NMFS and USFWS biological opinions and RPAs.

FIGURE 4-1 Modeling framework used in NMFS and USFWS biological opinions and RPAs.

Suggested Citation:"4 Use of Models." National Research Council. 2010. A Scientific Assessment of Alternatives for Reducing Water Management Effects on Threatened and Endangered Fishes in California's Bay-Delta. Washington, DC: The National Academies Press. doi: 10.17226/12881.
×

single time step (one month) optimization procedure based on a linear programming algorithm. CalSim-II represents the best available planning model for the CVP-SWP system, according to a CALFED Science Program peer review by Close et al. (2003) (USBR, 2008, p. 9-4). However, many users have suggested that its primary limitation is its monthly time step, and the model should be used primarily for comparative analysis between scenarios and discouraged its use for absolute predictions (Ferreira et al., 2005; USBR, 2008, Chapter 9). In response to the peer review by Close et al. (2003), DWR and USBR provided a list of development priorities (Table 2, DWR/USBR, 2004), including the use of a daily time step, but it is not clear how many of such planned improvements have been incorporated into the version of CalSim-II used in the biological opinions.

Several other tools and models were central in effects analysis and developing RPAs, including hydrodynamic and water-quality (DSM2, USBR’s temperature, SRWQM), habitat (SALMOD), and statistical and particle-tracking models (salvage, DSM2-PTM). Some of these models have already been evaluated in the literature for their individual strengths and limitations, though some (SALMOD and USBR’s mortality models) have not yet been formally peer reviewed. We first review some of the challenges of applying these individual models in the determination of RPAs, and then focus on examining the modeling process, including how the models contributed to the development of RPAs, and where the uncertainties and vulnerabilities in that process lie.

Model Scale and Management Implications

Very generally, the tiered modeling approach (Figure 4-1) applied the results of CalSim-II as input to various hydrodynamic and ecological models to predict impacts of project operations and, to a very limited extent, to explore RPAs. At one level, model simulations were also used or performed to investigate the feasibility of some proposed actions. For example, CalSim-II was used at the planning level to investigate whether the USBR could meet the 1.9 MAF (at the end of September) required by actions I.2.3 and I.2.4 (maintaining cold water supplies necessary for egg incubation for the following summer’s cohort of winter-run), and to recommend storage conservation in severe and extended droughts (NMFS, 2009, p. 596). Similarly, examination of CalSim results and hydrologic records demonstrated to the agencies that the first year of a drought sequence is particularly critical to storage and operations in the following drought year (NMFS, 2009, p. 596). The benefits of using models at this planning level, especially given the importance of water-year types, is clear, and there is little controversy about this application of the models.

Suggested Citation:"4 Use of Models." National Research Council. 2010. A Scientific Assessment of Alternatives for Reducing Water Management Effects on Threatened and Endangered Fishes in California's Bay-Delta. Washington, DC: The National Academies Press. doi: 10.17226/12881.
×

At another level, model scenarios were examined to investigate the relationships between operations and impacts on various life stages of the fish across the water-year types and operations scenarios. For example, NMFS used DWR’s Delta Survival Model (Greene, 2008) to estimate mortality of smolts associated with three CalSim-II Study scenarios (7.0, 7.1, 8.0). The USFWS used statistical models of salvage and total entrainment (Grimaldo et al., 2009; Kimmerer, 2008) to investigate the effects of proposed operations by comparing actual and predicted salvage and entrainment losses under modeled OMR flows (FWS, 2008, p. 211).

While some challenges exist in linking models in this tiered approach (see next section), concerns and controversies appear to be largely directed at the various forms of statistical relationships of salvage versus OMR flows, extrapolation of these relationships that describe impacts on single life stages to assess the population impacts on species, and the use of biological models without full consideration of their underlying uncertainties. In particular, this nested sequence of statistical models does not allow for uncertainties at one step to influence predictions at the next step. As a result, some of the RPA actions, especially those involving X2 and OMR flow triggers, are based on less reliable scientific and modeling foundations than others. In these cases, the incomplete data and resolution of the models do not closely match the resolution of the actions.

Adequacy of Current Models

Life-cycle Models

Both agencies have been criticized for the lack of adequate life-cycle models to address population level responses (e.g., Deriso, 2009; Hilborn, 2009; Manly, 2009). Nonlinear and compensatory relationships between different life-history stages are common in many fish species. Moreover, many life-history traits exhibit significant patterns of autocorrelation, such that changes in one life-history trait induce or cause related changes in others. These patterns can most effectively be understood through integrated analyses conducted in a modeling framework that represents the complete life cycle. However, complete life-cycle models were not used in either biological opinion to evaluate the effects of changes in operations. The agencies acknowledge that further model development is required, including the “cooperative development of a salmonid life-cycle model acceptable to NMFS, Reclamation (USBR), CDFG, and DWR” (NMSF biological opinion, p. 584). While one life-cycle model (Interactive Ob-

Suggested Citation:"4 Use of Models." National Research Council. 2010. A Scientific Assessment of Alternatives for Reducing Water Management Effects on Threatened and Endangered Fishes in California's Bay-Delta. Washington, DC: The National Academies Press. doi: 10.17226/12881.
×

ject-Oriented Salmon simulation) was available for winter-run salmon from the OCAP BA (USBR, 2008), this model was rejected based on model resolution and data limitation issues (NMFS, 2009, p. 65). Similarly, a better life-cycle model for delta smelt is critically needed (PBS&J, 2008). Such life-cycle models for delta smelt are currently under development. The committee recommends that development of such models be given a high priority within the agencies. The committee also encourages the agencies to develop several different modeling approaches to enable the results of models with different structure and assumptions to be compared. When multiple models agree, the confidence in their predictions is increased.

Particle-Tracking Models (PTMs)

Particle-tracking models (PTMs) are models that treat eggs and larval fishes as if they were particles and simulate their movements based on hydraulic models of flows. Criticisms have applied to the use of PTMs, which rely on some key assumptions (e.g., neutral buoyancy, no active swimming) that have been challenged at least for some life stages (Kimmerer and Nobriga, 2008) on the basis that fish live and move in three dimensions. Other limitations of the use of PTMs in this case include the reliance on the one-dimensional DSM2, use of random-walks to simulate lateral movements, and the lack of simulation of fish behavior. In view of these limitations, PTMs as used in this case may not be suitable for predicting the movement of fish of some life stages (juvenile and adults) where behavior becomes relevant to the question of potential entrainment (Kimmerer and Nobriga, 2008). The NMFS acknowledges these limitations, noting that “The acoustic tagging studies also indicate that fish behavior is complex, with fish exhibiting behavior that is not captured by the ‘tidal surfing’ model utilized as one of the options in the PTM simulations. Fish made their way downstream in a way that was more complicated than simply riding the tide, and no discernable phase of the tide had greater net downstream movement than another” (NMFS, 2009, p. 651).

However, while fish seldom behave like passive particles, results based on passive particles can provide insights. For example, the NMFS used a combination of models to simulate mortality rates of salmonids for three CalSim-II scenarios. The results were used to compare the inter- and intra-annual impacts of the three scenarios (NMFS, 2009, p. 381). Further, the agencies advocate improving the model through further study, such as Action iV.2.2, which includes an acoustic tag experiment in part to evaluate action benefits and in part to improve PTM results (USBR, 2008, p. 645). Thus, while there is uncertainty re-

Suggested Citation:"4 Use of Models." National Research Council. 2010. A Scientific Assessment of Alternatives for Reducing Water Management Effects on Threatened and Endangered Fishes in California's Bay-Delta. Washington, DC: The National Academies Press. doi: 10.17226/12881.
×

garding the accuracy of the mortality losses, the use of the models in a comparative way is probably acceptable. However, it should be made clear how the model is used, and the explicit consideration of the PTM assumptions and uncertainties should be more clearly documented in the biological opinions.

Although there has not been an assessment of the degree to which these limitations affect the conclusions, PTM results were used for RPA development. Although the DSM2 has been calibrated adequately for OMR flows, there is no clear evidence concerning the accuracy of the PTM’s ability to simulate smelt entrainment in relation to how the models are used for jeopardy determination and RPA development. This is particularly important because a number of actions driven by the RPAs recommend trigger values for OMR to curtail exports. As discussed in a later section, the science surrounding these OMR triggers is less clear than for many other aspects of the RPAs, and this trigger may result in significant water requirements. The committee’s recommendations for improving the modeling and associated science are intended to improve the best science available to the agencies. The committee will address such improvements in greater detail in its second report.

Other Biological Models

The NMFS used other biological models to simulate the effects of operations on various life stages of salmon. These models involve several key assumptions and data limitations that influence the reliability of their results.

For example, SALMOD, developed by the USGS, was used by the NMFS to investigate the population level responses of the freshwater life stages to habitat changes caused by project operations (NMFS, 2009, p. 269). A variety of weekly averaged inputs are required, including streamflow, water temperature, and number and distribution of adult spawners (USBR, 2008, p. 9-25). This model provides some valuable insight, but requires greater consideration of the model assumptions (e.g., linear stream, habitat as primary limiting factor, independence of food resources on flow and temperature, density independence for some life stages) and uncertainties. Otherwise, the use of this model is limited to comparative, rather than absolute, analysis of RPA actions. Further, it would be important to investigate the sensitivity of the model to initial conditions and input data, particularly those prone to measurement error (e.g., number and distribution of spawners) to provide some indication of the reliability of model outputs. While SALMOD has not been thoroughly peer-reviewed, criticisms of similar modeling approaches (e.g., NRC, 2008) have highlighted some key issues with habitat-suitability models (e.g., the need for greater clarity concerning

Suggested Citation:"4 Use of Models." National Research Council. 2010. A Scientific Assessment of Alternatives for Reducing Water Management Effects on Threatened and Endangered Fishes in California's Bay-Delta. Washington, DC: The National Academies Press. doi: 10.17226/12881.
×

the assumption that habitat is a limiting factor and the need for a thorough assessment of the representativeness of the areas sampled) and have provided extensive discussions of the use of models in an adaptive-management approach, which is relevant to this committee’s recommendations. Finally, the NMFS acknowledges that SALMOD is most appropriately applied to large populations that are not sensitive to individual variability and environmental stochasticity (NMFS, 2009, p. 270), which means that the predictions for the relatively small population in the delta river system are subject to considerable uncertainty. The uncertainties again highlight the need for an adaptive management approach.

The NMFS also used results from the USBR’s salmon mortality model (Hydrologic Consultants, Inc., 1996) to examine daily salmon spawning losses for early life stages (pre-spawned eggs, fertilized eggs, and pre-emergent fry) due to exposure of high temperatures. Temperature-exposure mortality criteria for the three life stages are combined with modeled temperature predictions and spawning distribution data to compute percents of salmon spawning losses. Because simulations of river temperatures are run on a daily or shorter time step, downscaling of monthly CalSim-II data is required (USBR, 2008, Attachment H-1). Moreover, the monthly temperature models do not adequately capture the range of daily temperature variability (USBR, 2008, pp. 9-109). In addition, several assumptions (e.g., density independence) and important data limitations (USBR, 2008, pp. L-6, L-7) challenge the reliability of this model. Finally, while this model has been applied in other systems, it is not thoroughly peer reviewed and no analysis of sensitivity or uncertainty has been performed. Addressing these model shortcomings would help increase confidence in the analyses.

Developing, Evaluating, and Applying Best Available Models

As the agencies work within the constraints of best available science, some recognition of the adequacy and reliability of the models should be reflected in the management decisions by making them adaptive. The following five factors, in particular, need better documentation.

1.
Incompatible temporal resolution and implications for management decisions.

The individual models used in this tiered analysis approach have a broad range of temporal resolutions (Figure 4-1). Care must be exercised in such

Suggested Citation:"4 Use of Models." National Research Council. 2010. A Scientific Assessment of Alternatives for Reducing Water Management Effects on Threatened and Endangered Fishes in California's Bay-Delta. Washington, DC: The National Academies Press. doi: 10.17226/12881.
×

situations so that the linkages of models with different temporal and spatial resolutions do not result in propagation of large errors that may influence decisions derived from the modeling results. For example, CalSim-II uses a monthly time step whereas the DSM2 uses a 15-minute time step. Although the tidal boundary condition in DSM2 is pre-processed at 15-minutes, average monthly flow, simulated by CalSim-II, is provided as the upstream flow boundary condition at many delta inflow points. The linkage of CalSim-II and DSM2 attempts to smooth out the step change in monthly simulated flows (USBR, 2008, pp. 9-14, 9-15), but this is not necessarily adequate to simulate the fluctuations of flows within the month. The use of the monthly time step certainly could have a significant influence on such performance measures as OMR flows, particularly when such flows are recommended in RPAs for triggering export curtailments. USFWS and NMFS should provide a comparison of daily versus monthly average simulations of DSM2 for a historical period to ascertain the reliability of using monthly CalSim output as input to DSM2.

The incompatibility of temporal resolutions is particularly important given that flows in the delta are strongly influenced by tides. The flows at such locations as Old River and Middle River are characterized by two flood-ebb cycles per day, with positive and negative values of much larger magnitude than the average net flow at these locations (Gartrell, 2010). In view of the fact that OMR flows have sub-hourly hydrodynamic components, averaging over a longer period such as 5 to 14 days to define the thresholds in the implementation of the RPAs could produce unnecessary changes in water exports. The use of monthly average flows produced by CalSim-II could further add to the concerns regarding the recommended thresholds of OMR flows. In view of these modeling uncertainties, further clarification as to how the modeled OMR flows were used for jeopardy determination and hence for the development and implementation of RPAs is needed.

2.
Inconsistent use of baselines.

Both biological opinions use historical data along with modeling results of the CALSIM-II scenarios. Study 7.0, which represents the existing condition, is expected to be closest to historical conditions. However, important differences between the two (historical and existing conditions) could exist due to differences in demands and more importantly due to deviations in operations. Because of the simplifying assumptions used in CalSim-II historical simulations, the FWS BO opted to use actual historical data to develop their baseline (FWS, 2008, p. 206) and continued to compare historical data with the modeling results

Suggested Citation:"4 Use of Models." National Research Council. 2010. A Scientific Assessment of Alternatives for Reducing Water Management Effects on Threatened and Endangered Fishes in California's Bay-Delta. Washington, DC: The National Academies Press. doi: 10.17226/12881.
×

of the numerous scenarios described above (see, for example, Figures E-3 through E-19).

The results suggest that often, actual data are very different in magnitude in comparison to Study 7.0 and furthermore, most scenarios (Studies 7, 7.1, 8, and study series 9) are clumped together with relatively small differences between them in relation to the magnitude of differences with the historical data. In view of these differences, the validation of Study 7.0 and consequently others, becomes even more important for the purpose of RPA development.

The use of historical data to make inferences is very typical and appropriate in the biological opinions. However, since the evaluation of project actions and the development of RPAs are based on the evaluation of modeling scenarios, which appear to greatly differ from historical data, a comparison of the two sets of data (historical and simulated) may incur errors in interpretation. The committee recommends that the biological opinions provide a better justification for the reasonableness of the baseline scenario, Study 7.0, as well as the comparison of scenario results with historical data.

3.
Challenges in calibrating and validating any of the models to historical observations and operations.

It is a standard practice to ensure the appropriate use of models through the processes of calibration and testing (ASTM, 2004; NRC 2008). Validation of CalSim-II is described in Appendix U of the OCAP BA (USBR, 2008), which provides a comparison of Study 7.0 (existing condition) with the recent historical data. A review of those results shows that there are significant deviations of the historical data from the simulated storages and exports that may be of the same magnitude as the differences between the scenarios being evaluated. Thus, while the tool itself performs well, some questions remain regarding the gross nature of generalized rules used in CalSim-II to operate CVP and SWP systems, relative to actual variability of dynamic operations (USBR, 2008, pages 9-4). In their peer review of the CalSim-II model, Close et al. (2003) suggested that “Given present and anticipated uses of CalSim-II, the model should be calibrated, tested, and documented for “absolute” or non-comparative uses.” It is not clear if the agencies that developed the model have responded to this suggestion in a comprehensive manner. As emphasized above, a clear presentation of the realism of Study 7.0 with respect to recent operations or observations would help avoid the criticism as to the results of Study 7.0 as well as other derivatives of it (Studies 7.1, 8.0 and series 9).

Suggested Citation:"4 Use of Models." National Research Council. 2010. A Scientific Assessment of Alternatives for Reducing Water Management Effects on Threatened and Endangered Fishes in California's Bay-Delta. Washington, DC: The National Academies Press. doi: 10.17226/12881.
×

The OCAP BA (USBR, 2008) provides sufficient information on the calibration and testing of temperature models, and the time steps vary among models, although all used the monthly output of CalSim-II in predictions. Thus, they appear to be adequate for predicting temperature variation and making comparisons at the monthly time scale. Information on the calibration of DSM2 and PTM is provided in part by DWR, which has been posted online (http://modeling.water.ca.gov/delta/studies/validation2000/) results of the calibration of this 1-D, hydrodynamic model of the delta. Based on the information provided, it appears to adequately mimic the historical data at a daily time-scale. However, the DSM2 simulations should demonstrate that the range of negative OMR flows used for calibration covers the high negative flows simulated by CalSim-II for future scenarios. There has been an attempt to test PTM (Wilbur, 2001), but clearly this tool needs further improvements. Wilbur (2001) reports that the existing velocity profiles used in PTM consistently over-predict the field observations (i.e., the predicted velocities exceed the observed velocities).

In addition, with the potential for changes in the historical patterns of climate and hydrology, calibrating models with historical data alone may be less meaningful for projection of future operations. Thus, in addition to providing support for model improvement and adaptive management, a more robust monitoring program will also support calibration and testing of models with more relevant representation of the current and future system. For example, drought-induced low flows of the past several years provide opportunities to calibrate and test models under infrequent but foreseeable conditions. Realistic modeling of the system that incorporates what actually happens in an operational setting with climate outlook will be important in the future.

The biological models such as USBR’s mortality model and SALMOD are essentially uncalibrated for the system, and further concerns about these models were addressed in previous sections.

4.
Challenges of the Tiered Modeling Approach.

Temperature, OMR flows, and X2 performance measures are particularly challenged by the tiered modeling approach, with limitations related to data availability and inconsistency in model resolution (spatial and temporal) and complexity (USBR, 2008, pp. 9-31). However, the use of models may still be beneficial in planning and triggering adaptive management needs. For example, for NMFS implementation of Action II.2 (Lower American River Temperature Management), forecasts will be used to simulate operations and compliance with thermal criteria for specific life stages in months when salmon would be present

Suggested Citation:"4 Use of Models." National Research Council. 2010. A Scientific Assessment of Alternatives for Reducing Water Management Effects on Threatened and Endangered Fishes in California's Bay-Delta. Washington, DC: The National Academies Press. doi: 10.17226/12881.
×

(NMFS, 2009, p. 614). However, if the USBR determines that it cannot meet the temperature requirement, and can demonstrate this through modeling of allocations and delivery schedules, consultation with the NMFS will occur. In this example, modeling results are used to evaluate the feasibility of meeting criteria, rather than trying to derive direct loss estimates. The RPA then leads to a process for adaptive management of the temperature operations based on updates to the hydrologic information. Thus, despite the particularly challenging example of managing temperature, the use of models appears to have allowed for flexibility.

However, no qualitative or quantitative analysis of the magnitude of errors across these model linkages and the resulting uncertainties are presented. While not required for the justification of RPAs, failing to consider error propagation across the models makes it difficult to evaluate the reliability of meeting the RPAs and their ability to provide the intended benefits.

5.
Lack of an integrative analysis of RPAs

Numerous RPA actions proposed in both biological opinions cover new projects as well as operational changes. However, the information provided to the committee did not include a comprehensive analysis of all RPA actions, either individually or, more important, jointly, with respect to their ability to reduce the risks to the fish or to estimate system-wide water requirements. Clearly, the agencies lacked properly linked operations/hydrodynamic/biological models at the appropriate scales for RPA development. The agencies should be complimented for using historical data as well as best available science when modeling was not adequate. However, the proposed RPAs could incur significant water supply costs, and there should be an attempt to provide an integrative analysis of the RPAs with quantitative tools. The committee also acknowledges the challenges associated with estimating water requirements for some RPAs, particularly those based on adaptive management strategies, but explicit and transparent consideration of water requirements and biological benefits of specific actions and of subsets of actions would provide the basis for a smoother implementation of the RPAs.

The committee recommends that the agencies consider investigating the use of CalSim-II and other quantitative tools (e.g., PTM, life-cycle models) to simulate appropriate RPA actions of both biological opinions. These linked models would allow an integrated evaluation of the biological benefits and water requirements of individual actions and suites of actions, and the identification of potential species conflicts among the RPAs. Although not required by the ESA,

Suggested Citation:"4 Use of Models." National Research Council. 2010. A Scientific Assessment of Alternatives for Reducing Water Management Effects on Threatened and Endangered Fishes in California's Bay-Delta. Washington, DC: The National Academies Press. doi: 10.17226/12881.
×

such an integrative analysis would be helpful to all concerned to evaluate the degree to which the RPAs are likely to produce biological benefits and to quantify the water requirements to those who might be affected by the future actions of the two biological opinions. In addition to further model development, efforts to improve documentation of model use would be beneficial. Documentation should include a record of the decisions, assumptions, and limitations of the models (e.g., NRC, 2008).

Thus, we find that, while used appropriately in this analysis, the PTM and biological models for both salmon and smelt should be further developed, evaluated, and documented. The models show promise for being quantitative tools that would allow for examination of alternative ideas about key relationships underlying the RPAs. In addition, complete life-cycle models capable of being linked to these other models should be developed. Although developing, testing, and evaluating such models would require a significant investment, the committee judges that the investment would be worthwhile in the long term.

CONCLUSION

Modeling is useful for understanding the system as well as predicting future performance. As long as modelers understand and accurately convey the uncertainties of models, they can provide valuable information for making decisions. The committee reviewed the models the agencies used to determine to what degree they used the models in developing the RPAs. The biological opinions have used results of a variety of operations, hydrodynamic, and biological models currently available to them for RPA development. However, the agencies have not developed a comprehensive modeling strategy that includes the development of new models (e.g., life-cycle and movement models that link behavior and hydrology); such models may have provided important additional information for the development of RPAs. Nonetheless, the agencies should be complimented for combining the available modeling results with historical observations and peer-reviewed literature. The committee also compliments the agencies for the extensive discussion and presentation of the rationale for the particular types of actions proposed in the RPAs.

The committee concluded that as far as they went, despite flaws, the individual models were scientifically justified, but that they needed improvements and that they did not go far enough toward an integrated analysis of the RPAs. The committee has raised several important issues related to the modeling process used, including the model scale and management information; the adequacy of models, particularly the particle-tracking model and the lack of life-cycle

Suggested Citation:"4 Use of Models." National Research Council. 2010. A Scientific Assessment of Alternatives for Reducing Water Management Effects on Threatened and Endangered Fishes in California's Bay-Delta. Washington, DC: The National Academies Press. doi: 10.17226/12881.
×

models; incompatibilities in both temporal and spatial scales among the models and between model output and the scale of the RPA actions; the use of baselines; inadequate calibration and testing of modeling tools (in some cases); and inadequate model documentation. A more-thorough, integrative evaluation of RPA actions with respect to their likelihood of reducing adverse effects on the listed fishes and their likely economic consequences, coupled with clear documentation would improve the credibility and perhaps the acceptance of the RPAs. Thus the committee concluded that improving the models by making them more realistic and by better matching the scale of their outputs to the scale of the actions, and by extending the modeling to be more comprehensive and to include features such as fish life cycles would improve the agencies’ abilities to assess risks to the fishes, to fine-tune various actions, and to predict the effects of the actions. Three-dimensional models are more expensive and time-consuming than simpler models, but they can contribute valuable understanding if used appropriately (e.g., Gross et al., 1999; Gross et al., 2009).

In addition, the committee concludes that opportunities exist for developing a framework to improve the credibility, accountability, and utility of models used in implementing the RPAs. The framework will be particularly important for some of the more-complex actions, such as those involving Shasta and San Joaquin storage and flows, which rely heavily on model predictions. The committee plans to address such issues, including the framework mentioned above, in more detail in its second report.

Suggested Citation:"4 Use of Models." National Research Council. 2010. A Scientific Assessment of Alternatives for Reducing Water Management Effects on Threatened and Endangered Fishes in California's Bay-Delta. Washington, DC: The National Academies Press. doi: 10.17226/12881.
×
Page 28
Suggested Citation:"4 Use of Models." National Research Council. 2010. A Scientific Assessment of Alternatives for Reducing Water Management Effects on Threatened and Endangered Fishes in California's Bay-Delta. Washington, DC: The National Academies Press. doi: 10.17226/12881.
×
Page 29
Suggested Citation:"4 Use of Models." National Research Council. 2010. A Scientific Assessment of Alternatives for Reducing Water Management Effects on Threatened and Endangered Fishes in California's Bay-Delta. Washington, DC: The National Academies Press. doi: 10.17226/12881.
×
Page 30
Suggested Citation:"4 Use of Models." National Research Council. 2010. A Scientific Assessment of Alternatives for Reducing Water Management Effects on Threatened and Endangered Fishes in California's Bay-Delta. Washington, DC: The National Academies Press. doi: 10.17226/12881.
×
Page 31
Suggested Citation:"4 Use of Models." National Research Council. 2010. A Scientific Assessment of Alternatives for Reducing Water Management Effects on Threatened and Endangered Fishes in California's Bay-Delta. Washington, DC: The National Academies Press. doi: 10.17226/12881.
×
Page 32
Suggested Citation:"4 Use of Models." National Research Council. 2010. A Scientific Assessment of Alternatives for Reducing Water Management Effects on Threatened and Endangered Fishes in California's Bay-Delta. Washington, DC: The National Academies Press. doi: 10.17226/12881.
×
Page 33
Suggested Citation:"4 Use of Models." National Research Council. 2010. A Scientific Assessment of Alternatives for Reducing Water Management Effects on Threatened and Endangered Fishes in California's Bay-Delta. Washington, DC: The National Academies Press. doi: 10.17226/12881.
×
Page 34
Suggested Citation:"4 Use of Models." National Research Council. 2010. A Scientific Assessment of Alternatives for Reducing Water Management Effects on Threatened and Endangered Fishes in California's Bay-Delta. Washington, DC: The National Academies Press. doi: 10.17226/12881.
×
Page 35
Suggested Citation:"4 Use of Models." National Research Council. 2010. A Scientific Assessment of Alternatives for Reducing Water Management Effects on Threatened and Endangered Fishes in California's Bay-Delta. Washington, DC: The National Academies Press. doi: 10.17226/12881.
×
Page 36
Suggested Citation:"4 Use of Models." National Research Council. 2010. A Scientific Assessment of Alternatives for Reducing Water Management Effects on Threatened and Endangered Fishes in California's Bay-Delta. Washington, DC: The National Academies Press. doi: 10.17226/12881.
×
Page 37
Suggested Citation:"4 Use of Models." National Research Council. 2010. A Scientific Assessment of Alternatives for Reducing Water Management Effects on Threatened and Endangered Fishes in California's Bay-Delta. Washington, DC: The National Academies Press. doi: 10.17226/12881.
×
Page 38
Suggested Citation:"4 Use of Models." National Research Council. 2010. A Scientific Assessment of Alternatives for Reducing Water Management Effects on Threatened and Endangered Fishes in California's Bay-Delta. Washington, DC: The National Academies Press. doi: 10.17226/12881.
×
Page 39
Suggested Citation:"4 Use of Models." National Research Council. 2010. A Scientific Assessment of Alternatives for Reducing Water Management Effects on Threatened and Endangered Fishes in California's Bay-Delta. Washington, DC: The National Academies Press. doi: 10.17226/12881.
×
Page 40
Suggested Citation:"4 Use of Models." National Research Council. 2010. A Scientific Assessment of Alternatives for Reducing Water Management Effects on Threatened and Endangered Fishes in California's Bay-Delta. Washington, DC: The National Academies Press. doi: 10.17226/12881.
×
Page 41
Next: 5 Other Stressors »
A Scientific Assessment of Alternatives for Reducing Water Management Effects on Threatened and Endangered Fishes in California's Bay-Delta Get This Book
×
Buy Paperback | $39.00 Buy Ebook | $31.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

California's Bay-Delta estuary is a biologically diverse estuarine ecosystem that plays a central role in the distribution of California's water from the state's wetter northern regions to its southern, arid, and populous cities and agricultural areas. Recently, the Fish and Wildlife Service and the National Marine Fisheries Service required changes (reasonable and prudent alternatives, or RPAs) in water operations and related actions to avoid jeopardizing the continued existence and potential for recovery of threatened species of fish. Those changes have reduced the amount of water available for other uses, and the tensions that resulted have been exacerbated by recent dry years.

The complexity of the problem of the decline of the listed species and the difficulty of identifying viable solutions have led to disagreements, including concerns that some of the actions in the RPAs might be ineffective and might cause harm and economic disruptions to water users, and that some of the actions specified in the RPAs to help one or more of the listed species might harm others. In addition, some have suggested that the agencies might be able to meet their legal obligation to protect species with less economic disruptions to other water users.

The National Research Council examines the issue in the present volume to conclude that most of the actions proposed by two federal agencies to protect endangered and threatened fish species through water diversions in the California Bay-Delta are "scientifically justified." But less well-supported by scientific analyses is the basis for the specific environmental triggers that would indicate when to reduce the water diversions required by the actions.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!