6
Enhancements and Future Uses of OST
Over a decade ago, the New York City Department of Environmental Protection (NYC DEP) undertook the development of the Operations Support Tool (OST) as part of a suite of actions intended to improve management of turbid waters flowing out of the Ashokan Reservoir in the Catskill system. A requirement to develop OST was part of both a 2006 State Pollutant Discharge Elimination System (SPDES) permit issued by the New York State Department of Health (NYS DOH) and the 2007 filtration avoidance determuination (FAD) issued by the U.S. Environmental Protection Agency (EPA). From the earliest stages of OST development, however, NYC DEP recognized that OST’s capability to account for the movement of waters of various quality through the many nodes and arcs of the Catskill and Delaware systems offered them a tool for exploring a wide range of critical operational and planning questions (NYC DEP, 2010). As discussed in Chapter 2, OST is now used to enhance water supply and water quality reliability and to gain insights on how water supply operations may affect downstream fish habitat and stream ecosystems. This chapter describes other applications of OST, some of which are already being pursued, that could support or enhance NYC DEP’s mission.
USE OF OST TO CAPTURE EXISTING STAFF KNOWLEDGE AND EXPERTISE
OST can be viewed as an expert system and repository of knowledge that reflects the collective professional experience, judgment, and institutional memory of its most experienced staff, many of whom are nearing
retirement. Institutional knowledge is a valuable asset of the current NYC DEP team. OST-supported decision making is at times guided by “soft” rules that are implemented using expert judgment. Some clearly defined decision-making rules do exist, such as the conditions of the Shandaken SPDES permit and the potable water turbidity standards of Subpart 5-1 of the New York State Sanitary Code. The use of best professional judgment is nevertheless an important element of the decision-making process.
For purposes of capturing the deep institutional knowledge of NYC DEP’s experts on OST, NYC DEP would benefit from a more systematic, documented approach to capturing information gained from OST runs. This knowledge capture serves two ends: to use the tool and its runs to preserve and enable retrieval of information about the system and to guide the professional development of NYC DEP’s next generation of water supply system operators.
When NYC DEP uses OST in position analysis (PA) mode, it draws on the expertise of its engineers and operators regarding how the water supply system should operate. This expert knowledge is captured both within OST’s model code and through the use of specific Operations Control Language (OCL) instructions that accompany each run of OST. OCL enables OST’s operators to set bounds on many constraints at individual nodes and arcs in the system related to flows, storage, and releases. OCL is also the means by which NYC DEP can easily incorporate its real-time knowledge of operational changes required to accommodate maintenance and construction schedules.
NYC DEP has a written standard operating procedure in place for when and why OST is run, but not for how to run OST. Water quality reports alert the OST team when it may be prudent to run W2. Non-routine runs take place regularly to support additional operational decision making, such as scheduling infrastructure maintenance. NYC DEP team members have acknowledged that there should be more detailed documentation of assumptions built into OST and decisions related to how and when it is used. As an example, each weight placed on a node or arc has a rationale and assumptions behind it that should be documented and updated as needed.
In addition, applications of the tool and the outcomes of those applications can provide valuable information for informing subsequent operational assumptions and would benefit current and future systems operators. By enabling retrieval of prior runs of OST and “reverse engineering” those runs, current and future operators would gain insights into how and why the system responded under, for example, varying streamflow and turbidity forcing conditions, management of extreme rainfall events, reservoir release strategies under persistent drought, and work-arounds to cope with major system shutdowns. The Committee encourages NYC DEP to continue to
build on its growing base of understanding of system performance under a range of challenging conditions by making maximum use of the OST PA runs.
USE OF OST AS A TRAINING TOOL
As an extension of OST’s use as a means of capturing staff knowledge and expertise in operating the water supply system, OST also could provide a sophisticated simulator for training staff to cope with “virtual” floods, droughts, and other off-normal or extreme events. One approach to using OST in a training mode could be to generate stressing scenarios (e.g., intense high flows or an extended period of low flows) using synthesized time series of streamflow and other input data to OST. Alternatively, NYC DEP could draw on historical input data, actual systems performance, and archived OST runs over the course of particularly notable historical events or time periods since OST’s inception.
A prime example would be OST’s performance during Hurricane Irene in August 2011. In drawing on past experience, NYC DEP could link varying streamflow and turbidity forcing conditions to the agency’s actual operational decisions, the extent to which those decisions tracked insights gained from OST, and observed system responses. This would provide insights into the ability of OST to capture complexities of the system under stress. It would further lend insight into NYC DEP’s past management of extreme rainfall events and work-arounds to cope with major system shutdowns. It might not be able to shed much light on reservoir release strategies under persistent drought, given the relatively benign drought conditions that the region has experienced since OST’s inception.
With both model improvement and training as objectives, this type of analysis could be fed back into OST’s model structure and code, at the same time offering NYC DEP’s staff valuable practice for handling “off normal” conditions. Lessons learned from previous experience could be incorporated into “play books” in which specific operational scenarios are documented in a form that could guide NYC DEP staff on future operational decisions.
LONG-TERM PLANNING UNDER A RANGE OF FUTURE CONDITIONS
With confidence that OST adequately captures the essential features and responses of the New York City water supply system when under stress, NYC DEP can exploit the many opportunities of using OST as an aid to long-term planning. OST could be—and indeed has been—used to explore alternative operating rules and the impact of new or reconfigured infrastructure within the system. Various patterns of shortages or other
emergency conditions could be simulated and potential vulnerabilities in current or future operations identified and explored.
As discussed in Chapter 5, OST can be used to explore the robustness of system operations under conditions of climate and other uncertainties with the potential to affect long-term changes in supply and demand. For example, OST could be used to explore whether under an extreme event or a series of events turbidity in upstream tributaries and reservoirs could rise high enough to cause exceedance of the 5-nephelometric turbidity units (NTU) limit in Kensico Reservoir, and whether enough alum could be dosed to counteract the turbidity influx. Future extreme flooding might also cause a shift in the dominant source of turbidity in the Catskill system (instream vs. terrestrial), which might be simulated by OST. A number of methods in the family of risk-based decision analytics could be used, with approaches varying by their decision criteria and assumptions about the extent to which climate uncertainties can be expressed with confidence as known probabilities (Brown et al., 2012; Kalra et al., 2014). For example, “deep uncertainty” describes a phenomenon in which probabilities of future conditions cannot be specified with any degree of confidence (Lempert et al., 2003). Under conditions of deep uncertainty, optimal solutions to operational problems cannot be identified, leading instead to the use of a robustness criterion in which future actions are favored that would do well over a wide range of possible future conditions.
One such application of OST for long-term planning purposes under assumptions of deep uncertainty was demonstrated by Groves et al. (2014) in which water reliability (percent drought days) and water quality (percent alum days) were projected over a wide range of climate, demand, and other uncertainties extending to the end of the 21st century. Using robust decision-making methods (Lempert et al. 2003, 2013; Groves et al., 2013; Fischbach et al., 2017), the Groves et al. (2014) analysis sought to identify vulnerabilities to NYC DEP’s operations and the conditions under which those vulnerabilities manifested themselves. The authors then explored a number of possible strategies, investments, and policy changes that could reduce vulnerabilities. Other types of risk-based methods also could provide NYC DEP and its regulators with useful insights about how policy changes could play out under a range of possible future climate and even funding conditions.
USE OF OST TO SUPPORT COMPLIANCE WITH THE FILTRATION AVOIDANCE DETERMINATION
A defining feature of OST is its interface between the OASIS model and the W2 water quality module, which enables the simulated interaction between reservoir operations and water quality throughout the system.
While OASIS tracks flows, OST can track projected changes in turbidity over the duration of a simulation period by passing daily flow data from OASIS to W2 and then sending W2 projections of turbidity back to OASIS. The modeling structure is thus already in place to use OST to simulate operations and turbidity levels in the absence of the Catskill Turbidity Control Program (CTCP) infrastructure and operational improvements (a no-action scenario) and compare these simulated results to actual operations and observed turbidity levels with the CTCP in place (see Chapter 3). This could provide useful information to NYC DEP and NYS DOH as they periodically assess the effectiveness of the CTCP and compliance with the FAD.
This application should be a complement to the kind of exploratory data analysis on observed turbidity levels and other measurements described in Chapter 3, an essential step in understanding system performance from a statistical perspective. Use of OST in this application would further rely on routine reevaluation of the estimates of turbidity built into W2. As described in Chapter 2, this application also would depend on the demonstration of OST’s capacity to reproduce observations of streamflows and turbidity levels under a wide range of streamflow conditions.
USE OF OST TO MANAGE OTHER WATER QUALITY METRICS
Given the capabilities of the W2 module, OST in theory could be exercised to estimate other water quality measures beyond NYC DEP’s sole focus on turbidity. For example, with sufficient observational data, it might be possible to incorporate precursors of disinfection byproducts, pathogens, and other water quality parameters into the W2 models within OST. These extensions would likely be more complicated than modeling turbidity, but working toward this end might enable NYC DEP to get ahead of issues that could arise in future regulatory proceedings.
The expansion of OST’s scope in simulating other water quality metrics could enable its use beyond system operations to include analysis of potential impacts of modifying current practices or implementing new actions under the authority of the Watershed Protection Program. The Watershed Protection Program currently includes a wide range of land-use restrictions and other watershed practices to reduce the possibility that contaminants such as fecal coliforms, pathogens such as Cryptosporidium and Giardia, and precursors of disinfection byproducts will enter the water supply. OST could simulate water quality impacts of existing and new program elements under a range of assumptions regarding systems operations, future land use, and other changes within the watershed.
USE OF OST TO HELP ILLUMINATE AND FRAME RESEARCH QUESTIONS
As with any mathematical model intended to simulate the dynamics of a complex system, use of OST should take into consideration the well-known maxim attributed to the statistician George E. P. Box that “Essentially, all models are wrong, but some are useful” (Box, 1987, p. 424). OST is structured to track the flow of water through a complex engineered system using streamflow forecasts and other inputs from an even more complex natural system. The distribution of precipitation falling across the watersheds of New York City’s water system is highly variable in time and space, as is the resultant streamflow, infiltration into the subsurface, and evapotranspiration. Further complexity is added by the runoff of suspended sediments, contaminants, and other constituents in the soil into the streams and reservoirs of the system.
OST serves as a repository of current understanding of the dynamics within the water supply system, but can also serve to highlight research questions that could improve its usefulness to NYC DEP. One example relates to structuring physical experiments to measure sediment fate and transport within the Catskill system to provide an improved understanding of how turbidity correlates with precipitation events of varying antecedent conditions, duration, and intensity. OST could be run in a mode in which turbidity is assumed to be an uncertain parameter rather than a deterministic input. This approach could provide some insight into the sensitivity of downstream reservoir operations and diversions to assumptions about the distribution of turbidity under given precipitation and streamflow forecasts.
Another line of research could explore the sensitivity of OST’s outputs to the weights embedded in the OASIS code. Insights could be obtainable from the underlying linear program solver. The Lagrange Multiplier and Slack solution results (sometimes called “reduced cost” and “range of basis” information) from such solvers can show how sensitive OST’s results might be to changes in individual weights assigned to the many nodes and arcs in OASIS. This type of sensitivity analysis also could provide insights into how much constraints and mass balances could affect OST’s results.
For some future research activities, significantly shortening OST’s run time would enable a range of studies designed to explore a wider range of possible future scenarios. Powerful parallel processing resources now found in a cloud computing environment can significantly speed up model run times at reasonable cost, although some adjustments in the model code may be required to take full advantage of parallel processing. If OST were to run faster, NYC DEP would be better positioned to more extensively explore uncertainty space. Specifically, NYC DEP could gain some insights into where the cascading uncertainties associated with OASIS and the W2
models are most problematic and how uncertainty is propagated through the modeled system.
Beyond NYC DEP’s immediate applied research needs, OST provides a valuable platform to support even more cutting-edge science. To this end, the Committee encourages NYC DEP and its academic partners to consider structuring a more formal research program and to take steps to improve documentation and timely publication of results in the peer-reviewed literature.
COMMUNICATION ABOUT OST AND HOW IT IS USED
Given the complexity of OST and the many assumptions built into its code, NYC DEP is challenged to provide the public with a clear and compelling explanation of OST’s workings, and how it is used to inform decision making in accordance with NYC DEP’s mission of providing drinking water to New York City. As NYC DEP’s use of OST continues to mature and offer additional insights to a wider range of operational and regulatory questions, the need for transparency and clarity about what OST is and what OST does will only grow. As important as developing clear explanations of what OST does and how it is used are equally clear explanations of what OST does not do and the uses for which OST is not well suited, such as modeling water quality and ecological impacts outside the bounds of the calibrated model.
REFERENCES
Box, G. E. P., and N. R. Draper. 1987. Empirical Model Building and Response Surfaces. New York: John Wiley & Sons.
Brown, C., and Y. Ghile, M. Laverty, and K. Li. 2012. Decision scaling: Linking bottom-up vulnerability analysis with climate projections in the water sector. Water Resources Research 48(9):W09537.
Fischbach, J. R., K. Siler-Evans, D. Tierney, M. T. Wilson, L. M. Cook and L. Warren May. 2017. Robust Stormwater Management in the Pittsburgh Region: A Pilot Study. Santa Monica, CA: RAND Corporation. https://www.rand.org/pubs/research_reports/RR1673.html.
Groves, D. G., J. R. Fischbach, E. Bloom, D. Knopman, and R. Keefe. 2013. Adapting to a Changing Colorado River: Making Future Water Deliveries More Reliable Through Robust Management Strategies. Santa Monica, CA: RAND Corporation. https://www.rand.org/pubs/research_reports/RR242.html.
Groves, D. G., J. R. Fischbach, N. Kalra, E. Molina-Perez, D. Yates, D. Purkey, A. Fencl, V. K. Mehta, B. Wright, and G. Pyke. 2014. Developing Robust Strategies for Climate Change and Other Risks: A Water Utility Framework. Denver, CO: Water Research Foundation.
Kalra, N., S. Hallegatte, R. Lempert, C. Brown, A. Fozzard, S. Gill, and A. Shah. 2014. Agreeing on Robust Decisions: New Processes for Decision Making Under Deep Uncertainty. Policy Research Working Paper No. 6906. June 1. World Bank. http://ssrn.com/abstract=2446310.
Lempert, R. J., S. W. Popper and S. C. Bankes. 2003. Shaping the Next One Hundred Years: New Methods for Quantitative, Long-Term Policy Analysis. Santa Monica, CA: RAND Corporation. https://www.rand.org/pubs/monograph_reports/MR1626.html.
Lempert, R. J., S. W. Popper, D. G. Groves, N. Kalra, J. R. Fischbach, S. C. Bankes, B. P. Bryant, M. T. Collins, K. Keller, A. Hackbarth, L. Dixon, T. LaTourrette, R. T. Reville, J. W. Hall, C. Mijere and D. J. McInerney. 2013. Making Good Decisions Without Predictions: Robust Decision Making for Planning Under Deep Uncertainty. Santa Monica, CA: RAND Corporation. https://www.rand.org/pubs/research_briefs/RB9701.html.
NYC DEP (New York City Department of Environmental Protection). 2010. New York City’s Operations Support Tool (OST) White Paper Prepared for the Delaware River Basin Supreme Court Decree Parties. October 8.
