Review of the Lake Ontario-St. Lawrence River Studies (2006)

This chapter draws together findings and recommendations from the NRC/RSC review of LOSLR studies and models. It begins by situating those studies and models within the historical context that shaped them, follows with the Statement of Task, noting the five qualifications of this review, and concludes with five overarching conclusions and recommendations. The conclusions look back at what has, and has not, been accomplished; while the recommendations look forward toward continuing scientific approaches needed to address the identified gaps, deficiencies, and water level and flow regulation challenges.

The historical context of the LOSLR studies and models includes a half-century of experience with water regulation plans and their effects (Clinton Edmonds and Associates, 2002). The first regulation plan for the Lake Ontario-St. Lawrence system (Plan 12-A-9) was adopted in 1956 (International Joint Commission, 1956) and replaced in 1960. The second Plan 1958-A was replaced after two years, and the third Plan 1958-B after only one year. The current Plan 1958DD was adopted in 1963 and has been in effect for 42 years, but it has required an increasing number and duration of deviations to accommodate hydroclimatic, ecological, and socioeconomic changes. In the 1990s, alternative plans were formulated to improve operations and reduce deviations, but they were not adopted in large measure because they did not examine the broad range of water regulation impacts or offer sufficient improvement.

The IJC responded to these deficiencies by commissioning the LOSLR Plan of Study (BACK 2). The LOSLR Study was authorized to expand the scope of inquiry to include environmental, social, and economic impacts. To achieve that aim, the LOSLR Study developed a Shared Vision Model approach to formulate and evaluate RPOs through a collaborative process with nine Technical Work Groups and a Public Interest Advisory Group. The LOSLR Study thus sought to address, in five years, water regulation issues that have developed over a half-century. These water regulation issues will continue to evolve in future decades in response to new socioeconomic, ecological, and hydroclimatic challenges. The LOSLR Study addresses some of these future challenges quantitatively (e.g., climate change scenarios, which were not part of this review), and others qualitatively (e.g., through Conceptual Narratives).

The IJC requested the National Research Council and Royal Society of Canada, “…to evaluate the appropriateness and sufficiency of the studies and models used to inform decisions related to regulation plan options.” The Statement of Task reads:

1. The studies reflect reasonable scientific methods, assumptions and supported findings;

2. The models sufficiently and appropriately integrate and display the key information needed for a comprehensive evaluation and understanding of the tradeoffs for selecting among the candidate RPOs; and

3. The models and reports are sufficient and appropriate to evaluate the various regulation plan options (RPOs) and impacts of changes in water levels and flows.

There are five qualifications to keep in mind when using this review:

• In some cases interim documents were presented for review. The documents presented for review were in various stages of completion. In cases where documents were incomplete, the committee tried to procure the most current version of the work. The committee treated the documents presented by the IJC as representative of the science under review and recognized that some documents would be modified after the review.

• This is a selective review of LOSLR studies and models. The committee reviewed selected documents for the SVM, FEPS, and environmental sciences. The review does not encompass all of the LOSLR research undertaken in these three fields. Nor does it include scientific fields in the LOSLR study that lie outside the scope of review but have some relevance for the overall charge, such as hydrology and hydraulics, navigation, hydropower, M&I, RSPM, etc. To the extent that the Shared Vision Model incorporates results from these other fields, this review offers a partial perspective on the overall sufficiency of LOSLR studies and models.

• The review occurs toward the end of the 5-year LOSLR Study. The NRC review was initiated in the final year of the LOSLR Study. This timing offers an opportunity to evaluate the studies and models used to inform to decision makers, as well as to identify opportunities for improvement prior to submission. However, some recommendations would have been more useful at the beginning or middle of the study period rather than this close to the Study’s completion.

• The review concentrates on the science for evaluating water level and flow effects of RPOs and for informing decision makers, and not on the

RPOs themselves or on decision making policies. Ten scientific evaluation criteria were used to evaluate the LOSLR studies and models. These criteria are common to the scientific and practical professional disciplines involved in evaluating complex studies, such as the water level and flow effects of regulation plan options in the Lake Ontario-St. Lawrence River system.

• The review distinguishes among conclusions and recommendations in terms of their certainty, importance, and ability to fix deficiencies. The conclusions of this report vary in terms of their certainty due to the state-of-the-science in different fields and gaps in study documentation. Some conclusions have more importance to the success of the LOSLR Study than others. Points of study weakness and recommendations vary in the degree to which they can be fixed and the amount of time and additional research needed to address them. The review strives to distinguish among recommendations that entail short- and long-term action. Depending upon the level of commitment made, short-term conclusions and recommendations are those that can be addressed in a period of weeks to months without new research. At this stage of the process, substantive research deficiencies cannot be fixed. Thus, short-term recommendations concentrate on improving the documentation, scientific communication, and disclosure of potential implications of these limitations for decision makers. Longer-term recommendations require investment in additional data collection, analysis, and interpretation. In making these distinctions, conclusions deemed to have the most certainty, importance, and timeliness are presented as declarative statements. Conclusions that have less certainty, importance, or timeliness are presented in conditional terms. Conclusions that have the least certainty, importance, or ability to be fixed are raised as questions.

With these five limitations in mind, this chapter presents five overarching findings about the LOSLR studies and model accomplishments, their scientific strengths and weaknesses, and the recommendations that emerge from this review.

This review commends some aspects of the studies and models and raises substantial concerns about others. If all of the individual reports, studies, and models reviewed here were found to be appropriate and sufficient, the committee might come to a qualified affirmation of their collective sufficiency for informing decision making. Instead, while commending advances over previous studies, this review finds deficiencies in LOSLR studies and models for each of the ten evaluation criteria. The overarching conclusions are:

1. LOSLR studies and models expand interdisciplinary scientific inquiry on the potential environmental effects of water level and flow regulation options in the Lake Ontario-St. Lawrence River Basin in ways that are useful for informing decision making in some respects. The LOSLR studies undertook a broad scope of inquiry and participatory process for understanding the potential effects of water level and flow regulation in a complex water system. The

Shared Vision Model compiles effects of flow regulation on environmental, coastal, and recreational effects, in addition to statutory obligations to consider municipal, navigation, and hydropower uses. LOSLR models go beyond previous Great Lakes water regulation efforts in compiling results of scientific inquiry and stakeholder input. Its iterative public participation process begins to treat water level and flow regulation as an adaptive process. Identification and inclusion of environmental performance indicators is a major contribution. The LOSLR studies have created large new databases, like those on wetland vegetation and coastal land use—that did not exist previously and that could, if archived and made readily accessible, have continuing value. However, the studies and models need to go further in encompassing ecological and socioeconomic scenarios and the linkages among them. Moreover, while the studies and models attempt to balance scientific and practical professional water management approaches, the findings below indicate where established standards of inquiry were, and were not, met.

2. The studies and models vary widely in empirical support, and they need stronger and more consistent quality control, quality assurance and treatment of error and uncertainty needed to inform decision making. This review finds that each of the studies has scientific merits and deficiencies that are summarized here. Following the framework used throughout the review, the scientific evaluation criteria are discussed under the headings of empirical foundations, quality assurance and quality control, and treatment of error and uncertainty.

• Empirical foundations (e.g., data, sampling, analysis). The studies and models reviewed here focus primarily on environmental and coastal investigations. The reviewed materials have varied types of data, data quality, sampling methods, and aggregation techniques that pose challenges for using results to inform management decisions. For example: (1) the coastal research developed a detailed land use parcel database, but that database differs in coverage for Canada and the USA; (2) although detailed for the sites selected, wetlands sampling is not statistically representative of wetland vegetation types along the coast or of deeper submergent vegetation in deeper water; (3) species at risk (SAR) data were collected to help develop some but not all SAR performance indicators; and (4) questions were raised about the way that some regression models were used. Empirical variation is to be expected in a multidisciplinary study where each discipline brings its own conventions, and in which long-term data collection is limited by the five-year timescale. Still, differences in empirical methods can propagate error and uncertainty in ways that have relevance for decision making. Recommendation: As no new data can be collected in the near-term, LOSLR study final reports should underscore empirical limitations, data gaps, and sampling problems, and discuss their implications for decision making.¹ For the longer-term,

research to correct deficiencies in data and models including replacement of regression equations with process models should be prioritized.

• Quality assurance and quality control (e.g., model validation, verification, and calibration; use of expert judgment; and independent peer review). The models generally lack adequate validation, verification, and calibration. In some cases, validation may have occurred or is briefly mentioned but is not documented. In other cases, it appears not to have been undertaken. For example, the SVM had not yet received its “stamps of approval” confirming model validation from technical work group members. Reports on the FEPS model suggest that model calibration has occurred but do not provide detailed documentation. Validating applications of proprietary models such as COSMOS is important for full scientific peer review. Environmental studies lack consistent protocols for quality assurance, and the IERM model acknowledges that validation was not attempted. The SVM and FEPS models and SAR studies make creative use of expert judgment, which should be subject to more formal quality assurance by using well-established protocols for eliciting expert judgments. Some studies rely on “peer review” by fellow team members while others involve refereed papers. Earlier external scientific review of the overall LOSLR study program would have been timely for identifying and rectifying deficiencies. Recommendation: In the short-term, LOSLR final reports should inform decision makers of the types of quality assurance measures that were, and were not, undertaken and discuss their potential implications for decision making. Further independent scientific review of final reports is recommended. In the longer-term, rigorous quality assurance methods should be put in place for evaluating the effects of water level and flow regulation.

• Treatment of error and uncertainty. This criterion was not fulfilled in the studies or models reviewed at the level expected for informing decision making. The SVM, FEPS, and IERM models do not present an overall framework for uncertainty analysis, which should include natural variability, data uncertainties, model uncertainties, model parameter uncertainties, and decision model uncertainties. Some individual studies, such as wetlands vegetation analysis, address natural variability and indicate error bars. The SAR 3A report provides a good model for qualitative discussion of uncertainty. The Shared Vision Model treats the uncertainty of environmental indicators with a simplistic, and unexplained, 10% criterion, and it does not apply any uncertainty estimate to economic indicators. Linkages among LOSLR studies and models lead inherently to the propagation of uncertainties, but SVM documentation does not analyze those cumulative uncertainties or discuss their implications for informing decision making. Without formal analysis and discussion, it is not possible to assess the types or magnitudes of error and uncertainty for particular water regulation plans, or to know whether differences between plans are significant. Recommendation: In the short-term, LOSLR final study reports should inform decision makers of the uncertainties that were analyzed, those that were not analyzed, and their potential implications for de-

cision making. Future studies of water level regulation effects in the LOSLR basin should develop a comprehensive approach to uncertainty analysis.

3. The LOSLR models and studies reviewed here do not adequately integrate and display the key information needed for comprehensive evaluation and understanding of the tradeoffs for selecting among the candidate regulation plans. This conclusion is based on the following four review criteria:

• Linkages and feedbacks among related studies and models. “Comprehensive evaluation and understanding of tradeoffs among RPO alternatives” requires a system dynamics approach that models the linkages and feedbacks among socioeconomic and environmental processes. The SVM compiles first-order effects on environmental, coastal, and other indicators generated by FEPS, IERM, and other models. But, as the IERM user’s manual indicates (IERM 4), it is not an ecosystem model that incorporates the feedback effects of water level variation on species and habitat conditions. Instead, it compiles initial impacts (first-order effects) on performance indicators, and it is thus an impact accounting model rather than an ecosystem model. In terms of model linkages, the FEPS model alters the bathymetry of shoreline environments, but those bathymetric changes were not fed into the IERM to alter the extent or depth of wetland inundation, which could in turn affect the vegetation, shoreline habitats, and other environmental performance indicators associated with water level variation. These vegetation changes could have feedback effects on sediment transport and coastal erosion. External model linkages include economic and demographic scenarios that are relevant for evaluating candidate water regulation plans to replace Plan 1958DD. For example, real estate values of coastal property continue to rise at rapid rates, and the demand for different water and related land uses is changing, but the SVM does not incorporate such scenarios in its structure.

This report acknowledges that some of these linkages and feedbacks require knowledge beyond the current limits, and that fact should be discussed in the final reports and presentation of SVM results. However, other linkages and feedbacks between the SVM and its sub-models, and externally between the SVM and scenarios of socioeconomic change, could have been addressed. The reviewed studies and models make progress toward comparing the effects of regulation plan options, but the comparisons reviewed here do not provide a comprehensive basis for evaluating and understanding tradeoffs among regulation plan options. Recommendation: In the short-term, the LOSLR final reports should inform decision makers of what has, and has not yet, been accomplished in the way of integrated water and environmental systems modeling. As part of an ongoing program, a LOSLR modeling system that dynamically links and reflects feedback among sub-models is recommended.

• Treatment of spatial and temporal resolution and scaling. Scaling issues in the Lake Ontario-St. Lawrence River basin are challenging. The LOSLR studies involve a wide range of spatial and temporal scales, which raise a number of concerns. For example, although more detailed hydrologic time series and station

data are available for use in the STELLA model in the SVM, the model generates a single series of quarter-monthly values for the level of Lake Ontario, based on historical water management practice. Use of these single series values can result in a loss of precision, as the quarter-month does not provide enough temporal variation for many environmental impacts for fish, SAR, and wetlands. This coarse time step was recognized as a potential problem in the LOSLR Plan of Study, which called for a 2D hydrodynamic model for the St. Lawrence River that operated on fine enough time scales to supplement the quarter-monthly time step generated by the SVM. As noted earlier, the LOSLR approach of using quarter-monthly values in Lake Ontario to calculate water levels for selected stations in the upper St. Lawrence River through regression analysis is inferior to hydrodynamic flow routing, and the combined use of regression and hydrodynamic models needs to be more fully explained. The FEPS model uses the single lake level elevations along with a grid of wind and wave fields that ultimately erode and flood individual shoreline parcels and reaches, the results of which are aggregated back to lake-wide effects. The errors and uncertainties associated with these varied resolutions and scales of inquiry need fuller analysis and discussion, as errors may exceed the differences between model outputs for some performance indicators and plans. Recommendation: In the short-term, the LOSLR final reports must inform decision makers of temporal and spatial scaling issues that affect the accuracy and uncertainty of predictions of regulation effects. In the longer term, choice of time step should better reflect the critical response times for system indicators, including those where transient fluctuations in water temperature and water level are critical, and appropriate hydraulic and hydrodynamic modeling approaches should be implemented.

• Thorough documentation of scientific studies and models. Of the ten criteria employed in this review, inadequate documentation was the most frequently cited deficiency. It is one of the few deficiencies that can be corrected in the near term. The Shared Vision Model had the least amount of documentation presented for this review, and the documentation that was presented was not at a level of completion ready for external scientific and public review, although it may have been adequate for internal working by the Study Board. FEPS included more detailed descriptions of modeled performance indicators than other studies, but did not fully document the models themselves. A user’s manual exists for the IERM and provides partial documentation, but insufficient explanations of weighting and aggregation in the model. Exceptions to these general patterns include the SAR 3A and 3B reports, which are well documented. Better documentation might have addressed some of the scientific concerns raised in this review, and might have raised additional questions. In either case, fuller disclosure of what was done and how it was done is needed to inform decision making.

Because the SVM is the primary tool for understanding and evaluating tradeoffs among RPOs, we concentrate on it in this recommendation. Documentation of the SVM should include more complete discussion of its role in the Shared Vision planning process. It should describe SVM development and refinement, including standard technical documentation of all component models. It should de-

scribe how scientific and stakeholder criteria were interactively used to formulate, screen, and evaluate the range of choice among RPOs. Recommendation: In the short-term, LOSLR final reports should include a thorough documentation of studies and models, especially the Shared Vision Model, and seek further independent scientific review of those reports.

• Effective scientific communication. This criterion involves the presentation of scientific information in ways that ensure its reception and comprehension by scientific, public, and decision making groups. Effective communication addresses public interests, communicates scientific findings and uncertainties clearly, is received accurately, and supports decision making.

The efficacy of scientific communication varies among LOSLR studies and models. Information about the environmental studies and performance indicator summaries were conveyed more clearly than the sub-model documentation, and sub-model documents more clearly than the SVM documentation. An example of deficient, or even misleading, communication is the differential treatment of economic and environmental indicators in Board Room presentations. On the other hand, the SVM Board Room spreadsheet tables and graphics have strong potential as a vehicle of scientific communication. As indicated in chapter 2, using the Board Room with broader public groups would require some editing and graphic design improvements. Recommendation: In the short-term the LOSLR final reports should communicate their scientific results with transparency to support decision making and to give full treatment of uncertainties and non-scientific dimensions of the studies. In the longer-term, the SVM Board Room may be refined for continued use as a vehicle for scientific communication.

4. Despite the breadth of LOSLR studies and models, ongoing analysis is needed to provide a strong scientific basis for long-term decision making about water level and flow regulation in the Lake Ontario-St. Lawrence River basin. Three points support this conclusion. First, current knowledge about the lower Great Lakes system is not comprehensive. The LOSLR studies and models broaden understanding about potential effects of RPOs, but a more comprehensive modeling approach is needed to understand system feedbacks, linkages, and uncertainties. A system dynamics model would: (a) improve the physical system description; (b) identify the most important feedback relationships; and (c) improve understanding of feedback effects on system behavior. Some feedback relationships would require expansion of model boundaries so that key processes, ranging from coastal urbanization and regional economic growth to climate change, are incorporated and their impacts are made visible within the model.

Second, previously selected regulation plans have been periodically reviewed and replaced in light of new knowledge. Initial RPOs were replaced due to unanticipated hydrologic and hydraulic events and their consequences. Plan 1958D accommodated some of those phenomena but required deviations to address less well-understood and emerging socioeconomic and environmental values, such as recreational boating and wetlands protection. The LOSLR studies and models

begin to address those effects. However, any plan adopted on the basis of current science without provision for regular updating as knowledge advances is likely to prove inadequate within a timeframe of years to decades.

Third, the LOSLR models evaluate effects of future RPOs and hydrologic scenarios on historical and current environmental and social performance indicators. This is important, given the significance of hydroclimatic variability for RPOs and the challenges of modeling current environmental and socioeconomic processes. Although this report does not review the climate change research and scenarios, it commends the inclusion of global processes that affect the robustness of regional RPO decisions (Sousounis, 2002). In the future, however, RPO decisions will require comparable scenario development and evaluation for other environmental and social processes. Changes in regional economic structure, demography, water demand, transportation technology, coastal land use, and socioeconomic values—to name a few—will likely transform the profile of stakeholder interests, performance indicators, and socioeconomic impacts associated with RPOs (Economic Advisory Committee, 2004). The past half-century indicates that these types of structural shifts in socioeconomic and environmental conditions and values, in conjunction with hydrologic variability, have had substantial implications for RPO decision making.

The LOSLR studies and models begin to address these issues through brief conceptual narratives with a planning horizon of 10 to 15 years that are linked to the Shared Vision Model, as recommended by the Economic Advisory Committee (2004). The conceptual narratives employ a common template, but they vary in detail, completeness, and level of peer review (Board Room, FEPS 11-13; and Leger, 2005). Correcting the scientific and modeling deficiencies identified in this review is necessary and appropriate, but not sufficient for informing water regulation decisions on a multi-decadal timescale. Recommendation: In the short-term, the LOSLR Study should complete the conceptual narratives and external peer review. For the longer-term, the IJC should consider commissioning an ongoing management and monitoring system to feed the results of current choices for water level regulation back into a dynamic model of the LOSLR system, in order to improve the scientific basis for future planning on a multidecadal timescale.

5. Build upon the LOSLR studies and models through an Adaptive Management Program. The LOSLR Study may begin a new approach to water level and flow regulation in the Great Lakes. Even when deficiencies are corrected, there will be an inherent need for continuing scientific study and modeling of the actual effects of present RPO decisions, as well as the possible effects of future RPO adjustments. These needs are anticipated in the LOSLR “Vision, Goals and Guiding Principles” (LOSLR, 28 August 2003):

• Criteria and Regulation Plans will incorporate flexible management of levels and flows in recognition of unusual or unexpected conditions affecting the Lake Ontario-St. Lawrence River System.

• Regulation of the Lake Ontario-St. Lawrence River System will be adaptable to reflect the potential for changes in water supply as a result of climate change and variability.

• Criteria and Regulation Plans will incorporate current knowledge, state-of-the-art technology and the flexibility to adapt to future advances in knowledge, science and technology.

These guiding principles were pursued in the Shared Vision planning and modeling processes, and they should continue in some form if the IJC is to achieve the type of long-term, comprehensive evaluation and understanding of tradeoffs among RPOs that is needed. The Shared Vision approach involves an adaptive process. While SVM documentation does not elaborate its links with the larger field of Adaptive Management, the remainder of this section briefly indicates how Adaptive Management might be used to inform future water level and flow regulation.

A recent NRC report defines adaptive management as “…a strategy that aims to create flexible resource management policies that can be adjusted as project outcomes are better understood and as stakeholder preferences change” (NRC, 2004, p. 13). Adaptive management emerged partly in response to unanticipated environmental variability, changing social objectives, and new scientific knowledge. It also addresses problems of protracted stakeholder conflict, inflexible management institutions, and unmonitored trial-and-error responses to these challenges (Gunderson 1999; Lee, 1999; NRC, 2004; Walters, 1997). Many of these issues apply to Plan 1958D: while deviations yield a practical understanding of adjustments to adverse effects, they lack formal scientific hypothesis testing, monitoring, and stakeholder input. Adaptive management strives to combine scientific analysis and stakeholder input for managing complex water and environmental systems such as the Lake Ontario-St. Lawrence River basin.

There are several approaches to adaptive management. A “passive” approach would focus on monitoring the effects of RPO decisions, along with hydrologic, environmental and social variability, and feed that evidence back to managers and stakeholders for consideration in alternative system adjustments and mitigation. An “active” approach would treat RPO decisions as quasi-experimental choices to test the behavior of the LOSLR system in ways that are hypothesized to reduce adverse impacts and/or increase net benefits. There is an increasing body of peer-reviewed scientific literature on the structure and performance of adaptive management programs in regions ranging from the Columbia River to Glen Canyon Dam, the Upper Mississippi River, and the Everglades–as well as programs in Europe (NRC, 1999, 2004). There are several common components of adaptive management programs (NRC, 2004, p. 16):

• Management objectives and performance indicators that are regularly revisited.

• A model of the system that is managed with feedback effects and exogenous driving-forces, as well as management variables.

• Consideration of a range of management alternatives and hypotheses about their possible outcomes.

• Monitoring and evaluation of outcomes.

• A collaborative structure and processes for stakeholder participation.

The LOSLR Shared Vision process prepares the way for these components of adaptive management through its studies, model development, performance indicators, supporting qualitative inquiry, technical work groups, and stakeholder processes.

Flaws in LOSLR models and studies need to be corrected or alternative models adopted prior to the establishment of an adaptive management program to avoid perpetuating existing problems. This would include representative wetland sampling, additional analysis of sandy shore environments, and replacement of regression models for flow routing and shore recession, along with a comprehensive approach to uncertainty analysis, quality assurance, and documentation. After that, an adaptive management program could progressively fill scientific gaps, analyze uncertainties, build cooperative stakeholder relationships, and improve system performance.

The challenges of implementing an adaptive management in the Lake Ontario-St. Lawrence River basin should not be underestimated (e.g., see Johnson, 1999; Lee, 1999; Walters, 1997). Models would need to be implemented in a manner that facilitates future updating at regular intervals. New models would need to be designed to incorporate environmental and socioeconomic processes currently treated as exogenous to RPO decision making. Water regulation decisions, monitoring effects, data analysis, and feedback via the models to subsequent decision making would need to be institutionalized as integral parts of the LOSLR management system. Strong stakeholder support would be needed to initiate and sustain an adaptive management program. The costs of these programs may be high. An “active” adaptive management could involve annual costs comparable to those of the LOSLR study, while “passive” adaptive management costs would be significantly lower, depending upon the scope of monitoring and management, but also less useful. Practical lessons learned in adaptive management programs in the U.S. and internationally can inform the design of a management program for the Lake Ontario-St. Lawrence River system (e.g., see the Collaborative Adaptive Management Network [http://www.adaptive-management.net/index.php]). Recommendation: In the short-term, identify adaptive management alternatives that build upon the LOSLR studies and models. In the longer term, the IJC should, in collaboration with other scientific and stakeholder organizations in the basin, develop an adaptive management program that would provide a con-

tinuing scientific basis for improving decisions about regulation plan options.

BACK 2. St. Lawrence River-Lake Ontario Plan of Study Team. September 1999. Plan of Study for Criteria Review in the Orders of Approval for Regulation of Lake Ontario-St. Lawrence River Levels and Flows.

Clinton Edmonds Associates. 2002. Lake Ontario and St. Lawrence River -Changes in the Institutional Structure and Their Impact on Water Levels, 1950-2001. Report to International Lake Ontario-St. Lawrence River Study Board. University of Ottawa, Institute of the Environment.

Economic Advisory Committee. January 7, 2004. Issues and Findings in the Economic Analysis of Water Level Management Plans: Consensus Report of the Economics Advisory Committee. Typescript.

FEPS 11. Contextual Narrative for “Sediment Budget PI” Lake Ontario and the Upper St. Lawrence River.

FEPS 12. Contextual Narrative for Erosion, Flooding, and Existing Shore Protection PIs Lake Ontario and the Upper St. Lawrence River.

FEPS 13. Contextual Narrative for “Beach Access” and “Barrier Beaches & Dunes PIs” Lake Ontario and the Upper St. Lawrence River.

Gunderson, L. 1999. Resilience, flexibility and adaptive management – antidotes for spurious certitude? Conservation Ecology 3(1): 7. Available on-line at http://www.consecol.org/vol3/iss1/art7.

IERM 4. IERM User’s Manual (two versions: a) html version with hyperlinks; and b) text version which is also Attachment B of the descriptive documentation).

International Joint Commission. 1956. Order of Approval for Regulation of Lake Ontario. October 29, 1952, as amended by a supplementary Order dated July 2, 1956. Office consolidation. Reprinted in St. Lawrence River-Lake Ontario Plan of Study Team (1999, Annex G).

Johnson, B. L. 1999. Introduction to the special feature: Adaptive management—Scientifically sound, socially challenged? Conservation Ecology 3(1): 10. Available on-line at http://www.consecol.org/vol3/iss1/art10/.

Lake Ontario-St. Lawrence River Study Board (LOSLR). 2005. Options for Managing Lake Ontario and St. Lawrence River Water Levels & Flows: Final Report. July 1, 2005 draft. Copy on file at The National Academies, Water Science and Technology Board, 500 5^th St., NW Washington, DC 20001.

Lee, K. N. 1999. Appraising Adaptive Management. Conservation Ecology 3(2): 3 Available on-line at http://www.consecol.org/vol3/iss2/art3. Accessed October 2005.

Leger, W. 2005. Position paper on conceptual narratives. Revised draft 22 January 2005; and updated 6/7/05 (LOSLR ftp site).

LOSLR (Lake Ontario-St.Lawrence River Study). 2003. Vision, Goals and Guiding Principles. Available on-line at http://www.losl.org/about/vision-e.html.

NRC (National Research Council). 1999. Downstream: Adaptive Management of Glen Canyon Dam and the Colorado River Ecosystem. Washington, DC: National Academies Press.

NRC. 2004. Adaptive Management for Water Resources Project Planning. Washington, DC: National Academies Press.

SAR 3A. Impact of Water Level Regulation on Nearshore Habitat Availability and SAR.

SAR 3B. (Supplement) Impact on dunes and SAR.

Sousounis, P. J. (Ed.). 2002. Special section on the potential impacts of climate change in the Great Lakes Region. Journal of Great Lakes Research 28(4): 493-642

Walters, C. J. 1997. Challenges of Adaptive Management of Riparian and Coastal Ecosystems. Conservation Ecology Available on-line at http://www.consecol.org/vol1/iss2/art1. Accessed October 2005.