2
Overarching Comments and Concerns

UTILITY

The Draft CCAM was developed in order to provide a tool that could help establish a carrying capacity for the Florida Keys—the maximum level of development that could be supported without damage to the natural and human resources of the area. The Draft CCAM is a valuable first step in the development of analytical tools to evaluate the impacts of development in the Florida Keys, but it is not capable of providing quantitative estimates of carrying capacity. As the committee emphasized in its interim report, it is not at all clear that it is possible to create a tool capable of credibly evaluating the impact of development on all of the Keys’ varied natural resources in such a short period of time, if at all (National Research Council, 2001).

That said, the Draft CCAM is currently the only analytical tool available that attempts to integrate quantifiable data from multiple sources and contexts to evaluate the environmental impacts of development in the Florida Keys. It incorporates information on land use, socio-economics, transportation, infrastructure, stormwater and wastewater management, and marine and terrestrial ecology. Though it is not as comprehensive as was intended, and although it does not estimate carrying capacity in the ecologically relevant sense of the term, it is nonetheless an important piece of work and has the potential to be a useful tool in managing the fragile ecosystems of the Florida Keys. Significant improvement is required in several key respects, however, if the CCAM is to be a useful impact assessment model. An impact assessment model differs from a model capable of fulfilling the mandate to determine carrying capacity—which may be beyond any



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 8
A Review of the Florida Keys Carrying Capacity Study 2 Overarching Comments and Concerns UTILITY The Draft CCAM was developed in order to provide a tool that could help establish a carrying capacity for the Florida Keys—the maximum level of development that could be supported without damage to the natural and human resources of the area. The Draft CCAM is a valuable first step in the development of analytical tools to evaluate the impacts of development in the Florida Keys, but it is not capable of providing quantitative estimates of carrying capacity. As the committee emphasized in its interim report, it is not at all clear that it is possible to create a tool capable of credibly evaluating the impact of development on all of the Keys’ varied natural resources in such a short period of time, if at all (National Research Council, 2001). That said, the Draft CCAM is currently the only analytical tool available that attempts to integrate quantifiable data from multiple sources and contexts to evaluate the environmental impacts of development in the Florida Keys. It incorporates information on land use, socio-economics, transportation, infrastructure, stormwater and wastewater management, and marine and terrestrial ecology. Though it is not as comprehensive as was intended, and although it does not estimate carrying capacity in the ecologically relevant sense of the term, it is nonetheless an important piece of work and has the potential to be a useful tool in managing the fragile ecosystems of the Florida Keys. Significant improvement is required in several key respects, however, if the CCAM is to be a useful impact assessment model. An impact assessment model differs from a model capable of fulfilling the mandate to determine carrying capacity—which may be beyond any

OCR for page 8
A Review of the Florida Keys Carrying Capacity Study modeling endeavor given the current level of understanding and data available.1 The CCAM can be used to guide professional judgement by knowledgeable experts, as long as its assumptions, uncertainties, and limitations are clearly acknowledged and understood. The CCAM can be used to evaluate some of the impacts of development on the Keys, but not with great precision due to the very nature of the techniques used and the data available. Given the policy decision to avoid new data collection as well as the short timeframe for the study, the consultants were forced to rely on standard techniques and parameters developed for general application that were not refined to reflect the unusual conditions found in the Florida Keys. As a result, much of the basic science underlying the model can be characterized as crude, and does not support the precise estimates of impacts necessary to establish impact thresholds and carrying capacity. When combined with a consideration of non-quantitative impact estimates based on professional judgement, the Draft CCAM can be used to guide decision-making. Such choices require a judicious balancing of uncertain impacts on the environment and quality of life with more certain financial consequences and impacts on individuals. Given the uncertainties involved, it may be advisable to make incremental decisions, monitor results, and proceed accordingly. The CCAM and other information can be used to guide research and data gathering efforts, the importance of which cannot be overstated. Adaptive management techniques that make use of tools like the CCAM as well as the fruits of information-gathering efforts may be the most appropriate way to ensure the continued vitality of the natural and human resources of the Florida Keys. The issues discussed below are not particular to any single module but are cut across all of the modules. Calibration Concerns Almost all of the individual models used in the Draft CCAM modules (e.g., the model that derives population estimates from land use, the model that estimates the impact of development on habitat for individual species, etc.) are highly simplified representations of the real world. When such models are used, it is customary to adjust the parameters (coefficients) in the model so that it predicts historical values of the outputs given historical values of the inputs, a process called calibration. Though the NRC Committee is generally satisfied with the form of the models used, virtually all of the modules are uncalibrated. For example, to estimate wastewater loading the Wastewater Module multiplies a coefficient by the number of dwelling units within a planning unit assumed to be served by a given wastewater management technology. The coefficient 1   See NRC, 2001 (Appendix B) Section 1 for a discussion of Carrying Capacity Model vs. Impact Assessment Tool.

OCR for page 8
A Review of the Florida Keys Carrying Capacity Study used is based on average water usage in gallons per day for the planning unit, as reported in the Monroe County Sanitary Wastewater Master Plan (Draft CCAM Appendix C; Monroe County Growth Management Division, 2000). The coefficient for each planning unit is based on averages that reflect the mix of land uses and socio-economic structure of the population within the planning units at the time they were calculated. The coefficients cannot be calibrated because they are not calculated from variables that actually determine water usage and associated wastewater generation. This does not mean, particularly in this case, that the model should not be used, or that it is inappropriate to use regional average values. It does mean that the model is uncalibrated and that the predictions made by the model are not precise and they may be biased toward high or low values. Incomplete Knowledge Base Lack of data and/or science has limited the ability of the Draft CCAM to represent a wide range of important processes, so in this sense the model is incomplete. Important but unmodeled processes exist in virtually every module. These include but are not limited to: the impact of external demand for vacation properties on land values, impacts on water quality in canals, explicit consideration of the impact of tourist activities, consideration of external sources of pollution loadings, and any consideration of marine biology. This does not invalidate the Draft CCAM, but it does mean that its results must be weighed against common sense and historical experience. Model Uncertainty Sensitivity testing is critical when attempting to use uncalibrated models because it allows the user to assess the amount of change in predictions induced by changing assumptions about parameters (coefficients) within reasonable ranges. If the sensitivity to the change in a parameter is small, then confidence in the model’s predictions increases. Conversely, if sensitivity is large, then further research is warranted in order to improve the accuracy of the estimate for that parameter. The Draft CCAM included no sensitivity testing. While it is possible that some testing occurred, a full sensitivity testing would require a very large effort, as the CCAM is very complex and involves many parameters, many of which would need to be varied concurrently. Since the run time for a single scenario is about two days, full sensitivity testing probably lies beyond the scope of the current effort. The Draft CCAM’s imprecision makes it important that the output displays produced by the model be designed to avoid communicating a false sense of precision. This task is made more difficult by the lack of sensitivity testing of the underlying models. The application of threshold values to CCAM output implies a precision that simply does not exist. For example, if a minimum patch size of

OCR for page 8
A Review of the Florida Keys Carrying Capacity Study 50 acres were the threshold for the survival of a species, then a patch would be coded red at 49.5 acres and green at 50.5 acres. The policy implied by those codings is quite different, despite a gap of only one acre: in the case of the latter, the patch would be added to the bird’s habitat, but not in the former. The nature of the models makes it impossible to establish that birds would in fact, colonize one site but not the other. To avoid the appearance of precision, continuous color shading could be applied to graphical displays or a weighting scheme could be devised to partially value habitats and other endpoints that fall at or near thresholds. Regardless of the method used, the uncertainty inherent in the modeling must be reflected in the displays of the results and should be fully described for each module. Endpoints Not Considered A number of endpoints (e.g., areal extent of propeller scarring, direct impacts on the coral reef, canal water quality, etc.) were proposed during briefings at a workshop in January 2001, but were not included in the Draft CCAM. Many (perhaps all) of these were dropped due to insufficient scientific basis for their inclusion in a quantitative analysis, which is well and good. Lack of ability to quantify impacts does not imply that useful information is unavailable, however. There needs to be recognition of the importance of these excluded endpoints, so that those who use the model can seek expert opinion, or otherwise consider those factors in decision making. A complete listing of the endpoints considered but not included and the rationale for excluding those endpoints should be a prominent part of the final CCAM documentation. Draft CCAM Conclusions Extreme caution is required when portraying Draft CCAM results as evidence of what would actually happen if a particular land use alternative should actually come to pass. The uncalibrated nature of the model and its lack of precision make it impossible to draw firm conclusions, particularly when those conclusions are called into question by existing data or anecdotal evidence. This does not mean that it is impossible to draw any conclusions. With implementation of the Monroe County Sanitary Wastewater Master Plan (Monroe County Growth Management Division, 2000), for example, it is quite reasonable to conclude that pollution loads to surrounding waters will be substantially lower in the long run than they would be if current conditions persist. It is important to note, however, that in the short run the background level of pollution due to past human activity may cause further deterioration. How significant such deterioration might be and how long it might persist is simply not known. It is not possible to conclude that current or projected levels of development do not or will not have significant impacts on the marine environment. Model

OCR for page 8
A Review of the Florida Keys Carrying Capacity Study results do indicate that the incremental contribution of some pollutants from land based sources should be small relative to background loads. This outcome contradicts evidence of beach closings due to elevated fecal coliform counts, historical measurements of decreased visibility for divers on the reefs, and trends in incidence of coral disease. Existing science is simply inadequate to establish a clear cause and effect relationship between land development in the Keys and these trends. Draft CCAM model results are, however, inadequate to allow the user to draw the conclusion that there is currently or will be no substantial impact. In general, model results must be considered in the context of historical experience. The uncalibrated nature of the model, its lack of completeness in terms of both processes and endpoints, and the lack of sensitivity testing of the model results must temper any conclusions to be drawn from the Draft CCAM. Documentation There are a great many editorial issues that must be dealt with by the contractor in both the text and appendix components of the Draft CCAM. Perhaps the most important is that the model logic as described in the documentation (particularly the appendices) does not match the logic as described by the contractor. The documentation clearly needs to be reviewed in order to ensure conformity with the actual model calculations. There are three general issues in relation to citations that are of concern to the NRC Committee. First, many citations are non-peer-reviewed documents that are almost impossible to acquire, and the committee expressed concern over the confidence placed in such documents. Some of those documents are not cited in the Literature Cited section and were not made available to the NRC Committee. Second, there are a number of documents noted in the answers to the NRC Committee’s questions (Appendix D) that were never seen by the committee (e.g., Delivery Order Reports). Finally, literature is poorly cited in general. A large number of citations throughout the text and appendices of the Draft CCAM cannot be found in the Literature Cited section. This is particularly true in the Appendices, where some are cited incorrectly. These citation issues need to be corrected in the final version of the CCAM report to provide maximum guidance to future users of the document. Besides the issues of citations discussed above, there is also a need for clear documentation of terminology and nomenclature. In the Draft CCAM, for example, any parcel of land with a structure on it is considered developed, regardless of whether structures occupy the entire parcel or only a fraction of the land area. The term “footprint” is used to depict the entirety of this developed land. If an undeveloped parcel is fitted with a single house or a shopping center, the entire parcel is included in the increased “footprint”; however, if an obsolete gas station is replaced with a strip mall, the “footprint” of the parcel does not change. Such

OCR for page 8
A Review of the Florida Keys Carrying Capacity Study use of the term “footprint” does not match the standard usage of the term in the design professions,2 which may confuse the reader. Another example is the use of the term “polygon,” a Geographic Information System (GIS) term that might be familiar to practitioners, but not to a general reader, such as a decision-maker or activist especially interested in the CCAM. Terms like these should be defined more clearly and more broadly if they are to be used as an instrument that may become the basis for development control. 2   In the design professions, the term footprint normally describes only that area of a site that is actually covered with a structure.