2

Specific Comments

ASSESSMENT END POINTS

Successful monitoring programs ultimately must be question-driven. In EMAP these questions are implicit in the designation of “assessment end points,” which reflect societal values and delineate the broad issues to be addressed with the monitoring data. EMAP-Surface Waters has designated three assessment end points for the lakes portion of their program:

  • biological integrity

  • trophic condition

  • fishability.

The choice of these particular end points was justified in the Research Strategy document on the grounds of their societal relevance and their relationship to environmental problems already of interest to EPA.

The choice of assessment end points provides the foundation for the EMAP Lakes Program. This first step, therefore, is of critical importance. This committee is troubled by EMAP-Surface Waters' current selection of end points for several reasons.

  • They do not specifically address several lake issues about which there is serious and widespread public and scientific concern (for example, acidification, salinization, exotic introductions, aesthetic quality, swimmability).

  • Not all of them are amenable to precise scientific definition.

  • The three end points are dissimilar in breadth conceptually.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 15
Review of EPA's Environmental Monitoring and Assessment Program: Surface Waters Monitoring Component 2 Specific Comments ASSESSMENT END POINTS Successful monitoring programs ultimately must be question-driven. In EMAP these questions are implicit in the designation of “assessment end points,” which reflect societal values and delineate the broad issues to be addressed with the monitoring data. EMAP-Surface Waters has designated three assessment end points for the lakes portion of their program: biological integrity trophic condition fishability. The choice of these particular end points was justified in the Research Strategy document on the grounds of their societal relevance and their relationship to environmental problems already of interest to EPA. The choice of assessment end points provides the foundation for the EMAP Lakes Program. This first step, therefore, is of critical importance. This committee is troubled by EMAP-Surface Waters' current selection of end points for several reasons. They do not specifically address several lake issues about which there is serious and widespread public and scientific concern (for example, acidification, salinization, exotic introductions, aesthetic quality, swimmability). Not all of them are amenable to precise scientific definition. The three end points are dissimilar in breadth conceptually.

OCR for page 15
Review of EPA's Environmental Monitoring and Assessment Program: Surface Waters Monitoring Component The Problem with Biological Integrity Of the three end points, biological integrity is the most problematic. As used by EMAP-Surface Waters this term is vague and all-inclusive, conceptually subsuming the content of the other end points and all other more specific environmental problems in lakes. Such a broadly defined term may be useful in summarizing diverse data or in addressing the multiple issues related to environmental quality, but it is not specific enough to be a useful end point upon which to design data monitoring activities. This point has already been made to EMAP-Surface Waters by its Peer Review Committee and we strongly concur. Although the term biological integrity seems intuitively appealing, and is part of the language of the 1972 Clean Water Act, there is no satisfactory technical definition of biological integrity. EMAP-Surface Waters defines it on page 9 of its 1991 pilot report as: “. . . the ability to support and maintain a balanced, integrated, adaptive community with a biological diversity, composition, and functional organization comparable to those of natural lakes and streams of the region (Frey, 1977; Karr and Dudley, 1981) and [it] includes various levels of biological, taxonomic and ecological organization (Noss, 1990).” This definition as written doesn't easily lead to useful, objective indicators, although it has a useful kernel. The phrase, “ability to support and maintain a balanced, integrated, adaptive community,” for example, defies technical interpretation. “Ability to support and maintain” is not an attribute of an ecosystem or EMAP resource class, but of our (human society's) interaction with ecosystems and resources. Furthermore, the meaning of the term “integrated” is obscure. Integrated with respect to what? All communities, whether altered by human action or not, integrate energy and material fluxes. Likewise, EMAP's use of the term “adaptive” has no straightforward interpretation. Ecologists typically do not speak of communities as being “adaptive” in an evolutionary sense, since community characteristics are not traits of the unit upon which natural selection acts (generally the individual, rarely the population). It is true that all communities may “adapt” in the

OCR for page 15
Review of EPA's Environmental Monitoring and Assessment Program: Surface Waters Monitoring Component sense of changing in response to selective pressures on component populations. This may lead (at least in theory) to highly specialized communities adapted to very specific physical environments. Such communities, however, are usually thought of as not being very “ adaptive” (i.e., resilient) to major shifts in the physical environment. Communities that are most “adaptive” in the sense of being resilient tend to be comprised of generalists and early successional species (commonly called weedy species) and would likely be considered by EPA to be communities with reduced biological integrity. The idea that “balance” is a desirable characteristic of aquatic communities, although widespread in popular environmental literature, is also problematic in its technical implications. Expanding on the terminology of Nicholson (1933), Swingle (1950) introduced the concept of balanced fish communities for application in small artificial pond systems. He proposed that predator and prey populations were in balance when the distribution of biomass across trophic levels maximized the ratio of predators to prey. This was of interest to Swingle and others managing small pond ecosystems because it provided a simple model for facilitating the exploitation of predator stocks (bass in Swingle's papers). It is not at all clear that maximizing energy transfer to predators is a universally desirable ecosystem attribute. Many pristine (i.e., unaltered) aquatic ecosystems are notably inefficient in this regard (e.g., prairie pothole wetlands, southern grayling streams, bog lakes). The useful kernel of EPA's definition is: “[a] community with a biological diversity, composition, and functional organization comparable to those of natural lakes and streams of the region.” This statement provides a clear definition of a quantifiable ecosystem attribute that would provide an additional useful and distinct assessment end point. The emphasis should be on measurable attributes of community structure, notably composition and diversity. In this context, “natural” is equated with the absence of human alteration and impact (Karr et al., 1986). Detecting alterations due to human activity requires both an assessment of the biological community present and a calibration against appropriate regional or historical controls. The assessment question becomes whether the community has a diversity

OCR for page 15
Review of EPA's Environmental Monitoring and Assessment Program: Surface Waters Monitoring Component and composition appropriate to its place in the landscape. Karr's Index of Biotic Integrity (IBI) (Fore et al., 1994) for stream fishes, which EMAP is testing for use, is an excellent example of a condition indicator that addresses an “appropriate diversity” end point. The IBI scores sites in terms of deviations from an expected composition based on regional controls and corrected for landscape position (stream order). Although the committee believes that “appropriate biological diversity” is a more useful assessment end point than “biological integrity”, we recognize that the application of the concept may still present some operational difficulties. The successful use of appropriate biological diversity will depend upon the occurrence and identification of suitable reference systems. It is possible that in degraded regions of the country no suitable reference system currently exists. Paleolimnological reconstruction of reference systems may be possible for well preserved taxa, such as diatoms, and we encourage EMAP Surface Waters to continue its work in this area. Another difficulty in implementing this concept that no single index is likely to work well over the entire nation, even if regionally specific reference systems are available. It is possible that the individual components of an index will need to be different from region to region, which will make comparisons among regions very difficult. The committee recommends that EMAP-Surface Waters (and the other resource groups) use the term “appropriate biological diversity”, defined above instead of “biological integrity” as an assessment end point. Even with the difficulties associated with implementing the concept of appropriate biological diversity, it is a clearer, more concise attribute, about which the public is obviously concerned (note the media attention given to the “biodiversity” crisis). It does not strongly overlap the other assessment end points EMAP-Surface Waters has proposed, and it provides a less ambiguous basis for the design of a monitoring program. Clearly, developing suitable indices for this assessment end point will require a major research effort.

OCR for page 15
Review of EPA's Environmental Monitoring and Assessment Program: Surface Waters Monitoring Component Fishability and Trophic Condition The other two assessment end points for EMAP-Surface Waters are trophic condition and fishability. In theory, each could be defined in reasonably unambiguous ways, and straightforward means can be developed to measure them quantitatively. Nonetheless, the committee concludes that further efforts are needed to refine the definitions and measurement strategies for both end points. EMAP-Surface Waters defines trophic condition (trophic state) as “the abundance or production of algae and macrophytes.” However, macrophyte problems in lakes often are quite distinct from issues of eutrophication, and measurement techniques (indicators) for these problems also are quite different. The committee suggests that for ease of measurement trophic condition should be defined primarily in terms of algal-related problems (and measured by the conventional trophic indicators: chlorophyll a, nutrient concentrations, and lake transparency). Macrophyte problems should constitute a separate assessment end point. These problems most commonly include invasions of exotic species, community loss and/or excessive production due to extreme shallowness. Fishability is defined by EMAP-Surface Waters to mean the “catchability and edibility of fish by humans and wildlife.” There are many potential and real problems with this definition. First, although both catchability and edibility are of concern to the public, it is not obvious that they can be combined in any meaningful way to provide a single index of fishability. Second, there are many other ways to define fishability, and the term will be subject to confusion and misinterpretation by the public and by aquatic scientists. This is true even if EMAP-Surface Waters clearly states its own definition. For example, public access is as important in defining fishability from an angler's perspective, as are questions related to the presence or abundance of gamefish. Also, fishability could be high as a result of supplementation by hatcheries, and thus any interpretation of the condition of the ecosystem based only on fishability might be misleading. Finally, the limited sampling protocol to be used in EMAP-Surface Waters precludes quantitative measurements of fish populations (age/size

OCR for page 15
Review of EPA's Environmental Monitoring and Assessment Program: Surface Waters Monitoring Component distribution, abundance by species). Instead, EMAP-Surface Waters will only be able to provide data on presence or absence of gamefish species and perhaps rough estimates of relative abundance. The committee questions whether the public (and decision makers) will find such limited information useful as a measure of fishability. The committee recommends that EMAP confine its assessment of fish communities to the issue of edibility (i.e., would consumption of fish from a lake constitute a health threat?). If EMAP-Surface Waters decides to obtain presence/absence or relative abundance data on fish species, it at least should select a less value-laden term for this information than “fishability.” There are other key issues (some regional, some national) regarding the ecological status of lakes that are not directly addressed by the current assessment end points. We recommend that a small number of precisely defined end points be added to more fully represent the current array of concerns facing lake management organizations. A useful starting point for selecting these assessment end points for lakes is the 13 classes of use-impairment defined by the EPA in its report to Congress under Section 314 of the Clean Water Act (U.S. EPA, 1989). These classes address such problems as acidification, sedimentation (particularly a concern in reservoirs), organic/heavy metals/toxic contamination, exotic species, and habitat damage. Most of the above-mentioned use-impairment classes represent categories of stress rather than explicit societal or ecological values. EMAP personnel recently have argued (U.S. EPA, 1994) that these classes are not appropriate as value-driven assessment end points. Instead, they contend that statements about these categories of stress will be developed when EMAP begins “to develop associations between biological integrity and possible causes for impairment of biological integrity ” (U.S. EPA, 1994, p. 15). Although the committee accepts the general logic of this statement (that is, values are distinct from causes of value impairment), EMAP's approach of restricting its assessment end points to the three described above nonetheless is likely to lead to an inadequate assessment program for at least three reasons:

OCR for page 15
Review of EPA's Environmental Monitoring and Assessment Program: Surface Waters Monitoring Component As discussed above, the concept of biological integrity is at best fuzzy and not subject to quantitative interpretation. The selected end points (biological integrity, trophic condition, fishability) still represent only a subset of the important societal values associated with surface waters. These end points do not directly address aesthetic quality, “swimmability,” and suitability as a drinking water supply, for example. Of course, it can be argued that trophic condition addresses all these values—a highly eutrophic lake with frequent, dense blooms of bluegreen algae has poor aesthetic quality, is not desirable for swimming, and poses economic and potential public health problems for use of the lake as a drinking water supply. However, there are other causes of impairment for these three values besides trophic condition (e.g., high turbidity caused by inorganic suspended sediment, microbial contamination, presence of toxic substances). The list of assessment end points thus needs to be expanded to be more inclusive of the important societal and ecological values for which we wish to protect our surface water resources. Unless the potential causes of value impairment are stated explicitly by EMAP as part of the design process for its monitoring program, there is a high likelihood that critical variables needed to make the association between value impairment and cause of impairment will not be included in the monitoring protocol. The committee is especially concerned about this issue in view of the fact that EMAP has recognized that it cannot measure every possible water quality variable and has reduced the number of variables it measures as it initiates and conducts pilot studies. Regarding the distinction between values and causes of use-impairment, it also might be argued that trophic condition is more a cause of impairment than an explicit societal value. The real societal values associated with trophic condition are swimmability, aesthetics (water clarity, presence/absence of odors), and fishability. The first two values are diminished monotonically with increasing eutrophic status, but the last value has a more complicated relationship. Warm-water fisheries are enhanced with increasing nutrient status up to fairly high levels of eutrophic status and then decline as habitat becomes degraded and rough fish assume dominance. The committee is not arguing that tro-

OCR for page 15
Review of EPA's Environmental Monitoring and Assessment Program: Surface Waters Monitoring Component phic condition should be eliminated as an assessment end point; it merely is making the point that the distinction between societal/ecological values and causes of impairment is imprecise. Finally, the committee does not accept the statement made by EMAP that drinking water usage is not within EMAP's scope (U.S. EPA, 1994, p. 15, lines 10-11). In an era where the concept of holistic watershed planning and management is becoming increasingly important, this is a curious position to take. The EPA has espoused the importance of holistic watershed management (e.g., Wayland, 1993), and when the Clean Water Act is reauthorized, it probably will emphasize this approach. Many impoundments and natural lakes are used both for recreational purposes and public drinking supplies. Although EMAP's financial resources will always be limited, the committee believes it would be imprudent to exclude drinking water a priori from consideration as a societal value in its surface water assessment program. This is an example where much closer cooperation between EMAP and the EPA Office of Water's 305b program (see section on Inter- and Intra-Agency Cooperation) could be highly beneficial to both programs. INDICATORS Selection Criteria and Procedures Once the major assessment end points have been decided, the next critical task is to determine what measurements are necessary to assess these end points. It is in this step that science is infused into the process. Once the problem has been stated, a conceptual model of how the particular system works with respect to the end points should be stated explicitly. Examination of the conceptual model then leads to the selection of candidate indicators, which are tested in the field. The indicators are selected on the basis of known or suspected cause-effect relationships that are identified in the conceptual model. Unfortunately, until March 1994, there were no satisfactory programwide guidelines for indicator selection strategy, and each resource group was left to fend for itself with little or no guidance from

OCR for page 15
Review of EPA's Environmental Monitoring and Assessment Program: Surface Waters Monitoring Component EMAP-Center. As a result, use of conceptual models to drive indicator selection is not well developed in EMAP. The conceptual model implicit in the EMAP-Surface Waters strategy document underestimates the complexity of freshwater ecosystems. An example is the statement: “Much variability in aquatic species composition and abundance not explained by physical habitat is likely to be associated with natural and anthropogenic differences in water quality…” (U.S. EPA, 1991, p. 68). There is no real discussion in the document of factors like biogeography, seasonal shifts in community structure associated with secondary nutrient depletion, competition, predation, or hydrologic factors. To simply note, as the document does, that all of these will contribute to additional spatial and temporal variance is quite an understatement. As pointed out in our previous report (NRC, 1994), an explicit conceptual model is needed to guide indicator development; without such a model, the indicator selection process is likely to proceed in a haphazard manner. In EMAP-Surface Waters, success of indicator selection has been mixed. EMAP-Surface Waters has selected appropriate pelagic measurements for the trophic condition end point. An index based on chlorophyll a concentration, Secchi disk depth, and nutrient concentration is used for pelagic condition. These parameters have been used as indicators of trophic condition over many years of research by the scientific community. Relationships among these parameters and between trophic conditions and nutrient loading rates are relatively well established. Moreover, the parameters are relatively easy and inexpensive to measure. The underlying conceptual model of eutrophication is so well known among limnologists that choices of measurement parameters were relatively straightforward. Success in selecting appropriate indicators for the other two assessment end points, fishability and biological integrity, has been more elusive. In the Lakes Pilot Study, EMAP-Surface Waters used multiple gears (gill nets, trap nets, minnow traps, eel pots, beach and short seines, and boat electrofishers) to estimate fish species composition and abundance. In addition, subsets of the catch were analyzed for about 45 contaminants. EMAP-Surface Waters used the initial pilot project to examine how best

OCR for page 15
Review of EPA's Environmental Monitoring and Assessment Program: Surface Waters Monitoring Component to deploy these gear types to maximize the number of species caught while minimizing sampling time. Sampling fish communities is difficult because of the biases of each of the sampling gears, and estimating species abundances is very difficult unless substantial effort is made on each lake. The emphasis on abundance may be misplaced. Population sizes of fish are known to be highly variable and are influenced by many factors including life history traits, development of strong year classes, water quality, weather, and angling pressure. Because EMAP-Surface Waters is able to perform only a limited sampling on each lake, we recommend that it focus on determining presence or absence of species rather than on estimating abundance. Determining what species are present and whether the gamefish present are edible would still provide useful information and simplify the sampling process. The committee is very concerned about indicator selection for the biological integrity end point. A strong conceptual model does not exist to guide choice of indicators for this end point. As a result EMAP-Surface Waters has chosen to measure characteristics of sedimentary diatoms, zooplankton, macroinvertebrates, fish, and riparian bird communities. At this point it is uncertain how EMAP will use these data to determine which lakes have biological integrity and which do not. It appears that EMAP will attempt to create Indices of Biological Integrity (IBI) for each of these community types. Too much emphasis is placed on future research to develop IBI-type indices for all surface waters. Karr's IBI works well in species-rich warm-water stream systems because traditional water quality parameters are extremely variable due to changing flow rates. The IBI approach may not be as useful in low-diversity streams nor in more hydrologically stable lentic systems. As stated previously, the committee recommends that EMAP-Surface Waters use appropriate biological diversity instead of biological integrity. Evaluation of Indicator Data The committee shares the concern raised by the EPA Surface Waters peer review panel regarding heavy reliance on indices with

OCR for page 15
Review of EPA's Environmental Monitoring and Assessment Program: Surface Waters Monitoring Component unknown properties. Use of indices to describe complex ecosystems has some advantages but also some important disadvantages. The major advantage is the ability of an index to condense many parameters into a single number, which at first glance may be easier to understand. A major disadvantage is that the statistical properties of the index are often not well understood and the indices are often nonlinear; that is, a change from 1 to 2 is not the same as a change from 2 to 3. Other approaches (including multivariate statistical analyses) are also being considered by EMAP-Surface Waters, and the committee recommends that these other approaches should be pursued. Rather than relying upon a univariate index with unknown statistical properties, it is possible to use the multi-response vector of the original parameters and apply multivariate statistical techniques for analysis (e.g., logistic regression, clustering and pattern recognition algorithms, neural network analysis), or exploratory data techniques involving better visualization of multidimensional data (Becker et al., 1987, Cleveland and McGill, 1988). Nonparametric multivariate procedures also exist (as in Zimmerman et al., 1985) for testing whether groups of multivariate data points are significantly different from each other (e.g., comparing disturbed to undisturbed areas). The committee recommends that EMAP-Surface Waters continue its efforts to develop indices using a number of different approaches including multivariate statistical and exploratory data analyses. If EMAP-Surface Waters is going to use indices, then the statistical perspectives for each index must be investigated as Fore does for Karr's Index of Biological Integrity (Fore et al., 1994). SAMPLING DESIGN The design for the Surface Waters component follows the overall EMAP design. The scheme for lakes is better developed than that for streams, which has not received detailed consideration to date. About 3,200 lakes nationwide will be selected using a probability-based sampling scheme. A different subset of 800 of them will be sampled each year so that every lake is sampled once every four years. Lakes will be stratified into size classes so

OCR for page 15
Review of EPA's Environmental Monitoring and Assessment Program: Surface Waters Monitoring Component Lack of statistically sound hypothesis evaluation. Because there was no replication of reference lakes, most of the analyses depended upon an arbitrary, nonstatistical test of the hypothesis that a lake differs from the appropriate control. EMAP-SW concluded that a response indicator was substantially different if it had a value < 1/2 or > 2 times the reference value, and it was clearly different if it had a value of 1.5 to 2.0 times the reference or 0.5 to 0.75 times the reference. These criteria were apparently applied to potential indicator variables irrespective of inherent scaling differences or variability! Furthermore, overall evaluation of the proposed indicators and of relative disturbance in each lake was based upon the degree to which summed indicator metrics (individually based on deviations from the reference) deviated from sums for the reference lake in each class. It is not at all clear that any meaningful signal remains in these aggregated values. Lack of any quantitative comparison between the various indicator variables measured. There was no discussion and apparrently no analysis of the correlation between various candidate indicators. Pilot Analyses of Sources of Variability The EMAP-Surface Waters group should be commended for its investigation into the critical ways different sources of variation will affect EMAP's ability to detect status and trends. Data on Secchi-disk transparency, total phosphorus concentrations, and chlorophyll a levels were obtained from existing studies, and statistical analyses were performed to determine the minimum trend that could be detected in a 12-year period if the trends were: (a) regional in scale and (b) monotonic (i.e., in the same direction over that period). This work was described in chapter 4 of the 1991 pilot study, and several manuscripts on this work have been prepared (Urquhart et al., 1993; Larsen et al., in review). The ability to detect trends depends on such factors as the number of lakes sampled each year (N = 50 was used for the analyses reported in the pilot study), the variability among lakes in a given year (variance across lakes), the variability in the mean

OCR for page 15
Review of EPA's Environmental Monitoring and Assessment Program: Surface Waters Monitoring Component value for a population or subpopulation of lakes over all years in the absence of a trend (variance across years), and various interaction terms. Results of the analysis indicated that a trend of about 1 percent per year (0.05 m/year) could be detected in the regional mean for Secchi transparency over a 12-year period if the sample size was 50 lakes per year. For total phosphorus, trends of 2 to 3 percent per year in the regional mean could be detected under the same circumstances. However, because the between-year variance was high for chlorophyll a, trend detection capability was much poorer; and regional trends of 3 to 4 percent per year could not be detected. The result for chlorophyll deserves attention. It illustrates the different emphasis EMAP places on two components of variance: variability among lakes in a given year and variation (across all lakes) between years. A high value for between-year variance for a particular parameter indicates that lakes are behaving similarly regionally with respect to that parameter (U.S. EPA, 1993a, p. 121). That is, if one lake has higher than average chlorophyll concentration in one year, other lakes in the region are also likely to have higher than normal concentrations. High between-year variance reduces the ability to detect regional trends. However, this type of “coherent” behavior (Magnuson et al., 1990) among lakes, whether or not it follows a monotonic trend, is of interest. The 1991 pilot report states, “[T]his common pattern of variation among lakes [high between year variances] is caused by regional-scale factors affecting the population in a consistent way, such as regional-scale climatological conditions” (U.S. EPA, 1993a). Variables that exhibit this type of regional coherence have the potential to be excellent regional indicators, especially if factors controlling these variables are likely to exhibit trends. The Surface Waters group should be encouraged to do more analyses along the lines described in the pilot study report and by Urquhart et al. (1993) and Larsen et al. (in review). It is especially important to conduct such studies on biological indicators, which generally have larger variance structure than Secchi-disk transparency. In addition, studies need to be done to relate the ability to detect a regional trend of a given magnitude to the likelihood that such trends would actually occur in regions

OCR for page 15
Review of EPA's Environmental Monitoring and Assessment Program: Surface Waters Monitoring Component of the size and heterogeneity on which EMAP intends to make trend analyses. In the above example for Secchi-disk transparency, a trend of 1 percent per year amounted to a change of 0.6 m in the average transparency over 12 years across the lake population. Is this a reasonable value to be able to detect? Limnologists generally would be satisfied if they could detect a trend of this magnitude for a single lake. However, not all the lakes in a region are likely to have a trend over any reasonable period of record, and some will have trends in opposite directions. Therefore, some lakes will need to change much more than 0.6 m if the minimum detectable 12-year trend region-wide is 0.6 m. The committee recognizes that trend-detection capabilities depend on sample size as well as the variance of the data, and that more than 50 lakes will be sampled in a region for full-scale implementation of EMAP. The above questions need to be addressed for the sample sizes and sampling strategies that EMAP proposes to use at full-scale implementation. STREAMS EMAP-Surface Waters also is conducting a pilot program on streams. Field work on small streams in the mid-Atlantic highlands was initiated in summer of 1993. It is difficult for the committee to evaluate this pilot study and EMAP's overall plans for a stream monitoring and assessment program because we have received little written information about them. Some members of the committee were given a short oral briefing on plans for stream studies at a meeting with EMAP scientists in August of 1992. EMAP also sent the committee two published papers (Kaufmann et al., 1991; Herlihy et al., 1991) describing a stream-acidification survey conducted as part of NAPAP (National Acid Precipitation Assessment Program). These papers describe the random sampling design being proposed for the stream program. In March 1994, the committee received a brief draft report (U.S. EPA, 1994) intended to describe the basic goals and general design of a regional-scale stream study that began in the mid-Atlantic highlands in 1993. This study, the Mid-Atlantic Highlands Assessment (MAHA) project, is described as a “combina-

OCR for page 15
Review of EPA's Environmental Monitoring and Assessment Program: Surface Waters Monitoring Component tion of three distinct but related activities: the base EMAP-Surface Waters stream pilot, Region III R-EMAP project (biological criteria in streams of the Ridge and Valley region), and the TIME project (assessment of streams sensitive to acidic deposition in response to the 1990 Clean Air Act Amendments)” (U.S. EPA, 1994). Because of the late date on which this document was received, the committee cannot provide a detailed analysis of it in this report. However, the committee has read and reviewed the information provided in that document, and it is part of the basis for our conclusions and recommendations regarding EMAP's stream studies. Based on the above limited information, we find the justification for stream pilot surveys to be inadequate for several reasons. First, the design for streams apparently is not really intended to sample river ecosystems, but rather channel segments. The classification of reaches within the sampling hexagon as large or small is a poor representation of what is in reality a size continuum easily described in terms of drainage basin area. Furthermore, the stream sampling design treats all segments as if they were independent units. There is no recognition that segments within the same drainage network are related. Second, there is no written document that defines the overall (national) objectives, assessment end points, sampling design, and indicators to be measured in stream surveys as there was in the strategy document (U.S. EPA, 1991a) for the lake monitoring and assessment phase of EMAP-Surface Waters. The draft report on the MAHA project (U.S. EPA, 1994) is not adequate as a strategy document for a national stream program. Moreover, this document seems to have been developed in a post facto manner rather than as a strategy document for the pilot study (MAHA) it describes. Third, the committee is not convinced that EMAP actually has developed a vision and a strategy for what it should achieve in a program to monitor and assess the status and trends of the nation's running waters. Based on the information now available, the committee concludes that it would be premature to continue stream pilot studies at this time. The committee recognizes that scientists “learn by doing” and that complicated programs like EMAP gain experience and can be improved through such exercises. Nonetheless, the

OCR for page 15
Review of EPA's Environmental Monitoring and Assessment Program: Surface Waters Monitoring Component committee concludes that insufficient planning has been done regarding a variety of critical issues for the stream survey and that the large amount of money and substantial energy required for pilot programs should not be invested at this time. One of the benefits of a pilot study is that it can provide valuable logistical information, i.e., is the proposed field sampling protocol feasible? Can the field and laboratory work be conducted according to the necessary standards and be done in a timely and cost-effective manner? Another benefit of pilot studies is in providing real data to analyze and interpret. The committee has no doubt that scientists within EMAP-Surface Waters would be able to take the data from a pilot survey (even one conducted at present), analyze them by a variety of statistical techniques, and develop (post facto) a reasonable interpretation of chemical and/or biological water quality trends. The committee doubts, however, that such analysis and interpretation would be very useful for the development of a full-scale stream program, given the current lack of clear objectives. The committee does not believe that the currently conceived sampling strategy is appropriate to characterize stream quality either chemically or biologically. The proposed plan (sampling sites at the upper and lower ends of a stream segment during some as yet unspecified index period) ignores basic principles of stream ecology—the continuum concept, hydraulic-geometry relationships, effects of variability in hydrologic source and flow on chemical/biological conditions, etc. The approach seems to be to sample enough sites to sort out all the potentially confounding factors of instream and riparian habitat, watershed land use, differences in soils, basin geochemistry, etc. This may work in theory, but in practice it leads to a highly inefficient design—many sites must be sampled to compensate for not establishing objective sampling criteria based on fundamental understanding of stream ecosystems. The number of randomly selected sampling sites needed to sort all these things out may be so large as to be impractical or prohibitively expensive. The two papers on the National Stream Survey (NSS) (Kaufmann et al., 1991; Herlihy et al., 1991) represent a sound application of science to address a practical question, and they

OCR for page 15
Review of EPA's Environmental Monitoring and Assessment Program: Surface Waters Monitoring Component demonstrate that it is possible to make reasonable inferences about factors affecting stream water chemistry (i.e., geochemical and anthropogenic mechanisms influencing stream water chemistry) from a large data base. They also demonstrate the feasibility of making eco-regional assessments of stream chemical conditions based on a large, randomly based sampling program, which was the primary objective of the NSS. One should not be misled, however, into thinking that the apparent success of the NSS can be translated into a similarly successful EMAP stream program or that the NSS results provide justification for conducting pilot-scale EMAP stream surveys. They cannot and do not. If anything, the NSS experience provides a strong rationale for not conducting pilot-scale EMAP stream studies at this time. The NSS stream survey was a narrowly focused effort that had clearly specified objectives and used widely accepted, straightforward chemical parameters to measure its assessment end points. Sampling was confined to a relatively narrow geographical region in which there was reason to expect spatial variations in the response variables. Almost none of these conditions apply to pilot studies that would be undertaken in preparation for a full-scale assessment of the nation's running waters. Objectives (assessment end points) are poorly defined, and a rational selection of measurement variables is not possible in this circumstance. The crucial questions of spatial heterogeneity in sampling reaches and temporal variability at a given site cannot be addressed adequately in the pilot study if the objectives and appropriate measurement parameters are not clearly understood. The question, If you don't know what you are looking for, how will you know when you have found it? would seem to be applicable here. The committee does not believe that EMAP has decided what its overall objectives are for assessing the status of the nation's rivers and streams. These objectives—and a strategy to achieve them—need to be developed within the context of existing federal monitoring programs. That is, what can EMAP contribute uniquely that is missing from such programs as the USGS NAWQA program, the EPA 305b program, monitoring work of the U.S. Fish and Wildlife Service, and the new National Biological

OCR for page 15
Review of EPA's Environmental Monitoring and Assessment Program: Surface Waters Monitoring Component Survey? In addition, how can EMAP cooperate and coordinate with these programs to leverage its own funds and resources? The objectives and strategy also must recognize the financial limitations of EMAP; it is unlikely that EMAP ever will be able to monitor all the kinds of resource groups it has defined in sufficient detail to satisfy everyone. It will need to set priorities, but the committee has seen little evidence that this has been done. With one exception (described below), it is unclear to what extent there has been substantive involvement of the larger scientific community in the planning done to date. It is crucial to involve scientists from a variety of disciplines—hydrologists, fisheries biologists, benthic invertebrate specialists, aquatic chemists, and ecologists —in planning the stream program of EMAP. Stream scientists in other branches of EPA, in other water-related federal agencies (USGS, USFWS, etc.), and in the academic community should be consulted to evaluate what should be done, what can be done (given resource constraints), and how the stream phase of EMAP should be designed. The committee received no briefings from EMAP personnel at the seven committee meetings held in 1992, 1993, and to date in 1994 related to the formation of a peer review team or the organization of a workshop to address the planning of EMAP stream studies. The committee thus was surprised to learn in the recently supplied draft document on the MAHA project (U.S. EPA, 1994) that a workshop on streams was conducted in February of 1992 and a report of the workshop was issued in July of 1993 (U.S. EPA, 1993b). The committee finds it regrettable that it did not receive this report until May 1994. The workshop included 27 participants from organizations outside of the EPA, one EPA official from the Office of Water, and an unstated number of EMAP scientists (U.S. EPA, 1994). Several of the outside scientists are well known for their expertise in stream ecology. The committee is unable to comment on the role this workshop and its report may have had in influencing the pilot stream study in the mid-Atlantic highlands or on EMAP' s general objectives and strategy for streams. Although some critical issues regarding stream studies can be discussed in workshop format, this should not be the sole format for involving the external community of stream scientists in

OCR for page 15
Review of EPA's Environmental Monitoring and Assessment Program: Surface Waters Monitoring Component EMAP's planning for stream studies. Many issues need more careful (reflective) and detailed analyses than can be provided in a workshop format. Other ways to involve external scientists effectively include commissioning review papers or assessing needs on a given topic by one or a few well-recognized scientists; bringing experts (singly or in very small groups) to the EPA Corvallis Laboratory to act as consultants for periods as short as one week to as long as 3 to 6 months; and issuing a call for proposals to develop alternative sampling-design strategies, alternative approaches to the stream pilot study (or studies), or alternatives to the EMAP stream program as a whole. EMAP-SW should also emphasize the idea of using the statistical results as a “red flag” warning system. That is, where statistical results are observed (e.g., statistically significant differences, changes, associations), one should then be able to biologically verify and interpret those results. Such a dual-pronged approach would provide protection against EMAP-SW presenting conclusions that are statistically significant but ecologically meaningless. The committee recognizes that not everything can be planned in advance, and there still is room for the trial-and-error approach in developing large-scale programs like EMAP. Nonetheless, the scale of financial and human resources required even for a pilot-level survey is sufficiently great that EPA must not only minimize the risk of error, but also maximize the likelihood it will successfully address the critical issues necessary for planning a full-scale stream survey. We do not believe that EMAP is presently in this position. EMAP-Surface Waters should use the next year or two to establish its objectives for the stream survey and to do a much more detailed design of the program. An essential component of this design will be an analysis of the specific information to be gained by the program that is not available elsewhere and the likely costs to achieve it. INTER- AND INTRA-AGENCY COOPERATION There is a tremendous opportunity for collaboration in field and laboratory programs of the Surface Waters component in EMAP. Water and air quality and fisheries data are collected by numerous

OCR for page 15
Review of EPA's Environmental Monitoring and Assessment Program: Surface Waters Monitoring Component federal and state agencies. Although EMAP-Surface Waters acknowledges this potential, we have seen no written plans to facilitate specific interactions. Regionally based state and federal scientists, with long-term familiarity with local systems, are likely to be more efficient in field sampling, given sufficient resources to accomplish prescribed goals. To the extent possible, EMAP should use such local talent (including individuals from academic institutions) for field data collection. Much of the routine water quality sampling done by state pollution control agencies on surface waters nationwide is funded through the U.S. EPA's Office of Water under Section 305b of the Clean Water Act. The committee recommends that EMAP and the EPA Office of Water work together to insure that data collected under the 305b program are useful not just for compliance monitoring (the primary focus of current programs in most states), but also to assess temporal and geographic trends in water quality. Closer collaboration between these two programs has the potential to enhance the effectiveness of both programs and reduce the overall cost of federal monitoring programs for surface water quality. OVERSIGHT AND COORDINATION AMONG EMAP RESOURCE GROUPS Although information about lakes in isolation has value, this value could be enhanced by coordinating among resource groups. The Surface Waters, Wetlands, Great Lakes, and Near Coastal resource groups of EMAP all share responsibility for assessing hydrologically linked ecosystems of various sorts. The operational organization of EMAP reflects bureaucratic realities, not ecological ones. Since resource group responsibilities cut across ecosystem boundaries, interactions between these resource groups will have to be strong if ecosystem assessment is really a goal of EMAP. Again, although this need for interaction is acknowledged, we have seen little concrete evidence of cooperation between resource groups. Assessment end points, indicators, resource-unit selection, and sampling protocols seem to be under separate development for each resource. It is very unclear how much real

OCR for page 15
Review of EPA's Environmental Monitoring and Assessment Program: Surface Waters Monitoring Component coordination exists. It is early in the life of the EMAP program, and it is still possible to develop substantive ties and coordination between the different groups. However, linking these resource groups in a meaningful way will be difficult if the process does not begin soon. Coordination among resource groups is especially important for the Surface Waters component. Surface waters are affected by processes occurring within the terrestrial ecosystems in their watersheds. Currently, the Surface Waters group is analyzing riparian vegetation. But it is not clear that the classification system being used is the same as those used by terrestrially focused resource groups. Without a close interaction with the terrestrial components of EMAP, an opportunity for comprehensive understanding of how and why lakes may be changing is likely to be missed. One of the most troubling aspects of the EMAP central organization is the lack of a clearly defined procedure for defining and prioritizing the assessment questions that can and will be addressed by the program. These questions are critically important, because they should drive the sampling strategies and clarify the goals of EMAP-Surface Waters and the other individual resource groups. A procedure needs to be developed to identify the most important assessment questions from a policy perspective, but at the same time ensure that it is scientifically feasible to address the questions. This is a particularly challenging task, because future assessment issues are likely to be interdisciplinary, numerous, and changing or evolving over the long term. Since the assessment questions define the overall direction of EMAP data gathering, it is imperative that these questions not be defined on an ad hoc basis without input from potential users outside EMAP. Questions and problems inherent in the choice of assessment end points for EMAP-Surface Waters might have been avoided if such a procedure were followed. The overall decision structure is not necessarily an issue of bottom-up or top-down, but rather of inclusion of the best available policy and scientific expertise in the process. One possible approach is to formalize a planning structure that would be composed of guidance panels associated with each re-

OCR for page 15
Review of EPA's Environmental Monitoring and Assessment Program: Surface Waters Monitoring Component source group. A central planning committee, composed of representatives from each of the “thematic” panels, would then make the hard decisions of resource allocation and attempt to optimize efficiency and coordination between groups. Some aspects of the planning structure exist already. However, given the importance of the decisions that the panels will make, it is critical that the guidance panels include representatives of EMAP 's clients, i.e., policy makers and the larger scientific community. Most panel members should be external to EMAP and to EPA. They should be leaders in their areas of expertise. The panels would play a decisive role in decision making and should not duplicate the advisory and planning functions of the current peer-review panels and of EPA' s Science Advisory Board.