Review of EPA's Environmental Monitoring and Assessment Program: Overall Evaluation (1995)

I want to thank you and the Committee for your recent review of the Forests and Estuaries components of EMAP. I am pleased that the Committee continues to support the goals of the program and sees progress in those components.

The recommendations in the Estuaries and Forest chapters are both thoughtful and helpful, and I understand that EMAP already has begun to take action on many of them. Matt Bills and I will make sure that all of the recommendations have been given careful attention before making substantial commitments to further implementing either component. I fully agree that articulation of a coherent, consistent, and comprehensive strategic plan is a priority for EMAP, and Ed Martinko is making this a top priority.

As we discussed during our visit, I am concerned that in the executive summary, the Committee questions the degree to which EMAP can meet its goals in a timely and effective manner. It is not clear to me whether this concern has more to do with the technical adequacy of the indicators and sampling design, or the costs and management of the program. Perhaps it would be helpful for me to respond to the Committee from a policy perspective.

Regarding the overall design of the program, I would like to speak to the EPA's reason for investing in EMAP. A critical component of Administrator Browner's National Goals Program is to establish measurable goals for improving the quality of the environment and to monitor progress in achieving these goals. EPA also was advised by its Science Advisory Board, in its report, Reducing Risk, to focus its limited resources on opportunities that offered the greatest potential for risk reduction. When in the past EPA has attempted to determine the quantitative extent of any particular environmental problem so as to assess relative risk, or to track progress resulting from regulatory programs, data from individual intensive

study sites selected for research or management attention seldom have provided the necessary, relevant information.

This is so because most environmental harm results from the aggregation of local actions (e.g., pollution discharges and habitat loss and alteration) on a matrix of differentially susceptible ecological systems. The resulting patterns of harm are spatially complex, making it very difficult to extrapolate from one research site to many, so as to gain an unbiased estimate of the conditions of the whole. On the other hand, the statistical sampling approach developed to assess the status and trends in surface water acidification from acid rain (and upon which the EMAP design was based) provided scientifically sound information that was exactly suited to EPA's decision-making needs.

The issue is not whether intensive monitoring will better meet EPA's needs (it won't), but whether the EMAP approach will. The questions that I hope the Committee will answer in this regard are: (1) do the chosen indicators accurately distinguish between systems about which we should be concerned and those about which we should not; and (2) will the sampling design detect differences in the relative proportions of these situations with sufficient precision to identify with adequate confidence which resources are at greatest risk, and in time to conduct cause/effect studies and institute management changes before significant or irreversible harm occurs to the resource as a whole? What is adequate, in either case, and is the "perfect" the enemy of the "good"?

There are both science and policy issues operating here, and I sense from the panel's comments that it is time to bring these two audiences together. I would be glad to help facilitate such a dialogue if the Committee would find it useful.

The committee continues to express concern over the way EMAP treats cause and effect. Early on, it was decided to focus on indicators of biological condition, rather than indicators of exposure to pollutants or habitat modification because of the uncertainty surrounding the cumulative impact of aggravating or mitigating effects of multiple stresses on ecosystems. It has been a "null hypothesis" in EMAP that collecting exposure and habitat indicator data to be used in an "epidemiological" search for statistical associations between indicators of poor condition and exposure to pollutants or altered habitat would be cost-effective, given the high fixed costs of visiting an EMAP sampling site. I suspect that exposure and habitat data are necessary in evaluating the credibility of the biological condition indicators early in the program, but its cost-effectiveness on a permanent basis needs to be rigorously evaluated.

Borrowing from Koch's Postulates, in no case would epidemiology be sufficient for presumption of a cause-effect relationship. Instead, it should help to prioritize ecosystems for exposure assessment and ecological effects research. ORD is now in the process of taking the long-awaited second step in its integrated ecological research strategy, which is to refocus its ecological research into conducting

intensive ecological research at a number of sites and regions across the country, in what has been termed the "place-centered" approach to integrated ecosystems management. It is here that cause-effect research will be centered, and we also expect to test the EMAP design for establishing baseline conditions and trends in ecological indicators at less than national scales. Many EPA Regional Offices are finding EMAP-developed technology useful at finer scales in the REMAP program. This government-wide initiative is coordinated through the White House Committee on Environment and Natural Resources.

Regarding costs, considering the enormous off-budget costs of over-protecting or under-protecting the environment based on inadequate information about environmental status and trends, EPA believes that a scientifically sound EMAP program would be a good investment even at $50-100 Million per year. Under the current discretionary domestic budget constraints, however, this may not be possible. We currently are reviewing our options for further development of the program, and accurate estimates of the cost of implementation are critical. It has been my experience, however, that because of economies of scale in procurement, eventual unit costs for field sampling programs such as EMAP are very difficult to predict with confidence in the early stages.

With regard to program management, having spent most of my career managing complex, multi-organizational programs, I can say with certainty that they are never managed as well as we would like them to be. In the case of EMAP, it has been made especially difficult by severe limitations on hiring, changes in contracts and grants management within EPA, and unique personnel issues beyond ORD's control. If the Committee has knowledge of particularly successful complex programs from which we could learn, I would appreciate specific management recommendations.

Finally, I would like to address the issue of communication. I can appreciate the committee's frustration with the "voluminous literature" of EMAP and the ponderous process of peer reviewed publication of research results. While this process seems to be speeding up (as of the end of 1993 EMAP counts 244 peer reviewed technical products), the process of analyzing samples, quality assuring the data, interpretation of results, and peer review seems to take at least a year. Implementation of a permanent program eventually will speed up the process for annual data reports, but that is cold comfort to the Committee.

If there is anything I can do to insure that the Committee has the latest possible information to guide its deliberations in the eleventh hour, I am at your service. In return, I would ask that the Committee be as specific as possible in its critiques and recommendations, for this will allow me to ensure that we are taking appropriate action to address its concerns. As we discussed at our meeting, I would like to offer the services of Jay Messer to you and the Committee to facilitate communication with EMAP, Matt Bills, and myself. Although Jay has not been involved directly with EMAP since his rotational assignment to the United States

Senate in 1991, as you know his knowledge of the program and the issues surrounding its instigation is extensive. He will give this assignment the highest priority. He can be reached as follows:

Dr. Jay J. Messer, Acting Director

Atmospheric Research and Exposure Assessment Laboratory

U.S. EPA (MD-75)

Research Triangle Park, NC 27711

Voice: 919-541-2107

Fax: 919-541-7588

E-mail: messer.jay@epamail.epa.gov

Again, please convey my appreciation to the Committee for their efforts on our behalf. EPA Administrator Browner is committed to insuring that EPA's science is of the highest quality and credibility, and the Committee's careful evaluation of EMAP is paramount to achieving this end.

EXECUTIVE SUMMARY AND OVERVIEW

NRC Comment: Some 18 months after the initial report many of the questions raised in the committee's interim report have not yet been answered.

EMAP Response: We agree. However, following the June 1992 report from the NRC, EMAP began to implement many of the suggestions contained in that report and initiated research efforts to answer questions that were raised. In addition, the Assistant Administrator for ORD convened a special workshop, attended by both SAB and NRC members, that dealt with several issues raised in that interim report. Research addressing some of these issues requires a longer period of time to produce results. Questions that require management and organizational answers extend beyond EMAP and are being actively pursued. Thus, EMAP has made significant progress since the interim report although some issues have not been fully addressed.

NRC Comment: The committee foresees many technical difficulties in detecting meaningful trends at scales relevant to policy decisions. This apparent lack of commitment to evaluating the best approach for temporal trend detection is a serious flaw in EMAP program development.

EMAP Response: This comment is pervasive in many comments of the committee. We concur that the description of ''meaningful trends at scales relevant to policy decisions" is an area of ongoing discussion that requires resolution prior to implementation of long-term EMAP monitoring. However, our initial technical information and policy discussions are confirming that our approach to detecting changes is developing useful information that is not available elsewhere.

The committee has recommended intensive site-based sampling approaches for the detection of trends. While EMAP concurs with the need for information based on such an approach. EMAP recommends that site-based intensive monitoring information continue to be collected through other ORD, federal, and academic programs, because site-based information alone is insufficient for detecting "meaningful trends at scales relevant to policy decisions".

Concems on the technical aspects of detecting trends are addressed both here and in the estuaries and forest sections. EMAP has sponsored statistical research on the properties of statistical sampling designs and their ability to detect trends and estimate status. These studies clearly show that detection of trends under EMAP sampling design alternatives is technically feasible. Based on this research and information gathered by the resource groups at least three groups Forests, Estuaries and Surface Waters will be able to meet

the EMAP Data-Quality Objectives (DQOs) for estimating status and detecting trends. While the other resource groups have not completed this analysis, initial indicators are favorable. EMAP is continuing discussions with EMAP information users and policy analysts to establish that the DQOs represent meaningful trends, acceptable levels of uncertainty, and appropriate scales.

NRC Comment: EMAP has chosen to use standard federal regions established by Office of Management and budget in its documentation. … These regions are not standard between federal agencies …. In addition, they do not capture known geographic, climatological, or ecological regimes or processes.

EMAP Response: EMAP has established Data-Quality Objectives (DQOs) on regional areas approximating the size of the standard federal regions. EMAP anticipates that the information will routinely be reported for a variety of the geopolitical regions (standard federal as well as those used by our partner agencies). For EPA information users, for example, we have committed to provide assessments of the status and trends in condition of the ecological resources within the EPA regions to EPA's Regional Administrators. The program is being designed, however, with additional flexibility to respond to a range of scales and regions, dependent upon the question and the DQOs of the information users. While the design phase is proceeding on the basis of the standard federal regions, EMAP fully anticipates that additional regional definitions based on geographic, climatological and ecological regimes and processes will be used in the analysis and assessment phases. The EMAP design is being develop to accommodate such analyses and provide information about the associated uncertainty.

NRC Comment: The committee expresses concern over the information management within EMAP. More specifically there are five significant areas of concern;

• 1)	No analysis of user requirements…

• 2)	No system design nor specific information concerning what the system will do…

• 3)	No short or long-term plan to implement IM.

• 4)	General consensus among the committee that Oracle and Arc INFO are inadequate to handle complex data from EMAP.

• 5)	An information system plan should include the transition from short-term to long-term. … and EMAP should use expertise in spatial data processing and handling at other federal agencies.

EMAP Response: Recently, EMAP has provided the NRC panel with a revised strategic plan for information management, along with the results of a peer review panel. In addition, a current status briefing was conducted for members of the NRC panel that included a demonstration of the current information management system. These briefings and documents address these areas of concem; however, we welcome the NRC's additional specific questions and recommendations for changes and will implement those that are within EPA policies and procedures.

Information management is indeed a complex issue: rapid advances are being made in scientific data management, computer information networks

and hardware; but these must be integrated within EPA's and EMAP's organizational structure. The use of ARC INFO and Oracle as the basis for EMAP's data base management system, was delayed by EPA's procurement of Oracle, but it is currently required as the Agency standard. This assures maximum transferability of information to other EPA as well as federal agency programs. EMAP is represented on the Federal Geographic Data Committee (FGDC) and does interact with expertise through out the federal government. In spite of the fact some federal agencies lag behind the latest scientific developments in information management systems.

NRC Comment: The EMAP documentation is a large and sometimes unclear body of literature and "typical of many large programs" it has lagged behind its initial schedule.

EMAP Response: We concur. The initial schedule was optimistic and the development and growth of the program research results, budget, infrastructure and management has proceeded at a slower rate. In part, the developmental nature of the program, while documented in over 200 reports, scientific papers and plans, has contributed to difficulties in the panel's obtaining a clear picture of EMAP. Some of the lack of clarity in the documentation arises from the diverse set of disciplines working on EMAP and that different people use the same term to mean different things. We have published a bibliography of common terms used within EMAP and have taken action to assure a more consistent set of teminology in all documents.

NRC Comment: Can EPA achieve overall purpose and basic goals in a timely and cost-effective manner?

EMAP Response: The Office of Research and Development has been established with a mission to conduct research including that necessary to implement a comprehensive monitoring and assessment program like EMAP. As stated by the Science Advisory Board, EPA should be conducting a program like EMAP that requires a commitment by EPA and all other federal partner agencies in order to be successful. EMAP's initial success with other agencies and EPA, leads us to anticipate that we will be successful in obtaining additional commitments of staff and budgets. With respect to timeliness issues, EMAP believes that, in an operational mode in partnership with other agencies, results can be delivered in a timely cost-effective manner.

EMAP agrees with the need to provide additional information on the cost and cost-effective-

ness of the program. We will continue to develop ways to further reduce costs and increase our effectiveness.

NRC Comment: The primary difference between this statement of objectives and earlier ones is that the analysis of cause-effect relationships has been downplayed. Instead, EMAP now states (perhaps more realistically) that it will "seek associations" between stress and indicators of ecological condition.

EMAP Response: The wording of the third objective was changed as the result of a joint meeting between members of the NRC committee, the SAB, and EMAP to clarify the original intent. Previously the objective read "Monitor indicators of pollutant exposure and seek associations between human induced stressors and ecological condition.…" We listened to the joint committee's advice and agreed that we should place our highest priority on biological indicators of condition and not the stressors themselves. We believe that we will be able to provide useful information much as an epidemiologist does when they draw associations between national health statistics and possible environmental contaminants.

Our preliminary feedback leads us to conclude that the EMAP information will significantly increase the scientific foundation and data available to administrators and policy makers. The SAB guidance, the degree of interagency activities, the reactions by EPA Program Offices, EPA Regions and several States, all point out that EMAP is providing useful information, that is not available elsewhere. EMAP anticipates continuing these interactions and improving the program's ability to provide useful information for policy makers.

NRC Comment: A coherent, consistent, comprehensive strategic plan for EMAP, should be an objective of the highest echelons of EPA that are involved with EMAP.

EMAP Response: We agree. We recognize this is long over due and the EMAP executive committee will give high priority to the preparation of this document.

NRC Comment: There is concern about the way "EMAP addresses cause and effect relationships". "It appears to the committee that the question of causality needs more focused planning by EMAP officials."

EMAP Response: EMAP has stated that its monitoring effort can not establish cause and effect; however, it does state an emphasis on seeking associations which do contribute to the scientific weight of evidence process for establishing cause and effect. Existing information on causality from the literature and ongoing scientific research is key in the development of conceptual modeis and indicator selection. Clearly cause and effect research results will play a critical role in this process and in the assessment of ecological conditions. ORD will focus additional efforts on the best strate-

gies for incorporating cause and effect results into both research and monitoring programs.

NRC Comment: There is a concern with the NRC committee about overall management and coordination within EMAP.

EMAP Response: We concur that management and coordination of the program can be improved. While EPA and ORD have made significant commitments of funds to the program, the program is understaffed. The hiring of some full-time staff at EMAP center has resulted in recent improvements in the coordination of the program. However, additional support in both areas is needed.

NRC Comment: EMAP should make better use of sampling and monitoring programs already developed by other agencies and parts of EPA.

EMAP Response: The NRC notes that: "The committee is encouraged by the extent to which the EMAP program has been pursuing opportunities for interagency cooperation. Such cooperation can reduce the cost and significantly increase the usefulness of monitoring programs. It will also result in the different government agencies basing their natural resource programs on consistent information." We concur that use of ongoing sampling and monitoring programs and cooperative efforts are critical to EMAP's success as a regional and national monitoring program. EMAP will continue to actively pursue these efforts with both interagency partners and within EPA.

SECTION 2 ESTUARIES

NRC Comment: The committee agrees with the review panel of the Estuarine Research Federation, which doubts that the indices generated by EMAP will have the power to detect the amount of environmental change expected.

EMAP Response: No data set presently exists that represents the type of population for which EMAP-Estuaries is attempting to ascertain trends (i.e., large biogeographic regions). Temporal trends related to estuary-specific data sets will not provide all of the information needed to address this issue, i.e. whether they support trends greater than or less than 2% per year for a decade. The type of data being collected by EMAP-Estuaries is the type of data set necessary to investigate the "expected" level of trajectory for any indicator.

At present, EMAP-Estuaries indicators have the potential to detect between a 1-2% change per year for a decade. This is based on an analysis of the Virginian and Louisianian Provinces data sets that will be published in 1994. With the possible exception of some data sets from Chesapeake Bay (which comprises 60% of the Virginian Province), no available historic data can be used to quantitatively estimate the expected long-term trends for biogeographical provinces. This is precisely what

makes the EMAP-Estuaries data unique and important.

For example, the EMAP design standard is the ability to detect a 20 percent change occurring over a decade. The published information on changes in various indicators shows, however, that some changes occur in estuaries at a much slower rate than this (Stanley, 1993).

The EMAP Data-Quality Objective for trends is to detect a +/-2% per year for a decade. Stanley (1993) documented trends of nutrients, chlorophyll, and bottom dissolved oxygen in the Pamlico River Estuary for 12-24 years at 3 sites (upper, middle, and lower river). Although the representativeness of these three sites for making trends statements concerning the Pamlico River Estuary can be debated, the site-specific data (not regional) presented tends to support a 2% per year change rate. While some information is missing, the overall monthly data at the three sites is available for trend analysis.

Seventy-eight percent of the nutrient-site combinations showed significant trends (7 of 9 sites). Of these seven, six showed trend rates ranging from +/-2.2 to 7.7% per year. The remaining site showed a trend of 1.6% per year. Another stressor indicator (bottom dissolved oxygen) showed only a 0.2% per year change. However, chlorophyll a showed a 6.6% per year change in concentration at a single site in the upper estuary. These rates relate to changes in the measured concentration of an indicator at a site, not the area characterized by a specific condition.

NRC Comment: It seems that in some cases EPA personnel have not researched the published literature.

EMAP Response: EMAP-Estuaries and its cooperators (University of Rhode Island, Rutgers University, University of Maryland, Virginia Institute of Marine Sciences, University of North Carolina, South Carolina Marine Resources Research institute, Florida Marine Research Institute, Gulf Coast Research Laboratory, Texas A&M University, University of California at Berkeley, and the Southern California Coastal Waters Research Project) have carefully studied the published literature as evidenced by the hundreds of references to this literature in the EMAP-Estuaries publications. EMAP-Estuaries welcomes any additional references and the specific information that should be applied, that the committee would like to suggest.

NRC Comment: By contrast, as is pointed out in Chapter 3, changes in ecosystems can be quite sudden and catastrophic, perhaps too fast to be adequately captured by EMAP's sampling scheme.

EMAP Response: If significant change is occurring too fast to be adequately captured by the EMAP-Estuaries design, then we would assume that detecting a 2% per year change would be more than adequate. If these catastrophic changes are ephemeral (i.e., recovery is evidenced in less than the one year between EMAP-Estuaries samples), then we should rethink our definition of catastrophic. Nonetheless, it is not an EMAP objective to detect short-term episodic events.

NRC Comment: There is a temptation to think that the next challenge is to carry out pilot projects on new provinces, one after another. However, as pointed out by the Estuarine Research Federation review committee, the real challenge is in obtaining the best possible set of indicators of ecological condition.

EMAP Response: EMAP-Estuaries does not disagree; however, we believe that the challenge is to develop a set of ecological indicators that is applicable for all biogeographic provinces. This challenge can only be met by evaluating, the utility, accuracy, precision, and interpretability of our selected indicators in all biogeographic provinces. Our present EMAP-Estuaries strategy calls for the collection of 4 years of data from the Virginian and Louisianian Provinces (the two largest in terms of estuarine resources), followed by intensive evaluation of that data and specific indicator testing and modification in the remaining 10 provinces. Making important decisions, such as the final selection of core indicators, without some testing in the remaining provinces would be imprudent at best and potentially disastrous. We must recognize that the estuarine resources of the Nation are diverse and the indicators must be shown to apply to all of them, prior to implementation of long-term monitoring.

NRC Comment: The committee considers the probability-based sampling approach to be satisfactory and flexible enough to meet the problem of how to sample large estuaries, small estuaries, and rivers. However, the value of the resulting data could be significantly enhanced by the addition of methods for detecting step functions of change, including threshold effects. This is an appropriate area for further research.

EMAP Response: EMAP-Estuaries, the EMAP Design & Statistics Team, and university cooperators are examining the effect of multiple design modifications on the detection of different types of trends functions, including step functions, nonlinear functions, and various other functions. We agree with NRC that this represents a fertile area for further research.

NRC Comment: The committee is concerned that the proposed probability-based sampling is not adequate for meaningful trend analysis. Consideration should be given to development of an alternative regional-sampling scheme that includes detailed site-based sampling, which would lend itself to more effective trend-analysis procedures.

EMAP Response: EMAP-Estuaries has, through a joint project with EMAP-Design and Statistics and

Oregon State University and several contracted statisticians, conducted analyses using the EMAP-Estuaries data sets for the Virginian and Louisianian provinces to evaluate our ability to discem 2% per year decadal trends. Trends of 1-2% per year for a decade were detectable for the Virginian and Louisianian Provinces from virtually all indicators comprising the EMAP-Estuaries data sets. Only data sets containing fewer than 40-50 observations over the four-year period could not attain the level of trend detection.

While EMAP-Estuaries agrees that the use of site-based intensive sampling is necessary to further evaluate the causes of the observed trends, it is not necessarily required for their detection. In addition, while this approach maybe more effective at detecting trends at a specific site, it has not been shown that it is sufficient for detecting trends within a resource class at province scales. EMAP is cooperating with NOAA's Status and Trends monitoring program, which includes more detailed site-based sampling.

NRC Comment: A more explicit conceptual model is needed to direct indicator research and provide integration across different estuarine provinces.

EMAP Response: EMAP agrees with NRC that the current conceptual model can be improved, and is working to provide a more explicit conceptual model¨ This conceptual model will appear in a manuscript, detailing its development, relationships to environmental values and indicators, and relationship to short- and long-term indicator research, to be submitted for publication by Environmental Monitoring and Assessment in 1994.

NRC Comment: Fish sampling should be reexamined carefully and either expanded significantly or dropped. In particular, the measurement of the contaminant body burden as currently implemented is fundamentally flawed because no account was taken of temporal (seasonal variability), and more importantly no account was taken of the relation between body burden and the size (age) of the fish.

EMAP Response: The present fish program concentrates on community measures of the fish species susceptible to the sampling gears used, pathologies, biomarkers, and body burdens. The number of trawls and their lengths have already been increased in EMAP-Estuaries in 1992-1994. The purpose and intent of the EMAP-Estuaries sampling is to reflect conditions during an index period, i.e. a short-temporal window (4-6 weeks) annually. While there was no direct adjustment of body burdens for the size (age) of the fish analyzed, there was care taken in both the Virginian and Louisianian Provinces to composite fish of the same size ranges at each station and across stations. Of course, this approach will not always be

successful based on the age structure of the fish populations at each site. However, length data were collected for all fish that were analyzed and EMAP-Estuaries will in its four-year assessments of the Virginian and Louisianian Provinces determine the effects of variability in length (age) on the body burdens determined for a sample. EMAP agrees that this research and analysis should be completed before additional decisions about the future of the fish program are made, i.e. expand or drop.

NRC Comment: The evaluation of current indicators should continue. The determination of the variability of the dissolved oxygen measurements carried out by EMAP-Estuaries is a good example of the care that must be taken in evaluating even a relatively simple indicator.

EMAP Response: EMAP agrees with the NRC in this regard. Evaluation of the variability associated with all EMAP-Estuaries indicators is one of the primary steps in the EMAP indicator development strategy. EMAP has and will continue its efforts to fully evaluate the utility of all indicators proposed for use. To this end, it is also important to test specific indicators under a wide range of conditions, including geographic and habitat differences. This necessitates the further testing of indicators in regions outside the Virginian and Louisianian Provinces and could not be accomplished if EMAP were to stop all pilot activities.

NRC Comment: Whenever possible, EMAP-Estuaries should develop remotely-sensed indicators. Candidate measures would include surface chlorophyll and extent of submerged aquatic vegetation.

EMAP Response: EMAP agrees with the NRC and has been utilizing remote sensing (aerial photography) to assess the extent of submerged aquatic beds through an interagency project with Fish and Wildlife Service (now National Biological Survey, NBS) and NOAA. More sophisticated remote measures based on satellite imagery are also being evaluated although they appear to have some difficulties based on interference factors. EMAP-Estuaries presently is measuring surface chlorophyll to evaluate its utility as an indicator, primarily to estimate its short-term temporal variability within the index sampling period.

NRC Comment: EMAP-Estuaries should develop indicators of ecosystem function, such as productivity.

EMAP Response: EMAP agrees with the NRC, although we have identified few short-term measures of ecosystem function. Since 1991, EMAP-Estuaries has been evaluating several functional indicators of ecological condition. Foremost among these indicators are indices of estuarine trophic state based upon productivity, nutrient concentrations, chlorophyll a, and stable isotopes.

NRC Comment: The benthic index should continue to be examined, particularly the validity of the use of weighting factors in the discriminant analysis.

EMAP Response: The benthic index has been and will continue to be examined. Changes in the specific structure of the benthic indices developed for the Virginian and Louisianian Provinces have been made since June, 1993. Validation studies are continuing. In addition, EMAP-Estuaries and the Chesapeake Bay Program (CBP) have been cooperating on a comparative study of the Virginian Province Index and the CBP's Restoration Goals Index (an index without weighting factors). That comparison shows approximately 85-88% overlap with most differences resulting from differences of opinion among benthic ecologists regarding the functional status of some species (i.e., whether species A is an equilibrium or opportunistic species).

NRC Comment: Research must continue on the best way to make measurements of fish pathology and on the best way to separate the various causes of the pathology.

EMAP Response: EMAP agrees. EMAP-Estuaries is continuing this research activity at EPA's Gulf Breeze and Narragansett laboratories and through an interagency agreement with the Fish and Wildlife Service (now National Biological Survey (NBS)). The EPA laboratory studies are examining the use of specific pathological indicators as indicators of fish community degradation. These activities focus on lesions and biomarkers, particularly splenic macrophage aggregates. The NBS work uses rearing studies to focus on the linkage between contaminant exposures and the subsequent development of increased density of size of macrophage aggregates culminating in lesions.

NRC Comment: A detailed analysis of the successes and failures of the various pilot and demonstration projects should be undertaken and the best ideas should be incorporated into the indicator plan for EMAP-Estuaries.

EMAP Response: EMAP agrees and will establish indicator workshops, including distinguished members of the estuarine research community, to accomplish precisely this goal. These workshops will establish through the results of the pilots, demonstrations, and indicator testing studies a suite of core indicators that will be used in developing a national implementation plan for coastal monitoring.

NRC Comment: Investigator-initiated research should continue to be supported to validate indicators in use and develop new ones.

EMAP Response: Largely in response to previous recommendations from the NRC and SAB, EMAP initiated in 1993 an investigator-initiated indicator research program through the Office of Exploratory Research to accomplish this goal. Nine research projects were begun in 1993. Currently, EMAP has solicited and is in the process of reviewing several

investigator-initiated research proposal for research to be conducted at Long-Term Ecological Research (LTER) and Land Margin Ecological Research (LMER) sites.

NRC Comment: EMAP-Estuaries should use the detailed historical data sets already identified in the Virginian province to provide temporal and spatial information on indicators and their ability to detect change.

EMAP Response: EMAP-Estuaries has examined several large-scale data sets in the Virginian and Louisianian provinces that have proven useful in determining annual variability. However, EMAP most detailed data sets have proven to be less useful because they; have not used consistent or comparable indicators or methods, their data-quality is not known, or they are specific to one estuarine system and not likely be representative of the larger province or regional scale.

NRC Comment: EMAP-Estuaries should give priority to determining the power of these indices to detect the changes that are reasonably expected over the next decade.

EMAP Response: We concur. The determination of the power of the presently used indicators to detect trends is a priority issue associated with the four-year assessments of data in the Virginian and Louisianian Provinces.

NRC Comment: EMAP-Estuaries should change to a standard relational database management tool for storage and retrieval in the near future.

EMAP Response: EMAP-Estuaries is presently converting from its initial data base storage tool of SAS to an ORACLE-based data base, the current EPA standard. This data base conversion is expected to be completed by mid-1994.

NRC Comment: Alternative multivariate analysis techniques should be carefully explored in addition to the linear discriminant function that was applied to data for the classification of estuarine habitats. EMAP might also explore methods of statistical computing conceming visualization of multi-dimensional data.

EMAP Response: EMAP agrees with the NRC and is currently examining the approaches mentioned. A new interagency agreement with the National Biological Survey is examining the use of GIS-analysis and evaluation as a research tool for EMAP-Estuaries data.

NRC Comment: Compatible data sets and efficient interfaces must be developed between EMAP-Estuaries and other segments of EMAP, NOAA, and other federal agencies with large-scale monitoring information. This would allow EMAP-Estuaries to be used as a model for development of data acquisition and management in other EMAP resource groups.

EMAP Response: EMAP agrees with the NRC. EMAP-Estuaries is currently working with NOAA and NBS to develop a retrieval system for EMAP-Estuaries data (a first step to the above goal) that can be accessed and used by other agencies and other public users. EMAP is also participating on the Intergovernmental Task Force on Monitoring Water Quality (ITFM) and the group addressing data management and information sharing. EMAP's Information Management Team is tackling the broader issue of integrating data sets across EMAP Resource Groups.

NRC Comment: A mechanism needs to be developed to ensure thoughtful, detailed analyses and interpretation of data. Outside scientists and managers should be involved at all stages, including review.

EMAP Response: EMAP includes outside scientists and managers in the analysis of all data but particularly for the 4-year cycle data. The Virginian Province Assessment Team is comprised of EMAP personnel, academic scientists, and Agency managers. Their task is to produce the 4-year assessment report for the Virginian Province.

NRC Comment: An updated plan needs to be prepared as to how the reports and analyses will incorporate the data on stressors. There has been a change from the original plans, and the entire EMAP now appears to be reducing its emphasis on stressors.

EMAP Response: EMAP-Estuaries continues to collect pertinent data on stressors to be used in condition or status reviews as well as to assess potential associations among biological indicators and stressors. EMAP, as a whole, is addressing the role of stressors in future monitoring and research activities. Dependent on the results of that effort, EMAP-Estuaries will prepare a plan for the use of internally and externally monitored stressor information within the EMAP context.

NRC Comment: It is imperative to assess EMAP-Estuaries in detail before going on or before adding additional provinces. This assessment should begin as soon as the Virginian Province demonstration completes its four-year cycle.

EMAP Response: EMAP-Estuaries is making significant efforts to analyze fully the four-year cycle data sets for the Virginian Province (1990-1993) and the Louisianian Province (1991-1994) while testing the applicability of indicators in other regions of the country in short, concise pilot studies. It is on the basis of these analyses that EMAP-Estuaries will propose a national implementation design plan and indicator suite. EMAP-Estuaries did stop collecting data in the Virginian Province in 1993 after a full 4-year cycle of data was collected to permit complete analysis of the information. Since the objectives of the EMAP program require comparisons among biogeographic regions, EMAP must develop a set of ecological indicators that are applicable for all biogeographical provinces. Consequently, evaluation of selected indicators in other provinces will be continued while detailed analysis of the Virginian Provinces four-year cycle demonstration is being completed.

NRC Comment: EMAP-Estuaries has begun development of indicators that should be useful to managers of estuarine resources. It now must begin to develop a program for continual involvement of these managers in improving the indicators and reporting and interpreting results.

EMAP Response: EMAP agrees with NRC and, to this end, we have included Agency and State Managers in our workshop teams for the development of the national design and indicator suite for EMAP-Estuaries. In addition, EMAP-Estuaries data is made available for Regional and State use and our interaction with these entities is continual. These efforts are anticipated to continue on a frequent basis.

NRC Comment: The scientific applications of the EMAP-Estuaries data are not yet clear. The present-day open availability of raw EMAP-Estuaries data must be maintained in order to foster the scientific use of the information.

EMAP Response: EMAP agrees with the NRC. To date, EMAP-Estuaries data has been transmitted to over 300 users. However, we believe the present system for request and transmittal of data is often cumbersome. Therefore, EMAP-Estuaries is developing, with the information Management Group within EMAP, a public access system that would allow ready access by modem to all the composite EMAP data. Raw data, inclusive of QA and QC data, will remain in its present form of accessibility.

NRC Comment: Greater emphasis must be placed on the timely production of annual statistical summaries and reports on demonstration projects. Availability of an up-to-date assessment of how EMAP-Estuaries is doing and what it is finding is important to managers and to the scientific community.

EMAP Response: EMAP agrees with the NRC and has taken steps in 1993-1994 to reduce the amount of time required before information can be released.

NRC Comment: The quality of the scientific review panel organized by the Estuarine Research Federation was high. EMAP-Estuaries needs to establish a regular, working, review panel of similar quality for each component of the program.

EMAP Response: The ERF panel was established as a result of a request from EMAP-Estuaries. EMAP-Estuaries will also establish, in a collaborative effort with NOAA's NS&T Program, a joint review panel to begin work in 1995. We intend to work with this panel to determine the national design and indicator suite for coastal monitoring. In addition, EMAP-Estuaries instituted in 1991, province-specific review panels comprised of state and regional environmental managers and academic estuarine researchers to provide a province-level review of demonstration plans. This combined national review and province review adds depth to the review process and we believe makes EMAP-Estuaries a better program.

NRC Comment: Coordination between EMAP-Estuaries and state, regional, and national monitoring programs should continue to be aggressively pursued. These connections may be the most useful outcome of the program.

EMAP Response: We agree that coordination and interaction at the state, regional, and local levels in very important to the success of EMAP. EMAP-Estuaries has worked with six of the 22 coastal states (at their request) to modify their state-level monitoring programs to incorporate all or parts of EMAP's monitoring technology. EMAP-Estuaries continues to work with the Chesapeake Bay (CBP) and Gulf of Mexico (GOMP) Programs to develop their comprehensive monitoring plans. EMAP-Estuaries has worked with seven National Estuary

Programs in the development of their Comprehensive Management Plans and three of these NEPs (thus far) have instituted EMAP-like monitoring programs. The collaborative efforts in California have become the catalyst for the promotion of a combined regional, estuary-specific, and local monitoring program with over 100 participants.

SECTION 3 FORESTS

The following response to the NRC comments was prepared jointly by the Forest Service Forest Health Monitoring Program and EMAP.

NRC Comment: However, it should be emphasized that many of the good features of this program derived from the previously established USFS FHM program and not from EMAP.

FS/EMAP: Both agencies view the forest monitoring portion of EMAP as an outstanding example of interagency cooperation. While many of the good features of the program came from FHM, many others came about as the result of the cooperative nature that has developed. Specifically EMAP has helped to improve the overall program by improving the sampling frame, looking at the forest as an ecological resource and not just a forest resource, indicator development criteria, quality assurance, information management structure and the assessment approach. We do not view the contributions from each agency separately, but rather as one federal government program.

NRC Comment: EMAP Forests relies too heavily on a purely ''epidemiological model." "Such models have little utility in predicting how nutrient cycles, nutrient losses, or biodiversity of ecosystems change in response to stress." It was recommended that EMAP-Forests develop a "theoretical basis from which predictions can be made of general types of forest response to different types of stress."

FS/EMAP Response: We agree that one must have a theoretical rationale which takes into account how forests respond to different types of stressors. However, the objective of EMAP is to describe the ecological condition, detect changes, and draw associations between condition known stressors. EMAP will rely on research from EPA ORD, the Forest Service, other federal agencies and the academic community to develop models to "predict response to stress". Consequently, Forest Health Monitoring looks at a wide range of parameters in assessing forest health status and trends. These parameters include factors as diverse as tree crown condition, lichen communities, and soil chemistry.

Our overall monitoring rationale is as follows: We can identify a small number of key indicators that reflect the overall health of the forest ecosystem. Using cost-efficient probabilistic sampling, we can use these key indicators to estimate status (or

a beginning point). We can then, by sampling over time, use these indicators to detect broad changes in the forest ecosystem.

NRC Comment: EMAP-Forests should choose a set of indicators as soon as possible and then conduct the staff work necessary to establish sampling methods and convey these to field crews.

FS/EMAP Response: We concur with this recommendation and are concentrating efforts in this area. Several indicators have been evaluated through EMAP's indicator development process and have reached operational status. A number of others are in the process of being evaluated. We recognize the need to add indicators "as soon as possible", particularly in the areas of biotic integrity and aesthetics. However, we weigh this urgency against the need to make sure each indicator receives sufficient and thorough scientific scrutiny during the development process.

Sampling methods are written and provided to the field crews during extensive training sessions each spring.

NRC Comment: The committee recommends that the current design of four-year plot rotations be replaced or augmented by a design in which some plots are revisited each year.

FS/EMAP Response: EMAP-Forests has conducted and continues to conduct research on alternative sampling designs over time and space. EMAP has sponsored statistical research at Oregon State University to investigate the properties of statistical sampling designs and their ability to detect trends and estimate status. The research shows that the FHM four panel sample approach would have increased power with an additional panel of sites with annual revisits, especially in the early years of the program. EMAP-Forests continues to investigate the benefits and costs associated with specific alternatives on the number of sites to be revisited annually and whether these sites should be annually revisited forever or new annual revisit sites should be started in future years. We are concerned about the impact on the individual site from continuous annual revisits. We plan to make a decision prior to implementing long-term monitoring, concerning supplementing our sampling with subsets of sites with annual revisits.

NRC Comment: EMAP-Forests should develop a comprehensive information-management plan that outlines user requirements, examines long-term implementation, and fits in with the overall plan for the information-management system.

FS/EMAP Response: We agree with this recommendation. Although still early in the development stage, Forest Health Monitoring has already collected a wide array of data. Our database has already grown to the point that it cannot be managed by the scientists who gathered the data. The existing SAS data base has been adequate for the initial development and testing of

indicators, but will prove inadequate in the long run. We have been working interactively with EMAP's Information Management Coordinator in developing the overall EMAP Information Management system that will be developed prior to implementation. Our information and data management, like our science, must be performed by a professional trained in that field. The Forest Health Monitoring database development effort is now being led by a professional database manager. We are also in the process of the transitioning from SAS data management software to an Oracle/ARC INFO system which is the EPA agency standard.

NRC Comment: The results of the Forest Health Monitoring efforts be published in peer reviewed science journals.

FS/EMAP Response: We agree. Peer reviewed publications are and will continue to be a factor in the performance evaluations of our technical staff. Forest Health Monitoring scientists continue to communicate FHM program information to the broader scientific community by publishing in recognized, peer reviewed journals. At least seven articles have been published in such journals as Environmental Monitoring and Assessment, The Canadian Journal of Forest Research, Communications in Soil Science and Plant Analysis, and the International Journal of Climatology. In addition, work by FHM was included in at least eight symposia proceedings and book chapters.

NRC Comment: The Forest Health Monitoring "should not be fully implemented until the results of demonstration projects have been fully evaluated and realistic estimates of cost to EPA and other agencies is available.

FS/EMAP Response: Forest Health Monitoring is being developed in an incremental fashion. States are gradually being brought into the program and new indicators are gradually being evaluated and, if validated, adopted. With this approach, the program will not be fully implemented without a detailed accounting of the cost of various activities and the total program. For example, in the indicator development phase, cost and time constraints are an important determinant in whether an indicator can be selected for use. We are constantly aware that Forest Health Monitoring can afford to spend only so much time on each plot and so much money for analysis. Therefore, when comparing one indicator with another, the information value of each indicator or measurement is evaluated relative to its cost.

NRC Comment: The committee made several references in the body of the report concerning approaches other than monitoring based on a probabilistic sample. For example, the committee seems to feel that a more appropriate approach to Forest Health Monitoring might be long-term research at key locations coupled with modeling.

NRC Comment: The committee made several references in the body of the report concerning approaches other than monitoring based on a probabilistic sample. For example, the committee seems to feel that a more appropriate approach to Forest Health Monitoring might be long-term research at key locations coupled with modeling.

NRC Comment: The committee made several references in the body of the report concerning approaches other than monitoring based on a probabilistic sample. For example, the committee seems to feel that a more appropriate approach to Forest Health Monitoring might be long-term research at key locations coupled with modeling.

FS/EMAP Response: We agree that the quoted sentence probably cannot be defended because of the unqualified "only." This last sentence by the Committee is an excellent one; they make an important point. Forest Health Monitoring must keep in mind that change, especially on the local scale, can sometimes be sudden. Nevertheless, EMAP-FHM was not designed to detect site-specific changes and other programs are capable of identifying catastrophic events.

NRC Comment: This may be due to the fact that the original purpose of the USFS FHM program was to detect the effects and extent of diseases and insects.

FS/EMAP Response: The original purpose of the USFS FHM program was to determine the status and condition of the forest ecosystem of which insects and diseases play significant roles.

NRC Comment: To date, the indicators proposed by EMAP-Forests clearly address productivity only.

FS/EMAP Response: Although productivity, in its broadest definition, is an important part of EMAP-FHM, biodiversity, sustainability, aesthetics, and extent are of equal importance.

NRC Comment: There is apparently no single set of indicators that is to be sampled nationwide.

FS/EMAP Response: Differences in the sets of indicators in the demonstration studies are the direct result of the indicator development process. All of the indicators used in the Detection Monitoring Demonstration are measured on all plots across all forest types, ecosystems and states (16 in 1994).

NRC Comment: The Western Pilot Study (EPA, 1992b) showed that trying to make many measurements on each plot is logically infeasible.

FS/EMAP Response: We have operationally measured as many as 15 indicators per plot within a one day visit. We have done this in different pilots and demonstrations across the U.S.

NRC Comment: No structure or criteria for selecting indicators have (sic) been accepted by both USFS and EPA.

FS/EMAP Response: The USFS and EPA came to an agreement on the criteria and process for selecting indicators about two years ago. This process has allowed the addition of lichen community structure, plant biodiversity, wildlife

habitat, photosynthetic active radiation and ozone bio-indicator plant measurements to the Detection Monitoring Demonstration being conducted in 16 states in 1994.