2—
Overall Assessment

Introduction

EMAP is a program of many parts, but all the parts share common goals. The purpose of this chapter is to review those matters that apply to all parts of the program. They concern the design and implementation of monitoring; the sampling protocols; the development of indicators; integration among various parts of the program; program coordination within EMAP, within EPA, and within the federal government; and external scientific review.

The EMAP Assessment Framework

Most ecological monitoring programs are driven by some explicit or implicit set of assessment questions. EMAP is no different. The topic of this section is the basic monitoring approach taken by EMAP—retrospective monitoring. This well-established monitoring approach is suited for many environmental problems, but not for all; it is essential that any evaluation of EMAP have a clear view of those kinds of environmental problems that EMAP is likely to help identify and those for which other monitoring approaches would be better.

Retrospective or effects-oriented monitoring is monitoring that seeks to find effects by detecting changes in status or condition



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 18
--> 2— Overall Assessment Introduction EMAP is a program of many parts, but all the parts share common goals. The purpose of this chapter is to review those matters that apply to all parts of the program. They concern the design and implementation of monitoring; the sampling protocols; the development of indicators; integration among various parts of the program; program coordination within EMAP, within EPA, and within the federal government; and external scientific review. The EMAP Assessment Framework Most ecological monitoring programs are driven by some explicit or implicit set of assessment questions. EMAP is no different. The topic of this section is the basic monitoring approach taken by EMAP—retrospective monitoring. This well-established monitoring approach is suited for many environmental problems, but not for all; it is essential that any evaluation of EMAP have a clear view of those kinds of environmental problems that EMAP is likely to help identify and those for which other monitoring approaches would be better. Retrospective or effects-oriented monitoring is monitoring that seeks to find effects by detecting changes in status or condition

OCR for page 18
--> of some organism, population, or community. Examples include monitoring the body temperature of a person, monitoring the productivity of a lake, monitoring the condition of foliage in forests, and so on. It is retrospective in that it is based on detecting an effect after it has occurred. It does not assume any knowledge of cause-effect relationships, although the intention is usually to try to establish a cause if an effect is found. It is EMAP's general approach. Predictive or stress-oriented monitoring is monitoring that seeks to detect the known or suspected cause of an undesirable effect (a stressor) before the effect has had a chance to occur or to become serious. Examples include monitoring the cholesterol level in a person's blood, monitoring the stress level along a geological fault, monitoring animal tissues for the presence of known carcinogens or other disease-causing compounds, and monitoring with a canary the toxic gas level in a mine. It is predictive in that the cause-effect relationship is known, so that if the cause can be detected early, the effect can be predicted before it occurs. The EMAP Assessment Framework (Thornton et al., 1994) is the formal exposition of the assessment context for the EMAP program. As such, it is the fundamental statement of the philosophical and practical requirements of EMAP's data gathering activities. This is an especially critical element in the EMAP program development because in recent years EPA has been actively promoting ecological risk assessment (NRC, 1993b, RAF, 1992) as a new operating paradigm. Early EMAP documents were curiously silent about this larger EPA perspective, raising some question about the extent to which these initiatives were coordinated. In February 1994, EPA-EMAP released a document entitled Environmental Monitoring and Assessment Program Assessment Framework (Thornton et al., 1994). It is a welcome—if long overdue—addition to the voluminous descriptive literature EMAP has generated. It contains the most developed and lucid descriptions to date of the overall philosophy and approach behind the EMAP program. Together with the Indicator Development Strate-

OCR for page 18
--> gy, the Assessment Framework clearly outlines the rationales, approach, and objectives of the EMAP program. The Assessment Framework document describes the proposed role of EMAP in terms of EPA's ecological risk assessment paradigm. It makes clear that the program is conceived of as having two distinctive roles in the ecological risk assessment process. As a supplement to the problem formulation phase of risk assessment, EMAP is to identify emerging problems that will require the attention of other programs within EPA to determine comparative ecological risk. EMAP's second task is to provide documentation of the success or failure of national risk management decisions. By reporting on long-term trends in environmental status, EMAP is expected to provide data on the effectiveness of regulatory decisions and risk management by the agency. Its purpose in this regard is to provide data to evaluate policy, thus helping to close the loop in the iterative Ecological Risk Framework (RAF, 1992). The authors argue effectively that, in this sense, EMAP is really an integral part of EPA's new operating paradigm. The Assessment Framework document is the appropriate place for a full exploration of the benefits and shortcomings of retrospective risk assessment. Unfortunately, it provides no wide-ranging examination of this issue. Given the proposed scale of EMAP, EPA has the responsibility to provide the public with a more detailed analysis. One of the most important features of the Assessment Framework is the discussion of the implications of EPA's decision to use a retrospective (effects-oriented) instead of predictive (stress-oriented) assessment model. The Assessment Framework describes both approaches in some detail and refers to them as being complementary, arguing that retrospective analysis will "become increasingly important as assessments of larger scale problems are conducted because it will become increasingly difficult to establish specific cause-effect relationships …" Because the whole of the EMAP monitoring strategy is based on retrospective analysis, EMAP needs to present a more rigorous exposition of its rationale for this strategic choice. It is not clear that the

OCR for page 18
--> quoted statement above is in fact true. For example, in the 1950s, the poor hatching success of birds of prey such as the bald eagle was a large-scale problem clearly caused by accumulation of DDT in the eggshells. It is significant that this cause-effect relationship between DDT and eggshell thickness was not established by a monitoring program such as EMAP, but by a research program driven by clearly stated hypotheses. Sound policy (the banning of DDT and subsequent recovery of eagles) was based on this cause-effect research. Predictive and retrospective approaches are complementary, but they are not equally useful in every risk assessment. For situations in which the risk or consequence is severe (e.g., a pedestrian crossing a busy highway), effects-based risk assessment is an inappropriate strategy. When the risk is lower, either because the effect is weak, or it can be mitigated, retrospective analysis is appropriate and probably more cost-efficient (e.g., exposure badges in low radiation environments). The EMAP approach will be more useful for some types of ecological risks than for others. In considering the value of EMAP as a national monitoring program, it is important to understand what kinds of ecological risk EMAP is likely to provide useful data on, and what kinds of risks will require more traditional predictive, or stressor-based, analyses. The Assessment Framework's assertion that spatial and temporal scales are the major variables determining whether or not EMAP can provide useful assessment data may not be correct. The usefulness of retrospective data in risk assessment at any scale varies with (1) the severity of risk, (2) the probability of detecting the effect (related to statistical power of sampling procedure), and (3) the time lag required for a mitigating response: Application of this simple model to a list of real or potential environmental risks (Table 2-1) suggests that there are many risks for which retrospective analysis is appropriate, two for which it may

OCR for page 18
--> Table 2-1 Applicability of retrospective assessment to various environmental threats Environmental Threat Applicability of Retrospective Assessment Reason landscape alterations appropriate low severity, good detection increased regional loadings of nitrogen, phosphorus, sulphur appropriate low severity, good detection regional habitat declines appropriate low severity, good detection chronic toxic contamination appropriate low severity, good detection efficacy of cumulative policy appropriate low severity, good detection point-source pollution inappropriate? poor detection acid precipitation inappropriate? large mitigation lag acute toxic contamination inappropriate high severity exotic species effects inappropriate large mitigation lag global warming inappropriate high severity, large mitigation lag regional nuclear contamination inappropriate high severity ozone depletion inappropriate high severity, large mitigation lag biological extinctions inappropriate poor detection, infinite mitigation lag

OCR for page 18
--> be appropriate, and many for which it is inappropriate. (These examples are ways to think about appropriate monitoring strategies, individual areas might need more detailed analyses.) It is noteworthy that many high-profile environmental risks fall into the latter category. The EMAP approach is no panacea, and it is important that claims by the program and expectations of the public be realistic. In general, when the probability of detecting an effect is high and the cost of failing to detect is not extremely large, EMAP's effects-based monitoring can provide useful input to the Ecological Risk Assessment process as indicated in the Assessment Framework document (Thornton et al., 1994). However, when the cost of failing to detect an effect early is high traditional predictive risk assessment, which emphasizes stressor monitoring and modeling, is a clearly preferable strategy. Implications for EMAP Design The practical usefulness of EMAP's retrospective monitoring design depends on achieving a sufficiently high probability of detecting ecologically important effects. As discussed elsewhere in this and in previous reports of this committee, EMAP's current design might not have sufficient statistical power to detect ecologically important changes. The program's data-quality objective of detecting a change of 2 percent per year over 10 years in the mean of an indicator across a standard federal region can be achieved (ASA Committee on EMAP, 1992). As discussed later in this chapter, the pertinent scales for ecological processes are considerably finer, however, and the usefulness of EMAP will depend on its ability to detect these smaller scale changes. EMAP should consider design augmentations to increase the probability of detecting smaller scale ecological signals. Some possibilities include: Increased revisitation rates at a subset of sample sites. Inclusion of a set of nonrandomly selected sentinel-sites with intensive data-collection, such as the Long-Term Ecological Research network.

OCR for page 18
--> Stratified random sampling by ecoregion with data-quality objectives specified for strata. Hybrid Assessment Models The Assessment Framework addresses the issue of enhancing the probability of detecting effects by calling for selection of indicators that are linked to specific environmental values. This has been termed ''stressor-cognitive" indicator selection by EMAP staff. In a sense the EMAP indicator development strategy leans towards including some aspects of a prospective assessment, or stressor-oriented, approach as well. In some of the resource groups, e.g., estuaries, this approach has been strongly implemented, with some of the indicators chosen to detect the most probable and common stresses (organic loading and toxic contamination). EMAP should consider further developing this hybridization of effects-oriented and stressor-oriented monitoring approaches. By focusing on indicators that are sensitive to the most likely known stressors for each resource group, an increase in the probability of detecting meaningful ecological changes may be achievable within any given sampling program. There remain issues of the EMAP design and assessment approach that have not been adequately addressed such as: the ability to detect changes at an appropriate scale, whether the sampling return period is adequate to detect cyclical events, and the efficacy of specific "stressor cognitive" indicators. EMAP should continue to evaluate features of the design in the light of its ability to detect important ecological change in a meaningful, timely, and useful manner. Indicators A fundamental premise of EMAP is that the status of large and complex ecological systems can be assessed and monitored using a limited set of indicators. Choosing appropriate indicators has been a major focus of EMAP activity since the program began. Despite the obvious centrality of indicator development to EMAP,

OCR for page 18
--> the completion of a comprehensive indicator strategy document has been slow in coming. An early version of a strategy document (Olsen, 1992) was withdrawn following a major program reorganization. A new indicator strategy document was developed more recently and was received by the committee in the spring of 1994 (Barber, 1994). A strong reliance on biological measurements is highly appropriate for a monitoring program with EMAP's goals. Because environmental managers increasingly emphasize issues of biological integrity and integrated approaches to watershed and ecosystem management, environmental monitoring programs can no longer rely solely or primarily on measuring physical and chemical conditions of ecosystem quality. In contrast to chemical indicators, which tend to reflect short-term or instantaneous conditions, biological indicators integrate conditions over time. This attribute is especially important for a monitoring program in which resource units are sampled at most once per year. In addition, advances in the development of multispecies biological indices using multivariate statistical methods offer some hope of defining and quantifying certain aspects of ecosystem status, and perhaps will allow useful quantification of the still nebulous concept of appropriate biological diversity, and the even more indeterminate concepts of ecological health and biological integrity. Exciting advances offer opportunities for development of innovative, molecular-level measurements of ecosystem functioning and responses to stress. Examples include the use of genetic markers (gene probes) to detect the presence of certain organisms or types of organisms, molecular indicators to detect the exposure of organisms to classes of toxic compounds, and molecular measures of biological functions. Moreover, EPA needs information about biota in the nation's ecosystems that goes beyond questions of biological diversity and structural aspects of biological integrity. For example, there is a long-term need to test biota for levels of toxic contaminants, not only because of human health issues but also because of concerns about the health of animals that accumulate contaminants from lower levels of the food-web (e.g., DDT and birds of prey).

OCR for page 18
--> On the other hand, there are several serious difficulties that must be overcome before EMAP is able to place more reliance on biological indicators in its long-term monitoring and assessment strategy. Many of these problems have been recognized by EMAP scientists in various documents discussing selection of indicators for specific resource groups (see Appendix A) and in the recent indicator strategy document. The following are among the most important of these difficulties. First, virtually every characteristic of biological integrity one can think of has an extremely wide range of "appropriate values," depending on the nature of the ecosystem being considered. There are few, if any, absolute biological characteristics that identify an ecosystem as healthy, pristine, or undegraded. Beginning in 1991, the EMAP-Estuaries pilot study used an index that is compared to a local reference system (NRC, 1994a). This approach could be extended to other parts of EMAP. The extent to which a system is degraded is decided by comparison with similar systems. Second, biological variables do not respond in simple or linear ways to stress, creating special statistical difficulties in developing indicators for them. Third, many community and ecosystem-level measures of ecosystem function are quite insensitive to stress. Fourth, from a practical perspective, taxonomic identifications can be tedious, time-consuming, and expensive, thus perhaps limiting the number of samples that can be measured. Finally, the most crucial problem is that for most ecosystems there are no quantifiable biological indicators of ecosystem health, biological integrity, or of several other societal values associated with ecosystems. A major research and development effort will be needed before workable, reliable, and cost-effective measures are available. To date, EMAP has not developed such an aggressive and comprehensive research program for development of biological indicators, and this is perhaps the most important research need facing EMAP. Given the scope of the problem, EMAP should initiate a major, focused research program on indicator development. This is a formidable scientific challenge. If it can be successfully met, it will need the involvement of many scientists for a period of at least 5 to 10 years. It should include a combination of research by EMAP scientists and external research that involves open

OCR for page 18
--> announcements of funding availability with peer reviewed grants. The difficulty and importance of this research requires that EPA attract the highest quality researchers in the environmental sciences to this program. Some of the needed research and development can be accomplished in association with existing pilot monitoring activities, but fundamental research also will be needed that cannot be addressed by sampling at the spatial and temporal scales of the pilot programs. As part of this indicator development program, scientists will need to study the relationship of the indicators being considered to assessment end points and the statistical properties and power of potential indicators. The research program should be directed not only at development of biological indicators appropriate for the various resource groups but also at applying new advances to develop new measures of ecosystem function as well as ecosystem structure. EMAP Sampling Density And Sampling Frequency EMAP has developed a probability-based sampling design to address certain questions regarding the status and trends of ecological resource populations of interest in the United States. EMAP focuses on regional surveys rather than a "sentinel- site" approach; an EMAP objective is to select samples that are spatially well distributed so that the results will apply to fairly large areas. EMAP is willing to forego intensive, site-specific monitoring data in return for reliable data on regional changes and trends in regional population statistics (Messer et al., 1991). According to Foley (1994), assessment of temporal trends using sentinel-site-based monitoring will be left to "other ORD [EPA's Office of Research and Development], federal, and academic programs, because according to EPA, site-based information alone is insufficient for detecting `meaningful trends at scales relevant to policy decisions'." The EMAP hexagonal, grid design does make it possible to sample at varying spatial densities such that the base grid is a subset of higher density grids (Stevens, 1993). Sites will be revisited every four-years. EMAP claims that the 4-year interval is consistent with the time scale of trends that EMAP must de-

OCR for page 18
--> tect. "Trends that result in immediate extreme changes will be reflected in annual estimates, by other monitoring programs, or by casual observation. However, faint trends require some time before the cumulative change is detectable, and as great a population coverage as possible is needed in order to isolate subpopulations that may respond differently than others" (Stevens, 1993, p. 20). A "large-scale, regional estimates only" approach is not compatible with one that claims to use the same spatial grid density to isolate sensitive subpopulations. By enhancing the grid density, such subpopulations might be identified and monitored. But they would have to be identified in advance by some other program or programs outside the EMAP sampling frameworks, because monitoring an enhanced grid at a regional or national scale would be prohibitively expensive. For instance, the selection of indicators of biological condition for the various resource groups may be driven more by what can be detected in four-years' time than by their ecological relevance. Regarding the issues of detection of temporal or spatial processes outside the current EMAP sampling frameworks, EMAP claims that other federal programs (e.g., LTER, LMER) and state and regional programs including the Regional Environmental Monitoring Assessment Program (REMAP) are really the programs designed to investigate those issues. In that case, the coordination between EMAP and other more site-specific programs should be greatly enhanced. Initially, EMAP had planned to augment the interpenetrating, 4-year-cycle design by sampling some sites annually; the contribution of this added component is described by Urquhart et al. (1993). Annual sampling during the initial years of the survey was mentioned specifically for lakes and streams as a way to increase the power of the EMAP design to detect trends (Larsen et al., 1993). As a result of further simulation studies, this augmentation sampling plan apparently was dropped and no lakes will be revisited annually for more than two consecutive years. This has the consequence of generating no site-specific information for longer than two years; the EMAP claim is that more information is to be gained by sampling more sites than by extensive repeat sampling at a single site (Appendix A).

OCR for page 18
--> tives and for summarizing and reporting data. Ecologically meaningful units, such as Bailey's or Omernik's ecoregions, should be the primary objects of statistical analysis and data reporting. Summarizing data at this level will facilitate the development of meaningful hypotheses regarding causes and effects. Summarizing data by ecoregions also will allow users to investigate the geographic pattern of responses to stresses in a way that lends itself to further investigations. For example, is a particular ecoregion responding to a regional or global stress such as acid deposition or global warming in the same way throughout its range? If not, why not? Such questions are meaningless if the data are primarily summarized by SFR. Ecological data that are focused on finer scales could be summarized for SFRs secondarily, and the data would then meet or exceed EMAP's data-quality objectives. The Regional Environmental Monitoring and Assessment Program (REMAP) operates at scales and with boundaries more appropriate to ecological processes. REMAP was not a part of the original EMAP concept, but it has become a potentially significant contribution. It uses EMAP indicators and the EMAP probability-based sampling scheme at an intensified sampling density to address local problems. EMAP should reconsider the scale at which the national program collects and reports data. Integration Need for Integration As the first national scale, multiresource monitoring program, EMAP represents a potentially significant addition to the myriad environmental monitoring programs run by other agencies. Two aspects of EMAP make it unique: the probability-based sampling design, and the inclusion of all of the biomes in the coterminous United States. In initial EMAP literature, this second aspect, inclusiveness of all resources, was the major justification for a new environmental monitoring program. The committee believes that integration among EMAP resource groups is crucial for the following reasons.

OCR for page 18
--> To a large extent, major trends within resource types may only be explained by interactions among resources. For example, most pollution problems occur across resource groups defined by EMAP. An explanation of a forest's condition may require the assessment of regional, urban or agricultural practices as they influence nitrogen gas emissions. Virtually every national environmental issue to date has involved such connections. Integration among resource groups explains trends in ecological status that are controlled by spatially explicit source-sink relationships. A large, national, cross-resource monitoring program could lead to important new advances in our knowledge of interactions among resources. Demonstrations of Integration EMAP has given several indications that it plans to carry out integration among resource programs in the future. The committee has been informed that the resource groups are developing plans to use common methods. The fact that EMAP has or is developing a central sampling design, indicator development strategy, and information management system also suggests that EMAP is planning for future integration. Additionally, the presence of EMAP Center as a central administration provides support for integration. Perhaps most promising, the new Integration and Assessment Program suggests that major new advances can be made with the integration of the program. Concerns The committee recognizes that a large, complex program like EMAP must continue to develop and evolve, and that this evolution is a process of iteration between the conceptual, integrative elements (top-down directives), and the empirical, on-the-ground, resource group elements (bottom-up guidance). The top-down directives of the program appear to have weakened considerably since this review began. In 1990, EMAP representatives stated that the Landscape Characterization program and the Indicator

OCR for page 18
--> Development Strategy were the foundations of the integration program for EMAP. Since that time, the Landscape Characterization program has been changed substantially, such that now it appears to be focused entirely on land cover mapping. The original elements that provided integration appear to be gone. These elements included guidance to resource groups on appropriate sampling intensity and locations, and analysis of indicators across resources. Finally, the technical support for cross-resource integration—the information system—still has not been fully developed. In sum, most of the positive aspects of integration listed above are not yet being implemented. Recommendations Coordinated Sampling Between Terrestrial, Aquatic, and Atmospheric Resources1 The groups are now sampling in separate locations resources that would appear to have very important connections because of their hydrologic linkages. The design would be much enhanced by a cooperative sampling scheme among resource groups in which lakes and streams were sampled in watersheds whose terrestrial systems (forests, agroecosystems, or arid systems) also were being sampled. A stratified random system such as this 1   One committee member has been deeply concerned about the apparent lack of communication between senior administrators and possibly senior scientists, in the Air and Deposition Section of EMAP and those in major federal, state, and international agencies (e.g., Canada and Mexico) who are also heavily involved in ecological risk assessments and environmental protection. This member feels strongly that such inter- and indeed intra-agency interactions are essential for effective coordination of monitoring and assessment efforts involving the atmospheric transport, transformations, and deposition (as well as associated intermedia transport) of a wide range of hazardous gaseous and particulate air pollutants.

OCR for page 18
--> would not compromise EMAP's ability to make regional scale generalizations, or to base those generalizations on probability-based samples. The data sets would be considerably stronger because the spatial covariance of the data sets could be used to test hypotheses related to cause and effect relationships. Several decades of measurements may be required to test these relationships on temporal data. In addition, the development of integration and assessment questions that crossresource groups could focus the program and significantly extend its value nationwide. Designate Resources for Integration As EMAP now stands, there are relatively few financial and human resources directed specifically at integration. Such resources could be directed in various ways, but several important needs must be met. Programs directed at integration must have access to the information management system, and must have computer and software resources to generate and test generalizations. One approach would support a team of individuals that focused on developing and addressing the integration and assessment questions. The team would either have one physical location or would be coordinated among resource groups by a central office. Key members of this group would include participants of the Landscape Characterization, Landscape Ecology, and Indicator Development groups. The new Integration and Assessment Program is a positive step in this direction, but there is no description to date of the activities of this program. A second approach would be to provide the funding for Request for Proposals to be extended to the scientific community. This approach could be implemented in a number of ways, either by targeting specific assessment questions, or by allowing the scientific community to develop the more general questions of most interest to EMAP. An intriguing possibility would include EMAP support (both financial and data support) of the new National Science Foundation Center for Ecological Analysis and Synthesis. The center is intended to serve as a resource for the

OCR for page 18
--> ecological community at large, and accessibility to EMAP data and technology would likely provide a great incentive for cross-resource analysis, assessment of ecology-policy links, and analysis of long-term and large-scale ecological status and trends. Program Coordination Within EMAP EMAP has improved its internal coordination over the past several years. The creation of EMAP Center has resulted in a concentration of technical personnel who provide support to the resource groups. During the reviews of the Surface Water, Forest, and Estuary programs, and during meetings with the NRC committee, EMAP staff commented favorably on coordination between EMAP Center and the resource groups. The recent publication of a newsletter facilitates communication among all components of EMAP as well as those outside of EMAP. None-theless, improvement is needed. Several features of EMAP inhibit internal coordination. EMAP personnel are dispersed in laboratories and offices across the country. Personnel in each resource group are typically clustered together, but they are separate from other resource groups and from EMAP Center. In addition, the autonomy of resource groups appears nearly absolute. The combination of geographic separation and administrative autonomy makes internal coordination difficult at best. The committee recommends that EPA consider the advantages of assigning all EMAP administrative personnel to a common location. Report review within EMAP appears to be cumbersome. Reports delivered to the committee for review are typically based on information gathered two or more years previously. During the early stages of a new program, it is especially important to learn from experience and to modify the program based on the lessons learned. It is not clear that experience is used in a timely and efficient manner to improve subsequent work. Possibly, the slowness of report reviews is at least partially responsible for such inefficiency. EPA should consider means by which report generation and review can be speeded up. Centralization of personnel, reports in peer reviewed journals, use of external review-

OCR for page 18
--> ers, reliance on scientific criteria for reviews, and reduced sensitivity to being wrong and to the prevailing political philosophy are potential means to shorten the time for report review. The rate of turnover in personnel appears high in comparison with other scientific programs and activities. EMAP has had three directors, three associate directors are leaving, and several resource and support groups have had multiple leaders. Some of this turnover is inevitable as positions are filled by contract employees with statutory limits on service. For example, the director and the coordinator of the Agroecosystem Resource Group serve under term-limited Interagency Personnel Agreements. Such turnover further disrupts coordination among EPA personnel. The committee questions whether contract employees can be as effective as direct employees in facilitating coordination within EPA. Individual relationships are the basis of collegiality and collaboration. Relationships between EPA employees and term-limited contract employees are frequently disrupted, inhibiting coordination. All federal agencies have complex hiring requirements, but EPA must address these issues. EPA should commit the senior-level positions required to assure continuity within the management of this important program and recruit qualified people to fill them. Coordination between EMAP and other parts of EPA Through its REMAP program, EMAP has demonstrated excellent coordination with its regional offices. Representatives from regional offices report that they have found the EMAP design and staff useful in assessing issues within the regions. EMAP should be commended for its efforts in finding the means to interact with its regional offices for mutual benefit. EPA senior management should facilitate close working relationships between EMAP and appropriate offices and divisions of EPA. For example, EMAP should continue its efforts to develop close working relationships with the Water Office of EPA to capture the benefits of EPA's past experience in collecting data on surface waters. In addition, EMAP should coordinate closely with other research programs in the Office of Research and Develop-

OCR for page 18
--> ment. EMAP's partnership with the Exploratory Research Program in administration of the solicitation for proposals for indicator development has been productive. Continued reliance on the experience of such programs leverages EMAP's resources and brings complementary expertise to the program. External Scientific Review EMAP is attempting to carry out a number of approaches that are new to ecology, including using indicators of ecological status or health, developing and applying indicators in many types of ecosystems, and making measurements over an entire continent. All of these novel approaches require the best possible judgment from the most experienced researchers. Is EMAP receiving the external scientific review it needs to carry out its mission? All parts of EMAP have brought in external reviewers at various stages of planning and implementation. For example, the indicator development program held extensive workshops of EPA contractors and independent scientists to come up with ideas about possible indicators. Panels of experts have advised the various resource groups, and the EMAP reports have received extensive review by up to 100 reviewers, many of them from outside of EPA. One recent activity, an EMAP-initiated funding of projects to develop new and better ecological indicators, used a National Science Foundation panel for scientific review and ranking of the proposed projects. One successful use of external reviewers was carried out at the early stages of EMAP-Estuaries. At the request of EMAP, the Estuarine Research Federation (ERF) set up an expert panel that was briefed by EMAP and produced reports based upon three meetings held a year apart. The ERF panel and EMAP scientists met at the end of each year to review progress and discuss problems. Participants report that the advice from the ERF panel was taken seriously. Several years later, the estuaries panel of this NRC committee found that the ERF reports dealt very well with the strengths and weaknesses of the EMAP program. EMAP has been willing to use external reviewers more than any other agency program. Nonetheless, the review process

OCR for page 18
--> could be improved. First, it is important to use external reviewers as advisors as well as reviewers. The advice of these external reviewers should be used in designing parts of the program and its overall structure, as well as in reviewing finished or partly finished activities. Second, at least in some cases, the reviewers could have been chosen better. For example, however skilled and knowledgeable EMAP contractors and employees might be, inasmuch as they have helped to design or implement the program, they are not independent of it, and therefore they should not be used as reviewers. Also, there is value to having reviewers who are active in other, related fields and whose funding and scientific publications indicate that they are leaders in developing the scientific research and technologies that EMAP needs. Third, EMAP seems to have convened a new review panel for each task. Given the complexity of the EMAP program, it is a major task to bring each panel up to the proper level of understanding. Permanent or long-term scientific review panels for each of the eight resource groups were a part of the EMAP plan, but have not yet been set up. In addition to the need for good external review of these various resource groups and funding initiatives, there is also a need for overview of the entire program that is not filled by reviews of each separate part. The current external review structure of EMAP should be modified so that its core is a permanent panel, with rotating membership, to provide continuity. This panel would advise both at the level of resource groups, such as the arid land or estuary resource level, and at the level of the entire EMAP program. The advice will be taken more seriously if good relationships are established between members of the panel and the directors of the various sections of EMAP. Above all, best scientists have to be recruited as advisors. Advice about membership should be sought from scientific societies, as was done with the Estuarine Research Federation, and from the program managers at the National Science Foundation, the Department of Agriculture, and the Department of Energy who organize competitive research programs in ecology. A structure for permanent advisory panels was proposed by EMAP personnel but has not been implemented.

OCR for page 18
--> EMAP's Place In The Federal Government The Administrative Setting for EMAP EMAP is a large program that will require an appropriate administrative setting to be successful. It is important to reflect on the extent to which EPA has provided an appropriate administrative environment for EMAP. Characteristics of the ideal administrative setting that would enhance EMAP's chances of success are described below. Of course, no agency has all these characteristics. Indeed, some of the characteristics are somewhat incompatible with each other. Conversely, some of the characteristics are related to each other—the first two, for example. EMAP's parent agency should be nonregulatory. A monitoring program housed within a regulatory agency will face problems of internal conflict of interest changing priorities, and data confidentiality. This may lead to a higher rate of denial of access to private lands. In addition, the need for short-term information for regulation might compromise the ability of the agency to commit resources to long-term monitoring programs. The agency needs to make EMAP one of its highest priorities internally and in its presentations to Congress. Large fluctuations in funding will seriously damage a program such as EMAP. The agency should have a strong administrative and scientific team capable of providing the initiative and scientific leadership required for such a large and highly visible program. The agency should make the commitment of long-term as well as rotating positions for key leadership and scientific advisory personnel. The agency should have strong familiarity with each of the resource types being monitored. The agency should have the capability to carry out strong research programs affiliated with EMAP to answer detailed questions raised by EMAP data.

OCR for page 18
--> The agency should have a strong scientific reputation, making it easier to attract top scientists to EMAP. The agency should be in close communication with agencies that will administer policy derived from data collected. EPA clearly has several of these characteristics. EPA scientists conceived the concept of EMAP and were successful in initiating and implementing its predecessor, the National Surface Waters Survey. Certainly, EPA will have some regulatory responsibility in enforcing policy derived from EMAP data, and lines of communication between monitoring and regulatory personnel should be strong within a single agency. EPA in its regional offices has started the promising Regional Environmental Monitoring Assessment Program to address specific questions using an EMAP-like approach. Close ties between REMAP and EMAP personnel will be mutually beneficial. On the other hand, EPA does not have some of the ideal characteristics. For some of this there may be little EPA can do. For example, EPA is a regulatory agency. That regulatory role was mandated by Congress. Although EPA cannot resolve all the conflicts associated with its regulatory role, it can take steps to reduce their impact, as other agencies have done, by separating research and monitoring functions as much as possible from regulatory ones. There are several other areas where EPA could act to improve the administrative setting of EMAP. As much as possible, funding for EMAP needs to be long-term and predictable. Fluctuations or uncertainty in funding levels will be seriously damaging to a monitoring program this large and ambitious. Within the constraints imposed by the congressional funding process, EPA must act to ensure the institutional commitment to this long-term monitoring effort. EPA should find some mechanism to allow flexibility in timely hiring of qualified personnel at all levels of EMAP. It appears that difficulty in hiring and keeping personnel in key positions has hindered EMAP's progress. While some of the difficulties in hiring personnel are beyond EPA's control, high enough priority may not have been given within the agency to filling key positions, such as the Indicator Coordinator. Finding ways to attract and keep top scientists in EMAP will

OCR for page 18
--> strengthen the program overall and lead to an enhanced scientific reputation.