National Academies Press: OpenBook
« Previous: Front Matter
Suggested Citation:"Executive Summary." National Research Council. 1995. Review of EPA's Environmental Monitoring and Assessment Program: Overall Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4931.
×

Executive Summary

EPA's Environmental Monitoring and Assessment Program (EMAP) was established to provide a comprehensive report card on the condition of the nation's ecological resources and to detect trends in the condition of those resources. At EPA's request, the National Research Council's Board on Environmental Studies and Toxicology and Water Science and Technology Board established the Committee to Review EPA's Environmental Monitoring and Assessment Program. This fourth and final report is the committee's overall evaluation of the program.

In 1988, the Science Advisory Board of the U.S. Environmental Protection Agency recommended that EPA ''undertake research on techniques that can be used to help anticipate environmental problems," and that "an office be created within EPA for the purpose of evaluating environmental trends and assessing other predictors of potential environmental problems before they become acute".

Following the Science Advisory Board's advice, EPA established the Environmental Monitoring and Assessment Program (EMAP) "to monitor ecological status and trends, as well as to develop innovative methods to anticipate emerging environmental problems before they reach crisis proportions". In 1993 EMAP's stated goals were to:

Suggested Citation:"Executive Summary." National Research Council. 1995. Review of EPA's Environmental Monitoring and Assessment Program: Overall Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4931.
×
  1. Estimate the current status, trends, and changes in selected indicators of condition of the nation's ecological resources on a regional basis with known confidence.
  2. Estimate the geographic coverage and extent of the nation's ecological resources with known confidence.
  3. Seek associations between selected indicators of natural and human stresses and indicators of the condition of ecological resources.
  4. Provide annual statistical summaries and periodic assessments of the nation's ecological resources.

As described by EPA, EMAP is unified by its approach to landscape characterization, the application of a coherent strategy for the choice and the development of indicators, and the use of a probability-based sampling approach that uses a hexagonal grid for identifying sampling sites. There are eight resource groups identified by the program: agroecosystems, arid (now rangeland) ecosystems, forests, the Great Lakes, estuaries, inland surface waters, wetlands (recently subsumed under surface waters and the Great Lakes), and landscape ecology. These resource groups are intended to represent ecosystem types or resources of national interest, and to provide a basis for incorporating ecological knowledge into the design of indicators and sampling programs.

The committee's reviews of other EMAP components such as forests and estuaries and surface waters were published as separate reports. The executive summaries of these reports are in Chapter 4.

After four-years of review, the committee retains its belief that EMAP's goals are laudable. However, because achieving the goals of this ambitious program will require that EMAP successfully meet many difficult scientific, practical, and management challenges, the committee continues to question whether and how well all these goals can be achieved. This final report reiterates that general assessment.

Suggested Citation:"Executive Summary." National Research Council. 1995. Review of EPA's Environmental Monitoring and Assessment Program: Overall Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4931.
×

As first conceived and presented to the committee in 1991, EMAP was significantly different than it is today. Several of its central features and components seem to have less importance in mid-1994 than they did in 1991. The reverse is also true: the resource groups have become much more important and are leading the program. One of the major strengths of EMAP as initially presented was that it planned to integrate information across regions and across resource types, but the nature and extent of that integration is still not clear.

Given the need for 10 years or more of data to sample regions and distinguish trends, nobody—including the members of this committee—can be certain whether, or how fully, EMAP will achieve its stated goals. This is to be expected for a large, ambitious, and novel program like EMAP. However, the program-wide concerns expressed in the committee's previous reports, in Chapter 2 of this report, and summarized below, are so important that EMAP will have little chance of achieving its goals if they are not addressed. Concerns revolve around the following issues.

  • The EMAP sampling program may operate at too coarse a scale in space and time to detect meaningful changes in the condition of ecological resources.
  • EMAP's success will be diminished if it does not develop reliable, scientifically defensible indicators for measuring change. The development of indicators of ecological health or integrity and of aesthetic quality appear to be particularly challenging.
  • EMAP's success will be diminished if it does not select the right assessment end points (i.e., the end effect that is the goal of the monitoring program), something it has not done so far.
  • EMAP's success will be diminished if the retrospective or prospective monitoring approach does not match the assessment needs and the needs of policy makers.
  • EMAP needs to incorporate the best scientific advice in the design, implementation, and review of its program.
  • EMAP has not yet fulfilled its promise of innovation and national comprehensiveness because the programs to integrate
Suggested Citation:"Executive Summary." National Research Council. 1995. Review of EPA's Environmental Monitoring and Assessment Program: Overall Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4931.
×
  • information across space, time and resource types have not been developed. The most important of these are an indicator development strategy, information management, and landscape characterization.
  • EMAP's information management system must support efficient access to a large, distributed database and application of an appropriate range of information processing tools.
  • Lack of continuity in staffing at EMAP has inhibited development of the program. EMAP cannot succeed unless the government (i.e., the administration and the Congress) makes a sufficient financial commitment to EMAP to support administrative and technical excellence, continuity, and efficiency in program management. That commitment is necessary for EMAP to succeed, but is not sufficient by itself.

A September 1994 letter from EMAP director Edward Martinko (Appendix A) describes EMAP's recent responses to earlier NRC reports and provides additional updates about the program. Many of the changes described appear to be in line with the earlier committee recommendations. EMAP has not provided more detailed documentation of these encouraging changes, so this report has not been substantially altered. However, recommendations in this report that deal with matters directly addressed by Dr. Martinko's letter are indicated with an asterisk.

Recommendations

Statistics, Sampling, and Design

  • EMAP should consider design changes that would increase the probability of detecting smaller scale ecological changes. Some possibilities include increasing revisitation rates at a subset of sample sites; inclusion of a set of nonrandomly selected sentinel-sites with intensive data-collection, such as the Long
Suggested Citation:"Executive Summary." National Research Council. 1995. Review of EPA's Environmental Monitoring and Assessment Program: Overall Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4931.
×
  • Term Ecological Research (LTER)/Land Margin Ecosystems Research (LMER) networks; and stratified random sampling by ecoregion with data-quality objectives specified for strata. If EMAP does not adopt these design changes itself, then it should become extremely closely and explicitly coordinated with a program that has these features.
  • EMAP should consider further combining effects-oriented and stressor-oriented monitoring approaches. Predictive, or stressor-oriented, monitoring seeks to detect the cause of an undesirable effect (a stressor) before the effect occurs or becomes serious. Retrospective, or effects-oriented, monitoring seeks to detect the effect after it has occurred. EMAP has relied mostly on the latter. Stressor-oriented monitoring will increase the probability of detecting meaningful ecological changes. As in the above point, if EMAP does not adopt these changes, it should become closely coordinated with a program that monitors in this way.
  • EMAP should undertake power analyses regarding the effectiveness of the sampling design for each resource group.* A power analysis is an analysis of the statistical strength of an approach to detect change if a change exists. Different resource groups have adopted different sampling approaches. All the resource groups should adopt the practice of the EMAP lakes component, which has assigned teams of statisticians to assess the effectiveness of EMAP for that particular resource.
  • EMAP should reconsider its detection criterion of a 20 percent change over 10 years. In some systems, such a large change is unlikely to occur in nature, while in other systems, a much smaller change would elicit concern. EMAP should also consider systems or indicators for which a change in the variance, rather than mean or median, is important.

*  

Recommendations marked with an asterisk are addressed in Dr. Martinko's 9/20/94 letter describing recent changes in EMAP.

Suggested Citation:"Executive Summary." National Research Council. 1995. Review of EPA's Environmental Monitoring and Assessment Program: Overall Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4931.
×

Indicators

  • EMAP should initiate a major, focused research program on indicator development.* Indicator development is at the heart of the EMAP program. Without a well-considered set of indicators for each resource group, EMAP will not fulfill its goal of presenting an evaluation of the nation's ecological resources. The difficulty and importance of indicator development requires EPA to attract the highest quality researchers in the environmental sciences to this program. The program should include a combination of internal research (by EMAP scientists) and external research involving open announcements of funding availability with peer reviewed grants.
  • Each EMAP resource group should develop one or more mechanistic conceptual models of its resource, based on current scientific knowledge. * These models should serve as explicit hypotheses about those aspects of ecosystem structure and functioning relevant to the assessment end points the group has chosen. The models must be detailed enough to include potential indicators, assessment end points, and key variables.
  • EMAP should provide program-wide guidance for numerous evaluation issues if the indicator selection strategy is going to yield the nationally applicable set of indicators EMAP envisions. The committee recommends as a high priority the explicit and early evaluation of the statistical properties of all potential indicators. Such evaluations should include analyses of the properties of the mean, variance, and behavior of the index in power tests. If this cannot be done analytically, then simulation analyses should be performed.
  • EMAP should carefully evaluate each potential indicator at incrementally larger spatial scales. EMAP needs to make sure that it has information on the domains of usefulness of its indicators—at what spatial and temporal scales are they reliable, and at what scales are they less reliable? The ways in which the various resource groups deal with this problem will have important consequences for the selection of nationally implemented
Suggested Citation:"Executive Summary." National Research Council. 1995. Review of EPA's Environmental Monitoring and Assessment Program: Overall Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4931.
×
  • indicator metrics. Program-wide strategies for dealing with this issue should be developed now, in time to be applied with some uniformity across the resource groups.

Integration

  • EMAP should develop key integration and assessment questions that cross-resources. This would help focus the program and significantly extend its value nationwide.
  • EMAP should designate resources for integration. As EMAP now stands, there are relatively few financial or other resources directed specifically at integration. Such resources could be directed in various ways, but several important needs must be met. Individual resource programs directed at integration must have access to the information management system, and must have computer and software resources to generate and test generalizations. One approach would support a team of individuals who focus on developing and addressing the integration and assessment questions, and who either work together at one physical location or were coordinated among resource groups by a central office. Key members of this group must be participants of the Landscape Characterization, Landscape Ecology, and Indicator Development groups. The new Integration and Assessment Program is a positive step in this direction, but we do not have a good description of the activities of this program.
  • EMAP should develop coordinated sampling between terrestrial, aquatic, and atmospheric resources.1 Resources

1  

One committee member has been deeply concerned about the apparent lack of communication between senior administrators and possibly senior scientists, in the Air and Deposition Section of EMAP and those in major federal, state, and international agencies (e.g., Canada and Mexico) who are also heavily involved in ecological risk assessments and environmental protection. This

Suggested Citation:"Executive Summary." National Research Council. 1995. Review of EPA's Environmental Monitoring and Assessment Program: Overall Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4931.
×
  • appearing to have very important ecological connections due to hydrologic linkages are now being sampled in separate locations. The design would be enhanced by a cooperative sampling scheme between resource groups in which lakes and streams were sampled in watersheds whose terrestrial systems (forests, agroecosystems, arid systems) also were being sampled. A stratified random system such as this would not compromise EMAP's ability to make regional scale generalizations based on probability-based samples. The data sets would be considerably stronger because the spatial covariance of the data sets could be used to test hypotheses related to cause and effect relationships.

    Possible examples include indicators reflecting net primary productivity, biological diversity, and aesthetic value. At present, it is unclear whether or not the assessment questions in each resource group are similar enough to lead to parallel sets of indicators. Such symmetry among resource groups, while not essential to basic EMAP objectives, would greatly enhance the scientific and analytical value of the data collected.

Appropriate Scale and Boundaries of Regions

  • EMAP should choose ecologically meaningful units as the primary scale for summarizing and reporting data. Ecologically meaningful units, such as Bailey's or Omernik's ecoregions, should be the primary objects of statistical analysis and data reporting rather than political units or EPA regions. In general,

   

member feels strongly that such inter- and indeed intra-agency interactions are essential for effective coordination of monitoring and assessment efforts involving the atmospheric transport, transformations, and deposition (as well as associated intermedia transport) of a wide range of harzardous gaseous and particulate air pollutants.

Suggested Citation:"Executive Summary." National Research Council. 1995. Review of EPA's Environmental Monitoring and Assessment Program: Overall Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4931.
×
  • EMAP should reconsider the scale and boundaries of units for which the national program summarizes and reports data.

Coordination And Management

  • EMAP is unlikely to succeed unless EPA commits permanent, senior-level positions to the program, and recruits qualified people to fill them. Commitment and continuity are crucial for the implementation of such an innovative national program. Too many important responsibilities in EMAP have been assigned to people on temporary Interagency Personnel Agreements (IPAs) or to contractors.
  • The committee recommends that EPA senior administrators facilitate close working relationships between EMAP and appropriate offices and divisions of EPA, including other research programs in the Office of Research and Development. In particular, EMAP should continue in its efforts to develop close working relationships with the EPA Office of Water to capture the benefits of EPA's past experience in collecting data on surface waters. Continued reliance on the experience of such programs leverages EMAP's resources and brings complementary expertise to the program.
  • EMAP should develop and maintain an administrative structure that demands close communication and interaction among EMAP-LC (Landscape Characterization), EMAP-IM (Information Management), and each of the resource groups. This structure could take several forms, such as locating lead personnel of each of these groups at a central office or some other mechanism that requires regular communication among these groups.
  • The committee recommends that EMAP continue its efforts to coordinate its activities with those of other agencies. The Memorandum of Understanding, signed by National Biological Service director H. Ron Pulliam and EPA Office of Research and Development director Robert Huggett (MOU, September 30,
  • Suggested Citation:"Executive Summary." National Research Council. 1995. Review of EPA's Environmental Monitoring and Assessment Program: Overall Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4931.
    ×
    • 1994) is an excellent example of such coordination. The committee encourages further efforts with programs like the National Water Quality Assessment of the U.S. Geological Survey.

    External Scientific Review

    • The current external review structure of EMAP should be modified so that its core is a permanent panel, with rotating membership, to provide continuity. A permanent board of accomplished scientists may provide more expertise and consistency of viewpoint than EMAP has had access to heretofore. The panel should advise both at the level of resource groups, such as the forests or estuary resource level, and at the level of the entire EMAP program.

    Information Management

    • While top-down planning for EMAP's information system is important, EMAP should base such planning on the viewpoint that the information system is a scientific database system, rather than an information system focused on the needs of management if the Information Management System is to function and facilitate integration among research groups as envisioned by EMAP. In particular, the planning should focus on the design of an environment that is sensitive to user requirements and that provides excellent hardware, software, and support personnel.
    Suggested Citation:"Executive Summary." National Research Council. 1995. Review of EPA's Environmental Monitoring and Assessment Program: Overall Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4931.
    ×
    Page 1
    Suggested Citation:"Executive Summary." National Research Council. 1995. Review of EPA's Environmental Monitoring and Assessment Program: Overall Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4931.
    ×
    Page 2
    Suggested Citation:"Executive Summary." National Research Council. 1995. Review of EPA's Environmental Monitoring and Assessment Program: Overall Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4931.
    ×
    Page 3
    Suggested Citation:"Executive Summary." National Research Council. 1995. Review of EPA's Environmental Monitoring and Assessment Program: Overall Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4931.
    ×
    Page 4
    Suggested Citation:"Executive Summary." National Research Council. 1995. Review of EPA's Environmental Monitoring and Assessment Program: Overall Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4931.
    ×
    Page 5
    Suggested Citation:"Executive Summary." National Research Council. 1995. Review of EPA's Environmental Monitoring and Assessment Program: Overall Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4931.
    ×
    Page 6
    Suggested Citation:"Executive Summary." National Research Council. 1995. Review of EPA's Environmental Monitoring and Assessment Program: Overall Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4931.
    ×
    Page 7
    Suggested Citation:"Executive Summary." National Research Council. 1995. Review of EPA's Environmental Monitoring and Assessment Program: Overall Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4931.
    ×
    Page 8
    Suggested Citation:"Executive Summary." National Research Council. 1995. Review of EPA's Environmental Monitoring and Assessment Program: Overall Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4931.
    ×
    Page 9
    Suggested Citation:"Executive Summary." National Research Council. 1995. Review of EPA's Environmental Monitoring and Assessment Program: Overall Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4931.
    ×
    Page 10
    Next: 1 Introduction »
    Review of EPA's Environmental Monitoring and Assessment Program: Overall Evaluation Get This Book
    ×
    Buy Paperback | $48.00
    MyNAP members save 10% online.
    Login or Register to save!
    Download Free PDF

    This is the fourth and final volume reviewing EPA's Environmental Monitoring and Assessment Program (EMAP). After 4 years of review, the authoring committee retains its belief that EMAP's goals are laudable. However, because achieving the goals of this ambitious program will require that EMAP successfully meet the difficult scientific, practical, and management challenges, the committee continues to question whether and how well all these goals can be achieved. This final overall review reiterates that general assessment.

    1. ×

      Welcome to OpenBook!

      You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

      Do you want to take a quick tour of the OpenBook's features?

      No Thanks Take a Tour »
    2. ×

      Show this book's table of contents, where you can jump to any chapter by name.

      « Back Next »
    3. ×

      ...or use these buttons to go back to the previous chapter or skip to the next one.

      « Back Next »
    4. ×

      Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

      « Back Next »
    5. ×

      Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

      « Back Next »
    6. ×

      To search the entire text of this book, type in your search term here and press Enter.

      « Back Next »
    7. ×

      Share a link to this book page on your preferred social network or via email.

      « Back Next »
    8. ×

      View our suggested citation for this chapter.

      « Back Next »
    9. ×

      Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

      « Back Next »
    Stay Connected!