4
Findings and Recommendations

The fishery management councils and that National Oceanic and Atmospheric Administration (NOAA) Fisheries have the difficult task of making decisions that affect people’s livelihoods and the health of the nation’s fishery resources. Sometimes these decisions are based on scientific information that contains substantial uncertainty. In addition, significant regional differences exist in fisheries, in the social and economic constraints on the industries involved, and in the types and amounts of scientific information available to decision makers. It will not be possible to eliminate discontent with management decisions among some stakeholders; however, guidelines for the “best scientific information available” (National Standard 2) could improve the credibility of and the confidence and trust in scientific information used to manage the nation’s fisheries.

This chapter has two objectives. First, it summarizes the findings that have been derived from examination of the application of the term “best scientific information available” in the preparation of federal fishery management plans (FMPs). Second, it provides recommendations for the uniform application of National Standard 2. The recommendations include proposed guidelines for the application of National Standard 2; how and when National Standard 2 should be employed; how adherence to the standard should be measured; and whether National Standard 2 should be applied to exclude information deemed inadequate or to rank information that would be used in relation to its relevance and rigor.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 51
Improving the use of the “Best Scientific Information Available” Standard in Fisheries Management 4 Findings and Recommendations The fishery management councils and that National Oceanic and Atmospheric Administration (NOAA) Fisheries have the difficult task of making decisions that affect people’s livelihoods and the health of the nation’s fishery resources. Sometimes these decisions are based on scientific information that contains substantial uncertainty. In addition, significant regional differences exist in fisheries, in the social and economic constraints on the industries involved, and in the types and amounts of scientific information available to decision makers. It will not be possible to eliminate discontent with management decisions among some stakeholders; however, guidelines for the “best scientific information available” (National Standard 2) could improve the credibility of and the confidence and trust in scientific information used to manage the nation’s fisheries. This chapter has two objectives. First, it summarizes the findings that have been derived from examination of the application of the term “best scientific information available” in the preparation of federal fishery management plans (FMPs). Second, it provides recommendations for the uniform application of National Standard 2. The recommendations include proposed guidelines for the application of National Standard 2; how and when National Standard 2 should be employed; how adherence to the standard should be measured; and whether National Standard 2 should be applied to exclude information deemed inadequate or to rank information that would be used in relation to its relevance and rigor.

OCR for page 51
Improving the use of the “Best Scientific Information Available” Standard in Fisheries Management FINDINGS The first set of findings is drawn from a previous National Research Council report titled Science and Its Role in the National Marine Fisheries Service (National Research Council, 2002a), which is relevant to this study: The use of science in the marine fisheries management decision-making process is impeded by the governance system created by the [Magnuson-Stevens Act] and the resulting mismatch between institutional authorities and responsibilities. A better structure to conduct science in [NOAA Fisheries] would improve outsiders’ perceptions of [NOAA Fisheries] scientists and science. A structure that allowed scientists to operate objectively and independently of the management body (but was responsive to requests for scientific investigations) could improve both the image and the performance of [NOAA Fisheries]. Chapter 2 includes a review of the legislative history of the phrase “best scientific information available,” describes the application of National Standard 2 by science centers and management councils, and analyzes the courts’ interpretations of National Standard 2. Based on this information, the committee arrived at the following findings: There is an implicit obligation in National Standard 2 to improve scientific information and reduce uncertainty over time. National Standard 2 embodies the idea that decisions regarding management and conservation should be made in a timely and effective fashion with available information despite recognized data gaps. It is not tenable to require a threshold of scientific certainty before making management decisions because National Standard 2 requires that managers make the best possible decisions based on the scientific information available. When presented with an explicit expression of the risks of management options, decision makers are better able to evaluate actions relative to the potential consequences of undesirable or irreversible outcomes.

OCR for page 51
Improving the use of the “Best Scientific Information Available” Standard in Fisheries Management The Magnuson-Stevens Act provides specific guidelines for the development of FMPs; however, no guidelines exist for the uniform application of National Standard 2. Fisheries science centers and fishery management councils report a common interpretation of National Standard 2; however, there are both institutional and regional differences in the application of the standard. There are regional, as well as fishery-specific, differences in the quality and quantity of scientific information available; the means of ranking data quality; and the degree of transparency about data inclusion in the development of stock assessments and FMPs. This makes it difficult, in some cases, to assess the degree to which scientific information is used from council to council. The form and function of the regional fishery management councils’ advisory panels in developing fishery regulations are poorly defined. Therefore the councils need guidelines to ensure accountability for their use of scientific information. Nearly all of the current 118 council members across the eight regional fishery management councils have no background in stock assessment science. Scientific information presented to fishery management councils sometimes is not clearly explained at open council meetings. One frequent consequence of the failure to convince stakeholders that a council has complied with National Standard 2 is litigation by stakeholders dissatisfied with the outcomes of fisheries management policies. Court decisions, through the hard-look doctrine, compel agencies to stay within the constraints of governing legislation, abide by fair procedures, explain and justify their decisions, and manifest a general commitment to reasoned decision making. Chapter 3 examined previous and ongoing efforts to define “best scientific information available.” The examination of those efforts led to the following findings: A statutory definition of what constitutes “best scientific information available” for fisheries management is inadvis-

OCR for page 51
Improving the use of the “Best Scientific Information Available” Standard in Fisheries Management able because it could impede the incorporation of new types of scientific information and would be difficult to amend if circumstances warranted change. Defining “best scientific information available” in legislation is unlikely to reduce the amount of controversy in the application of science to fisheries management. Establishing procedural guidelines is the preferred alternative for creating accountability and enhancing the credibility of scientific information used in fisheries management. There is widespread agreement in both scientific and policy communities on the criteria and procedures for determining the “best scientific information available.” RECOMMENDATIONS NOAA Fisheries should implement the guidelines presented below to govern the production and use of scientific information in the preparation of FMPs and supporting documents. The purpose of the guidelines is to promote consistency in both the production and the use of scientific information without unduly constraining the ability of scientists to adopt new scientific protocols for data collection and analysis. Procedural consistency would provide NOAA Fisheries with a stronger basis for defending controversial management decisions in court. More specifically, guidelines that address issues of relevance, inclusiveness, objectivity, transparency, timeliness, peer review, and the treatment of uncertainty are consistent with the procedural cues that have been sought in court cases. The procedures used for producing and obtaining scientific information should be uniform. This would provide greater consistency across regions so that given the same information, each region would be expected to develop similar assessments of what is the “best scientific information available.” The guidelines must be sufficiently flexible to accommodate the strong regional differences in fisheries and the scientific information available. The guidelines should be used by fisheries science centers, fishery management councils, and all other entities or parties that produce and/or use scientific information for fisheries management. A statutory definition of “best scientific information available” is inadvisable, but there is a need for clear guidelines that create explicit

OCR for page 51
Improving the use of the “Best Scientific Information Available” Standard in Fisheries Management and objective standards for deciding whether National Standard 2 has been met. A statutory definition would unduly restrict the incorporation of scientific advances into policy, thus increasing rather than decreasing the current gap between scientific information and the policy it is supposed to support. The most constructive alternative to such a definition is the development of agency guidelines that provide standards for ensuring consistency in the application of “best scientific information available,” as required by National Standard 2. However, scientific information is not readily categorized by its quality because it is conditional on the current state of knowledge. What is best at one point may be obsolete in the next as new data and analyses become available. The guidelines for ensuring the use of “best scientific information available” in fisheries management are based on the following widely accepted criteria (identified in Chapter 3): relevance, inclusiveness, objectivity, transparency and openness, timeliness, and peer review. Guidelines Relevance—Scientific information should be representative of the fish stock being managed, although the data need not be site specific or species specific. In some cases, analogous information from a different region or the biological characteristics of a related species or species with similar life-history strategies will be informative and relevant and may constitute the best information available. Stock assessments and economic and social impact assessments should clearly describe the strengths and weaknesses of the data used in analyses. Inclusiveness—Scientific advice should be sought widely and should involve scientists from all relevant disciplines. The goal should be to capture the full range of scientific thought and opinion on the topic at hand. Critiques and alternative points of view should be acknowledged and addressed openly.

OCR for page 51
Improving the use of the “Best Scientific Information Available” Standard in Fisheries Management Anecdotal (experiential, narrative or local) information should be acknowledged and evaluated during the process of assembling scientific information. When no other information is available, anecdotal information may constitute the best information that is available. In addition, anecdotal information can be used to help validate other sources of information and identify topics for research. Objectivity—Data collection and analysis should be unbiased and obtained from credible sources. Scientific processes should be free of undue nonscientific influences and considerations. Transparency and Openness—Congress has enacted laws intended to give the public full and open access to the development of federal policies, including advisory meetings, background documents, and other sources of information. Accordingly, the public should have information about each phase of the process from data collection to data analysis to decision making. Decision makers should provide a clear rationale for the choice of the information that they use or exclude when making management decisions. The processes of collecting data and selecting research for use in support of management decision making should be open, broad based, and carefully documented. All scientific findings and the analysis underlying management decisions should be readily accessible to the public. The limitations of research used in support of decision making should be identified and explained fully. Timeliness—There are at least two aspects to timeliness. First, timeliness refers to the acquisition of data in such a manner that sufficient time exists to analyze it adequately before it is used to make management decisions. Second, timeliness refers to whether the data are applicable to the current situation. Management decisions should give greatest weight to the most recent, reliable data available. Some types of information, such as the life-history characteristics of a species of fish, may not change over time do they remain current. Other types of information, such as population survey data, have to be updated on a regular basis. Timeliness can also mean that in some cases, results of important studies and/or monitoring programs must be brought forward

OCR for page 51
Improving the use of the “Best Scientific Information Available” Standard in Fisheries Management before the scientific team feels that the study is complete. Uncertainties and risks that arise from an incomplete study should be acknowledged, but interim results may be better than no new results at all. Management decisions should not be delayed indefinitely on the promise of future data collection or analysis. Except under extraordinary circumstances, FMP implementation need not be delayed to capture and incorporate data and analyses that become available after plan development. Peer Review—Peer review is the most accepted and reliable process for assessing the quality of scientific information. Its use as a quality control measure enhances the confidence of the community (including scientists, managers, and stakeholders) in the findings presented in scientific reports. Peer review is not infallible, but it has proved valuable for uncovering errors in, and providing diverse perspectives on, data collection, analysis, and interpretation. This includes cases in which documentation of the scientific information would be insufficient to validate or reproduce the results of an analysis of a given set of data. Reproducibility of data analysis is one important method for ensuring the validity of scientific information. NOAA Fisheries should establish an explicit and standardized peer review process for all documents that contain scientific information used in the development of FMPs. This is similar to a recommendation by the U.S. Commission on Ocean Policy (2004) that “[NOAA Fisheries], working with the Regional Fishery Management Councils and the interstate fisheries commissions, should develop a process for independent review of the scientific information generated by the Scientific and Statistical Committees in all regions.” Each region should have some flexibility to adapt peer reviews to individual circumstances. However, the following key elements should be included: the review should be conducted by experts who were not involved in the preparation of the documents or the analysis contained in them; the reviewers should not have conflicts of interest that would constrain their ability to provide honest, objective advice; all relevant information and supporting materials should be made available for review; and

OCR for page 51
Improving the use of the “Best Scientific Information Available” Standard in Fisheries Management a peer review should not be used to delay implementation of measures when a fishery has been determined to be overfished. Internal peer review of scientific information may be sufficient; however, an external review is advisable (and more consistent with the purpose of the guidelines) when one or a combination of the following circumstances applies: questions exceed the expertise of the internal review team, there is substantial scientific uncertainty, the findings are controversial, or there are a range of scientific opinions regarding the proposed action. Adherence to National Standard 2 NOAA Fisheries should require each fishery management council to provide explicit findings on how scientific information was used to develop or amend a FMP. It is important for the fishery management councils to explain how scientific information in each major component of an FMP was used and also to explain the reasoning when scientific information is presented to it and not used. The guidelines provided in this report allow for more uniform application of National Standard 2 and, if applied, may prevent situations in which the councils disregard scientific advice provided by their scientific and statistical committees. In addition, requiring fishery management councils to be explicit in their handling of scientific information provided sharpens their responsibility. It also will help the Secretary of Commerce to intervene to correct situations where the scientific information has been disregarded or misapplied. It is important for NOAA Fisheries to defend its own scientific information after council decisions. The Secretary of Commerce should steadfastly reject plans that clearly ignore current laws or regulations. Decision makers from the beginning to the end of the FMP process would benefit from clear articulation of and explicit findings on the use of their work products. NOAA science centers could incorporate feedback on how their work products are used to support the needs of the councils in future endeavors. Requirements for explicit findings are a way to assist fishery management councils and to moderate environments of raw politics with reasoned decision making. The findings of an earlier National Research Council (2002a) committee reiterated in this report will help NOAA Fisheries reviewers and the Secretary of Com-

OCR for page 51
Improving the use of the “Best Scientific Information Available” Standard in Fisheries Management merce to address their legal responsibilities for insisting on adherence to National Standard 2. They will also sharpen judicial review and allow courts to be more focused and discrete in their review functions. In addition, compelling the councils to document their interpretation and use of scientific information explicitly will clarify their decision-making processes and would provide the Secretary of Commerce with a clearer rationale for evaluating the merits of FMPs in terms of National Standard 2. Examples, such as the vermilion snapper (Box 2.1) fisheries in the Gulf of Mexico, illustrate that the very structure of the Magnuson-Stevens Act sometimes impairs the goals of employing, adhering to, and applying the “best scientific information available.” The reason for this is that the “best scientific information available” is only one goal among many in a set of national standards that include efficiency, cost minimization, bycatch avoidance, nondiscrimination, and protection of fishing communities. The “science” of the matter is thus in a constant “balancing” competition with other political and economic factors. This competition is ongoing in the review of all FMPs and is decided by fishery management councils having few members with scientific expertise (National Academy of Public Administration, 2002) based on information provided by scientists who often do not communicate risks effectively. In many cases, this situation is a competition that “best science” cannot win. In this environment, the best science is vulnerable to being swamped repeatedly for reasons of economics, convenience, or preference. As currently applied, the science of the subject can be freely passed over, rejected, remanded, and trumped. Steps to protect the independence of the scientific information developed under National Standard 2 may require legislative change and almost certainly will require rule change. For example, the U.S. Commission on Ocean Policy (2004) recommends that “Congress should amend the [Magnuson-Stevens Act] and related statutes to require regional fishery management councils and interstate fisheries commissions to rely on their scientific and statistical committees, incorporating the scientific and statistical committees’ findings and advice into the decision-making process.” There is great value in separating hard-won, consensus science from the unpredictable changes of day-to-day politics. The Secretary of Commerce should determine whether a plan adheres to National Standard 2 by the extent to which the guidelines have been followed as part of the review for compliance with all 10 national standards specified by the Magnuson-Stevens Act. The “best

OCR for page 51
Improving the use of the “Best Scientific Information Available” Standard in Fisheries Management scientific information available” standard should apply to all stages in the process of producing and using scientific information in fisheries management: data collection, data analysis, dissemination of data and information, and development of FMPs. Documentation of how scientific information is produced, validated, and applied to management decisions will improve accountability at all stages of the plan development process. A rigorous secretarial review of the use of scientific information will result in a feedback process that will improve the ability of fishery management councils to prepare FMPs that better meet National Standard 2 and minimize the need for intervention by the courts. The goal is to reduce the pressure on all parties to tailor the management plans to the interests of any one constituency, regardless of the scientific findings. Effective Communication The presentation of scientific information at regional fishery management council meetings should be concise and as free of scientific jargon as possible. Scientific information presented to fishery management councils often is not well understood by council members and stakeholders. In some regions, fishermen have a difficult time participating in reviews of stock assessments because they do not understand stock assessment methods. The scientific basis of management decisions may be challenged not only when the perceived quality of the scientific information is at issue, but also when the process of producing and validating the scientific information is neither transparent nor accessible to stakeholders. Data included or excluded from the analysis should be described, and a clear interpretation of the results should be given. Scientific reports should explicitly identify the level of uncertainty in results, provide explanations of the sources of uncertainty, and assess the relative risks associated with a range of management options. Decision making in fisheries requires an accurate and comprehensible assessment of uncertainty. Managers have to take into account both the short-term and the long-term effects of management actions. Scientists can help by estimating the risks to the fish population and to the fishery over different time periods and in relationship to the uncertainties involved. Descriptions of uncertainty can also provide an index of the quality of available information that should be used to help set research priorities. Fishery management council

OCR for page 51
Improving the use of the “Best Scientific Information Available” Standard in Fisheries Management members should receive an overview of scientific methodology that includes a discussion of the relationship between uncertainty and risk. Improving the Quality of Scientific Information Used in Fisheries Management NOAA Fisheries should develop and implement a plan to systematically improve the quality of the “best scientific information available” that includes regular assessments of the outcomes of management actions and evaluation of the predictive quality of the scientific information supporting those actions. NOAA Fisheries and the councils have attempted to improve the scientific information used in fishery management through a nationwide stock assessment improvement plan and an annual process of determining critical research priorities. Yet an additional process is necessary to improve accountability for use of the “best scientific information available” in the development of FMPs. Recently, the courts have acted as arbiters for determining whether National Standard 2 has been applied adequately in the preparation of contested FMPs. This default delegation of responsibility to the courts, however, results in additional costs, delays, and diversion of agency resources. Legislative history shows that one of Congress’ goals in establishing National Standard 2 “must be to achieve the best available scientific information” (Senate Committee, Fishery Conservation and Management Act, 94) for use in the preparation of FMPs. This suggests that the framers of the law intended NOAA Fisheries to acquire the scientific information necessary for effective fisheries management. Although use of the “best scientific information available” may involve applying incomplete information to determine management actions, it is not sufficient to rely on inadequate information over the longer term. NOAA Fisheries should formalize a system for establishing research priorities and funding levels across regions to gather the information needed to reduce uncertainty and improve understanding. After a management action has been passed by the council and approved by the Department of Commerce through NOAA Fisheries, follow-up evaluation of the effects of that management action is rarely undertaken. Yet evaluation of outcomes of management actions over time is necessary to ensure the continued use and refinement of scientific information. As the quality of scientific information improves, the basis for good management decision making will be stronger. Such an

OCR for page 51
Improving the use of the “Best Scientific Information Available” Standard in Fisheries Management evaluation process requires explicit hypotheses (statements about relationships) about potential actions and their related system components. Properly designed studies will provide new knowledge that tests these hypotheses and leads to a more refined understanding of the consequences of management actions. Clearly, there is broad agreement that the “best scientific information available” should form the basis of management decisions. Although there is room for improvement in what constitutes the best scientific information, the problem lies more in the way the scientific information is applied to policy. Some of the legal challenges of scientific information could be prevented by uniform application of operational guidelines that ensure uniform provision, use, and documentation of the “best scientific information available” in the development of FMPs. Indeed, a key goal of this report has been to develop guidelines to ensure that management actions are based on the “best scientific information available” as required under the Magnuson-Stevens Act, starting with the premise that the scientific information has undergone sufficient peer review to ensure that the methods of data collection, scientific analysis, and scientific conclusions are explicit, transparent (includes the rationale used for the inclusion or exclusion of specific data sets), and accepted as the best available. Legal challenges under National Standard 2 are not the only challenges to decision making for fisheries management. Other national standards also have been invoked in successful suits against the agency. Indeed, NOAA Fisheries has repeatedly and successfully been sued for ignoring the science in favor of more politically popular management decisions. This points to a problem with governance. Although it is not within the purview of this study to address governance, NOAA Fisheries is strongly urged to review recommendations made in previous National Research Council reports because governance speaks to the heart of the problem with the application of the “best scientific information available.”