Improving the Use of the "Best Scientific Information Available" Standard in Fisheries Management (2004)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

3 Characterizing “Best Scientific Information Available” PROBLEMS OF INTERPRETATION The term “best scientific information available” as used in National Standard 2 of the Magnuson-Stevens Act has been interpreted in a number of ways, depending on whether the emphasis is on the quality (best) or timeliness (available) of the scientific information. This chapter reviews the reasoning, results, and conclusions of previous activities and publications addressing the issue of how to incorporate the “best scientific information available” into policy. In so doing, common themes and areas of disagreement have been examined to help develop the types of criteria that could be used to evaluate the scientific information provided to decision makers. At its core, National Standard 2 affirms the role of scientific information in fisheries management. The basis for distinguishing scientific from other types of information has provided fertile ground for philosophical debate. It would be impractical to attempt to summarize and expand on this debate in a report of this limited scope and length. However, there are general attributes of scientific information that apply to the use of “best scientific information available” in fisheries management. The word science derives from the Latin word scire which means “to know.” Scientific information has been described as knowledge that “emerges from a process of observation, identification, description, and testing of explanatory hypotheses about fundamental principles that govern cause-and-effect relationships” (Bisbal, 2002). The hallmark of the scientific approach is the generation of hypotheses based on observations and the testing of these hypotheses through the meth- odological collection and analysis of data. It is important to note in the 41 

42 “BEST SCIENTIFIC INFORMATION AVAILABLE” STANDARD context of this study that the U.S. Supreme Court upheld this view of science in the Daubert decision (Box 2.4). The nature of scientific inquiry varies with the field of endeavor and the constraints imposed by the system under examination. For example, tightly controlled and replicated laboratory experiments form the basis of most research on the physiology and genetics of organisms. In contrast, this type of experimental approach often may be impractical for investigating the population biology, ecological relationships, and evolution of these same organisms because of the temporal and spatial scales and complexity of the systems. In these fields, much scientific information is derived from careful observation and analysis of empirical data. For each approach to scientific inquiry, potential sources of error or bias in either data collection or analysis may be identified through peer review. Ultimately, the quality of a given scientific analysis is revealed over time as additional information becomes available that either confirms or refutes earlier interpretations. In practice, most disputes over scientific information used in management have arisen from the addition of the modifiers “best” and “available.” The term best explicitly suggests that there is no better scientific information available and implicitly suggests the use of the most relevant and contemporary data and methods. Practically, best information will vary depending on the circumstances. The best information in a region where there is little scientific capacity or for a species about which little is known will differ in quality from the best information in a region with careful stock surveys and long-term monitoring by scientific experts. The term available suggests that management action is not contingent on the acquisition of new information. Hence, “best scientific information available” acknowledges the existence of scientific uncertainty, a feature of even the most robust biological population assessments, and dictates that prudent management be consistent with the scientific information that is available even though data gaps exist. SCIENCE AND ITS ROLE IN THE NATIONAL MARINE FISHERIES SERVICE A previous National Research Council report Science and Its Role in the National Marine Fisheries Service (National Research Council, 2002a) recognized that the governance structure has remained virtually 

CHARACTERIZING “BEST SCIENTIFIC INFORMATION AVAILABLE” 43 unchanged since 1976, even though the laws governing the management of marine fisheries have been amended many times. That report recommended explicitly that “Congress should initiate a review of the fisheries governance system and the use of science in governance” (National Research Council, 2002a). Although recommendations on structural changes in governance are beyond the statement of task for this study, improving adherence to National Standard 2 cannot be considered fully without an awareness of the governance structure. Therefore, a review of some relevant findings from that earlier report will help put the issues of this effort in context. The most relevant finding of the earlier report is that “the use of science in the marine fisheries management decision-making process is impeded by the governance system created by the [Magnuson-Stevens Act] and the resulting mismatch between institutional authorities and responsibilities” (National Research Council, 2002a). It was recognized that breakdowns in the application of National Standard 2 can occur in the process of fishery management plan (FMP) development (National Research Council, 2002a): Regional [fishery management councils] sometimes disregard the scientific advice provided by [the National Oceanic and Atmospheric Administration (NOAA) Fisheries] and their science and statistical committees in setting total allowable catches (TACs) and in deciding other aspects of FMPs. [NOAA Fisheries] has the legal right to approve, disapprove, or partially approve FMPs; but when councils have disregarded the scientific findings of [NOAA Fisheries] and the advice of their science and statistical committees, [NOAA Fisheries] has sometimes sought compromises with the councils rather than upholding their original findings. The entire process is subject to intense political pressure, directly from stakeholders and indirectly through their representatives in Congress. A recommendation to require explanatory findings from the [fishery management councils] on their treatment of scientific information will help improve outside perceptions of how the agency conducts its scientific work. 

44 “BEST SCIENTIFIC INFORMATION AVAILABLE” STANDARD CONSORTIUM FOR OCEANOGRAPHIC RESEARCH AND EDUCATION PANEL Congressman Don Young (R-Alaska) asked the Consortium for Oceanographic Research and Education (CORE) to convene a panel of experts to provide advice on the clarification of “best scientific information available” as specified in National Standard 2 and the application of this standard to fisheries management. CORE held a workshop in Washington, D.C., on February 18, 2000, to address this question. The CORE panel concluded that it was not possible to elaborate a specific definition or standard for identifying scientific information (Appendix G). Instead, the panel recommended the development of procedural standards or principles and identified five criteria for establishing procedural standards (Box 3.1) The CORE panel stated that a statutory definition was inadvisable because it would be written into law and therefore could not be adapted easily to different circumstances. Because science is continually evolving, any set of specific requirements may not be appropriate in all situations. A statutory requirement might specify exactly how and when the peer review must be conducted, whereas a variety of peer review mechanisms could produce a reasonable, perfectly acceptable result. For example, regulatory amendments or framework adjustments that simply represent part of an already-defined and approved rebuilding plan (e.g., BOX 3.1 CORE Report Procedural Criteria • Relevance. Scientific information must be applicable to the manage- ment issue. • Independent Scientific Review. Information must receive a positive review by independent experts. • Timeliness. Information must be provided in advance of manage- ment decisions to allow meaningful review by interested parties. • Reevaluation. Performance of management measures and under- lying science should be reexamined and updated as new information and methodologies become available. • Transparency and Inclusiveness. The process for developing scientific information should be open and broad-based; it should in- clude all relevant information and a clear rationale in cases where information is excluded. 

CHARACTERIZING “BEST SCIENTIFIC INFORMATION AVAILABLE” 45 increasingly stringent quotas, increased size limits, decreased bag limits) might require only an internal review before being sent forward. Similarly, with procedural guidelines there may be valid reasons for not following the guidelines precisely. It would be up to the Secretary of Commerce (or NOAA Fisheries), as part of FMP approval process, to determine whether the spirit of the guidelines has been followed. Under statutory guidelines, the process could not be completed until all guidelines were met fully. This could easily delay action necessary to protect living marine resources. NOAA FISHERIES ORGANIZED DECISION PROCESS NOAA Fisheries has published regulations for evaluating scientific information pursuant to its authority to define “dolphin-safe tuna” under the Marine Mammal Protection Act. NOAA Fisheries proposed an organized decision process to outline the types of information to be used by the agency to determine “whether the intentional deployment on or encirclement of dolphins with purse seine nets is having a ‘significant adverse impact’ on any depleted dolphin stock in the eastern tropical Pacific” (National Marine Fisheries Service, 2003). In the Federal Register notice, NOAA Fisheries provides guidelines for weighting information based on relevance, timeliness, independent peer review, and availability to NOAA Fisheries for verification. Scientific information is defined as “the results of properly designed scientific research” (National Marine Fisheries Service, 2003). Relevance and timeliness means that the information “is pertinent” to its use and is the “least degraded by the passage of time” (National Marine Fisheries Service, 2003). Independent peer review applies to information that …has been published in a refereed scientific journal in its field or independently read and criticized in writing by at least two peers; the criticism was disposed of either by acceptance or rebuttal, as appropriate, by the author(s); and the disposition of the criticism by the author(s) was independently determined to be appropriate and adequate. (National Marine Fisheries Service, 2003) 

46 “BEST SCIENTIFIC INFORMATION AVAILABLE” STANDARD PEW FELLOWS PROGRAM IN MARINE CONSERVATION WORKSHOP The “best scientific information available” was also the subject of a workshop sponsored by the Pew Fellows Program in Marine Conservation in 2001—The Best Available Science: A Workshop on the Role of Science in Marine Conservation Law. The purpose of the workshop was to consider ways in which policy makers identify and apply scientific information in conservation decisions. Participants examined recent litigation and related agency actions regarding the application of the Endangered Species Act and the National Environmental Policy Act to marine fisheries that are carried out in or near endangered wildlife habitat. The workshop highlighted many of the difficulties in interpreting “best scientific information available.” First, the goals of legislation (e.g., the Endangered Species Act, the Magnuson-Stevens Act, the Marine Mammal Protection Act) may be in conflict in some circumstances; hence, resolution will not depend on the quality of the scientific information but on which goal is considered the highest priority (e.g., to prevent overfishing or to minimize economic impacts). Second, different interpretations of the burden of proof in decision making among key players (the judiciary, scientists, politicians, industry, environmental groups, and the public) underlie many of the controversies in conservation and management issues. Should the resource or the resource user be subject to the greatest amount of risk when the scientific information is not definitive? Several themes raised in the Pew workshop were echoed in the committee’s 2003 workshop, such as (1) when decisions are contentious, scientific information is likely to be subject to more scrutiny and the quality of the scientific information may be challenged; (2) incomplete information should not delay management action, but agencies should have incentives for information gathering to support future decision making; and (3) the scientific advisory process should be separate from management to ensure objectivity. CANADIAN DEPARTMENT OF FISHERIES AND OCEANS Canada has been developing guidelines for the provision of scientific advice on decision making and policy setting. The guidelines and overarching principles warrant direct mention (Box 3.2). In particular, 

CHARACTERIZING “BEST SCIENTIFIC INFORMATION AVAILABLE” 47 the guidelines on “sound science and advice” are an attempt to produce explicit, operational guidelines for some of the same challenges that motivated this study (Rice, 2003). In 2001, the Canadian government adopted A Framework for Science and Technology Advice: Principles and Guidelines for the Effective Use of Science and Advice in Government Decision Making (Industry Canada, 2001) that embodies the six principles listed in Box 3.2. The Canadian Department of Fisheries and Oceans (Industry Canada, 2001) has identified six goals that emerge from trying to synthesize the principles described above. These goals address the peer review and advisory process and the specific challenges of decision making in marine and fisheries science. They are similar to the criteria for best scientific information identified in the CORE report and the NOAA Fisheries guidelines for the Marine Mammal Protection Act. The Canadian Department of Fisheries and Oceans goals are relevance, quality, inclusiveness, consistency, timeliness, and cost-effectiveness. BOX 3.2 Canada’s Framework for Science and Technology Advice—Six Principles • Early Issue Identification. The government has to anticipate, as early as possible, those issues for which science advice will be required, in order to facilitate timely and informed decision making. • Inclusiveness. Advice should be drawn from a variety of scientific sources and from experts in relevant disciplines, in order to capture the full diversity of scientific schools of thought and opinion. • Sound Science and Sound Advice. The government should employ measures to ensure the quality, integrity, and objectivity of the science and science advice it uses and to ensure that the science advice is used in decision making. • Uncertainty and Risk. Science in public policy always contains uncertainty that must be assessed, communicated, and managed. Government should develop a risk management framework that includes guidance on when and how precautionary approaches should be applied. • Transparency and Openness. The government is expected to employ decision-making processes that are open, as well as transparent, to stakeholders and the public. • Review. Subsequent review of science-based decisions is required to determine whether recent advances in scientific knowledge have an impact on the science advice used to reach the decision. 

48 “BEST SCIENTIFIC INFORMATION AVAILABLE” STANDARD The principles and goals adopted by Canada have many parallels with U.S. efforts to set standards for the scientific information used in conservation and management. For example, early issue identification necessitates improving communication among decision makers and the scientific community to ensure that the agency collects the appropriate information to address emerging issues. Although this principle is not discussed explicitly with regard to “best scientific information available” as used in the Magnuson-Stevens Act, it does embody the goal of timeliness. In the Canadian Department of Fisheries and Oceans draft guidelines, inclusiveness covers the consideration of disparate scientific opinion and scientific quality and involves experts from other, not necessarily scientific, disciplines. Sound science and sound advice embodies the application of rigor within the scientific process, from hypothesis formulation and data collection to internal and external review. Uncertainty and risk addresses the need for guidance on how to manage risk, especially when there is substantial scientific uncertainty, to determine when the precautionary approach should be applied. Scientists have the responsibility to explicitly identify uncertainty in scientific results and to communicate that uncertainty to decision makers. Transparency and openness raises issues similar to those identified in the CORE report. The development and application of scientific information should be conducted in a process that is open and accessible to stakeholders and other interested members of the public, including careful documentation of the decision-making process. Review is similar to the CORE report’s reevaluation criterion; this principle ensures that the most current scientific methods and concepts are employed through the establishment of follow-up procedures. Criticism of data, data analysis, or interpretation in a peer review context permits in-course corrections, the opportunity to revise analyses, and thereby the opportunity to respond to or rebut comments. This self-correcting mechanism is what “best scientific information available” is intended to build into the process. 

CHARACTERIZING “BEST SCIENTIFIC INFORMATION AVAILABLE” 49 STATUTORY-BASED CRITERIA FOR “BEST SCIENTIFIC INFORMATION AVAILABLE” A recent bill for reauthorization of the Magnuson-Stevens Act included a proposal to establish a statutory definition of “best scientific information available” (Fisheries Science and Management Improvement Act of 2003, 108th Cong., S.R. 482). The bill proposed by Senator Susan Collins (R-Maine) provides the following definition (Fisheries Science and Management Improvement Act of 2003, 108th Cong., S.R. 482): The term “best scientific information available”, with respect to fishery conservation and management and stocks of fish, means information that— (A) is directly related to the specific issue under consideration; (B) is based on a statistically valid sample such that any conclusions drawn are reasonably supported and not speculative; (C) has been independently peer-reviewed; (D) has been collected within a period that is reasonably related to the specific issue under consideration; (E) is consistent with information that is available from other reliable sources; and (F) may include, but not consist solely of, anecdotal information collected from the harvesting and processing of fish. (Fisheries Science and Management Improvement Act of 2003, 108th Cong., S.R. 482) There is a basis for concern that the term “best scientific information available” is too vague to provide sufficient guidance to NOAA Fisheries and the regional fishery management councils on the preparation and application of scientific information. The current disputes that end in litigation arise primarily over contesting the interpretation of the relative terms “best” and “available.” The proposed definition introduces terms that could be considered just as relative and subject to interpretation (e.g., “directly related,” “reasonably supported and not speculative,” “independently peer-reviewed,” “reasonably related,” other “reliable sources”) (Fisheries Science and Management Improvement Act of 2003, 108th Cong., S.R. 482). Adoption of the definition, therefore, would increase the number of relative terms and could result in more litigation, not less.