Committee on Fish Stock Assessment Methods

Ocean Studies Board

Commission on Geosciences, Environment, and Resources

National Research Council

• NOTICE
• COMMITTEE
• PREFACE
• CONTENTS
• EXECUTIVE SUMMARY

NATIONAL ACADEMY PRESS
Washington, D.C.1998

NATIONAL ACADEMY PRESS • 2101 Constitution Avenue, N.W. • Washington, DC 20418

NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competencies and with regard for appropriate balance.

This report has been reviewed by a group other than the authors according to procedures approved by a Report Review Committee consisting of members of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine.

The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Bruce Alberts is president of the National Academy of Sciences.

The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility of advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. William A. Wulf is president of the National Academy of Engineering.

The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Kenneth I. Shine is president of the Institute of Medicine.

The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy's purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Bruce M. Alberts and Dr. William A. Wulf are chairman and vice chairman, respectively, of the National Research Council.

This report and the committee were supported by the National Oceanic and Atmospheric Administration. The views expressed herein are those of the authors and do not necessarily reflect the views of the sponsor.

This study was supported by Contract No. 50-DGNC-3-00016 between the National Academy of Sciences and the National Oceanic and Atmospheric Administration. Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the view of the organizations or agencies that provided support for the project.

Library of Congress Catalog Card Number 98-84433
International Standard Book Number 0-309-05725-6

Additional copies of this report are available from:

National Academy Press
2101 Constitution Ave., NW
Box 285
Washington, D.C. 20055
800-624-6242
202-334-3313 (in the Washington metropolitan area)
http://www.nap.edu

Copyright 1998 by the National Academy of Sciences. All rights reserved.

Printed in the United States of America.



---

COMMITTEE ON FISH STOCK ASSESSMENT METHODS

RICHARD DERISO (Cochairman), Inter-American Tropical Tuna Commission, La Jolla, California

TERRANCE QUINN (Cochairman), University of Alaska Fairbanks, Juneau

JEREMY COLLIE, University of Rhode Island, Narragansett

RAY HILBORN, University of Washington, Seattle

CYNTHIA JONES, Old Dominion University, Norfolk, Virginia

BRUCE LINDSAY, Pennsylvania State University, University Park

ANA PARMA, International Pacific Halibut Commission, Seattle, Washington

SAUL SAILA, University of Rhode Island, Narragansett

LYNDA SHAPIRO, University of Oregon, Charleston

STEPHEN JOSEPH SMITH, Bedford Institute of Oceanography, Dartmouth, Nova Scotia, Canada

CARL WALTERS, University of British Columbia, Vancouver, Canada




Staff

EDWARD R. URBAN, JR., Study Director

LORA TAYLOR, Senior Project Assistant





---

OCEAN STUDIES BOARD

KENNETH BRINK (Chairman), Woods Hole Oceanographic Institution, Massachusetts

ALICE ALLDREDGE, University of California, Santa Barbara

DAVID BRADLEY, Pennsylvania State University, State College

WILLIAM CURRY, Woods Hole Oceanographic Institution, Massachusetts

ELLEN DRUFFEL, University of California, Irvine

RANA FINE, University of Miami, Florida

CARL FRIEHE, University of California, Irvine

ROBERT GAGOSIAN, Woods Hole Oceanographic Institution, Massachusetts

SUSAN HANNA, Oregon State University, Corvallis

JOHN HOBBIE, Marine Biological Laboratory, Woods Hole, Massachusetts

EILEEN HOFMANN, Old Dominion University, Norfolk, Virginia

JOHN KNAUSS, University of Rhode Island, Narragansett

ROBERT KNOX, University of California, San Diego

RAY KRONE, University of California, Davis

LOUIS LANZEROTTI, Bell Laboratories, Lucent Technologies, Murray Hill, New Jersey

JOHN MAGNUSON, University of Wisconsin, Madison

WILLIAM MERRELL, The H. John Heinz III Center for Science, Economics and the Environment, Washington, D.C.

B. GREGORY MITCHELL, University of California, San Diego

NEIL OPDYKE, University of Florida, Gainesville

MICHAEL ORBACH, Duke University Marine Laboratory, Beaufort, North Carolina

TERRANCE QUINN, University of Alaska Fairbanks, Juneau

C. BARRY RALEIGH, University of Hawaii, Honolulu

JAMES RAY, Shell Oil Company, Houston, Texas

GEORGE SOMERO, Stanford University, Pacific Grove, California

PAUL STOFFA, University of Texas, Austin




Staff

MORGAN GOPNIK, Director

M. ELIZABETH CLARKE, Associate Director

EDWARD R. URBAN, JR., Program Officer

DAN WALKER, Program Officer

ROBIN MORRIS, Administrative Associate

GLENN MERRILL, Research Associate

LORA TAYLOR, Senior Project Assistant

JENNIFER SWERDA, Project Assistant

SHARI MAGUIRE, Project Assistant

ANN CARLISLE, Project Assistant




---

COMMISSION ON GEOSCIENCES,
ENVIRONMENT, AND RESOURCES

GEORGE M. HORNBERGER (Chairman), University of Virginia, Charlottesville

PATRICK R. ATKINS, Aluminum Company of America, Pittsburgh, Pennsylvania

JAMES P. BRUCE, Canadian Climate Program Board, Ottawa, Ontario

WILLIAM L. FISHER, University of Texas, Austin

JERRY F. FRANKLIN, University of Washington, Seattle

THOMAS E. GRAEDEL, Yale University, New Haven, Connecticut

DEBRA KNOPMAN, Progressive Foundation, Washington, D.C.

KAI N. LEE, Williams College, Williamstown, Massachusetts

PERRY L. MCCARTY, Stanford University, California

JUDITH E. MCDOWELL, Woods Hole Oceanographic Institution, Massachusetts

RICHARD A. MESERVE, Covington & Burling, Washington, D.C.

S. GEORGE PHILANDER, Princeton University, New Jersey

RAYMOND A. PRICE, Queen's University at Kingston, Ontario

THOMAS C. SCHELLING, University of Maryland, College Park

ELLEN SILBERGELD, University of Maryland Medical School, Baltimore

VICTORIA J. TSCHINKEL, Landers and Parsons, Tallahassee, Florida

E-AN ZEN, University of Maryland, College Park




Staff

MYRON UMAN, Acting Executive Director

GREGORY SYMMES, Assistant Executive Director

JEANETTE SPOON, Administrative Officer

SANDI FITZPATRICK, Administrative Associate

MARQUITA SMITH, Administrative Assistant/Technology Analyst



---

Preface






Global marine fish harvest has plateaued and many important commercial stocks have been depleted. The Ocean Studies Board (OSB), at the request of the National Marine Fisheries Service (NMFS), has provided advice designed to improve management of marine fisheries in the United States (NRC, 1994a). Many of the changes suggested in the 1994 report Improving the Management of U.S. Marine Fisheries were incorporated in the reauthorized Magnuson-Stevens Fishery Conservation and Management Act (MSFCMA) in 1996.

NMFS also has asked the OSB for advice on specific issues of Atlantic bluefin tuna population biology; the OSB presented its results in another 1994 report, An Assessment of Atlantic Bluefin Tuna (NRC, 1994b). Following the publication of that report, there was a widespread expression of the need for reviews of several other specific stock assessments. NMFS responded by requesting a broad review of the methods used in the United States for stock assessments. The results of that review are presented in this report.

This study would not have been possible without the efforts of NMFS scientists who carried out blind runs of data provided by the study committee and participated in the committee's meetings. The cooperation between academic and agency scientists was commendable and actually led to advances in the state of the art of fish stock assessments. The OSB offers sincere thanks to NMFS analysts for their considerable efforts, including Ray Conser, Jeff Fujioka, Wendy Gabriel, Phil Goodyear, Jim Ianelli, Rick Methot, Jerry Pella, Clay Porch, Joe Powers, Mike Prager, Victor Restrepo, Gerald Scott, and Mike Sigler. Other individuals also contributed to the committee's work and deserve the thanks of the committee and the OSB: Jie Zheng (Alaska Department of Fish and Game), Andre Punt (Commonwealth Scientific and Industrial Research Organization, Australia), and David Fournier (Otter Research Ltd.).

The results of this study will serve as an important foundation for the National Research Council review of the U.S. Northeast fisheries stock assessments that was mandated by Congress in 1996 as part of the MSFCMA reauthorization. The recommendations presented herein should also be useful to ongoing international activities related to fish stock assessments, such as those of the International Council for the Exploration of the Sea.



Kenneth Brink Chairman, Ocean Studies Board






---

Contents






EXECUTIVE SUMMARY

1 INTRODUCTION

Overview of U.S. Fisheries
Overview of the Use of Assessments in Management
Approach of the Committee


2 DATA

Basics of Sample Survey Design
Commercial Surveys
Recreational Surveys
Fishery-Independent Data


3 ASSESSMENT METHODS

Review of Existing Methods
Bayesian Approaches
Meta Analysis
Retrospective Analysis in Stock Assessments
Data Weighting
Uncertainty in Stock Assessment Methods and Models


4 HARVEST STRATEGIES

Indicators of Performance
Biological Reference Points
Closed-Loop Management Strategies
Recreational Management Strategies
Impact of Responsible Fishing on Management Strategies and Closed-Loop Performance
Evaluation of Management Procedures and Decision Tables


5 SIMULATIONS

Simulation Approach
Simulation Results
Implications of Model Results


6 FINDINGS AND RECOMMENDATIONS

How Should Stock Assessments Be Conducted and by Whom?
Data Collection and Assessment Methods
Harvest Strategies
Rigorous Evaluation System
New Approaches
Peer Review of Assessments and Assessment Methods
Education and Training


REFERENCES

APPENDIXES
A. Letter of Request
B. Committee Biographies
C. Acronyms and Symbols
D. Checklist for Stock Assessments
E. Description of Age-Structured Simulation Model
F. Letter to Analysts
G. Retrospective Values
H. Information from Interstate Marine Fisheries Commissions and States
I. Model Results in Terms of Exploitable and Total Biomass
J. Excerpts from the Food and Agricultural Organization Report Precautionary Approach to Fisheries
K. Uncertainty in Stock Assessment Methods and Models



---

Executive Summary






Marine fisheries provide a vital contribution to food supplies, employment, and culture worldwide. Therefore, matching fishing activities with natural fluctuations so as to avoid unsustainable harvests and population crashes is an important goal. In an ideal world, accurate and precise estimates of the abundance of fish stocks and their dynamics (how and why population levels change) would be available to set sustainable harvest levels to accommodate commercial and recreational demand. In reality, fishery management is based on imperfect estimation of the number, biomass, productivity, and age structure of fish populations and incomplete knowledge of population dynamics. The ocean is relatively opaque to light, and acoustic techniques of remote sensing are not yet sufficiently developed for general use in estimating fish populations. Thus, it is difficult to count fish through nondestructive means and fish usually must be caught to be counted, weighed, and measured. Standardized techniques have been developed to sample a relatively small proportion of fish from a population and to combine such data with commercial and recreational catch information to estimate population characteristics. These techniques yield stock assessments used by managers at state, regional, national, and international levels.

In addition to monitoring the abundance and productivity of exploited fish populations, stock assessments can provide a quantitative prediction of the consequences of possible alternative management actions. The mechanisms that cause fish populations to change are poorly understood but include environmental and ecosystem effects, interactions among multiple species, and effects of humans through harvesting, pollution, habitat disruption, and other factors. Without accurate stock assessments and their proper use in management, exploited fish populations can collapse, creating severe economic, social, and ecological problems. Therefore, ensuring that stock assessment research progresses and that operational stock assessments use the best techniques for a given stock are fundamental for ensuring the sustainability of commercial and recreational marine fisheries.

Stock assessment is a multistage process. Steps include (1) definition of the geographic and biological extent of the stock, (2) choice of data collection procedures and collection of data, (3) choice of an assessment model and its parameters and conduct of assessments, (4) specification of performance indicators and evaluation of alternative actions, and (5) presentation of results. This report concentrates on evaluating assessment models, with less extensive treatment of the other steps. Chapter 1 discusses these steps in greater detail. Techniques of stock assessment range from informal estimates to more sophisticated modeling approaches used to combine data of various types. Assessment models predict rates of change in biomass and productivity based on information about yield from fisheries and the rates at which fish enter the harvestable population (recruitment), grow in size, and exit the population (natural and fishing mortality).

Stock assessments for fish living in the U.S. exclusive economic zone (3 to 200 nautical miles from shore) and for some highly migratory species are conducted by scientists from the National Oceanic and Atmospheric Administration's (NOAA's) National Marine Fisheries Service (NMFS) and independent species group commissions (e.g., the International Pacific Halibut Commission and the Inter-American Tropical Tuna Commission). In addition, interstate fishery management commissions were created to facilitate the coordination of state assessment scientists in working with each other and with federal scientists to assess and manage stocks shared among states in their coastal waters (within 3 nautical miles from shore on open coasts, as well as bays and estuaries). These organizations include the Atlantic States, Gulf States, and Pacific States Marine Fisheries Commissions. Some states (e.g., Alaska, Oregon, and Florida) also perform assessments for fisheries conducted in their own state waters.

Fishery management organizations use the results of stock assessments to design and implement various controls for the total catch that can be removed from fish populations under their jurisdictions. Commercial catch can be managed by specifying the amount of harvesting allowed; the areas of fishing and times of the year that fishing can take place; the gear that can be used; minimum fish size limits; and in some cases, the amount of fish that any single fisher, community, company, or other entity can catch. Recreational fisheries more often impose minimum size limits, daily catch limits, seasons, and sometimes gear restrictions and requirements to release fish that are caught.




STUDY PROCESS

The National Research Council (NRC) Committee on Fish Stock Assessment Methods was formed in early 1996 to review existing stock assessment methods and to consider alternative approaches for the future. The committee's statement of task was two-fold:


1. Conduct a scientific review of stock assessment methods and models for marine fisheries management.
2. Compare models using actual and simulated data having a variety of characteristics, to test the sensitivity and robustness of the models to data quality and type.

As part of this study, the committee asked selected stock assessment scientists to conduct blind runs of simulated data sets using five different models. Models tested included a production model, a delay-difference model, and three age-structured methods (described in detail in Chapter 3). The goal of the simulation study was to evaluate the performance of stock assessment methods for simulated fish populations for which the true population parameters were known (to the committee, but not to the analysts) and some of the assumptions usually made in stock assessments were violated. One type of data set was typical of the catch biomass, age composition of the catch, and catch per unit effort (CPUE) that are obtained from commercial and recreational fisheries. The other type of data set was typical of that collected by fishery-independent surveys.

Each analyst was asked to evaluate five 30-year sets of simulated commercial and survey data, alone and in combination. The five data sets provided different combinations of parameters in terms of the following:

• Increasing or decreasing stock size over time (population trend)

• Constant versus changing age of fish caught (fishery selectivity) over time

• Accuracy of catch reported by fishers

• Ability of fishery and survey vessels to catch fish (fishery and survey catchability)

The analysts were given essential information about fish growth and maturity, the probability of mis-estimating fish ages, and selected information about the structure of the populations and the data. Analysts were not provided information about natural mortality, catchability, selectivity, recruitment, or the amount of underreporting (although they were warned that underreporting might have occurred).

In addition to the results of these basic analyses, (1) some analysts repeated their model runs with the true average natural mortality (provided by the committee), (2) key management variables were calculated by analysts and the committee, and (3) retrospective analyses were conducted by the committee to determine the persistence of over- or underestimation of population parameters over time by the different models. Greater detail about the study process is given in Chapter 5 and Appendix E.




FINDINGS AND RECOMMENDATIONS

The committee focused its examination on the data that are used in assessments, model performance, use of harvest strategies, new assessment techniques, periodic review and quality control of assessments and assessment methods, and education and training of stock assessment scientists. The committee based its recommendations on the results of the simulations and on its collective experience. Caveats about how the analyses conducted for this study compare to actual stock assessments are given in Chapter 5. Accomplishing the recommendations of this report will require concerted and cooperative action by all interested parties (academic and government scientists, fishery managers, user groups, and environmental nongovernmental organizations) to improve the stock assessment process and products.




Data Collection And Assessment Methods

The committee concludes that stock assessments do not always provide enough information to evaluate data quality and to estimate model parameters, and it recommends a checklist that would promote more complete data collection for use in stock assessments. The results of the committee's simulations demonstrated that the availability of continuous sets of data collected by using standardized and calibrated methods is important for the use of existing stock assessment models. The best index of fish abundance is one for which extraneous influences (e.g., changes in gear and seasonal coverage, changes in fishers' behavior) can be controlled. The committee recommends that at least one reliable abundance index should be available for each significant stock. CPUE data from commercial fisheries, if not properly standardized, do not usually provide the most appropriate index. Likewise, CPUE data from recreational fisheries require standardization to serve as a good index of abundance.

Fishery-independent surveys offer the best opportunity for controlling sampling conditions over time and the best choice for achieving a reliable index if they are designed well with respect to location, timing, sampling gear, and other considerations of statistically valid survey design. NMFS should support the long-term collection of fishery-independent data, using either the NOAA fleet or calibrated independent vessels. Diminishing the quality of fishery-independent data by failing to modernize NOAA fishery research vessels or by changing sampling methods and gear without proper calibration could reduce the usefulness of existing and future data sets.

The simulation study demonstrated that assessments are sensitive to underlying structural features of fish stocks and fishery practices, such as natural mortality, age selectivity, catch reporting, and variations in these or other quantities. Auxiliary information in the form of indices or survey estimates of abundance, population structure information, and accurate estimates of other population parameters (e.g., natural or fishing mortality, growth, catchability) improves the accuracy of assessments.

Formally reviewed sampling protocols for collection of commercial fisheries statistics have not been implemented in many geographic regions. The lack of formalized, peer-reviewed data collection methods in commercial fisheries is problematic because bias and improper survey conduct may exist, with unknown impact on data reliability. Greater attention should be devoted to sampling design based on an understanding of the statistical properties of the estimators for catch at age and other factors. Sampling and subsequent analysis should also consider the issue of systematic biases that emerge with factors such as misreporting. Formalized sampling protocols have been developed for recreational fisheries in the form of the Marine Recreational Fisheries Statistics Survey (MRFSS). MRFSS data and methods, albeit imperfect, have undergone independent peer review, are readily available, and could serve as a model for commercial fisheries. The committee recommends that a standardized and formalized data collection protocol be established for commercial fisheries nationwide.




Models

Both harvesting strategies and decision rules for regulatory actions have to be evaluated simultaneously to determine their combined ability to sustain stocks. Simulation models should be realistic and encompass a wide range of possible stock responses to management actions and natural fluctuations consistent with experience. The committee recommends that fish stock assessments present realistic measures of the uncertainty in model outputs whenever feasible. Although a simple model can be a useful management tool, more complex models are needed to better quantify the unknown aspects of the system and to address the long-term consequences of specific decision rules adequately. Retrospective analyses performed by the committee showed that persistent over- or underestimation can occur over a number of years of assessment, regardless of which model is used. The committee recommends the use of Bayesian methods both for creating distributions of input variables and for evaluating alternative management policies. Other methods for including realistic levels of uncertainty in models also should be investigated.

In the simulations, model performance became erratic as more variability or errors were introduced to data sets. Newer modeling methods offer promise for reducing bias in key parameter estimates, although using mathematically sophisticated assessment models did not mitigate poor data quality. Different assessment models should be used to analyze the same data to help recognize poor data and to improve the quality of assessment results. Results from such comparisons can be used to direct survey programs to improve data quality and to assess the degree of improvement in data achieved over time. Greater attention should also be devoted to including independent estimates of natural mortality and its variability in assessment models. Further simulation work of this kind is also needed to determine whether the simulation results and the conclusions based on these results remain the same over multiple replications.

The committee believes that single-species assessments provide the best approach at present for assessing population parameters and providing short-term forecasting and management advice. Recent interest in bringing ecological and environmental considerations and multi-species interactions into stock assessments should be encouraged, but not at the expense of a reduction in the quality of stock assessments.




Harvest Strategies

Although the committee did not evaluate alternative harvest strategies, it believes that assessment methods and harvest strategies should be evaluated together because harvest strategies can affect stock assessments and the uncertainty inherent in stock assessments should be reflected in harvest strategies. Despite the uncertainty in stock assessments, fishery scientists may be able to identify robust management measures that can at least prevent overfishing, even if they cannot optimize performance. Conservative management procedures include management tools specific to the species managed, such as minimum biomass levels, size limits, gear restrictions, and area closures (for sedentary species). Management procedures by which the allowable catch is set as a constant fraction of biomass (used for many U.S. fisheries) generally perform better than many alternative procedures. However, errors in implementation due to assessment uncertainties could result in substantial reductions in long-term average harvests in some years if biomass estimates are highly uncertain. Assessment methods and harvest strategies need to be evaluated simultaneously to determine their ability to achieve management goals. Application of risk-adjusted reference points (based on fishing mortality or biomass) would immediately lead to reduced total allowable catch and thus create an economic incentive for investment in improved data gathering and assessment procedures to reduce the coefficient of variation of biomass estimates.

There are at least four alternatives to harvesting a constant fraction of exploitable biomass that may result in levels of total mortality that are consistent with maintaining a fish stock. First, target fishing mortality can be reduced as a stock decreases in size to reduce risks. Second, a minimum biomass level can be established, below which fishing would be halted (this is done for some U.S. fisheries). Third, the size of fish captured can be increased by changing requirements for harvest gear. This restriction might allow smaller fish to escape and spawn, but could be ineffective if harvesters apply more effort to the larger fish. Finally, geographic areas can be closed to limit mortality for sedentary species if the distribution of organisms is well known and if the fishing mortality in other areas is not increased. Area closures have been implemented or proposed for many fisheries worldwide in the form of marine reserves and sanctuaries.




New Approaches

NMFS and other organizations responsible for fisheries management should support the development of new techniques that can better accommodate incomplete and variable data and can account for the effects of environmental fluctuations on fisheries. Such techniques should allow the specification of uncertainty in key parameters (rather than assuming constant, known values), should be robust to measurement error, and should include the ability to show the risks associated with estimated uncertainty.

A few prominent recommendations for new approaches emerged from the study. Scientists that conduct stock assessments and organizations that depend on assessments should

• incorporate Bayesian methods and other techniques to include realistic uncertainty in stock assessment models;

• develop better assessment models for recreational fisheries and methods to evaluate the impacts of the quality of recreational data on stock assessments;

• account for effects of directional changes in environmental variables (e.g., those that would accompany climate change) in new models; and

• develop new means to estimate changes in average catchability, selectivity, and mortality over time, rather than assuming that these parameters remain constant.

The results from the simulation exercise should be sobering to scientists, managers, and the users of fishery resources. The majority of the estimates of exploitable biomass exceeded true values by more than 25%; assessments that used accurate abundance indices performed roughly twice as well as those that use faulty indices. A disturbing feature of the assessment methods is their tendency to lag in their detection of trends in the simulated population abundance over time. For example, some methods with some types of data consistently overestimate exploitable biomass during periods of decreasing simulated abundance and underestimate exploitable biomass during periods of increasing simulated abundance.

Although no stock assessment model was free from significant error in the simulations, it is also true that few of the models failed consistently. Hence, the message of this report is not that stock assessment models should not be used, but rather that data collection, stock assessment techniques, and management procedures need to be improved in terms of their ability to detect and respond to population declines. The simulation results and some actual fishery management examples suggest that overestimation of stock biomass and overfishing of a population can occur due to inaccurate stock assessments and that the overestimation can persist over time. The committee believes that the two most important management actions to mitigate this problem are: (1) to model and express uncertainty in stock assessments explicitly, and (2) to incorporate uncertainty explicitly into management actions such as harvesting strategies.

The absence of adequate data is the primary factor constraining accurate stock assessments. The differences between estimated and true values derived from the simulated data were most likely not introduced by any mistakes made by the analysts. Rather, the large differences that occurred under some scenarios were primarily the result of poor data and model misspecification stemming from incomplete knowledge of the true situation by the analysts. The surplus production and delay difference models did not include the ability to account for changes over time in key parameters for the simulated populations. The simulated data sets were better structured for analysis by age-structured methods; hence, these kinds of models performed better. When they did not perform well, it was generally because the models used biased information (e.g., the fishery CPUE index) or did not account for changes in selectivity and catchability over time. Had the analysts been told about these data features, it is likely that they could have compensated for them and obtained better assessments. Some of the newer models appear to be able to achieve such compensation through the introduction of process errors. Nevertheless, modeling will never be able to provide estimates that are as accurate as direct knowledge obtained by measurement and experimentation. Thus, if future stock assessments are to avoid some of the past problems, management agencies must devote the necessary resources to monitor and investigate fish populations in a stable research environment that fosters creative approaches.




Peer Review

It is imperative that stock assessment procedures and results be understood better and trusted more by all stakeholders. One means to achieve such trust is to conduct independent peer review of fishery management methods and results including (1) the survey sampling methods used in data collection, (2) stock assessment procedures, and (3) risk assessment and management strategies. When applied properly to stock assessments, peer review yields an impartial evaluation of the quality of assessments as well as constructive suggestions for improvement. Such reviews are most beneficial when conducted periodically, for example, every 5 to 10 years, as new information and practices develop. In addition, a complete review of methods for collection of data from commercial fisheries should be conducted in the near future by an independent panel of experts, which could lead to the adoption of formal protocols.




Education and Training

Reduction in the supply of stock assessment scientists would endanger the conduct of fishery assessments by the federal government, interstate commissions, and international management organizations and would hinder progress in the development and implementation of new stock assessment methods. NMFS and other bodies that conduct and depend on fish stock assessments should cooperate to ensure a steady supply of well-trained stock assessment scientists by using mechanisms such as personnel exchanges among universities, government laboratories, and industry and by funding stock assessment research activities. The training of stock assessment scientists should endow them with skills in applied mathematics, fisheries biology, and oceanography. Education of fisheries scientists should be organized and executed in such a way that it complements and augments the NMFS research mission and leads to improved management strategies for fisheries in the future.