The committee developed the following findings and recommendations based on the information presented in Chapters 1 through 4 and the results of the simulation studies described in Chapter 5. The committee makes specific recommendations about how stock assessments are conducted; data collection and assessment methods; harvest strategies; a rigorous evaluation system; continued development of new assessment techniques; periodic peer review of assessments and assessment methods; and education and training of stock assessment scientists.
Finding: Stock assessments do not always provide enough information to evaluate data quality and estimate model parameters.
The committee concluded that stock assessments are sometimes incomplete and that a checklist for stock assessments would be helpful to fishery managers. The committee's checklist is shown in Appendix D. It includes five parts: (1) stock definition, (2) choice of data collection procedures and actual data collection, (3) choice of an assessment model and its parameters, (4) evaluation of alternative hypotheses and possible actions and specification of performance indicators, and (5) presentation of results.
The committee believes it is important for assessments to continue to be conducted by individuals most familiar with the biology of managed species and the associated fishery (i.e., scientists of the federal government and interstate and international fishery management bodies). The National Marine Fisheries Service (NMFS) should be responsible for supporting the long-term collection of fishery-independent data, using either the National Oceanic and Atmospheric Administration (NOAA) fleet or calibrated independent vessels. Diminishing the quality of fishery-independent data by failing to modernize NOAA fishery research vessels or changing sampling methods and gear without proper calibration could imperil existing and future data sets (ASMFC, 1997). The committee did not evaluate the proficiency with which fishery managers communicate stock assessment results and methods to the public. Greater efforts to clarify the workings of stock assessment could be important to increase the credibility of those assessments.
Recommendation: Stock assessment scientists should conduct complete assessments using a checklist such as
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 111
--> 6— Findings and Recommendations The committee developed the following findings and recommendations based on the information presented in Chapters 1 through 4 and the results of the simulation studies described in Chapter 5. The committee makes specific recommendations about how stock assessments are conducted; data collection and assessment methods; harvest strategies; a rigorous evaluation system; continued development of new assessment techniques; periodic peer review of assessments and assessment methods; and education and training of stock assessment scientists. How Should Stock Assessments Be Conducted and by Whom? Finding: Stock assessments do not always provide enough information to evaluate data quality and estimate model parameters. The committee concluded that stock assessments are sometimes incomplete and that a checklist for stock assessments would be helpful to fishery managers. The committee's checklist is shown in Appendix D. It includes five parts: (1) stock definition, (2) choice of data collection procedures and actual data collection, (3) choice of an assessment model and its parameters, (4) evaluation of alternative hypotheses and possible actions and specification of performance indicators, and (5) presentation of results. The committee believes it is important for assessments to continue to be conducted by individuals most familiar with the biology of managed species and the associated fishery (i.e., scientists of the federal government and interstate and international fishery management bodies). The National Marine Fisheries Service (NMFS) should be responsible for supporting the long-term collection of fishery-independent data, using either the National Oceanic and Atmospheric Administration (NOAA) fleet or calibrated independent vessels. Diminishing the quality of fishery-independent data by failing to modernize NOAA fishery research vessels or changing sampling methods and gear without proper calibration could imperil existing and future data sets (ASMFC, 1997). The committee did not evaluate the proficiency with which fishery managers communicate stock assessment results and methods to the public. Greater efforts to clarify the workings of stock assessment could be important to increase the credibility of those assessments. Recommendation: Stock assessment scientists should conduct complete assessments using a checklist such as
OCR for page 111
--> given in Appendix D. Scientists from state and federal governments and from the independent fisheries commissions should continue to conduct fish stock assessments, with periodic peer review, as described below. Data Collection and Assessment Methods Abundance Indices Finding: Having an index that is proportional to abundance (called survey index in the committee's simulation) resulted in reasonably good fits of the models. Abundance indices subject to biases or other perturbations (called fishery-dependent in the simulation) can result in poor performance of the models. Combinations of indices, if one is poor, does not improve performance of stock assessment models. The best index of fish abundance is one for which extraneous influences (e.g., changes in gear, learning, changes in seasonal coverage) can be controlled. Catch per unit effort (CPUE) can vary over time in commercial and recreational fisheries, is subject to fishers' optimizing behaviors, and is not usually the most appropriate index (Chapters 2 and 3). There are examples, however, in which CPUE is an appropriate index if it is interpreted correctly (Quinn and Collie, 1990; NRC, 1994b; see model SS-P6 in Chapter 5). At present, fishery-independent surveys offer the best opportunity for controlling sampling conditions by maintaining consistent gear, spatial coverage, timing, and survey design. Good indices of abundance must be proportional to actual population values. Recommendation: At the minimum, at least one reliable abundance index should be available for each stock. Fishery-independent surveys offer the best choice for achieving a reliable index if designed well with respect to location, timing, sampling gear, and other statistical survey design considerations. Obtain Auxiliary Information Finding: The committee's simulation study demonstrated that assessments are sensitive to underlying structural features of fish stocks and associated fisheries, such as natural mortality, age selectivity, catch reporting, and variations in these and other quantities. For accurate estimation in stock assessment models, it has long been known that auxiliary information in the form of indices or survey estimates of abundance, model structure information, and information about other population parameters (e.g., natural or fishing mortality) improves assessments. Model performance varied across data sets, and no single method consistently outperformed the others. The more complex models, which allowed for trends in catchability and/or selectivity parameters, tended to work better when only biased abundance indices were used in the fit. However, sophisticated modeling techniques will not fix poor data. Performance of models became erratic as more variability was introduced to data sets. Priority should always be given to collecting the best data possible, by taking into account the biology of the species being studied, the behavior of the fishing fleets, environmental conditions, and survey sampling design. Application of different assessment models to the same data can help to recognize poor data. Results from such comparisons can be used to direct survey programs to improve data quality and to assess the degree of improvement achieved over time. There has been a tendency within NMFS for regional preference of specific assessment methods, which may hinder the identification of bad data. Instead, models should be chosen based on their performance using the data available for a given stock. Alternative methods to estimate parameters of stock assessment models include properly designed and executed tag-recapture and other survey experiments using modern technology to estimate abundance and mortality; methods designed to incorporate seasonal, spatial, and behavioral observations into assessment models; and the use of stomach content information to provide an index of natural mortality over time. The committee found that different values of natural mortality were assumed or estimated by different analysts, which would account for some of their differences in performance. Therefore, analysts were asked to repeat their analyses using the true value of natural mortality. Natural mortality rates are often assumed to be
OCR for page 111
--> constant and known. Bayesian techniques discussed in Chapter 3 and implemented in the Autodifferentiation Model Builder (ADMB) that was applied to the simulated data provide a natural way to incorporate independent information about M into the assessment while allowing for the existing uncertainty. A natural mortality index could be included easily in existing assessment models or used in an approach such as multispecies virtual population analysis (MSVPA) that estimates parameters for several species simultaneously. Although there have been many criticisms of the single-species assessment approach, the committee believes that single-species assessment offers the best approach at present for assessing population parameters and providing short-term forecasting and management advice. It is important to distinguish between such single-species assessment procedures and multispecies management; the latter topic is beyond the scope of this study. Another form of auxiliary data that could improve stock assessments are measurements of environmental conditions. It is not clear how such data (e.g., spawner-recruit relationships) could be incorporated in stock assessment methods, but there is increasing evidence that environmental conditions are a major factor driving changes in recruitment to some fisheries and that should be considered in developing sampling strategies. Recommendation: Because there are often problems with the data used in assessments, a variety of different assessment models should be applied to the same data; new methods may have to be developed to evaluate the results of such procedures. The different views provided by different models should improve the quality of assessment results. Greater attention should also be devoted to including independent estimates of natural mortality in assessment models. More Realistic Assessments of Uncertainty Finding: Fish stock assessment has often focused on obtaining point estimates (i.e., without uncertainty limits) of the key parameters of the biological system. Management has not tended to follow precautionary approaches. Precautionary principles established in recent UN agreements call for more emphasis on protection and sustainability of abundance levels and less emphasis on achievement of high levels of catches (FAO, 1995a). The role of stock assessment under these new guidelines is similar to its traditional role, except for the addition of new calculations of the consequences of alternative management actions (see Appendix J). Precautionary fishery management as described by the United Nations Food and Agriculture Organization (FAO, 1995a) requires "taking account of the uncertainties in fisheries systems" (p. 2). According to FAO (1995a): The precautionary approach is made more effective by development of an understanding of the sources of uncertainty in the data sampling process, and collection of sufficient information to quantify this uncertainty. If such information is available it can be explicitly used in the management procedure to estimate uncertainty and risk. If such information is not available, a precautionary approach to fishery management would implicitly account for uncertainty by being more conservative. (p. 13) A precautionary approach specifically requires a more comprehensive treatment of uncertainty than is the current norm in fishery assessment. This requires recognition of gaps in knowledge, and the explicit identification of the range of interpretations that is reasonable given the present information. (p. 14)* In particular, assessment scientists should provide probability estimates for various consequences, for example, the probability that the stock will exceed some predetermined benchmark levels of abundance 20 years in the future. The need to evaluate management consequences in probabilistic terms will change the way some assessments are conducted. The new types of calculations required can be approached using Bayesian methods; research on other methods (e.g., the applicability of bootstrap Monte Carlo methods to posterior probability calculations) may provide new tools (see Chapter 4). Fishery managers and assessment scientists will have to * These principles are not in the form of an international treaty and thus are guidelines rather than binding agreements.
OCR for page 111
--> cooperate to fulfill the requirements of the precautionary principle and to implement better feedback procedures. Calculation of the probability of future events requires specification of the rules that management will follow to establish regulations in the future. Because future management rules are difficult to predict, a practical solution is for managers and scientists to consider a range of rules and consequences for which probabilities can be assigned. The rules that managers follow influence the relative value of surveys and expenditures for other types of applied fisheries research. If a given population is at or near unexploited levels and rules are very cautious in the sense that catches are a small proportion of the population, only minimal survey monitoring may be necessary. Alternatively, heavily exploited populations with aggressive management rules require extensive survey monitoring. The committee's simulation study demonstrated that small, but realistic, departures from standard assumptions of stock assessment models can result in substantial assessment errors. All too frequently, stock assessments provide only point estimates of key population and management parameters without standard errors or confidence intervals. For the most part, this was the case in the simulation study. The reports from most analysts did not contain measures of uncertainty (the committee did not specifically request them, and the amount of time to do the simulation study was quite limited). Even if measures of uncertainty are given, they are likely to understate the true level of variation, because uncertainty cannot be included in all parameters. Only recently have posterior probability density functions for key parameters been presented in some stock assessment documents. In our simulation study, only the Kalman filter and ADMB approaches could handle both measurement and process error, both of which are certainly present. Only the most complex models can come close to capturing the uncertainty involved, but it remains unresolved whether the performance of complex models is superior to simpler, possibly more parsimonious, models. The committee evaluated uncertainty primarily by performing retrospective analysis. This analysis showed that persistent under- or overestimation can occur over a number of years of assessment, regardless of which model is used. A risk assessment framework has been adopted in many stock evaluations conducted by NMFS (see Rosenberg and Restrepo, 1994). More broadly, recent symposium volumes attest to the recognition of the importance of this issue in stock assessment (Shepherd, 1991; Smith et al., 1993). Recommendation: The committee recommends that fish stock assessments include realistic measures of the uncertainty in the output variables whenever feasible. Although a simple model can be a useful management tool, more complex models are needed to better quantify all the unknown aspects of the system and to address the long-term consequences of specific decision rules adequately. The implementation of this recommendation could follow the methods discussed in Chapter 3. Harvest Strategies Finding: Harvest strategies and stock assessments are linked inextricably, although this is not always acknowledged. Management procedures in which the allowable catch is set as a constant fraction of biomass perform better than many alternative procedures, although management based on constant fractional harvests could result in substantial reduction in harvests if biomass is highly uncertain. Even if the rate of natural mortality assumed in the assessment is correct, uncertainty, correlation in the errors made in successive years, and persistent bias in the estimation of stock abundance can significantly degrade management performance. The simulation results reported herein, albeit limited, illustrate that assessment errors can be substantial, even when unbiased abundance indices are available. Thus, harvesting policies that are robust to these levels of errors are necessary. Instead of harvesting a constant fraction of exploitable biomass, fishing mortality could be reduced sequentially at low stock sizes or a threshold biomass level could be established below which fishing would be curtailed (which is done for many North Pacific species). In addition to lowering target harvest levels, alternative controls may be needed as a safeguard against major assessment errors. Increasing the size limits with gear restrictions could allow a higher target fishing mortality rate (F) to be applied with less risk. Area closures may be appropriate
OCR for page 111
--> for sedentary species if the distribution of organisms is well known. Area closures can also be appropriate for migratory species, in that they can protect the species from overharvesting during part of the year. Recommendation: Precautionary management procedures should include management tools specific to the species managed, such as threshold biomass levels, size limits, gear restrictions, and area closures (for sedentary species). Rigorous Evaluation System Finding: In many U.S. fisheries, biological reference points are used to set target harvest rates. Justification for the use of various reference points comes from analyses of empirical and theoretical evidence encompassing a wide variety of stock categories. The aim of these studies has been to determine reference points that lead to sustainable harvest levels for stocks of a given life-history type. However, the performance of harvesting strategies and regulatory decision rules depend on the performance of the assessment method used to implement them (Chapter 4), which may vary widely among stocks depending on data availability. The total allowable catch (TAC) for many U.S. West Coast fisheries is specified by applying F40% to an estimate of exploitable biomass, corresponding to a harvest that reduces the equilibrium spawning biomass per recruit to 40% of what it would be under no harvesting. With this method, if the estimated natural mortality rate is too high, F40% the estimated exploitable biomass, and TAC will all be too high (Table 5.3, compare data sets 1 and 2). Even if the correct natural mortality rate is known, biases in the stock assessment, uncertainty, or correlated errors in the biomass projection can degrade management performance. FAO (1995a, pp. 14-15) noted: A precautionary approach to assessment and analysis requires a realistic appraised of the range of outcomes possible under fishing and the chances of these outcomes under different management actions. The precautionary approach to assessment would follow a process of identifying alternative possible hypotheses of states of nature, based on the information available, and examining the consequences of proposed management actions under each of these alternative hypotheses. This process would be the same in data rich and data poor analyses. A precautionary assessment would, at least, aim to consider (a) uncertainties in data; (b) specific alternative hypotheses about underlying biological, economic and social processes, and (c) calculation of the response of the system to a range of alternative management actions. Recommendation: Assessment methods and harvesting strategies have to be evaluated simultaneously to determine their ability to achieve management goals. Ideally, this involves implementing them both in simulations of future stock trajectories. For complex assessment methods, this may prove to be very computationally intensive, and an alternative is to simulate only the decision rule while making realistic assumptions about the uncertainty of future assessments. Simulation models should be realistic and should encompass a wide range of possible stock responses to management and natural fluctuations consistent with historical experience. The performance of alternative methods and decision rules should be evaluated using several criteria, including the distribution of yield and the probabilities of exceeding management thresholds. New Approaches Finding: Stock assessments are compromised by incomplete or variable data and new methods will be needed to deal with this situation. The committee was reluctant to create a prescriptive list of research needs for improving stocks assessments because it is difficult to foresee where the science and practice of stock assessments should proceed in the future and because such a list could hinder research in directions not included on it. The most important action that should be taken by NMFS and other organizations that depend on stock assessments would be to create and maintain an environment that fosters the development of a range of new techniques.
OCR for page 111
--> A few prominent themes emerged from the committee's discussions. The use of Bayesian methods and other means of incorporating uncertainty in stock assessment models should be pursued aggressively. One committee member believes that interval analysis, fuzzy arithmetic, and fuzzy logic may be preferable in some instances to Bayesian approaches and that the former techniques should be investigated for possible application. Assessment models for recreational fisheries are often less developed than models for commercial fisheries and require further development. Catchability, selectivity, and mortality are often assumed to be constant over time. When these factors are assumed to be constant but are not, it can lead to faulty stock assessments. New means to estimate changes in catchability, selectivity, and mortality over time should be developed along with models to include such data. The committee's simulations showed that the biomass of recovering populations tends to be underestimated when survey data are used. New means of accounting for stock recovery must be developed. Although few stocks seem to be in recovery phases, NMFS and the fishery councils are required by law to develop recovery plans for depleted stocks. It is important to track stock recovery accurately to minimize economic disruption in a fishery while protecting the stock from depletion in the future. Recommendation: NMFS and other bodies responsible for fishery management should support the development of new techniques for stock assessment that are robust to incomplete, ambiguous, and variable data and to the effects of environmental fluctuations on fisheries. Peer Review of Assessments and Assessment Methods Finding: External peer review of scientific procedures and results is standard practice throughout much of the scientific community. When applied properly to stock assessments, such reviews would yield an impartial evaluation of the quality of assessments as well as constructive suggestions for improvement. These reviews are most beneficial when conducted periodically, for example, every 5 to 10 years, as new information and practices develop. Stock assessments are often the focus of disputes among fishery managers, fishers, and environmental groups. It is imperative that stock assessment procedures be understood better and trusted more by all stakeholders. FAO (1995a, p. 14) focused on this issue: Specifically, the assessment process should include: scientific standards of evidence (objective, verifiable, and potentially replicable), should be applied in the evaluation of information used in analysis; a process for assessment and analysis that is transparent; and periodic, independent, objective, and in-depth peer review as a quality assurance. Recommendation: The committee recommends that NMFS conduct (at reasonable intervals) in-depth, independent peer review of its fishery management methods to include (1) the survey sampling methods used in the collection of fishery and fishery-independent data, (2) stock assessment procedures, and (3) management and risk assessment strategies. Commercial Data Collection Protocols Finding: The committee found that formal sampling protocols for collection of commercial fisheries statistics were unavailable for some geographic regions. Some regions, for example, the U.S. Northeast, are in the midst of the development and publication of new protocols, whereas other regions do not appear to use standardized methods. The lack of formalized, peer-reviewed data collection methods in commercial fisheries is worrisome. To the extent that formalized and standardized procedures are lacking, potential bias and improper survey conduct may exist, with unknown impact
OCR for page 111
--> on data reliability. The committee's simulations show that 30% underreporting of fisheries data diminishes the accuracy of stock assessments. Other than this result, there has been little study of the potential impact of unreliable sampling practices on assessment accuracy, and the magnitude of this problem is unknown. Formalized sampling protocols have been developed for recreational fisheries in the form of the Marine Recreational Fisheries Statistics Survey (MRFSS). MRFSS data and methods have undergone independent peer review, are readily available, and could serve as a model for commercial fisheries. MRFSS data and methods are not perfect, however. The MRFSS design does not provide precise estimates for the types of angling that require a specialized and targeted survey, such as for highly migratory species, charter boat fisheries, or species subject to a short fishing season. Recommendations: The committee recommends that a standardized and formalized data collection protocol be established for commercial fisheries data nationwide. The committee further recommends that a complete review of methods for collection of data from commercial fisheries be conducted by an independent panel of experts. Education and Training Finding: Reduction in the supply of stock assessment scientists would endanger the conduct of fishery assessments by the federal government, interstate commissions, and international management bodies and would hinder progress in the development and implementation of new stock assessment methods. The training of stock assessment scientists should endow them with skills in applied mathematics, fisheries biology, and oceanography. Such training should begin at the undergraduate level and continue throughout a scientist's professional career. Education of fisheries scientists can and should be organized in such a way that it complements and augments the NMFS research mission and leads to improved management strategies for fisheries. Skills needed include a knowledge of the biology of the target species and of the associated fisheries, as well as applied mathematics, including probability, statistics (maximum likelihood theory, nonlinear functions, statistical decision theory, Bayesian methods, survey sampling theory), and modeling. Training can be accomplished through a variety of mechanisms, including (1) coursework as part of graduate studies; (2) internships with experienced stock assessment professionals; (3) in-service training courses offered on-site; (4) professional leave to take university courses; (5) personnel exchanges of stock assessment scientists working in academia and government laboratories; and/or (6) time spent observing actual fishing or survey operations. It is commendable that NOAA has established some formal programs to foster education and collaborative exchange with academic scientists. Examples are the NOAA-University of Miami Cooperative Institute for Marine and Atmospheric Studies in the southeast and the NOAA-multiuniversity Cooperative Marine Education and Research program in the northeast. NMFS should support undergraduate participation in research to advance stock assessment research while providing educational opportunities that interest undergraduates in fishery science as a career. Such opportunities might include research internships for undergraduate juniors and seniors in federal or university fisheries laboratories. The Research Experience for Undergraduates program funded by the National Science Foundation could serve as a model. The committee reiterates the recommendation of the National Research Council (NRC) Ocean Studies Board that ''federal agencies with marine-related missions [should] find mechanisms to guarantee the continuing vitality of the underlying basic science on which they depend" (NRC, 1992, p. 10). Recommendations: NMFS and other bodies that conduct fish stock assessments should ensure a steady supply of well-trained stock assessment scientists to conduct actual assessments and to carry out associated research. NMFS should encourage partnerships among universities, government laboratories, and industry for their mutual benefit. This can be accomplished by exchanging personnel and ideas and by providing funding for continuing education at the graduate, postdoctoral, and professional levels, including elements such as cooperative research projects and specialized courses, workshops, and symposia.
OCR for page 111
This page in the original is blank.