Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
2 Current Situation and Problems in Effort and Catch Estimation This chapter highlights the complex nature of monitoring fishing effort and catches within the recreational fishing sector, discusses the data collection and estimation challenges posed by this complexity, and focuses on issues associated with the implementation of existing surveys. At present, there is a patchwork of methods and systems of data collection for recreational fishing throughout the United States, primarily as a result of historical anomalies and different regional and state management approaches. However, basic similarities in the methods used by component programs do exist because a two-phased process1 is generally needed to arrive at an estimate of the essential parameter, total catch, based on information about effort and catch per unit effort (CPUE). Survey programs must also consider design characteristics needed to address the requirements for information on indices of relative population abundance, biological sampling of fish species, and related parameters concerning economics and angler attitudes. The common feature of catch estimation by surveys discussed in this report is that estimates of total catch for each subcomponent (i.e., the design-based spatial and temporal strata, or the post-data collection strata, defined by species, primary fishing area, and type of catch) are obtained by multiplying together the estimates of effort and CPUE gathered from two separate surveys. Total catch is estimated in this way 1Note that the committee is not referring to a nested survey process here but instead is using two-phase to indicate the use of two different surveys, one to estimate CPUE and the other to estimate effort. 31
32 REVIEW OF RECREATIONAL FISHERIES SURVEY METHODS because of concern that nonsampling errors would invalidate direct estimates of total catch derived from data of either a dockside or telephone survey alone. (See Figure 2.1 for sources of error in survey estimates.) While a telephone survey might theoretically provide access to all anglers, anglers contacted in this manner may provide poor estimates of catch because they would be required to identify species caught or to recall the size or number of fish landed for fishing trips that may have occurred weeks or even months before being contacted. Similarly, reliance on dock-side intercept surveys alone is susceptible to problems of incomplete spatial sampling frames (see Box 2.1 for a discussion about sampling frames) wherein undercoverage bias results from the difficulties of accessing private fishing sites. In addition, poor precision of intercept surveys can result from financial constraints on the number of interviews that can be conducted, particularly to reach sites that are more remote or to sample dispersed but low-use sites adequately. Therefore, the Marine Recreational Fisheries Statistics Survey (MRFSS) uses a hybrid approach in which dockside intercept surveys are used to estimate CPUE and conduct biological sampling, and catch and telephone interviews are used to estimate effort. The results of the two surveys are combined to yield an estimate of total catch. The result of using these complementary strategies for assessing effort and CPUE and for obtaining biological information is that the estimation procedure is more complex than for many other demographic surveys since it requires two separate sampling operations. Additionally, numerous adjustments and extrapolations arise because the sample frames on which the surveys are based are incomplete or unrepresentative of the entire population. Evidence throughout this chapter will show the fundamental problems associated with the overall national MRFSS program and with some of the component state surveys. These problems are variations on several common elements. There are potentially large biases in the sample estimates, and neither their magnitude nor impact can be measured using the current data. These biases are due to the following reasons: · The sample frames for both catch rate estimation and for effort estimation are incomplete, contain errors, or both. · Fidelity to sampling protocols used in both effort estimation interviews and access-point intercept surveys is not monitored adequately.
SITUATION AND PROBLEMS IN EFFORT AND CATCH ESTIMATION 33 FIGURE 2.1 Sources of error in survey estimates (Groves et al., 2004; reprinted with permission from John Wiley & Sons, Inc.). · Assumptions of unknown validity are used in the expansion of estimates over the nonsampled segments of the angler popu- lation. Other potential biases within the sampling design can be estimated using the existing data, but these analyses have not been conducted. Inefficiencies arising from overcoverage in the list frame for effort estimation result in low precision of estimates and higher cost than
34 REVIEW OF RECREATIONAL FISHERIES SURVEY METHODS Box 2.1 Area Versus List Frames and Their Use in Angler Surveys Frame: A sampling frame is a collection of units from which a sample will be drawn. The frame is ideally identical to the population (a complete frame) about which one wishes to learn, but typically, the frame is a subset of the population (an incomplete frame). If the frame is different from the population in any way, bias can be introduced if the value of a parameter for the frame is not the same as the value of that parameter for the population. Two standard frame types are list frames and area frames. Coverage errors arise from errors in elements of the frame, more commonly in list frames, and will lead to bias in estimates based on sampling of the frame. Overcoverage can arise when frame references exist but do not provide access to sample elements (e.g., licenses without addresses, incorrect telephone numbers, households with telephones but no anglers). Undercoverage arises when some population units exist but are not linked to the frame and therefore have no probability of being sampled (e.g., fishing licenses sold that are not recorded in the list frame). List frame: A list of information that provides direct access to sample units. Through its random digit dialing (RDD) sample, the MRFSS uses a list frame of all working landline telephone numbers in coastal counties. This frame suffers from overcoverage since not all households contain anglers, undercoverage since some anglers do not live in coastal counties or live in coastal counties but have no landline telephones, and duplications since some anglers live in households with more than one working landline. Similarly, the For-Hire Survey uses a list frame of charter boat operators or licenses that may be incomplete. The access- point intercept survey used within the MRFSS and component programs is also an incomplete list frame. Even though the intercept sample sites may be geo-referenced, they are chosen from a master list of documented access sites (e.g., boat ramps, docks, piers) and therefore are not an area frame. Typically, the access site frame will not list all sites, resulting in undercoverage. Area frame: In the context of site access, an area frame would be a coastline map that could be sampled in portions, and each portion would be searched for access sites. An area frame provides indirect access to sampling sites; access is indirect because the geographic areas must be selected first and the direct access to sample units achieved through a second-stage sampling process. Currently, area frames are not used in the MRFSS.
SITUATION AND PROBLEMS IN EFFORT AND CATCH ESTIMATION 35 would be required if the list frame coincided with the angler population. Moreover, the data needs for management and analysis have changed since the inception of the program, including the following: · Management decisions require data on finer temporal and spatial scales. · Recreational fishing data are now required for use in stock assessments, sometimes as the sole data concerning stock status. · Managing recreational catch and retention has become a primary activity for fisheries management as recreational removals have supplanted commercial removals for many species and areas. Finally, the expertise and personnel needed to evaluate and improve the survey design and execution continually are lacking, and methods used to collect and analyze recreational fisheries data have not incorporated evolving statistical methodology or new innovations and technologies that would improve statistical efficiency and reduce costs. A number of regional surveys have been developed in recent years with the aim of addressing some of these problems. However, with such a wide range of surveys conducted, it is beyond the committee's ability to analyze all of their individual problems and potential solutions. Consequently, the issues raised in this chapter tend to focus on the MRFSS and the For-Hire Survey, but in most instances, these same issues are also common to the regional surveys. The issues and characteristics described here are not intended to be inclusive; rather they are meant to illustrate the general nature of the sampling situations and resultant problems. BIAS AND PRECISION As with all surveys, minimizing bias and maximizing precision of estimators of important parameters are the goals of the recreational fishing survey program. The problem with achieving these goals is that the nature of recreational fishing does not allow for data to be collected for all anglers. Ideally, representative samples that allow unbiased estimation of the catch by the total angler population should be collected. However, resource limitations, survey design characteristics, sample frame errors, and restricted access to anglers in some modes may result in nonrepresentative sampling of the angler population. Therefore,
36 REVIEW OF RECREATIONAL FISHERIES SURVEY METHODS adjustments, or expansions, in the estimation process are employed to account for the lack of information for some anglers. These adjustments require assumptions about the behavior of the unobserved anglers that are of unknown validity. Furthermore, the data do not exist to test the validity of these assumptions or to determine whether they result in large biases. Not knowing whether the adjustments introduce bias, and not being able to test for this bias, creates uncertainty about the quality of the estimates. Variation in an estimate among years is a source of major debate for recreational fishing surveys--especially where fluctuations in estimates result in equivalent fluctuations in regulations for subsequent years. It may be the case that these fluctuations are real, but they also may be artificial. They may result from low precision in the estimate (which can be corrected by increasing the sample size or sampling efficiency) so that the estimate may be unbiased but may vary from the true parameter value in any given period because of expected variation. It is currently difficult to assess if this is the problem because standard errors may be estimated incorrectly. Recreational fishing provides formidable challenges in estimating catch, effort, and economic expenditures by anglers, either regionally or nationally, due to the diversity of fishing sites and modes available to anglers. Recreational fishing can be an individual or group pursuit. It can be based on shore or on water and can be conducted on private boats or through a commercial for-hire vessel. Angler trips can originate from private residences that border fishing waters or involve travel over thousands of miles to a departure site, with additional travel on water to the fishing grounds. Effort can range from only minutes of active fishing for anything caught or for a favorite species to multiple-day trips involving multiple targets; often, trips can cover the entire 24-hour period. Furthermore, the target species for anglers may be varied and may include species entirely allocated to recreational fisheries, as well as those from mixed recreational and commercial fisheries. The difficulties of covering all fishing modes, access points, and duration of fishing has led to several additional surveys that complement the basic MRFSS approach. Yet, even these additional surveys are unable to measure all essential strata, leading to assumptions about unsampled fishing behavior. Below is a brief description of the different angler modes that highlights survey and estimation procedures that are used and how bias or imprecision may be introduced into estimates of effort, CPUE, and the resulting total catch. The issues discussed are not
SITUATION AND PROBLEMS IN EFFORT AND CATCH ESTIMATION 37 intended to be exhaustive for all surveys or even for a single survey but are intended to emphasize the issues that are described below. Private or Independent Fishing Shore-based Shore-based fishing refers to fishing directly from the shoreline (e.g., beaches, banks, headlands) or from artificial structures, such as docks, jetties, piers, bridges, breakwaters, and causeways. This is the most difficult sampling environment because of private property issues and because of a virtually unlimited number of small access points. Anglers who participate in fisheries from public or commercial property can be intercepted by onsite samplers and can be included in CPUE estimation; however, the extensive amount of publicly available property and structures makes attaining an efficient probability-based sample chal- lenging. Also, some shore-based anglers are not accessible through the public access-point frame used for estimating CPUE because they fish from private property. An angler fishing from a private residence might never be subject to an intercept interview, and therefore, his or her data never could contribute to CPUE estimate. Instead, his or her CPUE would be assumed to be the same as for anglers fishing and sampled though other modes. However, in order to expand the estimates based on sampled anglers to this unsampled portion, the assumption must be made that the species composition and catch rates of these anglers is the same as for the sampled anglers. This is assumed to be true, but data to test this assumption have not been collected. These anglers can be included in estimation of effort through the telephone frame. However, a consistent definition or duration of "angler trip" between shore-based and waterborne fishing is elusive. Effort for this shore-based private fishing is measured through the MRFSS random digit dialing (RDD) survey, but only for anglers who live in coastal counties. Anglers who reside beyond this area, but who fish from shore in the survey area, are excluded from the sampling frame.
38 REVIEW OF RECREATIONAL FISHERIES SURVEY METHODS Residential Boat Ramps and Docks Similar to shore-based anglers fishing from private property, waterborne anglers who launch from private residential property are not normally subject to access-point intercept sampling because samplers do not have access to private residential property. If CPUE for these anglers is the same as for those launching from public access sites, then no bias is introduced from this undercoverage. It seems possible, however, that the experience and knowledge of the local area among anglers in this mode may cause the two groups to differ in CPUE. The effort within coastal households for this mode can be estimated through the MRFSS RDD survey. Publicly Owned and Commercially Available Boat Ramps and Moorage This mode is similar to the use of public structures for shore-based anglers in that use of public facilities for boat launching or moorage provides the opportunity to conduct intercept sampling of waterborne anglers. However, sampling this subpopulation of anglers still can be problematic if there are a great many launching sites. The large number of sites and the limited survey budgets and time may result in a tendency to exclude many small sites from the list of sites chosen for sampler coverage. There may also be issues associated with the timing of angler presence at these sites; the intercept sample design must account for any such temporal stratification. Effort for local anglers (those residing in the selected RDD calling area) using this mode will be estimated through the MRFSS RDD survey, but effort for nonlocal anglers will not. For-Hire Fishing When anglers go with a guide, charter fishing on boats with crew, or on head boat trips, their participation and removals are estimated through a different framework than that used for private anglers. However, anglers who rent boats for independent, nonguided fishing are captured by the current MRFSS sampling approaches; these waterborne anglers are treated similarly to the private boat anglers discussed above. Head boats, charters, and guided boats are commercial enterprises, require registration, can be listed, and thus constitute a smaller and more
SITUATION AND PROBLEMS IN EFFORT AND CATCH ESTIMATION 39 efficient list sampling frame than that of the population of independent anglers. (Only some states have lists based on saltwater fishing licenses.) Effort in the For-Hire Survey, which measures number of boat trips, number of anglers, and areas fished, is determined from boat directory telephone surveys instead of RDD employed in the MRFSS. Because the list frame is complete, assuming that the directory is kept up to date, the potential bias associated with not collecting effort data from noncoastal county anglers is not an issue as it is with the MRFSS. Catch rate, however, may still be collected though dockside interviews, which share the same sampling issues associated with this type of sampling (discussed later in the chapter). In addition to these general problems, there are specific issues associated with the dockside interview for head boats. Each angler's data are likely to be highly correlated. This results in cluster effects that, if not accounted for, can have a significant impact on both the bias and the standard error calculation for the final estimates (see Chapter 3). (Cluster effects also should be expected for nonguided boat anglers, although probably to a lesser degree than for head boats.) In addition, biological sampling of these catches should account for cluster effects, and stock assessment analysts using these data also must be aware of these potential effects. The for-hire sector can provide an additional unique opportunity for recreational catch and effort sampling because records of angler participation generally are kept by for-hire companies. These records provide two capabilities: direct estimation of fishing effort (and, frequently, catch) and a source of validation for estimates obtained through alternate sampling methods, such as remote-access sampling of anglers based on a different sampling frame. Records of client participation are kept to varying levels of resolution. In the case of guide boats, records normally are associated with individual anglers. For example, guide boats taking anglers for high-prestige species, like tarpon or bonefish, may involve considerable expenditures, and records for an individual angler might have historical and future value for the guide. For head boat and charter boat fishing, records of fishing effort by anglers may or may not be accompanied by removal data at the individual level. Validation of charter boat records is recognized as an important component and source of error information for the estimation process. Access-point intercept samplers have noted inconsistencies between charter boat logbook records and observed presence and absence information on vessels at their normal home port. It is important to create a rigorous and objective sampling protocol for validations of this type.
40 REVIEW OF RECREATIONAL FISHERIES SURVEY METHODS Similarly, validation of angler participation also must accompany the use of charter boat data. It is important that charter boat anglers be included in alternate estimations of fishing effort, such as remote-access sampling, so that a validation of charter boat records can be achieved. It also is important, of course, that care be taken not to count anglers twice (i.e., once in contacting them individually and once through the for-hire survey). The implementation of the Party Charter Survey (For-Hire Survey) in California has improved the estimates of effort and therefore catch by this sector. The ability to define the sampling list frame through a directory of commercial enterprises also has improved the efficiency of sampling these anglers over what had been achieved previously in the MRFSS. In addition, more timely data are provided because a percentage of the vessels within the directory are sampled each week instead of waiting two months, as with the MRFSS. Additional improvements that can be made for this sector are discussed in Chapter 3. Tournaments are special cases that might have some potential use for assessing biases and for providing information for some species. Although angler catch and effort often are well documented, they do not represent typical angler activities and often focus on highly migratory species, which often are not included in the MRFSS. Night Fishing In some areas, night fishing is common and creates unique chal- lenges to estimation of catch rates and, to a lesser degree, fishing effort. Effort for night fishing can be estimated through the telephone survey in the same way as for other modes of fishing. However, estimation of catch rate for this mode is highly problematic because, while anglers participating in this mode may be accessed, in theory, through an existing frame, they are inaccessible because samplers normally do not intercept anglers at night. Therefore, a secondary temporal stratification within the access site sampling frame is required to estimate catch rate by this fishing mode. Such a program has been implemented in the Mississippi Shore Night Fishing Survey. Another method of obtaining angler-supplied night-catch information is to add some questions to the telephone survey; although, this will create additional complexities. In many cases, night-fishing catch will have to be ignored.
SITUATION AND PROBLEMS IN EFFORT AND CATCH ESTIMATION 41 Spatial and Temporal Issues of Sampling Coverage In most cases, CPUE is achieved via intercept sampling at access points. Access points are given different probabilities of selection into the sample, with sites weighted and chosen based on expected angler activity. The set of selection probabilities are referred to as the pressure matrix. This sample design is selected to improve the efficiency of the CPUE estimate and seems likely to do so if the pressure matrix effort estimates are accurate. While the MRFSS and its derivatives have attempted to keep the pressure matrix relevant to current effort distribution, the methods used to update the pressure matrix are not consistent across regions. In addition, the selection probabilities are not used in the estimation process, which will lead to bias in the estimators, except in unusual circumstances. This is discussed in more detail in a later section. A source of potential bias in the estimate of effort is due to the proportion of private anglers who are not part of the sampling frame. Effort estimation is based on telephone sampling of residents of coastal counties, and many private anglers do not reside in these counties. An adjustment based on information obtained in the intercept sample is attempted, but this will be adequate only under special circumstances. (Again, this is discussed in more detail in a later section.) This mismatch of the frames for estimating catch rate and effort results in a decreased capability for validation of fishing effort estimation through comparison of estimates from the two frames. The temporal stratification of the current MRFSS is based on two- month sampling periods, or waves. However, the timeliness of the estimation from each wave varies by region. In most regions, the lack of timeliness is not important because species' harvests are not managed in- season. However, for several major species on both coasts and in the Gulf of Mexico, in-season estimation is a key component of man- agement. The timeframe for estimation through the MRFSS process (because it takes a long time to accumulate enough fishing households to have an adequate sample size) does not address management require- ments consistently, in part due to the inefficient telephone sampling frame for estimating fishing effort.
42 REVIEW OF RECREATIONAL FISHERIES SURVEY METHODS GENERAL PROCEDURAL AND ESTIMATION ISSUES As a precursor to discussing the specific issues of precision, bias, and efficiency, it is necessary to describe the framework by which the estimated total catch is derived. Total estimated catch () (here total catch represents landed and released catch) is the product of total estimated effort (Ê), made from the telephone survey, and estimated catch rate of anglers (CPÛE) from the intercept survey. That is: C^ = E^ ×CPUE ^ Effort, in turn, is estimated as the ratio of the estimated total number of trips for those households sampled in the RDD frame (ÊRDD) and the estimated proportion of anglers who are sampled using the RDD frame (^ RDD ) (this number will be less than 1 since some anglers reside outside the frame, in noncoastal counties, and is estimated from the access survey): E^ = E^RDD /^RDD In other words, total estimated effort is the corrected effort after adjusting for the proportion included in the sampling frame ^ RDD. The variance of total catch is estimated using a delta method variance estimator for products and ratios. These estimates for individual areas or strata are summed across areas or strata to obtain aggregated estimates. Accounting for Anglers Not Included in the Effort Frame As previously discussed, some anglers are not surveyed for effort because they reside outside the coastal county RDD sampling frame; therefore, the frame used for estimating effort is incomplete. That is, the sample frame includes only a subset of the true population (i.e., those in telephone households in coastal counties), and estimates derived from sampling of this frame are then adjusted upward by "expanding" the frame. The expansion factor is generated through the intercept survey; when interviewed on the dock, anglers are asked where they live. The proportion of RDD-in-frame to total anglers (both in- and out-of-frame)
SITUATION AND PROBLEMS IN EFFORT AND CATCH ESTIMATION 43 is used to estimate RDD , and subsequently to expand ÊRDD to total estimated effort. In theory, this is a reasonable expansion, but in practice, there are some concerns with this method. In particular, the intercept frame that is used to correct for the incompleteness of the effort frame is itself incomplete. Private property and docks (without public access) are not included as sampling sites. Therefore, the estimate of RDD likely would be biased downward since noncoastal residents might be most likely to fish from areas that are subject to the intercept surveys. Conversely, if these anglers do not live on the water, they most likely will access the water through public sites (although a component may visit friends or relatives with private access). Therefore, a smaller fraction of coastal county residents most likely would be intercepted than there are in the angler population. If this sample proportion is biased downward, it will result in an overestimate of effort. However, the key point is that there is no information in the data collected that allows analysis of this bias. In addition, the intercept data may be biased simply due to a lack of a true probability sample because interviewers on the docks in many cases are allowed considerable freedom in choosing the sample and may select interview sites based on sufficient numbers of anglers, their receptivity, or other factors. This issue is discussed in more detail in a later section. Inefficiencies in the Effort Estimation RDD, even limited to coastal county residences, is not the most efficient way to gather angler effort information. In urban areas, less than 1 in 20 of the telephone intercepts reaches an angler. Improving the process whereby anglers are identified and contacted would not only improve the quality of the estimates but also should reduce costs. Remedies exist for other inefficiencies as well. For example, under the current sampling regime, identifying an angler costs more than the taking of information once the angler has been identified. Under this high ratio of costs for identifying anglers to costs for information gathering, it may be more efficient to repeatedly sample identified anglers (i.e., a panel survey). This also would allow comparison of angler effort levels and perhaps catch rates over time, yielding more precise estimates. Even if the costs were comparable, repeated sampling may be preferable because accuracy in trend detection may be improved.
44 REVIEW OF RECREATIONAL FISHERIES SURVEY METHODS Accounting for Anglers Not Included in the Catch per Unit Effort Frame Estimation of CPUE is achieved via intercept sampling at access points. However, the list frame used for the intercept sampling program is incomplete. The sample cannot yield information about CPUE from anglers who fish from small public access sites not included on the frame, anglers fishing from private shorelines, or those who leave from private docks. For the sample estimate of CPUE to be unbiased, one must assume that anglers who fish at sites not included on the intercept sampling frame have the same success in fishing and that they target the same species as those who fish at sites that are included on the frame. It also must be assumed that the unit of effort, the trip, is equivalent in the two groups. In fact, it seems likely that a trip actually has a different meaning for those who fish from their own private dock and those who do not. If access is easy, such as having a dock or boat near an angler's home, a "trip" could last only a couple of hours. Alternatively, for an angler who is paying for a charter, fishing may last the entire day. The catch per trip for these two groups could differ only due to the length of the trip. There is no way to assess the bias due to this undercoverage since no information about catch rate is available for anglers at the sites not in the frame. The daily timing of intercept surveys presents a second source of undercoverage. Most intercept surveys sample only during daylight hours and may have considerable flexibility in timing of sampling. In many venues, anglers do not return to access points until after dark or may engage entirely in night fishing. These anglers therefore are not captured in the CPUE survey frame. This unsampled component has been recognized through the development of some special-purpose surveys, such as the Mississippi Shore Night Fishing Survey, but such surveys are limited in number. Catch rates, by species, from daytime fishing are applied to the night fishing mode in the same fashion as extrapolations to other unobserved fishing, but such assignment of both catch rate and species composition may not be justified and may diminish the accuracy of the estimates. Even if catch rates are the same for night fishing, the angler community recognizes this undercoverage in the intercept frame and may perceive the results to be less credible. The intercept frame contains a list of angler access sites, along with estimates of fishing effort at those sites. The sample design uses unequal probability of selection of access sites, with selection probabilities proportional to estimated fishing effort. However, these measures of
SITUATION AND PROBLEMS IN EFFORT AND CATCH ESTIMATION 45 effort are not used in the estimation procedure, at least as carried out by the MRFSS. Instead, the design is treated as though it was self- weighting, and an unweighted estimator is computed. This estimator will be biased, except in very special circumstances. There are two conditions under which the use of the unweighted estimator would not result in a bias in estimation. If there is no difference in CPUE (by species or type) by anglers using different access points, then there will not be bias resulting from ignoring the weights. Alternatively, a self-weighting estimator would be unbiased if the selection probability is actually proportional to the number of angler trips at each site and a constant number of anglers were sampled at each selected site. In that case, the probability of selection for each angler trip would be: (ti /t)(k /ti) = k /t where ti is the number of angler trips at site i in the period, t is the total angler trips, and k is the number of angler trips selected at each site. If neither of these is true, however, then bias in the estimation of CPUE could result. The validity of this estimate could be investigated from the sample data by comparing estimated CPUE in sites with low and high selection probabilities. However, one could not assume that because this is true for a few subpopulations that it will remain generally true for all time, geography, species, and catch type cells. Rather, one would need to monitor this assumption regularly or develop an estimation system that does not require such an assumption (e.g., using weights computed from the design). The Possibility of Introducing Bias into the Intercept Survey It is randomness in the sampling process that assures that a probability sample is achieved. More specifically, in order for a sampling method to be called a probability sample, each unit must have a positive and known probability of selection. Thus, a probabilistic algorithm must be used (either before interviewers are given assignments or by the interviewer in the field) to decide which units will enter the sample and which will not. In a probability sample, interviewers will exercise no judgment in choosing who to interview. To the extent this protocol is
46 REVIEW OF RECREATIONAL FISHERIES SURVEY METHODS followed, the laws of probability will provide estimators with known properties. Currently, the onsite intercepts for all recreational surveys are assumed to be a random sample. However, the collection of intercept data has been tailored to the kinds of access sites that are present in a particular region, and interviewers frequently are allowed to make judgments about where, when, and which units to sample. This means that these samples may not be true probability (random) samples. Generally, the leeway afforded to onsite samplers is an attempt to reduce costs by reducing the time it takes to gather the target number of samples. This is problematic because such sampling is, in essence, a quota sample,2 rather than a probability sample in which all anglers have a known probability of being intercepted. This deviation from a probability sampling protocol has an unknown impact on estimates of both CPUE and effort. Besides the deviations from sampling protocol that are explicitly allowed, there may be other instances in which interviewers stray from instructions on sample selection. There is no regular interviewer monitoring program included in the sampling protocol, as is common in most survey operations. Indeed, it would be difficult to use the most common types of interviewer quality control programs in the intercept survey setting because they are based on a reinterview of a sample of respondents. The result of this problem is that it is not known, nor is there an easy way to determine, how much interviewer error affects the quality of the data gathered through the intercept survey. Making the development of a reinterview program especially difficult is the fact that intercept interviews are conducted by a wide range of people. Several states either have their own intercept surveys or have taken over the conduct of this portion of the MRFSS, but still others rely on contractors to complete the surveys. With multiple organizations involved, it is difficult to specify and monitor adherence to a common sampling protocol across survey efforts. 2A quota sample defines groups of people who are deemed important to reach, based on information about the target population. Quotas are set for each group based on the group's relative size in the population. Quota samples are not random samples, and their use can lead to bias if anglers who are difficult to reach differ from those who are easy to reach. In addition, the precision of estimates cannot be calculated (Pollock et al., 1994).
SITUATION AND PROBLEMS IN EFFORT AND CATCH ESTIMATION 47 Variance Estimation Standard error is an important indicator of the quality of an estima- tor, and yet, correctly estimating standard error may be the most difficult part of the estimation process. The sample design and estimator form for the MRFSS are nonstandard, so correct assessment of variance is challenging. There appear to be some problems with variance estimation as it is carried out currently. Addition of variances across subpopulations to obtain valid estimates of variance for aggregates requires that the estimators within each subpopulation be independent. For some subpop- ulation aggregates, this will be valid. For example, for strata representing time periods, it seems a reasonable assumption since both intercept and telephone samples are drawn independently in different time periods. (However, for sparse subpopulations, information from past time periods are imputed, which would invalidate this method.) For aggregations over post-strata, such as catch type (e.g., removals consisting of catch available for inspection [A] and catch unavailable for inspection because it is filleted, discarded dead, or refused for inspection [B1]), this is unlikely to be valid since the same sampling units are used to obtain each type of data. There is information in the data that would allow this correction to be attempted; in other words, it is possible to calculate correlations from the sample, and the variance estimation method could be changed to account for the correlation. Estimating Mortality The issue of catch and release of both target and nontarget species requires much greater attention and estimation of associated mortality rates than has occurred to date. Currently, catch released alive (B2) usually is not incorporated into the catch estimates used for quota monitoring because there is no verification of the catch and it ostensibly is released alive, although Oregon and Washington do apply hooking mortality rates to discards in ocean boat fisheries that vary by species and other factors. However, this assumption ignores the high hooking mortality rate associated with some fisheries, especially for those species with swimbladders that are caught at significant depths, such as rockfish or grouper. In some instances, mortality on released fish may represent the major mortality factor in total removals. For example, several species of west coast rockfish are under severe restrictions of total allowable
48 REVIEW OF RECREATIONAL FISHERIES SURVEY METHODS catch, and the recreational fishery (both retained and released) is the primary source of removals. Measuring the release of species is problematic because it relies on angler recall (in the absence of onboard observers) and angler knowledge of species. Estimates of size are even more difficult to collect and most likely are overestimated due to prestige bias or are subject to rounding errors. Better methods to estimate the number of released fish are needed. Some of the specific issues related to catch-and-release fisheries include (1) released catch cannot be inspected in an onsite survey, unlike the kept catch; (2) rounding errors are common; (3) exaggeration or under reporting due to memory problems are possible; (4) species identification errors may occur; and (5) the size and age distribution may be different compared to kept fish. All of these errors can be serious. Observers in boat-based recreational fisheries can be used to obtain direct estimates of fish release numbers. However, releases are likely to be different if observers are present. Research on released fish mortality estimation from cage studies and tagging studies is needed to help estimate the contribution of the auxiliary information collected about the depth from which fish were caught. However, cage experiments would provide only a minimum estimate of mortality and reflect only "physiological" mortality rather than the "ecological" mortality that would be measured through tagging studies. NEW DEMANDS ON RECREATIONAL FISHING DATA This committee identified a number of areas in which designers of sampling programs, data collectors, and users of recreational fisheries data appear to have incomplete communication, mismatched criteria, or other miscommunications. In most instances, these issues have arisen because the current uses of recreational fisheries data were not anticipated in the design of the MRFSS. Current users require data with higher resolution--spatially, temporally, and taxonomically--than the current MRFSS design can deliver. Two of the major recurring issues facing recreational surveys are adequate spatial and temporal resolution. These needs are driven primarily by the type of management applied in each area. Management tactics have changed since the inception of the MRFSS and continue to change as more stocks are monitored and managed. Survey designs now require greater coverage and more detail to estimate harvest and effort
SITUATION AND PROBLEMS IN EFFORT AND CATCH ESTIMATION 49 for national and interstate management (Henry, 2002; Lyle et al., 2002; Pollock, 2002). In addition, recreational data are now used for many stock assessments. For stocks that now have low catches from the commercial sector, recreational fishing data may be the primary data on which stock assessments are based. In addition, topographic and other differences between the coasts in various regions also affect demands on sampling design. For example, the rocky coasts of the Pacific Northwest, with their rough seas, provide far fewer potential access points for boats than the sandy coasts and calmer waters of Florida and the Gulf of Mexico. The application of recreational data to finer management scales and their use in assessments have highlighted potential issues with the bias and precision of current survey methods that previously were less important. However, if these uses of the data are to continue, changes in the survey methods are required to provide the needed information. Spatial and Temporal Resolution in Catch and Effort Estimation Currently, many fisheries are monitored at the state level, which is a finer stratification than intended originally for the data collected. In order to provide state estimates with reasonable precision, many states have increased their sample size, either by adding additional sampling by state personnel or by asking the MRFSS contractor to complete more calls and onsite intercepts. These actions, taken on the whole, seem to result in more precise estimates of total catch within these smaller areas. In addition, these measures appear to have increased angler confidence because increasing sample size is a straightforward premise that non- survey scientists can understand. In some cases, it also presents the state as taking a proactive approach that is appreciated by anglers--the states are no longer just saying that the data are not good enough to manage, they are actually doing something about it. Of course, additional samples require more money, but if quotas are to be allocated and monitored by each state, these additional samples are necessary. There are numerous other methods that can be used to increase precision on smaller scales than are employed in other national surveys. However, these methods generally have not been explored for their application to recreational fishing surveys. Some of these methods are discussed in Chapter 3. However, data gathering on smaller scales will only be useful if the data collection methods are not biased and the
50 REVIEW OF RECREATIONAL FISHERIES SURVEY METHODS assumptions made about extrapolations and imputations are valid, as dis- cussed in the previous section. The temporal scale of data collection also continues to be pushed to a finer level of resolution than originally intended. Within the MRFSS, effort sampling is conducted in two-month waves. Checking the data and completing the final estimates takes another two months, meaning that catch estimates are not released until at least four months following the actual fishing effort. This lack of timeliness raises many issues for man- agers. Obviously, this time lag does not allow in-season management, which is why different surveys have been implemented in states wanting to manage in-season, such as California, Washington, and Oregon. These states have reworked the fundamental components of the MRFSS--the intercept survey and the telephone survey--in order to compile more timely data. The most fundamental change is the implementation of an angler registry so that the sampling frame used to determine effort is more defined and efficient than that of RDD of the MRFSS. Finer spatial-scale management also has required larger sample sizes for each sampling wave to ensure sufficient precision of resultant estimates. Even with annual management, data timeliness is an issue. Often, the data from the previous year have not been analyzed completely until the following season is under way. This can result in adjustments to the total allowable catch once the season has begun. While this is not truly in- season management, the effects can be similar if the adjustments to the current year mean that fewer fish can be taken or if the season has to close earlier than expected. These situations are difficult for anglers and operators of for-hire vessels to deal with. For example, fishing trips can be planned many months to a year in advance; yet, there may be no guarantee the fishery will still be open for future planned trips. This uncertainty is perceived to be a much larger problem in recreational fisheries than in commercial fisheries because anglers can be infrequent users. The time it takes to collect, verify, and calculate fishing effort and catch using conventional survey approaches is too lengthy, even for annual management, if stability in the yearly total allowable catch is desired. Use of Data for Stock Assessments A large mismatch appears to exist between recreational sampling programs and the stock assessment scientists using them. The MRFSS
SITUATION AND PROBLEMS IN EFFORT AND CATCH ESTIMATION 51 and even some of the newer surveys were not designed to gather data for stock assessments; yet, the estimates of total catch and the biological data collected during the intercept survey are often used in assessments. Assessment scientists using these data generally do not have a clear understanding of the data collection process, or the data collection process may not be executed in the manner assumed by the scientist. The implications of this mismatch between those collecting and those using the data are profound. The lack of continuity in intercept samplers, differences in sampling methods applied to different modes of fishing (e.g., shore-based and boat-based private anglers or those using various for-hire vessels), differences in sample element definition, lack of incorporation of design elements in the estimation process (e.g., weighting of spatial or temporal sampling strata), lack of consistency (or accuracy) in species designation among fishing or sampling modes, and the inability to combine information based on different sampling modes all compromise the inclusion of these data in the assessment process. Data from different sampling modes may have unknown statistical properties because the data collection emerges from the implementation of general designs that are adapted to suit local circumstances. Scientists using these data may assume that their statistical properties are known and estimable. (More specific problems associated with recreational fisheries data and their incorporation into stock assessments [e.g., the difficulty of measuring which, if any, species are being targeted] are discussed in Chapter 4.) A common knowledge base among anglers, data collectors, and data users is required if surveys are to fulfill current data needs. This is not to say that all anglers must have a complete knowledge base of species, but the intercept samplers and the anglers must categorize catch to a jointly understood level. This is particularly important for taxonomic stratifi- cation of data. Stock assessment scientists must be able to employ data with confidence that species designations are applied accurately and consistently in the sampling process or with knowledge that higher groupings of taxonomic categories are used. INCORPORATING NEW IDEAS AND TESTING OLD ONES Surveys designed for monitoring long-term status of populations have considerable inertia and are resistant to change. In part, this resis- tance is appropriate if the data provided by the surveys are to be consistent and useful over long periods. Major design changes can break
52 REVIEW OF RECREATIONAL FISHERIES SURVEY METHODS the continuity of data and render them unusable for population moni- toring unless the new and old surveys are run in parallel for some years during the change. In addition, the original design objective for a given survey may continue to be relevant, in spite of subsequent objectives that may be of equal or greater priority than the original. Resistance to change also arises because of the fixed commitment of resources (human and material) to existing designs. The MRFSS has made some changes to accommodate better estimation of some fishing modes (e.g., For-Hire Survey of charter boats). However, the fundamental aspects of the two-phased survey have not evolved significantly since the inception of the program. Different survey designs exist that could possibly improve the quality of the collected data; yet, few new approaches have been undertaken by the MRFSS or the state surveys. Indeed, several previous reviews have offered suggestions for improvements, but most of these, including several from a previous report by the National Research Council (2000), have not been implemented, perhaps due to a lack of staff and additional funding. While several external reviews of the MRFSS or portions of it have been conducted (Essig and Holliday, 1991; Guthrie et al., 1991), there is presently no internal process of user feedback on evaluation and modification of the design within the MRFSS. Some users of recreational data have initiated dialogue with the survey project managers to address specific design issues, but the need exists for a structural feedback process. The rapid evolution of uses of and needs for data from recre- ational fisheries underscores the requirement for ongoing evaluation by survey managers. OUTREACH The committee heard from numerous groups and individuals expressing a lack of confidence in the estimates produced by several of the recreational surveys. While this is not a problem with the survey methodology per se, increasing understanding and confidence in the programs can be as essential as improving the data itself. The credibility gap arises from several causes, including a belief that alternate data sources are more credible; criticism of the temporal, spatial, group, or taxonomic stratification of the intercept sampling; lack of understanding of statistical methodology; or recognition that the existing sampling frames do not describe the angler population adequately.
SITUATION AND PROBLEMS IN EFFORT AND CATCH ESTIMATION 53 In addition to dialogue on design issues, survey managers also need to advise data users on constraints to some uses, as well as on funda- mental features of the data collection system. The websites for the MRFSS and the regional data programs (National Oceanic and Atmospheric Administration, 2005b) are information rich and provide general background for the average angler. In addition, in recent years, the MRFSS personnel have begun to conduct regular meetings with users to review results of sampling waves. However, the committee heard of a number of instances where users extracted sections of data histories but were unaware of the data characteristics, the methods of compilation, or the fundamental nature of sampling estimation versus census. These observations indicate that while the program has undertaken some outreach activities with users, misconceptions and lack of clarity on data characteristics continue to exist. Further, the lack of user understanding of the design basis of the survey clearly has created some lack of trust even in the underlying data. Considerably greater outreach effort appears necessary but with the recognition that user distrust may not be overcome completely. CONCLUSIONS AND RECOMMENDATIONS The designs, sampling strategies, and collection methods of recreational fishing surveys do not provide adequate data for management and policy decisions. Unknown biases in the estimators from these surveys arise from reliance on unverified assumptions. Unless these assumptions are tested and the degree and direction of bias reliably estimated, the extent to which the biases affect final estimates will remain unknown. The statistical properties associated with data collected through different survey techniques differ and often are unknown. The current estimators of error associated with various survey products are likely to be biased and too low. It is necessary at a minimum to determine how those differences affect survey results that use differing methods. It is impossible to assess the adequacy of recreational fishing surveys, particularly those associated with the MRFSS, when potential biases exist. Identifying and eliminating the sources of bias or estimating and correcting for the degree of bias is a fundamental requirement for the provision of reliable estimates from the MRFSS. The statistical properties of various sampling, data-collection, and data-analysis methods should be determined. Assumptions
54 REVIEW OF RECREATIONAL FISHERIES SURVEY METHODS should be examined and verified so that biases can be properly evaluated. The complexity of the recreational fishing surveys makes them susceptible to many forms of bias. Unfortunately, it is difficult to avoid this complexity due to the diverse nature of recreational fishing. However, it is important to eliminate sources of bias or appropriately adjust for them when bias is unavoidable. Biases can be addressed through expanding the sampling frame to better represent the population, through experiments used to derive the appropriate correction factors, and through better training and monitoring programs aimed at improving the quality of data sampling. All of these approaches are discussed in the following chapters. Some of what is viewed as bias by the public can be the result of variance of the mean estimate, arising from inadequate sample size or other sampling errors. While reduction in variance often can be achieved by increasing sample size, improving the statistical efficiency through appropriate choice of estimators and careful implementation of sampling protocols can also be useful. Improved precision commonly is achieved by increasing sample size, and improvements can be gained for estimates derived from recreational fishing surveys through just such an approach. However, gains in statistical efficiency also may be achieved by considering alternate estimators that make better use of the information available and by identifying and implementing mechanisms that improve the effectiveness of the sampling procedure, as for example through the creation of a complete sampling frame of anglers. Improvement will come not only as a result of greater precision but also in terms of reduced sampling effort and cost. Greater demands on recreational fishing data from both the science and the management sectors are being made. Management decisions are often made at finer spatial and temporal scales than they were earlier, the mix of recreational and commercial fishing has changed for many areas and species, and stock-assessment models now make greater use of data from recreational fisheries. Reallocation of harvest from commercial to recreational sectors has increased the need to gather stock assessment information in greater detail from recreational fisheries sources. As managers use recreational data on finer spatial and temporal scales, issues of precision and bias become more pronounced. Existing spatial and temporal sampling strata may be of too coarse a resolution to generate estimates that are adequate for the management requirements. The MRFSS is in need of additional financial resources so that technical and practical expertise can be added to assist in a major overhaul of the design, implementation, and analysis of data from
SITUATION AND PROBLEMS IN EFFORT AND CATCH ESTIMATION 55 the MRFSS. The goals and objectives of fisheries managers, as well as the different surveys, are evolving constantly. There has been progress in survey programs directed to some targeted fisheries with the implementation of new, tailored surveys; yet, additional improvement is required. There have been several reviews of the national program in the last 10 years, but a more fluid, continuous review and feedback would allow for evolution of the program to meet emerging needs. In addition, as statistical theory and sampling technologies improve, it is essential that the managers of these regional or national monitoring programs have greater access to expertise in statistical analysis and sampling design. It appears that the implementation of new survey methods is hampered by the inertia of existing surveys and that even when a need for change is identified, a lack of resources, staff time, and expertise may prohibit implementation of such changes. Development of new survey methods could be accomplished by an external, independent research group as discussed in Chapter 6.