Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
6 Program Management and Support As noted in earlier chapters, collection and processing of timely and scientifically credible data for recreational fisheries is extremely chal- lenging because of complex survey designs and measurement issues; furthermore, these challenges are evolving over time. Many short- comings in the current statistical system for marine recreational fisheries have been identified in this report and in earlier reviews (e.g., National Research Council, 2000, 2004). Sampling and measurement methods used by the Marine Recre- ational Fisheries Statistics Survey (MRFSS) are widely questioned, and several states have opted out of the current national system, leading to either a duplication of effort or difficulties in standardizing method- ologies. Estimation methods do not take full advantage of modern statis- tical techniques, particularly the use of auxiliary information. RESEARCH NEEDS AND PROBLEM SOLVING Currently, a great number of biases exist, and assumptions are made because the sampling methodology is often inadequate to allow for accurate data analysis. Therefore, research is needed to determine how best to reduce biases and assumptions. Since data collection is not supervised by an overarching group, research also needs to be done to improve the accuracy of data collection. Previous chapters outline the issues and concerns with the current survey methods. A number of ideas have been suggested for circum- venting or ameliorating these issues, including establishing a better sampling frame, tracking individuals through a panel survey, making use of auxiliary information where appropriate, and conducting experi- 107
108 REVIEW OF RECREATIONAL FISHERIES SURVEY METHODS ments to identify and account for biases and to estimate influential factors that otherwise cannot be determined (e.g., the estimation of release mortality rates). The hiring of additional fulltime statisticians would increase the expertise within the statistical offices and would improve data analysis and therefore fisheries management decision making. A NATIONAL STATISTICAL PROGRAM The development of a national statistical program for marine recreational fisheries data could be used to eliminate survey effort redundancies. Such a program might consist of a federal agency, regional and state offices, and an independent research group (IRG). The first two of these components already exist, and their specific responsibilities could be delineated to minimize duplication. Briefly, the federal agency provides a nationally consistent statistical system, while the regional offices offer on-the-ground expertise and implementation capability. The third program component, which would be new, is necessary to ensure sufficient staffing and other resources to implement changes, to develop state-of-the-art design and estimation systems, and to anticipate and adapt to evolving challenges. It is unlikely that a federal agency with so many other responsibilities would be able to deal with these issues as well. Since methods should not be reinvented region by region, regional offices also should not have this responsibility. Independent Research Group An IRG with recognized expertise in theory and methods of survey design; survey operations; human dimensions; and statistical estimation techniques, including modern survey regression techniques, time series analysis, and small area estimation, could revolutionize recreational fisheries data extrapolation and understanding. The IRG should have a proven record of research publication in the peer-reviewed literature, and ideally, the IRG would have ready access to additional expertise in fields like remote sensing, landscape ecology, cognitive science, computer science, and economics. Such a structure would provide the flexibility to anticipate and adapt to evolving challenges. Though a small number of private research organizations might meet these constraints, the IRG most likely would be a unit within a research university or consortium of
PROGRAM MANAGEMENT AND SUPPORT 109 universities. Such academic environments tend to produce research that is innovative and of high quality because much of it is ultimately sub- jected to rigorous peer review. Additionally, involvement of a research group outside the federal agency should help to increase stakeholder confidence in the statistical system. This IRG would not be responsible for data collection or dissem- ination and may or may not be responsible for online quality assurance and production of estimates, but it should be in a position to interact closely with other program components in these operations. This interaction ensures that methods developed by the IRG are operationally feasible and modifiable in real time, if necessary. Other responsibilities of the IRG might include some or all of the following: · Collaboration with the other program components on the establishment of nationally consistent standards for design of marine recreational fisheries surveys and for data and metadata · Production of detailed sampling designs and data collection protocols · Thorough evaluation of existing sampling designs, data collection protocols, and estimation techniques · Technical documentation of all design, edit, imputation, and estimation procedures and assistance to other program components in explaining these methods to stakeholders · Directed research on design and analysis to address specific data needs and known challenges, including continuing evaluation of developments in survey theory and methods that may have application to recreational fisheries · Basic research on new and innovative approaches to marine recreational fisheries surveys and to the analysis of such survey data · Outreach to the scientific community through conference presentations and peer-reviewed publications and to regional offices on best statistical practices for marine recreational fisheries A model for such an IRG in a university setting is the statistical unit supported by the large-scale, long-term cooperative agreement between the U.S. Department of Agriculture's Natural Resources Conservation Service (NRCS) and the Iowa State University Statistical Laboratory. NRCS and the statistical unit cooperate in producing the National
110 REVIEW OF RECREATIONAL FISHERIES SURVEY METHODS Resources Inventory (NRI), which is a stratified two-stage area sample that provides information to support agricultural and environmental policy development and program implementation (Nusser and Goebel, 1997; Breidt, 2002). In its current form, NRI is an annual rotating panel survey designed to assess conditions and trends for soil, water, and related natural resources on nonfederal lands of the United States (Breidt and Fuller, 1999). The survey has evolved over a period of several decades, and before adopting its current annual form, NRI was conducted every five years, assessing conditions on each of approximately 800,000 sample points throughout the United States (U.S. Department of Agriculture, 1999). Sample points are located in all counties and parishes of the 50 states and in Puerto Rico, the Virgin Islands, the District of Columbia, and selected portions of the Pacific Basin (Breidt, 2002). In addition to providing monitoring information, NRI is used as a frame for special studies conducted outside the normal sample rotation (e.g., wetland changes, soil quality). The Iowa State University Statistical Laboratory provides technical expertise on design and estimation for NRI. Specialized methodologies have been developed at Iowa State for activities, such as editing, imputation, small area estimation, and variance estimation. The relation- ship between Iowa State and NRCS goes back many years, with contracts renegotiated annually over most of that time. Most recently, they moved to a five-year contract obtained in a competitive bidding process. The current five-year contract is about $10 million and is for design and estimation with specific products, including an imputed dataset with appropriate weights. The funding does cover design of data collection protocols, including extensive interaction with three federal data collection centers, but does not cover actual data collection. The contract supports roughly 1.5 faculty, 12 professional and scientific staff (e.g., programmers, geographic information system [GIS] specialists, database managers), 5 graduate research assistants, and parts of several admin- istrative staff (e.g., accountants, secretaries). Four NRCS employees are located at Iowa State to allow them to work with Iowa State staff and participate in NRI design and estimation development, but these NRCS employees are not supported by the contract. Because the contract is in an academic department at a public research university, there is an expectation that theoretical and methodological research arising from the contract is broadly disseminated through scientific conferences and peer- reviewed publications. Some of the contract funding is used explicitly to support this dissemination.
PROGRAM MANAGEMENT AND SUPPORT 111 Two Science and Technology to Achieve Results (STAR) research assistance agreements, which are awarded by the Environmental Protection Agency (EPA) to faculty members in the Departments of Statistics at Colorado State University and Oregon State University, provide another model for an IRG in a university consortium that sup- ports an official statistical program. Additional key personnel are at other agencies and universities. The two agreements focus on statistical issues arising in the monitoring of aquatic resources, including lakes, streams, and estuaries, with particular emphasis on the EPA's Environmental Monitoring and Assessment Program and related programs. The agree- ment primarily funded at Colorado State University is the Space-Time Aquatic Resources Modeling and Analysis Program (STARMAP) (Colorado State University, 2005), and the agreement primarily funded at Oregon State University is Designs and Models for Aquatic Resource Surveys (DAMARS) (Oregon State University, 2005). STARMAP is somewhat more focused on estimation issues and DAMARS more on design issues, but there is considerable overlap in the research programs, particularly in small area estimation. Unlike the Iowa State/NRI agreement, these STAR agreements do not have specific deliverables like a weighted dataset. Instead, the agreements primarily are aimed at pure methodological research. How- ever, an explicit component of the agreements is outreach to federal, state, and tribal agencies responsible for aquatic resources monitoring. This outreach involves helping these agencies to implement the new statistical methodologies developed under the STAR agreements. There is considerable interaction with EPA scientists, primarily through short- term visits. Each of the agreements is a $3 million, four-year contract supporting faculty, graduate research assistants, and post-doctoral research associates. The two STAR agreements are tied to one another through subcontracts, joint annual meetings, and a common scientific advisory committee. Federal Agency The role of a federal agency in a national statistical program can vary from complete responsibility to coordination and oversight. Due to the complexities of data collection, processing, and dissemination and to the different scales of observed data, it is unlikely that a federal agency could oversee and meet the needs of all parties adequately. A federal agency is, however, ideally situated for ensuring a nationally consistent
112 REVIEW OF RECREATIONAL FISHERIES SURVEY METHODS statistical system and for setting standards to be coordinated among the different offices. Specific responsibilities of the federal agency might include some or all of the following: · Management of the overall program to ensure proper collabor- ation between the IRG and regional and state offices · Coordination with states or regional offices, including regional fishery management councils, on problems of mutual interest, such as creation and maintenance of a national list frame of marine anglers for efficient sampling and outreach programs · Collaboration with the other program components on establish- ment of nationally consistent standards for design of marine recreational fisheries surveys and for data and metadata (possibly including training of access site data collectors to ensure national consistency) · Cooperation with other program components on quality assur- ance and quality control for collected data · Cooperation with the IRG in the processing of data and produc- tion of estimates · Maintenance of a central data warehouse for marine recreational fisheries and development of appropriate dissemination tools · Outreach to national-level stakeholders, including organizing data product reviews by user groups, providing information to national media, and providing education to policy makers · Solicitation of feedback on data needs from user groups, such as scientists, policy makers, and managers · Collection of information from scientists, managers, and data collectors on evolving challenges in marine recreational fisheries surveys · Communication of data needs and evolving challenges to the IRG · Sponsorship of directed research on design and analysis to address specific data needs and known challenges · Sponsorship of basic research on new and innovative approaches to marine recreational fisheries surveys and to the analysis of such survey data One final responsibility that may fall to the federal agency is solici- tation of formal scientific reviews of the entire statistical program for marine recreational fisheries. Such reviews could be done on an ad hoc
PROGRAM MANAGEMENT AND SUPPORT 113 basis possibly through standing committees. Models for such committees include those maintained by the American Statistical Association (ASA). These standing advisory committees meet periodically to advise federal statistical agencies (American Statistical Association, 2006a). Current examples include the Census Advisory Committee, the Committee on Law and Justice Statistics (advisory panel for the Bureau of Justice Statistics, among other responsibilities), and the Committee on Energy Statistics (advisory panel for the Energy Information Administration [EIA]). The latter committee is a good example of an ASA advisory committee. EIA is an independent statistical agency within the U.S. Department of Energy. The ASA Committee on Energy Statistics is supported by a line item in EIA's annual budget. This support is admin- istered by ASA. The committee meets twice a year and advises EIA on a broad range of issues, not only on data gathering, data quality, and modeling, but also on strategic planning and stewardship of EIA's reputation in the scientific community. At each meeting, EIA updates the committee on its use of the committee's advice. The committee consists of a chair plus 12 members, including a balance of mathematical statisticians, survey methodologists, economists, energy modelers, and policy analysts. The committee's composition is dictated by the broad range of issues on which EIA needs advice: data collection, data presen- tation, energy modeling, economics, and policy analysis. Committee members serve rotating three-year terms with an option to renew for a second three-year term at the discretion of ASA and EIA. Regional and State Offices As opposed to the previous duties, maintaining access site list frames for catch per unit effort estimation would be accomplished at the regional or state level. Since local expertise on topics like geography, fishing modes, and species variation is essential, data collection is best done at a local level. Additional benefits to having state or local personnel conduct the intercept surveys include better training, the formation of an interviewerangler relationship, and a larger sample size. These benefits would come with higher costs, but survey reliability, data quality, analysis, and credibility would increase. Even with outstanding local expertise, access site list frames have a number of potentially serious deficiencies as outlined in earlier chapters and should, where possible, be supplemented with area samples or other dual-frame techniques to get at
114 REVIEW OF RECREATIONAL FISHERIES SURVEY METHODS catch per unit effort for anglers not accessing the water from listed public access points. The IRG could assist in designing appropriate supple- mentary samples. Other responsibilities of the regional offices might include some or all of the following: · Coordination with the federal agency on problems of mutual interest, such as creation and maintenance of a national list frame of marine anglers · Collaboration with the other program components on the establishment of nationally consistent standards for design of marine recreational fisheries surveys and for data and metadata · Collection of data, primarily from angler intercepts at access sites, which ideally would be done with a dedicated staff of state employees to ensure quality and continuity in the data · Cooperation with other program components on quality assur- ance and quality control for collected data · Communication of data needs and evolving challenges to a federal agency and the IRG · Outreach to local user groups, including communication with individual anglers at access sites INTERIM SOURCES OF PROGRAM SUPPORT Establishment of the outlined three-component statistical system for marine recreational fisheries will take time and resources. In the interim, other sources of much-needed technical support should be considered. Many federal agencies directly support relevant statistical research on a short-term basis (one to three years) through contracts and grants, such as EPA STAR grants, Research Joint Venture Agreements from the U.S. Forest Service, and cooperative agreements with the U.S. Census Bureau or the U.S. Bureau of Labor Statistics. These may be based on broadly advertised requests for proposals or may be directed by the agency to a particular researcher or group. Another mechanism by which federal agencies support academic researchers doing relevant research on survey methods is through the Methodology, Measurement, and Statistics Program (MMS) in the Division for Social, Behavioral, and Economic Research of the National Science Foundation (NSF). MMS works in collaboration with a consortium of federal statistical agencies represented by the Interagency
PROGRAM MANAGEMENT AND SUPPORT 115 Council on Statistical Policy and the Federal Committee on Statistical Methodology. (The National Oceanic and Atmospheric Administration is considered a federal statistical agency and is a member of the Federal Committee on Statistical Methodology.) MMS can fund relevant research in two ways: (1) directly through a call for proposals on topics of interest to the federal agencies or (2) indirectly through scanning proposals to other NSF programs (notably pertaining to statistics and probability in the Division of Mathematical Sciences) and jointly funding those of interest to the federal agencies. According to a recent call for proposals, MMS invited research proposals ...that further the development of new and innovative approaches to surveys and to the analysis of survey data... Although proposals submitted in response to this solicitation may address any aspect of survey methodology, priority will be given to basic research proposals that are interdisciplinary in nature, have broad implications for the field in general, and have the greatest potential for creating fundamental knowledge of value to the Federal Statistical System. Because methodological problems often require knowledge and expertise from multiple disciplines, collaborations are especially encouraged among the relevant sciences, including the social sciences, linguistics, cognitive science, statistics, computer science, and economics. (National Science Foundation, 2005) Such a call could be directly relevant to the development of survey methods for marine recreational fisheries. One other useful means of providing interim technical support to the statistical program is to bring short-term academic visitors into the federal agency through a formal fellowship program. ASA joins with NSF and various federal statistical agencies in supporting the ASA/NSF/Federal Statistics Fellowship program (American Statistical Association, 2006b). The program brings academic researchers into the federal agencies for up to one year for the purpose of fostering collaborative research efforts on methodology relevant to the agencies. The roles of ASA and NSF in the fellowship program include advertising the fellowships to the relevant audience, forming the committee that reviews the applicants, and lending prestige to the fellowship awards.
116 REVIEW OF RECREATIONAL FISHERIES SURVEY METHODS Agencies that have participated in the ASA and NSF fellowship program have included the Bureau of Labor Statistics, the Census Bureau, the Bureau of Economic Analysis, and the National Center for Education Statistics. Similar research programs have been jointly sponsored (without NSF) by ASA and the National Center for Health Statistics, the National Agricultural Statistics Service, and EIA. CONCLUSIONS AND RECOMMENDATIONS A large number of complex technical issues associated with sur- veys of marine recreational fishing remain unsolved, and a sig- nificant investment in intellectual and technical expertise therefore is needed. Research is required to determine how best to reduce biases and assumptions and to improve the accuracy of data collection. To address these needs additional fulltime statisticians should be hired by the MRFSS. A research group of statisticians should design new anal- yses based on current developments in sampling theory (as outlined in Chapter 3). These examinations should include experimentation, such as specific sampling of activities like night fishing or fishing from private property, whose current underrepresentation in the MRFSS sampling has the potential to create bias. A greater degree of coordination between federal, state, and oth- er survey programs is necessary to achieve the national perspective on marine recreational fisheries that is needed. The committee recommends the development of a national statistical program that might consist of three components: a federal agency, regional and state offices, and an IRG. A permanent and independent research group should be established and funded to continuously evaluate the statistical design and adequacy of recreational fishery surveys and to guide necessary modifications or new initiatives. Human dimensions expertise should be included as well. The recommended changes to the design and operation of the MRFSS and its continued development and operation will require additional funding above current levels. In the interim, the committee recommends that other sources of funds be considered for the technical assistance that is needed immediately for the MRFSS.