3
The Research Framework

IMPLEMENTATION OF RESEARCH FRAMEWORK

In preparing its initial advice to the AYK SSI about developing a research and restoration plan, this committee (NRC 2005) set forth the elements such a plan should contain. Reading the draft plan prepared by the AYK SSI leads this committee to amplify that advice. We begin with a diagram (Figure 3-1) of 11 steps that should be followed to implement the AYK SSI research program, as well as a description of each step and references to pages in the STC document that apply to each step. Reorganization and shortening of the current draft plan to follow these steps, and perhaps the inclusion of a diagram that shows the research program framework, would improve clarity and reduce confusion about the proper sequence of steps to be followed.

Steps for Implementation

Step 1. Problem Formulation

The first step in any research program is to identify clearly the overarching questions relevant to the program goal. Frequently, this is one clearly stated goal with a well-defined set of carefully chosen broad, universal questions. Specific research questions are developed later (Step 5). The purpose of Step 1 is to encourage creative thinking based on concepts used to understand issues in the world of salmon. This creative thinking, or problem formulation, is a process of identifying impor-



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 11
Review of the Draft Research and Restoration Plan for Arctic-Yukon-Kuskokwim (Western Alaska) Salmon 3 The Research Framework IMPLEMENTATION OF RESEARCH FRAMEWORK In preparing its initial advice to the AYK SSI about developing a research and restoration plan, this committee (NRC 2005) set forth the elements such a plan should contain. Reading the draft plan prepared by the AYK SSI leads this committee to amplify that advice. We begin with a diagram (Figure 3-1) of 11 steps that should be followed to implement the AYK SSI research program, as well as a description of each step and references to pages in the STC document that apply to each step. Reorganization and shortening of the current draft plan to follow these steps, and perhaps the inclusion of a diagram that shows the research program framework, would improve clarity and reduce confusion about the proper sequence of steps to be followed. Steps for Implementation Step 1. Problem Formulation The first step in any research program is to identify clearly the overarching questions relevant to the program goal. Frequently, this is one clearly stated goal with a well-defined set of carefully chosen broad, universal questions. Specific research questions are developed later (Step 5). The purpose of Step 1 is to encourage creative thinking based on concepts used to understand issues in the world of salmon. This creative thinking, or problem formulation, is a process of identifying impor-

OCR for page 11
Review of the Draft Research and Restoration Plan for Arctic-Yukon-Kuskokwim (Western Alaska) Salmon FIGURE 3-1 Numbered sequence of 11 steps that should be followed to develop and implement the AYK SSI research program.

OCR for page 11
Review of the Draft Research and Restoration Plan for Arctic-Yukon-Kuskokwim (Western Alaska) Salmon tant factors that affect salmon abundance. It is helpful to identify those factors, or stressors, and the regime that they operate in. For example, if temperature is a stressor (a factor that affects salmon), then the range and duration of temperature fluctuations is the stressor regime. The stressor also might primarily affect a specific life stage. For example, if sea temperatures over the continental shelf in summer are higher than average, that might not affect adults, but it might affect the marine survival of juveniles. Information for problem formulation should come from published literature, expert opinion, local and traditional knowledge (LTK), and the experience and expertise of the researchers. The AYK Salmon Research and Restoration Plan was funded because of low salmon returns in the region. Therefore, it makes sense for problem formulation to translate into questions about how the trends and causes of variation in AYK salmon abundance and human use of salmon can be compared and contrasted with trends and causes of variation elsewhere in the world. Any number of factors at global (climate change), international (for example, chum salmon hatcheries in Japan), regional (for example, commercial fishing in the Bering Sea), or local (for example, downriver commercial fishing) scales could contribute to the decline of AYK salmon. As indicated in Figure 3-1 of this report, LTK provides important input on local trends and causes of variation at this initial step. In the draft AYK Salmon Restoration and Research Plan, this component appears in the Plan Overview (p. 7, lines 19-22), section 1.2.2 (Goal, p. 11), and section 2.2 (Statement of Research and Restoration Program Goal, p. 36). Although the program goal is stated, the plan does not seem to include any clearly stated broad questions that are universal in scope. While some description, definition, and identification of causative factors or stressors, and stressor regimes are in the plan’s introduction (sections 1.3-1.3.9), the information in these sections is not clearly linked to the problem or overarching questions. We recommend deleting these sections (1.3-1.3.9) from the introduction. As discussed in Chapter 2 of this report, we suggest replacing the sections with ones that describe important sources of variability in salmon abundance and that review and integrate current concepts about the processes responsible for that variability. We also recommend deleting the program completion year of 2012 from the statement of the program goal because a date should not be a part of problem formulation. Problem formulation will lead to the development of a conceptual model (Step 2) that describes the problem and links the stressors to the

OCR for page 11
Review of the Draft Research and Restoration Plan for Arctic-Yukon-Kuskokwim (Western Alaska) Salmon problem (in this case, the research issue). The conceptual model is an integral component of problem formulation. Step 2. Development of a Conceptual Model A conceptual model can often be expressed through a visual representation and incorporate the fundamental problem, stressors, and stressor regimes to render a holistic understanding of the problem and its components. Figure 2.1 in the draft research and restoration plan is a good example of such a visual representation. There are several ways to develop a conceptual model, including through gaining expert scientific opinion, LTK, and from workshops led by broad and creative thinkers. This is a natural point to incorporate LTK, and in the case of the AYK plan, it is essential, as discussed below. An example of such a conceptual model is the Gulf Ecosystem Monitoring (GEM) model (http://www.evostc.state.ak.us/gem/concept_large.htm). (See also NRC 2002.) In the draft AYK SSI Salmon Research and Restoration Plan, this step appears in section 1.2.5 (Elements of the Conceptual Foundation, p. 1) and section 2.3 (Conceptual Foundation, pp. 36-38, Figure 2.1). It is unclear how the conceptual foundation illustrated in Figure 2.1 is related to the fundamental problem, as well as how this foundation is matched with the priority setting criteria (p. 9, 65-66). Step 3. Identification of Relevant Approaches and General Hypotheses Specific to the Conceptual Model Once the general problem has been formulated and the conceptual foundation has been illustrated, the next step is to identify the relevant approaches and general hypotheses that are specific to explaining, and ultimately testing, the validity of the conceptual model. These general hypotheses are developed by expert scientific opinion, LTK, and workshops and offer an explanation for processes that may contribute to the problem or relevant stressors. A general hypothesis for the AYK plan, for example, might be that climate change alters the timing and extent of spring freshwater runoff and that consequently juvenile salmon survival is affected by runoff. In this case, runoff is the stressor variable. The timing and magnitude of the runoff relative to the salmon’s life stage constitute the stressor regime.

OCR for page 11
Review of the Draft Research and Restoration Plan for Arctic-Yukon-Kuskokwim (Western Alaska) Salmon Relevant approaches (monitoring, process studies, retrospective studies, modeling studies) are described in section 4.6 (Types of Study Approaches, pp. 79-82). Adding experimental manipulations, especially of management actions, to the list of relevant approaches is also recommended. Feedback occurs between Step 3 and Step 4, which then leads to the prioritization of these hypotheses. In the case of the AYK plan, the prioritization may be facilitated by the use of the nine criteria listed on pages 65 and 66. Step 4. Collection and Synthesis of Existing Data and Metadata At this stage of implementing a research plan, an evaluation already is needed of what is already known that is pertinent to the problem and what data have already been collected but perhaps not analyzed specifically to address the problem. At this step, existing data are located and identified formally; this collection of information about primary data is known as metadata. In the metadata, the type, accuracy, sampling, availability, and other characteristics of the primary information (data) are addressed. All data relevant to the hypotheses must be gathered and evaluated, regardless of institutional source. LTK should be included at this stage. These data may be obtained from extant agency programs or through grant and contract mechanisms. For example, development of metadata involves expertise, time, and expense, and will need to be funded through a grant or contract. This may involve RFPs, and, if so, should include peer review. The information in Appendix 2 of the draft plan is the start of a metadatabase on freshwater assessment project data (although no specifics are provided on when, where, and by whom the data were collected) data types (for example, paper records, computer files), species, data formats, variables, where the data are located, how the data are stored, data condition, quality, and so on. Metadata for this region are available at www.pmel.noaa.gov/np/mdb/index.html. Without adequate attention to this step, the AYK SSI could spend time and money doing research that already has been done or gathering information that has already been gathered. The only place in the plan where metadata are specifically mentioned is in sections 4.10 (Communication of Research Results, pp. 88-89), 4.10.1 (Summary of Communication Obligations, p. 90), and 4.11 (Data Management, p. 91). The information presented in Table 4.1 and Table 4.2 is a form of metadata, although it pertains only to similar themes in other research plans and

OCR for page 11
Review of the Draft Research and Restoration Plan for Arctic-Yukon-Kuskokwim (Western Alaska) Salmon does not specifically address whether or not there are existing data available and the adequacy of these data. Appendix 2 (pp. 110-113) contains a partial list of freshwater salmon assessment projects that currently are collecting data in the AYK region, although there are not enough details to evaluate data availability and adequacy (such as species, spatial and temporal coverage) with respect to Step 3. This task will require considerable investment of time and money, although it will provide dividends for research far into the future. As recommended later, the AYK SSI should take the opportunity to work with other organizations that are doing similar work (such as the North Pacific Research Board’s [NPRB’s] Alaska Marine Information System and the North Pacific Ecosystem Metadatabase). Step 5. Generation of Specific Hypotheses Based on the collection of metadata and the evaluation of extant data and knowledge in Steps 3 and 4, the next step is to develop hypotheses specific to the problem and to prioritize them. The method of prioritization can include the potential for important results, the lack of overlap with other programs, and the lack of current funding opportunities for this type of research, among other factors. This step relies on RFPs to encourage scientists to test these hypotheses, prove peer review, and obtain funding through grants and contracts. Section 4.3 of the draft plan (Setting Research Priorities, pp. 65-77) is directly applicable to this step. In the layout of the draft document, however, the origin of the specific questions outlined in Chapter 3 (pp. 50-61) is unclear, and the questions would more logically follow rather than precede the description of how these specific questions were prioritized. The gap analysis based on the review of research plans of 10 potential partner organizations plus existing state and federal research programs from the AYK region does not seem to be a thorough and well-executed approach for prioritizing specific hypotheses, especially given that Table 4.1 (pp. 71-72) includes the category “Other programs?” and Appendix 1 (pp. 103-109) contains some organizations that were not included in the gap analysis. This shortcoming in the draft plan was recognized by the AYK SSI, which described the gap analysis “obviously incomplete” (see Note to Readers, p. 72). In the present draft, peer reviewers cannot evaluate the gap analysis. Whether the current research

OCR for page 11
Review of the Draft Research and Restoration Plan for Arctic-Yukon-Kuskokwim (Western Alaska) Salmon plans of these other programs have been or will be successfully implemented, or whether they adequately address hypotheses critical to the success of the AYK SSI Salmon Research and Restoration Plan is unknown. A clear statement of specific hypotheses for the AYK plan is buried near the end of the plan in section 4.8 (Restoration Studies, p. 83, lines 10-14): “Hence the working hypothesis of the Science Plan is that the most likely causes for declines in AYK salmon populations are human harvest in fresh and salt water, mortality factors related to short- and long-term climate change such as disease, and competition from hatchery fish in the marine environment.” Moving this statement forward in the plan would improve the presentation. Step 6. Testing of Hypotheses This step would likely be accomplished through preproposals, RFPs, and external peer review of the resulting proposals (and of course, research, Step 7, and a major topic of this whole report). The plan also should include methods for documenting progress on specific proposal objectives and on data management and archiving. This step involves the consideration of conflict of interest among the investigators, of whether the proposed investigation is of sufficient scope, and of the incorporation of capacity building, among other matters. No mention is made of the use of preproposals in the draft plan. Preproposals can be useful to researchers and funders by eliminating time spent in writing and evaluating proposals that do not match program goals. Preproposals should be short enough not to impose an undue burden on their authors or on the AYK SSI. Section 4.3.4 briefly describes guidelines for RFPs and states that “in future drafts a summary of components of the RFP and an overview of the proposal review process will be provided here.” Because this section of the draft was not yet complete, the NRC committee could not review this part of the plan. A sample of a complete RFP that was provided to the NRC committee by the STC at the August 8-11 meeting in Anchorage, Alaska, meets most of our recommendations; it is thoughtful and well written. We recommend that a process similar to the one used by National Science Foundation (NSF) or National Institutes of Health (NIH) be used in preparation, review, and selection of the proposals. Consideration should be given to an NSF-type proposal format and solicitation.

OCR for page 11
Review of the Draft Research and Restoration Plan for Arctic-Yukon-Kuskokwim (Western Alaska) Salmon By the above recommendation, the committee means that RFPs should be widely disseminated to the general scientific community throughout the United States—and beyond if appropriate. The RFPs should require structured elements in proposals, such as broader impacts of the work and the inclusion of LTK, among other things. There should be an anonymous, independent peer review of the proposals, which should include a mechanism to avoid conflicts of interest. If funds permit, the establishment of a panel of outside independent experts to evaluate proposals would be helpful. There should be a ranking procedure for the results of the peer review and rules for final selection that are established and made public before the process begins. Section 4.10 (Communication of Research Results, pp. 87-90) outlines approaches for documenting progress, and section 4.11 (Data Management, p. 91) briefly describes data management and archival. Step 7. Research and Findings This step is the core of the program, and its elements are discussed throughout this report. Step 8. Synthesis of Results Because there will be results from numerous projects, it is important that they be evaluated and synthesized in a holistic manner relevant to the problem and the program goals. An important step would be the periodic evaluation of results as mentioned in section 4.4 (Program Performance Measures, pp. 77-78). However, periodic evaluation is different from synthesis. We suggest that this is an excellent opportunity to use an independent scientific review or workshop that includes non-vested experts. Based on the synthesis, new questions may be identified and new priorities set; this is indicated by a feedback loop to Step 5. Recognizing the importance of synthesis, the draft plan introduces “synthesis” as a tool that will be used to address research themes under Framework 3 (see Figure A, p. 8). As described on page 43, the purpose of Framework 3 is based on the “synthesis of knowledge acquired from activities under Frameworks 1, 2, and other research: (1) to better understand the causes of variation and resilience of the AYK salmon, and (2)

OCR for page 11
Review of the Draft Research and Restoration Plan for Arctic-Yukon-Kuskokwim (Western Alaska) Salmon to refine and develop management tools (see Figure 2.3).” In particular, Research Theme 13 (p. 57) mentions “synthesis of historical population levels of salmon and their use by humans.” Step 9. Development of Management Options or an Action Plan Based on the understanding obtained in Step 8, management options would be developed and implemented by management partners. Management plans need to be periodically reviewed for their effectiveness in solving or ameliorating the identified problems. Because the AYK plan includes restoration in addition to research, the inclusion of management and its effectiveness is an integral component of this plan; it should not be viewed as an afterthought. We recommend that the concept of adaptive management as defined in section 1.4 (p. 33) be adopted, as identified in the feedback loop from Step 8 to Step 1. While this logically follows the previous steps, management options can be identified and implemented at any time the causative factors become clear and well defined. Management applications are briefly described in section 4.7 (p. 82). This section is the first to mention that the “guiding principle for implementing the AYK SSI Research Program is that projects will be undertaken with a view toward their potential application to enhance management decisions” (from section 4.7). We recommend inclusion of this guiding principal in the introduction to the plan. Step 10. Implementation of the Management Option Implementation of fishery management plans is normally based on as much scientific information as is available, as well as on other considerations. Peer review often is involved, at least in the scientific aspects of the plans, through such mechanisms as fishery management councils. This committee judges that peer review of the science aspects of fishery management plans is an essential step and therefore recommends peer review at this stage. Because adaptive management depends on the quality of the information obtained about management actions, it is essential that management actions be designed so as to facilitate the collection of

OCR for page 11
Review of the Draft Research and Restoration Plan for Arctic-Yukon-Kuskokwim (Western Alaska) Salmon relevant information. Chapter 5 of this report discusses peer review and other management actions in more detail. Step 11. Synthesis of Management Results and Incorporation into Problem Formulation This crucial step is the heart of adaptive management. It is essential that results from management feed back into the problem-formulation and hypothesis-generation steps of the research program diagrammed in Figure 3-1. Doing so in a transparent way has the additional benefit of making the management process itself more transparent. AN EXAMPLE OF DERIVING RESEARCH QUESTIONS FROM A HYPOTHESIS Figure 2.1 of the draft research and restoration plan illustrates the conceptual model the AYK SSI used in developing its plan. Here, as an illustrative example, we derive a hypothesis from that model and use it to develop a research question, following the advice we provided in our first report (NRC 2005) and in accordance with Figure 3-1 of this report. Hypothesis. The variability across space and time in smolt survival through the transition to the ocean from freshwater is a major factor affecting the variability of salmon returns in the AYK region. Importance. The hypothesis is relevant to understanding and predicting salmon population dynamics and stock-recruit relationships, setting escapement goals, and to predicting salmon abundance. Suggested RFP. Develop a research proposal to test the hypothesis for one or more salmon populations in one or more watersheds in the AYK region. The above would be the minimal sequence, allowing the investigators to develop research questions. However, if the AYK SSI wished to go further before issuing the RFP, the following steps might be taken.

OCR for page 11
Review of the Draft Research and Restoration Plan for Arctic-Yukon-Kuskokwim (Western Alaska) Salmon Research question. How are the numbers of outmigrating smolts in (one or more specified) streams related to the number of returning adults n, n − 1, and n + 1 years later, where n is the average number of sea years for the species under consideration? Data needs. Needed are the number of smolts migrating to sea in the chosen streams and the number of adult returns in those same streams the appropriate numbers of years later. The ADF&G already collects information on adult returns for many species in many streams. Further hypotheses. These will depend on research results and will also be likely to have relevance to program goals. ADDITIONAL SUGGESTIONS FOR THE RESEARCH PLAN Following the integrated review of factors affecting salmon abundance outlined in the previous chapter, it would be worthwhile to make some decisions about the SSI’s research strategy for improving understanding of variations in salmon abundance. This strategy should be based on the overall goals of the program and the integrated review of population dynamics. Issues to consider include the following: Should SSI-funded studies focus on populations where ADF&G or other organizations already collect information on escapement and age/weight/length, or should SSI fund projects to collect these kinds of data? Should SSI studies focus on representative populations that are particularly amenable to study or particularly important to the communities involved in SSI? To what extent should SSI focus on studies that use statistical and process-based modeling efforts to understand problems of interest, and to what extent should SSI focus on studies with significant fieldwork components? To what degree could the approaches profitably be combined? How should SSI divide its attention between the marine and freshwater environments?

OCR for page 11
Review of the Draft Research and Restoration Plan for Arctic-Yukon-Kuskokwim (Western Alaska) Salmon Following this section on research strategy, it would be helpful to revisit the question of how research should be prioritized; here we focus on a specific example, the stock-recruitment relationship. To what extent should the SSI focus on statistical and modeling research to better understand the stock-recruitment relationship? For example, to what extent should the SSI focus on reducing statistical uncertainty in explaining the stock-recruit relationship? To what extent should the SSI focus on process (that is, field and laboratory) research? For example, to what degree should the SSI focus on understanding the processes responsible for generating the stock-recruitment relationship and the way in which variation in environmental conditions generates variability in this relationship? Of course, some combination of these two approaches is expected; the question is one of emphasis. This committee has the following suggestions: The SSI should focus on studies that are the most cost effective in improving statistical descriptions of variations in and process-based explanations of salmon abundance. Similarly, the SSI should focus on studies that are likely to produce useful results within a reasonably short period. Both the short initial period (until 2012) of the program and the urgency of gaining information influence this suggestion. The SSI should focus on populations for which ADF&G has a long-term stock-assessment program and work with ADF&G to supplement these studies. The SSI should provide improved statistical descriptions of relationships between the stock-recruitment relationship and environmental characteristics, and an improved understanding of the processes responsible. The SSI should focus its efforts on a small group of representative populations. The SSI research should complement work by ADF&G and others, not duplicate it, or provide substitute funding. This implies the development of coordination between SSI-funded projects and ADF&G’s ongoing programs, and other regional marine-science and salmon studies.