The Gulf Research Program: A Strategic Vision (2014)

5
PROGRAM OPERATIONS

Scientific Integrity

Coordination and Communication

Planning and Evaluation

Being housed within the NAS gives the Program advantages that come with being placed in a long-established institution with significant resources, established procedures and protocols, and accumulated and transferable knowledge. It allows the Program access to a great diversity of volunteers and staff expertise, as well as to existing infrastructure and support systems. The Program will take advantage of existing institutional support and expertise whenever possible, from existing financial systems to oversee and manage funds, to the National Academies Press for publication needs, to the Office of Government and Congressional Affairs as an interface with Congress and the Executive Branch of the federal government.

Scientific Integrity

A fundamental purpose of the Program is to facilitate the advancement of knowledge and the application of science to address Gulf region challenges. All activities of the Program will be conducted to meet the highest standards of scientific integrity. All staff, volunteer advisers, fellows, and grantees have a responsibility to use the funds wisely. To ensure scientific integrity in preaward processes, the Program will develop peer review procedures modeled generally on National Science Board (2005) evaluation protocols. To avoid conflicts of interest in the selection process, independent reviews will be performed by scientific peers not affiliated with institutions who propose projects. The Program peer review process and criteria for selection will be detailed in each request for applications and on the Program’s website.

To continue the emphasis on scientific integrity throughout the award period, the Program will ask all researchers, trainees, and fellows to comply with professional standards as defined by the NAS report On Being A Scientist: A Guide to Responsible Conduct in Research (IOM et al., 2009). Dissemination of results through professional and public channels will be strongly encouraged. To promote data verification and reproducibility, and to maximize the value of the data generated by encouraging syntheses and reuse of data, data generated by the Program’s grantees should be made available in a timely manner and to the greatest extent possible, subject to any institutional review board or legal restrictions. Procedures will be described on the Program website and in awardee agreements. The Program will abide by the NAS’s institutional conflict-of-interest policies and procedures for board members, other volunteers, and staff.

Coordination and Communication

The Program will maintain relationships with a range of federal, regional, state, nonprofit, and industry leaders and organizations to ensure that Program activities are widely known, that opportunities for leveraging impact are identified, and that, cumulatively, Program efforts are as coordinated as is feasible given the diversity of mandates. Opportunities for two-way dialog will be essential, as will engagement and transfer of knowledge among those working on similar issues in other outer continental shelf regions. Given the Program’s mission, building relationships with industry and safety-focused centers and consortia will be essential. Mechanisms such as conferences, workshops, outreach meetings, or Web-based opportunities will be used to communicate Program opportunities and allow input into Program planning over time.

As Program activities develop, communications activities also will advance. The Program will develop a communications plan that identifies how best to communicate both its processes and the products of the work funded, and will work with others who specialize in communications to engage with and serve educators and students, program managers, decision makers, and industries and citizens who will benefit from the results of the Program’s work. Successful communications require an understanding of and long-term interaction with a variety of audiences, including

• Science, engineering, and health professionals;
• Managers of related science, engineering, and health programs involved in other Gulf of Mexico work (federal, regional, state, local, industry, and nonprofit);
• Gulf State regional and state-level decision makers with health, education, and environmental or resource management responsibilities;
• The offshore oil and gas industry, including major operators, drilling contractors and service providers, industry working groups, and organizations that address safety issues;
• Federal agencies and programs with research and restoration responsibilities in the Gulf region and in other outer continental shelf areas with relevance to oil and gas production;
• Regional planning and resource management bodies and local governments, community leaders, and key Gulf State decision makers;
• Organizations working at the boundary between producers and users of scientific results;
• Relevant nongovernmental organizations and community leaders; and
• Relevant units of the National Academies devoted to energy, health, or environmental topics of interest in the Gulf region.

Planning and Evaluation

Thirty years is a significant duration for a program, especially in science. The Program is committed to using the principles of adaptive management to guide the Program’s operations and areas of focus. The Program’s Advisory Board will play a critical role in planning, oversight, evaluation, and strategic course corrections. It will guide the development of Program procedures and systems that are efficient and effective, and that are flexible so that the Program can change to address new developments in science, technology, and priorities. The Program will strive to be transparent in its evaluation processes and will continue to engage with other programs and the region’s experts and decision makers in program planning. The Program will evolve as it matures, but several principles will ensure that development of the Program remains true to the vision of the initial Advisory Group. These include emphasizing a future-oriented perspective; encouraging excellence in science; engaging stakeholders in the Gulf region and beyond; catalyzing the development of potentially transformative science and technologies; and encouraging innovation, collaboration, and education.

Role of the Advisory Board

A program of this breadth and financial impact requires thoughtful leadership. Intellectual oversight and program strategic guidance will come from an appointed Advisory Board whose volunteer members will be selected through normal Academies processes to bring expertise and independent judgment to discussions. The Advisory Board will have approximately 18 members who will be appointed to rotating, 3-year terms (with a second term possible). Advisory Board members will bring expertise reflecting the Program’s mission and goals; they will include but not

be limited to residents of the Gulf States. Participation by members of the NAS, the NAE, and the IOM will continue.

Advisory Board members will meet two to three times per year, either in full or in subgroups, and will work as needed between meetings to set strategic directions for the program and oversee implementation. Advisory Board members will be responsible for

• Providing strategic leadership and identifying emerging issues and novel activities;
• Guiding the development of priorities, themes, and activities and related calls for applications and other opportunities;
• Participating in ongoing Program planning and portfolio balance discussions;
• Serving on working groups and/or subcommittees (e.g., selection of fellows);
• Maintaining relationships with relevant stakeholders, including federal agency staff, Gulf State officials, industry, nongovernmental organizations, and citizen groups;
• Advising on selection of external reviewers;
• Contributing to the award selection process; and
• Contributing to regular Program assessments and reviews, including an annual report and 5-year reviews.

Advisory Board members will be subject to normal NAS conflict-of-interest policies. In certain circumstances, members may be expected to recuse themselves from discussions in the event of a potential conflict. Because of conflict-of-interest rules and the impact they would have on Board composition, Advisory Board members will have input into, but not make, funding decisions. Peer review and Advisory Board review will be used to evaluate and rank proposals in competitive processes; final funding decisions will be made by the NAS. As part of the required 5-year program review, the Advisory Board’s structure and function will be evaluated and adjusted as necessary to improve effectiveness.

Program Evaluation and Metrics

Evaluation will be a critical component of the Program because it contributes to promoting self-understanding, accountability, and transparency; it also allows the identification and leveraging of Program strengths. Regular evaluation, including a required 5-year review, will help identify ways to improve operations and enhance effectiveness. Finally, evaluation will allow assessment of progress toward the Program’s goals and guide the Program as it adapts to changing needs and priorities.

Evaluation will take place at different time intervals and at the grantee, activity, and Program levels. The agreements that guide the Program require at least yearly reports from grantees on project accomplishments, expenditures, and final outcomes. Required reporting at the grant level will help the Program’s staff monitor the grantees’ progress, ensure financial accountability, and track deliverables.

At the activity level (e.g., exploratory grants and fellowships), metrics for evaluating success will be defined at the outset of each activity. Having a set of defined metrics will help identify what information needs to be collected, how and when the information will be collected, and how the collected information will be analyzed. Activity operations also will be assessed at the end of each cycle, so that inefficiencies can be identified and procedures modified to enhance operations of the next cycle.

In addition, information on an activity’s inputs, outputs, and outcomes will be collected, and all of the grantees’ reports for an activity will form the basis of the activity-level evaluation. Using exploratory grants and fellowships as an illustration, examples of inputs, outputs, and expected outcomes⁹ are shown in the table below.

TABLE Illustration of Inputs, Potential Outputs, and Outcomes of Program Activities

For many activities, outputs and outcomes might need to be tracked long after projects or fellowships end to gain a complete understanding of their impact. In academia, articles depicting research results (an output) may be published after the termination of activity. An applied-research project might result in a product developed as an output, but the project outcome might be insignificant if the product does not penetrate the market and is not used. Alternatively, market uptake could be slow initially, but the product could become widely used later and have

_______________

⁹Inputs are the resources needed to make the activity happen. Outputs are the direct result of the activity. An outcome is a change that has occurred as a result of the activity output over the longer term.

a significant impact. Other outcomes might be difficult to establish. For example, linking cause and effect between basic research and its contribution to advancing knowledge that ultimately leads to a wider economic impact could be difficult because of the multiple steps involved in the application of new knowledge.

All activity-level evaluations will contribute critical information to the Program’s annual report, which will be a summary of each year’s activities, including expenditures and accomplishments. The annual reports will help the Advisory Board determine whether the mix of activities in the Program’s portfolio is appropriate and recommend changes such as introducing, terminating, expanding, or reducing activities. The annual report will be produced in February of each year to allow accurate year-end financial summaries.

The activity-level evaluations and the Program-level annual reports will serve as inputs for a 5-year program review to be conducted by a team of outside experts who will assess the Program’s progress toward Program goals and objectives. In addition to quantitative data, the evaluation team may consider engaging stakeholders to ensure that the Program’s activities and the Program as a whole have proven relevant to their needs.

Given the early stage of Program formation and the complex issues that the Program aims to address, evaluation strategies will seek to provide feedback for continuous learning and program adaptation. Based on the stakeholders’ input and the data collected by the Program, the evaluation team will address the following:

• How did the Program perform against its stated goals and objectives?
• Has the Program been responsive to changing needs?
• Did the Program’s activities complement and/or leverage other existing activities?
• Does the Program need to alter its strategy/vision to respond to changes in needs or priorities?
• What can the Program do to improve its implementation and impact in the next 5 years?

REFERENCES

Environmental Law Institute and Tulane Institute on Water Resources Law and Policy. 2014. Deepwater Horizon Restoration & Recovery Funds: How Much, Going Where, For What? White Paper. Available at http://eli-ocean.org/gulf/files/Funding-DH-Restoration-Recovery.pdf (accessed September 2, 2014).

GoMRI (Gulf of Mexico Research Initiative). 2014. Gulf of Mexico Oil Spill & Ecosystem Science Conference Report. Available at http://2014.gulfofmexicoconference.org/wp-content/uploads/2014_GulfConferenceReport.pdf (accessed September 2, 2014).

Ingram, K. T., K. Dow, L. Carter, and J. Anderson. 2013. Climate of the Southeast United States: Variability, Change, Impacts, and Vulnerability. Washington, DC: Island Press.

IOM (Institute of Medicine). 2010. Assessing the Effects of the Gulf of Mexico Oil Spill on Human Health: A Summary of the June 2010 Workshop. Washington, DC: The National Academies Press.

IOM, NAS, and NAE (Institute of Medicine, National Academy of Sciences, and National Academy of Engineering). 2009. On Being a Scientist: A Guide to Responsible Conduct in Research, 3rd Ed. Washington, DC: The National Academies Press.

IPCC (Intergovernmental Panel on Climate Change). 2013. Climate Change 2013: The Physical Science Basis. Contribution of Working Group I to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change, T. F. Stocker, D. Qin, G.-K. Plattner, M. Tignor, S. K. Allen, J. Boschung, A. Nauels, Y. Xia, V. Bex, and P. M. Midgley, eds. Cambridge, UK, and New York: Cambridge University Press.

IPCC. 2014a. Climate Change 2014: Impacts, Adaptation, and Vulnerability. Part A: Global and Sectoral Aspects. Contribution of Working Group II to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change, C. B. Field, V. R. Barros, D. J. Dokken, K. J. Mach, M. D. Mastrandrea, T. E. Bilir, M. Chatterjee, K. L. Ebi, Y. O. Estrada, R. C. Genova, B. Girma, E. S. Kissel, A. N. Levy, S. MacCracken, P. R. Mastrandrea, and L. L. White, eds. Cambridge, UK, and New York: Cambridge University Press.

IPCC. 2014b. Climate Change 2014: Impacts, Adaptation, and Vulnerability. Part B: Regional Aspects. Contribution of Working Group II to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change, V. R. Barros, C. B. Field, D. J. Dokken, M. D. Mastrandrea, K. J. Mach, T. E. Bilir, M. Chatterjee, K. L. Ebi, Y. O. Estrada, R. C. Genova, B. Girma, E. S. Kissel, A. N. Levy, S. MacCracken, P. R. Mastrandrea, and L. L. White, eds. Cambridge, UK, and New York: Cambridge University Press.

IPCC. 2014c. Climate Change 2014: Mitigation of Climate Change. Contribution of Working Group III to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change, O. Edenhofer, R. Pichs-Madruga, Y. Sokona, E. Farahani, S. Kadner, K. Seyboth, A. Adler, I. Baum, S. Brunner, P. Eickemeier, B. Kriemann, J. Savolainen, S. Schlömer, C. von Stechow, T. Zwickel, and J. C. Minx, eds. Cambridge, UK, and New York: Cambridge University Press.

LSU AgCenter (Louisiana State University AgCenter). 2012. Racial and Ethnic Groups in the Gulf of Mexico Region: A Series. Baton Rouge: Author. Available at http://www.lsuagcenter.com/en/communications/publications/Publications+Catalog/research/Community/RacialandEthnicGroupsintheGulfofMexicoRegionSeries.htm (accessed September 2, 2014).

Mabus, R. 2010. America’s Gulf Coast: A Long Term Recovery Plan After the Deepwater Horizon Oil Spill. September. Available at http://www.restorethegulf.gov/sites/default/files/documents/pdf/gulfrecovery-sep-2010.pdf (accessed September 2, 2014).

McNutt, M. K., S. Chu, J. Lubchenco, T. Hunter, G. Dreyfus, S. A. Murawski, and D. M. Kennedy. 2012. Applications of science and engineering to quantify and control the Deepwater Horizon oil spill. Proceedings of the National Academy of Sciences of the United States of America 109(50):20222–20228.

NAE and NRC (National Academy of Engineering and National Research Council). 2011. Macondo Well Deepwater Horizon Blowout: Lessons for Improving Offshore Drilling Safety. Washington, DC: The National Academies Press.

National Academy of Sciences and Royal Society. 2014. Climate Change: Evidence & Causes. Washington, DC: The National Academies Press.

National Science Board. 2005. Report of the National Science Board on the National Science Foundation’s Merit Review System. Arlington, VA: National Science Foundation. Available at http://www.nsf.gov/nsb/documents/2005/nsb05119.pdf (accessed September 2, 2014).

NRC (National Research Council). 2010. Advancing the Science of Climate Change. Washington, DC: The National Academies Press.

NRC. 2011. America’s Climate Choices. Washington, DC: The National Academies Press.

Priest, T. 2013. History of offshore oil and gas in the Gulf of Mexico. Presentation at the Gulf Research Program Advisory Group Meeting, New Orleans, July 23–25.

Sidlauskas, B., G. Ganapathy, E. Hazkani-Covo, K. P. Jenkins, H. Lapp, L. W. McCall, S. Price, R. Scherle, P. A. Spaeth, and D. M. Kidd. 2009. Linking big: The continuing promise of evolutionary synthesis. Evolution 64(4):871–880.

Sung, N. S., W. F. Crowley, Jr., M. Genel, P. Salber, L. Sandy, L. M. Sherwood, S. B. Johnson, V. Catanese, H. Tilson, K. Getz, E. L. Larson, D. Scheinberg, E. A. Reece, H. Slavkin, A. Dobs, J. Grebb, R. A. Martinez, A. Korn, and D. Rimoin. 2003. Central challenges facing the clinical research enterprise. JAMA 289(10):1278–1287.

Weiss, J. L., J. T. Overpeck, and B. Strauss. 2011. Implications of recent sea level rise science for low-elevation areas in coastal cities of the conterminous U.S.A. Climatic Change 105:635–645.