Description of National Science Foundation’s Current Process

The following sections provide a description of NSF’s current process for its large research facilities from the time an idea originates through concept development, priority-setting, implementation, and oversight of its construction and operation.

ORIGINS OF CONCEPT AND DEVELOPMENT OF PROPOSALS1

The origins of large facility projects are as varied as the projects themselves. Some arise as logical outgrowths of previous research or facilities. Others originate as a consequence of new scientific development when the need for a new facility becomes apparent where no such need existed before. For example, high-speed networks and computers enable data acquisition and processing over widely dispersed geographic areas, creating the need for new large integration facilities.

The impetus for all new large facility projects originates in the scientific community, but ideas take various routes to fruition. The community processes vary greatly from field to field. Often, self-organizing groups within a field of science or engineering develop the initial ideas for a new facility and set scientific objectives for the facility by ranking

1  

To ensure that this description of NSF’s current process was accurate, given the evolving nature of NSF’s priority-setting procedures for the MREFC account, the committee decided to send the draft text to NSF for review. NSF reviewed, revised, and approved the portions of the text that reference this footnote.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 11
Setting Priorities for Large Research Facility Projects Supported by the National Science Foundation Description of National Science Foundation’s Current Process The following sections provide a description of NSF’s current process for its large research facilities from the time an idea originates through concept development, priority-setting, implementation, and oversight of its construction and operation. ORIGINS OF CONCEPT AND DEVELOPMENT OF PROPOSALS1 The origins of large facility projects are as varied as the projects themselves. Some arise as logical outgrowths of previous research or facilities. Others originate as a consequence of new scientific development when the need for a new facility becomes apparent where no such need existed before. For example, high-speed networks and computers enable data acquisition and processing over widely dispersed geographic areas, creating the need for new large integration facilities. The impetus for all new large facility projects originates in the scientific community, but ideas take various routes to fruition. The community processes vary greatly from field to field. Often, self-organizing groups within a field of science or engineering develop the initial ideas for a new facility and set scientific objectives for the facility by ranking 1   To ensure that this description of NSF’s current process was accurate, given the evolving nature of NSF’s priority-setting procedures for the MREFC account, the committee decided to send the draft text to NSF for review. NSF reviewed, revised, and approved the portions of the text that reference this footnote.

OCR for page 11
Setting Priorities for Large Research Facility Projects Supported by the National Science Foundation competing needs. At other times, facilities have been proposed at the initiative of an individual scientist or a small group of researchers with a bold vision. NSF program officers and staff foster these initiatives by providing funds for meetings and workshops that facilitate the scientific community’s internal evaluation and maturation of these concepts. In every case, the mission of NSF is to seek out the best ideas and the best scientists and to empower their investigations. This process of nurturing and maturation of a concept for a facility can take many years to develop fully or it can come together as a funded proposal quite quickly, depending on the nature of the proposal, the immediacy of the scientific need, and the potential payoffs scientifically and for society in general. NSF’s role in this process is reactive and responsive to the scientific community, rather than prescriptive, and this ensures that the highest quality proposals, as determined by peer review within the scientific community, are brought forward for implementation. NSF program officers are the key people who make the requirements for approval of such projects clear to the community. In identifying new facility construction projects, the science and engineering community, in consultation with NSF, develops ideas, considers alternatives, explores partnerships, and develops cost and timeline estimates. By the time a proposal is submitted to NSF, those issues have been thoroughly examined. ESTABLISHING PRIORITIES FOR LARGE FACILITY PROJECTS2 On receipt by NSF, large facility proposals are first subjected to rigorous external peer review that focuses on the criteria of intellectual merit and broad (probable) impacts. Only the highest rated proposals—those rated outstanding on both criteria—survive this process. These are recommended for further review by an MREFC panel that comprises the NSF assistant directors and office heads, who serve as stewards for their fields and are chosen for their breadth of understanding, and is chaired by the NSF deputy director acting in consultation with the director and later for review by the NSB. Both the MREFC panel and the NSB look for a consistent set of attributes in each project that they recommend: The project represents an exceptional opportunity to enable frontier research and education. 2   See footnote 1.

OCR for page 11
Setting Priorities for Large Research Facility Projects Supported by the National Science Foundation The impact on a particular field of research is expected to be transformational. The relevant research community places a high priority on the project. The resulting facility will be accessible to an appropriately broad user community. Partnership possibilities for development and operation are fully exploited. The project is technically feasible, and potential risks are thoroughly addressed. There is a high state of readiness—with respect to engineering cost effectiveness, interagency and international partnerships, and management—to proceed with development. The MREFC review panel evaluates the merit of a proposed project and then ranks it against other projects under consideration. It first selects the new projects that it will recommend to the director for future NSF support on the basis of a discussion of the merits of the science in the context of all sciences that NSF supports. Projects that are not highly rated according to the above criteria are returned to the initiating directorates and may be reconsidered later. Highly rated projects are placed in priority order by the panel in consultation with the NSF director. The review panel and the director emphasize the following criteria to determine the priority order of the projects: How “transformative” is the project? Will it change how research is conducted or alter fundamental science and engineering concepts or research frontiers? How great are the benefits of the project? How many researchers, educators, and students will it enable? Does it broadly serve many disciplines? How pressing is the need? Is there a window of opportunity? Are there interagency and international commitments that must be met? Those criteria are not assigned relative weights, because each project has its own unique attributes and circumstances. For example, timeliness may be crucial for one project and relatively unimportant for another. And the director must weigh the impact of a proposed facility on the balance between scientific fields, the importance of the project with respect to national priorities, and possible societal benefits.

OCR for page 11
Setting Priorities for Large Research Facility Projects Supported by the National Science Foundation NSF DIRECTOR AND NATIONAL SCIENCE BOARD3 Using the recommendations received from the MREFC panel, the NSF director selects candidate projects to be considered by the NSB during one of its meetings. According to the Guidelines, the director uses the following criteria in making this selection: Strength and substance of the information provided to the MREFC panel. The relationship to NSF goals and priorities, including NSF’s educational mission. Appropriate balance among various fields, disciplines, and directorates on the basis of consideration of needs and opportunities. Guidance from the NSB on overall decision boundaries for the MREFC account provided at the annual MREFC planning discussion (May). Opportunities to leverage NSF funds. The NSB’s Committee on Program and Plans (CPP) takes the lead in reviewing a proposed project; a member of the committee leads the discussion. The CPP uses these criteria: Need for such a facility. Research that will be enabled. Readiness of plans for construction and operation. Construction budget estimates. Operations budget estimates. After the CPP reviews the project, it makes recommendations to NSB for approving its inclusion in future budget requests and for approving project implementation. NSF DIRECTOR AND OFFICE OF MANAGEMENT AND BUDGET Once the NSB has approved a project for funding, the director may recommend the project for inclusion in a future budget request to the Office of Management and Budget (OMB). In August of each year, the director presents the priorities, including a discussion of the rationale for the priority order, to the NSB as part of the budget process. The NSB reviews the list and either approves or argues the order of priority. 3   The text of this section has been reviewed and approved by the NSB.

OCR for page 11
Setting Priorities for Large Research Facility Projects Supported by the National Science Foundation As part of its budget submission, NSF presents the rank-ordered list of projects to OMB. For projects included in the budget request, a capital asset plan and justification must be prepared; it follows a format developed by OMB. The capital asset plan summarizes how much the project will cost to build and operate, information on its management and cost, its schedule, and performance goals and milestones. The list of major projects in the budget may be modified during negotiations between OMB and NSF. During that process, other parts of the executive branch, such as the White House Office of Science and Technology Policy, may provide input on the projects included in the budget. Finally, NSF submits a priority list of projects to Congress as part of its budget submission. CONGRESSIONAL ACTION After submission of the president’s budget to Congress in February, congressional subcommittees and committees examine the proposed expenditures and begin the appropriations process. Congressional appropriators make decisions about whether to fund each of the large facility projects proposed for NSF in the president’s budget. In addition, because of budgetary constraints, the NSF director and OMB may decide not to request funds for large facility projects that the NSB has approved for inclusion in the budget. By 2001, the NSB had approved six large facility projects that had not yet been funded. Concerns were expressed in Congress and elsewhere that political pressures rather than scientific merit would increasingly determine which projects received appropriations. In 2001, Congress asked NSF to rank the six projects in order of priority. NSF responded by dividing the projects into two categories of three projects each; there was no ordering within a category. In its appropriations for FY 2003, Congress provided funds for two of the three projects in the high-priority category. In the 2004 budget request, NSF further ranked the projects, requesting funding for the remaining high-priority project in that fiscal year and proposing to start funding for the other three in FY 2005 and FY 2006. PROJECT IMPLEMENTATION AND OVERSIGHT4 Except for its facilities in the Antarctic, NSF does not directly operate research facilities. Rather, it makes awards to other organizations—such as universities, consortia of universities, and nonprofit organizations—to 4   See footnote 1.

OCR for page 11
Setting Priorities for Large Research Facility Projects Supported by the National Science Foundation construct, operate, and manage the facilities. NSF enters into partnerships with those organizations, whose details are most often defined through cooperative agreements, to accomplish that. A cooperative agreement defines the scope of work to be undertaken by an awardee and establishes the project-specific terms and conditions by which NSF will maintain oversight of the project. NSF has the final responsibility for oversight of the development, management, and performance of a facility. Each large facility project supported by NSF has a program manager in NSF who is the primary person responsible for all aspects of project oversight and management of the project within the foundation. The program manager carries out these responsibilities in accordance with an internal management plan (IMP) that has been crafted specifically for the project. The IMP defines a project advisory team (PAT) that consists of NSF personnel with expertise in the scientific, technical, management, and administrative issues associated with the project. The team works with the program manager to ensure the establishment of realistic cost, schedule, and performance goals for the project. The team also helps to develop terms and conditions of awards for constructing, acquiring, and operating a large facility. NSF’s deputy for large facility projects works closely with the program manager, providing expert assistance on non-scientific and nontechnical aspects of project planning, budgeting, implementation, and management to strengthen the oversight capabilities of the foundation. The deputy also facilitates the use of best management practices by fostering coordination and collaboration throughout NSF to share application of lessons learned from prior projects. The awardee designates one person to be the project director. This person has overall control and responsibility for the project within the awardee organization. Throughout the implementation stage, the awardee executes and manages the project—either construction or acquisition—in accordance with the cooperative agreement between the awardee institution and NSF. This phase of the project includes all installation, testing, commissioning, and acceptance. Oversight by NSF during this phase is accomplished through periodic reviews, written reports by the awardee to the foundation that include documentation of technical and financial status based on “earned value” reporting methods, annual work plans, periodic external reviews, and site visits. By the end of the implementation stage, a proposal is submitted to the program manager for operations and maintenance. The program manager reviews proposals in accordance with the merit-review procedures in Chapter V of NSF’s Proposal and Award Manual and presents a recommendation for funding to his or her division director and assistant director-office head. The director’s review board (DRB) reviews proposals for awards that exceed the DRB threshold (provided in Chapter VI of NSF’s

OCR for page 11
Setting Priorities for Large Research Facility Projects Supported by the National Science Foundation Proposal and Award Manual). After DRB review, the NSF director recommends awards above the NSB threshold to the NSB for approval. The NSB reviews and approves awards recommended by the director. The assistant director-office head, through the division director, then authorizes the program manager to recommend the making of an award in accordance with the proposal-processing procedures contained in Chapter VI of the Proposal and Award Manual. The program manager, with the Division of Grants and Agreements, drafts the cooperative agreement that will govern the project in accordance with the procedures contained in Chapter VIII of the Proposal and Award Manual. The Division of Grants and Agreements makes the award once the cooperative agreement is executed by it and the awardee. DEPUTY DIRECTOR, LARGE FACILITY PROJECTS, OFFICE OF BUDGET, FINANCE, AND AWARD MANAGEMENT5 As part of its Large Facility Projects Management and Oversight Plan drafted in 2001, NSF created a new position—deputy director, large facility projects—in the Office of Budget, Finance, and Award Management—to enable consistent management and oversight of all large projects, including all business and financial aspects, from their conceptualization phase through operations. The first person to serve in this capacity assumed office in June 2003. This official, who reports to the director of the Office of Budget, Finance, and Award Management (who is also NSF’s chief financial officer), is responsible for Providing expert assistance to NSF’s science and engineering staff on project planning, budgeting, implementation, and management. Developing, implementing, and managing—with NSF-wide input and concurrence—management and oversight policies, guidelines, and procedures. Ensuring shared learning of best practices by fostering coordination and collaboration throughout NSF to facilitate application of lessons learned from each project. Chairing and convening the NSF Facilities Panel, which establishes the appropriate level of management and oversight NSF will apply to each large facility project. Monitoring the business operations aspects of the facilities. Ensuring consistent representation of NSF staff on project advisory teams that advise and assist program officers in charge of large facility 5   See footnote 1.

OCR for page 11
Setting Priorities for Large Research Facility Projects Supported by the National Science Foundation projects in establishing realistic cost, schedule, and performance goals; developing the terms and conditions of cooperative agreements; overseeing projects; and providing assistance in moving projects through exceptional situations that may arise. This official is consulted on all policy matters involving facilities, including responses to inquiries made by NSF management, the NSF Office of Inspector General, the Office of Management and Budget, and Congress. The position is supported by permanent NSF staff with a mix of skills, qualifications, and extensive experience in project management, planning and budgeting, cost analysis, and oversight. NSF refers to this new position as the “deputy for large facility projects,” nomenclature that the committee respects throughout the rest of this report.