3
Project Selection, Implementation, Monitoring, and Review

PROJECT SOLICITATION

Process

Projects for the Reservoir Class Program have been solicited using two different procedures: (1) a Program Opportunity Notice (PON/DOE) and (2) a Request for Proposal (RFP/BDM-Oklahoma). DOE handled the solicitations and project management for Classes 1, 2, and 3 using the PON process. Class 4 responsibilities were contracted to BDM-Oklahoma which utilized an RFP process. Termination of the Class 4 procurement process due to pending budget cuts for FY96 prevented an evaluation of its effectiveness. The issuance of solicitations was followed by pre-proposal conferences designed to provide information about the program and to answer questions. PONs and RFPs have undergone an evolution in terms of length, deadlines, requirements, and objectives. In all cases, proposals have been solicited for near-term and mid-term projects. Near-term projects take place within a five-year period and have the goal of preserving access to similar reservoirs that have a high potential for increased productivity but are rapidly approaching their economic limit. Preserving access to these reservoirs is to be accomplished by conducting technology transfer that motivates operators to identify bypassed oil and apply underutilized technologies to produce that oil. Mid-term projects occur within a ten-year period and have



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 35
3 Project Selection, Implementation, Monitoring, and Review PROJECT SOLICITATION Process Projects for the Reservoir Class Program have been solicited using two different procedures: (1) a Program Opportunity Notice (PON/DOE) and (2) a Request for Proposal (RFP/BDM-Oklahoma). DOE handled the solicitations and project management for Classes 1, 2, and 3 using the PON process. Class 4 responsibilities were contracted to BDM-Oklahoma which utilized an RFP process. Termination of the Class 4 procurement process due to pending budget cuts for FY96 prevented an evaluation of its effectiveness. The issuance of solicitations was followed by pre-proposal conferences designed to provide information about the program and to answer questions. PONs and RFPs have undergone an evolution in terms of length, deadlines, requirements, and objectives. In all cases, proposals have been solicited for near-term and mid-term projects. Near-term projects take place within a five-year period and have the goal of preserving access to similar reservoirs that have a high potential for increased productivity but are rapidly approaching their economic limit. Preserving access to these reservoirs is to be accomplished by conducting technology transfer that motivates operators to identify bypassed oil and apply underutilized technologies to produce that oil. Mid-term projects occur within a ten-year period and have

OCR for page 35
the goals of developing and testing advanced technologies through an integrated multidisciplinary approach. Audience Reached The primary mechanism for distributing both PONs and RFPs was a mailing list developed by DOE and targeted primarily to the technical community. In addition, notices of upcoming projects and deadlines were published in trade journals for independent oil companies. Small oil producers were also reached through trade organizations. In the panel’s judgment, the distribution of PONs and RFPs has been effective in all class projects. This is important because the deadline for submitting a proposal is generally three months after the PON or RFP is released (Table 3.1). Project Eligibility The criteria used to determine project eligibility changed from Classes 1-3 to Class 4. Eligibility requirements for the Class 4 program are broader than those for Classes 1, 2 and 3. Class 4 proposals, which still call upon reservoir characterization as a major component, opened up funding to enhanced oil recovery projects. Although this type of project was not expressly excluded from Classes 1, 2, and 3, it was not openly solicited. No maximum length was imposed on proposals for Classes 1, 2, and 3, but for Class 4, an 80-page maximum (Executive Summary—5 pages; Technical Proposal—75 pages) was applied. This is a definite improvement; in fact, the panel feels that a 25-page proposal length is more appropriate. The level of financial detail required in proposals has been a frequently cited criticism of the selection process. Proposals for Classes 1, 2, and 3 TABLE 3.1 Proposal solicitation and selection data Project Date of PON/RFP No. Mailed Deadline for Proposals Near-term Proposals Received/Awarded Mid-term Proposals Received/Awarded Class 1 15 Oct 1991 1,850 15 Jan 1992 23/10 12/4 Class 2 8 Oct 1992 2,500 15 Jan 1993 27/8 17/3 Class 3 28 Feb 1994 900 8 Jun 1994 10/4 21/5 Class 4 28 Apr 1995 578 NA NA NA NOTE: Class 1 and Class 2 solicitations were open 90 days for proposal preparation and Class 3 was open 98 days. One Class 2 proposal was withdrawn after it was submitted. Class 4 was canceled by DOE in August 1995, in response to preliminary actions taken by Congress. NA = not applicable. Source: Department of Energy.

OCR for page 35
required detailed financial summaries during the negotiation phase, whereas Class 4 proposals were to require fixed cost contracts and costs determined using price-analysis techniques. All proposals require one or more teaming agreements in which subcontractors outline the terms of their partnership with the main contractor. Project Selection The project selection method for Classes 1, 2, and 3 was similar to the Source Evaluation Board model developed for clean coal projects with funding in excess of $25,000. Principal concerns in that program are in the areas of technical evaluation, confidentiality, and conflict of interest. Fairness has to be assured and demonstrable because authors of unsuccessful proposals typically challenge the evaluations. Reservoir Class Program evaluation teams have been variously composed of geologists, petroleum engineers, environmental specialists, chemists, geophysicists, and policy analysts. Evaluation teams for Class 1 and 2 proposals were made up of DOE, U.S. Geological Survey, and Minerals Management Service personnel. Class 3 evaluation teams were composed of DOE and BDM-Oklahoma personnel. Each team reported to a Source Evaluation Board chairperson who was the contact person for nongovernmental employees. Each proposal was reviewed in a four-step process. Step one ascertained whether the proposal met basic qualification requirements. Step two, the preliminary review, determined whether the proposal was consistent with program objectives and contained enough information to undergo comprehensive review. In step three, the comprehensive review, each proposal was scored (Excellent, Good, Fair, or Poor) in six categories: (1) technical and project management approach; (2) evidence of technical readiness; (3) technology transfer; (4) relevance of project; (5) environmental, health, and safety aspects; and, (6) funding plan, financial capability, and commitment. Scores were weighted differently in various classes and in near-term versus mid-term projects. In all cases, technical criteria received the highest weighting. Cost was considered, but not point scored. The selection process occurred at a single site over a three- to four-week period. During this time, review teams spend 100 percent of their time performing this task. Although the process is tedious, DOE feels it is effective. The fourth step in proposal selection involved a Source Selection Official who made the final selection of projects from among all proposals that were highly ranked and met program objectives. In making the final decision, the Source Selection Official also considered factors such as technical and geographic diversity, and type and size of the proposing organization. Applicants were informed in advance that these factors would be considered.

OCR for page 35
In response to concerns about the solicitation, review, selection, and contracting processes that arose during the Class 1 and 2 selection process, DOE selected BDM Oklahoma to assist in the Class 3 selection process. DOE planned to have BDM-Oklahoma handle the entire process for Class 4. Although the Class 4 program was discontinued before proposals were received, it is instructive to review some of BDM’s plans to streamline and simplify the selection process for the Class 4 proposals. BDM first organized a pre-proposal conference to clarify the RFP and answer questions from potential contractors. BDM planned to assemble an evaluation team of 10 to 12 individuals from among BDM employees. This team would have been given two months to evaluate technical, business, past performance, and cost criteria. Technical criteria would have been given the greatest weight, and proposals would have been rated as exceptional, acceptable, marginal, or unacceptable. BDM planned to use price-analysis techniques to determine standard prices for proposed expenditures in order to reduce the burden of detailed financial statements from the contractors. They anticipated firm fixed-price contracts rather than reimbursements now used by DOE. Contracting Process The length of contract negotiation has been a contentious issue for some of the Class 1, 2, and 3 projects. Figure 3.1 shows the length of time between the announcement of an award and the signing of a contract. Lengthy contract negotiations were in some cases caused by DOE and in others caused by individual contractors, but both parties were responsible for extended delays in many specific projects. The post-selection/pre-award process involves fact-finding and negotiation. During the fact-finding stage, DOE performs a comprehensive review of the proposal by functional area. Following this review, a fact-finding letter is issued to the potential contractor with a response expected in 30 days. The participant must then prepare a detailed cost estimate for the project for DOE to audit. The entire fact-finding stage generally takes several months. After the audit has been completed, DOE is able to authorize pre-award costs, allowing the project to begin. The negotiation stage involves subcontracts to third parties, joint ventures, and/or partner agreements. Depending on the complexity of the proposal, this stage can also take several months. A pre-negotiation plan, negotiation, and a post-negotiation summary are required. The form of award that follows is a Cooperative Agreement. DOE payment begins at this point, although payments can be retroactive to the date of the pre-grant authorization. For Class 4, BDM-Oklahoma planned to implement a simplified contract-

OCR for page 35
FIGURE 3.1 Contracting process: length of time from award announcement to contract signing. Source: Unpublished data provided by DOE.

OCR for page 35
ing process. Contracts were to be firm, fixed-price agreements with a minimum 50 percent cost share. Costs were to be determined using price-analysis techniques, which would have reduced the detailed cost estimates required from contractors in the past. Some of the negotiation steps were to be eliminated by having certain issues addressed at the pre-proposal conference. Award would be made without discussions whenever possible. The objective of these intended changes was to reduce the length of the contracting period to two to five months. Due to the cancellation of the Class 4 solicitation, however, the overall effect of these proposed changes is uncertain. MONITORING AND REVIEW OF PROJECTS, PROGRESS, AND RESULTS Objectives Because the projects are cost shared, it is DOE’s view that review and monitoring of the project are a joint industry-DOE responsibility. The DOE’s approach is to allow the industry, university, or state survey participant to manage the day-to-day operation of projects, whereas DOE’s main function is to monitor the project to assure that program goals are met and government funds are used properly. Process The DOE review process consists of review and approval of the quarterly progress reports, milestone reports, cost reports and invoices, and annual and final reports. Site visits are occasionally conducted, and DOE participates in project team and industrial consortia meetings. This review is conducted by the contracting officer or his/her technical representative. In addition, DOE requires the review and approval of any change in the statement of work. Three criteria must be met for the approval of a change in the contract: (1) the revision complies with the scope of work; (2) the revision contributes to the program objectives; and (3) the cost is no more than the original proposal. For each project, there is also a distinct phase-1/phase-2 decision point when DOE and the contractor meet to evaluate the status of the project before entering the field demonstration phase. To date, several projects, including American Oil Recovery/Mattoon Oil Field and Anderman/Smith Operating Company/Black Warrior Basin, have been canceled for a variety of reasons. Examples of significant changes the DOE has approved include: Lomax Exploration Co., Uinta Basin Waterflood: DOE approved the use of an alternate source for the injection water, and the delay in drilling a well until unexpected nearby well performance had been evaluated.

OCR for page 35
Fina Oil and Chemical Co., Integrated reservoir management for optimizing infill drilling: Crosswell seismic tomography has been deleted because of expense and mechanical problems, and no alternative has been found. The rest of the project is proceeding without the data from tomography. The DOE funding has been adjusted to reflect this. University of Tulsa, Application of horizontal wells to improve a waterflood performance: The drilling of a horizontal injection well has been deleted due to unacceptable economic projections and targeted recompletions are being used instead. DOE feels that the project still complies with the primary objective of comparing waterflood optimization using state-of-the-art technology and conventional technology. BDM has not formalized their progress monitoring and review process, but at this time it is anticipated to be similar to that of DOE. EVALUATION OF RESULTS OF PROJECTS BY DOE The evaluation of the technical results by DOE is performed by the contracting officer and/or the contracting officer technical representatives (COTR). There is no additional review, even within the DOE. Although this is common practice for many federal and private funding agencies, the Reservoir Class Program is significantly different from most federal programs due to the length of the project involved. For example, in some projects the length of time from proposal submission and project selection to the end of the project will exceed eight years. Due to the significant changes in technology and economic conditions that are possible over such a length of time, it is important that Reservoir Class Program projects be evaluated on a technical basis throughout their duration. QUALIFICATIONS AND CREDIBILITY OF DECISION MAKERS Most of the internal DOE/BDM reviewers appear to have the required education to effectively evaluate proposals. It is not known, however, whether they have the necessary industrial experience or are up-to-date with the current technology because a number of reviewers have not been active participants in professional society meetings and workshops that play an important role in the petroleum industry. As a result, the research and industry communities are unfamiliar with the DOE staff who reviewed the proposals and have little faith in the project selection process. Any mistakes in project selection, whether real or perceived, only work to enhance this impression. To put it simply, a number of DOE decision makers have not been an active part of the petroleum industry through participation in the normal professional society meetings and workshops. Thus, an image of directing the program from a distance is pro-

OCR for page 35
jected, despite the fact that monitoring of the projects is left to the industry. It certainly does not help that in the Reservoir Class Program, factors other than technical merits are acknowledged to be considered as heavily as the technical merits of the projects. In fact, it appears that some of the projects may have been awarded primarily on the basis of geographic location and political considerations. While this is certainly a reality of the funding process, some of these projects do not appear to meet a minimal technical standard. FINDINGS AND RECOMMENDATIONS Finding The likelihood of having the best possible set of projects is enhanced if there is a large number of proposals from a broad range of organizations. The panel feels the quantity of proposals submitted is not adequate to ensure the number of high quality projects necessary to satisfy the various goals of the Reservoir Class Program. Recommendation DOE should take measures to increase the number of proposals submitted. Two possible mechanisms are suggested: (1) open future proposal solicitations to meritorious projects from all previously funded classes, as well as the class currently being solicited; and (2) simplify the procedure used in proposal preparation. Changes in the proposal preparation procedure planned for Class 4, such as a maximum proposal length and fixed cost contracts to reduce negotiation time, should be implemented for future DOE programs. In addition, DOE should ensure that the technical documentation that accompanies any future program solicitation includes current information about depositional models of the targeted reservoir class. Finding An important feature of the Reservoir Class program is that the contract period is divided into two phases: an initial phase dominated by reservoir characterization, followed by a second phase of field demonstration if the project is deemed feasible. This provides both the contractor and DOE the option to terminate a project that is proceeding unsatisfactorily. Recommendation The panel recommends that DOE not be reticent in exercising the option to terminate a project after Phase 1, where appropriate. In particular,

OCR for page 35
the DOE should consider this option for projects in which there was a major change in the technology employed. Finding The cost-sharing arrangement adopted for the reservoir class program has resulted in the successful application of new technologies to marginal oil fields in the United States. The 50 percent contribution from DOE has resulted in innovative projects that might not have been attempted otherwise and the financial commitment from industry insures financial accountability from industry. Recommendation We recommend that DOE continue the cost-sharing aspect of the Reservoir Class Program in future programs. Finding The current anonymous internal DOE/BDM review has led to uncertainty and mistrust among applicants about the project selection process. Recommendation DOE should implement a system of external technical reviews in the proposal evaluation phase of the Reservoir Class Program. The review process should include experts from outside the federal government and the contracting companies. Finding The panel saw little evidence that the present DOE review process, both during the projects and after completion, is producing mid-course modifications that are normally called for in projects of this type. The lack of an external project review process contributes to this shortcoming Recommendation DOE should implement a system of external technical reviews for monitoring the technical status of all Reservoir Class Program projects on an annual basis. In addition, upon completion of a project, thorough technical review by external experts should be carried out prior to the distribution of project results.

OCR for page 35
This page in the original is blank.