Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 61
49 Table 3.2. Summary of proposed best-value parameters and evaluation criteria. *Final Evaluation Parameter Designation Criteria Includes Remarks Cost A.0 Initial Capital Construction and Sometimes called the "bid" Cost procurement costs (also price include design costs in a DB project) Time B.0 Schedule Time to build project (also Sets contract performance include design time in a DB period project) Qualifications & P.0 Prequalification Financial and corporate Typically a routine Performance information as well as government form used for all bonding requirements contracting opportunities P.1 Past Project Project experience on past Preference is given to offerors Performance projects that are similar to with the most relevant the project at hand. Also experience might include past history of claims and litigation P.2 Key Personnel Qualifications of key Licenses, registrations, and Experience & personnel past project experience of Qualifications individuals P.3 Subcontractors' Subcontracting plan Often requires that goals for Information including small business participation by certain types utilization of firms be met P.4 Project Plans for logistics, material Often related to schedule Management management, equipment, constraints Plans traffic control, etc. P.5 Safety Record Corporate safety record and Often uses the Workers' and/or Plan plans for specific safety Compensation Insurance hazards Modifier as a metric to measure safety record Quality Q.0 Quality Typical QA/QC program May include design QC if bid Management submitted prior to award alternates or DB is used Plans Design D.0 Pro posed Design Owner allows contractor to Bid is submitted with and Alternates Alternate propose an alternate material without alternates. Owner or technology for a given makes decision as to which feature of work alternates will be accepted prior to award D.1 Technical Proposals are considered Requires that a measurable Proposal responsive if they receive a standard be developed for each Responsiveness minimum technical score evaluation criteria D.2 Environmental Plans to prevent and/or Many are required by law Considerations mitigate pollution during and/or regulation construction * Note: Best-value parameter designations have been changed to simplify the final implementation of the results of this report. DB = design build the best value on a basis of cost and time, also supported the 3.2 Proposed Best-Value Award decision to include this criterion. Algorithms and Rating Systems On the basis of Table 3.1, Table 3.2 was developed to allow the direct association of proven case study best-value evalua- The selection of a best-value award algorithm is also proj- tion criteria with the underlying best-value parameters. The ect specific. Some projects will need a more complex system owner can now apply this suite of evaluation criteria to the than others. For example, a project with little variability in the development of a best-value evaluation plan. In addition to experience of potential contractors will not benefit from an the evaluation criteria, the overall evaluation plan also con- extensive evaluation of qualifications, although an urban sists of a rating system and an award algorithm. A discussion freeway project having no schedule constraints may benefit of these two remaining best-value concepts is provided in greatly by including the schedule in the competitive bid Section 3.2. process. As an additional example, a project requiring special

OCR for page 61
50 technical expertise that is not present in the agency may ben- agency best-value award algorithm practices. These data efit from contractors proposing certain aspects of the design were drawn from the case study projects as well as from other as alternates. Each of these situations requires a different best- published information in the literature. The sample contains value award algorithm to ensure, throughout the process of 36 transportation agencies. Combining the two samples evaluation and award, the continued emphasis of salient furnishes a means to gain insight as to the applicability of aspects of the project that factored in its selection to be pro- existing best-value award algorithms to highway construc- cured using best value. Seven best-value award algorithms tion. The results of this analysis appear in Table 3.4. were identified in the first phase of this study and are shown It should be noted that the sum of the Qualitative and in Table 3.3. The table further shows the variables used in each Quantitative Cost-Technical Tradeoff (53%) is highest in the algorithm and the method used to determine the award. It horizontal case study project sample because of the large per- should be noted that five of the seven award algorithms entail centage (61%) of Federal projects that are in the population. point scoring or some mathematical combination of price Cost-Technical Tradeoff is mandated by federal regulation if and non-price scores. a low-bid award is not used. Three of the six low-bid case These algorithms were all drawn from the case study study projects were federal low-bid best-value projects. Thus, projects identified in Phase 1 of this research. Many of even though it is the most prevalent in the case study popu- the case study projects were not highway projects, because lation, the reader should not interpret that statistic to mean the research team was committed to looking for possible that it is the best algorithm for all horizontal projects. Look- best-value procurement solutions across both highway and ing at the number of transportation agencies that use the var- building construction industries. If the vertical projects are ious types of algorithms, if the two types of cost-technical eliminated and only those algorithms that were used to tradeoff algorithms are added together, one can see that procure horizontal construction projects are included, the actual usage is almost evenly split among the algorithms. sample becomes much smaller. However, it becomes more Therefore, no clear trend seems to exist. Thus, it must be con- relevant because the case study best-value award algorithm cluded that flexibility in the selection of a best-value award results are restricted to those that may be most suitable for algorithm should be maintained. highway construction projects. Twenty-eight of the case Using this analysis as a starting point, the best-value award study projects fell into the horizontal category. Additionally, algorithms can be condensed into three basic types. It should Table 2.16 in Chapter 2 contains a summary of published be noted that each has an associated best-value evaluation Table 3.3. Summary of best-value award algorithms. Best-Value Award Algorithm Algorithm Variables Award Determination Meets Technical If T > Tmin, Award to Pmin T = Technical Score Lowest Price Criteria--Low Bid If T < Tmin, Non-Responsive P = Project Price Adjusted Bid AB = P/T AB = Adjusted Bid Numerical analysis Award ABmin using point scoring, a Adjusted Score AS = (T x EE)/P AS = Adjusted Score mathematical Award AS max EE = Engineer's combination of price Estimate and non-price factors, or a quantitative Weighted Criteria TS = W1S1 + W2S2 + ... + WiSi + W(i+1)PS TS = Total Score tradeoff analysis Award TS max Wi = Weight of Factor i Si = Score of Factor i PS = Price Score Quantitative Cost- TIncrement = [(Tj/Ti) ­ 1] x 100% T = Technical Score Technical Tradeoff PIncrement = [(Pj/Pi) ­ 1] x 100% P = Project Price If TIncrement > PIncrement, Award Proposali If TIncrement < PIncrement, Retain Proposalj for possible award and repeat with Proposalj+1 Repeat Process until TIncrement > PIncrement Fixed Price--Best Award T max, Fixed P T = Technical Score Proposal P = Project Price Qualitative Cost- Similar to above, only no quantitative analysis of Evaluation panel Qualitative tradeoff Technical Tradeoff difference. Award to proposal that has best value reaches consensus as to analysis of cost and in proposed scope. See Figure 3.3. which proposal is the technical factors best

OCR for page 61
51 Table 3.4. Case study best-value award algorithm usage. Number of Number of Horizontal Case Best-Value Transportation Agencies Study Projects Using Award Award Algorithms Using Award Algorithm Algorithm Meets Technical Criteria­Low Bid 7 19 % 6 21% Adjusted Bid 7 19 % 3 11% Adjusted Score 6 17% 1 4% Weighted Criteria 8 22 % 3 11% Quantitative Cost-Technical Tradeoff 2 6% 3 11% Fixed Price­Best Proposal 1 3% 0 0% Qualitative Cost-Technical Tradeoff 5 14% 12 4 2% Totals 36 10 0% 28 10 0% rating system that best fits the mechanics of the award algo- screening and selection tool provided in Appendix F can rithm. The results are as follows: facilitate this screening process. If the project appears to be a good candidate, capture the essential screening criteria that 1. Meets Technical Criteria--Low Bid: All non-cost criteria made it a good candidate and rank them in order of impor- are evaluated using a satisficing rating system. Direct point tance to the project. scoring may be used to determine if the technical proposal 2. Develop qualifications and technical evaluation criteria meets the minimum technical score. Those proposals found based on the screening criteria. For each evaluation criteria, to be fully responsive make up the "competitive range"(FAR the owner must develop a measurable standard against term that fits this case). The bids are then opened, and the which responsiveness will be measured. project is awarded to the lowest price proposal. 3. Publish the best-value solicitation. The solicitation will 2. Cost-Technical Tradeoff (Qualitative): All non-cost crite- contain the following items as a minimum: ria are evaluated using either an adjectival or modified sat- a. Scope of work, plans, and specifications isficing rating system. Those proposals found to have no b. Bid form fatal deficiencies make up the competitive range, and then c. Contract completion date or days the bids are opened and the project is awarded to the best d. Best-value evaluation plan listing the evaluation criteria value, without any mathematical manipulation or combi- with corresponding standards nation of price and non-price factors. e. Description of what constitutes a non-responsive 3. Value Unit Price ($/technical point): All non-cost crite- proposal ria are evaluated using a direct point scoring system. Those 4. Receive best-value proposals and sealed bids. proposals found to have no fatal deficiencies make up the 5. Evaluate best-value proposals against published standards competitive range. The bids are then opened, and the proj- and determine which proposals are fully responsive in ect is awarded to the best value using a mathematical meeting the technical and qualifications criteria. manipulation or combination of both price and non-price 6. Return the sealed bids to the authors of non-responsive factors, a tradeoff analysis, or points. This category would proposals. include adjusted bid, adjusted score, quantitative cost- 7. Open the bids of those competitors that remain in the technical tradeoff, weighted criteria, and fixed price--best competitive range. proposal (technical score only). The use of this algorithm 8. Award to the lowest bid from within the competitive range. permits the owner to put a dollar value on a point of score, creating a "best-value unit price." It is important in this award algorithm to limit the number of qualification and technical criteria to those from categories that carried high importance in the project's best-value screen- Meets Technical Criteria--Low Bid ing. The evaluation plan should be written to be completely transparent to members of industry. To avoid the possibility of To implement this best-value award algorithm, the owner dispute or bid protest, the owner should will use the process that is illustrated graphically in Figure 3.2 with the following steps: "Clearly state the evaluation criteria and the weight assigned to each item and ensure that the evaluation team uses them. Clearly 1. Screen the candidate project and determine its potential to state the requirements of the RFP including what will be consid- accrue benefits by using best-value procurement. The project ered a non-responsive proposal." (Parvin 2000)

OCR for page 61
52 Screen Project* for BV Award BVP Responsive No Drop from to All Competition/ Return Sealed Bid No Q/TECs? Good BV Procure Using Candidate? Low Bid * Please refer to the screening and Yes selection steps presented in Appendix F. Yes Announce BV Competitive Range Develop Qualifications &/or Technical Evaluation Step 1 Criteria (Q/TEC) {Q/TECi to Q/TECn} Step 2 Open Sealed Bids for BVPs in Competitive Range Publish BV Solicitation Award to Lowest Responsive Bid in BV Proposal Competitive Range (BVP) {BVPi to BVPn} Figure 3.2. Two-step meets technical criteria--low bid best-value procurement flowchart. The goal is to have as many responsive competitors at the criteria that made it a good candidate and rank them in end of the first step as possible, thus ensuring the greatest pos- order of importance to the project. sible price competition in the second step of the procure- 2. Develop qualifications, technical, schedule, and cost eval- ment. Therefore, only evaluation criteria that will assist the uation criteria (QC, TC, SC, and CC, respectively, in Fig- owner in differentiating among the pool of potential com- ure 3.3) as appropriate based on the screening criteria. petitors should be included in the evaluation plan. For each evaluation criteria, the owner must develop a measurable standard against which responsiveness will be measured. Cost-Technical Tradeoff (Qualitative) 3. Publish the best-value RFQ. The solicitation will contain To implement this best-value award algorithm, the the following items as a minimum: owner will follow the process that is illustrated graphically a. Description of scope of work in Figure 3.3. Implementation includes the following steps: b. SOQ forms c. Contract completion date or days 1. Screen the candidate project and determine its potential to d. List of qualifications evaluation criteria with correspon- accrue benefits by using best-value procurement. The proj- ding standards ect screening and selection tool provided in Appendix F e. Description of process to be followed for the best-value can facilitate this screening process. If the project appears proposal evaluation plan to be a good candidate, capture the essential screening f. Description of what constitutes a non-responsive SOQ

OCR for page 61
53 Screen Project* for BV Award SOQ Responsive No Drop from to All QCs? Competition No Good BV Procure Using Candidate? Low Bid Yes * Please refer to the screening and selection steps presented in Yes Announce Appendix F. Competitive Range Step 1 Develop Evaluation Criteria Qual {QCi ­ QCn} Step 2 Publish BV Tech {TCi ­ TCn} Request for Sched {SCi ­ SCn} Proposals Cost {CCi ­ CCn} BV Proposal Publish BV (BVP) Request for {BVPi to BVPn} Qualification Rate BVPs Using T/S/CCs Statements of {T/S/CCi to T/S/CCn} Qualifications (SOQ) {BVPi to BVPn} {SOQi to SOQn} Yes BVP No Responsive Drop from Rate SOQs Using QCs to All Competition {QCi to QCn} T/S/CCs {SOQi to SOQn} Conduct Cost- Technical Tradeoff Analysis Award to proposal Found to Offer the Best Value Figure 3.3. Two-step cost-technical tradeoff (qualitative) best-value procurement flowchart. 4. Receive SOQs. b. Proposal forms 5. Evaluate SOQs against published standards and determine c. Contract completion date or days (if applicable) which statements are fully responsive and meet the qualifi- d. Method to carry forward Step 1 qualifications rank- cations criteria. ing/scores into final evaluation (if applicable) 6. Announce the list of prequalified firms. e. Best-value proposal evaluation plan listing the techni- 7. Publish the best-value RFPs. The solicitation will contain cal, schedule, and cost evaluation criteria with corre- the following items as a minimum: sponding standards a. Scope of work and relevant plans and specifications f. Description of what constitutes a non-responsive proposal

OCR for page 61
54 8. Evaluate proposals against published technical, schedule, 1. Screen the candidate project and determine its potential and cost standards and determine which proposals are to accrue benefits by using best-value procurement. The fully responsive in meeting the qualifications criteria. project screening and selection tool provided in Appen- 9. Eliminate any non-responsive proposals from the compet- dix F can facilitate this screening process. If the project itive range. appears to be a good candidate, capture the essential 10. Roll up evaluation results. screening criteria that made it a good candidate and rank 11. Convene selection panel and conduct qualitative cost- them in order of importance to the project. technical tradeoff analysis to identify the best proposal. 2. Develop qualifications, technical, schedule, and cost 12. Award to the firm within the competitive range offering evaluation criteria (QC, TC, SC, and CC, respectively, in the best-value proposal. Figure 3.4) as appropriate based on the screening crite- ria. For each evaluation criterion, the owner must This is the most subjective of the three best-value award develop a measurable standard against which respon- algorithms, and as a result, it will be the least popular to siveness will be measured. implement. However, numerous conversations with pro- 3. Publish the best-value RFQs. The solicitation will contain curement officials in the federal sector indicate that they have the following items as a minimum had more award protest problems with the quantitative cost- a. Description of scope of work technical tradeoff than with this more subjective approach. b. SOQ forms Feldman's Government Contract Awards treatise states as c. Contract completion date or days follows various issues associated with quantitative and qual- d. List of qualifications evaluation criteria with correspon- itative ratings in Sections 10:20 and 10:21 (Feldman 1994, ding standards footnotes omitted): e. Description of process to be followed for the best-value proposal evaluation plan The General Accounting Office (GAO) has approved the use f. Description of what constitutes a non-responsive SOQ of qualitative ratings as opposed to numerical ratings, so long as 4. Receive SOQ. they give the source selection official a clear basis for considering 5. Evaluate SOQs against published standards and deter- the merits of proposals. [§10:21] mine which statements are fully responsive and meet the Indeed, the Comptroller General has stated that such ratings `may be a more direct and meaningful method' than the numeri- qualifications criteria. cal evaluation of technical proposals, even though both evaluation 6. Announce the list of prequalified firms. approaches characteristically reflect the disparate, subjective judg- 7. Publish the best-value RFPs. The solicitation will contain ments of the evaluators. As with numerical rating systems, the the following items as a minimum: GAO has said that qualitative ratings are best used as guides to a. Scope of work and relevant plans and specifications intelligent decision making and are not generally controlling for award. [§10:21] b. Proposal forms Sometimes, both agency and industry personnel assign talis- c. Contract completion date or days (if applicable) manic importance to point scores. The Comptroller General has d. Method to carry forward Step 1 qualifications ranking/ stated repeatedly, however, that point scores are useful only as scores into final evaluation (if applicable) guides to intelligent decision making and are not generally con- e. Best-value proposal evaluation plan listing the technical, trolling for award because they reflect the subjective and sometimes schedule, and cost evaluation criteria with corresponding disparate judgments of the evaluators. [§10:20] [T]he Comptroller General (and some agency regulations) standards consistently have disapproved of agencies' establishing predeter- f. Description of what constitutes a non-responsive mined cutoff scores for deciding technical acceptability. [§10:20] proposal 8. Evaluate proposals against published technical, sched- This qualitative approach enables owners to differentiate ule, and cost standards and determine which propos- between competitors when the relative merits of each pro- als are fully responsive in meeting the qualifications posal are difficult to quantify using a point scoring system, criteria. but the project has specific technical or experiential require- 9. Eliminate any non-responsive proposals from the com- ments to be successful. petitive range. 10. Roll-up evaluation results and determine the final point score for each responsive proposal. Value Unit Price 11. Compute the $/technical point using the formula pub- To implement this best-value award algorithm, the lished in the RFP to identify the best proposal. owner will follow the process that is illustrated graphically 12. Award to the firm within the competitive range offering in Figure 3.4. Implementation includes the following steps: the lowest best-value unit price.

OCR for page 61
55 Screen Project* for BV Award SOQ Responsive No Drop from to All QCs? Competition No Good BV Procure Using Candidate? Low Bid Yes * Please refer to the screening and selection steps presented in Yes Announce Appendix F. Competitive Range Step 1 Develop Evaluation Criteria Qual {QCi ­ QCn} Step 2 Publish BV Tech {TCi ­ TCn} Request for Sched {SCi ­ SCn} Proposals Cost {CCi ­ CCn} BV Proposal Publish BV (BVP) Request for {BVPi to BVPn} Qualification Rate BVPs Using T/S/CCs Statements of {T/S/CCi to T/S/CCn} Qualifications (SOQ) {BVPi to BVPn} {SOQi to SOQn} Yes BVP No Responsive Drop from Rate SOQs Using QCs to All Competition {QCi to QCn} T/S/CCs {SOQi to SOQn} Compute BV $/Tech pt Award to Proposal with Lowest BV Unit Price Figure 3.4. Two-step value unit price best-value procurement flowchart. The value unit price algorithm assumes that the owner will integrity of the scoring process. Unfortunately, the extremely develop a specific formula that can be used to calculate the important decision regarding the numerical range of possible best-value objective decision criterion. Table 3.3 contains the points to be awarded is often made arbitrarily without regard formulae that are currently in use for the various best-value to its overall impact on the scoring system. award methods. This algorithm is extremely dependent on Owners must make sure that the cost-value of a single point of the proper implementation of a thoughtfully developed score is consistent with its actual value to the project. In a direct direct point scoring system. Once the direct point scoring point scoring system, the total number of points awarded in each system has been selected, the owner must make a number of rated category is usually determined by the weight that the owner decisions about the details of the system to ensure the allocates to that category. For example, the qualifications of the