National Academies Press: OpenBook

Best-Value Procurement Methods for Highway Construction Projects (2006)

Chapter: Chapter 3 - Interpretation, Applications, and Recommendations for Implementation

« Previous: Chapter 2 - Findings
Page 59
Suggested Citation:"Chapter 3 - Interpretation, Applications, and Recommendations for Implementation." National Academies of Sciences, Engineering, and Medicine. 2006. Best-Value Procurement Methods for Highway Construction Projects. Washington, DC: The National Academies Press. doi: 10.17226/13982.
×
Page 59
Page 60
Suggested Citation:"Chapter 3 - Interpretation, Applications, and Recommendations for Implementation." National Academies of Sciences, Engineering, and Medicine. 2006. Best-Value Procurement Methods for Highway Construction Projects. Washington, DC: The National Academies Press. doi: 10.17226/13982.
×
Page 60
Page 61
Suggested Citation:"Chapter 3 - Interpretation, Applications, and Recommendations for Implementation." National Academies of Sciences, Engineering, and Medicine. 2006. Best-Value Procurement Methods for Highway Construction Projects. Washington, DC: The National Academies Press. doi: 10.17226/13982.
×
Page 61
Page 62
Suggested Citation:"Chapter 3 - Interpretation, Applications, and Recommendations for Implementation." National Academies of Sciences, Engineering, and Medicine. 2006. Best-Value Procurement Methods for Highway Construction Projects. Washington, DC: The National Academies Press. doi: 10.17226/13982.
×
Page 62
Page 63
Suggested Citation:"Chapter 3 - Interpretation, Applications, and Recommendations for Implementation." National Academies of Sciences, Engineering, and Medicine. 2006. Best-Value Procurement Methods for Highway Construction Projects. Washington, DC: The National Academies Press. doi: 10.17226/13982.
×
Page 63
Page 64
Suggested Citation:"Chapter 3 - Interpretation, Applications, and Recommendations for Implementation." National Academies of Sciences, Engineering, and Medicine. 2006. Best-Value Procurement Methods for Highway Construction Projects. Washington, DC: The National Academies Press. doi: 10.17226/13982.
×
Page 64
Page 65
Suggested Citation:"Chapter 3 - Interpretation, Applications, and Recommendations for Implementation." National Academies of Sciences, Engineering, and Medicine. 2006. Best-Value Procurement Methods for Highway Construction Projects. Washington, DC: The National Academies Press. doi: 10.17226/13982.
×
Page 65
Page 66
Suggested Citation:"Chapter 3 - Interpretation, Applications, and Recommendations for Implementation." National Academies of Sciences, Engineering, and Medicine. 2006. Best-Value Procurement Methods for Highway Construction Projects. Washington, DC: The National Academies Press. doi: 10.17226/13982.
×
Page 66
Page 67
Suggested Citation:"Chapter 3 - Interpretation, Applications, and Recommendations for Implementation." National Academies of Sciences, Engineering, and Medicine. 2006. Best-Value Procurement Methods for Highway Construction Projects. Washington, DC: The National Academies Press. doi: 10.17226/13982.
×
Page 67
Page 68
Suggested Citation:"Chapter 3 - Interpretation, Applications, and Recommendations for Implementation." National Academies of Sciences, Engineering, and Medicine. 2006. Best-Value Procurement Methods for Highway Construction Projects. Washington, DC: The National Academies Press. doi: 10.17226/13982.
×
Page 68
Page 69
Suggested Citation:"Chapter 3 - Interpretation, Applications, and Recommendations for Implementation." National Academies of Sciences, Engineering, and Medicine. 2006. Best-Value Procurement Methods for Highway Construction Projects. Washington, DC: The National Academies Press. doi: 10.17226/13982.
×
Page 69
Page 70
Suggested Citation:"Chapter 3 - Interpretation, Applications, and Recommendations for Implementation." National Academies of Sciences, Engineering, and Medicine. 2006. Best-Value Procurement Methods for Highway Construction Projects. Washington, DC: The National Academies Press. doi: 10.17226/13982.
×
Page 70
Page 71
Suggested Citation:"Chapter 3 - Interpretation, Applications, and Recommendations for Implementation." National Academies of Sciences, Engineering, and Medicine. 2006. Best-Value Procurement Methods for Highway Construction Projects. Washington, DC: The National Academies Press. doi: 10.17226/13982.
×
Page 71
Page 72
Suggested Citation:"Chapter 3 - Interpretation, Applications, and Recommendations for Implementation." National Academies of Sciences, Engineering, and Medicine. 2006. Best-Value Procurement Methods for Highway Construction Projects. Washington, DC: The National Academies Press. doi: 10.17226/13982.
×
Page 72
Page 73
Suggested Citation:"Chapter 3 - Interpretation, Applications, and Recommendations for Implementation." National Academies of Sciences, Engineering, and Medicine. 2006. Best-Value Procurement Methods for Highway Construction Projects. Washington, DC: The National Academies Press. doi: 10.17226/13982.
×
Page 73
Page 74
Suggested Citation:"Chapter 3 - Interpretation, Applications, and Recommendations for Implementation." National Academies of Sciences, Engineering, and Medicine. 2006. Best-Value Procurement Methods for Highway Construction Projects. Washington, DC: The National Academies Press. doi: 10.17226/13982.
×
Page 74
Page 75
Suggested Citation:"Chapter 3 - Interpretation, Applications, and Recommendations for Implementation." National Academies of Sciences, Engineering, and Medicine. 2006. Best-Value Procurement Methods for Highway Construction Projects. Washington, DC: The National Academies Press. doi: 10.17226/13982.
×
Page 75
Page 76
Suggested Citation:"Chapter 3 - Interpretation, Applications, and Recommendations for Implementation." National Academies of Sciences, Engineering, and Medicine. 2006. Best-Value Procurement Methods for Highway Construction Projects. Washington, DC: The National Academies Press. doi: 10.17226/13982.
×
Page 76
Page 77
Suggested Citation:"Chapter 3 - Interpretation, Applications, and Recommendations for Implementation." National Academies of Sciences, Engineering, and Medicine. 2006. Best-Value Procurement Methods for Highway Construction Projects. Washington, DC: The National Academies Press. doi: 10.17226/13982.
×
Page 77
Page 78
Suggested Citation:"Chapter 3 - Interpretation, Applications, and Recommendations for Implementation." National Academies of Sciences, Engineering, and Medicine. 2006. Best-Value Procurement Methods for Highway Construction Projects. Washington, DC: The National Academies Press. doi: 10.17226/13982.
×
Page 78
Page 79
Suggested Citation:"Chapter 3 - Interpretation, Applications, and Recommendations for Implementation." National Academies of Sciences, Engineering, and Medicine. 2006. Best-Value Procurement Methods for Highway Construction Projects. Washington, DC: The National Academies Press. doi: 10.17226/13982.
×
Page 79
Page 80
Suggested Citation:"Chapter 3 - Interpretation, Applications, and Recommendations for Implementation." National Academies of Sciences, Engineering, and Medicine. 2006. Best-Value Procurement Methods for Highway Construction Projects. Washington, DC: The National Academies Press. doi: 10.17226/13982.
×
Page 80
Page 81
Suggested Citation:"Chapter 3 - Interpretation, Applications, and Recommendations for Implementation." National Academies of Sciences, Engineering, and Medicine. 2006. Best-Value Procurement Methods for Highway Construction Projects. Washington, DC: The National Academies Press. doi: 10.17226/13982.
×
Page 81
Page 82
Suggested Citation:"Chapter 3 - Interpretation, Applications, and Recommendations for Implementation." National Academies of Sciences, Engineering, and Medicine. 2006. Best-Value Procurement Methods for Highway Construction Projects. Washington, DC: The National Academies Press. doi: 10.17226/13982.
×
Page 82
Page 83
Suggested Citation:"Chapter 3 - Interpretation, Applications, and Recommendations for Implementation." National Academies of Sciences, Engineering, and Medicine. 2006. Best-Value Procurement Methods for Highway Construction Projects. Washington, DC: The National Academies Press. doi: 10.17226/13982.
×
Page 83
Page 84
Suggested Citation:"Chapter 3 - Interpretation, Applications, and Recommendations for Implementation." National Academies of Sciences, Engineering, and Medicine. 2006. Best-Value Procurement Methods for Highway Construction Projects. Washington, DC: The National Academies Press. doi: 10.17226/13982.
×
Page 84
Page 85
Suggested Citation:"Chapter 3 - Interpretation, Applications, and Recommendations for Implementation." National Academies of Sciences, Engineering, and Medicine. 2006. Best-Value Procurement Methods for Highway Construction Projects. Washington, DC: The National Academies Press. doi: 10.17226/13982.
×
Page 85
Page 86
Suggested Citation:"Chapter 3 - Interpretation, Applications, and Recommendations for Implementation." National Academies of Sciences, Engineering, and Medicine. 2006. Best-Value Procurement Methods for Highway Construction Projects. Washington, DC: The National Academies Press. doi: 10.17226/13982.
×
Page 86
Page 87
Suggested Citation:"Chapter 3 - Interpretation, Applications, and Recommendations for Implementation." National Academies of Sciences, Engineering, and Medicine. 2006. Best-Value Procurement Methods for Highway Construction Projects. Washington, DC: The National Academies Press. doi: 10.17226/13982.
×
Page 87
Page 88
Suggested Citation:"Chapter 3 - Interpretation, Applications, and Recommendations for Implementation." National Academies of Sciences, Engineering, and Medicine. 2006. Best-Value Procurement Methods for Highway Construction Projects. Washington, DC: The National Academies Press. doi: 10.17226/13982.
×
Page 88
Page 89
Suggested Citation:"Chapter 3 - Interpretation, Applications, and Recommendations for Implementation." National Academies of Sciences, Engineering, and Medicine. 2006. Best-Value Procurement Methods for Highway Construction Projects. Washington, DC: The National Academies Press. doi: 10.17226/13982.
×
Page 89

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

47 3.1 Development of Best-Value Procurement Methods This section of the report presents a best-value procure- ment framework that was developed from the performance benchmarking case study results and the analysis of best-value concepts addressed in Chapter 2. The proposed framework presents practical, objective criteria and processes (including a scoring system). The results of the national transportation agency survey validated the proposed framework. Objective screening criteria to identify suitable projects for best-value procurement methods are also provided. To facilitate the project-selection process, a screening and selection tool has been developed and is included in Appendix F. The screening criteria are important because they also provide the basis from which the content of the best-value pro- curement method is selected considering the needs of the proj- ect. Both one- and two-phase procurement processes, similar to those in the case study projects, are integrated with the best- value parameters and are implemented in the following manner: 1. One-Step Best-Value Procurement: For those projects that the owner determines will derive no benefit from using a competitive screening system to develop a shortlist, the single phase best-value procurement would generally follow these steps: a. The owner selects which best-value parameters are most appropriate for a given project. From the list of possible best-value evaluation criteria shown in Table 3.1, the owner then selects those criteria associated with the chosen best-value parameters whose formal evaluation will add value to the project. b. These criteria then make up the evaluation plan, and a best-value rating (scoring) system is selected to complete the evaluation plan. c. A best-value award algorithm is then selected based on the scope and complexity of the given project, and the best-value RFP will be published detailing both the award algorithm and the requirements to submit information and documentation to be responsive to the best-value evaluation plan. d. Depending on the nature of the evaluation plan, an evaluation panel may be formed to conduct the formal evaluation of best-value proposals. This will happen when some element of the design must be evaluated to ensure that it complies with agency regulations, design policy, and specifications. e. Proposals are then received and evaluated in accordance with the published evaluation plan and the award is made using the selected best-value award algorithm. 2. Two-Step Best-Value Procurement: For those projects that would benefit from the use of a competitive screening system to develop a shortlist, the two-phase best-value procurement generally follows these steps: a. Step 1 is evaluation of qualifications and quality informa- tion (P.1–P.5 and Q.4) and development of a shortlist of best-qualified bidders. It must be noted that this method involves a more detailed evaluation of qualifications than the current administrative prequalification process in use by many state construction agencies. It is anticipated that the agency will publish a formal “request for qualifica- tions”for each individual project using evaluation criteria that have been customized to the needs of the given high- way construction project. For each evaluation criteria, the agency must develop a measurable standard against which the qualifications would be measured. b. The “statements of qualifications” (SOQs) will be evalu- ated, and the list of prequalified firms will be announced. c. A best-value RFP will be published detailing both the award algorithm and the method by which the Step 1 qualifications ranking/scores will be carried over into the final evaluation. d. The evaluation panel will evaluate all responsive pro- posals in accordance with the published evaluation plan and award will be made according to selected best-value award algorithm. C H A P T E R 3 Interpretation, Applications, and Recommendations for Implementation

48 The use of both systems allows the proposed best-value procurement method to retain maximum flexibility while maintaining an appropriate focus and tradeoff between cost and non-cost parameters. Additionally, it provides the owner with a powerful selection tool that draws the bulk of its details from the methods used to successfully procure the case study projects. Best-Value Parameters Figure 2.1 in Chapter 2 illustrated the research team’s initial attempt at conceptualizing a best-value framework. As described in that chapter, the best-value parameters, which reflect the ultimate goals of the project, form the foundation of the best-value contracting framework. Using Figure 2.1 as a basis, the research team then developed Fig- ure 3.1 to better depict the relationship between the param- eters and the components making up the evaluation plan. Once the owner has determined which parameters are most appropriate for a given project, the remainder of the details of the best-value procurement can be determined. The evaluation criteria stem directly from the best-value parameters, with the selected criteria then yielding the appro- priate rating system and award algorithm. MAP Case Study Evaluation Criteria to Best-Value Parameters The first step in deriving a proposed best-value procure- ment framework is to map the results of the best-value project case study content analysis to the best-value param- eters and evaluation criteria. To do this, the research team determined that the best measure of potential success for a given generic evaluation criterion was repetitive use by those agencies that have experimented with best-value pro- curement. Accordingly, the following standard was devel- oped for selecting a given best-value evaluation criterion to be recommended for use in the proposed framework: To be recommended, the criterion must appear in >50% of the sample population solicitations, or, if none are >50%, the single highest occurrence will be used. With this standard in mind, Table S.1 was revised into Table 3.1. It should be noted that the only evaluation crite- rion that did not meet the 50% rule was “Project Schedule Evaluation.” It was included based on the advisory panel survey indicating that it had a “high potential” for successful implementation. The widespread use in the highway indus- try of A+B contracts, through which a contractor-proposed schedule is integrated into the award algorithm to determine Figure 3.1. Best-value concepts. Table 3.1. Summary evaluation criteria as identified with best-value parameter from total case study project population. Evaluation Criteria Best-Value Parameter Designation Number of Contracts Using Evaluation Criteria (Total = 50) Price Evaluation A.0 42 Project Schedule Evaluation B.0 19 Financial & Bonding Requirements P.0 35 Past Experience/Performance Evaluation P.1 44 Safety Record (or Plan) P.1 25 Key Personnel & Qualifications P.2 41 Utilization of Small Business P.3 30 Subcontractor Evaluation/Plan P.3 29 Management/Organization Plan P.4 31 Quality Management Q.4 27 Proposed Design Alternate D.0 26 Technical Proposal Responsiveness D.1 37 Environmental Considerations D.1 25

49 the best value on a basis of cost and time, also supported the decision to include this criterion. On the basis of Table 3.1, Table 3.2 was developed to allow the direct association of proven case study best-value evalua- tion criteria with the underlying best-value parameters. The owner can now apply this suite of evaluation criteria to the development of a best-value evaluation plan. In addition to the evaluation criteria, the overall evaluation plan also con- sists of a rating system and an award algorithm. A discussion of these two remaining best-value concepts is provided in Section 3.2. 3.2 Proposed Best-Value Award Algorithms and Rating Systems The selection of a best-value award algorithm is also proj- ect specific. Some projects will need a more complex system than others. For example, a project with little variability in the experience of potential contractors will not benefit from an extensive evaluation of qualifications, although an urban freeway project having no schedule constraints may benefit greatly by including the schedule in the competitive bid process. As an additional example, a project requiring special Table 3.2. Summary of proposed best-value parameters and evaluation criteria. Parameter *Final Designation Evaluation Criteria Includes Remarks Cost Initial Capital Cost Construction and procurement costs (also include design costs in a DB project) Sometimes called the “bid” price Time B.0 A.0 Schedule Time to build project (also include design time in a DB project) Sets contract performance period Qualifications & Performance P.0 P.1 P.3 P.4 P.5 Q.0 Prequalification Financial and corporate information as well as bonding requirements Typically a routine government form used for all contracting opportunities Past Project Performance Project experience on past projects that are similar to the project at hand. Also might include past history of claims and litigation Preference is given to offerors with the most relevant experience P.2 Key Personnel Experience & Qualifications Qualifications of key personnel Licenses, registrations, and past project experience of individuals Subcontractors’ Information Subcontracting plan including small business utilization Often requires that goals for participation by certain types of firms be met Project Management Plans Plans for logistics, material management, equipment, traffic control, etc. Often related to schedule constraints Safety Record and/or Plan Corporate safety record and plans for specific safety hazards Often uses the Workers’ Compensation Insurance Modifier as a metric to measure safety record Quality Quality Management Plans Typical QA/QC program submitted prior to award May include design QC if bid alternates or DB is used Design Alternates D.0 D.1 D.2 posed Design Alternate Owner allows contractor to propose an alternate material or technology for a given feature of work Bid is submitted with and without alternates. Owner makes decision as to which alternates will be accepted prior to award Technical Proposal Responsiveness Proposals are considered responsive if they receive a minimum technical score Requires that a measurable standard be developed for each evaluation criteria Environmental Considerations Plans to prevent and/or mitigate pollution during construction Many are required by law and/or regulation * Note: Best-value parameter designations have been changed to simplify the final implementation of the results of this report. DB = design build Pro

50 technical expertise that is not present in the agency may ben- efit from contractors proposing certain aspects of the design as alternates. Each of these situations requires a different best- value award algorithm to ensure, throughout the process of evaluation and award, the continued emphasis of salient aspects of the project that factored in its selection to be pro- cured using best value. Seven best-value award algorithms were identified in the first phase of this study and are shown in Table 3.3. The table further shows the variables used in each algorithm and the method used to determine the award. It should be noted that five of the seven award algorithms entail point scoring or some mathematical combination of price and non-price scores. These algorithms were all drawn from the case study projects identified in Phase 1 of this research. Many of the case study projects were not highway projects, because the research team was committed to looking for possible best-value procurement solutions across both highway and building construction industries. If the vertical projects are eliminated and only those algorithms that were used to procure horizontal construction projects are included, the sample becomes much smaller. However, it becomes more relevant because the case study best-value award algorithm results are restricted to those that may be most suitable for highway construction projects. Twenty-eight of the case study projects fell into the horizontal category. Additionally, Table 2.16 in Chapter 2 contains a summary of published agency best-value award algorithm practices. These data were drawn from the case study projects as well as from other published information in the literature. The sample contains 36 transportation agencies. Combining the two samples furnishes a means to gain insight as to the applicability of existing best-value award algorithms to highway construc- tion. The results of this analysis appear in Table 3.4. It should be noted that the sum of the Qualitative and Quantitative Cost-Technical Tradeoff (53%) is highest in the horizontal case study project sample because of the large per- centage (61%) of Federal projects that are in the population. Cost-Technical Tradeoff is mandated by federal regulation if a low-bid award is not used. Three of the six low-bid case study projects were federal low-bid best-value projects. Thus, even though it is the most prevalent in the case study popu- lation, the reader should not interpret that statistic to mean that it is the best algorithm for all horizontal projects. Look- ing at the number of transportation agencies that use the var- ious types of algorithms, if the two types of cost-technical tradeoff algorithms are added together, one can see that actual usage is almost evenly split among the algorithms. Therefore, no clear trend seems to exist. Thus, it must be con- cluded that flexibility in the selection of a best-value award algorithm should be maintained. Using this analysis as a starting point, the best-value award algorithms can be condensed into three basic types. It should be noted that each has an associated best-value evaluation Table 3.3. Summary of best-value award algorithms. Best-Value Award Algorithm Algorithm Variables Award Determination Meets Technical Criteria—Low Bid If T > T min, Award to Pmin If T < T min, Non-Responsive T = Technical Score P = Project Price Lowest Price Adjusted Bid AB = P/T Award ABmin AB = Adjusted Bid Adjusted Score AS = (T x EE)/P Award AS max AS = Adjusted Score EE = Engineer’s Estimate Numerical analysis using point scoring, a mathematical combination of price and non-price factors, or a quantitative tradeoff analysis Weighted Criteria TS = W1S1 + W2S2 + … + WiSi + W(i+1)PS Award TS max TS = Total Score Wi = Weight of Factor i Si = Score of Factor i PS = Price Score Quantitative Cost- Technical Tradeoff TIncrement = [(Tj/Ti) – 1] x 100% PIncrement = [(Pj/Pi) – 1] x 100% If TIncrement > PIncrement, Award Proposali If TIncrement < PIncrement, Retain Proposalj for possible award and repeat with Proposalj+1 Repeat Process until TIncrement > PIncrement T = Technical Score P = Project Price Fixed Price—Best Proposal Award T max, Fixed P T = Technical Score P = Project Price Qualitative Cost- Technical Tradeoff Similar to above, only no quantitative analysis of difference. Award to proposal that has best value in proposed scope. See Figure 3.3. Evaluation panel reaches consensus as to which proposal is the best Qualitative tradeoff analysis of cost and technical factors

51 rating system that best fits the mechanics of the award algo- rithm. The results are as follows: 1. Meets Technical Criteria—Low Bid: All non-cost criteria are evaluated using a satisficing rating system. Direct point scoring may be used to determine if the technical proposal meets the minimum technical score. Those proposals found to be fully responsive make up the “competitive range”(FAR term that fits this case). The bids are then opened, and the project is awarded to the lowest price proposal. 2. Cost-Technical Tradeoff (Qualitative): All non-cost crite- ria are evaluated using either an adjectival or modified sat- isficing rating system. Those proposals found to have no fatal deficiencies make up the competitive range, and then the bids are opened and the project is awarded to the best value, without any mathematical manipulation or combi- nation of price and non-price factors. 3. Value Unit Price ($/technical point): All non-cost crite- ria are evaluated using a direct point scoring system. Those proposals found to have no fatal deficiencies make up the competitive range. The bids are then opened, and the proj- ect is awarded to the best value using a mathematical manipulation or combination of both price and non-price factors, a tradeoff analysis, or points. This category would include adjusted bid, adjusted score, quantitative cost- technical tradeoff, weighted criteria, and fixed price—best proposal (technical score only). The use of this algorithm permits the owner to put a dollar value on a point of score, creating a “best-value unit price.” Meets Technical Criteria—Low Bid To implement this best-value award algorithm, the owner will use the process that is illustrated graphically in Figure 3.2 with the following steps: 1. Screen the candidate project and determine its potential to accrue benefits by using best-value procurement.The project screening and selection tool provided in Appendix F can facilitate this screening process. If the project appears to be a good candidate, capture the essential screening criteria that made it a good candidate and rank them in order of impor- tance to the project. 2. Develop qualifications and technical evaluation criteria based on the screening criteria. For each evaluation criteria, the owner must develop a measurable standard against which responsiveness will be measured. 3. Publish the best-value solicitation. The solicitation will contain the following items as a minimum: a. Scope of work, plans, and specifications b. Bid form c. Contract completion date or days d. Best-value evaluation plan listing the evaluation criteria with corresponding standards e. Description of what constitutes a non-responsive proposal 4. Receive best-value proposals and sealed bids. 5. Evaluate best-value proposals against published standards and determine which proposals are fully responsive in meeting the technical and qualifications criteria. 6. Return the sealed bids to the authors of non-responsive proposals. 7. Open the bids of those competitors that remain in the competitive range. 8. Award to the lowest bid from within the competitive range. It is important in this award algorithm to limit the number of qualification and technical criteria to those from categories that carried high importance in the project’s best-value screen- ing. The evaluation plan should be written to be completely transparent to members of industry. To avoid the possibility of dispute or bid protest, the owner should “Clearly state the evaluation criteria and the weight assigned to each item and ensure that the evaluation team uses them. Clearly state the requirements of the RFP including what will be consid- ered a non-responsive proposal.” (Parvin 2000) Table 3.4. Case study best-value award algorithm usage. Best-Value Award Algorithms Number of Transportation Agencies Using Award Algorithm Number of Horizontal Case Study Projects Using Award Algorithm Meets Technical Criteria–Low Bid 7 19% 6 21% Adjusted Bid 7 19% 3 11% Adjusted Score 6 17% 1 4% Weighted Criteria 8 22% 3 11% Quantitative Cost-Technical Tradeoff 2 6% 3 11% Fixed Price–Best Proposal 1 3% 0 0% Qualitative Cost-Technical Tradeoff 5 14% 12 42% Totals 36 100% 28 100%

52 The goal is to have as many responsive competitors at the end of the first step as possible, thus ensuring the greatest pos- sible price competition in the second step of the procure- ment. Therefore, only evaluation criteria that will assist the owner in differentiating among the pool of potential com- petitors should be included in the evaluation plan. Cost-Technical Tradeoff (Qualitative) To implement this best-value award algorithm, the owner will follow the process that is illustrated graphically in Figure 3.3. Implementation includes the following steps: 1. Screen the candidate project and determine its potential to accrue benefits by using best-value procurement. The proj- ect screening and selection tool provided in Appendix F can facilitate this screening process. If the project appears to be a good candidate, capture the essential screening criteria that made it a good candidate and rank them in order of importance to the project. 2. Develop qualifications, technical, schedule, and cost eval- uation criteria (QC, TC, SC, and CC, respectively, in Fig- ure 3.3) as appropriate based on the screening criteria. For each evaluation criteria, the owner must develop a measurable standard against which responsiveness will be measured. 3. Publish the best-value RFQ. The solicitation will contain the following items as a minimum: a. Description of scope of work b. SOQ forms c. Contract completion date or days d. List of qualifications evaluation criteria with correspon- ding standards e. Description of process to be followed for the best-value proposal evaluation plan f. Description of what constitutes a non-responsive SOQ Screen Project* for BV Award Good BV Candidate? Procure Using Low Bid Develop Qualifications &/or Technical Evaluation Criteria (Q/TEC) {Q/TECi to Q/TECn} Publish BV Solicitation BVP Responsive to All Q/TECs? Drop from Competition/ Return Sealed Bid Announce BV Competitive Range BV Proposal (BVP) {BVPi to BVPn} Open Sealed Bids for BVPs in Competitive Range Award to Lowest Responsive Bid in Competitive Range Step 1 Step 2 No No Yes Yes * Please refer to the screening and selection steps presented in Appendix F. Figure 3.2. Two-step meets technical criteria—low bid best-value procurement flowchart.

53 4. Receive SOQs. 5. Evaluate SOQs against published standards and determine which statements are fully responsive and meet the qualifi- cations criteria. 6. Announce the list of prequalified firms. 7. Publish the best-value RFPs. The solicitation will contain the following items as a minimum: a. Scope of work and relevant plans and specifications b. Proposal forms c. Contract completion date or days (if applicable) d. Method to carry forward Step 1 qualifications rank- ing/scores into final evaluation (if applicable) e. Best-value proposal evaluation plan listing the techni- cal, schedule, and cost evaluation criteria with corre- sponding standards f. Description of what constitutes a non-responsive proposal Cost {CCi – CCn} Screen Project* for BV Award Good BV Candidate? Procure Using Low Bid Publish BV Request for Qualification Rate SOQs Using QCs {QCi to QCn} {SOQi to SOQn} SOQ Responsive to All QCs? Drop from Competition Step 1 Step 2 No No Yes Yes Announce Competitive Range Publish BV Request for Proposals BV Proposal (BVP) {BVPi to BVPn} Sched {SCi – SCn} Tech {TCi – TCn} Develop Evaluation Criteria Qual {QCi – QCn} Statements of Qualifications (SOQ) {SOQi to SOQn} Rate BVPs Using T/S/CCs {T/S/CCi to T/S/CCn} {BVPi to BVPn} BVP Responsive to All T/S/CCs Drop from Competition No Yes Conduct Cost- Technical Tradeoff Analysis Award to proposal Found to Offer the Best Value * Please refer to the screening and selection steps presented in Appendix F. Figure 3.3. Two-step cost-technical tradeoff (qualitative) best-value procurement flowchart.

54 8. Evaluate proposals against published technical, schedule, and cost standards and determine which proposals are fully responsive in meeting the qualifications criteria. 9. Eliminate any non-responsive proposals from the compet- itive range. 10. Roll up evaluation results. 11. Convene selection panel and conduct qualitative cost- technical tradeoff analysis to identify the best proposal. 12. Award to the firm within the competitive range offering the best-value proposal. This is the most subjective of the three best-value award algorithms, and as a result, it will be the least popular to implement. However, numerous conversations with pro- curement officials in the federal sector indicate that they have had more award protest problems with the quantitative cost- technical tradeoff than with this more subjective approach. Feldman’s Government Contract Awards treatise states as follows various issues associated with quantitative and qual- itative ratings in Sections 10:20 and 10:21 (Feldman 1994, footnotes omitted): The General Accounting Office (GAO) has approved the use of qualitative ratings as opposed to numerical ratings, so long as they give the source selection official a clear basis for considering the merits of proposals. [§10:21] Indeed, the Comptroller General has stated that such ratings ‘may be a more direct and meaningful method’ than the numeri- cal evaluation of technical proposals, even though both evaluation approaches characteristically reflect the disparate, subjective judg- ments of the evaluators. As with numerical rating systems, the GAO has said that qualitative ratings are best used as guides to intelligent decision making and are not generally controlling for award. [§10:21] Sometimes, both agency and industry personnel assign talis- manic importance to point scores. The Comptroller General has stated repeatedly, however, that point scores are useful only as guides to intelligent decision making and are not generally con- trolling for award because they reflect the subjective and sometimes disparate judgments of the evaluators. [§10:20] [T]he Comptroller General (and some agency regulations) consistently have disapproved of agencies’ establishing predeter- mined cutoff scores for deciding technical acceptability. [§10:20] This qualitative approach enables owners to differentiate between competitors when the relative merits of each pro- posal are difficult to quantify using a point scoring system, but the project has specific technical or experiential require- ments to be successful. Value Unit Price To implement this best-value award algorithm, the owner will follow the process that is illustrated graphically in Figure 3.4. Implementation includes the following steps: 1. Screen the candidate project and determine its potential to accrue benefits by using best-value procurement. The project screening and selection tool provided in Appen- dix F can facilitate this screening process. If the project appears to be a good candidate, capture the essential screening criteria that made it a good candidate and rank them in order of importance to the project. 2. Develop qualifications, technical, schedule, and cost evaluation criteria (QC, TC, SC, and CC, respectively, in Figure 3.4) as appropriate based on the screening crite- ria. For each evaluation criterion, the owner must develop a measurable standard against which respon- siveness will be measured. 3. Publish the best-value RFQs. The solicitation will contain the following items as a minimum a. Description of scope of work b. SOQ forms c. Contract completion date or days d. List of qualifications evaluation criteria with correspon- ding standards e. Description of process to be followed for the best-value proposal evaluation plan f. Description of what constitutes a non-responsive SOQ 4. Receive SOQ. 5. Evaluate SOQs against published standards and deter- mine which statements are fully responsive and meet the qualifications criteria. 6. Announce the list of prequalified firms. 7. Publish the best-value RFPs. The solicitation will contain the following items as a minimum: a. Scope of work and relevant plans and specifications b. Proposal forms c. Contract completion date or days (if applicable) d. Method to carry forward Step 1 qualifications ranking/ scores into final evaluation (if applicable) e. Best-value proposal evaluation plan listing the technical, schedule, and cost evaluation criteria with corresponding standards f. Description of what constitutes a non-responsive proposal 8. Evaluate proposals against published technical, sched- ule, and cost standards and determine which propos- als are fully responsive in meeting the qualifications criteria. 9. Eliminate any non-responsive proposals from the com- petitive range. 10. Roll-up evaluation results and determine the final point score for each responsive proposal. 11. Compute the $/technical point using the formula pub- lished in the RFP to identify the best proposal. 12. Award to the firm within the competitive range offering the lowest best-value unit price.

55 The value unit price algorithm assumes that the owner will develop a specific formula that can be used to calculate the best-value objective decision criterion. Table 3.3 contains the formulae that are currently in use for the various best-value award methods. This algorithm is extremely dependent on the proper implementation of a thoughtfully developed direct point scoring system. Once the direct point scoring system has been selected, the owner must make a number of decisions about the details of the system to ensure the integrity of the scoring process. Unfortunately, the extremely important decision regarding the numerical range of possible points to be awarded is often made arbitrarily without regard to its overall impact on the scoring system. Owners must make sure that the cost-value of a single point of score is consistent with its actual value to the project. In a direct point scoring system, the total number of points awarded in each rated category is usually determined by the weight that the owner allocates to that category. For example, the qualifications of the Cost {CCi – CCn} Screen Project* for BV Award Good BV Candidate? Procure Using Low Bid Publish BV Request for Qualification Rate SOQs Using QCs {QCi to QCn} {SOQi to SOQn} SOQ Responsive to All QCs? Drop from Competition Step 1 Step 2 No No Yes Yes Announce Competitive Range Publish BV Request for Proposals BV Proposal (BVP) {BVPi to BVPn} Sched {SCi – SCn} Tech {TCi – TCn} Develop Evaluation Criteria Qual {QCi – QCn} Statements of Qualifications (SOQ) {SOQi to SOQn} Rate BVPs Using T/S/CCs {T/S/CCi to T/S/CCn} {BVPi to BVPn} BVP Responsive to All T/S/CCs Drop from Competition No Yes Compute BV $/Tech pt Award to Proposal with Lowest BV Unit Price * Please refer to the screening and selection steps presented in Appendix F. Figure 3.4. Two-step value unit price best-value procurement flowchart.

56 project’s quality control engineer might carry a total possible score of 5 points, whereas, the quality management plan may carry a maximum total score of 45 points. If the maximum total score for all rated categories adds up to 1,000 points, then the weight assigned to each rated category is proportional to its individual maximum total score.Continuing with this hypothetical example, if this project’s estimated cost is $20 million,the value of each point will be $20,000.Thus,the cost value of the engineer’s qualifications will be $100,000 and the cost value of the quality management plan feature of the design is $900,000. Thus, the overall evaluated value of the two is $1.0 million or 5% of the project. The reader must remember that these values are not absolute.However, if the amount of money at risk if the project is not properly constructed is estimated at $5.0 million, then these rated categories are under- weighted relative to the entire project value. Therefore, more weight should be given (i.e., more points assigned) to the quality management feature of the project in the evaluation plan. If, on the other hand,the quality management aspects of this project are a minor portion of the work, and the technical and performance risk lies in other rated categories, then these two evaluation cate- gories may be over-weighted, and the points assigned to them should be reduced and moved to other more important categories. A detailed discussion of best-value evaluation rating systems for the other two award algorithms is found in Chapter 2. The details of each evaluation rating system should ultimately be based on the requirements of the individual project under analysis.Those projects that are relatively straightforward should have a simple rating system. On the other hand, those projects that are technically complex will need a more complex rating system to be able to identify the best value. Additionally, the owner must ensure that the rating system can be mapped back to the project screening system and ensure that those areas are thoroughly evaluated.Any weighting that is developed must be consistent with the project screening criteria as well and ensure that those areas that have the greatest importance in the procurement are the most heavily weighted. Finally, the owner should test the weighting with a small number of pilot projects to ensure that the system behaves as anticipated. 3.3 Summary of Proposed Best- Value Procurement Framework Table 3.5 is a summary of the proposed best-value pro- curement framework. It shows how the practical, objective Award Algorithm BV Parameter and Evaluation Criteria Meets Technical Criteria—Low Bid (Cost) Cost-Technical Tradeoff (Qualitative) Value Unit Price Price Price: A.0 X X X Time Schedule: B.0 X X X Cost Cost: C.0 X X Qualifications Prequalification: P.0 X Past Project Performance: P.1 X X Key Personnel Experience: P.2 X X Subcontractor Information: P.3 X X Project Management Plans: P.4 X X X Safety Record/Plan: P.5 X X Quality Quality Management: Q.0 X X X Design Alternates Design with Proposed Alternate: D.0 X X Technical Proposal Responsiveness: D.1 X Environmental Considerations: D.2 X X Rating System Satisficing Adjectival or ModifiedSatisficing Direct Point Scoring Table 3.5. Summary of best-value procurement framework.

57 evaluation criteria are related to both the best-value award algorithms and the rating (scoring) system. Table 3.5 shows how the four elements of best-value pro- curement can be brought together in a cogent manner that allows the owner to develop a best-value procurement method- ology systematically on a project-by-project basis. When used in conjunction with the best-value project screening and selec- tion system, this framework will permit state agencies to create a standardized procurement policy for best-value projects. 3.4 Implementing the Proposed Best-Value Procurement Method Figure 3.5 is a flow chart that illustrates the process by which an agency would implement best-value procurement. The process is designed to be project-specific and stems from the output of the project screening and selection process that was used to pick a given project to be delivered using best-value Figure 3.5. Implementing best-value procurement flowchart. Select BV Cost Parameter & Criteria A.0 BV Project Screening and Selection Process Project Selected for BV Award Benefits from BV Procurement Identified in Screening Select Project BV Parameters & Evaluation Criteria Project Complexity Use Meets Technical Criteria Low- Bid BV Award Algorithm Barrier to Subjective Award? BV Award Algorithm Selected Based on Selected Best- Value Award Algorithm Move to Flow Charts Shown in Figures 3.2, 3.3, & 3.4, and Skip Screening Steps Yes No Complex Simple Project Selected for BV Award Cost Savings Benefits Time Savings Benefits Quality Enhancement Benefits Qualifications Benefits Use Cost- Technical Tradeoff BV Award Algorithm Select BV Schedule Parameter & Criteria B.0 Select BV Design Alternate Parameter & Criteria D.0 – D.2 Select BV Qualifications Parameter & Criteria P.0 – P.5 and/or Q.0 Use Value Unit Price BV Award Algorithm

58 procurement. Note that Section 3.5 discusses the screening and selection process in detail. In essence, the best-value procurement process involves a series of decisions that are constrained by the best-value pro- curement framework. It tracks through the following series of steps: 1. Having identified those potential benefits that may be accrued by delivering a project by best-value procurement, the owner then lists those benefits and identifies the specific best-value parameters that are appropriate to the project from the list of potential parameters shown in Table 3.2. 2. For each of the appropriate parameters, the relevant eval- uation criteria are selected. There will always be a cost parameter with evaluation criteria in the final set. If the schedule is fixed by the agency, then no schedule parame- ter will be selected, but, if the contractor is allowed to pro- pose some element of the schedule, then it will also be included. This set forms the foundation on which the remainder of the procurement is built. 3. Next, the best-value award algorithm is selected based on project characteristics. Project complexity must be con- sidered because it will impact the choice of award algo- rithms. 4. If the project is a relatively simple and technically straight- forward job, then the simplest best-value award algorithm, meet technical criteria—low bid, is a logical choice. If the owner is concerned about project quality, the process may involve prequalification or shortlisting or could allow the owner to factor in its own costs into the selection decision. The previously identified parameters and evaluation criteria make up the set that is published in the best-value solicitation (see Figure 3.2). A measurable standard is developed for each best-value evaluation criterion, and a satisficing (“go/no-go”) rating system is established. The project is then advertised and awarded in accordance with the process described in Figure 3.2. 5. If the project’s scope of work is judged to be complex, then the owner must decide whether it will use the cost- technical tradeoff or value unit price award algorithm. As previously stated, cost-technical tradeoff gives the owner maximum flexibility in its best-value award decision, and experience of federal agencies indicates that its use results in better decisions and also reduces the potential for bid protests based on improper application of the published evaluation plan. However, legislatures may be reluctant to allow agencies to use this algorithm due to the major paradigm shift from the conventional design-bid-build procurement process. For the same reason, agencies may be reluctant to use this process even if legal authorization exists. An owner that has authority to use this process and is interested in doing so would take the parameters and evaluation criteria identified in the screening process and develop appropriate evaluation standards for each crite- rion. A modified satisficing or adjectival rating system would then be established. The criteria associated with the qualifications and quality parameters would form the basis for the best-value RFQ. The remaining parameters and criteria would be published in the RFP. 6. The process would follow that shown in Figure 3.3. Once the Step 2 evaluation of those proposals that remain in the competitive range after Step 1 is completed, the selection panel would conduct the cost-technical tradeoff, could elect to proceed with discussions and final proposals followed by re-evaluation, and would select the proposal that offered the greatest value to the agency. 7. If there are legal, institutional, or political barriers to using cost-technical tradeoff on a relatively complex project, the logical alternative is value unit price. Development of this procurement is the same as described in the previous para- graph except that the owner must develop a formula to compute the best-value unit price. As previously stated, there are a number of possible formulae that have been successfully used by transportation agencies across the country that could be adopted or adapted by the procur- ing agency for this step. However, the research team believes that the weighted criteria formula shown in Table 3.3 is the most flexible approach to determining the best- value unit price, and allows the owner the ability to control most completely the relationship between the mathemat- ical outcome and the project’s requirements. Therefore, this formula is recommended. The impact of using the other formulae will be demonstrated in the next section of this report. 8. The process would follow that shown in Figure 3.4. Once the Step 2 evaluation is completed for those proposals that remain in the competitive range after Step 1, the evalua- tion panel would compute the value unit price and award the project to the proposal that best satisfied the formula’s objective decision criterion. Again, the procurement process could include the opportunity for discussions and final proposals, if permitted by enabling legislation and deemed advisable by the procuring agency. The final point concerning implementing the proposed best-value procurement method deals with the owner’s learn- ing curve. The research team’s personal experience in apply- ing best-value procurement techniques in the federal sector and with design-build best-value awards in state highway agencies leads it to believe that each agency will decide on an optimum process for delivering best-value projects only after a number of best-value projects are completed. Thus, it needs to be recognized that the procurement method proposed in this report provides a theoretical basis to which an agency

59 can add its personal legislative and institutional constraints, thus producing a customized method that fits its market and its mission. Illustrative Examples This somewhat complex process is best illustrated by examples. Three examples are provided to allow the reader to see the dynamics of each of the proposed best-value award processes. Meets Technical Criteria—Low-Bid Award Algorithm Example Starting with the simplest award algorithm, meets techni- cal criteria—low bid, a hypothetical chip seal project will be introduced. The project’s details are as follows: • The owner restricts the competition to prequalified chip seal contractors that have completed at least three previous projects in the state. • Safety is to be measured using the standard that the firm must have a Workers’ Compensation Insurance Rate Modifier of 1.00 or lower. • The owner lists two types of allowable binder and aggregate combinations: AC15-5TR binder with precoated grade 3 aggregate and CRS-2P binder with uncoated grade 3 aggre- gate. The contractors must state in their proposal which combination they intend to use to be technically responsive. • The owner requires that a quality management plan be submitted based on the contractor’s binder-aggregate choice that complies with the minimum standards shown in the state standard specifications for chip seal projects. • The owner will allow a maximum of 150 days for this job, and the contractor most propose its own detailed schedule for completing the project in the stipulated period. This schedule essentially consists of identifying the dates on which each of the major chip seal sections will be shot, because this project is not continuous and involves four different highways. Table 3.6 shows the results of the technical criteria evalu- ation for five typical contractor proposals for this project. One can see that two of the five firms were eliminated for not having met one or more of the technical criteria. In both cases, their price proposal was returned unopened. Of the three remaining firms, Firm C had the lowest price proposal. Two points should be noted about allowing the competitors to select from two predetermined binder/aggregate combi- nations. First, it does not make this a design-build project. The agency is neither allowing the contractor to design the final product nor is it shifting any performance liability with the product selection. Second, the owner’s engineer obvi- ously felt that either of these alternatives would furnish a sat- isfactory product. By allowing the contractor to make the selection, the agency is creating an environment in which a contractor can base the bid and the schedule on the alternate with which it has the most, and perhaps the best, experience. This would be directly reflected in the bid price and the schedule. The next two award algorithms require a more complex evaluation plan and a more involved evaluation process. Hence, a more detailed example has been developed to illus- trate the dynamics of the cost-technical tradeoff and value unit price best-value award algorithms. An example project was found in the Florida DOT procurement policy guide (FDOT 1996). It furnishes enough basic information to Table 3.6. Best-value evaluation results for hypothetical chip seal project with meets technical criteria—low-bid award algorithm. Firm Prequal. Safety Binder/Agg. Quality Plan Time Price Proposal Proposed GO/ NO GO Proposed Modifier GO/ NO- GO Proposed GO/ NO GO Proposed GO/ NO- GO 150 max A 3 projects GO 0.97 GO AC15-5TR w/precoat G3 GO Meets specs GO 150 $2,859,890 B 2 projects NOGO 0.87 GO AC15-5TR w/precoat G3 GO Does not meet specs NO- GO 150 Bid returned C 9 projects GO 0.91 GO CRS-2P w/G3 GO Meets specs GO 150 $2,832,489 D 4 projects GO 1.03 NO- GO CRS-2P w/G3 GO Meets specs GO 150 Bid returned E 3 projects GO 0.95 GO AC15-5TR w/precoat G3 GO Meets specs GO 150 $2,840,049 Firm C is the winner. Lowest bid with all GOs.

60 allow the process to be demonstrated. The author will fill in hypothetical information where actual project information is not known. The project was a reconstruction of an exist- ing suburban highway. The majority of the work is the recon- struction of the pavement. Table 3.7 shows the project proposal data that comes from the FDOT example project. To furnish the input necessary to adequately demonstrate the proposed procurement processes, the following hypo- thetical project information is assumed: • The DOT was willing to allow the contractors to propose the option of either recycling the millings from the existing asphalt pavement in the project mix or furnishing new hot mix asphaltic concrete pavement and stockpiling the millings for future use by the DOT. Accordingly, the con- tractor was allowed to propose the type of asphalt binder and a mix design that conformed to state specifications. The RFP stated that “recycling was preferred if quality could be maintained at a reasonable cost.” This is consid- ered a design alternate. • Traffic control was an issue as this road was on the route to a major tourist attraction in the area. The DOT desired that disruption to traffic be minimized if possible. The proposed traffic control plan was to be furnished in the proposal. • The maximum number of scheduled working days was 500. A daily user cost of $6,000 per day was specified for use in those best-value award algorithms where a value must be placed on time. • The DOT was specifically concerned about the following qualifications issues: the qualifications of the quality control engineer, the qualifications of the superintendent, the num- ber of similar projects the firm had successfully completed in the region using similar mix designs, the level of small business utilization, and the firm’s safety record as measured by its Workers’ Compensation Insurance Modifier. • This project was screened and selected as a good candidate for best-value procurement because it seemed to have the potential to accrue benefits in the following areas: – The probability of success was enhanced by the selection of a highly competent and experienced contractor. – It had the potential for quality enhancements by com- peting pavement design components. – There was an opportunity that an innovative traffic con- trol plan could accrue real time savings. – Work zone safety was a particular concern, and the DOT wanted to ensure that the successful contractor had a strong institutional safety program. • The agency published the fact that a proposal must score a minimum of 70 points in the technical evaluation to be considered responsive. A minimally satisfactory proposal in each category would receive 50% of the available points. The final scores and breakdown of the details of the tech- nical score are shown in Table 3.8. At this point, there is no need to explain the reasons for the individual scores. Table 3.7. Best-value selection on example project (FDOT 1996). Firm Technical Score Time Price Proposal A 450 $11,880,000 B 460 10,950,000 C 500 9,850,000 D 500 9,760,000 E 68 74 76 86 92 500 9,700,000 Table 3.8. Best-value selection on example project technical score breakdown. Totals Technical Score Breakdown Firm Time Price Proposal Tech. Score (100) Design Alternate (20) Schedule (20) Traffic Control Plan (20) Quals. (20) Past Performance (10) Safety (10) A 450 $11,880000 92 18 20 17 19 10 8 B 460 10,950,000 86 18 19 17 16 7 9 C 500 9,850,000 76 15 10 15 18 10 8 D 500 9,760,000 74 14 10 16 16 8 10 E 500 9,700,000 68 13 10 15 14 7 9

61 Cost-Technical Tradeoff Example The proposed version of cost-technical tradeoff involves the qualitative determination of best value without a direct mathematical comparison of scores. Thus, the scoring results need to be broken out in a manner that facilitates the discus- sion of the merits of each proposal and the arrival at consensus regarding the best value. Table 3.9 demonstrates how the eval- uation results can be organized in an adjectival manner. It essentially looks to determine which proposal had the best score in each category. It then identifies the second best score. To further amplify the results, any proposal that received 80% of the possible points is also called out as “good.” The results in Table 3.9 are then sorted from lowest respon- sive bidder. Firm E is eliminated because its technical score was lower than 70 points and is therefore considered not responsive. Table 3.10 shows the reorganized evaluation information. At this point, the evaluation panel must come together and compare the cost of awarding based on a proposal that is rated higher than the lowest responsive bid. This can obviously go many ways and no attempt will be made at this point in the report to cover all the possible outcomes. However, looking at Table 3.10, one can arrive at several conclusions that would influence the evaluation panel’s decision: • Firm D, the lowest priced proposal, was satisfactory in all categories and furnished the best safety record and received “good” ratings in traffic control plan, qualifications, and past experience. • Firm A is clearly the best proposal having been the best in 5 out of 6 categories. However, its price is $2,120,000 more than the low bid. • For an additional $90,000, Firm C offers the best past per- formance, the second best qualifications, a good safety record, and a slightly better score than Firm D in the Design Alternate category. Finally, it furnishes the same schedule as Firm D. Thus, one possible outcome is for the evaluation panel to decide that the enhanced proposal offered by Firm C is worth Table 3.9. Cost-technical tradeoff best-value selection on example project technical score/adjectival breakdown. Totals Technical Score Breakdown Firm Time Price Proposal Tech. Score (100) Design Alternate (20) Schedule (20) Traffic Control Plan (20) Quals. (20) Past Performance (10) Safety (10) A 450 $11,880,000 92 18 best 20 best 17 best 19 best 10 best 8 good B 460 10,950,000 86 18 best 19 2nd best 17 best 16 7 9 2nd best C 500 9,850,000 76 15 1510 1510 18 2nd best 10 best 8 good D 500 9,760,000 74 14 10 16 good 16 good 8 good 10 best E 500 9,700,000 68 NR* 13 14 7 9 2nd best *NR = not responsive Table 3.10. Cost-technical tradeoff best-value selection on example project adjectival comparison with price increment. Firm Price Proposal Price Increment (over low bid) Tech. Score (100) Des Alt. (20) Sched. (20) TC Plan (20) Quals. (20) Past Perf. (10) Safety (10) D $9,760,000 -- 74 14 10 15 1510 16 good 16 good 8 good 10 best C 9,850,000 90,000 76 18 2nd best 10 best 8 good B 10,950,000 1,190,000 86 18 best 19 2nd best 17 best 16 7 9 2nd best A 11,880,000 2,120,000 92 18 best 20 best 17 best 19 best 10 best 8 good

62 in federal procurements), Firm C becomes the best value. This is because the weighted value of the technical evaluation cri- teria became greater with respect to price. Finally, if one were to reverse the TTA specified weighting and give technical 85% of the total, Firm A, the high bidder, is awarded the contract. In this case, the agency is deciding that the value of the evalu- ated technical criteria justifies the increased cost. This example illustrates two very important points about best-value procurement. First, the determination of the weights assigned to the various portions of the best-value parameters must be made carefully, with great thought as to the ultimate impact of those weightings in the final award decision. Secondly, discrete point values assigned to each eval- uation criterion must take into consideration the actual value that the criterion brings to the project after award. In this case, where one point is worth $100,000, the agency should review the scoring system point by point and ensure that each point reflects a commensurate return on investment. 3.5 Screening Criteria for Best- Value Procurement Best-value procurement has obvious advantages, and some federal agencies employ it for 100% of their procurements. While some form of best-value procurement can theoretically be used on every project, certain projects will benefit more from its application. Conversely, there are instances when the benefits of the best-value system are outweighed by the ben- efits of open low-bid competition. Establishing a process for selecting the most appropriate projects for best-value procurement has two distinct advantages. First, projects that are more appropriate will perform better, saving the taxpayers money while deliver- ing higher quality projects. Second, as highway agencies begin to use best-value procurement, there will unques- tionably be a learning curve for the agency and its industry partners. By selecting the most appropriate project for a best-value procurement, the agency can “flatten the learn- ing curve” and help make the transition to a best-value cul- ture smoother. Table 3.11. Value unit price best-value selection on example using weighted criteria formula. Evaluated Values Adjusted Values 85% Price/15% Technical 50% Price/ 50% Technical 15% Price/ 85% Technical Firm Time Price Proposal Tech. Score Adjusted Score Adjusted Price Total Score Total Score Total Score A 450 $11,880,000 92 100 78 80.90 88.76 96.63 B 460 10,950,000 86 93 87 88.07 90.30 92.52 C 500 9,850,000 76 83 98 96.08 90.53 84.99 D 500 9,760,000 74 80 99 96.54 89.91 83.28 E 500 9,700,000 68 74 100 96.09 86.96 77.83 an extra $90,000 (less than a 1% increase), whereas the enhanced proposals offered by Firms A and B (22% and 12% increases over low bid, respectively) are not worth the addi- tional quality indicated by the technical evaluation score. Thus, Firm C would be declared the best value and be awarded the contract. It must be noted that the use of this rationale would preclude awarding to the proposer submit- ting the higher priced proposals if the evaluation panel agreed that the additional factors in each proposal beyond the min- imum were not worth the additional incremental cost. Value Unit Price Example A number of formulae are in use throughout the country for calculating a value unit price. For this example, the team uses the weighted criteria formula used by the Texas Turnpike Authority (TTA) in a recent best-value award (TTA 2001). In that evaluation plan, the following formulae was used: In this system, the price carries an effective weight of 85% and the technical score carries a corresponding weight of 15%. It is possible to use weights other than these values in the best- value award algorithm to compute the value unit price. Table 3.11 shows the results of the value unit price calcula- tion using the weighted criteria formula for three different weights. The first used the TTA specified weighting of 85% price and 15% technical. In this case, Firm D, the second low bidder, would be determined to be the best value because it had the highest adjusted total score. It can be seen in this sys- tem that one point of technical score was worth approximately $100,000. If the weighting was modified so that price was equal to all other factors combined (a very common practice Contract Bid Price Score oposal Bid = −100 (Pr Price Score 100) Lowest Bid Price Score × Adjusted Technical Score Proposal Technic = al Score 100 Lowest Bid Price Score × Total Score = (Adjusted Technical Score 0× .15) + (Contract Bid Price Score 0.85×

63 The process of selecting projects for best-value procure- ment is intertwined with the process of selecting best-value parameters and evaluation criteria. Any project can use best- value selection, but the project’s complexity, specialization, quality requirements, opportunities for innovation, and pro- curement risk will determine whether a best-value selection process is appropriate. Additionally, the agency’s project goals and performance measures must be considered in the decision to select a project for a best-value procurement approach. For example, the use of a past experience/past per- formance evaluation criterion is appropriate when high pre- cision or quality is considered critical, and the use of a project schedule evaluation is appropriate when user delay costs are a significant concern. While significant project benefits may be generated through time savings, cost savings, and quality enhance- ments, agencies considering whether to use a best-value pro- curement methodology should keep in mind that such procurements may generate additional costs in the form of higher procurement costs or higher administrative costs. Agencies must determine whether the benefits of using best- value procurement outweigh the costs. Each project must be examined on an individual basis. The primary objective for best-value procurement project selection can be summarized as follows: Select projects with characteristics that provide significant benefit from using an alternative form of procurement. Once identified, develop the evaluation plan and project scope to con- firm that the benefits are real, the negative impacts are minimal, and the risks are manageable. The best-value project screening criteria can be used in con- junction with the planning phase steps presented in NCHRP Report 451, Chapter 4, “Guidelines for Warranty, Multi- Parameter, and Best Value Contracting.” NCHRP Report 451 presents planning phase guidelines that detail the steps an owner should take to begin a best-value program and select a pilot project. Key steps presented in that report include • Determine the agencies current level of experience with best value and • Determine the motivation for implementing best value. The guidelines presented in NCHRP Report 451 make a few key recommendations. First, the report recommends that new users determine the motivation for implementing best value as well as review and understand best practices for best-value contracting. The report specifically addresses these two rec- ommendations. Second, NCHRP Report 451 recommends that low-to-moderately experienced users (one to five proj- ects) obtain input from industry in deciding how to proceed and select pilot projects to test and measure the performance of the best-value system. Users are strongly recommended to refer to NCHRP Report 451 concerning the implementation and evaluation of this second recommendation. Cost versus Benefit Best-value procurement is not currently the highway industry’s standard way of doing business. The decision to use a non-traditional procurement for a particular project should be based on an analysis of the value to be added to the proj- ect, recognizing that the system may create additional costs such as increased agency staff time and industry proposal preparation time, as well as the possibility of higher initial construction prices (due to costs incurred by the contractor to achieve the best value) compared with traditional design- bid-build procurements. Best-value procurement can add significant value through cost savings, time savings or quality enhancement directly. Although best-value procurement allows the contracting agency to take the contractor’s experi- ence and reputation into account in the selection process, that capability should not be the sole basis for a decision to use the process, and the analysis that is the basis for the decision to use the procurement system should focus on projected added value. The potential costs and benefits projected to result from a best-value process should align with the project goals, and those goals must be communicated to the contracting com- munity for the procurement to be successful. One important early step in project planning is to determine the project goals. It should also be noted that changes in agency policies that have the effect of changing the predetermined project- specific goals will adversely impact the chances of project success. The decision to use best-value procurement should be based on projected benefits to the project. Again, best- value procurement is not currently the business paradigm in the U.S. highway industry, and for a decision to use this process to gain acceptance, it is advisable for agencies to communicate their reasons for using the system to industry and other interested parties. Similarly, to encourage compe- tition and also to obtain responsive proposals meeting the agency’s needs, the basis for making the best-value determination should be clearly stated in the procurement documents. The remaining sections of this chapter elaborate on possi- ble screening criteria for selecting projects for best-value pro- curement, starting with the potential costs and benefits associated with implementing this procurement methodol- ogy. Other criteria considered include opportunities for innovation, specialization requirements, and risks in pro- curement. Appendix F contains a flowchart and a best-value project selection tool intended to facilitate the decision- making process.

64 Potential Costs Agency Staff Time Where traditional procurement employs a responsibility standard for qualification of contractors (addressed through prequalification in some states and in others through review of qualifications after bids are received) and a project-specific evaluation of bids, best-value procurement requires a project- specific evaluation of contractor qualifications and price pro- posals. Best-value procurement may also involve a technical evaluation of proposals when the contractor’s scope includes some element of design. These evaluations may require the agency to assemble a project-specific team to evaluate the offeror’s proposals. The cost of developing the evaluation plan and implementing the evaluation process itself are added costs on each project. Staff training may also be an additional cost. Best-value procurement is similar to existing methods of a qualifications-based consultant selection. The staff must have training or experience in qualifications-based selection for the procurement to be successful. Additionally, when technical proposals are part of the procurement, the evaluators must have design review experience. Industry Preparation Time Depending on the nature of the project and process, best- value procurements can be costly and burdensome for the industry. Contractors who participate in traditional procure- ments typically pay for the costs of annual or periodic prequalification and bidding in their general overhead. Preparation of project-specific qualifications responses and technical proposals are not typically part of their overhead. Highway agencies must be conscious of what their best-value procurement requirements will cost the proposers, and they must strive to keep these costs to a minimum to maximize the level of competition. When using best-value procurement on complex design-build projects with significant proposal preparation costs, owners often include a proposal prepara- tion stipend to offset the industry proposal costs (Smith and Ryan 2004). In some states, these stipends have been used as consideration for innovative concepts included in proposals submitted by the unsuccessful proposers. In best-value design-bid-build or smaller design-build procurements, stipends are typically not applied. Potential Benefits Cost Savings Cost savings stemming from best-value procurement can be difficult to measure and predict at the time of procure- ment. By definition, best-value procurement can provide justification for choosing a proposal that is not the low bid, thereby increasing initial construction costs. Initial construc- tion costs, potential cost growth after award, and life-cycle costs should all be considered in examining the potential best-value procurement cost savings. Initial construction cost savings will only be realized when there is an opportunity for contractors to save money through schedule compression, the application of innovative means and methods, or more con- structible designs. Adding best-value selection criterion such as an exemplary safety record, past performance, and man- agement plans may in fact add to the initial construction price. By nature, contractors with better qualifications are likely to spend more on safety and management practices. However, use of these evaluation factors may ultimately result in lower cost growth as a result of better management and fewer bidding errors. A best-value process may also include evaluation factors encouraging lower life-cycle costs through higher quality construction or through the inclusion of a life- cycle analysis in the best-value procurement solicitation. Time Savings Best-value procurement allows for the evaluation of time in procurement. Traditional procurements ask for prices based on fixed project start and finish dates. Best-value pro- curement can reward the contractor for bidding a shorter construction schedule, thus allowing the contractor to deter- mine the optimum schedule with reference to its increased costs of accelerating the project. There is a potential for an increase in initial construction costs because of the acceler- ated schedule—although to some extent the increased costs of acceleration will be offset by the reduction in overhead. In addition, the bidder with a higher initial construction cost can be selected if the time savings is determined to be more valuable than the cost increase on the basis of user costs or agency overhead costs. However, it is also possible that the owner will receive both the lowest initial cost and the short- est schedule from the same contractor. In either case, if the agency is interested in accelerating the schedule, it must be willing to take steps to remove constraints that are likely to impact the critical path for the project. Quality Enhancements Quality benefits can be even more difficult to measure than cost or schedule benefits. Agencies should strive to include those quality enhancements that are easily convertible to a measurable dollar benefit, such as improved design resulting in lower operations and maintenance costs. While there is a belief that more stringent quality control plans, more com- prehensive safety plans, better past performance, better per- sonnel or better management plans will improve the project’s

65 safety record and result in higher quality construction, those benefits are difficult to correlate to specific performance out- comes. This does not mean that these items should not be included in a best-value procurement, it just means that agen- cies must apply these criteria prudently. Key Project Characteristics to Consider As previously stated, best-value procurement is not the highway industry’s standard way of doing business. A whole- sale change to best-value procurement in the highway indus- try is not feasible or prudent given the industry’s long ties to the low-bid method. The choice to use best-value procure- ment should be made judiciously on a project-by-project basis. Key project characteristics should be considered when making the decision to determine whether best-value pro- curement is appropriate for a given project. Agency Staff Capacity and Experience Qualifications and availability of agency staff are a key project characteristic that must be examined when consider- ing best-value procurement. As previously stated, best-value procurement can require more staff time and a different level of training and education than traditional procurements. Staff considerations are particularly important when the pro- curement requires an evaluation team or design review. Market Capacity and Experience Best-value procurement requires contractors to prepare proposals that include details of schedules, qualifications, management plans, and even designs. Contractors must have the capacity and skills to develop these proposals. Contractors that have only performed work for the agency based on a low- bid selection process will not have this experience and, there- fore, will need to make a greater investment in responding to a best-value RFP. As contractors gain experience, this process will become less burdensome, but highway agencies must be cognizant of the level of effort required to respond to a best-value RFP when selecting projects for best-value con- tracting. Some best-value procurements, particularly those involving designs, will require that contractors carry different insurance or obtain different surety bonds. Market factors affecting the ability of contractors to obtain such insurance and bonds should also be considered when making this deci- sion to use best-value procurement. Project Complexity Project complexity will impact the possible benefits resulting from best-value procurement. Project complexity primarily stems from technical complexity or management complexity. In either case,best-value projects can offer opportunities for added value because the contractor can bring its knowledge and expertise to the project. Complex projects seem to offer more opportunity for benefit from best-value procurement. Relatively simple projects can also benefit from best-value procurement, but the benefits might not be as significant. Numerous federal agencies use best-value procurement proj- ects with all levels of complexity, including simple projects. Benefits for such projects can be realized in areas such as past experience, quality plans, and safety. Drawbacks associated with use of best-value procurement for a simple project include the decreased opportunity for participation by smaller and less experienced contractors and the fact that best-value procurement can be administratively burdensome for the highway agency and the industry. Using best-value procure- ment on less complex projects should be tempered with sound judgment concerning its effects on open competition and the administrative burden on the procurement process itself. Quality Requirements Where low-bid methods typically only stipulate minimum quality requirements through contract specifications, best- value procurement allows for quality-related elements to be included as part of the competition. Quality management plans and tighter tolerance on materials or end products are two examples of items that can be factored into the evalua- tions. Through competition, higher quality may be achieved at the same or even lower costs. Furthermore, even though the initial costs may be higher, the life-cycle cost may ultimately be less than the life-cycle cost that would have resulted from a low-bid procurement process. Opportunities for Innovation Best-value procurement offers a framework for agencies to take advantage of innovative proposals from the industry. These innovations may result in cost savings, time savings, or even higher quality products. In their simplest form, these innovations may be contractor traffic maintenance plans or construction schedules. At the other end of the spectrum, the innovations may come in the form of design-build delivery with the industry completing more than 80% of the design. When projects have elements that can be precisely defined through measurable performance outcomes, they may be suitable for design-build delivery. Specialization Requirements Projects that require specialized equipment, knowledge of construction, or exclusive technology are ideally suited to

66 best-value procurement. Specialized requirements occur when there are highly unique aspects to a project. Adding qualitative factors for these projects can result in higher qual- ity projects or projects that require less rework due to con- tractor inexperience. Small but highly specialized contractors could likely see a benefit from best-value procurement. In fact, a number of the federal case studies conducted for this report used best-value procurement to speed the process of hiring disadvantaged businesses (DBEs). However, FHWA’s current policy is that achievement of DBE goals or good faith efforts to achieve them should be considered as a pass/fail cri- terion and not considered in the best-value evaluation, although past performance with respect to use of DBEs could presumably be considered. Risk in Procurement A best-value procurement system can increase the likeli- hood that the contractor will successfully perform the work (known as performance risk). However, it also creates a risk relating to the ability of the evaluators to properly evaluate a contractor’s proposal, known as proposal risk (Army Source Selection Guide 2001). Each project will have characteristics that create risks in procurement. An attempt should be made to select projects with minimal best-value procurement risks. Additionally, selection of appropriate evaluation criteria can help to mini- mize these risks. Best-Value Project Screening Decision Flowchart and Selection Tool A flowchart is presented in Appendix F to describe the project screening process. Successful navigation of this decision flowchart allows the user to proceed to the next step in best-value procurement, which entails the actual selection of projects. There are a number of critical decisions in the planning stages of a project that must be made before a best-value procurement can commence. The flowchart allows for quick identification of the critical deci- sion points and provides advice regarding how to proceed if fatal flaws to the process are discovered. A user may only need to refer to the decision flowchart on the first few proj- ects because it primarily deals with organizational and political hurdles that must be overcome. Once these pro- grammatic barriers have been overcome, the user will be able to “shortcut” the flowchart and proceed directly to the project selection tool. The project selection tool, also included in Appendix F, further guides the user’s project selection process. Please note that the project selection tool is also available electronically at http://construction. colorado.edu/best-value. 3.6 Implementation Strategies Even the best and most convincing research will not suc- ceed in the implementation phase if it does not adequately address the concerns of the owner’s organization and achieve industry support. For this to happen, all parties must perceive that a best-value procurement system will articulate common objectives; be advantageous to owners and bidders; and be legal, practical, impartial, and relatively simple to implement. Furthermore, the research results must be structured in a way to clearly and convincingly communicate the advantages (or disadvantages) of a particular approach. The first step is to identify and understand barriers to implementation and then devise effective strategies to overcome these barriers. Legal and Regulatory Considerations As described in Chapter 2, at both the federal and the state levels, legislation has moved toward increased acceptance of alternative procurement practices using best-value selec- tion. However, the laws are far from uniform. Each agency must carefully examine its enabling authorization in deter- mining how to proceed with a best-value procurement. In addition to reviewing the Model Code and statutes identi- fied in Appendix B, agencies wishing to obtain general best- value legislation may want to review the enabling legislation allowing use of design-build for transportation projects. A survey of design-build legislation can be reviewed at http://www.nossaman.com/db30/cgi-bin/news/NCS_BJD_ 50%20State%20Survey%20of%20Design%20Build%20Aut hority_4.20.06.pdf. The federal best-value process has been in place much longer than similar processes at the state level. The FAR 15, Contracting by Negotiation, sets forth best-value concepts under a competitive acquisition. Best value under the source selection process might entail the tradeoff of weighted factors or selection of the lowest-priced technically acceptable pro- posal. Excerpts from the FAR 15 are provided in Appendix B (FAR 2004). The FAR 15 process is available for all types of contracts. Many federal agencies have implemented compet- itive negotiation or design-build and have developed instruc- tions or procedures for development and implementation of these methods. For example, the U.S. Postal Service, the U.S. Army Corps of Engineers, the Navy, the Department of Vet- erans Affairs, the Federal Bureau of Prisons, and other agen- cies have developed procedures and guidelines for source selection and design-build contracting applicable to their construction programs. The federal government has imposed certain procurement restrictions on state and local agencies wishing to use federal- aid funds to pay for transportation infrastructure. For many years, federal law mandated that construction of federal-aid

67 projects be undertaken “by contract awarded by competitive bidding”(see 23 U.S. Code § 112(b)), unless FHWA approved use of an alternative procurement process. FHWA’s SEP-14 program has been the vehicle for such approvals. In 1998, TEA-21 created an exception to the general competitive bid- ding requirement, authorizing use of best-value procure- ments for federal-aid design-build contracts over a specified dollar amount (TEA-21 1998). In 2002, FHWA issued regula- tions establishing the procurement process to be followed for such projects, thus avoiding the need for agencies to obtain SEP-14 approval to use a best-value procurement process for such projects. FHWA’s design-build rule includes best-value procurement requirements that are based on FAR 15. It should be noted that the competitive bidding requirement remains in effect with respect to federal-aid construction con- tracts that do not meet the TEA-21 definition of qualified design-build projects, unless FHWA approves an alternative process. It should also be noted that federal permission to use a best-value procurement process for federal-aid contracts does not constitute enabling authorization for state and local agencies wishing to use such a process. Enabling authoriza- tion must be provided by state and local legislative action. On the state and local levels, until recently most agencies have been subject to legislatively imposed requirements that construction contracts be awarded to the lowest responsible bidder after the project is fully designed. These statutes do not expressly prohibit best-value selection, but are inconsistent with use of any selection factors other than responsibility of the bidder, responsiveness to the procurement requirements, and price. One question that has been the subject of argument and case law going both directions is whether A+B bidding is consistent with a statute requiring award to be made to the low bidder. The consensus in the industry (notwithstanding case law in at least one state to the contrary) is that cost-plus- time bidding is consistent with a requirement to award to the lowest responsible bidder. During the past decade, a legislative trend has emerged to permit use of best-value procurement by state and local agen- cies for the reasons described in this report. Many agencies have been granted specific authority to procure design-build contracts on a best-value basis. In addition, the desire to unequivocally allow use of A+B bidding and to incorporate other best-value elements into the selection process for con- struction contracts has led to more general legislation allow- ing best-value procurement to be used in selecting any contractor provided the decision to use it can be justified. In 2000, after many years of research, analysis, and discussions, the ABA issued a revised Model Procurement Code that can be used as the basis for legislative changes. As described in Section 2.2, the Model Code allows use of a “competitive sealed bidding” process so that the project owner can award to the responsive bidder who provides the lowest priced bid (i.e., using the meets technical criteria—low-bid algorithm) or take costs outside of the bid price into account in making selection decisions with award made to the bidder who pro- vides the proposal that results in the lowest cost to the agency (using the meets technical criteria—low cost algorithm). If the agency determines that competitive sealed bidding is impracticable, it can use a competitive sealed proposal process with any of the other six algorithms. In drafting legislation for the purpose of allowing trans- portation agencies to use a best-value procurement process, the drafter must consider the needs of the public agencies and public policy considerations. On the one hand, public agen- cies procuring contracts on a best-value basis will need flexi- bility to adapt the procurement process for a wide variety of projects and circumstances. From a public policy perspective, however, it is advisable to include certain requirements to ensure that the selection decision will be made rationally and without favoritism, and as a result, the legislature will typi- cally include requirements in enabling legislation regarding the procurement process to be followed. On one end of this legislative spectrum, a statute might grant contracting authority to the agency without imposing any restriction on procurement methodology. At the other end of the spectrum, legislation may impose requirements so cumbersome that it is unlikely the process will ever actually be used. In some cases, these requirements may be included in the original bill based on the author’s belief that they will be helpful. In some cases, they may simply have been carried forward from a prior bill without further analysis. Often such requirements are the result of compromises necessary to obtain passage of the bill. In the middle of the spectrum is legislation based on the Model Procurement Code published by the ABA. The Model Code has been used as the basis for legislation in a number of states and establishes a framework for best-value procure- ments consistent with public policies while allowing the agency significant flexibility to address its needs with respect to individual projects. It should be noted that most states have adopted separate enabling legislation for their DOTs. As a result, even though a particular state may have adopted legis- lation based on the Model Code, that authorization may not necessarily extend to the DOTs. Refer to Appendix B for a list of eleven states that may have best-value authority for con- struction contracts not using design-build. As previously noted, a number of states have adopted legislation specific to their DOTs allowing use of best value for design-build pro- curements, and some states have adopted best-value legisla- tion for other types of contracts. Institutional and Industry-Related Issues As the transportation industry has gained more experience in the use of best-value selection within traditional low-bid,

68 design-build, and negotiated procurements, concerns and questions have been raised by participants from the owner and industry perspectives that must be addressed before best- value procurement will be widely supported and imple- mented. Some of the issues relate to the procurement process itself while others address the effect of incorporating addi- tional selection parameters (time, quality, or other factors) on construction: • It is necessary to establish criteria and a decision-making process to determine if a project is a viable candidate for best-value procurement. • If the best-value selection process is administratively bur- densome, it will not sustain support from the owner organ- ization tasked with administering it. • If best-value procurement requirements are too time con- suming and costly, it will discourage smaller or DBE con- tractors with limited resources from bidding and reduce competition. • If the selection parameters are not clearly defined or are overly subjective, the owner risks that awards will be chal- lenged, delaying or negating the award. • It is necessary to determine appropriate pass/fail criteria or factors and to consider under what circumstances they should be used. • Under best-value procurement, a higher initial cost for the same work procured under low bid will discourage wide- spread implementation unless the additional value received can be reasonably determined. • The selection process must be structured to limit the num- ber of qualified bidders, yet allow sufficient competition. • Procedures must be established to maintain confidential- ity and to document the evaluation process. • Concerns regarding the subjectivity inherent in a best- value selection process make contractors reluctant to par- ticipate in best-value procurements. • The use of alternate bids or design alternates in the context of competitive bidding in the United States is limited to specific material or equipment items, pre-engineered items, and specific construction processes rather than com- plex designs such as buildings or bridges. • The Associated General Contractors of America has expressed a strong preference that highway agencies con- tinue to award highway construction contracts on a low- bid basis. • Accelerated schedules and extended overtime associated with some A+B projects challenge agency and contractor resources, raising concerns about reduced quality and safety. • Multi-parameter or A+B bidding for time shifts more responsibility and risk for estimating time to the contrac- tor. As a result, the owner may incur higher bid prices reflecting the more aggressive schedule or increase the risk of delay claims. • Contractors have expressed concern that multi-parameter bidding may result in a contractor submitting unrealisti- cally low numbers for time or high numbers for quality to be more competitive. • Uncooperative third parties have the ability to “throw a monkey wrench” into plans to accelerate the project sched- ule. As a result, even though an innovative procurement methodology may result in an accelerated completion deadline, the accelerated schedule may be delayed. • Some contractors have expressed concerns that warranty projects will tie up funds and reduce bonding capacity for extended durations. • Industry organizations often have opposed warranties, because they would impose greater hardships and risk on small engineering and construction firms, because con- tractors would be held responsible for designs they did not create and because such firms have no control over future uses of the highway or other conditions that might give rise to a warranty claim. Legal protests have arisen on best-value projects that involve one or more of the noted issues. These highlight pit- falls of implementation and serve as lessons learned to guide future implementation. The following case study, involving a recent federal best-value procurement, illustrates how an arguably subjective best-value selection criterion can raise concerns and potentially give rise to legal protests, delaying or derailing the procurement process. The Butt Construction Case A Protest was filed by Butt Construction Company, Inc., (Comptroller General No. B-284270, March 20, 2000) for the renovation of the Avionics Research Laboratory at Wright- Patterson Air Force Base using best-value selection (Scott and Geisen 2002). The RFP contemplated award to the firm offer- ing the best value to the government. Price was given equal weighting with a combination of technical factors listed in order of importance as follows: 1. Qualifications and experience 2. Design and engineering 3. Project management Five firms submitted proposals and were found to be in the competitive range. The technical evaluation panel scored the most significant technical factors in the proposals as shown in Table 3.12. Offeror C had the top-rated technical proposal, but its price was $1.5 million higher than Monarch’s second ranked

69 technical proposal. Butt had the lowest price, but was ranked fourth in the most important technical area, qualifications/ experience. The panel concluded that Offeror C’s technical proposal did not offer enough advantages over Monarch’s technical proposal to justify award to C. The panel also found that the technical strengths of Monarch’s proposal offset the $239,000 difference in price between Monarch and Butt. Monarch had demonstrated a significant amount of prior work experience and had completed several recent projects of similar scope, size, and complexity. Based on this evaluation, the panel recommended Monarch for award. The source selection authority agreed with the recommendation. Butt challenged the agency’s price/technical tradeoff, argu- ing that although Monarch’s scores may have been higher, the evaluators did not find that these scores were indicative of technical superiority justifying the price premium. The Comptroller General rejected this argument, noting that source selection officials have broad discretion to determine the manner and extent of technical and price evaluation results under a negotiated procurement: In deciding between competing proposals, price/technical tradeoffs may be made; the propriety of such tradeoffs turns not on the difference in technical scores or ratings per se, but on whether the source selection official’s judgment concerning the significance of that difference was reasonable and adequately jus- tified in light of the RFP evaluation scheme. ...The discretion to determine whether the technical advantages associated with the higher-priced proposal are worth the price premium exists notwithstanding the fact that price is equal to or more important than other factors in the evaluation scheme. Looking at the record of decision, the Comptroller General found that Monarch’s experience was sufficient to justify the higher price and that the technically superior proposal inher- ently would result in superior performance. This example highlights the issues faced by public sector owners attempting to move from a strictly lowest cost selec- tion process to one evaluating price with other technical fac- tors. While, in this particular case, the decision affirmed the selection committee’s use of “broad discretion” in evaluating technical and price tradeoffs consistent with federal procure- ment rules, this discretion is often the source of disputes related to the process. Similarly, controversy regarding the selection process may stem from the perception that adding parameters to the bid price representing the value of time or improved quality is a departure from competitive bidding or may increase project risks. Issues may also arise with respect to evaluation of bid alternates as part of a competitive bidding process as exem- plified in the following case (Scott and Geisen 2002). White Contracting Case In September 2000, Massport solicited bids for the renova- tion of the Maurice H. Tobin Memorial Bridge in Boston. Massport invited bidders to submit alternative bids based on the use of “type 5” cement concrete and “silica fume” con- crete, reserving the right to award the contract based on the alternative that was “in the best interests of Massport.” J.F. White Contracting Company was the low bidder for the type 5 cement at $6,443,912. DeMatteo was the low bidder for the silica fume concrete at $6,455,174. Massport selected the silica fume concrete alternative because of its superior anti-corrosive properties and awarded the contract to DeMat- teo, even though its bid was higher. White filed suit seeking a preliminary injunction prohibiting Massport from proceed- ing with an award of the contract to any contractor other than White. White asserted that under Mass. Ann. Laws, Chapter 30, §39M (2000), it was entitled to an award of the contract because its bid on the type 5 concrete was lower than DeMat- teo’s bid on the silica fume concrete. Mass.Ann. Laws, Chapter 30, §39M (2000), governing com- petitive bidding practices for public works contracts requires that every contract for construction and repair shall be awarded to “the lowest responsible and eligible bidder on the basis of competitive bids publicly opened and read.” The pur- pose of this statement was to “create an open and honest com- petition with all bidders on an equal footing, and to enable the public contracting authority to obtain the lowest bidder.” Table 3.12. Comparison of technical scores. Technical Score Offeror Qualifications/Experience Design/Engineering Total Score Maximum Points 8,000 3,600 11,600 Butt 4,500 2,405 6,905 Monarch 5,380 2,120 7,500 Offeror A 5,090 2,060 7,150 Offeror B 4,090 1,825 5,915 Offeror C 5,380 2,475 7,855

70 The Superior Court judge denied White’s request, on the merits of the claim “since all bidders were afforded full com- petition as to the two alternatives and there was no claim of improper favoritism.” The Appeals Court affirmed the deci- sion noting “there is no language in §39M which prohibits a public authority such as Massport from using the type of alternative bidding procedures at issue in this case, so long as it accepts the lowest bid for the alternative ultimately selected.” The award to DeMatteo was acceptable because it was the lowest bidder on the silica fume concrete alternative. The Appeals Court found that Massport did not violate §39M in awarding the contract, fully defined both alternatives in the bid solicitation documents, and made it clear that “a bidder could be assured of an award of contract only if it submitted the lowest bid on both alternatives.” This case provides an example of a procurement process that gives the owner discretion to determine that the higher cost alternative is more advantageous, without any require- ment to specifically quantify the cost benefit of that alternative in terms of reduced maintenance or other savings. Although no improper favoritism was proved in this case, it is apparent industry is concerned about the possibility of abuse in this type of process. A requirement for the owner to quantify the benefits in connection with the decision to select a particular alternate (or provide a life-cycle cost adjustment factor) would help to avoid the use of alternate bids as a means of circum- venting the low-bid process. Another approach used by some owners is to have officials make the decision regarding selec- tion of the alternative based on the prices provided, without knowing which bidder supplied which price. Implementation Strategies As part of a comprehensive implementation plan, the issues and questions raised by industry must be addressed. Past research addressed critical success factors for implemen- tation of proposed contracting methods and quality-based rating systems. NCHRP Report 451 contained guidelines for implementing three contracting methods. These guidelines addressed implementation in general terms. They cited the importance of senior management support, more up-front investment by the agency, communication, training, appro- priate project selection, and industry buy-in. All of these factors are important contributors to the success of imple- mentation. However, to develop workable implementation strategies, more specifics are needed regarding an approach for implementing best-value procurement within a tradi- tional contracting environment, such as guidelines for leg- islative reform, sample best-value language, training tools, and steps to achieve industry acceptance. Strategies prepared for particular procurement approaches that are not clearly defined will be speculative at best. However, based on success with moving innovative ideas, policies, and procurement and contracting approaches into practice, a number of strategies can be identified and dis- cussed at this stage. Some of these strategies are identified in NCHRP Report 451 and in the NCHRP Project 10-54 final report. The following elaborates on some of the steps that are likely to be necessary to move the results of this research into practice. Step 1—Clearly communicate the results and products of the research and advantages of implementing it, and enlist champions to promote its use and test its effectiveness The research results must address the relative advantages of best-value procurement and communicate these results to members of the implementing organizations and to industry as a whole. This report provides the background information needed for stakeholders to appreciate the advantages of best- value procurement, the challenges and concerns raised by industry related to its use, strategies to address these concerns, a decision framework for selectively implementing best-value leg- islative guidelines and model provisions. The research findings as a whole have shown that best-value procurement has resulted in improved performance and that industry perceptions to the contrary may reflect a lack of experience. Concerns regarding increased likelihood of protest can be countered by experience in the federal sector that the likelihood of a successful protest is reduced when the more advanced processes are used. Concerns regarding the additional burden placed on staff during the pro- curement process can be offset by the reduced burden on staff during the contract administration phase. Implementers should also consider the following as part of implementation: 1. The implementation process presented in this report allows for maximum flexibility in the design of the best- value procurement system to accommodate the different types of projects and different experience levels of the agency and industry stakeholders. An agency implement- ing best-value procurement for the first time would be more inclined to select a system, for example a one-step meets technical criteria—low-bid system, more closely aligned with its traditional procurement process. With more experience, the agency might move to a more sophisticated or complex best-value model similar to the approaches used by the federal agencies with significant best-value experience. This flexible and graduated approach will increase the likelihood that agencies will experiment with best-value procurement. 2. The procurement policy should clearly require that the criteria used for technical evaluation, the weighting or

71 relative importance of each criterion (including price), the rating system, and the award algorithm be clearly defined in the procurement documents. This creates a level play- ing field, reduces the uncertainty related to the selection process, and focuses the proposers on what is most impor- tant to the agency. 3. As noted in the research findings, it is advisable to use selection criteria that are important, add value, and relate to desired performance. For example, if time performance is a critical criterion, proposers can be asked to provide a completion date that meets or beats the owner’s estimated completion date. If the owner wishes to use past perform- ance as a selection factor, it may wish to consider asking for information regarding specific performance measures in terms of issues such as cost control, rate of progress, qual- ity in terms of degree of conformance with specifications or standards, and safety in terms of accidents or lost work- days. Selection criteria that include factors that are diffi- cult to assess or do not directly relate to the performance goals of the agency result in procurements that are overly complex and should therefore be avoided. 4. Under low bid, or a one-step meets technical criteria— low-bid (cost) award process, it is often advantageous to establish pass/fail or minimum performance criteria to determine bidder responsibility and whether the bidder’s technical proposal is responsive. Responsibility can be addressed through prequalification, whether through a blanket prequalification for multiple projects, or through prequalification/shortlisting as the first step of a two-step, best-value procurement. Pass/fail criteria relating to responsibility might include a specified number of years of specialized expertise, demonstrated quality levels for sim- ilar projects, and a minimum safety rating. Technical pass/fail criteria are tied to the responsiveness determina- tion and could include matters such as provision of meet- ing a schedule or bettering certain milestones. It is also possible to require a bid to be within a competitive pricing range to be considered responsive. 5. When performing a best-value tradeoff analysis to justify award to other than the lowest priced offeror or other than the highest technically rated offeror, systematic compar- isons of price and technical criteria should be conducted. The federal procurement model requires that the owner advise the proposers regarding the relative importance of the evaluation factors. Furthermore, the rationale for the decision, including benefits associated with the additional costs (or reduced costs), must be documented, although the tradeoffs that led to the decision are not required to be quantified. 6. It is useful to conduct pre-proposal conferences and debriefings to clarify potential ambiguities in the solicita- tion documents. Interested parties should be offered the opportunity to submit questions in advance of the pro- posal due date. Questions should be answered in writing and provided to all proposers. One or more pre-proposal meetings can be held to answer questions or clarify aspects of an RFP. For a best-value procurement involving the consideration of complex technical criteria, this initial opportunity to request clarification is even more critical. Additionally, it should be noted that the rules applicable to procurements by federal agencies require notification to unsuccessful offerors and allow for pre-or post-award debriefings if requested by the offeror. A debriefing is also strongly recommended for best-value procurement at the state and local levels as well to further clarify the basis for award, the selection process, and the rationale for elimi- nating the offeror, if this was not apparent in the written notification of contract award. 7. If the project complexity and objectives require a more intensive effort to respond to a best-value proposal, par- ticularly one involving work product such as alternative designs or technical solutions, the agency should consider payment of a stipend. Although various owners choose to refer to the payment as stipend, stipulated fee, honorar- ium, and so forth, the basic premise is that the proposer will be partially compensated for its costs of preparing the proposal. There is no set range of values for this payment for work product fee. The amount of the fee can be estab- lished based on the project budget, the estimated proposal costs, the estimated construction costs, or some other basis. Payment of compensation to the responsive pro- posers can be an effective means of retaining contractor interest in the procurement and encourages preparation of quality proposals. Step 2—Devise solutions to legal barriers and procurement regulations Agencies interested in gaining the potential benefits from implementing best-value procurement must identify and analyze laws and rules affecting the agency that would limit or prevent its use. Depending on the results of the analysis, the implementation of a best-value procurement may start with crafting solutions to legal barriers. The trend toward greater use of best-value procurements has yielded a number of statutes and rules incorporating best-value concepts. The research team recommends that the Model Code and model regulations associated with the Model Code be used as the starting point. In theory, the fact that best-value procurement has been generally authorized for use by federal agencies and various state transportation agencies should make it easier to obtain legislation in the remaining states, but in practice it will probably be necessary to “reinvent the wheel” every time

72 new legislation is desired, due to the need to educate the leg- islature regarding best practices in public procurement as well as the need to deal with interest groups that are opposed to any change in the existing procurement requirements, not to mention that decisions to vote for or against a particular bill are often wholly unrelated to the subject matter of the bill. The first step in obtaining new legislative authorization is to develop draft language producing the desired result. This will entail review of the agency’s existing authority as well as examples of comparable legislation passed in the state in question, and review of legislation in other states. The agency should enlist the aid of its attorneys in drafting the bill as well as involving its legislative liaison. Once the proposed language passes muster within the agency, it will need to be submitted to a legislator for intro- duction. The language proposed by the agency will be reviewed by the legislator’s staff and may be revised prior to introduction. The agency’s legislative liaison will be respon- sible for obtaining information from staff and ensuring that any changes are reviewed by appropriate agency personnel. The proposed language could be introduced as a stand-alone bill, or could be appended to an existing bill involving a sim- ilar subject matter. In some cases, the proposed language may entirely replace the provisions in a previously proposed bill. The process for introducing, amending, and passing legisla- tion varies from state to state, but will always involve oppor- tunities for interested parties to propose modifications. Again, the agency’s legislative liaison will need to pay close attention to proposed modifications and must ensure that any changes are reviewed by appropriate agency personnel. It may be advisable for agency staff to meet with interest groups seeking changes to the bill, particularly if they have the abil- ity to “kill” the bill, to try to reach a compromise acceptable to the agency that will allow the legislation to proceed. The agency’s legislative liaison will need to pay attention to the legislative calendar and take appropriate steps to ensure that all required actions relating to the bill are timely. The process to be followed in adopting implementing reg- ulations will be simpler since they do not require legislative approval (although it is likely that one or more politicians will be contacted by interest groups if they have any objection to the regulations). However, the same general concepts apply. Agency staff and attorneys will need to draft the regulations; following initial publication the agency will receive com- ments and decide how to address them; after the comment period ends, the agency will issue the final regulation. Specific procedures will vary state by state and agency by agency. Widespread implementation of best-value procurement will require creative and flexible solutions to legal and pro- curement-related barriers. Appendix G includes a matrix identifying legal, regulatory, social, and business barriers, indicating the level at which each barrier must be addressed, possible solutions to each barrier, and an estimate of the probability that each barrier can be solved without legislative restructuring. Step 3—Training Training is an essential tool to formally communicate changes in policies to a wider audience as part of implemen- tation. Training ideally should include owner and industry members in the process. The process should • Introduce the basic concepts to agency and contractor per- sonnel. • Be concise and clearly communicate the new procedures and the relative benefits of implementing them to all stake- holders. • Address methods of selecting projects, parameters, and best-value procurement systems. • Provide guidance for evaluation and scoring of technical proposals. Ensure that there is a consistent scale for scoring and that all scoring officials understand the scale. A simple example of this concept is that all officials must agree that an average score is 50 out of 100 points or 70 out of 100 points when using a direct point scoring system. Training has the added benefit of recruiting additional champions to further promote and implement the proposed changes. The summary results of the study can be incorpo- rated into an introductory training package consisting of the training tool shown in Appendix H. Step 4—Collaborate with industry in the implementation process The successful implementation of best-value procurement practices must include industry participation and comment; thus, it is prudent to reach out to owner and industry members affected by the change, explain the proposed changes, and obtain their insights, concerns, and ideas regarding the process. There are a number of reasons for this. Primary among these is the recognition that there will always be opposition to change. For example, strong industry opposition exists with regard to certain innovative procurement practices. If stakeholders are serious about implementing the results of research, then the implementation plan must provide the implementers and champions with the tools they will need to push through change. These tools include collaboration with industry. The research team has consulted with its advisory board, particularly members representing industry organizations, regarding strategies to build industry support. Their feedback included recommendations for agencies to incorporate the following into their best-value contracting programs:

73 1. Identification of common objectives and advantages for best-value procurement; 2. Analysis and allocation of risks in the procurement process; 3. Involvement of an owner and industry task force in the development and review of proposed legislation or pro- posed best-value procurement procedures; and 4. Involvement of owner and industry team in testing the new approach through a pilot or demonstration project. Step 5—Pilot projects Pilot projects are a proven tool for validating and fine-tuning new practices resulting from research. Using traditional proj- ects as a benchmark, pilot projects or programs have been used extensively to measure the relative success of new procurement and contracting methods. The results of pilot projects, though in some cases difficult to attribute to one specific cause, have served to effectively promote the long-term implementation of new industry practices. It is recommended that an agency champion the use of best-value procurement through a pilot program, partner with industry in testing various best-value systems, and develop criteria to measure its relative success compared with traditional low-bid projects. The project screening and selection tool developed for the implementation of best-value procurement can be used by agencies to identify those projects that will make good pilot test beds and will furnish the project performance metrics that can be used to evaluate the results of the local pilot proj- ect program against a baseline of traditional projects. It is essential for the agency to maintain a long-term commit- ment, providing ongoing technical and troubleshooting sup- port, and adjust and revise procedures as appropriate to overcome recognized problems and pave the way for more widespread implementation. Typically, institutionalizing the process through the development of appropriate governmen- tal and private support groups or associations, annual con- ventions or meetings, websites, and regular periodicals will facilitate long-term support. 3.7 Model Best-Value Specification This model specification represents a framework for the development of best-value procurement specifications. This document should be considered a template. To integrate these specifications into a proposal, special care must be exercised to ensure compatibility with the agency’s standard specifica- tions, especially the General Provisions. For more complex procurements, it is highly advisable to include separate Instructions to Bidders instead of incorporating procurement and award requirements in the standard specifications. This model specification refers to the AASHTO Guide Specifica- tions for Highway Construction (AASHTO 1998) where appropriate. Under each of these sections, options, insertions, or alternate approaches are italicized. XXX.01 General/Description. A. The Agency is using a best-value procurement procedure to select the Bidder that will be awarded the Contract. The selection process will take into account the price offering and other factors that the Agency considers essential to the successful performance of the work. In addition to price, best-value parameters will include [insert addi- tional best-value parameter(s) based on project objectives identified in the project screening process. Parameters may include time, qualifications, quality, design alternates, or some combination of these factors aligned with the project objectives.] B. Refer to Section XXX.05 for the Agency’s evaluation plan, criteria, and selection method. C. This procedure consists of a [insert one-step or two-step] procurement process. Refer to Sections XXX.03 and XXX.05 for detailed requirements. XXX.02 Definitions and Terms. The following definitions are added to Section 101.03 Definitions: A. Best value—a procurement process where price and other key factors are considered in the evaluation and selection process to minimize impacts and enhance the long-term performance and value of construction. B. Parameters—categories describing the Agency’s procure- ment objectives in terms of cost or time savings, qualifica- tions, or quality enhancements. Parameters are expressed as cost, time, qualifications & performance, quality, and design alternates. C. Evaluation Criteria—those factors associated with each best-value parameter that will add value to the procure- ment and will be used to systematically evaluate proposals as part of the evaluation plan. D. Rating Systems—a decision system that measures how well an offeror’s response meets the solicitation’s requirements. The system ranges from a relatively simple satisficing or go/no-go decision to more complex adjectival and direct point scoring systems. E. Award Algorithm—methods for combining parameters and evaluation rating systems into an award decision. Algorithms are described through a formula or a step-by- step decision process.

74 XXX.03 Preparing the Proposal. The following is added to the conditions listed in Section 102.06 If the Agency specifies a one-step process, submit a pro- posal consisting of [insert description of the required process, evaluation criteria, rating system, and award algorithm]. The proposal includes separate price and technical submissions. Submit price proposals on Agency-supplied forms. For tech- nical proposals, submit a sealed package, containing concise written material (or drawings) that enables a clear under- standing and evaluation of technical criteria. Legibility, clar- ity, and completeness of the responses are essential. Present the Technical Proposal such that the Agency can easily sepa- rate and evaluate each criterion. [Insert specific requirements for technical responses, page limits, and format]. If the Agency specifies a two-step process, the Agency will issue a step-one Request for Qualifications (RFQ) including [insert qualifications evaluation criteria, standards, and eval- uation plan]. Submit a Statement of Qualifications (SOQ) addressing the requirements of the RFQ. If determined to be fully responsive to the qualifications, the Agency will issue a step-two Request for Proposal (RFP) to the qualified bidders. Considerations: If the proposal consists of price and other parameters expressed in terms of an equivalent price, as in the case of A+B bidding, the Agency will limit the submission to a price proposal form with an explanation of prices for Part B or other parameters. For A+B bidding, Part A is the total dollar amount of the unit price bids in the Bid Schedule and Part B is the number of calendar days that the Bidder will require to substantially complete the project multiplied by the Daily User Cost listed in this special provision. The Bidder shall enter this calendar day number on the Bid Schedule in the Proposal Form. The number of calendar days shall not exceed [insert maximum number of days] days or the bid will be considered non-responsive. The Agency will evaluate each bid as the sum of Parts A and B. The successful bid is the lowest combination of Parts A and B. The Agency will award the Contract in the amount specified in Part A of the bid. The B time will be the calendar-day time period specified in Part B of the bid. XXX.04 Irregular Proposals. The following is added to the conditions listed in Section 102.07 Irregular Proposals, under which proposals are considered irregular and may be rejected. A. The proposal fails to meet a minimum standard or pass/fail requirement. B. When A+B bidding is specified, the proposed number of days bid to complete the project or listed contract segments is outside the range specified for the project or segment. XXX.05 Consideration of Proposals. The following replaces Section 103.01 Consideration of Proposals. The Agency will evaluate proposals based on [describe best- value system including the specified evaluation criteria, award algorithm, and rating system. Refer to commentary for sum- mary table describing evaluation criteria and award algo- rithms]. The Agency will select the successful proposer based on [describe the evaluation plan and method of scoring using a mathematical combination of price and technical score, cost- technical tradeoff, or fixed-price best proposal. The specification must clearly document the evaluation process, and specify the method of scoring and computation or qualitative determina- tion of the best-value proposal]. Considerations: The following are some considerations for Agency personnel when developing a best-value solici- tation: • The following table includes the recommended best-value award algorithm formulas. [Please note that other award algorithms are possible. Consult the applicable statutes and procurement guidelines to determine if a particular award mechanism is required.] BV Award Algorithm Algorithm Variables Meets Technical Criteria— Low Bid If T > Tmin, Award to Pmin If T < Tmin, Non-Responsive T = Technical Score P = Project Price Value Unit Price (Weighted Criteria) TS = W1S1 + W2S2 + ... + WiSi + W(i+1)PS Award TSmax TS = Total Score Wi = Weight of Factor i Si = Score of Factor i PS = Price Score Qualitative Cost-Technical Tradeoff Similar to above, only no quantitative analysis of difference. Award to proposal that has best value in proposed scope. Evaluation Panel reaches consensus as to which proposal is the best.

75 • The following table includes a recommended framework for combining evaluation criteria with an award algorithm and rating system. • The criteria used for technical evaluation, the weighting or relative importance of each criterion (including price), the scoring system, and the award algorithm should be clearly defined in the solicitation documents. This creates a level playing field, reduces the uncertainty related to the selec- tion process, and focuses the proposers on what is most important to the agency. • Use selection criteria that are important, add value, and relate to desired performance. Selection criteria often include factors that are difficult to assess or do not directly relate to the performance goals of the agency. Use a project screening system to identify key selection criteria that add value to the procurement process. • If performing a best-value tradeoff analysis to justify award to other than the lowest priced offeror or other than the highest technically rated offeror, conduct systematic com- parisons of price and technical criteria. Furthermore, the rationale for the decision, including benefits associated with the additional costs (or reduced costs), must be doc- umented, but need not quantify the tradeoffs that led to the decision. • It is prudent to ask that interested parties submit questions in advance of the submission stage and hold a pre-proposal meeting to answer questions or clarify aspects of an RFP. For a best-value procurement involving the consideration of complex technical criteria, this initial clarification is even more critical. Additionally, federal law requires noti- fication to unsuccessful offerors and allows for pre- or post-award debriefings if requested by the offeror. A debriefing is also strongly recommended for best-value procurement to further clarify the basis for award, the selection process, and the rationale for eliminating the offeror, if this was not apparent in the written notification of contract award. Example 1: Meets Technical Criteria—Low Bid The final award decision is based on price. Technical pro- posals are scored before any cost proposals are reviewed. The price proposal is opened only if technical proposal is above the minimum technical score. If it is below the technical score, the proposal is deemed non-responsive, and the price proposal is not considered. Award will be determined by the lowest priced, fully qualified offeror. A generic algorithm and example follow: Algorithm: If T > Tmin, Award to Pmin If T < Tmin, Non-Responsive T = Technical Score P = Project Price Award Algorithm BV Parameter and Evaluation Criteria Meets Technical Criteria—Low Bid or Low Cost Value Unit Price (Weighted Criteria) Cost-Technical Tradeoff (Qualitative) Cost Cost: A.0 X X X Time Schedule: B.0 X X X Qualifications Prequalification: P.0 X Past Project Performance: P.1 X X Key Personnel Experience: P.2 X X Subcontractor Information: P.3 X X Project Management Plans: P.4 X X Safety Record/Plan: P.5 X X X Quality Quality Management: Q.0 X X X Design Alternates Design with Proposed Alternate: D.0 X X Technical Proposal Responsiveness: D.1 X Environmental Considerations: D.2 X X Rating System Satisficing Adjectival or ModifiedSatisficingDirect Point Scoring

76 Example 2: Value Unit Price (Weighted Criteria) In the value unit price algorithm, the technical proposal and the price proposal are evaluated individually. A weight is assigned to the price and each of the technical evaluation fac- tors. The sum of these values becomes the total score. The offeror with the highest total score is selected. A generic algo- rithm and example follow: Algorithm: TS  W1S1  W2S2  ...  WiSi  W(i1)PS Award TSmax TS = Total Score Wi = Weight of Factor i Si = Score of Factor i PS = Price Score Example 3: Qualitative Cost-Technical Tradeoff The qualitative cost-technical tradeoff is used by many fed- eral agencies under the FAR. This method relies primarily on the judgment of the selection official and not on the evalua- tion ratings and scores (Army 2001). The final decision con- sists of an evaluation, comparative analysis, and tradeoff process that often require subjectivity and judgment on the part of the selecting official. The figure below depicts the qualitative cost-technical tradeoff algorithm as described in the Army Source Selection Guide (Army 2001). The tradeoff analysis is not conducted solely with the rat- ings and scores alone. The selection official must analyze the differences between the competing proposals and make a rational decision based on the facts and circumstances of the Offeror Technical Score (60 maximum) (40 minimum) Price Proposal 1 51 $1,400,000 2 53 $1,200,000 3 44 $1,100,000 4 39 NR Value Unit Price Example Offeror Technical Score (60 maximum) Price Proposal Price Score (40 maximum) Total Score (100 maximum) 1 51 $1,200,000 36 87 2 53 $1,250,000 35 88 3 44 $1,100,000 38 82 4 39 $1,000,000 40 79 Lowest priced proposal is the superior proposal in terms of non-cost proposal Proposals are essentially equal in terms of non-cost factors Conduct tradeoff analysis Award to offeror that represents the best value Award to lowest priced offeror NO NO YES YES Meets Technical Criteria—Low-Bid Example

77 specific acquisition. Two selection officials may not necessar- ily come to the same conclusion, but both must satisfy the fol- lowing criteria: • Represent the selection official’s rational and independent judgment, • Be based on a comparative analysis of the proposal, and • Be consistent with the solicitation evaluation factors and subfactors. 3.8 Summary The research team has developed practical criteria and processes for implementing best-value procurement for con- struction. The approach is to furnish a limited suite of possi- bilities that allows each agency to select the parts that best fit its individual needs and legislative constraints. Additionally, each project is unique and a given agency may want to use different best-value contracting systems for different kinds of projects. The research has further shown that to be success- ful, the selection of appropriate best-value projects is essen- tial. There will be those projects that should not be procured using a best-value contract, and they should remain in the procurement realm defined by the lowest responsive bid. Thus, the coupling of the best-value project screening and selection tool to the best-value procurement system using the parameters, evaluation criteria, rating/scoring system, and award algorithm is both logical and essential to the successful implementation of best-value contracting for highway con- struction projects.Agencies should also be open to use of adjec- tival rating systems, based on recommendations from federal agencies that the best results are achieved with such a system. Finally, the report recommends strategies to implement best-value procurement. These include legislative and regula- tory guidelines, a graduated approach to implementation, suggested ways to collaborate with industry, suggested sam- ple training tools, case studies of pilot projects, and a model specification.

Next: Chapter 4 - Conclusions and Suggested Research »
Best-Value Procurement Methods for Highway Construction Projects Get This Book
×
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

TRB’s National Cooperative Highway Research Program (NCHRP) Report 561: Best-Value Procurement Methods for Highway Construction Projects examines procurement methods, award algorithms, and rating systems for use in awarding best-value highway construction contracts. The report also explores screening criteria for selecting projects for application of best-value procurement, implementation strategies, and a model best-value specification.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!