A Guidebook for Selecting Airport Capital Project Delivery Methods (2009)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Introduction The Tier 2—Weighted-Matrix Delivery Decision Approach provides a means for airports to further examine project delivery methods for an individual project when an obvious choice was not found in the Tier 1 approach. The Tier 2 approach involves prioritizing project objec- tives and selecting the delivery method that best aligns with these objectives. The Tier 2 approach is founded upon successful delivery decision tools developed by academics and professionals over the past 20 years (Construction Industry Institute 2003, Loulakis 2000, Skitmore and Marsden 1988). Airports should complete a Tier 1 approach before conducting a Tier 2 approach. The Tier 1 approach provides airports with two key pieces of information. First, the Tier 1 approach requires airports to define their project goals in terms of cost, schedule, quality, maintainability, sustain- ability, and other options. These project goals are critical to the Tier 2 approach. Second, the Tier 1 approach provides a shortlist of available project delivery options. Only those project deliv- ery methods that are feasible and have the best potential for successful application will pass through the Tier 1 filtering process. The filtering process involves examination of go/no-go issues and consideration of 19 pertinent issues involved in the project delivery decision. Knowledge of these pertinent issues is helpful in the Tier 2 approach. Forms for the Tier 2 approach are provided in Appendix E, which is available on the TRB web- site. To find Appendix E, go to www.trb.org and search for “ACRP Report 21”. The Tier 2 approach has three primary objectives: • Present a structured framework to assist airports in prioritizing their unique project goals and delivery selection issues, • Assist airports in aligning their unique goals and issues with the most appropriate project delivery method, and • Further document the project delivery decision in the Project Delivery Decision Report estab- lished in Tier 1. The Tier 2 approach provides a framework for airports to use in prioritizing their project goals and selecting a project delivery method that best aligns with these goals. Priorities for project goals and critical selection issues are unique to each project. Likewise, project delivery methods vary in their ability to achieve these goals and their suitability with regard to various issues. The Tier 2 approach will align these two facets of the delivery decision. The Tier 2 approach is composed of five distinct steps listed below and shown in Figure 5-1. Step 1. Define Selection Factors Step 2. Weight Selection Factors 75 C H A P T E R 5 Tier 2—Weighted-Matrix Delivery Decision Approach

Step 3. Score Project Delivery Methods Step 4. Choose Most Appropriate Project Delivery Method Step 5. Document Results Step 1 of the Tier 2 process begins by defining a concise set of selection factors. These selec- tion factors consist of the project goals and any of the pertinent issues examined in the Tier 1 approach that were deemed critical (see Chapter 4 for the Tier 1 approach). The Tier 1 approach asks airports to establish their project goals at the beginning of the process. The first step in Tier 2 is for airports to develop a concise set of selection factors by combining their project goals with the most relevant of the 19 pertinent issues examined in Tier 1. These selection factors will be used throughout the Tier 2 approach. In Step 2, airports rank and then weight selection factors. Some selection factors may overlap with others, in which case they can be combined. Other selection factors may stand alone for analy- sis. Completion of Step 2 results in a list of up to seven selection factors for further analysis. Step 3 of the Tier 2 approach requires airports to score each delivery method in terms of the selection factors. A further examination of the advantages and disadvantages for each delivery method will form the basis for these scores. Since the scores will be subjective, airports will need to be diligent in documenting the rationale for the scores. Step 4 involves a determination of the most appropriate delivery method through the com- pletion of the weighted-decision matrix. Airports will make the determination by multiplying the selection factor weights by the project delivery scores and then summing the values for each delivery method. The highest score will indicate the best choice. However, since the scores will be subjective, airports are encouraged to review the totals to determine whether the values are logical and defensible. The objective of Step 5 is to supplement the Project Delivery Decision Report developed in Tier 1. The Tier 1 report will provide a project description, project goals, delivery methods con- sidered, advantages and disadvantages, delivery method decision, and any relevant appendices. The Tier 2 documentation will add the weighted-decision matrix to the Tier 1 documentation to sup- 76 A Guidebook for Selecting Airport Capital Project Delivery Methods Figure 5-1. Tier 2 approach overview. Step 1. Define Selection Factors Time: _______________ Cost: _______________ Quality: _______________ … : _______________ … : _______________ … Step 5. Document Results PROJECT DELIVERY DECISION REPORT Tier 1 Project Description Project Goals Delivery Methods Considered Advantages and Disadvantages Delivery Method Decision Tier 2 Weighted-Matrix Decision Chart Step 2. Weight Selection Factors Step 3. Score Project Delivery Methods Step 4. Choose Most Appropriate Project Delivery Method Project Delivery Method DBB CMR DB Specify Procurement (______________)* Selection Factor Factor Weight Score Weighted Score Score Weighted Score Score Weighted Score Factor 1 (e.g., Project Goals) Factor 2 (e.g., Airport experience) Factor 3 (e.g., Market issues) Factors 4 to 7 … Total Score *Refer to the section titled “Definition of Delivery Methods” in Chapter 2 for procurement options. The DB procurement options considered in Tier 2 are primarily Best-Value Procurement with Fixed Price and DB Qualifications-Based Procurement with Negotiated Price. DB Low Bid is an option, but it is not recommended in this guidebook for the majority of DB projects.

plement the archival record of the project delivery decision. The Project Delivery Decision Report will serve to communicate the decision to interested stakeholders and to justify the decision if issues arise years later, after the project is completed. The five steps of the Tier 2 approach are discussed in more detail below. In this report, to better illustrate how the Tier 2 approach works, the selection of a delivery method for an example proj- ect is followed through the first three steps of the Tier 2 analysis. Following the description of Steps 1, 2, and 3 in the Tier 2 approach is an illustration of how each step was handled in the delivery selection process for the example project. Step 1. Define Selection Factors As stated in Step 1 of the Tier 1 approach, understanding and communicating a concise set of project goals is perhaps the most important element in selecting an appropriate project delivery method. The definition of project goals is a key success factor not only in the project delivery deci- sion, but also in the development of procurement documents and the administration of a project. It is the project performance goals (e.g., time, cost, quality, maintainability, and sustainability) that typically drive the project delivery decision. The first step in the Tier 2 approach requires airports to combine the project goals and pertinent issues into a set of selection factors for use in the weighted-decision matrix. This step requires a review and filtering of the project goals and pertinent issues for use as selection factors. Figure 5-2 depicts this process. To create the goal-based selection factors, airports should review the project goals that were established in Tier 1. The Tier 1 review of the delivery method advantages and disadvantages may have revealed overlaps or gaps in the originally established project goals. While the original proj- ect goals should not change, these overlaps and gaps will need to be removed for the development of the Tier 2 selection factors. Step 1 in the Tier 2 approach allows and encourages editing of these goals as they are rewritten into selection factors. In developing selection factors from the project goals, airports should consider the following questions: • Are there significant overlaps in the project goal statements that can be revised to make them more independent? Tier 2—Weighted-Matrix Delivery Decision Approach 77 Figure 5-2. Tier 2 selection factor development. Pertinent IssuesProject Goals TIER 2 SELECTION FACTORS Develop Goal-Based Selection Factors • Remove overlaps • Fill in gaps • Comprehensive • Non-overlapping • Concise Develop Issue-Based Selection Factors • Choose critical issues • Remove overlaps

• Are there goal statements missing that are needed to define the ultimate project success? • Can any of the goals be stated more concisely? The Tier 1 approach provides an opportunity to review 19 pertinent project delivery issues. However, in the Tier 1 approach, all of the issues are treated as equally important. Upon reviewing the issues, airports will certainly find that some issues are more important than others. A small number of issues are likely to be important to the final project delivery decision. The next task in Step 1 is to select up to 7 of the 19 pertinent issues to examine and develop into selection factors. The airport should select the pertinent issues based on the following criteria: • The pertinent issues should be independent of the project goals, • The pertinent issues should be independent of each other, and • No more than seven critical issues should be chosen. The final task of Step 1 is to consolidate the goals-based and issues-based selection factors into one comprehensive list. The intent is to limit the total number of selection factors in this con- solidated list to seven, so depending on the number of goals chosen, some of the pertinent issues may be eliminated in the final analysis. Delivery Selection Process for the Example Project—Step 1. The selection factors for the example project were determined to be the following: • Project complete by November 1, 20XX. • Cost not to exceed $200 million. • Minimize impact on operations and passengers. • Minimize staffing requirements during design and construction. This list of selection factors includes project goals relating to time, cost, operations, and staffing. While other issues of technical quality, maintainability, security, and so forth, undoubt- edly exist on the project, the list of selection factors includes the goals and issues by which the success of the project will be primarily measured at its completion. Step 2. Weight Selection Factors The Tier 2 approach is based on the premise that airports can establish a unique hierarchy of selection factors. In other words, success will be defined differently for each project and the cri- teria for success can be described by a few key selection factors. The objective of Step 2 is to weight the list of selection factors. Step 2 involves first ranking and then weighting the selection factors. There are numerous meth- ods that can be used to achieve a weighted ranking of the factors. The most straightforward method is developing a ranking and weighting through discussion among project decision-makers. The decision will by nature be somewhat subjective, so a diligently documented group decision is preferable. To achieve the weighted ranking, airports should do the following: • List the selection factors in rank order from highest to the lowest with regard to their influ- ence on project success. • Include a minimum of four and a maximum of seven factors. – Remove factors not ranked in the top seven. • Using 100 total points, weight the factors according to their influence on project success. – Avoid equal weighting of factors. – Remove any factors with a value of less than 5 of the 100 points and redistribute points. 78 A Guidebook for Selecting Airport Capital Project Delivery Methods

The result of Step 2 will be a weighted ranking of up to seven selection factors. A maximum of seven is selected because research has found that when there are more than seven variables it becomes difficult for people to make distinctions among them (Miller 1956). The weightings should total 100 points. Equal factor weightings are not recommended because distinguishing the level of importance of each factor (goals and critical issues) is necessary for the decision process. Additionally, no single factor should have a point value of less than five because a point value that low will not have a sufficiently significant influence on the final decision and may in fact make the selection more difficult. The next steps (Steps 3, 4, and 5) involve combining the weighted rankings with a scoring of the project delivery methods to arrive at a final selection of the most appropriate delivery method. Delivery Selection for the Example Project—Step 2. Table 5-1 shows how weighted rank- ing worked in the example project. In Table 5-1, selection factors for the example project have been weighted to reflect their influence on the success of the example project’s delivery. These weightings are project dependent and should be agreed upon by key airport team members. Step 3. Score Project Delivery Methods The third step involves a scoring of the alternative delivery methods that survived the screen- ing process of Tier 1 analysis. Each of these delivery methods will have a bearing or influence on the selection factors, which stem from the project goals and pertinent issues. The key decision- makers must translate this influence into a score to arrive at a decision. To achieve the total scores for each delivery method, airports should do the following: • Using the scale provided in Table 5-2, assign a score to each delivery method that represents its influence or bearing on each selection factor. Score all delivery methods for each factor before moving to the next factor. • Repeat the previous step for each selection factor. • When all of the delivery methods have been scored, multiply each delivery method’s factor weight by its score to achieve a weighted score for each delivery method. • Sum all of the weighted scores to arrive at a total score for each delivery method. Table 5-2 provides a scale for scoring each delivery method’s bearing on each selection factor. The scores range from 1 to 10 so that when they are multiplied by the factor weight, the total score will range from 0 to 1,000. The scores are subjective, so a detailed definition for each numerical score is provided adjacent to the score in Table 5-2. When scoring the delivery methods, airports should discuss the advantages and disadvantages of each delivery method (see Chapter 3 and Step 4 of the Tier 1 approach). The alignment of these advantages and disadvantages with the selec- tion factors forms the basis for the scoring. In assigning the scores, the airport should work in a Tier 2—Weighted-Matrix Delivery Decision Approach 79 Table 5-1. Weighted ranking of selection factors for the example project. Weight Selection Factor 50 Project complete by November 1, 20XX. 25 Cost not to exceed $200 million. 15 Minimize impact on operations and passengers. 10 Minimize staffing requirements during design and construction. 100 Total

team to come to a decision by consensus. The rationale for each individual score should also be carefully documented. Consideration should also be given to the relative scores for each delivery method to ensure consistency. Similar to the development of factor weights, the scoring can be done simply through a group discussion among key airport team decision makers. Table 5-3 provides a weighted-matrix template. The matrix shown contains three delivery meth- ods. However, a larger or smaller number of delivery methods can be analyzed, depending upon the results of Tier 1. For example, two types of DB delivery methods with various procurement 80 A Guidebook for Selecting Airport Capital Project Delivery Methods Table 5-3. Weighted-matrix template. Score Definition 10 The evidence that the delivery method positively aligns with the project objective or issue is of the highest possible order of affirmation. 8 The delivery method strongly aligns with the objective or issue and is demonstrated in practice. There is a slight risk that the objective or issue may not be beneficial. 6 Experience and judgment point to the delivery method strongly aligning with the objective or issue. There is a mild risk that the objective may not be beneficial. 4 Experience and judgment slightly point to the delivery method aligning with the objective or issue. There is a strong risk that the objective will be negatively affected. 2 There is little benefit to applying the delivery method for this goal or objective. There is a strong likelihood that the object will not be achieved. 9,7,5,3,1 Intermediate values between two adjacent judgments. (Adapted from Saaty 1990.) Table 5-2. Project delivery scoring scale. Project Delivery Method DBB CMR DB Specify Procurement (______________)* Selection Factor Factor Weight Score Weighted Score Score Weighted Score Score Weighted Score Factor 1 (e.g., Project Goals) Factor 2 (e.g., Airport experience) Factor 3 (e.g., Market issues) Factors 4 to 7 … Total Score *Refer to the section titled “Definition of Delivery Methods” in Chapter 2 for procurement options. The DB procurement options considered in Tier 2 are primarily Best-Value Procurement with Fixed Price and DB Qualifications-Based Procurement with Negotiated Price. DB Low Bid is an option, but it is not recommended in this guidebook for the majority of DB projects.

Tier 2—Weighted-Matrix Delivery Decision Approach 81 Table 5-4. Weighted matrix for the example project. methods could be competing in this matrix. The matrix can also contain up to seven selection fac- tors for each project. The result of Step 3 will be a scored ranking of the delivery methods in question. The delivery method with the highest total score will be the most appropriate method for the given project. The next steps involve documenting the individual scores, making a decision, and creating a Project Delivery Selection Report. Delivery Selection for the Example Project—Step 3. Table 5-4 shows how an airport might score the project delivery methods for the example project. Note that only the CMR and DB project delivery methods made it through the Tier 1 filter for further consideration in Tier 2. Also note that the scores in the example below are project dependent and will certainly change from project to project. Explanations of the scores for the project delivery methods for the example project are the following: • Project completion factor. The project completion factor relates to a project goal. In this case, the project has a fixed end date of November 1, 20XX. The airport believes that the completion date can be achieved with CMR delivery. The airport also believes that CMR will require the use of multiple bid packages to achieve the schedule, which adds a risk for meeting the schedule date, so CMR = 6 (in this case). DB delivery provides for a single entity to coordinate design and con- struction. DB also allows for an airport to specify a fixed end date in the procurement documents and the contract. According to what has been demonstrated in practice, the airport is confident that the end date can be achieved through a DB delivery, so DB = 8 (in this case). • Cost containment factor. The cost containment factor relates to a project goal. The project has a maximum budget of $200 million. DB delivery has demonstrated in practice that a fixed price can be negotiated early in the project development process. It has also been demonstrated that DB provides the lowest average cost growth of the two methods in question, so DB = 8 (in this case). CMR also provides the ability to meet a fixed price, but the airport is not as confident that it will be able to negotiate a fixed price as early in the process as it can with DB, so CMR = 6 (in this case). • Impact on passengers and operations factor. This factor stems from a pertinent issues analy- sis in Tier 1. The project involves work on an operating airport. The airport desires to keep operations and passengers flowing smoothly throughout the construction of the project. In this case, the airport has met with designers who can help define operational goals that can be Project Delivery Method CMR DB (QBS) Selection Factors FactorWeight Score Weighted Score Score Weighted Score Project complete by November 1, 20XX 50 6 300 8 400 Cost not to exceed $200 million 25 6 150 8 200 Minimize impact on operations and passengers during construction 15 10 150 6 90 Minimize staffing requirements during design and construction 10 8 80 6 60 Total Score 100 680 750

achieved through their independent designs if they are hired directly by the airport. The CMR delivery method will provide for a direct contract between the airport and the designer to ensure that the goals are achieved, so CMR = 10 (in this case). While the airport can develop DB performance criteria related to operations and passenger impacts, it is not as confident that it can accurately articulate its goals in the performance criteria, and it believes that there is a risk that the goals will not be fully achieved, so DB = 6 (in this case). • Airport staffing factor. The airport staffing factor relates to a pertinent issue examined in Tier 1. The airport does not have a large staff and wants to minimize staffing requirements during design and construction. The CMR option will allow the airport to supplement its staff during both design and construction, either with the designer or with the CMR. The airport is confi- dent that qualified professionals exist to meet its staffing needs, but is slightly concerned about exactly how the working relationship will be executed between the CMR and the airport, so CMR = 8 (in this case). The DB option will require the airport to mass its resources (or build up for a short time) during the procurement and design review process. The airport believes that it can supplement its staff with a general engineering consultant, but it is not confident that the DB option will be as effective as the CMR option, so DB = 6 (in this case). Step 4. Choose the Most Appropriate Project Delivery Method At this point, choosing the appropriate delivery method is simply a matter of reviewing the total scores and making the project delivery decision. Since the factor weighting and the scores are sub- jective, the airport should review the totals and confirm that they are logical and defensible. If, upon further discussion, a factor weight or project delivery score appears to be incorrect or to overly influence the selection, it is acceptable to make changes and create a new total project score. The key is to document the reasons for each change. If the airport is not confident about a partic- ular weight or score, it can conduct more research on a particular delivery method and revisit the scoring after gathering more information. Step 5. Document Results As in Tier 1, documentation of the delivery decision is a key part of the process. Whether one delivery method clearly achieves the highest score or no dominant choice appears, documenta- tion is a vital step. Documentation will assist in developing procurement and contracting strate- gies for the ultimate project delivery method. Documentation will also serve to communicate the project delivery choice to interested stakeholders. Documentation of results includes the Project Delivery Decision Report developed in Tier 1. It should also contain the weighted matrix of Tier 2 and a detailed documentation of the reason- ing that was used to assign each criterion weight and project delivery score. A Project Delivery Decision Report outline is offered below: • Executive Summary • Project Description • Project Goals • Delivery Methods Considered • Selection Factors • Weight Selection Factors • Score Project Delivery Methods • Delivery Method Decision • Appendices 82 A Guidebook for Selecting Airport Capital Project Delivery Methods

Conclusion The Tier 2—Weighted-Matrix Delivery Decision Approach extends the Tier 1 approach through an examination of how project delivery methods align with project goals and pertinent issues as they are consolidated into selection factors. The weighted ranking of project selection factors requires decision-makers to define their priorities and more closely examine the attri- butes of each delivery method that passed through the Tier 1 filter. At the end of Step 4, there should be a single, clear, and logical choice for a project delivery method, and this choice can be documented in the Project Delivery Decision Report. Tier 2—Weighted-Matrix Delivery Decision Approach 83