Quality Assurance in Design-Build Projects (2008)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

This study has generally confirmed that quality issues per- vade the procurement, design, and construction phases of a design-build (DB) project. The issues range from the quality of the design-builder’s personnel who will accomplish the actual work, to the quality of the experience of the various firms that make up the design-builder’s team, to the quality of the plans that will be used to implement quality manage- ment practices after the DB contract is awarded, not to men- tion the classic design and construction quality issues present in traditional projects. Because of this, the management of quality in the DB project is of utmost importance and requires that a department of transportation (DOT) contem- plating the use of a DB project delivery prepare a thorough and thoughtful approach to communicating the DB project’s quality requirements as well as the administrative and tech- nical mechanisms that the DOT intends to use to manage both design and construction quality. The study of the litera- ture combined with the analysis of the content of the DB solicitation and policy documents and the responses to the two surveys have converged on a number of points that are summarized in this chapter. CONCLUSIONS This study used an intersection between the literature, the content analyses, and the survey responses as the criterion to making the following conclusions. When there was a dis- agreement between those sources, greater weight was given to the output from the content analyses and survey responses from DOTs with more than five DB projects’ worth of expe- rience. Based on that output, an overarching theme for the outcome of this study that applies to all stages of DB projects is that there is no one-size-fits-all quality management orga- nization for every DB project. Each DB project is unique and requires careful planning and execution to reach a successful conclusion. This was confirmed in the Request for Proposal (RFP) content analysis by finding that state DOTs that have mature DB programs—such as Florida, North Carolina, and Virginia—are using more than one type of quality manage- ment organization based on the individual requirements of each project. This was also confirmed by the survey re- sponses that indicated that 45% of the agencies use two or more approaches to DB quality assurance (QA), depending on the project. The literature validates this conclusion in a Design Build Institute of America document on DB QA and quality control (QC) that recognizes “quality as a flexible 68 aspiration that is fixed only as the business drivers of the owner and specific performance needs of the project become apparent” (italics added). In addition, survey responses indi- cated that the more experienced DOTs retain more quality responsibilities in DB than less experienced DOTs. This is again validated in the literature. An American Council of Engineering Companies report on DB QA and QC specifi- cally advocates that an owner with a third-party design con- sultant should be involved in design QA and that, when a design consultant is the engineer-of-record on a DB team, that consultant should take responsibility for construction QA. Although there are many insights than can be derived from the foregoing analysis, the synthesis yielded seven major conclusions. • Quality Management Guidance in Design-Build Policy Documents DOT DB policy documents, such as guidelines and model RFPs that contain specific guidance with regard to the agency’s quality management approach and how it can be modified and adjusted to fit the specific needs of each proj- ect, are most useful as a foundation from which to develop DB solicitation document quality requirements. By creating a specific requirement to evaluate the activities that are nec- essary to the delivery of a quality project at this fundamental policy level, the DOT is indicating the need to consider proj- ect quality from its inception and then creating a mechanism to map the conceptual quality requirements directly into the procurement documents. This will furnish not only much needed guidance to the DOT’s project managers but also pro- mote consistency in its DB projects and make estimating the cost of quality management activities easier for the compet- ing design-builders. Additionally, it will make the agency less reliant on the qualifications-based short-listing process (i.e., “quality by qualifications”) to ensure that quality man- agement has been included in the DB evaluation plan. • Design-Build Project Quality Assurance Model The content analysis identified a new set of quality manage- ment activities. This finding was confirmed in the literature by Transportation Research Circular E-C090: Design- Build: A Quality Process, which states that, “a new definition of QA [is needed that] includes oversight to provide confi- dence that the design-builder is performing in accordance with the QC plan” (italics added), as well as by the survey CHAPTER SEVEN CONCLUSIONS AND COMMONLY USED PRACTICES

69 Responsible Entity Design Construction Coordination/co mm unication as required Construction quality control: the responsibility of the construction quality ma nager Design- Builderís Project Quality Manager Design-Builder’s Construction Quality Manager Design-Builder’s Design Quality Manager Design quality control: the responsibility of the Design Quality Manager Construction quality assurance: the responsibility of the design quality ma nager Depart me nt of Transportation plus 3rd-party quality consultants (if required) Design quality assurance: verification and reviews by the DOT and/or 3rd-party quality consultants; includes over-the-shoulder reviews, com pliance checks, and audits Project quality assurance: audits and inspections of the construction performed by the DOT and/or 3rd-party quality consultants; includes testing, audits, and independent verification/acceptance TABLE 15 POSSIBLE ASSIGNMENT OF QUALITY MANAGEMENT RESPONSIBILITIES IN A DESIGN-BUILD PROJECT responses (“Design-builder is responsible for QA/QC and owner established an extensive quality oversight program”). Those are the activities that the owner undertakes to ensure a quality project when it has assigned QA responsibilities to the design-builder. This was termed “project quality assur- ance (PQA).” The content analysis also showed that experienced DOTs are not opting for a one-size-fits-all quality management ap- proach. They tailor the quality management assignments on a project-by-project basis to fit the individual needs of the project. Although a variety of possible quality management organizational structures were identified through both the RFP content analysis and the general survey responses, an analysis of the projects completed by DOTs with more than five DB projects’ worth of experience showed that they were stepping back somewhat from completely outsourcing the re- sponsibility for quality management in DB projects. The consensus seems to indicate that experienced DOTs assign the design-builder responsibility for both design and con- struction QC as well as portions of construction QA. These DOTs retain design QA and perform construction PQA with their own forces or with the assistance of a third-party con- sultant. Table 15 provides the distribution of quality manage- ment responsibilities for a typical DB project that has no post-construction operations and/or maintenance options used by experienced DOTs. Ninety-two percent of the expe- rienced DOTs rated level of agency involvement in the qual- ity management process as having a high or very high impact on the project’s ultimate quality. The one exception to the above is for projects with post- construction operations and/or maintenance options. The re- spondents that used design-build-maintain, design-build- operate-maintain, or public-private partnership (PPP) believed that there was less need to be involved in design QA, trusting the post-construction operational period to act as a warranty on the design. They typically assigned all QA and QC responsibilities to the design-builder and then conducted PQA on both the design and the construction. • Two-Step Selection Process Both the literature [the two-step selection process as “essential for success” (Bourne) and “the best overall budget and sched- ule performance” (Molenaar et al.)] and the RFP content analysis [89% of the projects analyzed used the two-step Request for Qualification (RFQ)/RFP process] showed that most DOTs use a two-step selection process for awarding DB projects. They can leverage this process to promote quality by including a quality management submittal in both steps. DOTs can require that the design-builders’ Step 1 submittal include qualifications and past performance information on the key quality management personnel on the design-builder’s team. As a minimum, the design-builder’s project quality manager, design quality manager, and construction quality manager can be identified and evaluation criteria can be established for their qualifications and past experience. Of the experienced DOTs, 92% rated the qualifications of the design-builder’s staff as having a high or very high impact on the project’s ultimate quality. During Step 2, the RFP can contain evaluation criteria for a summary/outline quality management plan in the pro- posal and clearly indicate the requirement to submit a com- plete plan after award for review and final approval. The summary quality management plan would be focused on the salient features of work and any quality challenges that the DOT has identified during its preliminary engineering work. Both the Step 1 and 2 submittals would be given an appropriate weight within the overall context of the project’s evaluation and award scheme. Including some form of the pro- posed quality management plan in the proposal is essential if the DOT wants to know each design-builder’s quality manage- ment approach before awarding the DB contract. • Best Value Award Process DOT’s use of a best value award process was also indicated by the literature and the DB solicitation document analysis. Most of the RFPs analyzed (90%) used some form of best value award. Qaasim states that “best value award is a good way to add extra weight to quality components,” and one DOT

70 indicated that “placing a quality component in the RFQ or RFP brings extra attention to the design-builder that quality is an im- portant issue for the DOT and that a proposal emphasizing quality will be evaluated more favorably” (Gladke). In the eval- uation and award scheme, a best value award process furnishes an incentive for the design-builder to propose a level of quality that is better than the stated minimum, thereby enhancing the competitiveness of its proposal and its chance to win. After award, the contents of the design-builder’s proposal are incor- porated into the DB contract, which potentially opens the door for the DOT to receive quality betterments on the project. The same is true for the quality management plans and personnel that are contained in the proposal. • Design Quality Management Emphasis Design quality management may not be not receiving suffi- cient emphasis by DOTs in their DB quality management pro- grams. The RFP content analysis revealed that only about one-third of the documents contained specific references to design quality, whereas two-thirds of the survey respondents indicated that they rated detailed design criteria as having a high or very high impact on the project’s ultimate quality. The literature clearly shows that the design phase is the time when the quality of the constructed product is articulated, and a pre- vious study found that only half of transportation project DB RFPs included design quality management aspects. There- fore, it would make sense that DOTs place a special quality emphasis on that aspect of the DB project. The second conclu- sion discussed previously supported this premise when it found that experienced DOTs were not assigning design QA responsibilities to the design-builder but rather were retaining that responsibility. Therefore, it seems that DOTs need to de- velop specific policy to pay strict attention to design quality management and promulgate that policy in their DB solicita- tion documents by making it a part of both their best value award proposal evaluation plan and the final DB contract. • Incorporation of Standard Specifications by Reference The more experienced DOTs chose to incorporate the DOT’s standard specifications by reference and allow the design- builder to optimize specific construction means and methods with its design approach. Of the experienced DOTs, 82% in- dicated that the use of performance criteria and specifications have a high or very high impact on the project’s ultimate qual- ity. Only 67% of the same group assigned that impact rating to the use of standard agency specifications and design details. Therefore, one can infer that this group is in favor of allowing the design-builder greater design freedom. These DOTs are comfortable with this because they remain involved in the design process by retaining the responsibility for design QA. • Perceptions Barrier to Implementation Perceptions that teaming the engineer-of-record with the construction contractor will degrade the quality of the proj- ect remain a barrier to implementation in spite of the FHWA DB effectiveness study that found DB quality was un- changed from that found in DBB projects. DOTs might remain sensitive to this issue in developing their DB quality management programs. By retaining design QA responsibil- ities, as cited in other previously stated conclusions, a DOT may overcome this perception. COMMONLY USED PRACTICES The analyses have also led to the identification of 10 com- monly used practices as defined in chapter one (i.e., found in the literature or confirmed by the survey or content analysis). Once again, if there was a conflict in the output more weight was given to the responses and solicitation documents that came from DOTs with a more mature DB program, as evi- denced by having more than five DB projects. The practices fall within the entire life cycle of the DB project. These are listed here without the supporting information, which can be found in the body of this report. These will be covered in life cycle order; however, it should be noted that because some of them span more than one phase, they will be included in the phase to which they are most strongly associated. • Procurement Phase There were four commonly used practices identified in this particular phase: 1. Use of the best value two-step DB award process: Ask- ing for quality-oriented qualifications for key members of the design-builder’s quality management team in the first step and requiring that summary-level or outline quality management plans be submitted for both design and construction during the second step. Stating that full design and construction quality management plans be required for review and approval after award of the DB contract in the project’s RFP. Indicating the weight of the quality-related components of the proposal in re- lation to the other evaluated factors including price. 2. Clearly identifying the quality management organiza- tional approach that will be used on the DB project in the solicitation documents. Clearly assigning the re- sponsibility for all levels of quality management in both design and construction, ensuring that those roles that are reserved for the DOT or its third-party quality consultant are also clearly indicated. 3. Demanding that the design-builder provide highly qualified and experienced personnel on its DB projects and if the DOT is new to DB, use those projects as a training ground on which the DOT staff can gain the DB experience it lacks. Asking that the RFQ/RFP list require quality-specific qualifications on both the design and construction members of the DB team. 4. To ensure that the competitors understand the requisite level of design and construction quality, preliminary design documents in the RFP clearly state the specifi-

71 cations, design criteria, and standards to be used in the final design and construction of the project. • Design Phase There were two commonly used practices identified for the design phase: 1. Determining the number of design reviews that will be conducted during the DB project design phase and clearly assigning the responsibility for conducting those reviews. These are normally published in the project’s solicitation documents to create the necessary contrac- tual requirements for both parties to the DB contract. 2. Unless the DB project has a follow-on maintenance or operations option, the DOT usually retains the respon- sibility for design QA. This can be accomplished either with its own forces or through the employment of a third-party quality consultant. • Construction Phase There were four commonly used practices identified for the construction phase: 1. The design-builder’s engineer-of-record is usually as- signed the responsibility of conducting construction QA. 2. The design-builder’s construction quality manager is usually assigned the responsibility of conducting con- struction QC. 3. The DOT conducts PQA activities during construction to satisfy its federally mandated oversight responsibil- ities. This can be accomplished either with its own forces or through the employment of a third-party quality consultant. 4. Incorporating standard state specifications by reference in the DB contract and allowing the design-builder to optimize construction means and methods with the design approach. SUGGESTIONS FOR FURTHER STUDY The final objective of this synthesis was to identify those areas in which further study is warranted. There are five such areas. • Design-Build Project Quality Assurance Model Defi- nition The owner’s role in the DB quality management process needs further definition. This report coined the term PQA to capture all those activities that the DOT would conduct to satisfy its federally mandated oversight responsibility. The literature contains a profusion of terminology that is used to identify owner quality management tasks that fall into this category. Many of the terms, such as independent assurance and verification testing, are used in traditional DBB projects. Thus, there is confusion on both the part of the DOT and more importantly on the part of the competing DB teams as to what activities might be included under the PQA umbrella. Because of this confusion, design-builders may have diffi- culty pricing their DB quality management efforts and possi- bly may include unnecessary contingencies. A respondent comment on the perceptions survey confirms this by saying that: “DB changes roles and the level of owner QA involve- ment is not yet clear.” Additionally, the effectiveness of var- ious quality management organizations needs to be studied to determine which organizations best allocate the risk of unsat- isfactory project quality. This may differ between projects based on their order of magnitude and technical complexity. Therefore, the future review must seek to differentiate between different sizes of projects as well as different types of projects. It could also investigate the quality management requirements for urban versus rural projects. • Design Quality Management Program The synthesis showed that there is a lack of clear guidance on how to properly develop and administer design quality man- agement activities in the DB context. DOTs are faced with a dilemma of determining the requisite level of design devel- opment that might be conducted for the DB RFP to properly articulate the scope of work. This dilemma stems from a natural aversion to creating liability for the preliminary engi- neering and the need to clearly articulate the technical con- straints associated with the project so that the design-builders can prepare accurate price proposals. This issue is further complicated by the traditional DBB design phase, which is often completed by DOT design engineers. Thus, the theoret- ical and contractual foundation that provides the details of an owner’s design quality management plan has not yet been produced to the level of detail currently available for the con- struction phase. Therefore, it might be prudent to investigate the marriage of the project’s design development during both the procurement and design phases of DB project delivery and optimize that with the roles and responsibilities required to develop a proper design quality management plan. Future study could use the same set of information as this synthesis to identify the possible forms that the DOT design quality management plan could take and produce a guidebook that could be used by DOTs that have not developed a specific set of DB design quality management policies and procedures. It could validate those recommendations by seeking test proj- ects on which case studies could be developed and against which project performance metrics could be benchmarked to furnish a quantitative as well as qualitative result. • Design-Build Program Evolution The process of how experienced DOTs’ DB programs matured from the first projects to the current forms could be docu- mented to furnish a reference for DOTs that have not experi- mented with DB. This would allow these less-experienced DOTs to capture lessons learned and be able to start their own

72 programs at a point that is higher up the learning curve than is currently possible. The synthesis found that experienced DOTs had started their early projects giving the design-builder the majority of the QA responsibility and then, over time and with experience, the DOTs had pulled back to a point at which they were more involved in later projects than in earlier ones, especially in the area of design QA. The reasons behind this shift in quality management policy might be documented and carefully analyzed to furnish guidance for the DB programs of less-experienced DOTs. This proposed review would examine the five or six most experienced DOTs, gathering both DB project QA policy and implementation data as well as project performance data to document the change in performance over time as the DOTs’ DB programs matured. It would also collect information on legal and contractual issues that may have arisen, causing a change in DB QA policy implementation. • Evaluating Quality Management Plans During Procure- ment The subject of evaluating quality management plans during the procurement phase also needs additional study. The liter- ature contained a number of examples in which authors indicated that applying a weighting to the proposed quality management plans during proposal evaluation could result in design-builders offering quality enhancements over the min- imums specified in the RFP. At this point in time this is merely an assertion that needs to be proven. The analysis in this area could take a two-pronged approach, with the first phase examining completed projects and comparing their results with the weights that quality aspects received in the RFP proposal evaluation plan. This phase would allow a measuring of whether this assertion is really successful in the form of proposed betterments that actually end up being con- structed in the DB project. Next, the project would seek to recruit DOTs to experiment with the weights for quality aspects illustrated in upcoming DB projects and determine if the resulting proposals indeed offer betterments as a result of the weighting and the value of those betterments. The result of the study could be a model DB quality management eval- uation plan and a decision-making tool to assist DOTs in developing DB solicitation documents that encourage inno- vation and creativity in the quality management arena. • Quality Management Program for Post-Construction Finally, the area of QA in post-construction options for oper- ations and maintenance, as well as the impact of private fund- ing in PPP projects, deserves immediate attention. FHWA’s SEP-15 program specifically allows DOTs to use PPP and other forms of post-construction options, and the DOTs are responding to the opportunity by developing large, complex projects whose values often exceed $1 billion. This is being done without the benefit of fundamental research on this sub- ject. PPP projects are common overseas and could furnish the foundation from which a study could begin and then expand into how the overseas agencies’ concepts can be “American- ized” for our industry, legal, and regulatory environment. It is very important to quantify the changes that must occur to the QA process when the design-builder not only is liable for operations and maintenance but also furnishes much, if not all, of the financing. The role in project QA of private bank- ing institutions that provide financing for these kinds of proj- ects is not well understood and must be analyzed as soon as possible to permit DOTs to leverage this source of funding for the good of this nation’s transportation infrastructure.