Alternative Quality Management Systems for Highway Construction (2015)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

CHAPTER 2 : STATE OF THE PRACTICE 2.1 Introduction and Overview 2.1.1 Background During the 1990’s, the transportation industry began catching up with the building construction industry by experimenting with various forms of alternative project delivery (Songer and Molenaar 1996; Anderson and Damnjanovic 2008) as well as alternative quality management systems (Miron et al 2008). In the state department of transportation sector this was facilitated by the Federal Highway Administration (FHWA) Special Experimental Project 14 (SEP-14) (FHWA 2006), which authorized the use of DB and CMR (also called Construction Manager/General Contractor or CM/GC) project delivery. In the past decade, transportation agency procurement programs have matured and project delivery methods, which in the 1990s were termed as either “alternative” or “innovative,” have been institutionalized and now are viewed as merely tools in the typical agency’s procurement toolbox (Trauner 2007). A paper written at the turn of the 21st century (Miller et al 2000) proposed “simultaneous use of multiple project delivery methods” as the “new paradigm.” Those authors were prescient because recent project delivery research has confirmed that many transportation agencies across the country employ more than one project delivery method (Touran et al 2010; Scott et al 2006). Driving the shift in public procurement culture was the perception by practitioners both in government and industry that benefits could be accrued by integrating the project team and bringing the constructor into the project before design was complete to furnish substantive input on cost, schedule, and constructability to the final design (Miller et al 2000; Touran et al 2010). Beyond time and cost savings, the salient question has been to determine if the quality of the ultimate product was degraded through either the speed at which the design and construction had to be completed or by an agency loss of control over the design and construction process (Gransberg and Molenaar 2008). 2.1.2 Objective Most authors that have written about the dangerous condition of the nation’s highway network concluded that public transportation agencies must find ways to deliver infrastructure projects “better, faster, cheaper” (Atzei et al. 1999; Avant 1999; Richmond et al. 2006). Yet, the political will to make fundamental changes in the methods used to deliver infrastructures is often difficult to muster. For example, the length of time California Department of Transportation (Caltrans) needed to gain authorization to utilize design-build (DB) contracting is a typical example of the constraints faced by many transportation agencies. The Federal Highway Administration (FHWA) Special Experimental Projects No. 14 – Alternative Contracting (SEP-14) was introduced in 1990. By 2009, the FHWA had authorized more than 400 DB highway projects (FHWA 2006; FHWA 2009). California has led the nation in terms of traffic congestion for decades and as a result accelerating the delivery of projects to increase the capacity of 12

California’s highways will generate enormous benefits to the traveling public and the environment (Dowall and Whittington 2003). However, it took Caltrans 19 years from the inception of SEP-14 to obtain the legislative authorization to proceed with a demonstration program of six DB projects. In June, 2010, the FHWA introduced its “Every Day Counts” (EDC) initiative to address this and other issues of similar importance. The program is designed to accelerate the implementation of innovative practices that are immediately available, as described by then FHWA Administrator Victor Mendez: “Our society and our industry face an unprecedented list of challenges. Because of our economy, we need to work more efficiently. The public wants greater accountability in how we spend their money. We need to find ways to make our roads safer. And we have an obligation to help preserve our planet for future generations. But it’s not enough to simply address those challenges. We need to do it with a new sense of urgency. It’s that quality—urgency—that I’ve tried to capture in our initiative, Every Day Counts.”(Mendez 2010, italics added). Again, many authors have documented the “urgent need to replace aging infrastructure” (Dowall and Whittington, 2003), but creating an atmosphere of urgency inside technocratic public transportation agencies is a challenge in and of itself. The resistance to change is rooted in the fear that the agency’s historic set of checks and balances will be upset and control over cost, schedule and quality will be lost (NSPE 1995). Hence, the FHWA EDC focus is on innovations that have already been successfully employed by typical State Transportation Agencies (STAs), which theoretically will no longer be considered “experimental” as the SEP-14 label implies. “EDC is designed to identify and deploy innovation aimed at shortening project delivery, enhancing the safety of our roadways, and protecting the environment…it’s imperative we pursue better, faster, and smarter ways of doing business” (Mendez 2010, italics added). Note that Administrator Mendez changed the “better, faster, cheaper” mantra to “better, faster, and smarter.” This relieves a great deal of pressure in the political process when a state agency no longer must find the cheapest solution to obtain federal-aid funding. Plus, the notion of “shortening,” rather than accelerating, the project delivery process subtly implies lowering the risk of the agency losing control over its design and construction quality and costs. 2.1.3 Scope There are two key concerns to all public transportation agencies: project quality management and project delivery method. With the growth of alternative project delivery methods in the past few decades, the issues have become interrelated. It is important to understand how agencies that are using alternative project delivery are approaching the quality management (QM) issue on their projects. In project delivery methods where selection of the contractor occurs before design completion so that the contractor can contribute to the design, the agency ought to consider the impact of that shift on quality management planning and execution at every phase of project 13

development. Table 2-1 compares the potential for meeting three quality objectives among three project delivery methods based on an analysis of federal projects (Uhlik and Eller 1999). Table 2-1 – Quality Management Comparison of Project Delivery Methods (Adapted from Uhlik and Eller 1999) Likelihood of Meeting Objective Quality Objectives DBB CMR DB A system of checks and balances exists between design and construction High High Low Input on quality is provided during design by someone with construction expertise Low High High Single point of responsibility for design and construction quality Low Low High This study concludes that CMR project delivery has a high likelihood of delivering two of three quality objectives. The third objective, single point of responsibility, can only be achieved by DB project delivery and DB has a low probability of achieving the checks and balances objective. The chart indicates that CMR and DB may be the preferred project delivery methods for projects where assuring quality is difficult. Another author reached the same conclusion when it stated: “CMR improves quality and value… [by keeping] focus on quality and value – not low bid” (Ladino et al 2008). The scope of this section is to discuss the state-of-the practice regarding QM as implemented in projects delivered using alternative project delivery methods. 2.1.4 Essential Questions Chapter 2 will seek to answer the following questions regarding QM in alternative project delivery:  What is the purpose of a Quality Management program?  Why do alternative project delivery methods need a different approach to quality management?  What quality management tools, techniques and methods are currently in use on alternative project delivery methods and currently in use for highway design and construction? 2.2 Relevant Definitions Throughout the construction industry, there are certain terms that are used to define aspects of quality programs. The literature review exposed “confusion” among the various authors as to precise definitions for the various aspects of “quality” and the terminology used to describe the tasks involved in design and construction quality management. The American Society for Quality defines quality as, “the totality of features and characteristics of a product or service that bears on its ability to satisfy given needs” (ASQ2013). That definition is quite broad, but the focus on “satisfy given needs” is cogent to this section. The owner must clearly articulate the “given needs” for design and construction quality in 14

project documents (i.e. RFPs, specifications, etc.). One way to do that is by requesting specific quality-related submittals as a part of any pre-award proposals if applicable. The other way is to include the requirements for design and construction quality management as submittals required after contract award. The ASQ goes on to define five varying types of quality as follows (ASQ 2013):  Relative Quality: loose comparison of product features and characteristics.  Product-Based: quality is a precise and measurable variable and differences in quality reflect differences in quantity of some product attribute.  User-Based: fitness for intended use.  Manufacturing-Based: conformance to specifications.  Value-Based: conformance at an acceptable cost. (ASQ 2013) Thus, one can see that the concept of quality has many facets. As a result, an owner attempting to articulate the requirements for both design and construction quality needs to be very precise in the working definition of quality for each feature of work. One way to measure quality is by conformance to a quality plan (Arditi and Lee 2004). Additionally, as the methods used to deliver highway projects evolve, the transportation industry must develop new definitions to describe the altered state of roles and responsibilities for managing the quality process. A previous NCHRP report by Halstead (1979) simplified the definition a quality management system to four basic questions that provide a concise reference to ensure that a quality management system fulfills its needs. The questions are: 1. What do we want? 2. How do we order it? 3. Did we get what we ordered? 4. What do we do if we do not get what we ordered? (Halstead 1979) The Transportation Research Circular E-C090 provides an example of the need for new definitions, which recognized the specific need for new definitions for quality in projects delivered using alternative delivery methods and stated: As it relates to QA, the owner is responsible for oversight management and a new definition of QA. This new definition includes oversight to provide confidence that the design–builder is performing in accordance with the QC plan, design monitoring and verification through auditing, spot-checking, and participation in the review of the design (Warne 2006 italics added). 2.2.1 Standard Definitions For the purposes of this report, the Transportation Research Circular E-C074: Glossary of Highway Quality Assurance Terms (Leahy 2009) will be used to define exactly what the quality assurance terms in this report mean. The major definitions are reproduced below. 15

 Quality. (1) The degree of excellence of a product or service; (2) the degree to which a product or service satisfies the needs of a specific customer; or (3) the degree to which a product or service conforms with a given requirement.  Quality assurance (QA). All those planned and systematic actions necessary to provide confidence that a product or facility will perform satisfactorily in service. (QA addresses the overall problem of obtaining the quality of a service, product, or facility in the most efficient, economical, and satisfactory manner possible. Within this broad context, QA involves continued evaluation of the activities of planning, design, development of plans and specifications, advertising and awarding of contracts, construction, and maintenance, and the interactions of these activities).  Quality control (QC). Also called process control. Those QA actions and considerations necessary to assess and adjust production and construction processes so as to control the level of quality being produced in the end product.  Independent assurance (IA). A management tool that requires a third party, not directly responsible for process control or acceptance, to provide an independent assessment of the product and/or the reliability of test results obtained from process control and acceptance testing (The results of independent assurance tests are not to be used as a basis of product acceptance).  Acceptance plan. An agreed–upon method of taking samples and making measurements or observations on these samples for evaluating the acceptability of a lot of material or construction. To the above definitions, the report will add two that are not contained in Transportation Research Circular E-C074. These definitions are for “quality management” and “project quality assurance.”  Quality Management (QM): The totality of the system used to manage the ultimate quality of the design as well as the construction encompassing the quality functions described above as QA, QC, quality acceptance, independent assurance, and verification.  Project Quality Assurance (PQA): All those actions necessary for the owner to ensure that the design-builder performed quality acceptance activities given a true representation of the quality of the completed project. This may include owner verification and acceptance testing or independent assurance as owner oversight actions when assigning the design-builder the responsibility for design and/or construction QA activities. Additionally, these also include owner oversight, verification, validation, acceptance, and other activities necessary to satisfy FHWA Technical Advisory 6120.3 (FHWA 2004) for projects with federal funds and the employment of independent quality consultants that may be necessary in DB projects with post-construction operations and/or maintenance options. Finally, to understand the relationship between quality and the project procurement process, the following list defines project delivery methods, procurement procedures, contract payment provisions and other terms used commonly throughout the report. 16

 Design-bid-build (DBB): A project delivery method where the design is completed either by in-house professional engineering staff or a design consultant before the construction contract is advertised. Also called the “traditional method.”  Design-build (DB): A project delivery method where both the design and the construction of the project are simultaneously awarded to a single entity.  Construction Manager/General Contractor (CM/GC): A project delivery method where the contractor is selected during the design process and makes input to the design via constructability, cost engineering, and value analysis reviews. Once the design is complete, the same entity builds the projects as the general contractor. CMGC assumes that the contractor will self-perform a significant amount of the construction work.  Construction Manager-at-Risk (CMR): A project delivery method similar to CMGC, but where the CM does not self-perform any of the construction work.  Project delivery method: the comprehensive process by which designers, constructors, and various consultants provide services for design and construction to deliver a complete project to the owner. While names can vary in the industry and owners often create hybrid delivery methods, there are essentially three primary project delivery methods: design-bid-build (DBB), construction manager-at-risk (CMR), and design-build (DB).  Procurement procedure: the process of buying and obtaining the necessary property, design, contracts, labor, materials, and equipment to build a project. The four primary procurement procedures are low-bid, best-value, qualifications-based, and sole-source procurement.  Contract payment provision: the contract language that defines how design and construction professionals will be paid for their services. The four primary contract payment provisions are fixed price lump sum, guaranteed maximum price (GMP), cost plus fee, and cost reimbursable.  Design deliverable: A product produced by the design-builder’s design team that is submitted for review to the agency. (i.e. design packages, construction documents, etc.)  Construction deliverable: A product produced by the design-builder’s construction team that is submitted for review to the agency. (shop drawings, product submittals, etc.)  Administrative prequalification: A set of procedures and accompanying forms/ documentation that must be followed by a construction contractor to qualify to submit bids construction projects using traditional project delivery.  Performance based prequalification: A set of procedures and back-up documents that must be followed by a construction contractor to qualify to submit a bid on a construction project based on quality, past performance, safety, specialized technical capability, project-specific work experience, key personnel, and other factors. 2.3 History of Alternative Quality Management Program Development The requirement for quality pervades the life cycle of a transportation project. Decisions made in the concept and feasibility phases of a project’s life create impacts that may not be evident until 17

many years after the completed project has been in operation. The failure of the I-35W bridge in Minneapolis in 2007 is an extreme example of how an early design decision, in this case deciding to exclude the gusset plates that ultimately failed in the bridge load rating design guidance, can ultimately degrade the quality of a large project (NTSB 2008). The literature contains many models for a typical transportation project’s life cycle. However, the simplest one specifically oriented to highway construction is offered by Anderson and Blaschke (2004) and is shown in Figure 2-1. This model tracks with the traditional DBB project delivery where the project is developed in linear fashion with little overlapping of the various phases of the project’s life cycle. Thus, quality checks occur in each phase before the next phase starts. In alternative project delivery, these phases can run in parallel and that has the overall effect of compressing the project delivery period. It also furnishes less opportunity for iterative quality verification activities and makes controlling ultimate quality more difficult. Figure 2-1 – Project Life Cycle (adopted from Anderson and Blaschke 2004) However, using CMGC or DB project delivery does not make the components of the phases disappear. The owner must still conceive, study, develop, design, construct, commission, and operate the project regardless of how many activities are being done in parallel. Each of these life cycle components has a requisite quality standard associated with its deliverables and each of those must be managed to ensure that the ultimate quality of the constructed product is responsive to the operational requirements of the infrastructure project. Therefore, it is useful to study the quality management requirements that occur in each phase of the CMGC or DB project's life cycle without attempting to account for parallelism. This feature of alternative project delivery can then be approached as an additional constraint imposed by the project delivery method, i.e. the need to control quality in a manner that permits the overlap of tasks that were traditionally completed in series. 2.4 Methodologies An efficient, well-built, and sustainable highway project needs highly qualified designers and constructors to build it. For each transportation project, the quality process starts in the Programming Final Design Planning Advanced Planning/ Preliminary Design Award Letting Construction Transportation Improvement Needs 18

procurement phase. This report then defines the procurement phase as including all actions taken by the STA prior to awarding the design or construction contracts. Thus, it will encompass all preliminary planning and design activities necessary for the identification of right-of-way requirements, environmental clearances, as well as advancing the design to a point where it becomes an adequate description of the scope of work. This phase also includes the activities undertaken during advertising, qualifications review, proposal evaluation, discussions, and best and final offers as applicable to the project delivery method. Once the STA awards the contract, the procurement phase is complete. The early phases of programming and preliminary engineering are the points where the requirements for design and construction are identified. These phases lead to the concurrent development of required qualifications for the designers, builders, and others that must be present to verify the quality of the final projects. These requirements are defined, evaluated, and assured in the project’s procurement phase. NCHRP Syntheses 376 (Gransberg et al. 2008) and 402 (Gransberg and Shane 2010) found that owners, designers, and builders agreed on the key role of defining the necessary qualifications to ensure the selection of highly qualified teams during this phase. Each synthesis asked both the agencies and their contractors to comment on the impact of various aspects of the procurement phase in the CMR and DB delivery methods on a project’s final quality. Table 2-2 illustrates the results. Table 2-2 – Impact on Final Project Quality of Procurement Components for DB & CMR Procurement Phase Component Agency Ratings DB and CMR Contractor Ratings Very High or High Impact Some or Slight Impact No Impact Very High or High Impact Some or Slight Impact No Impact Qualifications of DBr or CMR’s staff 91% 9% 0% 100% 0% 0% DBr or CMR’s past project experience 76% 24% 0% 100% 0% 0% Use of performance criteria/specifications 72% 28% 0% 67% 33% 0% Early contractor involvement in design 70% 30% 0% 100% 0% 0% Level of agency involvement in the QA process 69% 31% 0% 33% 67% 0% Level of detail in the procurement documents 68% 32% 0% 0% 83% 17% Preconstruction services 63% 30% 7% 100% 0% 0% Quality management plans 61% 39% 0% 83% 17% 0% Warranty provisions 55% 38% 8% 33% 50% 17% Use of agency specifications and/or design details 51% 42% 7% 17% 67% 17% DBr = design-builder There was nearly unanimous agreement that the aspects, which have the greatest impact on project quality, are the qualifications of the Design-builder or CMR’s personnel and its past project experience. The Oregon DOT interviewee stated that “qualifications are critical to achieving quality.” This corresponds with the information cited in the literature with regard to the value of these aspects. Taken together, it leads to the conclusion that tailoring procurement 19

documents to fit the project’s specific technical and management requirements will attract the kinds of competitors that have the correct set of personnel and experiences. 2.4.1 FHWA Quality Assurance Policy for Alternative Project Delivery Understanding FHWA alternative project delivery quality assurance policy involves remembering that the foundation for it comes from 23 CFR 627, which springs from Title 23 US Code. The 1968 Federal Highway Act revised Title 23 USC which required construction contracts to be awarded “. . . only on the basis of the lowest responsive bid.” As such, the 1968 revision was written with the expectation that all federal-aid highway projects would be delivered using DBB. Thus, to permit delivery by another method required special authorization, which was provided by Special Experimental Programs 14 and 15. This meant that until 2004, a state that wanted to use DB had to apply for the authority through SEP-14, and states wishing to use CM/GC or PPP must still submit an SEP 14 or SEP 15 application if they are using federal- aid highway funding. As a result, the statutory underpinning of alternative QA views it as an exception to the established practice. Developing the necessary information to change alternative QA from an exception to the rule to a different set of recognized rules is the purpose of this particular research project. In the past decade, the use of alternative project delivery methods has challenged the traditional quality management approach where the contractor performed QC while the owner conducted QA. Figure 2-2 is a graphical representation of the FHWA approach to the definition of QA in federal-aid highway projects. A presentation at a recent FHWA “Every Day Counts” (EDC) Summit explained the federal alternative QA model in these terms: “The term Quality Assurance (QA) is a broad term that addresses all actions necessary to achieve project quality in both Design and Construction. QA is an umbrella term that includes Quality Control (QC) activities by the Contractor, as well as Acceptance activities by the Agency. Contrary to past misuse, QA is not the activity performed by the Agency.” (Yakowenko 2010). 20

Figure 2-2 – Quality Assurance Concept as Described by the FHWA (adopted from Yakowenko 2010) One can see that the major shift in perspective is the idea that QA is an umbrella under which all quality management activities take place and that the owner’s primary QA role is one of acceptance. AASHTO R10-06 (2006) provides the following definition: “Quality Assurance (QA) – (1) All those planned and systematic actions necessary to provide confidence that a product or facility will perform satisfactorily in service; or (2) making sure the quality of a product is what it should be [QA addresses the overall process of obtaining the quality of a service, product, or facility in the most efficient, economical, and satisfactory manner possible. Within this broad context, QA includes the elements of quality control, independent assurance, acceptance, dispute resolution etc. The use of the term QA/QC or QC/QA is discouraged and STA should use the term QA. QA involves continued evaluation of the activities of planning, design, development of plans and specifications, advertising and awarding of contracts, construction, and maintenance, and the interactions of these activities.]” (AASHTO 2006 italics added). Additionally, the FHWA EDC presentation went on to state: “The Design-Builder cannot be assigned responsibility to perform any Acceptance (Verification) functions” (Yakowenko 2010). It is important to note that 23 CFR 637.207 (b) only applies to construction QA and not to design QA as shown in Figure 2-3. This not only validates the notion that the regulation was drafted contemplating DBB delivery, but it also creates a void with regard to federal guidance on design QA. The documents produced during the design process define a CM/GC or DB project’s overall 21

quality (Gransberg and Molenaar 2004). The construction QA program will use the construction documents as the standard for compliance of the constructed product. Another study found that only 17 of 75 DB RFPs required the submission of a design QA with the design-builder’s proposal or after award of the DB contract (Gransberg and Windel 2008). Figure 2-3 – FHWA Concept of the Relationship between Design QA and Construction QA (Yakowenko 2010) A look at Figure 2-3 furnishes the answer to the rhetorical question: If the project’s quality is defined in the design, why would it not be imperative to require a QA plan for design that is as robust as the construction QA plan? The answer is that currently there is no federal regulatory requirement to assure design quality. The same study also inferred that since many STAs have robust in-house design capability and complete much of their design using these resources, that “public sector owners have not seemed to make the philosophical transition from DBB to DB with regard to design quality management” (Gransberg and Windel 2008). The EDC presentation validates this understanding in one of the “Key Messages” developed to reinforce the initiative’s purpose: “To ensure that goals for Project quality will be met, the Owner Agency should establish Quality Assurance (QA) requirements for DB projects. As outlined below, this includes a Design QA Program to address quality in the design process and a Construction QA Program to ensure the quality of construction. These QA Programs provide the skeleton or overall framework necessary for Quality Assurance implementation and clearly delineate the proper Agency and Design-Builder roles and responsibilities in achieving quality.  Design QA Program - Research shows that many DB project RFPs totally ignore Design QA and focus only on Construction QA. While not explicitly required by Federal regulation, the Owner Agency is strongly encouraged to implement a Design QA 22

Program as a requirement of each DB project. The Design QA Program for DB projects includes the following two elements:  Design Quality Control (QC) system by the Design-Builder  Design Acceptance/Approval system by the Agency  Construction QA Program - The Owner Agency should implement a Construction QA Program for each DB project. For Federal-Aid DB projects, this is a requirement (See 23CFR637.207 (b)). A complete Construction QA Program includes the following six core elements:  Construction Quality Control (QC) system by the Design-Builder  Construction Acceptance system by the Agency (or its Designated Agent)  Independent Assurance (IA) by the Agency (or its Designated Agent)  Dispute Resolution system established by the Agency  Qualified/Accredited Laboratories (both Design-Builder and the Agency)  Qualified/Certified Inspection & Testing Personnel (Design-Builder and the Agency)” (Yakowenko 2010) Taking the FHWA quality management division of responsibilities detailed above and comparing it to the QM organizations found in NCHRP Synthesis 376 leads to the conclusion that the EDC presentation advocates a “Type 8” DB QA organization similar to the one used by the Minnesota DOT organization as outlined in the following RFP excerpt. “To ensure the quality of the Design-Build project, MnDOT will manage and perform construction and design QA and construction IA functions. Construction QC and design QC plans and functions will be the responsibility of the Proposers.” (MnDOT 2001b italics added) The NCHRP 10-83 Proposal contained Figure 2-4, the Integrated Quality Management Model (IQ2M). The FHWA alternative QA notion is that project acceptance is the ultimate goal of a QA program. Additionally, the universe of Project Quality Assurance (PQA) shown in Figure 2-4 corresponds with the QA umbrella in Figure 2-2 from the FHWA EDC presentation. Both models allow for QM activities separated by design and construction as well as independent assurance and owner verification. The only real difference, other than the artistic representation, is the IQ2M model adds two decision points that reflect the result of owner acceptance of the quality of the two major deliverables (final design and constructed facility). The first is the point in the project where the design or the design of a feature of work is released for construction. The second is the point where the constructed product is accepted and released for final payment. Therefore, one concludes that the IQ2M model complies with the spirit of QA expressed in the EDC presentation. 23

Figure 2-4 – Integrated Quality Management Model (IQ2M) [adapted from Synthesis 376 (Gransberg et al 2008)] To be fully compliant with the FHWA EDC model, Figure 2-4 needs to be modified to indicate the distinction between QA/QC approach used by the agencies reviewed in the synthesis (i.e. “Contrary to past misuse, QA is not the activity performed by the Agency.”). The modification is shown in Figure 2-5 and seeks to conform to the AASHTO R10-06 (2006) guidance that “The use of the term QA/QC or QC/QA is discouraged” by replacing the term “QA” shown inside the universe with “acceptance” and reorganizing the structure to show the acceptance decision points before the issuing of formal acceptance. Project Quality Assurance Project Acceptance Design Released for Construction Design Quality Assurance Construction Released for Final Payment Design Quality Control Independent Assurance (if req’d) Construction Quality Assurance Construction Quality Control Independent Assurance (if req’d) Owner Verification Testing (if req’d) 24

Figure 2-5 – IQ2M Model Adapted to Conform to the FHWA EDC QA Approach (after Yakowenko 2010) 2.4.2 USACE Quality Assurance Policy for Alternative Project Delivery The US Army Corps of Engineers (USACE) has been using alternative project delivery methods since the 1980s (Henner 2007). USACE has a highly developed, detailed set of QM policies, regulations, and guidance documents for all project delivery methods. The overarching document is Engineer Regulation 1110-1-12 Quality Management (USACE 2006). This document describes the contents of what it calls “Project Quality Documents.” It furnishes the following definitions for each component to the overall USACE QM program:  “Project Management Plan (PMP)… a PMP is required for the execution of all work. The PMP identifies the scope, schedule, and resources needed to accomplish the work. It has sections that detail how to accomplish the project work items. The customer/local sponsor participates in development of the PMP and endorses it once completed.  Quality Management Plan (QMP). The QMP is the quality component of the PMP. Its purpose is to document the project-specific quality control and quality assurance procedures appropriate to the size, complexity, and nature of the project. The QMP will identify customer quality objectives, and their thresholds, and project specific requirements determined by the command. The QMP will include Quality Control Plans and Quality Assurance Plans required for product deliverables and identify quality control and quality assurance requirements for the overall project, including work Project Acceptance Independent Assurance (if req’d) Construction Acceptance Design Acceptance Design Quality Control Construction Quality Control Design Released for Construction Construction Released for Final Payment Independent Assurance (if req’d) Owner Verification (if req’d) Quality Assurance 25

performed by contractors. The QMP must be consistent with the organizational QM unless otherwise documented.  Quality Control Plan (QCP). The QCP is the quality control component of the QMP and defines how quality control will be executed for products and services.  Quality Assurance Plan (QAP). The QAP the quality assurance component of the QMP and defines how quality assurance will be executed for products and services that are completed by outside resources, including architect-engineer (A-E) contractors as well as other USACE Districts and Centers.  Contractor Quality Control Plan (CQCP). The CQCP is a written plan, provided by an AE contractor that defines how quality control will be executed on products and services that are completed with A-E resources.” (USACE 2006). One can see that the USACE approach to QM is different from the approach described for the FHWA where “QA is not the activity performed by the Agency.” (Yakowenko 2010). The references to QA in ER 1110-1-12 clearly assign the QA responsibility to the agency. Additionally, the USACE model sees QM as the “umbrella” that covers both QA and QC and it rolls the “acceptance” activities into the agency’s QA role. Therefore, understanding the USACE approach entails carefully applying the USACE definitions for key terminology and not confusing it with the FHWA definitions for the same terms. The USACE model operates on Total Quality Management (TQM) principles laid down by Deming. Specifically, the model prescribes a “‘Plan-Do-Check-Act’ (PDCA) Cycle (commonly referred to in industry as the Deming (1986) Cycle) which is the guiding quality management procedure for USACE business processes” (Yakowenko 2010). The model is graphically presented in Figure 2-6. Each step in this process is defined as follows:  “Plan – design the Project Management Plan to achieve customer requirements and provide for high-quality products and services.  Do – implement the PMP, including the quality control and quality assurance procedures.  Check – evaluate the project results.  Act – identify and implement process changes for continual improvement” (USACE 2006). 26

Figure 2-6 – USACE QA Model for Alternative Project Delivery (USACE 2006) USACE utilizes a systems approach to quality and does not break it out as a separate category of agency responsibility. As seen above, the QM plans are included as an integral part of the project management plan. The QM regulation goes on to develop further that concept by requiring the project management team to “ensure that other key PMP components are structured to optimize project quality.” Those components are defined to include:  “Production Schedule – All projects and associated technical documents will have a formal production schedule.  Risk Management Plan – A Risk Management Plan is required for the PMP. The PM will effectively engage with the customer and other PDT members to identify risks to project scope, quality, schedule and cost. These risks will be clearly defined in the project Risk Management Plan. The PDT will ensure that the necessary work breakdown activities and resources are specified in the PMP to effectively address the defined risks. Starting with the PMP, Resource Providers and Independent Technical Review Team (ITRT) members will provide continuous review to ensure that the PDT has adequately defined and addressed project risks.  Value Management Plan – A Value Management Plan is required for the PMP. The PDT will ensure that the Value Management Plan effectively applies VM/VE policies and procedures to provide the best value project for the customer.  Change Management Plan – Change Management Plan is required for the PMP. The CMP will stipulate performance metrics for project scope, schedule, cost, quality and risk. PDT and ITRT members will evaluate all proposed project changes and report potential impacts to the performance metrics per the project Communications Plan. The • 3. Check for problems • Revise procedures • Work the plan - Build in quality • Plan for quality 1. PLAN 2. DO 3. CHECK 4. ACT Quality 27

goal for the change management process will be to optimize project performance and customer satisfaction throughout the project life cycle.” (USACE 2006). 2.4.2.1 USACE DB QM Philosophy After analyzing the USACE QM approach, one can see how it directly relates with the STAs’ primary reason for delivering a project using alternative methods: to compress the delivery schedule (FHWA 2006). USACE makes QM an integral part of project management rather than a separate process that requires specialized skills. To accomplish this aim, USACE specifically states its expectations in a chapter devoted to applying ER 1110-1-12 to DB project delivery. The following list contains selected excerpts that illustrate that philosophy shift from DBB to DB project delivery:  “With D-B the contractor is the single point of responsibility for the design and construction services.  The PDT [USACE Project Delivery Team] is responsible for the quality of the design performance criteria in the D-B solicitation. The D-B contractor is otherwise responsible for design quality. The PDT will ensure that appropriate design quality control provisions are included in the D-B contract.  The PDT will develop and provide quality control review of the performance criteria and prescriptive requirements in the RFP. The PDT will review and evaluate D-B proposals for compliance with the contract requirements.  Compared to an [DBB] A-E [architect-engineer design] contract, the D-B Contractor and its Designer-of-Record (DOR) are charged with a higher standard of care to correct construction associated with faulty design.  The contract will have provisions for the contractor’s construction function to provide input during the design. The D-B Contractor’s construction function will address constructability, coordination, and ensure that the project cost is within the contract budget/price amount.  The D-B contract will include “Warranty of Design” provisions that provide for an extended callback for design errors and omission, and for correction of construction related to faulty design.  The D-B contract will address QC for design and design-related activities during construction. As a minimum, the design QCP must designate a qualified design quality control manager, incorporate independent peer reviews, utilize a design deficiency tracking system and develop procedures for design reviews, DOR reviews and approved construction submittals.  The D-B Contractor, through its DOR, will ensure the project construction is in accordance with the accepted design and the contract. The DOR’s quality role during construction includes, but is not limited to, reviewing and approving shop drawings, correcting design errors and omissions, revising the design for official changes and approved deviations, resolving field questions or problems and approving final as-built drawings. 28

 The PDT remains responsible for the quality of the design criteria and for assuring that the construction conforms to the accepted design as well as to the contract requirements. The PDT’s role is that of quality oversight by concurrence of the DOR and contract quality control activities, including spot-checking submittals to ensure that they conform to the contract and accepted design.” (USACE 2006). 2.4.2.2 USACE DB QM Philosophy USACE pioneered the use of CMGC project delivery as a method to bring the construction contractor onto the project that is faster than DBB and DB. Figure 2-7 shows the USACE perspective in relation to a given project planning year (PY) which begins when Congress authorizes planning and development (P&D) funding. USACE calls the project delivery method “Early Contractor Involvement” (ECI), which should not be confused with the ECI project delivery used by many international public transportation agencies that are structured similarly (Scheepbouwer 2010). Essentially, USACE sees ECI as a process that starts in planning with the construction contractor furnishing preconstruction services throughout the planning and preliminary design phases of an ECI project’s life cycle. Presumably, procurement of the ECI contractor could happen before the project’s designer-of-record, including both in-house and outsourced design resources. FAR 16.403-2 “Fixed Price Incentive Contract-Incentive Price Revision (Successive Targets)” methodology is used to procure the ECI contractor. The project delivery method allows the owner to determine whether to award the ECI contractor the construction contract based on the ability to negotiate a mutually agreed guaranteed maximum price (GMP). Since CMGC project delivery has the owner holding separate contracts for design and construction, CMGC contractually falls between DBB and DB (Yakowenko 2010) and the need to promulgate new USACE QM regulations and policies is minimal. At this writing, USACE has not published any specific ECI QM guidance, and contacts with a senior USACE official indicate that none is forthcoming. ECI has been used extensively on nearly $7.0 billion worth of projects associated with Hurricane Katrina recovery effort. The project delivery teams merely applied the QM regulations (ER 1110-1-12) in much the same manner as discussed above for DB project delivery. 29

Figure 2-7 – USACE Relative Project Delivery Timelines (USACE 2009) 2.4.2.3 Summary of USACE Alternate QM The USACE QM approach for projects delivered using alternative methods can be summarized as follows:  USACE sees QA as the agency’s role in QM.  USACE uses a systems approach to QM and has detailed guidance for QA in each phase of life cycle of a CMGC (ECI) or DB project.  USACE uses a standard series of QM plans to codify, quantify and assess quality performance on both design and construction quality. 2.4.3 ISO 9000 QA Principles Applied to Alternative Project Delivery ISO 9000, promulgated by the International Organization for Standardization (ISO), offers a series of standards for an effective quality management program. The ISO 9000 requirements provide a common foundation to instill a quality culture in organizations that “embrace eight quality principles:  Customer focus,  Leadership,  Involvement of personnel, 30

 Process-based approach,  Systems approach to management,  Continual improvement,  Factual approach to decision-making, and  Mutually beneficial relationships with suppliers” (Miron et al 2008). ISO 9000’s QM objective is to “break down communication barriers, change paradigms, and ensure that every department in an organization knows how its work affects other processes or areas in the organization. Aligning a quality management system with the organization's current management system facilitates planning, allocating resources, defining complementary objectives, and evaluating the organization's overall effectiveness” (Miron et al 2008). At first glance, ISO 9000 appears to have no specific guidance with regard to highway design and construction. It does contain specific guidance for building construction that is directly applicable to the vertical projects that STAs may construct. The ISO web site contains no direct references to project delivery methods or highway construction. However, that fact does not mean there is nothing within the ISO body of knowledge that is not applicable to QM for highway projects delivered using alternative methods. ISO does provide a wealth of information on QM in design and manufacturing that has direct application to highway projects. For example, an article written in 1998 describes how an ISO 9002-certified Canadian contractor applies the fundamental principles of ISO 9000 to “produce, monitor and control its own concrete mixes, not only to ensure mix quality, but to more efficiently feed its continuous concrete paving operations” (Dufferin 1998). The primary application of ISO 9000 was on the contractor’s mobile concrete batch plant. This fits neatly into the manufacturing standards available from ISO. Since highway construction is equipment-intensive and becoming more highly automated, the opportunity to cross manufacturing standards to the appropriate construction system, such as GPS-driven construction machine guidance or in-place recycled asphalt paving trains is clearly transferable as shown by the ISO-certified Canadian batch plant. The FHWA used ISO 9000 principles to guide an initiative entitled: “Advanced Quality Systems (AQS)” (Miron et al 2008). The definition of AQS is: “An AQS is an integrated quality management system to fulfill the customer's expectations of pavement performance by making optimum use of the available tools and resources to continuously improve the system processes and the quality of the product delivered while fostering cooperative working relationships among all parties.” The term “integrated” is used in this definition in the same sense as it is used in project delivery method classification. An integrated project delivery method is one where the designer and the constructor work together during the design and construction phases. Thus, both CMGC and DB are integrated project delivery methods. Table 2-3 shows AQS tools developed by the FHWA and one can see that most of the tools are applicable across the project’s life cycle. For example, the Construction Quality Database is used in planning, design, construction, and post- construction. Therefore, the application of this ISO 9000-based initiative to alternative project delivery requires little or no alteration to be able to implement. 31

Table 2-3 – FHWA Developed AQS Tools (Miron et al 2008). Tool PP D V Ad C PC Ac M As CI T Construction Quality Database, www.fhwa.dot.gov/pavement/concrete/pubs/ 07019/index.cfm www.fhwa.dot.gov/pavement/concrete/pubs/ 07020 X X X X X X X X X X PaveSpec, www.fhwa.dot.gov/pavement/pccp/pavespec/ index.cfm X X X X X X X SpecRisk, Available on CD only; X X X X X X X Prob.O.Prof, http://dx.doi.org/ X X X X X X X HIPERPAV, www.hiperpav.com X X X X X X COMPASS, www.pccmix.com X X X X X X QA Program Effectiveness, http://dx.doi.org/ X X X Percent Within Limits: The Quality Measure of Choice (workshop) X Basic Pavement Warranty Workshop X SpecRisk Training (FHWA-NHI-134070) Web-based course currently under development. X PP = project planning; D = design; V = design verification; Ad = design adjustment; C = construction; PC = construction process control; Ac = construction acceptance; M = monitoring; As = assessment; CI = continuous improvement; T = Training 2.4.3.1 ISO 9000 Applied to Design QM in CMGC and DB Design QM is the place where ISO 9000 principles may be most applicable. The principles of “involvement of personnel” and “factual approach to decision-making” personify integrated delivery methods. Both DB and CMGC bring the members of the project team together at early stages in the design and involve them in developing the facts regarding costs, time, and technical performance for design alternatives before making the final design decisions. The Oregon DOT lists the major services that the construction contractor can perform during the design phase of a CMGC project as follows:  Cost Estimates  Schedule analysis  Work sequence  Risk identification/mitigation/pricing  Constructability reviews  Develop work packages for bid  Develop a GMP that meets owner requirements and budget restraints (Lee 2008) A study of the delivery of a water treatment plant in Florida contained an interesting design QM role for the CMGC. The CMGC preconstruction services contract contained a clause that required the CMGC to include a licensed design professional on its preconstruction team to 32

perform a technical peer review of the design at the 30%, 60% and 90% phases of design (Kwak and Bushey 2000). Water treatment projects are often rife with ISO 9000 process equipment design and manufacturing requirements and as such, often require both the designer and the construction contractor to be ISO 9000 certified (Battikha and Russell 1998). This requirement makes using an integrated method such as CMGC or DB an elegant way to an ISO 9000 certified team and ensuring the application of ISO 9000 QM principles from concept to ribbon cutting. ISO 9000 advocates that a critical factor in achieving high quality design is free and open communication between all parties during the design phase (Miron et al 2008; Beard et al 2001), and DBIA’s Manual of Policy Statements states: “DBIA advocates both formal and informal project partnering and considers the partnering philosophy to be at the foundation of design-build delivery” (DBIA 1998). During proposal preparation, the design-builder will make a number of design assumptions around which it will develop its proposed price. The FHWA AQS initiative recognizes this and prescribes: “Because the design and associated drawings and specifications define what the transportation agency wants, they all need to be consistent. Especially important: The design assumptions used to specify the quality of constructed pavement and predict its performance must be consistent with the quality and performance requirements called for in the specifications” (Miron et al 2008). Therefore, the ISO 9000 principles demand that the design- builder reconcile its design assumptions with the agency’s technical criteria and ensure that the requisite level of quality is achieved on a performance basis. 2.4.3.2 ISO 9000 Applied to Construction QM in CMGC and DB Construction QM in a CMGC project will not differ greatly from that found in a DBB project. The owner still occupies the same contractual position with respect to the designer and builder. Therefore, the ISO 9000 systems in use in DBB projects will directly apply to CMGC projects with little alteration. The key difference is the change in motivation of the constructor with regard to quality. In DBB, it has no input to the design and must build what is shown in the construction documents. In CMGC, the contractor assists in developing the final design and as a result assumes a significant degree of ownership in the design product. NCHRP Synthesis 402 (Gransberg and Shane 2010) described the idea of having “buy-in” to the design making the CMGC less prone to submit a claim for additional compensation for design problems in features of work for which the CMR had been paid to review and furnish input. This notion was confirmed a Florida study of the delivery of a water treatment plant using CMGC (Kwak and Bushey 2000). In that project, much of the project’s process equipment had ISO 9000 standards associated with it. Because of the rate at which water treatment technology is evolving and the constant threat of environmental standards changing, these types of projects often have need to implement “supplemental technology late in the design process and construction.” The study concluded: “This method of delivery provides for flexibility in the implementation of design changes late in the design process without impacting construction schedules and final delivery dates. The ability of the CM [GC] to input constructability reviews, construction phasing, material availability, and cost estimating as well as technical peer review throughout the design process reduces the probable occurrences of change orders [due to design errors], project construction delays, and increased project costs due to contractor identification of these elements in the design phase instead of the construction phase…Costs in terms of 33

initial, operational, and maintenance would tend to favor the CM [GC] quality and reject the low bid traditional method of project delivery” (Kwak and Bushey 2000). STAs are accustomed to describing, in detail, the means and methods used to carry out the construction of their transportation projects in the standard specifications for construction. The use of prescriptive specifications for DBB projects has been proven over time to be successful in yielding a quality product. In DB, however, STAs have the opportunity to allow design-builders to use specific construction means and methods to differentiate themselves from their competitors and to provide efficiencies that may not have been contemplated by the project’s owner. This creates an opportunity to use ISO 9000 certification of construction companies as a mechanism to mitigate the risk that the construction means and methods used by the design- builder may not achieve the same quality as those prescribed in DBB contracts. Thus, ISO 9000 furnishes a criterion to articulate the amount of flexibility the design-builder will have over construction means and methods in a DB highway project (Battikha and Russell 1998). 2.5 Design Quality Management Tools The design phase of a project is when STAs quantify and define the ultimate quality of the constructed facility through the production of construction documents. A previous study of DB quality management stated: “Quality cannot be assumed into the project. It must be designed and built into the project in accordance with the DB contract itself” (Gransberg and Molenaar 2004). It is intuitively obvious that there is a relationship between the final quality of the construction and the quality of the project’s design. Thus, design QM activities are necessary to assure the final quality of the products produced during design development as indicated by the following quote: “Quality documents facilitate quality construction… Review of the constructability of transportation facilities in the planning and design phases, specifically [for] deficiencies in quality and clarity of construction plans is critical…Constructability reviews…are the key mechanism for insuring that plans and specifications fulfill these quality objectives.” (Dunston et al 2002 italics added) An important factor in producing a design that meets a high standard is free and open communication between all parties during the design phase (Beard et al 2001). Harking back to the I-35W bridge collapse, the investigators found, “Insufficient bridge design firm quality control procedures for designing bridges, and insufficient Federal and State procedures for reviewing and approving bridge design plans and calculations” (NTSB 2008). Although delivery of this bridge used DBB, the designer’s QM program and owner’s design QM program were both found lacking. This indicates a failure to communicate the requirements for quality in both directions. 2.5.1 Design Quality Management in DB Projects As design details define construction quality requirements, it would follow that STAs who must commit themselves to the cost of construction prior approving the project’s final design, as happens in DB, would devote a significant portion of their DB solicitation packages to defining 34

the required design quality management process, and that, in turn, would cue design-builders to prepare design quality management plans that detail their proposed process for each specific project that can be evaluated as a part of the selection process. Unfortunately, in practice this is not occurring. A previous study of design quality management requirements in 75 DB projects across the nation found that only 18% of the DB solicitation documents required submission of a design quality management plan as part of the DB proposal (DeCorso 2004). Additionally, only 17 of these projects required a design QC plan after award and only two took the next step by requiring a complimentary design QA plan. Thus, the literature shows that design quality management is an area where the greatest potential for improvement is present. Perhaps this is due to a lack of policy guidance in the area of design management due to the DBB practice of public engineering agencies traditionally doing much of their design work using in-house professional engineers. Thus, STAs are not availing themselves of the opportunity to evaluate different design-builders’ approaches to ensuring design quality by not asking for design quality management plans prior to award. STAs give up control of the details of design by selecting DB project delivery, thus depending merely on the qualifications process to guarantee design quality, like in a pure design contract, may not be sufficient. With the dominant organizational type being a constructor-led DB team (Songer and Molenaar 1996), the designer’s client is no longer the owner. Therefore, it would seem to be imperative that the DB teams’ approach to producing a quality design be evaluated prior to award. Thus, having a clear definition of design quality management is imperative. The Minnesota Department of Transportation (MnDOT) furnishes an excellent definition for quality management during the design phase of a DB project when it laid out the objectives of the Design Quality Management Plan. The intent of the plan is to:  Place the primary responsibility for design quality on the design-builder and its designer(s).  Facilitate early construction by the design-builder.  Allow the Department to fulfill its responsibilities of exercising due diligence in overseeing the design process and design products while not relieving the design-builder from its obligation to comply with the contract. (Gonderinger 2001). One can see that MnDOT’s three-pronged approach not only satisfies its obligations for project oversight due to federal funding, but also ensures that the responsibility for the quality of the design is placed clearly on the design-builders’ shoulders. It also speaks toward achieving a major benefit accrued by the owner when selecting DB project delivery: project schedule compression through overlapping design and construction activities. Thus, it becomes important to adopt not only a good definition for design quality management but to also define clearly the allocation of responsibilities between the STA and the design-builder after project award. 2.5.1.1 STA Guideline Approaches for Design QA When reviewing the guidelines published by various STAs, there are three different policies established in determining the QC and acceptance roles in DB projects: 35

 Variable assignment of design QC and acceptance responsibilities on a project-by-project basis.  Assigning design QC to the design-builder and the design acceptance to the STA.  Assigning design QC and acceptance to the design-builder with the STA performing oversight and verification (i.e. design PQA). Variable Approach In the first approach, the assignment of design QC and acceptance roles can be varied from project to project. This recognizes that every project is different and that, depending on size, delivery speed and technical complexity, the optimum assignment of QC and acceptance responsibilities will be different based on individual project needs. This is shown in the Arkansas Design-Build Guidelines and Procedures: “The D/B package shall address any quality assurance requirements that the selected firm must follow in addition to those already in the referenced specifications, policies and procedures that will assure quality products (plans, materials, construction, etc.). Quality management criteria require at least three independent roles, including (1) quality control by the selected firm, (2) acceptance or verification by the Department’s Resident Engineer (RE) office, and (3) independent assurance by the Department’s central office staff. The responsibilities for all three roles and minimum sampling, testing and inspection frequencies shall be defined in the scope.” (Arkansas HTD 2006). STA Design Quality In the second type of design QC and acceptance approach, the design-builder is responsible for the design QC and the STA is responsible for the design acceptance. This approach parallels the DBB assignment of responsibilities for construction quality management. The Colorado DOT, Massachusetts Highway Department and the Florida DOT use this method and an example of this comes from the Arizona Design-Build Procurement and Administration Guide: The Design-Builder shall be required to submit a design quality management plan, which describes how the Design-Builder will control the accuracy and completeness of the plans, specifications, and other related design documents produced by the Design- Builder. …ADOT will retain a quality verification role as it does for other quality management issues. For design work, quality verification will be accomplished by design reviews led by the PM and performed by ADOT’s technical groups or the general consultant, if one is used. (Arizona DOT 2001) Design-Builder Design Quality Finally, the third approach assigns the design-builder both design QC and acceptance and the STA steps back from active participation and responsibility and, instead, only performs oversight and verification of design quality. The New York State DOT follows this approach. The contractual requirements for design management and QA/QC [quality control and quality acceptance] are the primary responsibility of the Design-Builder rather than the 36

Department…The Department’s project staff Oversight role during design and Design Review consists of monitoring and auditing design progress, interpreting contract requirements, and verifying design compliance with contract requirements. (NYSDOT 2005). Regardless of how the design QM responsibilities are assigned they must be performed. When the STA will perform the design acceptance activities, the contract documents (i.e. the RFP) need not further explain design acceptance activities. However, when the design-builder is assigned the design acceptance responsibilities, it is imperative to lay out the requirements in the RFP so that confusion is eliminated and the STA understands exactly what services they will receive with the proposal as well as how to integrate their QM activities during design. Reviews of the DB Design Deliverables One of the traditional ways that STAs have ensured quality design is by being able to fully review the design before it is advertised for bids. In DB, STAs do not have this same opportunity. One of the major advantages of DB is schedule compression, which happens by being able to start construction before the full design is finalized. In fact, in the survey 85% of STA respondents to the general survey indicated this as a reason for implementing DB. Another advantage of DB is the transfer of risk from the STA to the design-builder, and in the survey, 53% of STA respondents also indicated this as a reason for implementing DB. In a DB contract, the design-builder is responsible for the adequacy of the design in relation to the contract documents. STAs must be aware that “increased control over project design might not only reduce potential design-build benefits but might also carry with it the risk of liability for the entire project” (Wichern 2004). Thus, many states that do place the responsibility for design QC and acceptance on the design- builder use specified design review checkpoints, a design PQA activity, to ensure that the design is proceeding according to contract requirements. This also fulfills the STAs’ responsibility to the public to deliver projects designed and built in accordance with public law and good engineering practice. These checkpoints exist so that the design-builder’s final design is acceptable to the STA and is in accordance with the performance criteria contained in the contract documents. Design Review Checkpoints The NCHRP Synthesis 376 RFP content analysis found that there are two general ways to determine design review checkpoints. These are summarized in Table 2-4. The first method defines them in the RFP. The STA states in the RFP when to conduct reviews and what to include in the review. The design-builder must then account for the required reviews in their proposal and schedule of the project. This is by far the most common way to identify the required reviews. In the solicitation document analysis, 41 projects had design reviews as a requirement of the contract. Of these 41 projects, 83% told the design-builder at what point to review the design. 37

Table 2-4 – General DB Design Review Categories Type Design-builder responsibility STA responsibility Comments % of Projects in Content Analysis Defined Reviews To be responsive, must follow defined reviews in contract documents Defines reviews in the RFP Reviews may be performed by design-builder, STA or 3rd party 83% Proposed Reviews Propose design reviews for project as part of proposal or after award of contract Accepts or rejects proposed design reviews Reviews may be performed by design-builder, STA or 3rd party 17% The second approach, proposed reviews, is to allow the design-builder to propose the schedule of design reviews in their response to the RFP or during negotiations after the award of the contract. This is the stated policy of the Arkansas DOT: “There will be no pre-defined reviews scheduled by the Department. The selected firm and the Department will decide on the appropriate timing of reviews during execution of the contract” (Arkansas HTD 2006). 2.5.1.2 Appropriate Number of Design Reviews In addition to how the design reviews are defined, the number of required design reviews by the STAs varies from state to state. However, the content analysis identified three main trends:  No formal review prior to final (release-for-construction) design review,  One review prior to the final design being released for construction,  Multiple reviews prior to the final design review. Also, in many instances the design-builder is encouraged to request informal reviews that are not required but allow the STA to provide frequent input. This helps to ensure that the final design will meet the contract requirements. These reviews are often called “over-the-shoulder” or “oversight” reviews to indicate that the design process will not stop proceeding to wait for comments that result from these informal reviews. Table 2-5 provides a summary of the different categories of required number of design reviews and the corresponding percentage of occurrences in the RFP analysis. Table 2-5 – Required Number of Design Reviews % of Projects in Content Analysis Comments No review prior to final 15% STA still provides oversight and comments informally One review prior to final 56% Can be anywhere from preliminary design until just before the final design review Multiple reviews prior to final 29% The exact number of reviews can range from two reviews total to one review for every major feature of work 38

No Mandated Reviews When there is no STA-mandated design review checkpoint required before final design, the burden of design compliance is fully placed on the design-builder. In theory, this is one of the benefits of utilizing DB project delivery. However, the STA must still provide assurance that the contract will be completed with all the requirements met in a timely manner. In the RFPs analyzed for this project, 41 mentioned the design review requirements. Fifteen percent used the approach of no STA-mandated design review checkpoints before the release-for-construction design review. The Minnesota DOT detailed its design QA approach in one RFP as follows: The Department will participate in oversight reviews and reviews of early construction as part of its due diligence responsibilities. If the Department, in its review, observes that the Design-Builder is not complying with contract requirements and/or that the QC/QA [quality control and quality acceptance] checks are not complete, it will notify the Design-Builder in writing that construction may not proceed until the noted items are corrected. The Department’s oversight review and comments will not constitute approval or acceptance of the design or subsequent construction. (MnDOT 2001b) This QA activity (sometimes termed as due diligence) must be accomplished through an oversight approach as stated in the Minnesota RFP referenced above or by via a formal audit approach such as used by the Utah DOT (2005). The Utah RFP goes on to discuss the design review process for the final design deliverable. When the designer has completed a design package to 100% and the package has been checked and audited, a formal design submittal is assembled and distributed for review, including plan sheets, calculations, specifications, and other pertinent data. The Designer shall prepare for these reviews a full set of drawings and other documents stamped “Checked and Ready for Review.”…After the 100% comments have been addressed and the design documents have been checked and audited, a "ready to be released for construction" submittal package is assembled and distributed to the Design-Builder and the Department for release for construction. (UDOT 2005) To preserve the definition of design liability, Utah also requires the design-builder to complete a certification process on the final design package and specifies the time limit to which the STA must adhere to furnish timely acceptance. Single Design Review The second category of DB design review is where the STA requires a single official review of the design before the review of the final design deliverable. This gives the STA an intermediate point at which to verify that the design development is proceeding in accordance with the contract requirements and to ensure that it is progressing according to the schedule. The Mississippi DOT uses this type of design review for their DB projects. An example is listed below. 39

The CONTRACTOR will prepare and submit a single preliminary design submittal for the entire project. Preliminary design shall include roadway plan and profile, bridge type, selection layout, drainage, erosion control, signing, architectural and traffic control plans. MDOT will review Preliminary Design Submittals within 21 Days of the submittal…. (MissDOT 2005a) The requirement of only one official review by the STA is, by far, the most popular design review process currently used as found in the RFP analysis. Fifty-six percent of the RFPs analyzed for NCHRP Synthesis 376 used this type of design review process. Multiple Design Reviews In the final category of design reviews, the STA requires more than one official agency review before releasing the design for construction. This process was found in 29% of the RFPs that included information about design reviews. The Maine DOT required in one RFP that “formal design package submittals shall be made…at the 50% and 80% design development stage of any design package intended to be RFC [released-for-construction]” (Maine DOT 2003). Two variations on this category that were found and require mentioning in this section. The first is when the STA requires an independent design quality assurance firm to do the design reviews with the STA only providing limited oversight. This is the current situation with the SH 130 project in Texas. The RFP states: DQAM [design quality assurance manager] will conduct a formal over-the-shoulder review presentation to the TTA [Texas Turnpike Authority] at the TTA’s office. The over-the-shoulder review presentation will be held, following the DQAF’s [design quality assurance firm’s] approval of: the Corridor Structure Type Study Report; the Preliminary (30%) Design Submittal; the Intermediate (65%) Design Submittal; and the Final (100%) Design Submittal….Developer’s designer shall furnish to the DQAF at least five (5) mandatory design submittals, and if necessary, any resubmittals. (TTA 2001) The second variation is when the STA requires certain design reviews, attends the reviews, but is not the responsible party for the review. In the following example, the DB firm was responsible for the formal design reviews with the STA in attendance. The DQA [design quality assurance] Manager will conduct formal milestone reviews at the 30%, 60%, and 90% (or as otherwise agreed by the WSDOT and Design-Builder) stage of project elements to determine whether the Contract requirements and design are being followed and that QC/QA [quality control and quality acceptance] activities are following the approved QMP…The DQA Manager shall compile and maintain documentation of the review. The Department will be invited to attend these reviews. (WSDOT 2004) In the vein of deciding the appropriate number of STA design reviews for a given project, it is interesting to note that the US Army Corps of Engineers (USACE) changed its policy for DB design reviews, reducing the number of reviews from four (30%, 60%, 90%, and final) to two (intermediate and final) (USACE 2006). The reason for the change was to reduce the potential for delays due to waiting for government reviews. In a personal communication with the author, 40

Joel Hoffman of USACE explained the rationale as: “Philosophy is that once the designer-of- record approves construction and extension of design submittals, the builder can proceed - don't wait on us, unless there is a specific government approval required.” Thus, one critical issue regarding determining the appropriate number of design reviews is the need for the design- builder to maintain an aggressive schedule. If the project is not schedule-constrained, the DOT can afford to inject more design review points. Whereas, design reviews can be minimized on a fast-track project. Over-the-Shoulder Reviews In addition to the design reviews outlined above, another noticeable trend is the inclusion in the RFP of a statement inviting the design-builder to request informal “over-the-shoulder” reviews to ensure that the design is progressing according to the contract requirements without the need to prepare a specific design submittal package and to provide owner input to the design where it will be both desired and helpful. These statements are included in RFPs regardless of the number of required design reviews. Usually, however, a statement is also included that removes liability from the STA for any comments that may be incorporated from the informal reviews. Design reviews are an integral part of any design QM program. They ensure the constructability of the project, and they ensure that the design meets the contract requirements. Even though the design-builder is responsible for both of these in DB, STAs must provide themselves with assurance that the design-builder is carrying out its responsibility. Design Review Responsibility Communicating who is responsible for the design reviews is also essential to the smooth execution of these quality activities. This can be done in various ways, including lists, charts, diagrams, or designating responsibility in contract clauses. Table 2-6 is an example from a Louisiana DOTD DB RFP that provides a good example of how to communicate effectively design review responsibility. 41

Table 2-6 – Communicating Design Review Responsibilities (Louisiana DOTD 2005). Stage Of Design Development Design Check And Certification To Design-Builder Design Review Definitive Design Designer and Design Quality Control Manager Design Quality Control Manager Interim Review Designer and Design Quality Control Manager Design Quality Control Manager Readiness for Construction Design Designer and Design Quality Control Manager Design Quality Control Manager Final Design Designer and Design Quality Control Manager Design Quality Control Manager Working Plans and Related Documents Designer and Design Quality Control Manager Design Quality Control Manager As-Built Plans Designer and Design Quality Control Manager LA DOTD’s designated representative Major Temporary Components Designer and Design Quality Control Manager Design Quality Control Manager Temporary Components Designer and Checker Not applicable 2.5.2 Design Quality Management in CMGC Projects CMAA commissioned a study in 2005 (Doren et al) to survey owners about their perceptions on how project quality can be improved. The study’s top five responses all relate to enhancing the project’s quality during design by collaboration between the designer and builder.  A/E teams [designers] need to be more conscious of the cost to build their designs  More coordination/collaboration among team members  Need quality reviews from CMs  There needs to be a thorough review of the technical design details  Need to bring contractors, subs, and suppliers on board in the design phase (Doren et al 2005) The UDOT CMR report confirmed the same notions specifically for design quality. It also indicated agreement from both the design and construction industries in Utah. The report stated: The program managers and AGC representatives agree that contractor participation in design minimized risk and improved schedule. Design consultants preferred this method because UDOT controlled the design and innovations selected for the project. This gave them a greater ability to develop a quality design (Alder 2007). Another public agency report on its CMR project found that “[d]esign and peer review of the 30%, 60%, and 90% detail designs are required to ensure quality and constructability” (Kwak and Bushey 2000). This agency points to constructability as a measure that goes hand and hand 42

with quality. Design reviews are an integral part of any design QA program. They ensure the constructability of the project, and they ensure that the design meets the contract requirements (Dunston et al 2002). A survey on the benefits of constructability reported the following responses regarding its impact on a project:  Minimizes contract change orders and disputes,  Reduces project cost,  Enhances project quality,  Reduces project duration,  Increases owner satisfaction,  Enhances partnering and trust among project team (Pocock et al 2006). 2.6 Construction Quality Management Tools In DBB, STAs make their standard set of specifications for bridges and roads a contract requirement and then use the agency’s QM program to assure meeting the standards for construction quality expressed in the construction documents. Since the CMGC contractual structure is virtually the same as DBB, one would expect that there would be no significant change with regard to construction QM and there is no functional reason to make a change if the agency is comfortable with its traditional QM process. Such is the not the case in DB project delivery. DB project delivery inherently changes the traditional QM roles merely because the design- builder (i.e. the agency’s single prime contractor) produces the design and thus writes its own construction quality standards (FHWA 2012). DB project delivery is usually chosen to accelerate project delivery and that aspect in itself creates a greater demand on the QM system to be well defined and be designed to function at a pace of work not often seen in DBB or CMGC projects. 2.6.1 Construction Quality Management in DB Projects NCHRP Synthesis 376 (Gransberg et al 2008) used a content analysis of DB RFPs to identify trends regarding construction QA and QC functions on DB projects. It specifically sampled four major components of the construction QM process, normally retained by the agency, and identified the change in responsibilities for the following QA and QC activities:  Performance of submittal, shop and/or working drawing review and approval,  Performance of routine QC inspections,  QM testing,  Nonconforming work (punchlist). Table 2-7 shows the results of that analysis. With the exception of preparing and verifying the report of nonconforming work, the responsibility for construction QA was delegated to the design-builder. This may because the designer-of-record now works for the design-builder and 43

many of the traditional QA activities are performed by the agency that either completed the design in-house or holds a design contract with a design professional (Gransberg et al 2008). Nevertheless, the table clearly illustrates the shift in quality culture that is demanded by the employment of DB project delivery. Table 2-7 – NCHRP Synthesis 376, Quality Assurance in Design-Build Projects (Gransberg et al 2008) DB Construction QM Task Agency Only Design-builder Only Independent Consultant Combination of 2 or more Submittal approval 15% 62% 12% 11% Routine inspection 13% 51% 31% 5% QM testing 0% 95% 5% 0% Nonconforming work 64% 32% 0% 4% The central aspect of construction QM is QC testing. This is also an instance where the NCHRP Synthesis 376 content analysis found that STAs have overwhelmingly given responsibility to the design-builder. In fact, in the 39 projects that specifically mentioned construction QC testing in the DB RFP, the STA did not retain control in any of them. In all but two cases, the design- builder was assigned direct control. In the two exceptions, a third party was required to perform part of the testing. STAs, however, did not give up the right to make further inspections or to perform their own verification and acceptance testing. The following example from an RFP issued by the Mississippi DOT illustrates how they gave the responsibility of construction QC testing to the design-builder while retaining verification and acceptance testing responsibilities. “The CONTRACTOR is required to conduct concrete and asphalt sampling and testing in accordance with MDOT Standard Specifications for Road and Bridge Construction ... The CONTRACTOR may elect to conduct other sampling and testing for his own benefit ... [MDOT] or its duly authorized representative may conduct QA inspections, verification sampling and testing for concrete and hot mix asphalt, all other acceptance testing, and independent assurance testing.” (MissDOT 2005b) The Texas DOT utilized an independent consultant in their DB RFP for State Highway 130, a mega-project with its own unique characteristics. Because of the magnitude of the project, TxDOT chose to rely on an independent QA firm. The RFP indicated the independent firm’s responsibilities as follows: For quality assurance purposes, the department shall provide or contract for, independently of the design-build firm, any inspection services or verification testing services necessary for acceptance of the transportation project. (Koch et al 2010). The above discussion leads to the conclusion that that the STA must ultimately ensure that the construction QM system supports rather than constrains project progress. It can accomplish this by either giving the design-builder full responsibility for construction QM or by inserting an independent quality firm to furnish full-time, on-site resources. In doing so, the STA is ensuring that it has clearly delineated the requirement for QC testing during the course of construction. 44

2.6.2 Construction Quality Management in CMGC Projects Establishing construction phase quality management relationships occurs in the procurement phase, nurtured during the design phase, and applied when work begins on the project. Before starting work, the contractor needs to know and understand the roles of the agency’s QA personnel as well as the designer’s responsibilities during the construction of a CMGC project. “It is also recognized that, because of constrained staffing and budgets, it is not possible for state agencies to ’inspect’ quality into the work. Therefore, a procurement process is needed that considers value-related elements in awarding contracts” (Scott et al 2006). The value found in CMGC is the in-depth knowledge of the design that the constructor gained during the preconstruction when it starts construction. Additionally, the presence of contingencies further ameliorates the motivation to cut corners on quality as a means of recovering financial losses due to contractor or subcontractor error (Trauner 2007). Finally, the issue of remaining competitive for future work with the agency is a strong motivator to achieve the requisite construction quality and satisfy the owner. “CM At-Risk is still largely a position of representing the Owner’s interests, and if it is pursued as simply a negotiated general contract it will leave the Owner dissatisfied and the CM without future work” (Strang 2002). NCHRP Synthesis 402 found that 80% of the STAs studied made no changes to their traditional DBB QM system. The ones that did were all delivering projects that were nontraditional STA projects with strong vertical construction content. The synthesis found four possible entities to which these responsibilities were assigned are shown in table 2-8. The independent consultant is not the same as the designer. The Alaska, Florida, Michigan, and Utah DOTs as well as the Utah Transit Authority and the Memphis Airport all retained consultants to perform construction quality assurance services. The sum of the number of times a given entity was assigned a task is at the bottom and gives an indication of the distribution of construction quality management responsibilities in the case study CMGC projects. The agency and its independent consultant share the most responsibility, then the design consultant and finally the CMGC firm. The CMGC’s assigned roles appear to line up almost with what one would expect to see in a DBB construction QC plan. The only anomalies are one instance of shop drawing review and two instances of verification testing. The shop drawing review occurred in Michigan where the CMGC was required to conduct joint reviews with the designer of certain subcontractor submittals. The CMGC verification testing was done jointly with the agency’s staff in the Pinal County project and the Utah Transit Authority assigned this responsibility to the CMGC exclusively. The Utah project put 4% the construction fee at risk on each monthly progress payment (Touran et al 2009). The decision as to how much was awarded was made by a panel and included the evaluation of construction quality performance. 45

Table 2-8 – Distribution of Construction Quality Management Tasks in Case Study Projects Construction Quality Management Tasks Assigned Responsibility* Does not apply Agency Designer CMGC Independent consultant Technical review of construction shop drawings 0 1 9 1 6 Technical review of construction material submittals 0 2 7 0 7 Checking of pay quantities 1 4 3 0 6 Routine construction inspection 0 3 3 4 4 Quality control testing 0 0 2 10 1 Establishment of horizontal and vertical control 0 0 3 6 2 Verification testing 0 3 3 2 4 Acceptance testing 0 7 2 0 3 Independent assurance testing/inspection 0 3 3 0 5 Approval of progress payments for construction 0 9 0 0 3 Approval of construction post-award QA/QC plans 0 8 1 0 2 Report of nonconforming work or punchlist. 0 7 1 0 4 Total responses 1 47 37 23 47 *Can total > 10 as some agencies assign dual responsibility for same QM task. 2.7 Independent Assurance The Glossary of Highway Quality Assurance Terms (TRB 2005) defines independent assurance (IA) as follows: “A management tool that requires a third party, not directly responsible for process control or acceptance, to provide an independent assessment of the product or the reliability of test results, or both, obtained from process control and acceptance. [The results of independent assurance tests are not to be used as a basis of product acceptance.]” (TRB 2005). The joint Australian/New Zealand Standards Committee (A/NZS 1996) terms the independent assurance process a “configuration audit.” That group defines configuration as: “Functional and physical characteristics of a product as defined in technical documents [i.e. the design] and achieved in the product [i.e. constructed facility]. The definition of a configuration audit is “examination [by an independent entity] to determine whether a configuration item conforms to its configuration documents.” The operating verbs in this approach are “functional” and “physical.” As a result this approach is uniquely qualified to be adapted to alternative project delivery in the US, since DB project by definition are performance-based and CMGC projects furnish the agency an ability to verify/validate the physical constraints of the design before committing to construction. The A/NZS configuration audit has two parts. First, the auditor completes the “functional configuration audit” by verifying that the project’s contractual functional characteristics perform as designed. Secondly, a “formal examination of the ‘as-built/produced’ configuration” is completed and verifies that physical requirements are met. The one fundamental difference between this definition and the one from TRB is that projects in these two countries often contain 46

a contractual requirement for a configuration audit as part of the project acceptance process. The A/NZS approach externally validates the IQ2M model proposed in Figure 2-5 in the previous section. In fact, it allows the independent assurance aspect of the model to be further defined by adding the functional and physical components to the model as shown in Figure 2-8. Figure 2-8 – Expanded IQ2M Model to Further Define Independent Assurance Activities 2.7.1 Independent Assurance in DB Projects The Washington State DOT (WSDOT) provides a comprehensive definition of what it includes for independent assurance in its DB projects. WSDOT includes the following IA activities:  “Observation of initial qualification, annual re-qualification of sampling and testing procedures for quality verification (QV) testing staff.  Monthly initially then moving to Quarterly review/audit observations of the qualifications of the Design/Builder’s QA testing staff, with Non-conformance issues documented for Design/Builder—WSDOT resolution. Project Acceptance Construction Acceptance Design Acceptance Design Quality Control Construction Quality Control Design Released for Construction Construction Released for Final Payment Independent Assurance (if req’d) - functional audit -physical audit Owner Verification (if req’d) Quality Assurance Independent Assurance (if req’d) - functional audit -physical audit 47

 Monthly initially then moving to Quarterly review/audit observations of verification records of the testing equipment used to perform WSDOT QV and Design/Builder’s QA testing activities.  Initially month by month then moving to Quarterly review/audit observations of the Design/Builder’s QA tester training records.  Practical guidance for specification interpretation and procedural technique for WSDOT QV oversight personnel” (WSDOT 2007). WSDOT believes that the IA program can be used to “validate the:  WSDOT QV oversight of the Design-Builder QA sampling and testing processes.  WSDOT QV sampling and testing processes.  QA/QV statistical abnormalities ...”(WSDOT 2007). The WSDOT document states that “this work may include:  Auditing QA/QV testing records.  Observing the Design/Builder’s QA testing technicians.  Taking independent split samples with the Design/ Builder on a random basis for verification of Design/Builder’s QA testing procedures and equipment.  Taking independent split samples with the WSDOT on a random basis for verification of WSDOT QV testing procedures and equipment.  Act as a third party for analysis and recommend resolution of statistical abnormalities between Design/Builder’s QA and WSDOT QV.” (WSDOT 2007). The notable feature of the WSDOT IA program for DB projects is the use of audits in much the same manner as the A/NZS configuration audit approach. 2.7.2 Independent Assurance in CMGC Projects Since the contract structure in CMGC closely matches that of DBB contracts, one would expect that most agencies would feel comfortable using the same approach to CMGC IA activities as they do for DBB projects. NCHRP Synthesis 402 found this to be true for 80% of the case study agencies. It also found that the IA role was assigned to an independent third party consultant in only half the cases. In the others, either agency itself or the designer-of-record was given the responsibility for conducting IA. The Utah Transit Authority connected the quality of on-going construction with its progress payment clause and put 4% the construction fee at risk on each monthly progress payment (Touran et al 2009). The decision as to how much of the at-risk fee was awarded for each monthly pay application was made by a stakeholder panel and included the evaluation of quality performance. Thus, it created a de-facto IA body that essentially validated the QA information produced for each month’s construction progress as well as a disincentive for the constructor if it was not diligent in executing the approved QA plan. In nearly 3 years of construction, there was only one month where the CMGC did not receive the full amount of fee, 48

and the panel refunded the penalty amount the next month based on the speed and effectiveness of the corrective actions taken to remedy the problem. 2.8 Validation, Verification and Acceptance Activities It is important to remember the differences between the three terms to be able to understand how these concepts are applied in DB and CMGC projects. The Glossary of Highway Quality Assurance Terms (TRB 2005) offers the following definitions:  “Validation - The process of confirming the soundness or effectiveness of a product (such as a model, a program, or specifications) thereby indicating official sanction. [The validation of a product often includes the verification of test results.]  Verification- The process of determining or testing the truth or accuracy of test results by examining the data and/or providing objective evidence. [Verification sampling and testing may be part of an independent assurance program (to verify contractor QC testing or agency acceptance) or part of an acceptance program (to verify contractor testing used in the agency’s acceptance decision).]  Acceptance - The process of deciding, through inspection, whether to accept or reject a product, including what pay factor to apply” (TRB 2005). One can see that the three terms are both interrelated and distinct. Essentially the three forms of QM activities compose the tasks for necessary to release the design product for construction and to justify final payment for construction and project acceptance. 2.8.1 Validation, Verification and Acceptance Testing for DB Projects To support final project acceptance, the responsibility for final inspections as well as any verification or acceptance testing must be determined. The Synthesis 376 content analysis looked to see who was responsible for the verification and acceptance testing. As with the report of nonconforming work, STAs have generally retained this quality function. In fact, of the 40 projects that listed who was responsible for verification and acceptance testing, 88% assigned the responsibility to the STA. Another 5% required an independent firm that worked directly for the STA to accomplish this. Interestingly, three RFPs assigned this responsibility to the design- builder. The synthesis survey responses support these findings by indicating that 88% of the time the STA or a third party hired by the STA performs verification and acceptance testing while the design-builder is responsible only 12% of the time. While the majority of the states retained the verification and acceptance testing responsibilities, this does not mean that they will perform the actual tests with their own forces. Some STAs indicated in the RFP that the STA reserved the right to appoint a representative to perform the tests (see the example from the Florida DOT below). Standard RFP clauses are represented below.  Arizona: The design-build firm shall be responsible for the quality of the construction and materials incorporated into the project and is responsible for most QC actions. The Department has the responsibility of determining the acceptability of the construction and the materials incorporated into the project. The Department will use the results of the firm's inspection, sampling and testing, and the Department's surveillance inspection, and 49

verification sampling and testing to determine the acceptability of completed work items and for final project acceptance. Verification Sampling and Testing will be performed by the Department to validate Design-builder Sampling and Testing as well as the quality of the material produced. (Arizona DOT 1997)  Florida: The Department or Department’s representative will perform independent assurance, verification and resolution testing services in accordance with the latest Specifications. The Design-Build Firm will provide quality control testing in accordance with the latest Specifications. (FDOT 2003)  Minnesota: The Department, through its owner quality assurance (QA), will have the primary responsibility for verification of the quality of both the design and construction work. The Department reserves the right to conduct inspection, sampling, testing, and evaluation associated with QA and IA [independent assurance]. (MnDOT 2002)  Mississippi: [Mississippi DOT] or its duly authorized representative may conduct QA inspections, verification sampling and testing for concrete and hot mix asphalt, all other acceptance testing, and independent assurance testing. (MissDOT 2005a)  Utah: The Department will be responsible for construction QA. The Department will perform the same inspections and tests it performs on a standard design-bid-build project. (UDOT 2005) In three of the Synthesis 376 case study projects, the design-builder was made responsible for the verification and acceptance testing. One of these, the RFP for the Eastern Transportation Corridor in California, was for a toll collection and revenue management system where the verification and acceptance testing was based on a 60-day trial period after completion. For projects involving highway construction, however, the two projects that used design-builder verification and acceptance testing were a mega-project, the Colorado South East Corridor (SEC) Multi-Modal Project, and a major urban interstate makeover, I-64 in Missouri. Excerpts from the RFP are given for both projects.  Colorado: In cases where inspections are to serve as the basis for compliance verification, the Contractor shall prepare detailed inspection procedures and submit these to the SEC Representative for review. The Contractor shall conduct each inspection in accordance with the inspection procedures reviewed by the SEC Representative; no inspection shall be performed prior to obtaining the SEC Representative review of such inspection procedures. The Contractor in a suitable inspection report clearly showing if the inspection passed or failed based on the “pass/fail criteria” established in the procedure, shall document the results. (CDOT 2000 italics added)  Missouri: The following quality planning aspects shall be included in the Quality Manual…the Quality Assurance staff position responsible to perform the verification responsibilities including inspection, checking and testing…the method of performing Quality Assurance verification responsibilities including inspection, checking and testing. (MoDOT 2006 italics added) In another mega-project, the San Joaquin Hills Transportation Corridor, an independent firm was to be retained for the acceptance and assurance responsibilities. The RFP states: 50

The Construction Engineering Manager [employed by an independent firm] shall be responsible for coordinating and directing all Acceptance and Assurance inspections, sampling and testing to he conducted hereunder. (San Joaquin Hills 1991) Finally, in Utah’s I-15 mega-project, UDOT specifically listed a third party specifically to assist in fulfilling verification and acceptance testing responsibilities. (UDOT 1997) 2.8.2 Validation, Verification and Acceptance Testing for CMGC Projects Once again, the typical CMGC project’s validation, verification and acceptance testing program would not vary much from that found in typical QM in DBB projects. In Synthesis 402, only two of 10 case study projects assigned the role of verification testing to the CMGC and none released control of acceptance testing. In about half the cases, the agency retained an independent consultant to conduct validation, verification and acceptance tests. A concern found in the literature was that the faster pace of CMGC projects would create an environment where validation, verification and acceptance test results might be compromised and hence the quality of the ultimate project would suffer. NCHRP Synthesis 379 (Anderson and Damnjanovic 2008) evaluated the potential for and impact of alternative construction method to accelerate completion. The authors found that quality was not reported to be degraded by the use of nontraditional contracting methods such as CMR and also made this observation: “The majority of respondents stated that quality was the same for the contracting methods evaluated as compared with typical projects. This result seems to counter the perception that accelerating project completion negatively impacts quality, which was cited as a perceived disadvantage for some contracting methods.” (Anderson and Damnjanovic 2008) 2.9 Benchmarking Survey 2.9.1 Purpose The purpose of this survey was to define the state-of-the-practice with regard to the use of alternative quality management systems. The survey was designed not only to benchmark STA practices on projects delivered using alternative project delivery methods, such as CMGC, DB, and PPP, but also to uncover alternative quality management systems that are in use on traditional DBB projects. The survey questionnaire was prepared based on the principles prescribed by Oppenheim (1992) for survey questionnaire design. The survey contained the following four sections (See Appendix B for survey details): 1. General Information: This section captured the data necessary to identify the agency and quantify its experience with alternative QM systems. 2. Agency Quality Management Policy/Procedures Information: This section sought to identify the responding agency’s approach to QM is the following phases of the project life cycle: a. Procurement phase: Actions taken regarding the quality management process that are reflected in the agency’s contractor prequalification requirements and/or 51

solicitation documentation such as in the Invitation to Bid (IFB), Request for Qualifications (RFQ) and the Request for Proposals (RFP). b. Design Phase (in-house): Actions taken after approval to start design work regarding ensuring the quality of the design deliverables as well as that the final design complies with contractual requirements. OR c. Design Phase (out-source): Actions taken after design contract award regarding ensuring the quality of the design deliverables as well as that the final design complies with contractual requirements. d. Construction Phase: Actions taken after contract award regarding the quality of the final constructed product to ensure that it complies with both the completed design and other contractual requirements. 3. Quality Management Planning: This section for information regarding how the agency planned QM operations on typical projects. 4. Quality Management Program Effectiveness: This section asked the respondents cogent questions regarding the effectiveness of the agency’s QM program in a variety of typical situations. 2.9.2 Survey General Information Results The survey drew 22 responses from 19 states. Figure 2-9 shows the respondents, which also shows each state’s experience with various project delivery methods. The map shows that the responses were not only from a cross-section of STAs but also from a group with varied experience with alternative project delivery methods. One fact that was important to the analysis of survey responses was the amount of experience each respondent had with various project delivery methods. Table 2-9 shows the results for this question. Since the roles and responsibilities for various QM tasks are often assigned based on type of project delivery method (Gransberg et al 2008) and research has documented a learning curve as public owners become familiar with a new project delivery method (Gransberg and Molenaar 2004), it is important to analyze the survey responses in light of the context of a given agency’s experience with a specific project delivery method. Since DB is the most often used alternate project delivery method, the population was split into those with greater than 5 years of experience with DB and those with less than 5 years of experience. The idea that since DB requires an owner to delegate design and construction QM tasks to the design-builder, STAs with significant DB experience will have developed an approach to QM based on lessons learned that might be different than the ones used by STAs that are new to alternative project delivery and STAs with only DBB experience (Ernzen and Feeney 2002). 52

Figure 2-9 – Survey Responses Table 2-9 – Survey Respondents’ Experience with Each Project Delivery Method Project Delivery Method Years of Experience with Given Project Delivery Method 1-5 6-10 >10 DBB 0 0 22 All CMGC 4 1 0 AK; CO; CT; OR UT - DB 5 3 9 CA; MO; NV; TN; VT AK; KY; NM CO; GA; MN; NJ; OH; OR; UT; VA; WA PPP 9 1 2 AK; CO; GA; MN; MO; NV; OH; OR; TN UT CA; VA To ensure that Table 2-9 is not misinterpreted, it should be noted that based on the wording of the survey question, a state with authorization to use a given project delivery method but no experience would be forced to answer “1-5 years.” Tennessee is an example of an agency with the authority to use PPP but with no actual project experience. Additionally, the questionnaire 53

instruction directed a state that has not completed a project using a given method but has one or more projects under development to also answer “1-5 years.” California is an example of this having only received DB authority within the past two years. In spite of these two special cases, the impact on the survey analysis is nil since the two subpopulations were split between those with five or less years of DB experience and those with more. Respondents were also asked to differentiate alternative project delivery authority between authority for only pilot projects or only specific project types (i.e. PPP only allowed for tolling projects) and general authority to use the given methods without constraint. Table 2-10 shows the results of that query. Table 2-10 – Legislative Authority for Alternative Project Delivery Use Project Delivery Method Pilot/Specific Projects Only General Authorization DBB 0 22 CMGC 4 1 DB 7 10 PPP 7 5 2.9.3 Agency Quality Management Policy/Procedures Information The survey sought to identify the state-of-the-practice regarding agency QM policy and procedures for all project delivery methods. Alternative project delivery methods require the alteration of QM policies and procedures used in DBB and thus are inherently classified as alternative QM systems (Arditi and Lee 2004; Blanding 2006; Drennon 1998; Lee 2008; Scheepbouwer 2010; VDOT 2006; Yuan et al 2007). However, the survey and literature review found alternative QM systems, such as contractor acceptance testing, in use of DBB as well (Hughes 2005; Turochy et al 2007; Qaasim 2005; Parker and Turochy 2006; Miron et al 2008). Since QM is a concept that permeates the entire project’s life cycle, the survey attempts to differentiate QM policies and procedures by project phase: procurement, design, and construction. “The design phase . . . is the phase where the ultimate quality of the constructed facility is quantified through the production of construction documents . . . the point of the project where quality is defined . . . it is imperative that the design quality management responsibilities be clearly defined in the [procurement phase] solicitation documents” (Gransberg et al. 2008). Hence, CMGC, DB, and PPP project delivery requires adjusting the linear QM approach commonly used in DBB and sometimes called the “catch and punish method,” (Postma et al 2002; Koch et al 2010) where the management of project quality is transferred at the end of each phase. This notion gains confirmation through the survey results shown in Figure 2-10. Two- thirds of the respondents retain traditional QA roles in DBB and CMGC, the two methods where the owner retains control of the design (Koch et al 2010). Whereas, 75% of the respondents outsource QA on PPP projects and 71% do not retain traditional QA roles on DB projects. Also, one-third of the respondents delegate traditional QA tasks to the contractor on CMGC projects, which corresponds to the FHWA policy shown in Figure 2-2. 54

Figure 2-10 – Agency Quality Management Approaches by Project Delivery Method Finally, the survey asked if the QM system used on projects delivered using alternative methods was different than the one used for DBB projects and 64% responded that they were the same. Based on the responses shown in Figure 2-10, this is inferred to mean that the agency’s QM policy does not change. 2.9.3.1 Procurement Phase Agency Quality Management Policies NCHRP Synthesis 402 (Gransberg and Shane 2010 italics added) states: “Construction phase quality management relationships are established in the procurement phase, nurtured during the design phase, and applied when work begins on the project.” As such, articulation of agency policies regarding QM requirements occurs before any work actually begins or any design or construction product is produced. Put another way: “It is also recognized that, because of constrained staffing and budgets, it is not possible for state agencies to ‘inspect’ quality into the work. Therefore, a procurement process is needed that considers value-related elements in awarding contracts” (Scott et al. 2006 italics added). The survey sought to identify procurement policies that related to QM systems for each project delivery method. The first procurement phase decision deals with whether to complete the design or to procure a design consultant. Table 2-11 shows the survey responses regarding the amount of outsourced design completed by each respondent. The response ranged from a low of 10% in California to a high of 90% in Utah. The relative percentage of outsourced design is important to design quality because it creates an enforceable contractual relationship between the STA and the design consultant. It also permits the STA to require certain specific design personnel qualifications and 22.2% 0.0% 66.7% 5.6% 5.6% 0.0% 0.0% 66.7% 33.3% 0.0% 35.7% 7.1% 28.6% 0.0% 28.6% 0.0% 25.0% 0.0% 0.0% 75.0% 0% 10% 20% 30% 40% 50% 60% 70% 80% Agency uses two or more of the above depending on the project Agency retains an independent party to perform QA roles Agency retains traditional QA roles Contractor primarily responsible for QA/Agency audits contractor program Design consultant (or Design-Builder or Concessionaire) primarily responsible for QA/Agency audits consultant program P3 DB CMGC DBB 55

experience requirements that impact design quality. Table 2-11 – Level of Design Outsourcing Survey Responses Percent of Design Program that is Outsourced State 0-20% CA, MO, NV, OR 21-50% AK, CO,MN, NE,NM, OH, VT, WA 51-70% NJ, VA >71% CT,GA, KY, UT While in-house designs are in no way inferior to those completed by design consultants, STA project managers have less control over work assigned to agency design professionals (Alder 2007). Additionally, a STA’s internal design staff is a fixed entity with regard to professional qualifications and past experience, increasing the possibility that the design will be completed by a design professional with little experience in a given project. An analysis of the responses from those STAs that outsourced design percentages of >20%, >30%, and >40% versus the respondents that fell below those thresholds was conducted to determine if there were discernible differences in QM approaches was conducted as the amount of outsourced design increased. No trends were identified, which leads to the conclusion that quality is not affected by whether or not the project’s design is outsourced. The procurement phase furnishes the owner an opportunity to influence the “quality” of project design and construction personnel by establish criteria for individual qualifications and corporate experience with specific types of projects. Table 2-12 details the response to a survey question that asked respondents to articulate the content of their project solicitation documents. In DBB and CMGC projects, the solicitation documents included RFQs for the designer. For the reason discussed in the previous paragraph, it is unrealistic to sample projects that are designed in-house since there are no solicitation documents to procure the design services. The table ranked ordered the possible responses from the most used to the least used. The table shows that factors that related to design quality were more often cited than those related to construction quality. In fact, all the responses that had more positive than negative answers were design quality related. This indicates a shift in perspective from the results of earlier studies that found that design quality was rarely mentioned in DB solicitation documents (Gransberg and Windel 2008; Gransberg et al 2008). 56

Table 2-12 – Procurement Solicitation Document Content Do your project advertising/ solicitation documents contain the following? Required proposal/bid package submittal? (of 18 responses) If YES: Is it evaluated to make the award decision? If NO: Is it a required submittal after contract award? Yes No Yes No Yes No Qualifications of the Design Quality Manager 14 4 10 4 0 4 Qualifications of other Quality Management Personnel 12 6 8 4 3 3 Design quality management plan 11 7 11 0 3 4 Design quality assurance plan 11 7 11 0 2 5 Design quality control plan 11 7 10 1 3 4 Qualifications of the Construction Quality Manager 8 10 6 2 2 8 Quality management roles and responsibilities 8 10 6 2 3 7 Construction quality management plan 7 11 5 2 6 5 Construction quality assurance plan 6 12 5 1 4 8 Construction quality control plan 6 12 5 1 7 5 Construction testing matrix 5 13 1 4 2 11 Design criteria checklists 4 14 0 4 6 8 Quality-based incentive/ disincentive features 4 14 1 3 0 14 Warranties 2 16 0 2 1 15 Optional warranties 0 18 0 0 2 16 Table 2-13 contains a comparison of those STAs with less than five years of experience with alternative project delivery methods and those with more than that amount of experience. The table breaks the various solicitation factors into three categories: 1. Qualifications of Key Personnel 2. QM Plans 3. Contract Content The difference between the two subpopulations is striking. The experienced respondents’ solicitation documents contain more criteria than the less experienced agencies in all but one category, qualifications of other QM personnel. There is also a clear trend to more heavily emphasize QM planning during the procurement phase and inclusion of contractual constraints, incentives, and warranties. In addition, the experienced group emphasizes design QM more heavily than construction QM in their solicitation documents, which confirms the notion that ultimate project quality is defined the construction documents prepared in the design phase. This leads to the conclusion that QM starts in the procurement phase and that important QM issues 57

can successfully be addressed through the evaluation of competing design and construction proposals before awarding a DBB design contract, a CMGC preconstruction contract, or DB contract. It also lends weight to the conclusion that experienced STAs use the procurement phase as a tool to articulate project quality requirements and constraints. Table 2-13 – Comparison of Inexperienced Respondents with Experienced Respondents Do your project advertising/ solicitation documents contain the following <5yrs exp (% of subpopulation) >5yrs exp (% of subpopulation) Qualifications of Key Personnel Qualifications of the Design Quality Manager 75% 82% Qualifications of the Construction Quality Manager 20% 55% Qualifications of other Quality Management Personnel 75% 55% QM Plans Design quality management plan 25% 73% Design quality assurance plan 25% 73% Design quality control plan 25% 73% Construction quality management plan 20% 45% Construction quality assurance plan 0% 45% Construction quality control plan 0% 45% Quality management roles and responsibilities 25% 55% Contract Content Design criteria checklists 0% 18% Construction testing matrix 0% 36% Quality-based incentive/disincentive features 0% 27% Warranties 0% 18% Optional warranties 0% 0% 2.9.3.2 Design Phase Agency Quality Management Policies Defining the construction quality requirements takes place during the design phase. As a result, agency QM policies and procedures during the design phase create the environment in which the design is delivered and ultimately accepted by the STA. Central to the process of alternative project delivery is the definition and assignment of responsibilities for the suite of QM tasks that must be accomplished before the final design is released for construction (Gonderinger 2001; Arditi and Lee 2008; Pantazides 2005; Yakowenko 2010; Yuan et al 2007). To assess the state- of-the-practice in this phase, the survey asked a number of specific questions including the STAs’ policies for distribution of QM roles and responsibilities on project delivered using alternative quality systems. Table 2-14 and table 2-15 contain the output from that portion of the survey showing the total population and the two subpopulations. 58

Table 2-14 – Design Phase QM Responsibilities for Design Process Who performs the following design quality management tasks? RANK % RANK % RANK % Technical review of design deliverables Total <5yrs exp >5yrs exp Agency design staff 1 64% 1 60% 1 77% Agency project management staff 3 41% 2 40% 3 46% Project design consultant 2 50% 2 40% 2 69% Independent quality consultant 4 32% 5 20% 4 38% Project constr. staff in CMGC, DB, PPP 6 14% 6 0% 5 23% Other 5 18% 2 40% 6 15% Checking of design calculations Total <5yrs exp >5yrs exp Agency design staff 1 50% 1 40% 2 62% Agency project management staff 3 14% 3 0% 4 15% Project design consultant 2 45% 2 20% 1 69% Independent quality consultant 3 14% 3 0% 3 23% Project constr. staff in CMGC, DB, PPP 5 9% 3 0% 4 15% Other 5 9% 3 0% 4 15% Checking of quantities Total <5yrs exp >5yrs exp Agency design staff 1 59% 1 60% 1 62% Agency project management staff 4 14% 3 0% 4 23% Project design consultant 2 45% 2 20% 1 62% Independent quality consultant 3 23% 3 0% 3 31% Project constr. staff in CMGC, DB, PPP 6 9% 3 0% 5 15% Other 4 14% 3 0% 5 15% Acceptance of design deliverables Total <5yrs exp >5yrs exp Agency design staff 1 64% 1 40% 1 85% Agency project management staff 2 36% 2 20% 2 46% Project design consultant 3 14% 3 0% 3 23% Independent quality consultant 5 5% 3 0% 5 8% Project constr. staff in CMGC, DB, PPP 5 5% 3 0% 5 8% Other 4 9% 3 0% 4 15% Review of specifications Total <5yrs exp >5yrs exp Agency design staff 1 68% 1 60% 1 85% Agency project management staff 2 55% 2 40% 2 69% Project design consultant 4 32% 4 20% 3 46% Independent quality consultant 3 41% 4 20% 3 46% Project constr. staff in CMGC, DB, PPP 6 5% 6 0% 6 8% Other 5 23% 2 40% 5 15% NOTE: The percentages will not add to 100% because respondents were permitted to check all that applied. So if the agency design staff and a consultant shared a responsibility it would checked twice. Hence the important data is the ranking for each category. 59

Table 2-15 – Design Phase QM Responsibilities for Design Close-out Who performs the following design quality management tasks? RANK % RANK % RANK % Approval of final construction documents Total <5yrs exp >5yrs exp Agency design staff 1 55% 1 40% 1 69% Agency project management staff 2 45% 2 20% 2 62% Project design consultant 5 5% 3 0% 5 8% Independent quality consultant 3 14% 3 0% 3 23% Project constr. staff in CMGC, DB, PPP 6 0% 3 0% 6 0% Other 4 9% 3 0% 4 15% Approval of payments for design progress Total <5yrs exp >5yrs exp Agency design staff 2 32% 1 40% 2 31% Agency project management staff 1 59% 1 40% 1 77% Project design consultant 4 0% 2 0% 4 0% Independent quality consultant 3 14% 2 0% 3 23% Project constr. staff in CMGC, DB, PPP 4 0% 2 0% 4 0% Other 4 0% 2 0% 4 0% Approval of post-award design QM/QA/QC plans Total <5yrs exp >5yrs exp Agency design staff 2 27% 1 20% 2 38% Agency project management staff 1 41% 1 20% 1 62% Project design consultant 4 5% 2 0% 4 8% Independent quality consultant 3 23% 2 0% 3 31% Project constr. staff in CMGC, DB, PPP 4 5% 2 0% 4 8% Other 4 5% 2 0% 6 0% NOTE: The percentages will not add to 100% because respondents were permitted to check all that applied. So if the agency design staff and a consultant shared a responsibility it would checked twice. Hence the important data is the ranking for each category. Looking at the rankings in Table 2-14 and table 2-15, STAs retain the responsibility for design QM in most instances. The agency staff is ranked either first or second in every category of design QM task except checking quantities. There is also no difference in the assigned responsibilities between the experienced and inexperienced agencies. This leads to the conclusion that design QM is the primary responsibility of the owner regardless of project delivery method. 2.9.3.3 Construction Phase Agency Quality Management Policies Construction QM is “a domain once strictly held by the agency… [the] combination of increased pressure to perform faster along with decreased numbers of personnel and an increased dependence on outsourcing of the quality function has some in the transportation community concerned” (Ernzen and Feeney 2002). Table 2-16 and table 2-17 portray the responses to survey questions regarding construction phase QM roles and responsibilities. 60

Table 2-16 – Construction Phase QM Responsibilities for Routine QM Tasks Who performs the following construction quality management tasks? RANK % RANK % RANK % Technical review of construction shop drawings Total <5yrs exp >5yrs exp Agency design staff 1 59% 1 60% 1 54% Agency project management staff 3 36% 2 40% 4 38% Project design consultant 2 45% 2 40% 3 46% Independent quality consultant 4 32% 4 0% 1 54% Project constr. staff in CMGC, DB, PPP 5 9% 4 0% 5 15% Other 6 5% 4 0% 6 8% Technical review of construction material submittals Total <5yrs exp >5yrs exp Agency design staff 3 32% 1 40% 3 31% Agency project management staff 1 45% 1 40% 2 46% Project design consultant 3 32% 1 40% 3 31% Independent quality consultant 1 45% 5 0% 1 62% Project constr. staff in CMGC, DB, PPP 6 9% 5 0% 5 15% Other 5 18% 4 20% 5 15% Checking of pay quantities Total <5yrs exp >5yrs exp Agency design staff 5 5% 3 20% 5 0% Agency project management staff 2 50% 1 60% 2 46% Project design consultant 6 0% 5 0% 5 0% Independent quality consultant 1 55% 3 20% 1 62% Project constr. staff in CMGC, DB, PPP 4 9% 5 0% 3 8% Other 3 18% 2 40% 3 8% Routine construction inspection Total <5yrs exp >5yrs exp Agency design staff 5 9% 3 20% 6 0% Agency project management staff 2 45% 1 60% 2 38% Project design consultant 5 9% 3 20% 5 8% Independent quality consultant 1 55% 3 20% 1 62% Project constr. staff in CMGC, DB, PPP 3 23% 6 0% 3 23% Other 3 23% 2 40% 4 15% Quality control testing Total <5yrs exp >5yrs exp Agency design staff 6 0% 4 0% 6 0% Agency project management staff 3 23% 2 20% 3 23% Project design consultant 5 5% 4 0% 5 8% Independent quality consultant 1 55% 4 0% 2 31% Project constr. staff in CMGC, DB, PPP 4 14% 4 0% 4 15% Other 2 41% 1 40% 1 46% NOTE: The percentages will not add to 100% because respondents were permitted to check all that applied.. 61

Table 2-17 – Construction Phase QM Responsibilities for Acceptance and Close-out Who performs the following construction quality management tasks? RANK % RANK % RANK % Verification testing Total <5yrs exp >5yrs exp Agency design staff 3 18% 2 20% 6 8% Agency project management staff 1 41% 1 40% 2 38% Project design consultant 5 9% 2 20% 5 8% Independent quality consultant 1 41% 5 0% 1 46% Project constr. staff in CMGC, DB, PPP 5 9% 5 0% 3 15% Other 3 18% 2 20% 3 15% Acceptance testing Total <5yrs exp >5yrs exp Agency design staff 4 18% 3 20% 5 8% Agency project management staff 1 41% 1 40% 2 38% Project design consultant 6 9% 3 20% 5 8% Independent quality consultant 1 41% 5 0% 1 46% Project constr. staff in CMGC, DB, PPP 4 18% 5 0% 3 23% Other 3 23% 1 40% 4 15% Approval of progress payments for construction progress Total <5yrs exp >5yrs exp Agency design staff 5 5% 3 20% 5 0% Agency project management staff 1 59% 1 80% 1 54% Project design consultant 6 0% 4 0% 5 0% Independent quality consultant 2 45% 4 0% 2 46% Project constr. staff in CMGC, DB, PPP 4 14% 4 0% 3 15% Other 3 18% 2 40% 4 8% Approval of construction post-award QM/QA/QC plans Total <5yrs exp >5yrs exp Agency design staff 3 23% 3 0% 2 31% Agency project management staff 1 45% 1 60% 1 46% Project design consultant 5 9% 3 0% 4 15% Independent quality consultant 2 32% 3 0% 2 31% Project constr. staff in CMGC, DB, PPP 6 0% 3 0% 6 0% Other 4 18% 2 20% 4 15% Report of nonconforming work or punchlist Total <5yrs exp >5yrs exp Agency design staff 4 9% 2 20% 4 8% Agency project management staff 2 50% 1 60% 2 46% Project design consultant 4 9% 2 20% 4 8% Independent quality consultant 1 55% 5 0% 1 62% Project constr. staff in CMGC, DB, PPP 4 9% 5 0% 3 15% Other 3 14% 2 20% 4 8% NOTE: The percentages will not add to 100% because respondents were permitted to check all that applied. 62

A glance at table 2-16 and table 2-17 confirms the assertion made by Ernzen and Feeney (2002) with regard to “increased dependence on outsourcing of the quality function.” The majority of the experienced agencies delegated the construction QM authority to an independent quality consultant for every QM task, achieving a rank of either first or second for all possible categories. There is also greater disparity between the experienced and inexperienced agencies in construction QM than there was in design QM. The rankings show inexperienced agencies retain the construction QM responsibility in most instances. The lone exception is QC testing, where most experienced and inexperienced agencies selected “other” as their answer. The explanation for all other responses indicated that the construction contractor was assigned this responsibility and is consistent with the FHWA policy shown in Figure 2-3. The difference between the two subpopulations in the construction QM area infers that the experienced agencies advanced on the alternate QM system learning curve and are successfully outsourcing traditional construction QM responsibility to an independent quality consultant. Federal Regulation 23 CFR 637.207(b) mandates agency oversight (sometimes termed “due diligence”) of QM on federal-aid highway projects. The experienced agencies have satisfied this requirement by augmenting the STA personnel with consultants that specialize in QM. The Arizona DOT is one of these experienced agencies and a case study of a major urban freeway construction project found the following: “The reconstruction of Interstate 17 in Phoenix, Arizona, used two innovative concepts: design–build and a contractor-led quality management program… [the study found that] though the material quality is marginally higher with traditional design–bid–build contracting and agency controlled QA, the differences are very small and the results are very encouraging for other state highway agencies that are considering moving in this direction and that are concerned about project quality” (Ernzen and Feeney 2002). The survey also asked for information regarding the use of contractor QA test results. Figure 2-11 shows the output. Of the agencies that indicated the use of contractor tests, only one was an inexperienced agency. The major difference between DBB and the various alternative project delivery methods is the ability to select the constructor on a basis of qualifications rather than on a low bid. Adding qualifications to the award method apparently permits the experienced agencies to select contractors to whom they are willing to delegate QA acceptance testing. This may be necessary to achieve the aggressive schedules that justify the implementation of alternative project delivery. This notion is confirmed by the Arizona DOT I-17 DB project where “[d]espite the constraints of working on one of the most heavily traveled roads in the state, the design–build team… completed the project in 30% less time than Arizona DOT had originally estimated” (Ernzen and Feeney 2002). 63

Figure 2-11 – Contractor QA Test Usage 2.9.4 Quality Management Planning QM planning spans the entire life cycle of the transportation project. Ensuring that the requirements for alternative design and construction QM systems are clearly defined is critical to their successful execution (Gransberg and Windel 2008). The survey specifically sought to determine how prescriptive the responding STAs were in the alternative QM planning process. Table 2-18 illustrates the output from a series of questions regarding this specific topic. The responses were aggregated using a weighted average technique so that they could be ranked. The two highest ranked QM plan elements are very prescriptive with all agencies mandating the use of standard specifications and design details. This permits greater agency control over the final design. Research has found that the loss of control over the details of design is a major barrier to the implementation of alternative project delivery methods (Smith 2001; Lee 2008; Gransberg and Molenaar 2008; Scheepbouwer 2010; Touran et al 2010). The next highest ranked elements are procedures that allow the STA to customize its design and construction QM planning based on the requirements of a given projects. When combing the flexibility of the third ranked elements with prescription of the first two, one finds that agencies that utilize alternate QM systems are able to control design quality issues by mandating tried and true details and specifications while allowing their contractors the flexibility needed to achieve aggressive project delivery schedules with minimal agency interference. The Minnesota DOT, an experienced agency, is a typical example of this prescriptive yet flexible approach to alternative QM. Minnesota is one of the agencies that requires no formal design reviews. Its DB design QM policy is synopsized as follows: “The Department will participate in oversight reviews and reviews of early construction as part of its due diligence responsibilities” (MnDOT 2001a). It also uses an innovative process to control design details by meeting with each competing design-builder before proposals are due to confidentially and individually 0 2 4 6 8 10 12 14 Does your agency utilize contractor QA test results for acceptance on any of its projects? If yes, do you use a performance based prequalification process in conjunction with the contractor acceptance testing program? If no, would you use QA acceptance testing if you could prequalify contractors and/or their quality assurance personnel on a performance basis? Agency utilization of contractor QA test results for acceptance on any projects Yes No 64

approve alternative technical concepts as well as lock down the content of the design before awarding the contract (MnDOT 2010). The ability to influence design decisions before award then allows MnDOT to be flexible during design and construction and minimize its potential interference with progress while satisfying its statutory obligations under 23 CFR 637.207(b). Table 2-18 – Quality Management Plan Components. Elements of the QM Plan Always Some-times Never Weighted Average Rank Agency mandate the use of standard agency specifications? 14 4 0 16.67 1 Agency mandate the use of standard agency design details? 10 5 0 13.33 2 Construction QM plans used on CMGC/DB/PPP projects significantly different from the QM plans used on traditional DBB construction projects? 5 8 6 12.33 3 Design QM plans used on CMGC/DB/PPP projects different from the QM plans used on traditional design projects? 4 10 5 12.33 3 Agency mandate the use of standard agency construction means and/or methods? 3 11 3 11.33 5 Agency mandate the use of its own standard QM plans? 6 5 4 10.67 6 Agency specify what must be included in the CMGC/DB/PPP QM plans? 6 4 3 9.67 7 Agency mandate a specific set of qualifications for the QM staff of design consultants and construction contractors in DBB projects? 3 7 5 9.33 8 Agency mandate a specific set of qualifications for the design-builder/PPP concessionaire's design QM staff? 4 5 2 8.00 9 Agency mandate a specific set of qualifications for the design-builder/PPP concessionaire's construction QM staff? 3 6 1 7.33 10 Agency mandate a specific set of qualifications for the QM staff of design consultants and construction contractors in CMGC projects? 1 2 7 4.67 11 2.9.5 Quality Management Program Effectiveness The final topic covered in the survey was the effectiveness of alternative QM systems. The survey asked the respondents to rate the impact on quality of a number of different factors found in the literature (Gransberg et al 2008) to identify trends between the two subpopulations regarding perceived effectiveness of those factors. The factors are organized from least to most impact based on the inexperienced agency responses. A glance at Figure 2-12 confirms that there are indeed differences in the two groups. 65

Figure 2-12 – Rated Impact on Final Quality of Various Factors 0 2 4 6 8 10 12 14 16 18 20 Qualifications of the construction contractor’s staff Qualifications of agency project management staff Qualifications of the design consultant’s staff Qualifications of agency design staff Qualifications of agency construction staff Construction contractor’s past project experience Incentive/disincentive provisions Use of performance criteria/specifications Design consultant’s past project experience Use of agency specifications and/or design details Allowing flexibility in choice of design standards and construction specifications Detailed design criteria Use of manuals, standards and specifications developed for DBB type projects Warranty provisions Submittal of QM plans prior to work start Level of agency involvement in the QM process Level of detail expressed in the procurement documents (IFB/RFQ/RFP) Innovative financing (PPP/concession) Follow-on maintenance provisions >5yrs exp <5yrs exp 66

2.9.6 Impact on Quality To understand better the differences in the perceived impact on quality, the absolute value of the difference between the rank assigned by the inexperienced agencies and the experienced agencies was calculated and listed in table 2-19. Combining these with the information in Figure 2-12 allows one to infer the trends in this data. The greatest difference was the impact of the information contained in the procurements documents (IFB/RFQ/RFP). Experienced agency ranked this having high impact whereas the inexperienced agencies ranked it very low. This comparison shows the value of dividing the two groups to benchmark the state-of-the-practice. In DB and PPP projects, the technical scope of work contained in the procurement documents becomes an operating part of the final awarded contract (Loulakis and Shean 1996; Yakowenko 2010). Hence, the high rank assigned by the experienced agencies is justified and the low rank assigned by the inexperienced agencies demonstrates a lack of understanding that QM begins in the procurement phase and scoping decisions expressed in the solicitation documents ultimately impacts final project quality. Table 2-19 – Difference in Ranking of Experienced and Inexperienced Respondents Factor |∈| Factor |∈| Factor |∈| Level of detail expressed in the procurement documents 9 Use of agency specifications and/or design details 5 Qualifications of agency construction staff 2 Qualifications of agency project management staff 8 Use of performance criteria/specifications 3 Detailed design criteria 2 Construction contractor’s past project experience 7 Allowing flexibility in choice of design standards and construction specifications 3 Qualifications of the construction contractor’s staff 1 Level of agency involvement in the QM process 7 Warranty provisions 3 Submittal of QM plans prior to work start 1 Design consultant’s past project experience 6 Qualifications of the design consultant’s staff 2 Innovative financing (PPP/concession) 1 Use of manuals, standards and specifications developed for DBB 6 Qualifications of agency design staff 2 Follow-on maintenance provisions 1 |∈| = Absolute value of the difference in impact rank Incentive/disincentives 0 The next big differences are in the perceived impact of the agency’s project management staff and the constructor’s past project experience on quality. The inexperienced agencies rated these higher than the experienced agencies. The experienced agencies felt that the qualifications of the agency’s design and construction staffs as well as the qualifications of the designer had more of a significant impact. The experienced group placed relatively high ratings on the use of standard specifications, design details and allowing flexibility in choosing which of these to use on a given project. This reinforces the conclusion drawn in Section 2.8.2 that experienced agencies control project quality by mandating the use of proven design details and specifications while allowing their contractors flexibility to make decisions in a manner that facilitates the increased speed when using alternative QM systems. 67

2.9.7 Challenges to Achieving Quality The final section of the survey asked the respondents to relate their experience with respect to achieving project quality. Again, it provided a list of factors found in the literature (Yuan et al 2007) that had been cited as creating a challenge to a project manager’s ability to achieve the desired levels of final project quality. Figure 2-13 shows the output from that exercise. Again, there are perceptional differences between the two groups. Both agreed that documenting the quality aspects of a project was the greatest challenge and in a similar vein, rated getting complete design documentation from their contractors as the next largest challenge. The opinions diverged after that with the experienced agencies rating misunderstood roles and responsibilities and the use of DBB contract language without regard to project delivery method much higher than the inexperienced group. This probably reflects a frustration with internal contracting policies and procedures where agency contracting personnel lag behind the agency project delivery personnel on the alternate QM learning curve. Koch et al (2010) describe this particular phenomenon as “creative tension” between process-oriented entities who prefer to standardize procedures and product-oriented entities that see each project as a different product demanding procedures customized to fit project requirements. Figure 2-13 – Challenges to Implementing Effective QM Using Alternative Project Delivery The experienced group also rated subcontractor quality control as a bigger challenge than the inexperienced agencies. This may reflect the fact that the agency has no privity with the project’s subcontractors and must make demands on the prime contractor to promulgate QM standards, policies, and procedures to its subcontractors. This also may be a challenge because the subcontractors often are new to alternative project delivery methods and are unfamiliar with the changes inherent to the move from traditional DBB to another delivery method. The reasons for both the differences rating discussed in this paragraph are conjecture and no conclusions can be drawn. 0 1 2 3 4 5 6 7 8 Obtaining complete verification documentation Designers that produce incomplete designs Poorly defined acceptance criteria Subcontractors that cannot control the quality of their work Contractors that cannot control the quality of their work Agency staff that cannot give up the need to control the quality of work Use of contract language that is from DBB and is not applicable to alternative project delivery Misunderstood roles and responsibilities 1= Least Challenging to 8= Most Challenging >5yrs exp <5yrs exp 68

2.10 Summary NCHRP Synthesis 379 comment by Anderson and Damnjanovic (2008) aptly points out the “popular mythology” that surrounds the culture shift from traditional project delivery and its “catch and punish” QM system to something different. There are always champions that promote the new method with evangelistic zeal and opponents that can see all sorts of unsolvable problems being spawned by the change in contractual relationships. Degraded quality is usually one of the disadvantages cited by opponents to change and the champions cite reasons why quality is in fact enhanced. The real issue with regard to quality is not how to guarantee that it will improve but rather to ensure that the change does not create a set of circumstances that causes it to decline. The FHWA Design-Build Effectiveness Study (FWHA 2006) effectively debunked this quality issue. The FHWA study team found that: “On average, the managers of design-build projects surveyed in the study estimated that design-build project delivery reduced the overall duration of their projects by 14 percent, reduced the total cost of the projects by 3 percent, and maintained the same level of quality as compared to design-bid-build project delivery”(FHWA 2006 italics added) Strang (2002) holds that public owners who “are not bold enough” to implement DB will be drawn to CMGC as a more palatable alternative to DBB. Design consultants that work for UDOT have indicated a preference for CMGC over DB because UDOT controls the details of the design (Alder 2007). Other agencies have found that CMGC furnishes a good option to both DBB and DB project delivery. “The performance to date of Tri-Met’s [Portland, Oregon] two major CMGC contracts for the Interstate MAX light rail line is exemplary from the perspective of keeping the project on schedule and maintaining good quality” (FTA 2003). The same report goes one to say: “With a more cooperative working partnership between the owner, the designer and CMGC contractor—Tri-Met calls these entities the CMGC team—work quality should improve. The contractor has been selected on factors other than just price, many of which are strong indicators of ability to complete the job successfully. Also, the CMGC contractor has increased responsibility for quality control over all aspects of the job under this method.” (FTA 2003) This critical review of the literature on QM as applied to alternative project delivery methods can be summed up as a success with regard to furnishing project delivery methods that “improves quality and value… [by keeping the] focus on quality and value – not low bid” (Ladino et al 2008). Thus, the notion that quality must be “engineered” into the project from the very start is confirmed and that using project delivery methods that don’t have a bias to minimize construction costs in fact reap the benefits of enhanced quality while controlling both cost and schedule of badly needed highway projects. This conclusion was confirmed by the survey results where experienced respondents reported success using alternate QM systems on projects delivered using alternative delivery methods. These agencies have struck a balance between prescriptive requirements to control design quality to allowing maximum flexibility to their contractors to facilitate aggressive schedules. 69