Guidelines for Optimizing the Risk and Cost of Materials QA Programs (2017) / Chapter Skim
Currently Skimming:
Pages 131-200
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 131... ...
P a r t 2 Guidelines
|
From page 132... ...
135 How much time and money should we devote to quality assurance? Will the results be worth the resources invested?
|
From page 133... ...
136 Guidelines for Optimizing the risk and Cost of Materials Qa programs The conceptual foundation for analyzing the economics of quality is therefore well established in the literature. Over the years researchers have proposed and refined a theoretical model for optimizing QA based on the principle of diminishing marginal returns (e.g., Juran, 1951; Kilpatrick, 1970; Plunkett and Dale, 1988; Hylton Meier, 1991; Morse, 1993; Schiffauerova and Thomson, 2006)
|
From page 134... ...
Introduction 137 activities on the basis of the probability and consequence of material failure. What such a qualitative process may lack in academic rigor, it can make up for in accessibility and ease of use.
|
From page 135... ...
138 Guidelines for Optimizing the risk and Cost of Materials Qa programs Level 1 Level 2 Level 3 Objective Adjust QA effort (e.g., greater reliance on certification and inspection) based on qualitative assessment of the risk of material failure Adjust owner QA testing effort based on material property importance and risk Obtain the optimal QA investment point based on an explicit consideration of the cost of QA versus the cost of non-conformance Inputs • Materials of interest • Standard acceptance plans • Expert judgment regarding material risk • Standard acceptance plans • Primary and secondary properties and test methods • Traditional and advanced properties and test methods • Expert judgment regarding material risk • Cost to implement different levels of QA (testing, inspection, certification)
|
From page 136... ...
Introduction 139 Subsequent chapters then present a detailed discussion of the framework methodology itself: • Chapter 3 describes a "Level 1" materials-based optimization approach, which uses a qualitative risk-based rating process to prioritize materials and QA activities on the basis of the probability and impact of material failure or non-conformance; • Chapter 4 describes a "Level 2" optimization of material properties (e.g., strength, density) for materials subject to sampling and testing; and • Chapter 5 describes a "Level 3" cost-based optimization process that balances the cost of different QA protocols against the cost of potential material defects to determine an optimal QA investment point.
|
From page 137... ...
140 C h a p t e r 2 To provide context for the optimization processes presented later in this manual, this chapter describes the current state of materials QA in the highway construction industry. A general summary of standard methods used by DOTs to assure materials quality is presented first, followed by a discussion of some existing optimization strategies being used by DOTs to achieve better efficiencies in quality management.
|
From page 138... ...
Materials Qa State of the practice 141 for managing quality, increased understanding of materials behavior, and innovations in nondestructive testing and related technologies are providing additional motivation for DOTs to revisit their existing QA practices. Examples of strategies DOTs are implementing to optimize materials QA are discussed in the following subsections.
|
From page 139... ...
142 Guidelines for Optimizing the risk and Cost of Materials Qa programs The DOTs are expected to perform enough verification sampling and testing to be able to identify statistically valid differences between its results and those of the contractor. [The F-test (comparison of variances)
|
From page 140... ...
Materials Qa State of the practice 143 the perspective of difficulty to repair or replace, safety, maintenance cost, or cost of rework. Examples of such strategies include the following: • The California DOT (Caltrans)
|
From page 141... ...
144 Guidelines for Optimizing the risk and Cost of Materials Qa programs or ability of the material to perform as intended is in question, the project engineer must determine if visual acceptance is appropriate or if additional acceptance testing or certification is necessary. The materials included on the list include items such as erosion control materials and miscellaneous fittings and hardware.
|
From page 142... ...
Materials Qa State of the practice 145 base course and embankment construction would require frequent to constant inspection based on the specific inspection item in question (e.g., embankment lift height would require frequent inspection whereas density would require constant inspection)
|
From page 143... ...
146 Level 1: Materials-Based Optimization As introduced in Chapter 1, this guidebook presents a three-level analytical framework to help DOTs identify the QA acceptance plan necessary to meet specification requirements within an acceptable level of risk. Briefly, these levels can be described as follows: • Level 1 entails conducting a qualitative risk rating of materials and then aligning these ratings with QA methods that can provide reasonable assurance of acceptable quality.
|
From page 144... ...
Level 1: Materials-Based Optimization 147 Given unlimited time and resources, DOTs would likely benefit from reviewing and optimizing the QA protocol for all materials included in their Standard Specifications. If, however, such an overarching study is not practical, smaller subsets of materials could be explored on a programmatic basis (e.g., safety-critical materials, fabricated items, project or field-produced materials)
|
From page 145... ...
148 Guidelines for Optimizing the risk and Cost of Materials Qa programs As described in greater detail below, determining a given material's risk of failure requires assessing: • The probability (or likelihood) of receiving non-conforming material and • The impact (or consequence)
|
From page 146... ...
Level 1: Materials-Based Optimization 149 Similar to estimating probabilities, all participants must share a common understanding of "impact" for the process to be effective. Possible perspectives from which to consider impacts include safety, difficulty to repair or replace, maintenance costs, and cost of work.
|
From page 147... ...
150 Guidelines for Optimizing the risk and Cost of Materials Qa programs In the example matrix shown in Figure 3.2, the probability rating is multiplied by the impact rating to arrive at an overall risk score. This score will then be used in the following steps to prioritize materials for the purpose of efficiently allocating QA resources on the basis of material criticality.
|
From page 148... ...
Level 1: Materials-Based Optimization 151 Risks related to materials quality are generally managed using some combination of prevention and appraisal techniques: • Prevention techniques refer to those measures taken to avoid poor quality. This could include prequalification (e.g., of materials, sources of supply, contractors, fabricators)
|
From page 149... ...
152 Guidelines for Optimizing the risk and Cost of Materials Qa programs • Submittals – Review of working drawings – Review of mix designs – Review of quality management plans Appraisal Methods: • Contractor quality control sampling and testing • Certificates of compliance – Certificate of compliance from a producer – Certificate of compliance from a producer with test results • Sampling and testing – Source and field – Contractor and agency • Inspection – Benchmark, intermittent, or continuous • Warranties Similar to most quality systems, the methods listed above represent a mix of measures used to both prevent and appraise material quality. Although it is possible that under certain circumstances one method alone could be used to assure the desired level of quality and performance of a material (e.g., requiring the use of materials selected from a qualified products list)
|
From page 150... ...
Level 1: Materials-Based Optimization 153 similar characteristics regarding variability and stability of properties, and will thus likely require comparable levels and types of QA to assure product acceptability. Additional examples of risk-based materials acceptance methods are provided in case studies from the WSDOT and Caltrans (see boxes 3.1 and 3.2, respectively)
|
From page 151... ...
154 Guidelines for Optimizing the risk and Cost of Materials Qa programs Box 3.1. WSDOT Risk Rating of Materials Washington State DOT (WSDOT)
|
From page 152... ...
Level 1: Materials-Based Optimization 155 Box 3.2. Caltrans Risk Rating of Fabricated Materials The Caltrans Office of Structural Materials (OSM)
|
From page 153... ...
156 Guidelines for Optimizing the risk and Cost of Materials Qa programs Quality Assurance Methods Production Mode Material Risk Tier Pr oj ec t Pr od uc ed Fa br ic at ed M an uf ac tu re d Ti er 1 Ti er 2 Ti er 3 Materials Prequalification Qualified (or Authorized) Products List (1)
|
From page 154... ...
Level 1: Materials-Based Optimization 157 Quality Assurance Methods Production Mode Material Risk Tier Pr oj ec t Pr od uc ed Fa br ic at ed M an uf ac tu re d Ti er 1 Ti er 2 Ti er 3 Sampling and Testing by Agency Certificate of Compliance (1) X x x x x Certificate of Compliance with Test Results x x x x Inspection Quality Control Inspection X x x x x x Continuous Inspection of Work In Progress X x x Intermittent Inspection of Work In Progress X x x Benchmark Inspection X x x Warranties Material and Workmanship Warranty X x x x Performance Warranty X x Manufacturer's Guarantee x x x x (1)
|
From page 155... ...
158 Guidelines for Optimizing the risk and Cost of Materials Qa programs Factor Considerations Possible Optimization Strategies Contractor/supplier qualifications and experience Does the contractor, fabricator, or supplier (as applicable) have a history of consistently acceptable performance (i.e., compliance with specifications or with national or regional quality standards, such as NTPEP)
|
From page 156... ...
Level 1: Materials-Based Optimization 159 consequences that could result from its failure. In essence, this entails providing a response to the following questions: What can cause the material to fail?
|
From page 157... ...
160 Guidelines for Optimizing the risk and Cost of Materials Qa programs {ART} Box 3.3. Using Risk Bowties to Refine QA Strategy The Bowtie method is a structured risk evaluation technique that can be used to qualitatively assess and demonstrate effective control of risks.
|
From page 158... ...
Level 1: Materials-Based Optimization 161 2. Assess the specific threats that could cause the hazard.
|
From page 159... ...
162 Guidelines for Optimizing the risk and Cost of Materials Qa programs 5. Identify recovery measures that can be used to mitigate the potential impacts of the threat should it occur.
|
From page 160... ...
163 Level 2: Property-Based Optimization The Level 1 optimization process discussed in Chapter 3 provides a framework for identifying suitable method(s) for assuring that a given material will be in reasonably close conformance to the approved specifications.
|
From page 161... ...
164 Guidelines for Optimizing the risk and Cost of Materials Qa programs have historically produced satisfactory quality, they may fail to take advantage of possible efficiencies to be gained from • recent developments in the understanding of materials behavior, • advances in non-destructive testing technology, and • increasing use of performance specifications and alternative delivery methods that shift more responsibility for quality management to industry.
|
From page 162... ...
Level 2: property-Based Optimization 165 • Is the property more strongly or less strongly associated with distresses? • What is the likelihood that property non-conformance would result in reduced or impaired performance?
|
From page 163... ...
166 Guidelines for Optimizing the risk and Cost of Materials Qa programs indicates that a property is strongly associated with common failure modes, then the property could be considered a primary indicator of performance. It is also important to remember that, as a general rule, properties tested in the end product are more closely related to the end-product performance than when tested during production.
|
From page 164... ...
Level 2: property-Based Optimization 167 In general, the DOT may choose to modify the normal inspection or testing procedures for lower risk projects or project elements. For primary properties, sampling frequency may be reduced for tests that are under control (e.g., after ten consecutive samples taken at the normal testing frequency indicate full conformance with the specifications)
|
From page 165... ...
168 Guidelines for Optimizing the risk and Cost of Materials Qa programs a. Primary Properties For primary properties, verification testing frequency is typically a minimum of 10% to 25% of the QA testing frequency.
|
From page 166... ...
Level 2: property-Based Optimization 169 Box 4.1. Continuous-Cumulative and Chain Lot Methods The continuous-cumulative method shown in the figure below consists of accumulating incrementally test results from sequential lots (i.e., results from lots 1 and 2; results from lots 1, 2, and 3; results from lots 1, 2, 3, and 4)
|
From page 167... ...
170 Guidelines for Optimizing the risk and Cost of Materials Qa programs Box 4.2. TXDOT Risk-Based Approach to Verification Testing Texas DOT (TXDOT)
|
From page 168... ...
Level 2: property-Based Optimization 171 The figure below provides an example of how TXDOT's guide applies these analysis categories to specific materials and properties.
|
From page 169... ...
172 Guidelines for Optimizing the risk and Cost of Materials Qa programs Property Importance Material Risk Example Properties High Risk (e.g., structural or safety critical elements, high user impacts, large quantities) Moderate Risk (e.g., structural elements with moderate safety or user impacts)
|
From page 170... ...
173 Level 3: Cost-Based Optimization In contrast to the Level 1 and 2 analyses presented in Chapters 3 and 4, respectively, the Level 3 optimization process described below explicitly compares the direct costs of different QA protocols to the associated cost of material failure to arrive at the optimum QA investment point. 5.1 Conceptual Framework 5.1.1 Cost of Quality To apply the Level 3 optimization process, it is important to first understand what is meant by the cost of quality.
|
From page 171... ...
174 Guidelines for Optimizing the risk and Cost of Materials Qa programs Table 5.1. Cost of quality.
|
From page 172... ...
Level 3: Cost-Based Optimization 175 Figure 5.2. Level 3 optimization.
|
From page 173... ...
176 Guidelines for Optimizing the risk and Cost of Materials Qa programs Step 1. Identify the Materials and Properties of Interest As QA protocols, costs, and risks are material and property-specific, the first step when attempting to optimize the costs and risks of QA -- whether on a programmatic or project-level basis -- is to identify the materials and properties of interest.
|
From page 174... ...
Level 3: Cost-Based Optimization 177 0.00% 2.00% 4.00% 6.00% 8.00% 10.00% 12.00% 14.00% 16.00% L E V E L 1 L E V E L 2 L E V E L 3 L E V E L 4 L E V E L 5 P ER CE N TA G E O F M AT ER IA L CO ST LEVEL OF QUALITY ASSURANCE EFFORT Cost of QA Figure 5.3. Estimated cost of QA effort.
|
From page 175... ...
178 Guidelines for Optimizing the risk and Cost of Materials Qa programs 0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00% 80.00% L E V E L 1 L E V E L 2 L E V E L 3 L E V E L 4 L E V E L 5 P ER CE N TA G E O F M AT ER IA L CO ST LEVEL OF QUALITY ASSURANCE EFFORT EV of non-conformance Cost of QA Figure 5.4. Expected value of non-conformance.
|
From page 176... ...
Level 3: Cost-Based Optimization 179 0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00% 80.00% L E V E L 1 L E V E L 2 L E V E L 3 L E V E L 4 L E V E L 5 P ER CE N TA G E O F M AT ER IA L CO ST LEVEL OF QUALITY ASSURANCE EFFORT EV of non-conformance Cost of QA CoQ Figure 5.5. Total cost of quality (CoQ)
|
From page 177... ...
180 Implementation This chapter provides a high-level recap of the optimization process, a discussion of potential implementation strategies, and examples to illustrate the application of the QA optimization framework. 6.1 Recap of the Optimization Process This guidebook applies a three-level analytical framework to help DOTs identify the QA acceptance plan necessary to meet specification requirements within an acceptable level of risk.
|
From page 178... ...
Implementation 181 specifications, manuals, test equipment) associated with these traditional practices, some resistance may be encountered when attempting to implement a QA strategy that is new or different.
|
From page 179... ...
182 Guidelines for Optimizing the risk and Cost of Materials Qa programs place among the experts to allow a consensus to be reached on the data input values. Members of this team should include internal DOT subject matter experts on materials and products, and construction and maintenance personnel.
|
From page 180... ...
Implementation 183 4. After participants have identified the perceived optimum levels of QA necessary for materials management for the given scope and materials of interest, the facilitator will then compile and present the optimized materials QA plan for final discussion and validation by the expert panel.
|
From page 181... ...
184 References AASHTO.
|
From page 182... ...
185 Current State of Materials QA While not an exhaustive list, the following table summarizes several QA practices in standard use today. Although it is possible that under certain circumstances one method alone could be used to assure a material's desired level of quality and performance (e.g., requiring use of materials selected from a Qualified Products List)
|
From page 183... ...
186 Guidelines for Optimizing the Risk and Cost of Materials QA programs Contractor Qualifications: Specification of minimum requirements for contractors. î Ensure construction is performed by qualified contractors having the requisite experience Qualification Requirements for Sampling/Testing Personnel: Requirement that sampling, testing, and inspection personnel are certified to a recognized standard or certification program (e.g., ACI)
|
From page 184... ...
Current State of Materials QA 187 QA Method Description QA Strategy/Objectives System-Based Acceptance: Agency monitoring and management of material quality on a statewide basis (would likely require implementation of a materials management system)
|
From page 185... ...
188 Guidelines for Optimizing the Risk and Cost of Materials QA programs QA Method Description QA Strategy/Objectives Intermittent Inspection of Work In Progress: Agency monitoring of the contractor's construction processes on an intermittent basis to ensure that the construction quality and workmanship are in compliance with the plans and specifications. î Provide a reasonable degree of confidence in the quality of workmanship and fitness for purpose (e.g., Inspect 30– 80% of the time work is in progress with assistant(s)
|
From page 186... ...
189 Optimization Tool Description of Tool The Level 3 model is designed to use actual or estimated QA costs and the potential cost impacts related to material non-conformance to optimize materials quality assurance (QA)
|
From page 187... ...
190 Guidelines for Optimizing the Risk and Cost of Materials QA programs Figure B.1. Material/property inputs.
|
From page 188... ...
Optimization Tool 191 Input Cost of QA Input the cost of QA as a percentage of cost as shown in Table B.1 for each QA level. DOTs typically account for the costs of various QA activities as a percentage of a program budget (lab tests and plant inspections)
|
From page 189... ...
192 Guidelines for Optimizing the Risk and Cost of Materials QA programs Input Cost of Non-Conforming Material Input the cost of a non-conforming property as shown in Table B.3. The user can assign expected values based on the perceived impact of the material non-conformance considering the range of options previously discussed.
|
From page 190... ...
193 Case Study Examples of Optimization Process Example 1: Level 2 QA Optimization for HMA Pavement Reconstruction This example illustrates the use of the Level 2 property-based optimization for HMA pavement. HMA is categorized as a project-produced material.
|
From page 191... ...
194 Guidelines for Optimizing the Risk and Cost of Materials QA programs specific material properties. The worksheet can be used to determine the optimal QA frequency for acceptance testing of material properties.
|
From page 192... ...
Case Study examples of Optimization process 195 start of or early during production as a measure of mixture performance, particularly if recycled material is used in the mix. It is an important test in that production can be suspended until mix design is adjusted if the test results do not meet the performance standard for Hamburg rutting.
|
From page 193... ...
196 Guidelines for Optimizing the Risk and Cost of Materials QA programs Dense-graded Hot-Mix Asphalt (QC/QA) , Item xxx, high volume roadway mix Test For Location Std.
|
From page 194... ...
Case Study examples of Optimization process 197 Step 1: Define Material(s) and Properties (Categories)
|
From page 195... ...
198 Guidelines for Optimizing the Risk and Cost of Materials QA programs Factors Scenarios 1 2 3 4 Industry Experience High Low High High Material Quantity Large Large Small Large Project delivery method DBB DBB DBB DBOM Criticality/ Complexity High High Low High Table C.5. Project scenarios.
|
From page 196... ...
Case Study examples of Optimization process 199 Bridge Deck QA Effort\Scenarios 1 2 3 4 1 Visual inspection 2% 1% 1% 1% 2 Certification 3% 2% 2% 2% 3 Certification w/data 8% 5% 3% 3% 4 Verification sampling and testing 9% 8% 6% 9% 5 Full sampling and testing 12% 12% 10% 10% Drainage Structure QA Effort\Scenarios 1 2 3 4 1 Visual inspection 1% 2% 1% 1% 2 Certification 2% 3% 2% 2% 3 Certification w/data 2% 0% 3% 3% 4 Verification sampling and testing 6% 7% 4% 8% 5 Full sampling and testing 9% 10% 6% 10% Figure C.3. QA effort for each scenario.
|
From page 197... ...
200 Guidelines for Optimizing the Risk and Cost of Materials QA programs Spec\Scenario 1 2 3 4 1 Bridge Deck 115% 115% 115% 115% 2 Drainage Structure 115% 115% 115% 115% Figure C.5. Estimated impact of non-conforming element.
|
From page 198... ...
Case Study examples of Optimization process 201 expected total CoQ of the material. The green color coding represents the lowest cost of QA and the red coding represents the highest cost of QA.
|
From page 199... ...
Abbreviations and acronyms used without definitions in TRB publications: A4A Airlines for America AAAE American Association of Airport Executives AASHO American Association of State Highway Officials AASHTO American Association of State Highway and Transportation Officials ACI–NA Airports Council International–North America ACRP Airport Cooperative Research Program ADA Americans with Disabilities Act APTA American Public Transportation Association ASCE American Society of Civil Engineers ASME American Society of Mechanical Engineers ASTM American Society for Testing and Materials ATA American Trucking Associations CTAA Community Transportation Association of America CTBSSP Commercial Truck and Bus Safety Synthesis Program DHS Department of Homeland Security DOE Department of Energy EPA Environmental Protection Agency FAA Federal Aviation Administration FAST Fixing America's Surface Transportation Act (2015) FHWA Federal Highway Administration FMCSA Federal Motor Carrier Safety Administration FRA Federal Railroad Administration FTA Federal Transit Administration HMCRP Hazardous Materials Cooperative Research Program IEEE Institute of Electrical and Electronics Engineers ISTEA Intermodal Surface Transportation Efficiency Act of 1991 ITE Institute of Transportation Engineers MAP-21 Moving Ahead for Progress in the 21st Century Act (2012)
|
From page 200... ...
TRA N SPO RTATIO N RESEA RCH BO A RD 500 Fifth Street, N W W ashington, D C 20001 A D D RESS SERV ICE REQ U ESTED N O N -PR O FIT O R G .
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.