Short-Term Laboratory Conditioning of Asphalt Mixtures (2015)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

D-1 Scope of the Study ............................................................. D-1 Specimen Preparation and Testing Plan .......................... D-2 Data Analysis ..................................................................... D-2 Laboratory Results ............................................................ D-4 Sources of Variability and Bias ....................................... D-14 Summary and Recommendations .................................. D-16 References ........................................................................ D-17 This appendix summarizes the results of a round robin study between the University of California Pavement Research Center (UCPRC), Texas A&M Transportation Institute (TTI), and National Center for Asphalt Technology (NCAT) to assess the consistency, repeatability, and reproducibility of laboratory testing results between the different laboratories participating in this project. Results from the study indicate that specimen fabrication was generally consistent between the three laboratories. However, some of the results were inconsistent indicating that aspects of the testing procedures might be open to interpretation and hence followed differ- ently between the laboratories. A few of the results (specific tests within individual laboratories) also appeared to have high variation in the results across the five specimens tested. The results of the round robin testing and the test procedures were reviewed to assess possible reasons for variability within and between laboratories. It was found that all of the labora- tories were performing the tests in a similar manner, except for a few minor details. Appropriate statistical approaches for analyzing the results of field testing to meet the objectives of the project were also identified. The conclusion of the study was that project testing should proceed as planned. Scope of the Study To ensure consistency, repeatability, and reproducibility of laboratory testing results between the different laboratories participating in this project, round robin testing was con- ducted prior to starting the planned project testing. Given that the three laboratories participating in the study (UCPRC, TTI, and NCAT) all have AASHTO Materials Reference Lab- oratory (AMRL) accreditation for standard hot mix asphalt tests, only those tests that are not part of the AMRL accredita- tion system were evaluated. Tests included in the round robin testing program included unconfined flow number, resilient modulus (MR), and dynamic modulus (E*)/phase angle. The round robin employed aggregate batches arranged by the UCPRC and sent to the participating laboratories for preparation and testing of specimens (referred to as in- laboratory fabricated in this appendix), as well as testing of specimens prepared and compacted at the UCPRC and sent to participating laboratories (referred to as prefabricated in this appendix). Testing of specimens prepared from supplied aggregate batches and binder assessed consistency, repeat- ability, and reproducibility between participating laborato- ries of mixing, compacting (gyratory), sawing/trimming, air-void (AV) content testing, and performance-related test- ing. Testing of specimens prepared at the UCPRC and sent to participating laboratories assessed the performance-related tests only. All results were submitted to the UCPRC for analy- sis and reporting. The following tasks were completed to achieve the study’s objective: 1. Prepare a test plan for specimen preparation, testing, and reporting. 2. Collect aggregate and binder samples. Prepare material batches and specimens for all laboratories. 3. Perform unconfined flow number, MR, E*/phase angle tests according to the test plan. Submit test results for analysis. 4. Analyze the results. The results of the analysis were reviewed to identify poten- tial sources of error and differences in practice due to possible gaps or different interpretations of the test procedure. A P P E N D I X D Round Robin Study

D-2 Specimen Preparation and Testing Plan One set of five test specimens for each test was produced by each laboratory from aggregate/binder material provided by UCPRC and a set of five test specimens for each test was prepared by UCPRC. All specimens were prepared from a California crushed granite aggregate source and a California coastal crude based PG 64-16 binder. Tables D-1 and D-2 show the specimen preparation tem- peratures, dimensions, AV, and tolerances used by UCPRC for fabricating specimens. NCAT and TTI followed these guide- lines for consistent specimen preparation among all three laboratories. The flow number test was conducted according to NCHRP TP 79-12 with NCHRP 09-33 and 09-43 recommended param- eters and modifications, as shown below: • Test temperature: 55°C • 30 kPa (4.35 psi) contact stress • 600 kPa (87 psi) deviator stress • Unconfined (0 psi) • Five replicates each for both the laboratory-mixed, laboratory-compacted (LMLC) samples and the pre- fabricated samples • Test ending criteria: 10,000 loading cycles or accumulated 30,000 microstrain • Data analysis in accordance with AASHTO TP 79-12 (Francken model) The output of the flow number test was reported according to Section 10.5 in TP 79-12. The flow number was calculated following the Francken model. Resilient modulus or MR stiffness was measured in accor- dance with the current ASTM D7369. TTI and NCAT replaced the on-specimen linear variable differential transducers (LVDT) setup with external LVDT aligned along the horizontal diam- etral plane (i.e., gauge length as a fraction of diameter of the specimen = 1.00). The MR test was conducted through repeti- tive applications of compressive loads in a haversine wave- form along a vertical diametral plane of cylindrical asphalt concrete specimens. Testing parameters for the E* test are shown in Table D-3. Testing was performed in accordance with AASHTO TP 79-12 and under unconfined conditions (0 psi). Five replicates for each of the LMLC samples and the prefabricated samples were tested. Data analysis and presentation were done in accordance with Section 11 of AASHTO PP 61-10. Data Analysis The unconfined flow number, MR, E*, phase angle, and AV content data from the specimens prepared from the supplied loose aggregate/binder samples were analyzed in accordance with ASTM E691-13 (Standard Practice for Conducting an Interlaboratory Study to Determine the Pre- cision of a Test Method) to develop precision statements for each test. Following the same analysis procedure, the data Temperature Compaction Binder/Mixture 145°C MR (61 x 150 mm) AV = 7.0 ± 0.5% (as compacted), Height ± 2.5 mm E* (150 x 100 mm) AV = 7.0 ± 0.5% (cut specimen), Height ± 2.5 mm Flow # (150 x 100 mm) AV= 7.0 ± 0.5% (cut specimen), Height ± 2.5 mm Compaction using height control mode Binder Content 5% by Dry Weight of Aggregate Aging/Curing of mix is per PP 60-09 (R 30 short-term aging) Aggregate 160°C Cure Mix 4 h 135°C Compact 140°C Table D-1. Specimen preparation parameters. Parameter Acceptable Tolerance Air-Void Tolerance Average Sample Diameter Standard Deviation of Sample Diameter Sample Height End Flatness End Perpendicularity 7.0 ± 0.5% 100 to 104 mm ≤ 0.5 mm 147.5 to 152.5 mm ≤ 0.5 mm ≤ 1.0 mm Table D-2. Asphalt Mixture Performance Tester sample fabrication tolerances (AASHTO PP 60-09). Test Temperature (°C) Loading Frequencies (Hz) 4.0 10, 1, 0.1 20.0 10, 1, 0.1 40.0 (for PG 64-16 binder) 10, 1, 0.1, 0.01 Table D-3. Temperatures and frequencies for E* testing.

D-3 from the pre-fabricated specimens were analyzed to deter- mine to what extent sample fabrication affected the preci- sion of the tests. It should be noted that the purpose of the analysis was not to provide final statements of precision of the test methods. Since only three laboratories were involved in the study, conclusions regarding precision and bias in test results are not appropriate in that this study did not provide sufficient statistical evidence to identify laboratories with problematic specimen preparation and testing methods. However, it was possible to identify the levels of differences in test results based on the results of the analysis. The following steps were followed to analyze the test results. Equations were adapted from ASTM E691 and NCHRP Report 702 (Bonaquist 2011). Step 1. Compute Consistency and Initial Estimate of Repeatability and Reproducibility Statistics The following equations were used to compute the consis- tency and initial estimate of repeatability and reproducibility statistics: Eq. (D-1) 2 1 s s p r p∑= Where: sr = repeatability standard deviation, s = within-laboratory standard deviation, and p = number of laboratories in the interlaboratory study. % 100 Eq. (D-2)s s X r r = × Where: sr% = repeatability coefficient of variation, sr = repeatability standard deviation, and – X = average of the laboratory averages. s s s n 1 n Eq. (D-3)R x 2 r 2( ) ( )= + −   Where: sR = reproducibility standard deviation, sx = standard deviation of the laboratory averages, sr = repeatability standard deviation, and n = number of tests in each laboratory. % X 100 Eq. (D-4)R R s s = × Where: sR% = reproducibility coefficient of variation, sR = reproducibility standard deviation, and – X = average of the laboratory averages. k s s Eq. (D-5) r = Where: k = within-laboratory consistency statistic, s = within-laboratory standard deviation, and sr = repeatability standard deviation. h s Eq. (D-6) x X x ( ) = − Where: h = between-laboratory consistency statistic, sx = standard deviation of the laboratory averages, –x = laboratory average, and – X = average of the laboratory averages. Step 2. Evaluate the Consistency Statistics to Identify Questionable Data Groups The consistency statistics, k and h, were used to evaluate the consistency of the data collected. The k statistic is a mea- sure of consistency of the data within a laboratory, and the h statistic is a measure of the consistency of the data between laboratories. These values were compared to critical values that depend on the number of laboratories and number of tests per laboratory that were conducted. For the design used in this interlaboratory study (three laboratories with five tests per laboratory), the critical values of k and h are 1.56 and 1.15, respectively (ASTM E691). Plots of the k and h statistics were also used to identify data requiring further review and to understand the characteristics of the test variability. Step 3. Investigate Difference in Between-Laboratory Test Results Differences in between-laboratory test results were inves- tigated using the Welch modified two-sample t-test, which is recommended for evaluating small datasets (Ruxton 2006). This method assumes unequal dataset variances and, if F1 and F2 are two distributions, the possible hypotheses and alternatives concerning these distributions are: • H0: F1(x) = F2(x) • HA: F1(x) ≠ F2(x)

D-4 • Decision rule: Reject H0 if p-value < 0.10; accept H0 if p-value ≥ 0.10 For example, if the p-value from the Welch modified two- sample t-test for two distributions is equal to 0.07, H0 will be rejected. In other words, distributions of the two test results are not equal. p-values closer to one suggest that the selected two test result distributions are similar for the laboratory test under consideration. Step 4. Evaluate Trends in the Repeatability and Reproducibility Statistics Repeatability is the variability between independent test results on the same material obtained in a single laboratory. Reproducibility is the variability between independent test results on the same material obtained in different laborato- ries. Repeatability and reproducibility statistics were calcu- lated to identify trends in these statistics so that appropriate precision statements could be developed. Step 5. Evaluate the Effect of AV Content on Test Results The effects of AV content variability on test result precision and bias were also investigated. Laboratory Results Test result summaries are provided in this section. The laboratories are identified in the figures as follows: • Lab 1: UCPRC • Lab 2: TTI • Lab 3: NCAT Unconfined Flow Number The unconfined flow number tests were conducted accord- ing to the AASHTO TP 79-12 test method, with NCHRP 09-33 and NCHRP 09-43 recommended parameters and modifications as previously mentioned. Within-laboratory (k) consistency statistics for the un- confined flow number tests on in-laboratory fabricated and prefabricated specimens are shown in Figure D-1. None of the data exceeded the limit for the k statistic. However, due to the high variability in the UCPRC test results, within- laboratory consistency appeared to be lower. TTI and NCAT had closer within-laboratory consistency statistics, compared to the UCPRC statistics. The coefficients of variation of the flow number test results were also calculated to further inves- tigate the levels of within-laboratory variability. Results are shown in Figure D-2. The test result variability for UCPRC was significantly higher than those for TTI and NCAT. Between-laboratory (h) consistency statistics for the pre- fabricated and in-laboratory fabricated specimens are shown in Figure D-3. None of the data exceeded the limits for the h statistic. However, the h statistic for TTI appeared to be close to the limit, which suggests that the TTI flow number test results were different from the UCPRC and NCAT test results. Differences in between-laboratory test results are listed in Table D-4. For the in-laboratory fabricated specimens, unconfined flow number test results from UCPRC can be considered as similar to those from NCAT. The results from TTI were statistically different to those from the UCPRC and NCAT laboratories. For the prefabricated specimens, the results from UCPRC were again similar to those from NCAT, but only marginally similar to those from TTI. When comparing results between TTI and NCAT, the results were statistically different. Comparisons of results for the different fabrication methods for each laboratory indicate that fab- rication did not influence the TTI results, but did influence Figure D-1. Flow number: within-laboratory (k) consistency statistics.

D-5 Figure D-2. Flow number: variability of test results. Figure D-3. Flow number: between-laboratory (h) consistency statistics. Comparison p-Value In-Laboratory Fabricated Prefabricated by UCPRC Prefabricated vs. In-Lab Fabricated UCPRC – TTI UCPRC – NCAT TTI – NCAT TTI – TTI NCAT – NCAT 0.19 0.49 0.01 – – 0.09 0.56 0.02 – – – – – 0.45 0.01 Table D-4. Flow number: p-values from the Welch modified two-sample t-test.

D-6 Figure D-4. Flow number: relationship with AV content for in-lab fabricated specimens. Figure D-5. Flow number: relationship with AV content for prefabricated specimens. Figure D-6. Resilient modulus: within-laboratory (k) consistency statistics. the NCAT results. This analysis was not conducted for the UCPRC laboratory since all specimens were prepared at the same time. AV content variability can contribute to test result variability and bias between different laboratories. Specimen AV contents were plotted against flow numbers (Figures D-4 and D-5) to investigate this effect. AV content variability had an effect on measured flow number variability for the UCPRC and NCAT laboratories. The results from TTI were less influenced by specimen AV content. Resilient Modulus Within-laboratory (k) consistency statistics for the pre- fabricated and in-laboratory fabricated specimens are shown in Figure D-6. None of the data exceeded the limit for the k statistic. However, due to the high variability in the UCPRC and NCAT test results, within-laboratory consistency appeared to be lower overall. Within-laboratory consistency statistics for TTI were lower than those for UCPRC and NCAT. The coefficients of variation of the measured resilient modulus values were also calculated and are shown in Figure D-7. Test result variability was significantly smaller than the variability for the flow number test results discussed in the previous sec- tion. Although resilient modulus values from all three labora- tories had similar variability, the TTI results appeared to have the lowest level of within-laboratory variability. Between-laboratory (h) consistency statistics for the pre- fabricated and in-laboratory fabricated specimens are shown in Figure D-8. The prefabricated specimen results from TTI exceeded the limits for the h statistic while its results for the in-laboratory fabricated specimens were close to the limit. These results suggest that the TTI resilient modulus test results from prefabricated specimens were different from those from UCPRC and NCAT. Results for the differences in between-laboratory test results are listed in Table D-5. For the in-laboratory fabricated speci- mens, the UCPRC results were statistically different from those

D-7 Figure D-7. Resilient modulus: variability of test results. Figure D-8. Resilient modulus: between-laboratory (h) consistency statistics. Comparison p-Value In-Laboratory Fabricated Prefabricated by UCPRC Prefabricated vs. In-Lab Fabricated UCPRC – TTI UCPRC – NCAT TTI – NCAT TTI – TTI NCAT – NCAT 0.00 0.02 0.18 – – 0.00 0.90 0.00 – – – – – 0.72 0.01 Table D-5. Resilient modulus: p-values from the Welch modified two-sample t-test.

D-8 from TTI and NCAT. The results from TTI show some simi- larity to those from NCAT. For the prefabricated specimens, the results from UCPRC and NCAT were statistically similar, but both these were statistically different from those from TTI. Comparisons of results for the different fabrication methods for each laboratory indicate that fabrication did not influence the TTI results but did influence the NCAT results. This analysis was not conducted for the UCPRC lab- oratory since all specimens were prepared at the same time. Plots of AV content versus resilient modulus for in- laboratory fabricated and prefabricated specimens are shown in Figures D-9 and D-10, respectively. AV content variability had an effect on the measured resilient modulus variability for the prefabricated specimens, but was not evident in the test results for in-laboratory fabricated spec- imens shown in Figure D-9. Dynamic Modulus Within-laboratory (k) consistency statistics for the pre- fabricated and in-laboratory fabricated specimens are shown in Figure D-11. The test results on in-laboratory fabricated specimens from TTI exceeded the limit for the k statistic; all other results were below the limit, with the NCAT results for in-laboratory fabricated specimens appearing to have the low- est level of variability and highest level of within-laboratory consistency. Temperature and loading frequency did not affect the within-laboratory consistency. The coefficients of variation of the measured E* values for all temperatures and loading frequencies are shown in Figure D-12. Test result variability for E* was significantly higher than the variability for the resilient modulus test results (see Figure D-7). Between-laboratory (h) consistency statistics for the in- laboratory fabricated and prefabricated specimens are shown in Figure D-13. Results from TTI for the in-laboratory fabri- cated specimens exceed the limit for the h statistic. The results from UCPRC for the prefabricated specimens are also close to the limit. Results for the differences in between-laboratory test results are listed in Table D-6. For the in-laboratory fabricated speci- mens, the UCPRC results were similar to the NCAT results for six of the ten testing configurations and only similar to Figure D-9. Resilient modulus: relationship with AV content for in-lab fabricated specimens. Figure D-10. Resilient modulus: relationship with AV content for prefabricated specimens. Figure D-11. Dynamic modulus: within-laboratory (k) consistency statistics.

D-9 Figure D-12. Dynamic modulus: variability of test results. Figure D-13. Dynamic modulus: between-laboratory (h) consistency statistics. one of the TTI results (20°C/1 Hz). The TTI and NCAT results were also only similar for one testing configuration (40°C/10 Hz). For the prefabricated specimens, the UCPRC results were similar to the NCAT results for four of the ten testing configurations and again only similar to one of the TTI results (40°C/10 Hz). The TTI results were similar to the NCAT results for eight of the ten testing configurations. Comparisons of results for the different fabrication methods for each laboratory indicate that fabrication did not influence the TTI results but did influence the NCAT results (three out of ten testing configurations were similar). This analysis was not conducted for the UCPRC laboratory since all specimens were prepared at the same time. Figures D-14 and D-15, respectively, compare the repeat- ability and reproducibility statistics for the E* tests. Repeatabil- ity and reproducibility were both higher for the prefabricated specimens. Consequently, specimen fabrication could have affected the between-laboratory test result variability. The effect of AV content on E* test results for the 20°C and 1 Hz loading frequency test configuration on the in-laboratory fabri- cated and prefabricated specimens are plotted in Figures D-16 and D-17, respectively. AV content variability had an effect on the measured E* variability for the test results from TTI and NCAT but did not appear to influence the UCPRC results. Phase Angle Within-laboratory (k) consistency statistics for the in- laboratory fabricated and prefabricated specimens are shown in Figure D-18. The TTI results again exceeded the limit for the k statistic. The NCAT test results for in-laboratory fabri- cated specimens appeared to have the lowest level of variability and a high level of within-laboratory consistency. Tempera- ture and loading frequency did not affect any of the within- laboratory test result consistency. Between-laboratory (h) consistency statistics for the in- laboratory fabricated and prefabricated specimens are shown in Figure D-19. The UCPRC phase angle values from prefab- ricated specimens were on the limit for the h statistic. TTI values for in-laboratory fabricated specimens were also close to the limit.

D-10 Table D-6. Dynamic modulus: p-values from the Welch modified two-sample t-test. Test Parameter Comparison p-Value In-Laboratory Fabricated Prefabricated by UCPRC Prefabricated vs. In-Lab Fabricated 4°C, 10 Hz UCPRC – TTI UCPRC – NCAT TTI – NCAT TTI – TTI NCAT – NCAT 0.02 0.39 0.04 – – 0.03 0.09 0.35 – – – – – 0.92 0.15 4°C, 1 Hz UCPRC – TTI UCPRC – NCAT TTI – NCAT TTI – TTI NCAT – NCAT 0.03 0.38 0.05 – – 0.03 0.08 0.35 – – – – – 0.79 0.10 4°C, 0.1 Hz UCPRC – TTI UCPRC – NCAT TTI – NCAT TTI – TTI NCAT – NCAT 0.02 0.40 0.05 – – 0.02 0.06 0.36 – – – – – 0.64 0.05 20°C, 10 Hz UCPRC – TTI UCPRC – NCAT TTI – NCAT TTI – TTI NCAT – NCAT 0.10 0.00 0.03 – – 0.08 0.31 0.37 – – – – – 0.57 0.02 20°C, 1 Hz UCPRC – TTI UCPRC – NCAT TTI – NCAT TTI – TTI NCAT – NCAT 0.11 0.00 0.03 – – 0.09 0.27 0.47 – – – – – 0.46 0.02 20°C, 0.1 Hz UCPRC – TTI UCPRC – NCAT TTI – NCAT TTI – TTI NCAT – NCAT 0.10 0.00 0.03 – – 0.04 0.19 0.39 – – – – – 0.48 0.01 40°C, 10 Hz UCPRC – TTI UCPRC – NCAT TTI – NCAT TTI – TTI NCAT – NCAT 0.09 0.36 0.17 – – 0.10 0.12 0.75 – – – – – 0.43 0.15 40°C, 1 Hz UCPRC – TTI UCPRC – NCAT TTI – NCAT TTI – TTI NCAT – NCAT 0.07 0.55 0.09 – – 0.02 0.09 0.19 – – – – – 0.52 0.06 40°C, 0.1 Hz UCPRC – TTI UCPRC – NCAT TTI – NCAT TTI – TTI NCAT – NCAT 0.02 0.41 0.03 – – 0.00 0.02 0.00 – – – – – 0.98 0.03 40°C, 0.01 Hz UCPRC – TTI UCPRC – NCAT TTI – NCAT TTI – TTI NCAT – NCAT 0.01 0.01 0.01 – – 0.00 0.00 0.00 – – – – – 0.17 0.02

D-11 Figure D-14. Dynamic modulus: comparison of repeatability. Figure D-15. Dynamic modulus: comparison of reproducibility. Figure D-16. Dynamic modulus: relationship with AV content for in-lab fabricated specimens. Figure D-17. Dynamic modulus: relationship with AV content for prefabricated specimens.

D-12 Figure D-18. Phase angle: within-laboratory (k) consistency statistics. Figure D-19. Phase angle: between-laboratory (h) consistency statistics. Results for the differences in between-laboratory test results are listed in Table D-7. For the in-laboratory fab- ricated specimens, the UCPRC results were similar to the TTI results for just two of the ten testing configurations (20°C/10 Hz and 20°C/1 Hz) and similar to the NCAT results for another two of the ten configurations (4°C/1 Hz and 4°C/0.1 Hz). The TTI and NCAT results were similar for five of the ten testing configurations. For the prefabri- cated specimens, the UCPRC results were statistically dif- ferent from the TTI results for all testing configurations. The UCPRC results were similar to the NCAT results for three of the ten testing configurations, while the TTI results were similar to the NCAT results for six of the ten testing configurations. Comparisons of results for the different fab- rication methods for each laboratory indicate that fabrica- tion generally did not influence the TTI results (seven out of ten testing configurations were similar) but did influence the NCAT results (only two out of ten testing configura- tions were similar). This analysis was not conducted for the UCPRC laboratory since all specimens were prepared at the same time. The results for phase angle were generally incon- sistent with the dynamic modulus test results, despite the tests being carried out on the same specimens and as part of the same testing sequence. The repeatability and reproducibility statistics for phase angle are compared in Figures D-20 and D-21, respectively. Repeatability and reproducibility were both higher for the prefabricated specimens. Consequently, specimen fabrication could have affected the between-laboratory test result vari- ability. Reproducibility for both the in-laboratory fabricated and prefabricated specimens appeared to be similar. The effect of AV content on phase angle test results for the 20°C and 1 Hz loading frequency test configuration on the in-laboratory fabricated and prefabricated specimens are plotted in Figures D-22 and D-23, respectively. AV content variability did not have a significant and consistent effect on the measured phase angle variability for the test results from all three laboratories.

D-13 Table D-7. Phase angle: p-values from the Welch modified two-sample t-test. Test Parameter Comparison p-Value In-Laboratory Fabricated Prefabricated by UCPRC Prefabricated vs. In-Lab Fabricated 4°C, 10 Hz UCPRC – TTI UCPRC – NCAT TTI – NCAT TTI – TTI NCAT – NCAT 0.03 0.09 0.21 – – 0.05 0.04 0.92 – – – – – 0.53 0.07 4°C, 1 Hz UCPRC – TTI UCPRC – NCAT TTI – NCAT TTI – TTI NCAT – NCAT 0.03 0.20 0.10 – – 0.05 0.05 0.97 – – – – – 0.36 0.03 4°C, 0.1 Hz UCPRC – TTI UCPRC – NCAT TTI – NCAT TTI – TTI NCAT – NCAT 0.04 0.28 0.10 – – 0.03 0.03 0.99 – – – – – 0.45 0.01 20°C, 10 Hz UCPRC – TTI UCPRC – NCAT TTI – NCAT TTI – TTI NCAT – NCAT 0.17 0.01 0.04 – – 0.06 0.21 0.37 – – – – – 0.65 0.00 20°C, 1 Hz UCPRC – TTI UCPRC – NCAT TTI – NCAT TTI – TTI NCAT – NCAT 0.13 0.00 0.03 – – 0.07 0.26 0.56 – – – – – 0.42 0.01 20°C, 0.1 Hz UCPRC – TTI UCPRC – NCAT TTI – NCAT TTI – TTI NCAT – NCAT 0.07 0.04 0.03 – – 0.01 0.13 0.20 – – – – – 0.47 0.01 40°C, 10 Hz UCPRC – TTI UCPRC – NCAT TTI – NCAT TTI – TTI NCAT – NCAT 0.01 0.03 0.06 – – 0.00 0.00 0.01 – – – – – 0.67 0.02 40°C, 1 Hz UCPRC – TTI UCPRC – NCAT TTI – NCAT TTI – TTI NCAT – NCAT 0.00 0.00 0.01 – – 0.00 0.00 0.00 – – – – – 0.03 0.02 40°C, 0.1 Hz UCPRC – TTI UCPRC – NCAT TTI – NCAT TTI – TTI NCAT – NCAT 0.01 0.00 0.11 – – 0.00 0.00 0.00 – – – – – 0.02 0.25 40°C, 0.01 Hz UCPRC – TTI UCPRC – NCAT TTI – NCAT TTI – TTI NCAT – NCAT 0.02 0.00 0.29 – – 0.00 0.00 0.01 – – – – – 0.02 0.71

D-14 Sources of Variability and Bias The reasons behind the differences in TTI, NCAT, and UCPRC results for the flow number, resilient modulus, and, to a lesser degree, E* and the phase angle were analyzed. Possible sources of variability and bias introduced during specimen preparation and testing included test and specimen prepara- tion procedures, test setup, and equipment differences. Specimen Preparation All three laboratories outlined their step-by-step speci- men preparation procedures. No significant differences were identified in these. Details of the specimen preparation for all three laboratories are listed in Table D-8. Unconfined Flow Number None of the laboratories exceeded the limit for the within- and between-laboratory consistency statistics. TTI results were slightly higher than the other two laboratories (not statistically significant). Possible causes of bias and their resolution are: • Testing with or without steel ball (top platen free to rotate)— none of the laboratories used the steel ball during the tests. • Friction reducers—all three laboratories used latex friction reducers. Resilient Modulus Load Level NCAT and UCPRC ran their resilient modulus testing with loading set at 150 lb, while TTI ran tests at 75 lb. This was dis- cussed earlier in the study; however, the different load levels were agreed upon given that the NCAT and UCPRC equip- ment achieved reasonable signal-to-noise ratios only at 150 lb Figure D-22. Phase angle: relationship with AV content for in-lab fabricated specimens. Figure D-23. Phase angle: relationship with AV content for prefabricated specimens. Figure D-20. Phase angle: comparison of repeatability. Figure D-21. Phase angle: comparison of reproducibility.

D-15 load level while noise was high at 75 lb. On the other hand, TTI’s test equipment is capable of testing at only 75 lb. This dif- ference in load levels was expected to be a source of bias in the test results. However, NCAT tested some samples at both 75-lb and 150-lb to determine the effect of load level on test results and did not observe any significant effect on average resilient modulus. At 75 lb load level, UCPRC could not get a reason- able wave form for the determination of the resilient modulus. LVDT Configuration NCAT and TTI used LVDT around the sample while UCPRC had LVDTs mounted on the sample (Figure D-24). Although this difference in LVDT position is expected to affect the test results due to the non-uniformity in AV distributions within the specimens, results were not conclusive since the average resilient modulus results measured by NCAT and UCPRC were close. Different load levels and LVDT position might be the sources of bias between the three laboratories. However, since the mixtures from the identified projects/asphalt plants will be tested at one laboratory only and the focus of the project was to identify the differences in properties for the mixture vari- ables, possible sources of bias were not expected to affect the statistical analysis of the test results in the project as a whole. Dynamic Modulus TTI test results for in-laboratory fabricated specimens slightly exceeded the limits for both the within-laboratory and between-laboratory consistency statistics. However, the difference was not very high. Possible causes of bias and their resolution are: • Equipment differences—all laboratories have IPC Global equipment. • Data quality—all laboratories checked data quality sta- tistics requirements given in AASHTO TP 79 and each met the requirements. • Testing with or without steel ball (top platen free to rotate)—all laboratories used the steel ball. Procedure NCAT UCPRC TTI Binder break down Compaction temp. and time Aggregate mixing temp. and time Binder mixing temp. and time Short-term aging 145°C for 2 hours 135°C for 2 hours 160°C for 2 hours 145°C for 2 hours 135°C for 4 hours 145°C for 2 hours 140°C for 2 hours 160°C for 2 hours 145°C for 2 hours 135°C for 4 hours 145°C for 2 hours 140°C for 2 hours 160°C for 2 hours 145°C for 2 hours 135°C for 4 hours Table D-8. Comparison of specimen preparation variables. (a) TTI and NCAT (b) UCPRC (according to ASTM D7369) Figure D-24. LVDT configurations.

D-16 • Friction reducers—NCAT used Teflon friction reducers for E* testing (which is allowed according to the specification) while TTI and UCPRC used latex friction reducers. • Temperature conditioning time—all laboratories followed the procedures given in the specifications and no impor- tant differences were noted. Summary The conclusion from the analysis of sources of variability and bias was that all of the laboratories were performing the tests in a similar manner, except for a few minor details. The minor differences did not have major influence on the results, and were corrected. Statistical approaches for analyzing the results of field testing to meet the objectives of the project were also identified. Summary and Recommendations The following lists summarize the results from the differ- ent tests. Unconfined Flow Number • None of the data exceeded the limit for the within- laboratory consistency statistic. • The UCPRC test result variability was significantly higher than those of TTI and NCAT. • None of the data exceeded the limit for the between- laboratory consistency statistic. • The TTI flow number test results were different from those from UCPRC and NCAT. • Specimen fabrication did not appear to significantly affect the flow number test results from TTI. The in-laboratory fabricated and prefabricated specimen test results from NCAT were different. • AV content variability influenced the UCPRC and NCAT results but had limited effect on the TTI test results. Resilient Modulus • None of the data exceeded the limit for the within-laboratory consistency statistic. • Due to the high variability in the UCPRC and NCAT test results, within-laboratory consistency appeared to be lower. • Within-laboratory test result variability was significantly smaller than the variability for the flow number test results. • TTI test results on prefabricated specimens exceeded the limit for the between-laboratory consistency statistic. The UCPRC and NCAT test results for the prefabricated speci- mens were similar to each other, but both were different from those from TTI. • For the in-laboratory fabricated specimens, TTI and NCAT results were similar, but both were different from those from the UCPRC. • Specimen fabrication did not appear to significantly affect the test results from TTI. The in-laboratory fabricated and prefabricated specimen test results from NCAT were different. • AV content variability had some effect on measured resilient modulus variability for the prefabricated specimens. This effect was not evident in test results from the in-laboratory fabricated specimens. Dynamic Modulus • TTI test results for in-laboratory fabricated specimens exceeded the limits for both the within-laboratory and between-laboratory consistency statistics. • NCAT test results for in-laboratory fabricated specimens appeared to have the lowest level of variability. • Temperature and loading frequency did not affect the within-laboratory consistency. • Variability in the E* test results between and within the laboratories was significantly higher than the variability for the resilient modulus test results. • UCPRC and NCAT test results for the in-laboratory fab- ricated specimens were generally similar, but both were statistically different from those from TTI. • Test results for the prefabricated specimens from TTI and NCAT were generally similar. UCPRC results were signifi- cantly different from those from TTI for all loading fre- quencies and temperatures and different from those from NCAT for six of the ten testing configurations. • Specimen fabrication did not appear to significantly affect the TTI E* test results. The NCAT results for the in-laboratory fabricated and prefabricated specimen test results were different. • Repeatability and reproducibility were both better for the prefabricated specimens. • AV content variability appeared to have some effect on mea- sured E* variability at TTI and NCAT but did not appear to influence the test results from UCPRC. Phase Angle • The TTI test results for in-laboratory fabricated specimens exceeded the limit for the within-laboratory consistency statistic. • NCAT test results for in-laboratory fabricated specimens appeared to have the lowest level of variability and highest level of within-laboratory consistency. • Temperature and loading frequency did not affect the within-laboratory consistency.

D-17 • The UCPRC phase angle values for prefabricated specimens were on the limit for the between-laboratory consistency statistic. • Test results from all three laboratories appeared to be different. • Specimen fabrication did not appear to significantly affect the phase angle results from TTI. The in-laboratory fabri- cated and prefabricated specimen phase angle results from NCAT were different. • Repeatability was better for the prefabricated specimens. Reproducibility for sets of specimens appeared to be simi- lar for all laboratories. • AV content variability did not appear to have a significant and consistent effect on measured phase angle variability at all three laboratories. Based on the outcome of the variability and bias analy- sis, the recommendation was to proceed with field testing, and use standard statistical procedures to handle the lev- els of between-laboratory bias and variability identified in this study and to arrive at sound conclusions for the overall project. References Bonaquist, R. (2011). NCHRP Report 702: Precision of the Dynamic Modulus and Flow Number Tests Conducted with the Asphalt Mixture Performance Tester. Transportation Research Board, Washington, D.C. Ruxton, G. D. (2006). “The Unequal Variance t-test Is an Underused Alternative to Student’s t-test and the Mann-Whitney U Test.” Behavioral Ecology, Vol. 17(4), pp. 688–690.