Validation of Urban Freeway Models (2014)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

167 Overview This appendix presents the validation analysis of the L03 data-poor models. Appendix H of the L03 Report contains a set of models that predict the following travel time index (TTI) reliability statistics: • 95th-, 90th-, and 80th-percentiles • Standard deviation • Percentage of on-time trips relative to space mean speeds of 50 mph, 45 mph, and 30 mph In the L03 project, these models were termed “data poor” because they enable the prediction of a wide set of reliability measures based only on estimates of the mean travel time index. The L03 project calibrated these data-poor models using data collected in a number of metropolitan areas but it did not perform any validation of the final predictive equations. The goal of this stage of the L33 project is to quantify the effective- ness of these models using new data sets collected from around the country. The rest of this appendix is organized as follows. The next section presents the validation procedure, including the data gathering and the techniques used to measure the effectiveness of the L03 data-poor models. The following section presents the validation results for each model overall and by region. The final section summarizes the conclusions. There is also an attachment that contains detailed validation outputs (shown in Tables D.23 to D.50 and Figures D.57 to D.140). Validation Procedure Models The seven L03 data-poor models validated in this task are 1. 95th-percentile TTI = 1 + 3.6700 * ln(meanTTI) 2. 90th-percentile TTI = 1 + 2.7809 * ln(meanTTI) 3. 80th-percentile TTI = 1 + 2.1406 * ln(meanTTI) 4. Standard deviation of TTI = 0.71 * (meanTTI - 1)0.56 5. PctTripsOnTime50mph = e(-0.20570*[meanTTI-1]) 6. PctTripsOnTime45mph = e(-1.5115*[meanTTI-1]) 7. PctTripsOnTime30mph = 0.333 + [0.672/ (1 + e(5.0366*[meanTTI-1.8256]))] Appendix H of the L03 Report, which contains these models, does not include any outputs from the statistical analyses used to form these equations. Without these outputs, much of the L33 validation had to focus on evaluating the extent to which these models adhere to the assumptions required for general- ized regression. Data The data used in the validation were collected from the Los Angeles, San Francisco Bay Area, Sacramento, and San Diego metropolitan regions (grouped together into a “California” data set); Minneapolis–St. Paul, Minnesota; Salt Lake City, Utah; and Spokane, Washington. Details about the study seg- ments, data sets, and data processing techniques are discussed in the L33 validation plan. The California, Salt Lake City, and Spokane data were collected from the three-year period between January 1, 2010, and December 31, 2012. The Min- nesota data were collected from the three-year period between June 1, 2009, and May 31, 2012. Validation was performed using data collected on week- days during the midday period (11:00 a.m.–2:00 p.m.) and the peak period (a continuous time period of at least 75 min during which the space mean speed is less than 45 mph). This is consistent with the time periods that L03 used to calibrate the data-poor models. Table D.1 summarizes the sample size of data by region and time period used in the validation. Each value represents the number of section-years for which the mean TTI and TTI reliability statistics were calculated from the collected data. In A P P e n d i x d Data-Poor Validation

168 the validation, the mean TTI was plugged into the model equations to calculate the reliability statistics, which were then compared to the measured values. Table D.1 shows that far fewer section-year data points were generated for the peak period than for the midday period. This is because many segments did not meet the L03 definition of having a peak period, defined as a time period of at least 75 min during which the mean speed is less than 45 mph. In Spokane, none of the sections met these criteria. In Salt Lake City, only three section-years (representing one section over 3 years) met these criteria. This reduces the regional variation among the validation data sets and sug- gests that the peak period definition needs to be reevaluated in the model enhancement stage. In addition to the lack of a notable peak period in the Spokane and Salt Lake City data sets, in general, the travel times in these data sets exhibited much less variation and unreliability than in the California (CA) and Minnesota (MN) sites. This should be kept in mind when evaluating the validation results. D.2.3 Measures For each model, the goals of the validation were to quantify the model error and determine whether the model follows the key assumptions of generalized regression. This section first describes the assumptions that were tested and then presents the performance measures that were evaluated. D.2.3.1 Generalized Regression Model Assumptions As part of model validation, we examine if the key assump- tions of generalized regression models are violated. General- ized regression models have the following basic assumptions: 1. Generalized nonlinear functional form: the following for- mula states that the conditional mean of yi given xi is a continuous differential function f, that is, (a) E[yi | xi] = f(xi, b), i = 1, . . . , n (b) If this assumption is satisfied, the residuals should not show any nonrandom pattern (e.g., concave shape) in the residual plot. Otherwise, the model form may not be adequate. 2. Zero residual mean: the distribution of residuals has a mean of zero. 3. Homoscedasticity: the distribution of residuals has a con- stant variance. 4. Normal distribution of residual: it is assumed that residu- als follow the normal distribution. Performance Measures For a systematic evaluation of the model assumptions, we used the performance measures proposed in the L33 valida- tion plan: (1) root mean square error (RMSE); (2) residual plots; and (3) Student’s t-test of zero residual mean. Each of these is described below. Root Mean SquaRe eRRoR Denote the predicted response values from the model as yˆ and the measured response values as y. The prediction error (resid- ual) r is thus defined as = −r y yˆ A positive mean r implies that the model systemically over- estimates values based on new data. RMSE is defined as ∑( ) ( )= = −  = =RMSE MSE ˆ ˆ 2 2 1y E y y r n i n RMSE measures the magnitude of differences between the pre- dicted and measured responses. However, there is no simple benchmark or threshold for an acceptable RMSE. ReSidual plotS Ideally, residual r is a random variable that follows a normal distribution with zero mean. Plotting out the distribution of residuals allows for an assessment of the goodness of fit and the likelihood of the presence of bias and heteroscedasticity (unequal variance). Student’S t-teSt of ZeRo ReSidual Mean The one sample Student’s t-test can be used to determine if the mean of the residuals is significantly different from zero in a statistical sense, which tests for systematic bias. With an unbiased model, the difference should be statistically insig- nificant. The t-value is calculated as = − µ t r s n 0 where r– is the residual mean, s is the standard deviation of residuals, n is the sample size, and µ0 is the specific mean Table D.1. L33 Validation Data Sample Size (Section-Years) Period CA MN Salt Lake City Spokane All Data Midday 144 60 42 12 258 Peak period 43 19 3 0 65 Total 187 79 45 12 323

169 value for comparison, set here to be zero. To draw a conclu- sion, if the calculated t-value is larger than some threshold ta (e.g., a = 5%) using a two-tailed t distribution table, the null hypothesis that the residuals have a mean of zero can be rejected with (1ta) level of confidence. Or we say that the residual mean is significantly different from zero at a level of probability. If the corresponding p-value is used to draw a conclusion, it means that if the null hypothesis were correct, then we would expect to obtain such a large t-value on at most p percentage of occasions. For the validation, we use a 95% level of confidence. data-Poor Model Validation Results This section contains the data-poor validation results, with subsections for each of the seven data-poor models. Each sub- section is further divided into the following sections: (1) All Sites, which presents results aggregated across all regions; (2) Region Specific, which details model performance in each individual region; and (3) Summary, which includes conclu- sions on the model validation. Each All Sites model section includes the following: • A table listing the RMSE values at each study site; • A scatter plot comparing the predicted TTI curve with the measured values; • A scatter plot of the residuals by predicted TTI; • A histogram showing the distribution of the residuals; • A quantile-quantile plot (normality plot) of the residuals; and • A series of tables showing various statistics about the resid- uals and the results of the t-test to check if the mean of the residuals is statistically different from zero. Each region-specific section contains a scatter plot com- paring the predicted TTI curve with the measured values, as well as a general discussion on the validation results. The outputs listed above are also included for each region in the attachment. Each summary section contains a table listing whether the basic regression assumptions are satisfied based on statistical results and subjective observations of plots. The judgment is concluded in a qualitative way determining the satisfaction level of the assumptions, categorized as S (satisfactory) or NS (not satisfactory). Four criteria are summarized: 1. Systematic nonlinear trend. It is evaluated as “satisfac- tory” if the model can describe the validation data trend adequately, without some systematic biased pattern shown in the residual plot, in which case the residuals should be symmetrically distributed on both sides of the zero refer- ence line in the residual plot. 2. Residual: zero mean. It is evaluated as “satisfactory” if the Student’s t-test shows strong confidence (95% level) that the null hypothesis of zero residual mean cannot be rejected. 3. Residual: constant variance. It is evaluated as “satisfactory” if the distribution of residuals along the zero reference line does not show a cone shape or double bow shape. 4. Residual: normal distribution. It is evaluated as “satisfac- tory” if the residual distribution closely follows a normal distribution. 95th-Percentile TTI Model All Sites Table D.2 shows the RMSE for the 95th-percentile prediction by site. In terms of RMSE, the data-poor 95th-percentile TTI model fits the Spokane data the best, with an RMSE of only 0.0688. The error is highest with the California data set, which has an RMSE over 0.2. However, the RMSE criterion alone does not tell the whole story. As discussed in the Overview section of this appendix, the Spokane data set has very little variance compared to that of California. One needs to keep this issue in mind when examining the statistical results. The scatter plot in Figure D.1 shows that the data-poor model can generally predict the trend of the 95th-percentile TTI data. Nonetheless, the residual plot in Figure D.2 exhibits a clear cone shape increasing in range, indicating a nonconstant variance Table D.2. RMSE Summary, 95th-Percentile TTI CA MN Salt Lake City Spokane All Data Sets 0.2064 0.1716 0.0883 0.0688 0.1820 Figure D.1. Mean TTI versus 95th-percentile TTI.

170 and suggesting that some data transformation may be needed to get a better fit to the data. The histogram in Figure D.3 and the normality plot in Figure D.4 show that the distribution of residuals close to the mean varies less than that for a nor- mal distribution. The t-test results in Table D.3 show that the residual mean is larger than zero, meaning that the model tends to predict a higher 95th-percentile TTI than the data show. Region Specific califoRnia The scatter plot of the California data set in Figure D.5 shows that the predicted 95th-percentile TTI curve follows a similar pattern to that of the measured data points. However, most of the residuals are positive, with an increasing error variance as the mean TTI increases. This indicates that the model tends to overpredict the 95th-percentile travel times in the California Figure D.2. Residual plot of the 95th-percentile TTI— AllData. Figure D.3. Histogram of residuals—95th-percentile TTI—AllData. Table D.3.c. Student’s t-Test of Zero Residual Mean Test Statistic p-Value Student’s t-test 4.0291 <0.0001 Table D.3. Residual Analysis Results— 95th-percentile TTI—AllData Table D.3.a. Basic Summary Location Variability Mean 0.0399 Std deviation 0.1779 Median 0.0108 Variance 0.0316 Min -0.7684 Range 1.5732 Max 0.8048 Interquartile range 0.0767 Table D.3.b. Estimated Confidence Limits Assuming Normality Parameter Estimate 95% Confidence Limits Mean 0.0399 0.0204 0.0593 Std deviation 0.1779 0.1651 0.1927 Variance 0.0316 0.0273 0.0371 Figure D.4. Normality plot of residuals— 95th-percentile TTI—AllData.

171 regions. The t-test results reject the null hypothesis that the average of the residuals is zero. MinneSota The Minnesota data points are more evenly scattered on both sides of the data-poor model curve than were the California data points, as we can see from the scatter plot in Figure D.6. Analysis of the residuals shows a nonconstant error variance, implying that some type of variable transformation may be needed. The distribution of residuals is negatively skewed. The t-test results reject the null hypothesis that the average of the residuals is zero. In general, the model tends to underpredict the 95th-percentile travel times in the Minneapolis region. Salt lake city The Salt Lake City data samples were mostly collected dur- ing a noncongested condition, as we can see from the scatter plot in Figure D.7. It has only three sparse points whose mean TTI is larger than 1.1. When the mean TTI is below 1.1, the real data pattern is much more flattened than the predicted curve, meaning that the model tends to overpredict the 95th- percentile travel time when mean conditions are relatively uncongested. As a result, there is some nonrandom pattern in the residuals when the predicted value is below 1.5. The normality analysis indicates less variability among the resid- uals than that of a normal distribution. The t-test results reject the null hypothesis that the average of the residuals is zero. Spokane The scatter plot of the Spokane results in Figure D.8 shows a cone shape, but this may be partly attributable to the scale of the plot. All section-years had a mean TTI of less than 1.08, which indicates a nearly free-flow condition. The residuals Figure D.5. Mean TTI versus 95th-percentile TTI, California. Figure D.6. Mean TTI versus 95th-percentile TTI, Minnesota. Figure D.7. Mean TTI versus 95th-percentile TTI, Salt Lake City. Figure D.8. Mean TTI versus 95th-percentile TTI, Spokane.

172 show a nonconstant variance, but none of the residuals are large. The residuals do not follow a normal distribution. The t-test of the zero residual mean gives a p-value of 0.0767, which means that we cannot reject the zero mean null hypoth- esis with a 95% level of confidence. Summary From the validation analysis for the 95th-percentile TTI, we can conclude that, in general, the existing model can explain the variation in the 95th-percentile TTI. However, the model does not fit each region’s data set equally well. This may be due to the unique traffic flow characteristics of each region, which are difficult to generalize into a single model. The residual analysis showed violations of the basic regres- sion assumptions. Nonconstant variance of the residuals is the common problem in all of the regions. The zero residual mean assumption was rejected in all sites except for Spokane, with the model tending to overpredict the 95th-percentile TTI in California and Salt Lake City, and underpredict it in Minnesota. This implies that the nonlinear model form assumption may be violated. The Spokane data set exhibited the minimum RMSE among the four regional data sets. How- ever, the sample size and variability of this data set are not large enough to draw confident statistical conclusions. Table D.4 lists a summary of whether the generalized regres- sion assumptions are satisfied (S) or not (NS). As mentioned in the beginning of this section, the conclusions are based on subjective observation of the plots as well as objective statistical analysis. The standards for the conclusions in the table are dis- cussed in the previous section. 90th-Percentile TTI Model All Sites The RMSE table in Table D.5 shows that the 90th-percentile TTI model predicts consistently better than the 95th-percentile TTI for all four regional data sets. The largest RMSE shows up in the validation of the Minnesota data set, followed by that of the California data set. The largest RMSE is 0.15023. Overall, RMSE is 0.11890, close to that of the California data set, which may be because the dominant samples are coming from the California data set. The scatter plot in Figure D.9 and the residual plot in Fig- ure D.10 clearly show that the data-poor model is unable to fully capture the data trend, resulting in a concave shape in the residual plot. The histogram in Figure D.11 and the Table D.4. 95th-Percentile of TTI Model Validation Summary Assumptions: All CA MN Salt Lake City Spokane Systematic nonlinear trend NS NS S NS S Residual: zero mean NS NS NS NS S Residual: constant variance NS NS NS NS NS Residual: normal distribution NS NS NS NS NS Note: NS = generalized regression assumptions are not satisfied; S = satisfied. Table D.5. RMSE Summary, 90th-Percentile TTI CA MN Salt Lake City Spokane All Data Sets 0.1187 0.1502 0.0483 0.0604 0.1189 Figure D.9. Mean TTI versus 90th-percentile TTI. Figure D.10. Residual plot—90th-percentile TTI— AllData.

173 normality plot in Figure D.12 indicate that the residual distri- bution has less variance compared with a normal distribu- tion. The 95% confidence interval of estimated residual mean does not include zero, and the t-test yields a p-value of 0.0233 (Table D.6), indicating that the hypothesis of zero residual mean can be rejected. Region Specific califoRnia The California data samples fall around the predicted curve but tend to be smaller (Figure D.13). Residual analysis Figure D.11. Histogram of residuals. Table D.6.c. Student’s t-Test of Zero Residual Mean Test Statistic p-Value Student’s t-test 2.2790 0.0233 Table D.6. Statistical Residual Analysis Results—90th-percentile TTI—AllData Table D.6.a. Basic Summary Location Variability Mean 0.0150 Std deviation 0.1181 Median 0.0118 Variance 0.0140 Min -0.6737 Range 1.0585 Max 0.3848 Interquartile range 0.0503 Table D.6.b. Estimated Confidence Limits Assuming Normality Parameter Estimate 95% Confidence Limits Mean 0.0150 0.0020 0.0279 Std deviation 0.1181 0.1097 0.1280 Variance 0.0140 0.0120 0.0164 Figure D.12. Normality plot of residuals— 90th-percentile TTI—AllData. Figure D.13. Mean TTI versus 90th-percentile TTI, California.

174 confirms that most of the residuals are above the zero refer- ence line, with a positive skew in the residual distribution. The estimated residual mean is 0.0358 with an estimated con- fidence interval of 0.0194 to 0.0522. The t-test rejects the zero residual mean hypothesis. Note that there is a potential out- lier resulting in the maximal absolute residual value of 0.6737. However, since we do not have enough evidence to conclude that it should be removed from the data set, it is included in the validation analysis. MinneSota Unlike in California, in Minnesota the measured 90th- percentile TTI increases faster with the mean TTI than does the prediction. Like the 95th-percentile model, the 90th-percentile model in Minnesota tends to underpredict the measured 95th-percentile TTI (Figure D.14). The variance among the residuals is nonconstant, with the absolute value of the resid- ual reaching its maximum as the predicted value reaches its maximum. The distribution of the residuals is negatively skewed compared with a normal distribution. The t-test results reject the zero residual mean hypothesis. Salt lake city The Salt Lake City data set shows similar results to those in the 95th-percentile TTI model validation. In the area around the origin when the mean TTI is below 1.1, the measured data consistently fall below the predicted curve, as is shown in the scatter plot (Figure D.15) and the residual plot in the attach- ment (Figure D.75). The t-test results reject the zero residual mean null hypothesis. Spokane This 90th-percentile TTI data-poor model does not fit the Spokane data set very well, as shown in the scatter plot (Figure D.16). However, since the data has relatively little variance, the absolute values of the residuals are small, with a maximum of 0.1118. The main problem with this model fit is that the residuals increase almost linearly with the predicted value. The distribution of the residuals is positively skewed and not normally distributed. The t-test results reject the zero residual mean null hypothesis. Summary The L03 90th-percentile TTI data-poor model generally fits the data better than the 95th-percentile model does, as shown by the lower RMSE values. This is probably due to the fact that the 90th-percentile TTI validation data has less overall variance than the 95th-percentile TTI data. The 90th-percentile TTI model tends to overpredict the 90th-percentile TTI in Figure D.14. Mean TTI versus 90th-percentile TTI, Minnesota. Figure D.15. Mean TTI versus 90th-percentile TTI, Salt Lake City. Figure D.16. Mean TTI versus 90th-percentile TTI, Spokane.

175 the California, Salt Lake City, and Spokane data sets and underestimate it in the Minnesota data. Residual analysis indicates violation of the basic normality, zero error mean, and constant error variance assumptions (summarized in Table D.7 with all “not satisfactory” assessments). 80th-Percentile TTI Model All Sites The RMSE values for the 80th-percentile TTI model, shown in Table D.8, are even smaller than those for the 90th-percentile TTI model. This is probably due to the fact that the 80th- percentile TTI validation data set has less variance. For each of the five data sets, RMSE is less than 0.1, which means that the mean 80th-percentile travel time prediction error is less than 10% of the corresponding free-flow travel time. In this sense, the model performs satisfactorily. However, we need more complicated validation analysis to see if the regression assumptions are satisfied. The scatter plot (Figure D.17) and the residual plot (Fig- ure D.18) clearly show that the data-poor model for the 80th-percentile TTI is unable to capture the trend of the response variable. In fact, the scattered data samples show a linear or convex shape, but the data-poor model shows a concave-like shape, resulting in concavely scattered resid- ual points in the residual plot. The histogram (Figure D.19) and the normality plot (Figure D.20) both show that the residual distribution does not perfectly follow a normal distribution. The Student’s t-test yields a p-value less than 0.0001, implying that we can reject the zero mean hypothesis (Table D.9). Table D.7. 90th-Percentile of TTI Model Validation Summary Assumptions: All CA MN Salt Lake City Spokane Systematic nonlinear trend NS NS NS NS NS Residual: zero mean NS NS NS NS NS Residual: constant variance NS NS NS NS NS Residual: normal distribution NS NS NS NS NS Note: NS = generalized regression assumptions are not satisfied. Table D.8. RMSE Summary, 80th-Percentile TTI CA MN Salt Lake City Spokane All Data Sets 0.0660 0.0896 0.0290 0.0447 0.0684 Figure D.17. Mean TTI versus 80th-percentile TTI. Figure D.18. Residual plot—80th-percentile TTI—AllData. Figure D.19. Histogram of residuals— 80th-percentile TTI—AllData.

176 Region Specific califoRnia The scatter plot of California data samples shows an initial tendency to fall below the predicted curve until the mean TTI exceeds 1.7, at which point the data samples tend to be above the predicted curve (Figure D.21). This obviously shows that the data-poor model fails to capture part of the variability in the response variable. The residual analysis shows non- constant variance and a positively skewed distribution. The t-test results reject the zero residual mean hypothesis. MinneSota As in California, the 80th-percentile TTI data-poor model fails to sufficiently capture the measured data trend, espe- cially when the mean TTI is beyond 1.5. In fact, the predicted curve shows a concave shape while the real data show a slight convex shape (Figure D.22). The nonconstant error variance problem also exists. The residuals are negatively skewed, just like for the 90th and 95th-percentile models in Minnesota. However, the t-test results show a p-value of 0.5049, meaning that we cannot reject the null hypothesis of zero mean. Salt lake city The Salt Lake City data points all fall below the predicted line (Figure D.23), but the residuals are all very small (less than 0.1), meaning that the predicted values are very close to the true values. However, the residual distribution does not closely follow a normal distribution. The t-test results reject the zero residual mean null hypothesis. Spokane In the scatter plot for the Spokane results, sample points largely fall below the predicted line (Figure D.24), indicating that the model overestimates the 80th-percentile TTI. In addition, this Table D.9. Statistical Residual Analysis Results—80th-percentile TTI—AllData Table D.9.a. Basic Summary Location Variability Mean 0.0184 Std deviation 0.0660 Median 0.0135 Variance 0.0044 Min -0.3750 Range 0.6014 Max 0.2263 Interquartile range 0.0379 Table D.9.c. Student’s t-Test of Zero Residual Mean Test Statistic p-Value Student’s t-test 5.0051 <0.0001 Table D.9.b. Estimated Confidence Limits Assuming Normality Parameter Estimate 95% Confidence Limits Mean 0.0184 0.0111 0.0256 Std deviation 0.0660 0.0612 0.0715 Variance 0.0043 0.0037 0.0051 Figure D.20. Normality plot of residuals— 80th-percentile TTI—AllData. Figure D.21. Mean TTI versus 80th-percentile TTI, California.

177 overestimation tends to increase as the mean TTI increases, meaning that the residuals are positively correlated with the predicted values. The residuals are not normally distributed. The t-test results reject the zero residual mean null hypothesis. Summary Although the RMSE table shows that the average prediction error is within 10% of TTI, the validation analysis for the 80th-percentile TTI data-poor model shows that the lack of fit is obvious for all four data sets. Specifically, the model fails to capture the correct curvature of the California data and Minnesota data and tends to overestimate the 80th-percentile TTI for the Salt Lake City and Spokane data sets. Essentially they are the same problem, since the last two data sets have less variance and thus only represent the area with low mean TTIs. Nonnormally distributed residual, nonzero residual mean, and nonconstant error variance problems were all shown in the validation analysis (Table D.10), but the primary concern is the lack of fit for the curvature. Standard Deviation of TTI Model Overview Table D.11 summarizes the RMSE for the data-poor standard deviation (std) of the TTI model. The magnitude of these values is not large since the standard deviation of TTI data itself has a small magnitude, mostly less than 1.1. The highest RMSE is in Minnesota while the lowest is in Salt Lake City. The scatter plot (Figure D.25) and the residual plot (Fig- ure D.26) together show that the model has lack of fit problems. Figure D.22. 80th-percentile TTI versus mean TTI. Figure D.23. 80th-percentile TTI versus mean TTI, Salt Lake City. Figure D.24. 80th-percentile TTI versus mean TTI, Spokane. Table D.10. 80th-Percentile of TTI Model Validation Summary Assumptions: All CA MN Salt Lake City Spokane Systematic nonlinear trend NS NS NS NS NS Residual: zero mean NS NS S NS NS Residual: constant variance NS NS NS NS NS Residual: normal distribution NS NS NS NS NS Note: NS = generalized regression assumptions are not satisfied; S = satisfied. Table D.11. RMSE Summary, Standard Deviation of TTI CA MN Salt Lake City Spokane All Data Sets 0.0839 0.1028 0.0586 0.0672 0.0855

178 The measured standard deviation increases faster than the predicted standard deviation. The histogram (Figure D.27) and the normality plot (Figure D.28) both indicate that the residual distribution approximately follows a normal curve, although a negative skew exists. In Table D.12, a p-value of 0.0430 in the Student’s t-test demonstrates that the residual distribution does not have a zero mean. Region Specific califoRnia The scatter plot of the California data set in Figure D.29 shows that most data points fall near the predicted curve. However, the shapes of the scattered points and the predicted line do not match very well; the predicted line initially tends to overesti- mate the standard deviation of TTI and then under estimates it at high mean TTIs. The residuals indicate non constant vari- ance and a lack of fit problem. The residual distribution is close to normal, with a small positive skew. The t-test results reject the null hypothesis that the residual mean is zero. MinneSota The problem of lack of fit is obvious in the Minnesota data scatter plot. The model line fails to follow the upward trend in the measured data set, instead gradually flattening as the mean TTI increases (Figure D.30). The residual analysis confirms this problem and shows that the residual variance is non constant. Figure D.25. Standard deviation TTI versus mean TTI. Figure D.26. Residual plot—standard deviation of TTI—AllData. Figure D.27. Histogram of residuals—standard deviation of TTI—AllData. Figure D.28. Normality plot of residuals—standard deviation of TTI—AllData.

179 (Figure D.31). The residual analysis shows the nonconstant variance problem. The t-test has a p-value of 0.0907, meaning that we cannot reject the null hypothesis that the residual mean is zero. Spokane The mean TTI in the Spokane data spreads from 1.0 to 1.08. Thus the data has very little variance and is collected all in near free-flow conditions. Figure D.32 shows that only two data points fall to the left of the data-poor model curve while all other 10 points fall to the right of that curve. The residuals are scattered in an unbalanced way and have non- constant variance. The distribution of residuals does not closely follow a normal distribution. The t-test results show that we can reject the null hypothesis that the residual mean is zero. Table D.12. Statistical Residual Analysis Results—standard deviation of TTI Table D.12.a. Basic Summary Location Variability Mean 0.0096 Std deviation 0.0850 Median 0.0110 Variance 0.0072 Min -0.4453 Range 0.7044 Max 0.2590 Interquartile range 0.0626 Table D.12.b. Estimated Confidence Limits Assuming Normality Parameter Estimate 95% Confidence Limits Mean 0.0096 0.0003 0.0189 Std deviation 0.0851 0.0790 0.0922 Variance 0.0072 0.0062 0.0085 Table D.12.c. Student’s t-Test of Zero Residual Mean Test Statistic p-Value Student’s t-Test 2.0322 0.0430 Figure D.29. Standard deviation TTI versus mean TTI, California. Figure D.30. Standard deviation TTI versus mean TTI, Minnesota. Figure D.31. Standard deviation TTI versus mean TTI, Salt Lake City. The distribution of residuals is negatively skewed. The t-test results reject the null hypothesis of zero residual mean. Salt lake city The standard deviation of a TTI data-poor model largely predicts the trend in the measured Salt Lake City data

180 Summary Overall, nonconstant variance is a problem for all data sets, and the nonzero residual mean is a problem for all data sets except for Salt Lake City. The California data set presents rela- tively good residual distributions that closely follow a normal distribution. Overall, though, we conclude that the standard deviation of TTI model does not adequately capture the mea- sured data pattern. Table D.13 summarizes the validation results for this model based on subjective observation and objective statistical analysis. Percentage of On-Time Trips with Over 50 mph Mean Speed All Sites Let “PctTripsOnTime50mph” denote a short name for “per- centage of on-time trips with over 50 mph mean speed.” Similarly, in the next two sections, “PctTripsOnTime45mph” and “PctTripsOnTime30mph” are used for mean speed thresholds of 45 mph and 30 mph, respectively. The RMSE table for PctTripsOnTime50mph shows that the smallest RMSE comes from the Salt Lake City data, while the largest comes from the California data (Table D.14). However, all RMSE values are less than 0.1, or 10% of the number of all trips, which indicates good model performance. The scatter plot (Figure D.33) shows that the data-poor model can generally predict the trend of PctTripsOnTime 50mph. However, the residual plot (Figure D.34) shows a clear non random pattern when mean TTI is around 1.0, indi- cating that the model may be improved by some form of data transformation. The histogram (Figure D.35) and the nor- mality plot (Figure D.36) both show that the residual distri- bution displays an almost perfect normal distribution shape when the residual is less than zero but differentiates from the normal distribution reference line when the residual is larger than zero. It is also shown that more samples have negative residuals than have positive residuals. The t-test results in Table D.15 demonstrate that the null hypothesis of the zero residual mean can be rejected with a 95% level Figure D.32. Standard deviation TTI versus mean TTI, Spokane. Table D.13. Standard Deviation of TTI Model Validation Summary Assumptions: All CA MN Salt Lake City Spokane Systematic nonlinear trend NS NS NS S NS Residual: zero mean NS NS NS S NS Residual: constant variance NS NS NS NS NS Residual: normal distribution S S NS NS NS Note: NS = generalized regression assumptions are not satisfied; S = satisfied. Table D.14. RMSE Summary, PctTripsOnTime50mph CA MN Salt Lake City Spokane All Data Sets 0.0891 0.0617 0.0552 0.0721 0.0784 Figure D.33. PctTripsOnTime50mph versus mean TTI.

181 Figure D.34. Residual plot—PctTripsOnTime50mph— AllData. Figure D.35. Histogram of residuals— PctTripsOnTime50mph—AllData. of confidence since the p-value is less than 0.0001. Overall, the model performs satisfyingly to some extent but leaves room for improvement. Region Specific califoRnia The PctTripsOnTime50mph data-poor model tends to underestimate the response variable in California when the mean TTI is close to 1.0 (uncongested conditions). How- ever, when the mean TTI becomes larger than 1.4, the data- poor model tends to overestimate the response variable Figure D.36. Normality plot of residuals— PctTripsOnTime50mph—AllData. Table D.15. Statistical Residual Analysis Results—PctTripsOnTime50mph— AllData Table D.15.a. Basic Summary Location Variability Mean -0.0221 Std deviation 0.0754 Median -0.0172 Variance 0.0057 Min -0.2374 Range 0.6062 Max 0.3687 Interquartile range 0.0565 Table D.15.b. Estimated Confidence Limits Assuming Normality Parameter Estimate 95% Confidence Limits Mean -0.0221 -0.0303 -0.0138 Std deviation 0.0754 0.0700 0.0817 Variance 0.0057 0.0049 0.0067 Table D.15.c. Student’s t-Test of Zero Residual Mean Test Statistic p-Value Student’s t-Test -5.2626 <0.0001

182 (Figure D.37). The residuals show nonconstant variance. The residual distribution approximately follows a normal distribution but with a positive skew. The t-test results show that evidence is not sufficient to reject the null hypothesis of zero residual mean. MinneSota In the Minnesota data validation, the data-poor model tends to consistently underestimate the response variable, as shown in Figure D.38. The nonconstant variance problem exists in this data set. The residuals do not closely follow a normal distribution and have a slight negative skew. The t-test results show strong evidence to reject the null hypothesis of zero residual mean. Salt lake city The data-poor model fails to capture the Salt Lake City data pattern when the mean TTI is below 1.1, as shown in the scatter plot (Figure D.39). The nonconstant variance problem still exists, but the primary concern is the inability to predict the measured data trend. The distribution of residuals does not closely follow a normal distribution and is negatively skewed. The t-test results reject the zero residual mean hypothesis. Spokane The Spokane measured data all fall above the data-poor model line (Figure D.40). There is a positive correlation between the residuals and the predicted values. The residuals are not normally distributed but rather appear uniformly distributed. Figure D.37. PctTripsOnTime50mph versus mean TTI, California. Figure D.38. PctTripsOnTime50mph versus mean TTI, Minnesota. Figure D.39. PctTripsOnTime50mph versus mean TTI, Salt Lake City. Figure D.40. PctTripsOnTime50mph versus mean TTI, Spokane.

183 The t-test results show sufficient evidence to reject the zero residual mean hypothesis. Summary The PctTripsOnTime50mph data-poor model can largely pre- dict the data trend of the measured validation data sets but tends to underestimate PctTripsOnTime50mph, especially when the mean TTI is small. The RMSE values are all below 0.1, indicating an average prediction error of less than 10% of the total number of trips. The California data has a weak indi- cation of zero residual mean, but other data sets all show strong evidence of violating this assumption. The residual plots show that there is some uncaptured pattern in the data sets. The constant variance assumption cannot be satisfied in any data set; neither can the normal residual distribution assumption. Table D.16 summarizes these conclusions in a qualitative way. Percentage of On-Time Trips with Over 45 mph Mean Speed All Sites The PctTripsOnTime45mph model validation results show that all the RMSE values are between 4% and 7%, which is an indication of relatively good performance (Table D.17). The scatter plot demonstrates that the data-poor model can largely predict the trend of the validation data, but it tends to underestimate the response when the mean TTI is below 1.1 (Figure D.41). A corresponding pattern along with an indi- cation of nonconstant variance can be found in the residual plot (Figure D.42). The histogram (Figure D.43) and the normality plot (Figure D.44) both show that the residual Table D.16. PctTripsOnTime50mph Model Validation Summary Assumptions: All CA MN Salt Lake Spokane Systematic nonlinear trend NS NS NS NS NS Residual: zero mean NS S NS NS NS Residual: constant variance NS NS NS NS NS Residual: normal distribution NS NS NS NS NS Note: NS = generalized regression assumptions are not satisfied; S = satisfied. Table D.17. RMSE Summary, PctTripsOnTime45mph CA MN Salt Lake City Spokane All Data Sets 0.0681 0.0480 0.0433 0.0553 0.0602 Figure D.41. PctTripsOnTime45mph versus mean TTI. Figure D.42. Residual plot— PctTripsOnTime45mph—AllData. Figure D.43. Histogram of residuals— PctTripsOnTime45mph—AllData.

184 distribution closely follows a normal distribution when the residual is less than zero but is skewed when the residual is larger than zero. The Student’s t-test yields a p-value less than 0.0001, indicating that the null hypothesis can be rejected at a 95% level of confidence, as shown in Table D.18. Region Specific califoRnia The data-poor model predicts a curve that generally approxi- mates the measured trend. However, it tends to under estimate the response variable when the mean TTI is smaller than 1.1 while overestimating when the mean TTI is larger than 1.4 (Figure D.45). The residuals closely follow a normal distribu- tion when the residuals are negative but are skewed when the residuals are positive. The t-test results imply that the zero mean residual assumption can be rejected. MinneSota The scatter plot (Figure D.46) and the residual analysis indicate that the data-poor model can predict the general pattern of the Minnesota data but tend to underestimate the response vari- able. The nonconstant variance problem also exists. The resid- uals do not appear to be normally distributed. The t-test shows strong evidence to reject the zero residual mean hypothesis. Salt lake city The data-poor model tends to underestimate the response variable in the Salt Lake City validation data (Figure D.47). The nonconstant residual variance is evident, but the inade- quate model trend problem is the primary concern. The resid- ual distribution does not closely follow a normal distribution. Figure D.44. Normality plot of residuals— PctTripsOnTime45mph—AllData. Table D.18. Statistical Residual Analysis Results—PctTripsOnTime45mph— AllData Table D.18.a. Basic Summary Location Variability Mean -0.0196 Std deviation 0.0570 Median -0.0156 Variance 0.0033 Min -0.1877 Range 0.4463 Max 0.2587 Interquartile range 0.0438 Table D.18.b. Estimated Confidence Limits Assuming Normality Parameter Estimate 95% Confidence Limits Mean -0.0196 -0.0258 -0.0133 Std deviation 0.0570 0.0530 0.0618 Variance 0.0033 0.0028 0.0038 Table D.18.c. Student’s t-Test of Zero Residual Mean Test Statistic p-Value Student’s t-test -6.1626 <0.0001 Figure D.45. PctTripsOnTime45mph versus mean TTI, California. The t-test results indicate that we can reject the zero residual mean assumption. Spokane In the Spokane scatter plot (Figure D.48), sample points all fall above the data-poor model curve. As the mean TTI increases,

185 the measured PctTripsOnTime45mph decreases much more slowly than the predicted curve. The residuals are positively correlated with the predicted value, which implies lack of fit. Because the Spokane data have little variance, the prediction error is in fact not large. The residuals appear to be uniformly, rather than normally, distributed. The t-test results indicate that we can reject the zero residual mean assumption. Summary The PctTripsOnTime45mph data-poor model can predict the general trend of the validation data sets. However, evi- dence suggests that the models violate basic regression assumptions, meaning that the model has room for improve- ment. Specifically, the model tends to underestimate when mean TTI is small, which indicates lack of fit. Note that all residual means are negative. The constant residual variance assumption is found to be violated through examining the residual plot. Normal distribution of residuals and zero resid- ual mean assumptions cannot be satisfied either. Table D.19 summarizes the performance evaluation results qualitatively. Percentage of On-Time Trips with Over 30 mph Mean Speed All Sites The PctTripsOnTime30mph data-poor model predicts the four validation data sets with a maximum RMSE of 0.0329, indicat- ing good performance (Table D.20). The model largely captures the data trend (Figure D.49), and the residual plot (Figure D.50) shows a much better pattern than that in the previous two sec- tions. However, a potential lack of fit is evidenced by the Figure D.46. PctTripsOnTime45mph versus mean TTI, Minnesota. Figure D.47. PctTripsOnTime45mph versus mean TTI, Salt Lake City. Figure D.48. PctTripsOnTime45mph versus mean TTI, Spokane. Table D.19. PctTripsOnTime45mph Model Validation Summary Assumptions: All CA MN Salt Lake Spokane Systematic nonlinear trend NS NS NS NS NS Residual: zero mean NS NS NS NS NS Residual: constant variance NS NS NS NS NS Residual: normal distribution NS NS NS NS NS Note: NS = generalized regression assumptions are not satisfied. Table D.20. RMSE Summary, PctTripsOnTime30mph CA MN Salt Lake City Spokane All Data Sets 0.0247 0.0329 0.0134 0.00674 0.0254

186 concave-like shape. A nonconstant variance problem is also indicated by the cone shape in the residual plot. The histogram and the normality plot (Figure D.51 and Figure D.52) show that the residual distribution has much less variance than that of a normal distribution but the residual mean is highly likely to be zero. The Student’s t-test demonstrates this zero residual mean assumption with a p-value of 0.6689 (Table D.21). Region Specific califoRnia The scatter plot of California data shows that the data-poor model closely predicts the measured data trend (Figure D.53). However, it is also clear that the model can still be improved Figure D.49. PctTripsOnTime30mph versus mean TTI. Figure D.50. Residual plot— PctTripsOnTime30mph—AllData. Figure D.51. Histogram of residuals— PctTripsOnTime30mph—AllData. since the model tends to underestimate the response variable when the mean TTI is below 1.6 and to overestimate it when the mean TTI is above 1.6. The residual analysis indicates lack of fit and nonconstant variance. The residuals do not appear to be normally distributed. The t-test results show that the null hypothesis of zero residual mean cannot be rejected. MinneSota The scatter plot of Minnesota data also shows that this data- poor model generally follows the measured data trend but is obviously not adequate (Figure D.54). The residual plot included in the attachment shows a concave pattern, indi- cating some lack of fit. The nonconstant variance problem Figure D.52. Normality plot of residuals— PctTripsOnTime30mph—AllData.

187 Table D.21. Statistical Residual Analysis Results—PctTripsOnTime30mph— AllData Table D.21.a. Basic Summary Location Variability Mean -0.0006 Std deviation 0.0254 Median -0.0055 Variance 0.0006 Min -0.1103 Range 0.2336 Max 0.1233 Interquartile range 0.0021 Table D.21.b. Estimated Confidence Limits Assuming Normality Parameter Estimate 95% Confidence Limits Mean -0.0006 -0.0034 0.0022 Std deviation 0.0254 0.0236 0.0275 Variance 0.0006 0.0006 0.0008 Table D.21.c. Student’s t-Test of Zero Residual Mean Test Statistic p-Value Student’s t-test -0.4281 0.6689 Figure D.53. PctTripsOnTime30mph versus mean TTI, California. Figure D.54. PctTripsOnTime30mph versus mean TTI, Minnesota. Figure D.55. PctTripsOnTime30mph versus mean TTI, Salt Lake City. may also exist. The t-test cannot reject the null hypothesis of zero residual mean. Salt lake city The scatter plot of the Salt Lake City data set shows that the data-poor model follows the measured data but tends to underestimate the response variable when the mean TTI is small, while overestimating it when the mean TTI is large (Fig- ure D.55). The residuals do not appear to be normally distrib- uted. The t-test results indicate that we cannot reject the zero residual mean null hypothesis. Spokane The scatter plot of the Spokane data shows that the data-poor model tends to underpredict the measured values (Figure D.56). The residuals do not appear to be normally distributed. The t-test results indicate that we can reject the zero residual mean hypothesis. Summary This PctTripsOnTime30mph data-poor model can largely pre- dict the measured data trend but not with adequate fit. All but

188 the Spokane data satisfy the zero residual mean assumption. The normal-distributed residual assumption and the constant residual variance assumption are not satisfied in any regional data set. Potential improvements should address the slight con- cave pattern shown in the residual plot. Table D.22 summarizes the model validation results for the PctTripsOnTime30mph model. Conclusions The validation of the L03 data-poor models was performed on four regional data sets (California, Minnesota, Salt Lake City, and Spokane), as well as on the combined data sets over- all. The main conclusion is that, while the average prediction error (measured by the RMSE) of each model is generally acceptable across the regions, the models violate many of the assumptions of generalized regression and thus have room for enhancement. Most of the models in nearly all of the regions violate the zero residual mean assumption, meaning that the models tend to either systematically overpredict the reliability mea- sure (i.e., indicate that a section is less reliable than it actually is) or underpredict the reliability measure (i.e., indicate that a section is more reliable than it actually is). Interestingly, this systematic bias appears to vary regionally, with the models tending to underpredict the reliability measures in Minne- sota and overpredict them in California. This lends support for building regional models rather than cross-sectional models, although insufficient data are an obstacle to regional modeling. Additionally, most of the models in most of the regions vio- late the assumption of constant variance of the residuals. In just about all cases, the variance of the residuals increases with the mean TTI. This makes intuitive sense, as higher levels of baseline, recurrent congestion lead to more unreliable and unpredictable conditions. The models also tend to violate the assumptions that the residuals are normally distributed and that the model form can adequately predict the data trend. These problems all indicate that the data-poor model perfor- mance can be improved. Another conclusion is that the model error is larger for the prediction of higher moments of the TTI distribution (i.e., the RMSEs are larger for the 95th-percentile model than for the 90th and 80th-percentile models). This makes sense because the 95th-percentile TTI is likely associated with very rare events (like a major incident or bad weather). We would thus expect these TTIs to vary greatly from section to section, making them harder to accurately model based solely on the mean TTI. In interpreting these results and conclusions, it is important to understand how they are affected by the validation data set characteristics. Eighty percent of the section-year data included in this validation effort were collected during the weekday midday period, where mean TTIs were heavily clustered around 1. The RMSEs for this mean TTI area were very low, since section time periods that operate in free-flow conditions are relatively reliable. As illustrated by the consistent violation of the constant variance of residuals assumptions, the model error is much higher for larger mean TTIs, and these are the conditions under which systematic bias in the prediction is most evident. Unfortunately, due to the stringent definition of a peak period used in L03, very little of the Salt Lake City and Spokane data were able to contribute to this congested regime analysis, so we were only able to observe the model response to mean congested conditions at two sites (technically, the Cali- fornia data represent four regions). In the model enhancement phase, the research team hopes to loosen the peak period defi- nition to be able to consider and evaluate more of the Salt Lake City and Spokane congestion. Figure D.56. PctTripsOnTime30mph versus mean TTI, Spokane. Table D.22. PctTripsOnTime30mph Model Validation Summary Assumptions: All CA MN Salt Lake City Spokane Systematic nonlinear trend NS NS NS NS NS Residual: zero mean S S S S NS Residual: constant variance NS NS NS NS NS Residual: normal distribution NS NS NS NS NS

189 Appendix D Attachment 95th-Percentile TTi Model California Figure D.57. Residual plot—95th-percentile TTI— California. Figure D.58. Residual histogram—95th-percentile TTI—California. Figure D.59. Residual normality plot—95th-percentile TTI—California. Table D.23. Residual Analysis— 95th-Percentile TTI—California Table D.23.a. Basic Summary Location Variability Mean 0.0799 Std deviation 0.1908 Median 0.0214 Variance 0.0364 Min -0.7684 Range 1.5732 Max 0.8048 Interquartile range 0.1181 (continued on next page)

Minnesota Table D.23. Residual Analysis— 95th-Percentile TTI—California (continued) Table D.23.b. Estimated Confidence Limits Assuming Normality Parameter Estimate 95% Confidence Limits Mean 0.0799 0.0524 0.1074 Std deviation 0.1908 0.1732 0.2124 Variance 0.0364 0.0300 0.0451 Table D.23.c. Student’s t-Test of Zero Residual Mean Test Statistic p-Value Student’s t-test 5.7275 <0.0001 Figure D.60. Residual plot—95th-percentile TTI— Minnesota. Figure D.61. Residual histogram—95th-percentile TTI—Minnesota. Table D.24. Residual Analysis— 95th-percentile TTI—Minnesota Table D.24.a. Basic Summary Location Variability Mean -0.0474 Std deviation 0.1660 Median -0.0016 Variance 0.0276 Min -0.6480 Range 1.0239 Max 0.3760 Interquartile range 0.0618 Table D.24.b. Estimated Confidence Limits Assuming Normality Parameter Estimate 95% Confidence Limits Mean -0.0473 -0.0845 -0.0102 Std deviation 0.1660 0.1435 0.1968 Variance 0.0276 0.0206 0.0387 Table D.24.c. Student’s t-Test of Zero Residual Mean Test Statistic p-Value Student’s t-test -2.5351 0.0132 Figure D.62. Residual normality plot—95th-percentile TTI—Minnesota.

191 Salt Lake City Figure D.63. Residual plot—95th-percentile TTI—Salt Lake City. Figure D.64. Residual histogram—95th-percentile TTI—Salt Lake City. Figure D.65. Residual normality plot—95th-percentile TTI—Salt Lake City. Table D.25. Residual Analysis— 95th-Percentile TTI—Salt Lake City Table D.25.a. Basic Summary Location Variability Mean 0.0279 Std deviation 0.0848 Median 0.0063 Variance 0.0072 Min -0.2786 Range 0.6060 Max 0.3274 Interquartile range 0.0482 Table D.25.b. Estimated Confidence Limits Assuming Normality Parameter Estimate 95% Confidence Limits Mean 0.0279 0.0025 0.0534 Std deviation 0.0848 0.0702 0.1071 Variance 0.0072 0.0049 0.0115 Table D.25.c. Student’s t-Test of Zero Residual Mean Test Statistic p-Value Student’s t-test 2.2096 0.0324

192 Spokane Figure D.66. Residual plot—95th-percentile TTI— Spokane. Figure D.67. Residual histogram—95th-percentile TTI—Spokane. Table D.26. Residual Analysis— 95th-Percentile TTI—Spokane Table D.26.a. Basic Summary Location Variability Mean 0.0349 Std deviation 0.0620 Median 0.0309 Variance 0.0038 Min -0.0667 Range 0.1806 Max 0.1139 Interquartile range 0.1072 Table D.26.b. Estimated Confidence Limits Assuming Normality Parameter Estimate 95% Confidence Limits Mean 0.0349 -0.0044 0.0743 Std deviation 0.0620 0.0439 0.1052 Variance 0.0038 0.0019 0.0111 Table D.26.c. Student’s t-Test of Zero Residual Mean Test Statistic p-Value Student’s t-test 1.9534 0.0767 Figure D.68. Residual normality plot—95th-percentile TTI—Spokane.

193 90th-Percentile TTi Model California Table D.27. Residual Analysis— 90th-Percentile TTI—California Table D.27.a. Basic Summary Location Variability Mean 0.0358 Std deviation 0.1135 Median 0.0169 Variance 0.0129 Min -0.6737 Range 1.0585 Max 0.3848 Interquartile range 0.0611 Table D.27.b. Estimated Confidence Limits Assuming Normality Parameter Estimate 95% Confidence Limits Mean 0.0358 0.0194 0.0522 Std deviation 0.1135 0.1030 0.1263 Variance 0.0129 0.0106 0.0160 Table D.27.c. Student’s t-Test of Zero Residual Mean Test Statistic p-Value Student’s t-test 4.3133 <0.0001 Figure D.69. Residual plot—90th-percentile TTI— California. Figure D.70. Residual histogram—90th-percentile TTI—California. Figure D.71. Residual normality plot—90th-percentile TTI—California.

194 Minnesota Figure D.72. Residual plot—90th-percentile TTI— Minnesota. Figure D.73. Residual histogram—90th-percentile TTI—Minnesota. Table D.28. Residual Analysis— 90th-Percentile TTI—Minnesota Table D.28.a. Basic Summary Location Variability Mean -0.0416 Std deviation 0.1453 Median 0.0064 Variance 0.0211 Min -0.5958 Range 0.7700 Max 0.1742 Interquartile range 0.0297 Table D.28.b. Estimated Confidence Limits Assuming Normality Parameter Estimate 95% Confidence Limits Mean -0.0416 -0.0741 -0.0090 Std deviation 0.1453 0.1256 0.1723 Variance 0.0211 0.0158 0.0297 Table D.28.c. Student’s t-Test of Zero Residual Mean Test Statistic p-Value Student’s t-test -2.5415 0.0130 Figure D.74. Residual normality plot—90th-percentile TTI—Minnesota.

195 Salt Lake City Figure D.75. Residual plot—90th-percentile TTI—Salt Lake City. Figure D.76. Residual histogram—90th-percentile TTI—Salt Lake City. Table D.29. Residual Analysis— 90th-Percentile TTI—Salt Lake City Table D.29.a. Basic Summary Location Variability Mean 0.0195 Std deviation 0.0446 Median 0.0048 Variance 0.0020 Min -0.1603 Range 0.2729 Max 0.1126 Interquartile range 0.0284 Table D.29.b. Estimated Confidence Limits Assuming Normality Parameter Estimate 95% Confidence Limits Mean 0.0195 0.0061 0.0329 Std deviation 0.0446 0.0369 0.0564 Variance 0.0020 0.0014 0.0038 Table D.29.c. Student’s t-Test of Zero Residual Mean Test Statistic p-Value Student’s t-test 2.9347 0.0053 Figure D.77. Residual normality plot—90th-percentile TTI—Salt Lake City.

196 Spokane Figure D.78. Residual plot—90th-percentile TTI— Spokane. Figure D.79. Residual histogram—90th-percentile TTI—Spokane. Table D.30. Residual Analysis— 90th-Percentile TTI—Spokane Table D.30.a. Basic Summary Location Variability Mean 0.0457 Std deviation 0.0412 Median 0.0491 Variance 0.0017 Min -0.0033 Range 0.1151 Max 0.1118 Interquartile range 0.0773 Table D.30.b. Estimated Confidence Limits Assuming Normality Parameter Estimate 95% Confidence Limits Mean 0.0457 0.0195 0.0719 Std deviation 0.0412 0.0292 0.0700 Variance 0.0017 0.0009 0.0049 Table D.30.c. Student’s t-Test of Zero Residual Mean Test Statistic p-Value Student’s t-test 3.8418 0.0027 Figure D.80. Residual normality plot—90th-percentile TTI—Spokane.

197 80th-Percentile TTi Model California Figure D.81. Residual plot—80th-percentile TTI— California. Figure D.82. Residual histogram—80th-percentile TTI—California. Table D.31. Residual Analysis— 80th-Percentile TTI—California Table D.31.a. Basic Summary Location Variability Mean 0.0282 Std deviation 0.0598 Median 0.0159 Variance 0.0036 Min -0.1918 Range 0.4181 Max 0.2263 Interquartile range 0.0473 Table D.31.b. Estimated Confidence Limits Assuming Normality Parameter Estimate 95% Confidence Limits Mean 0.0282 0.0196 0.0369 Std deviation 0.0598 0.0543 0.0665 Variance 0.0036 0.0029 0.0044 Table D.31.c. Student’s t-Test of Zero Residual Mean Test Statistic p-Value Student’s t-test 6.4630 <0.0001 Figure D.83. Residual normality plot—80th-percentile TTI—California.

198 Minnesota Figure D.84. Residual plot—80th-percentile TTI— Minnesota. Figure D.85. Residual histogram—80th-percentile TTI—Minnesota. Table D.32. Residual analysis— 80th-Percentile TTI—Minnesota Table D.32.a. Basic Summary Location Variability Mean -0.0068 Std deviation 0.0899 Median 0.0123 Variance 0.0081 Min -0.3750 Range 0.5394 Max 0.1644 Interquartile range 0.0211 Table D.32.b. Estimated Confidence Limits Assuming Normality Parameter Estimate 95% Confidence Limits Mean -0.0068 -0.0269 0.0134 Std deviation 0.0899 0.0777 0.1066 Variance 0.0081 0.0060 0.0114 Table D.32.c. Student’s t-Test of Zero Residual Mean Test Statistic p-Value Student’s t-test -0.6700 0.5049 Figure D.86. Residual normality plot—80th-percentile TTI—Minnesota.

199 Salt Lake City Figure D.87. Residual plot—80th-percentile TTI—Salt Lake City. Figure D.88. Residual histogram—80th-percentile TTI—Salt Lake City. Table D.33. Residual Analysis—80th-Percentile TTI—Salt Lake City Table D.33.a. Basic Summary Location Variability Mean 0.017496 Std deviation 0.02340 Median 0.004669 Variance 0.0005477 Min 0.000173969 Range 0.08334 Max 0.0835138 Interquartile range 0.01919 Table D.33.b. Estimated Confidence Limits Assuming Normality Parameter Estimate 95% Confidence Limits Mean 0.017496 0.010465 0.024527 Std deviation 0.023403 0.019374 0.029563 Variance 0.000548 0.000375 0.000874 Table D.33.c. Student’s t-Test of Zero Residual Mean Test Statistic p-Value Student’s t-test 5.015032 <0.0001 Figure D.89. Residual normality plot—80th-percentile TTI—Salt Lake City.

200 Spokane Figure D.90. Residual plot—80th-percentile TTI— Spokane. Figure D.91. Residual histogram—80th-percentile TTI—Spokane. Table D.34. Residual Analysis— 80th-Percentile TTI—Spokane Table D.34.a. Basic Summary Location Variability Mean 0.0332 Std deviation 0.0312 Median 0.0310 Variance 0.0010 Min -0.0093 Range 0.0887 Max 0.0794 Interquartile range 0.0549 Table D.34.b. Estimated Confidence Limits Assuming Normality Parameter Estimate 95% Confidence Limits Mean 0.0332 0.0134 0.0531 Std deviation 0.0312 0.0221 0.0530 Variance 0.0010 0.0005 0.0028 Table D.34.c. Student’s t-Test of Zero Residual Mean Test Statistic p-Value Student’s t-test 3.6870 0.0036 Figure D.92. Residual normality plot—80th-percentile TTI—Spokane.

201 Standard deviation of TTi Model California Figure D.93. Residual plot—standard deviation of TTI—California. Figure D.94. Residual histogram—standard deviation of TTI—California. Table D.35. Residual Analysis— Standard Deviation of TTI—California Table D.35.a. Basic Summary Location Variability Mean 0.0235 Std deviation 0.0808 Median 0.0160 Variance 0.0065 Min -0.2805 Range 0.5395 Max 0.2590 Interquartile range 0.0739 Table D.35.b. Estimated Confidence Limits Assuming Normality Parameter Estimate 95% Confidence Limits Mean 0.0808 0.0118 0.0351 Std deviation 0.0065 0.0733 0.0899 Variance 0.5395 0.0054 0.0081 Table D.35.c. Student’s t-Test of Zero Residual Mean Test Statistic p-Value Student’s t-test 3.9710 0.0001 Figure D.95. Residual normality plot—standard deviation of TTI—California.

202 Minnesota Figure D.96. Residual plot—standard deviation of TTI—Minnesota. Figure D.97. Residual histogram—standard deviation of TTI—Minnesota. Table D.36. Residual Analysis— Standard Deviation of TTI—Minnesota Table D.36.a. Basic Summary Location Variability Mean -0.0309 Std deviation 0.0987 Median 0.0033 Variance 0.0097 Min -0.4453 Range 0.5404 Max 0.0950 Interquartile range 0.0693 Table D.36.b. Estimated Confidence Limits Assuming Normality Parameter Estimate 95% Confidence Limits Mean -0.0309 -0.0530 -0.0088 Std deviation 0.0987 0.0853 0.1170 Variance 0.0097 0.0073 0.0137 Table D.36.c. Student’s t-Test of Zero Residual Mean Test Statistic p-Value Student’s t-test -2.7848 0.0067 Figure D.98. Residual normality plot—standard deviation of TTI—Minnesota.

203 Salt Lake City Figure D.99. Residual plot—standard deviation of TTI—Salt Lake City. Figure D.100. Residual histogram—standard deviation of TTI—Salt Lake City. Table D.37. Residual Analysis— Standard Deviation of TTI—Salt Lake City Table D.37.a. Basic Summary Location Variability Mean 0.0148 Std deviation 0.0574 Median 0.0021 Variance 0.0033 Min -0.1706 Range 0.2922 Max 0.1216 Interquartile range 0.0645 Table D.37.b. Estimated Confidence Limits Assuming Normality Parameter Estimate 95% Confidence Limits Mean 0.0148 -0.0024 0.0320 Std deviation 0.0573 0.0475 0.0724 Variance 0.0033 0.0023 0.0052 Table D.37.c. Student’s t-Test of Zero Residual Mean Test Statistic p-Value Student’s t-test 1.7297 0.0907 Figure D.101. Residual normality plot—standard deviation of TTI—Salt Lake City.

204 Spokane Figure D.102. Residual plot—standard deviation of TTI—Spokane. Figure D.103. Residual histogram—standard deviation of TTI—Spokane. Table D.38. Residual Analysis— Standard Deviation of TTI—Spokane Table D.38.a. Basic Summary Location Variability Mean 0.0415 Std deviation 0.0551 Median 0.0504 Variance 0.0030 Min -0.0645 Range 0.1829 Max 0.1184 Interquartile range 0.0616 Table D.38.b. Estimated Confidence Limits Assuming Normality Parameter Estimate 95% Confidence Limits Mean 0.0415 0.0065 0.0766 Std deviation 0.0551 0.0390 0.0936 Variance 0.0030 0.0015 0.0088 Table D.38.c. Student’s t-Test of Zero Residual Mean Test Statistic p-Value Student’s t-test 2.6108 0.0242 Figure D.104. Residual normality plot—standard deviation of TTI—Spokane.

205 Percentage of On-Time Trips with Over 50 mph Mean Speed California Figure D.105. Residual plot—percentage of on-time trips over 50 mph—California. Figure D.106. Residual histogram—percentage of on-time trips over 50 mph—California. Table D.39. Residual Analysis— Percentage of On-Time Trips Over 50 mph—California Table D.39.a. Basic Summary Location Variability Mean -0.0117 Std deviation 0.0886 Median -0.0153 Variance 0.0079 Min -0.2374 Range 0.6062 Max 0.3687 Interquartile range 0.0559 Table D.39.b. Estimated Confidence Limits Assuming Normality Parameter Estimate 95% Confidence Limits Mean -0.0117 -0.0245 0.0011 Std deviation 0.0886 0.0804 0.0986 Variance 0.0079 0.0065 0.0097 Table D.39.c. Student’s t-Test of Zero Residual Mean Test Statistic p-Value Student’s t-test -1.8041 0.0728 Figure D.107. Residual normality plot—percentage of on-time trips over 50 mph—California.

206 Minnesota Figure D.108. Residual plot—percentage of on-time trips over 50 mph—Minnesota. Figure D.109. Residual histogram—percentage of on-time trips over 50 mph—Minnesota. Figure D.110. Residual normality plot—percentage of on-time trips over 50 mph—Minnesota. Table D.40. Residual Analysis— Percentage of On-Time Trips Over 50 mph—Minnesota Table D.40.a. Basic Summary Location Variability Mean -0.0353 Std deviation 0.0510 Median -0.0221 Variance 0.0026 Min -0.2230 Range 0.3729 Max 0.1499 Interquartile range 0.0526 Table D.40.b. Estimated Confidence Limits Assuming Normality Parameter Estimate 95% Confidence Limits Mean -0.0353 -0.0467 -0.0239 Std deviation 0.0510 0.0441 0.0604 Variance 0.0026 0.0019 0.0037 Table D.40.c. Student’s t-Test of Zero Residual Mean Test Statistic p-Value Student’s t-test -6.1620 <0.0001

207 Salt Lake City Figure D.111. Residual plot—percentage of on-time trips over 50 mph—Salt Lake City. Figure D.112. Residual histogram—percentage of on-time trips over 50 mph—Salt Lake City. Table D.41. Residual Analysis— Percentage of On-Time Trips Over 50 mph—Salt Lake City Table D.41.a. Basic Summary Location Variability Mean -0.0318 Std deviation 0.0457 Median -0.0066 Variance 0.0021 Min -0.1350 Range 0.1927 Max 0.0577 Interquartile range 0.0568 Table D.41.b. Estimated Confidence Limits Assuming Normality Parameter Estimate 95% Confidence Limits Mean -0.0318 -0.0455 -0.0181 Std deviation 0.0457 0.0378 0.0577 Variance 0.0021 0.0014 0.0033 Table D.41.c. Student’s t-Test of Zero Residual Mean Test Statistic p-Value Student’s t-test -4.6715 <0.0001 Figure D.113. Residual normality plot—percentage of on-time trips over 50 mph—Salt Lake City.

208 Spokane Figure D.114. Residual plot—percentage of on-time trips over 50 mph—Spokane. Figure D.115. Residual histogram—percentage of on-time trips over 50 mph—Spokane. Table D.42. Residual Analysis— Percentage of On-Time Trips Over 50 mph—Spokane Table D.42.a. Basic Summary Location Variability Mean -0.0601 Std deviation 0.0417 Median -0.0570 Variance 0.0017 Min -0.1295 Range 0.1266 Max -0.0029 Interquartile range 0.0728 Table D.42.b. Estimated Confidence Limits Assuming Normality Parameter Estimate 95% Confidence Limits Mean -0.0601 -0.0866 -0.0336 Std deviation 0.0417 0.0295 0.0708 Variance 0.0017 0.0009 0.0050 Table D.42.c. Student’s t-Test of Zero Residual Mean Test Statistic p-Value Student’s t-test -4.9958 0.0004 Figure D.116. Residual normality plot—percentage of on-time trips over 50 mph—Spokane.

209 Percentage of On-Time Trips with Over 45 mph Mean Speed California Figure D.117. Residual plot—percentage of on-time trips over 45 mph—California. Figure D.118. Residual histogram—percentage of on-time trips over 45 mph—California. Table D.43. Residual Analysis—Percentage of On-Time Trips Over 45 mph—California Table D.43.a. Basic Summary Location Variability Mean -0.01220 Std deviation 0.06721 Median -0.01415 Variance 0.00452 Min -0.1876805 Range 0.44630 Max 0.2586168 Interquartile range 0.04294 Table D.43.b. Estimated Confidence Limits Assuming Normality Parameter Estimate 95% Confidence Limits Mean -0.0122 -0.0219 -0.0025 Std deviation 0.0672 0.0610 0.0748 Variance 0.0045 0.0037 0.0056 Table D.43.c. Student’s t-Test of Zero Residual Mean Test Statistic p-Value Student’s t-test -2.4815 0.0140 Figure D.119. Residual normality plot—percentage of on-time trips over 45 mph—California.

210 Minnesota Figure D.120. Residual plot—percentage of on-time trips over 45 mph—Minnesota. Figure D.121. Residual histogram—percentage of on-time trips over 45 mph—Minnesota. Figure D.122. Residual normality plot—percentage of on-time trips over 45 mph—Minnesota. Table D.44. Residual Analysis - Percentage of On-Time Trips Over 45 mph—Minnesota Table D.44.a. Basic Summary Location Variability Mean -0.0306 Std deviation 0.0372 Median -0.0208 Variance 0.0014 Min -0.1719 Range 0.2103 Max 0.0383 Interquartile range 0.0400 Table D.44.b. Estimated Confidence Limits Assuming Normality Parameter Estimate 95% Confidence Limits Mean -0.0306 -0.0389 -0.0222 Std deviation 0.0372 0.0322 0.0441 Variance 0.0014 0.0010 0.0019 Table D.44.c. Student’s t-Test of Zero Residual Mean Test Statistic p-Value Student’s t-test -7.3038 <0.0001

211 Salt Lake City Figure D.123. Residual plot—percentage of on-time trips over 45 mph—Salt Lake City. Figure D.124. Residual histogram—percentage of on-time trips over 45 mph—Salt Lake City. Table D.45. Residual Analysis— Percentage of On-Time Trips Over 45 mph—Salt Lake City Table D.45.a. Basic Summary Location Variability Mean -0.0238 Std deviation 0.0366 Median -0.0043 Variance 0.0013 Min -0.1053 Range 0.1695 Max 0.0642 Interquartile range 0.0425 Table D.45.b. Estimated Confidence Limits Assuming Normality Parameter Estimate 95% Confidence Limits Mean -0.0238 -0.0348 -0.0128 Std deviation 0.0366 0.0303 0.0462 Variance 0.0013 0.0009 0.0021 Table D.45.c. Student’s t-Test of Zero Residual Mean Test Statistic p-Value Student’s t-test -4.3584 <0.0001 Figure D.125. Residual normality plot—percentage of on-time trips over 45 mph—Salt Lake City.

212 Spokane Figure D.126. Residual plot—percentage of on-time trips over 45 mph—Spokane. Figure D.127. Residual histogram—percentage of on-time trips over 45 mph—Spokane. Table D.46. Residual Analysis— Percentage of On-Time Trips Over 45 mph—Spokane Table D.46.a. Basic Summary Location Variability Mean -0.0459 Std deviation 0.0322 Median -0.0458 Variance 0.0010 Min -0.0973 Range 0.0953 Max -0.0020 Interquartile range 0.0594 Table D.46.b. Estimated Confidence Limits Assuming Normality Parameter Estimate 95% Confidence Limits Mean -0.04591 -0.0664 -0.0254 Std deviation 0.0322 0.0228 0.0547 Variance 0.0010 0.0005 0.0030 Table D.46.c. Student’s t-Test of Zero Residual Mean Test Statistic p-Value Student’s t-test -4.9369 0.0004 Figure D.128. Residual normality plot—percentage of on-time trips over 45 mph—Spokane.

213 Percentage of On-Time Trips with Over 30 mph Mean Speed California Figure D.129. Residual plot—percentage of on-time trips over 30 mph—California. Figure D.130. Residual histogram—percentage of on-time trips over 30 mph—California. Figure D.131. Residual normality plot—percentage of on-time trips over 30 mph—California. Table D.47. Residual Analysis— Percentage of On-Time Trips Over 30 mph—California Table D.47.a. Basic Summary Location Variability Mean -0.0009 Std deviation 0.0247 Median -0.0055 Variance 0.0006 Min -0.1009 Range 0.2143 Max 0.1134 Interquartile range 0.0037 Table D.47.b. Estimated Confidence Limits Assuming Normality Parameter Estimate 95% Confidence Limits Mean -0.0009 -0.0045 0.0026 Std deviation 0.0247 0.0224 0.0275 Variance 0.0006 0.0005 0.0008 Table D.47.c. Student’s t-Test of Zero Residual Mean Test Statistic p-Value Student’s t-test -0.5185 0.6048

214 Minnesota Figure D.132. Residual plot—percentage of on-time trips over 30 mph—Minnesota. Figure D.133. Residual histogram—percentage of on-time trips over 30 mph—Minnesota. Table D.48. Residual Analysis— Percentage of On-Time Trips Over 30 mph—Minnesota Table D.48.a. Basic Summary Location Variability Mean 0.0021 Std deviation 0.0331 Median -0.0056 Variance 0.0011 Min -0.1103 Range 0.2336 Max 0.1233 Interquartile range 0.0016 Table D.48.b. Estimated Confidence Limits Assuming Normality Parameter Estimate 95% Confidence Limits Mean 0.0021 -0.0053 0.0095 Std deviation 0.0331 0.0286 0.0392 Variance 0.0011 0.0008 0.0015 Table D.48.c. Student’s t-Test of Zero Residual Mean Test Statistic p-Value Student’s t-test 0.5652 0.5736 Figure D.134. Residual normality plot—percentage of on-time trips over 30 mph—Minnesota.

215 Salt Lake City Figure D.135. Residual plot—percentage of on-time trips over 30 mph—Salt Lake City. Figure D.136. Residual histogram—percentage of on-time trips over 30 mph—Salt Lake City. Figure D.137. Residual normality plot—percentage of on-time trips over 30 mph—Salt Lake City. Table D.49. Residual Analysis— Percentage of On-Time Trips Over 30 mph—Salt Lake City Table D.49.a. Basic Summary Location Variability Mean -0.0026 Std deviation 0.0133 Median -0.0054 Variance 0.0002 Min -0.0092 Range 0.0671 Max 0.0579 Interquartile range 0.0009 Table D.49.b. Estimated Confidence Limits Assuming Normality Parameter Estimate 95% Confidence Limits Mean -0.0026 -0.0065 0.0014 Std deviation 0.0133 0.0110 0.0168 Variance 0.0002 0.0001 0.0003 Table D.49.c. Student’s t-Test of Zero Residual Mean Test Statistic p-Value Student’s t-test -1.2873 0.2047

216 Spokane Figure D.138. Residual plot—percentage of on-time trips over 30 mph—Spokane. Figure D.139. Residual histogram—percentage of on-time trips over 30 mph—Spokane. Figure D.140. Residual normality plot—percentage of on-time trips over 30 mph—Spokane. Table D.50.c. Student’s t-Test of Zero Residual Mean Test Statistic p-Value Student’s t-test -6.4287 <0.0001 Table D.50. Residual Analysis— Percentage of On-Time Trips Over 30 mph—Spokane Table D.50.a. Basic Summary Location Variability Mean -0.0060 Std deviation 0.0032 Median -0.0062 Variance 0.0000 Min -0.0095 Range 0.0119 Max 0.0024 Interquartile range 0.0034 Table D.50.b. Estimated Confidence Limits Assuming Normality Parameter Estimate 95% Confidence Limits Mean -0.0060 -0.0080 -0.0039 Std deviation 0.0032 0.0023 0.0055 Variance 0.0000 0.0000 0.0000