The National Academies Press

Currently Skimming:

Pages 65-166

The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.

From page 65... ... 65 Overview This appendix presents the validation analysis of the L03 datarich models. Pages 143–145 of the L03 report contain six model equations that predict the following travel time index (TTI) Read the entire page →
From page 66... ... 66 Midday Models 1. mean TTI 0.2599 dccrite= ( ) Read the entire page →
From page 67... ... 67 Performance Measures Root Mean SquaRe eRRoR Because the data-rich models are in the exponential form, the prediction error is defined differently here than it was for the data poor validation. The research team assumes that the data-rich models are produced by first taking the logarithm transformation of the response data and then fitting it with a linear regression model. Read the entire page →
From page 68... ... 68 data-Rich Model Validation The data-rich models are categorized as peak period, peak hour, midday, and weekday models. For each category, there are six models for the different travel time reliability measures: mean TTI, 99th-percentile TTI, 95th-percentile TTI, 80th-percentile TTI, 50th-percentile TTI, and 10th-percentile TTI. Read the entire page →
From page 69... ... 69 the models tend to underestimate the response variable as most residuals are under the zero reference line. These nonrandom patterns show that the model fits the validation data unsatisfactorily. Read the entire page →
From page 70... ... 70 Figure C.4. Residual plot of peak period -- 99th-percentile TTI -- AllData. Read the entire page →
From page 71... ... 71 values does not indicate a good fit, evident from the increasing trend of the scattered points. The Student's t-test shows that the null hypothesis of zero residual mean can be rejected with a confidence level of 90%, while the Shapiro-Wilk normality test shows that null hypothesis of normal distribution cannot be rejected with 90% confidence level but can be rejected at a confidence level of 85%. Read the entire page →
From page 72... ... 72 The residual analysis results (Table C.9) show that the zero residual mean hypothesis can be rejected in the Student's t-test with a confidence level of 90%. Read the entire page →
From page 73... ... 73 confidence level of 90%. All the associated results and plots for MN region are included in the attachment. Read the entire page →
From page 74... ... 74 Figure C.15. Residual normality plot of peak period -- 50th-percentile TTI -- AllData. Read the entire page →
From page 75... ... 75 versus the predicted values still exists, and the histogram and the normality plot demonstrates the normality test results that the residuals are not closely following a normal distribution. All the associated results and plots for the MN data are included in the attachment. Read the entire page →
From page 76... ... 76 that the residuals follow a normal distribution cannot be rejected in the Shapiro-Wilk normality test as the p-value is 0.6593. The residual plot shows that the residuals are all positive except for one. Read the entire page →
From page 77... ... 77 shows that the null hypothesis of normal distribution cannot be rejected with the threshold confidence level of 90%. The Student's t-test rejects the null hypothesis of zero residual mean with a high confidence level, which corresponds to the fact that the 95% confidence limits for the mean of residual both fall on the negative side. Read the entire page →
From page 78... ... 78 peak houR -- Mean tti -- MinneSota The validation of the peak hour mean TTI model using the MN data set shows that the null hypothesis of the zero residual mean cannot be rejected with a confidence level of 90%. The normality test also shows that the null hypothesis cannot be rejected with the preset threshold confidence level but can be rejected with a confidence level of 85%. Read the entire page →
From page 79... ... 79 peak houR -- 99th-peRcentile tti -- califoRnia Validation using the CA data presents similar results to that using the AllData set. Again, the model performs best when the predicted ln(99th-percentile TTI) Read the entire page →
From page 80... ... 80 peak houR -- 95th-peRcentile tti -- MinneSota The validation results using the MN data set show that the research team cannot reject the zero residual mean assumption or the normal distribution of residual assumption with a confidence level of 90%, as the p-values in the Student's t-test and the Shapiro-Wilk test are both larger than 0.1. The histogram and the normality plot manifest these hypothesistesting results. Read the entire page →
From page 81... ... 81 Peak Hour -- 80th-Percentile TTI peak houR -- 80th-peRcentile tti -- all data The RMSE values for the validation of the 80th-percentile TTI model using the AllData, CA, and MN data sets are close to one another, at around 35% (Table C.20) Read the entire page →
From page 82... ... 82 the normal distribution of residual assumption. The residual versus predicted value plot shows that the model tends to overestimate when the predicted value is large, and that there are more negative residuals than positive ones. Read the entire page →
From page 83... ... 83 side of the zero reference line, which corresponds to the 95% limits for residual mean shown in Table C.23. peak houR -- 50th-peRcentile tti -- califoRnia The validation using the CA data set yields similar results to that using the AllData set: the Student's t-test rejects the null hypothesis of zero residual mean while the Shapiro-Wilk normality test cannot reject the null hypothesis of normal distribution. Read the entire page →
From page 84... ... 84 from the CA data set. The RMSE for this model in the L03 report is 15.2%, which is larger than the MN RMSE but smaller than the other two. Read the entire page →
From page 85... ... 85 peak houR -- 10th-peRcentile tti -- MinneSota The validation using the MN data set shows that the zero residual mean and the normal distribution of residuals assumptions are likely violated as the Student's t-test and the Shapiro-Wilk normality test both reject their null hypotheses. The plot of residuals versus predicted values shows that most of the residuals are positive, and there is one potential outlier located at the bottom right of the figure. Read the entire page →
From page 86... ... 86 the Student's t-test with a confidence level of 90%. The null hypothesis of normal distribution can also be rejected with the threshold confidence level in the normality test. Read the entire page →
From page 87... ... 87 a confidence level of 90%. The histogram and the normality plot show the deviation of the residual distribution from a normal distribution. Read the entire page →
From page 88... ... 88 to increase with the increase in predicted values. All these observations imply that the predictive capability of the model may be insufficient. Read the entire page →
From page 89... ... 89 normal distribution. The plot of residual versus the predicted value (Figure C.44) Read the entire page →
From page 90... ... 90 t-test and the normality test. The residual plot shows that the model tends to overestimate the response variable. Read the entire page →
From page 91... ... 91 shows similar problems noted in the validation results of the AllData set (i.e., more positive residuals and potential outliers) Read the entire page →
From page 92... ... 92 training data set used in L03 is similar to the validation data sets, then the RMSE value in the L03 report should not have been this large, which indicates that it could be erroneous. For the validation data sets the research team can see that the largest value still comes from the CA data set while the smallest ones come from the MN and the Salt Lake City sets. Read the entire page →
From page 93... ... 93 perform satisfactorily. All associated results and plots are included in the attachment. Read the entire page →
From page 94... ... 94 Midday -- 10th-peRcentile tti -- califoRnia The validation of the midday 10th-percentile TTI model using the CA data set presents similar results to that using the AllData set, except that zero residual mean hypothesis has passed the Student's t-test. However, this success of passing the hypothesis test could be attributed to the existence of large negative residuals, which can be seen from the residuals versus the predicted values plot. Read the entire page →
From page 95... ... 95 problematic issue is the nonrandom patterns shown in the residual plots, which indicate that the model is not adequate. It may be mentioned again that the comparison of the L03 RMSE values and the current RMSE values is based on the assumption that they are defined in the same way. Read the entire page →
From page 96... ... 96 Weekday -- Mean tti -- califoRnia In the validation of the weekday mean TTI model using the CA data set, the null hypothesis of zero residual mean can be rejected with a confidence level of 90%, and the null hypothesis of normal distribution can be rejected with the same threshold confidence level. The plot of residuals versus the predicted values shows problematic patterns, with the residuals increasing as the predicted values increase and that the model tending to overestimate the response variable. Read the entire page →
From page 97... ... 97 Weekday -- 99th-peRcentile tti -- califoRnia In this validation both the 95% confidence limits for the zero residual mean are positive, which accords with the Student's t-test results. The normality test also rejects the null hypothesis with a confidence level of 90%. Read the entire page →
From page 98... ... 98 Weekday -- 99th-peRcentile tti -- MinneSota In this validation the zero residual mean assumption and the normal distribution assumption seem to be violated as the Student's t-test and the Shapiro-Wilk normality test reject their null hypotheses. The residual versus the predicted value plot renders unusual patterns, with the residuals increasing with the predicted values. Read the entire page →
From page 99... ... 99 residuals versus the predicted values shows that the variation of residuals is relatively large given the scale of the predicted value. The increasing tendency and the fact that there are more positive residuals than negative ones indicate that the model may be biased and unreliable. Read the entire page →
From page 100... ... 100 Weekday -- 80th-peRcentile tti -- califoRnia In this validation the Student's t-test and the normality test reject their null hypotheses, indicating violation of the zero residual mean assumption and the normal distribution assumption. The plot of residuals versus predicted values reveals a nonconstant residual variance problem; additionally, the residuals are unbalanced with more positive residuals than negative. Read the entire page →
From page 101... ... 101 and the normal distribution null hypotheses are rejected. The plot of residuals versus the predicted values shows an obvious increasing trend with mostly positive residuals. Read the entire page →
From page 102... ... 102 test rejects the normal distribution assumption. The residual normality plot is shown in Figure C.69. Read the entire page →
From page 103... ... 103 residual histogram and normality plots are shown in Figures C.71 and C.72. Weekday -- 10th-peRcentile tti -- califoRnia In this validation the Student's t-test rejects the null hypothesis of zero residual mean with a confidence level of 90%, and the normality test rejects the null hypothesis of normal distribution with the same threshold confidence level. Read the entire page →
From page 104... ... 104 Weekday -- 10th-peRcentile tti -- MinneSota In this validation the null hypothesis of zero residual mean and the null hypothesis of normal distribution are rejected with a confidence level of 90%. The plot of residuals versus the predicted values shows an almost linearly increasing trend with the exception of three samples located in the lower right corner of the plot. Read the entire page →
From page 105... ... 105 Table C.50. Summary of RMSE Values for Each Model by Region Model Details RMSE Value by Region Analysis Time Slice Model All Data CA MN Salt Lake City Peak period Mean TTI 96.94% 127.55% 21.59% 99th Percentile 403.44% 607.76% 63.67% 95th Percentile 251.95% 359.19% 45.85% 80th Percentile 151.95% 206.54% 30.95% 50th Percentile 89.55% 116.63% 23.15% 10th Percentile 12.13% 14.43% 6.23% Peak hour Mean TTI 25.45% 26.97% 24.68% 99th Percentile 50.74% 52.78% 47.46% 95th Percentile 38.38% 40.19% 37.27% 80th Percentile 35.13% 36.89% 34.06% 50th Percentile 28.85% 32.41% 24.22% 10th Percentile 18.50% 22.24% 12.14% Midday Mean TTI 6.24% 7.57% 4.07% 3.52% 99th Percentile 32.32% 34.95% 25.86% 34.01% 95th Percentile 15.62% 17.29% 14.01% 12.55% 80th Percentile 8.99% 10.86% 6.61% 3.60% 50th Percentile 5.43% 6.93% 2.09% 2.08% 10th Percentile 1.81% 2.20% 0.80% 1.33% Weekday Mean TTI 19.74% 12.81% 35.99% 5.95% 99th Percentile 72.91% 50.04% 141.72% 30.87% 95th Percentile 83.82% 40.46% 197.82% 22.85% 80th Percentile 29.28% 14.84% 59.43% 5.75% 50th Percentile 4.68% 5.92% 1.71% 2.16% 10th Percentile 0.81% 0.74% 0.48% 1.30% the null hypothesis of the normality test was rejected for all models, indicating that the residuals are nonnormally distributed. Read the entire page →
From page 106... ... 106 Table C.51. Summary of Student's t-Test and Shapiro-Wilk Normality Test Results for AllData Set Model Details Null Hypothesis Result Analysis Time Slice Model t-Testa Normality Testb Peak period Mean TTI Reject Reject 99th Percentile Reject Reject 95th Percentile Reject Reject 80th Percentile Reject Reject 50th Percentile Reject Reject 10th Percentile Reject Reject Peak hour Mean TTI Cannot Reject Cannot Reject 99th Percentile Cannot Reject Reject 95th Percentile Cannot Reject Cannot Reject 80th Percentile Reject Cannot Reject 50th Percentile Reject Cannot Reject 10th Percentile Cannot Reject Reject Midday Mean TTI Reject Reject 99th Percentile Reject Reject 95th Percentile Reject Reject 80th Percentile Reject Reject 50th Percentile Cannot Reject Reject 10th Percentile Reject Reject Weekday Mean TTI Reject Reject 99th Percentile Reject Reject 95th Percentile Reject Reject 80th Percentile Reject Reject 50th Percentile Cannot Reject Reject 10th Percentile Reject Reject a t-test results indicate whether the null hypothesis assumption that the residuals satisfy zero residual mean can be rejected or not with a certain confidence level. Read the entire page →
From page 107... ... 107 Peak Period Mean TTI Model Peak Period -- Mean TTI Model -- California Appendix C Attachment Table C.52. Residual Analysis of Peak Period -- Mean TTI -- California Table C.52.a. Read the entire page →
From page 108... ... 108 MinneSota Figure C.75. Residual normality plot of peak period -- mean TTI -- California. Read the entire page →
From page 109... ... 109 99th-Percentile TTI Model California Table C.54. Residual Analysis of Peak Period -- 99th-Percentile TTI -- California Table C.54.a. Read the entire page →
From page 110... ... 110 Minnesota Table C.55. Residual Analysis of Peak Period -- 99th-Percentile TTI -- Minnesota Table C.55.a. Read the entire page →
From page 111... ... 111 95th-Percentile TTI Model California Table C.56. Residual Analysis of Peak Period -- 95th-Percentile TTI -- California Table C.56.a. Read the entire page →
From page 112... ... 112 Minnesota Table C.57. Residual Analysis of Peak Period -- 95th-Percentile TTI -- Minnesota Table C.57.a. Read the entire page →
From page 113... ... 113 80th-Percentile TTI Model California Table C.58. Residual Analysis of Peak Period -- 80th-Percentile TTI -- California Table C.58.a. Read the entire page →
From page 114... ... 114 Minnesota Table C.59. Residual Analysis of Peak Period -- 80th-Percentile TTI -- Minnesota Table C.59.a. Read the entire page →
From page 115... ... 115 50th-Percentile TTI Model California Table C.60. Residual Analysis of Peak Period -- 50th-Percentile TTI -- California Table C.60.a. Read the entire page →
From page 116... ... 116 Minnesota Table C.61. Residual Analysis of Peak Period -- 50th-Percentile TTI -- Minnesota Table C.61.a. Read the entire page →
From page 117... ... 117 10th-Percentile TTI Model California Table C.62. Residual Analysis of Peak Period -- 10th-Percentile TTI -- California Table C.62.a. Read the entire page →
From page 118... ... 118 Minnesota Table C.63. Residual Analysis of Peak Period -- 10th-Percentile TTI -- Minnesota Table C.63.a. Read the entire page →
From page 119... ... 119 Peak Hour Mean TTI Model California Table C.64. Residual Analysis of Peak Hour -- Mean TTI -- California Table C.64.a. Read the entire page →
From page 120... ... 120 Minnesota Table C.65.d. Tests for Normality Test Statistic p-Value Shapiro-Wilk W 0.9361 Pr < W 0.1203 Table C.65.c. Read the entire page →
From page 121... ... 121 99th-Percentile TTI Model California Table C.66. Residual Analysis of Peak Hour -- 99th-Percentile TTI -- California Table C.66.d. Read the entire page →
From page 122... ... 122 Minnesota Table C.67. Residual Analysis of Peak Hour -- 99th-Percentile TTI -- Minnesota Table C.67.a. Read the entire page →
From page 123... ... 123 95th-Percentile TTI Model California Table C.68.c. Tests for Location: Mu=0 Test Statistic p-Value Student's t t -0.877 Pr > t 0.3854 Table C.68.b. Read the entire page →
From page 124... ... 124 Minnesota Table C.69. Residual Analysis of Peak Hour -- 95th-Percentile TTI -- Minnesota Table C.69.a. Read the entire page →
From page 125... ... 125 80th-Percentile TTI Model California Table C.70.b. Basic Confidence Limits Assuming Normality Parameter Estimate 95% Confidence Limits Mean -0.127 -0.216 -0.037 Std deviation 0.2908 0.2398 0.3696 Variance 0.0846 0.0575 0.1366 Table C.70.c. Read the entire page →
From page 126... ... 126 Minnesota Table C.71. Residual Analysis of Peak Hour -- 80th-Percentile TTI -- Minnesota Table C.71.a. Read the entire page →
From page 127... ... 127 50th-Percentile TTI Model California Table C.72.d. Tests for Normality Test Statistic p-Value Shapiro-Wilk W 0.9844 Pr < W 0.8187 Table C.72.c. Read the entire page →
From page 128... ... 128 Minnesota Table C.73.b. Basic Confidence Limits Assuming Normality Parameter Estimate 95% Confidence Limits Mean 0.0219 -0.069 0.1128 Std deviation 0.2202 0.1719 0.3063 Variance 0.0485 0.0296 0.0938 Table C.73.c. Read the entire page →
From page 129... ... 129 10th-Percentile TTI Model California Table C.74.d. Tests for Normality Test Statistic p-Value Shapiro-Wilk W 0.8385 Pr < W <0.0001 Table C.74.c. Read the entire page →
From page 130... ... 130 Minnesota Table C.75. Residual Analysis of Peak Hour -- 10th-Percentile TTI -- Minnesota Table C.75.b. Read the entire page →
From page 131... ... 131 Midday Mean TTI Model California Table C.76.b. Basic Confidence Limits Assuming Normality Parameter Estimate 95% Confidence Limits Mean 0.0164 0.0045 0.0283 Std deviation 0.0714 0.0639 0.0809 Variance 0.0051 0.0041 0.0065 Table C.76.d. Read the entire page →
From page 132... ... 132 Minnesota Table C.77. Residual Analysis of Midday -- Mean TTI -- Minnesota Table C.77.a. Read the entire page →
From page 133... ... 133 Salt Lake City Table C.78. Residual Analysis of Midday -- Mean TTI -- Salt Lake City Table C.78.a. Read the entire page →
From page 134... ... 134 99th-Percentile TTI Model California Table C.79. Residual Analysis of Midday -- 99th-Percentile TTI -- California Table C.79.a. Read the entire page →
From page 135... ... 135 Minnesota Table C.80.b. Basic Confidence Limits Assuming Normality Parameter Estimate 95% Confidence Limits Mean 0.1419 0.0948 0.1891 Std deviation 0.1825 0.1547 0.2225 Variance 0.0333 0.0239 0.0495 Table C.80.c. Read the entire page →
From page 136... ... 136 Salt Lake City Table C.81. Residual Analysis of Midday -- 99th-Percentile TTI -- Salt Lake City Table C.81.a. Read the entire page →
From page 137... ... 137 95th-Percentile TTI Model California Table C.82. Residual Analysis of Midday -- 95th-Percentile TTI -- California Table C.82.a. Read the entire page →
From page 138... ... 138 Minnesota Table C.83. Residual Analysis of Midday -- 95th-Percentile TTI -- Minnesota Table C.83.a. Read the entire page →
From page 139... ... 139 Salt Lake City Table C.84. Residual Analysis of Midday -- 95th-Percentile TTI -- Salt Lake City Table C.84.a. Read the entire page →
From page 140... ... 140 80th-Percentile TTI Model California Table C.85. Residual Analysis of Midday -- 80th-Percentile TTI -- California Table C.85.a. Read the entire page →
From page 141... ... 141 Minnesota Table C.86. Residual Analysis of Midday -- 80th-Percentile TTI -- Minnesota Table C.86.a. Read the entire page →
From page 142... ... 142 Salt Lake City Figure C.178. Residual plot of midday -- 80th-percentile TTI -- Salt Lake City. Read the entire page →
From page 143... ... 143 50th-Percentile TTI Model California Figure C.181. Residual plot of midday -- 50th-percentile TTI -- California. Read the entire page →
From page 144... ... 144 Minnesota Table C.89. Residual Analysis of Midday -- 50th-Percentile TTI -- Minnesota Table C.89.a. Read the entire page →
From page 145... ... 145 Salt Lake City Table C.90. Residual Analysis of Midday -- 50th-Percentile TTI -- Salt Lake City Table C.90.a. Read the entire page →
From page 146... ... 146 10th-Percentile TTI Model California Table C.91. Residual Analysis of Midday -- 10th-Percentile TTI -- California Table C.91.a. Read the entire page →
From page 147... ... 147 Minnesota Table C.92. Residual Analysis of Midday -- 10th-Percentile TTI -- Minnesota Table C.92.a. Read the entire page →
From page 148... ... 148 Salt Lake City Table C.93. Residual Analysis of Midday -- 10th-Percentile TTI -- Salt Lake City Table C.93.a. Read the entire page →
From page 149... ... 149 Weekday Mean TTI Model California Table C.94. Residual Analysis of Weekday -- Mean TTI -- California Table C.94.a. Read the entire page →
From page 150... ... 150 Minnesota Table C.95. Residual Analysis of Weekday -- Mean TTI -- Minnesota Table C.95.a. Read the entire page →
From page 151... ... 151 Salt Lake City Table C.96. Residual Analysis of Weekday -- Mean TTI -- Salt Lake City Table C.96.a. Read the entire page →
From page 152... ... 152 99th-Percentile TTI Model California Table C.97. Residual Analysis of Weekday -- 99th-Percentile TTI -- California Table C.97.a. Read the entire page →
From page 153... ... 153 Minnesota Table C.98. Residual Analysis of Weekday -- 99th-Percentile TTI -- Minnesota Table C.98.a. Read the entire page →
From page 154... ... 154 Salt Lake City Table C.99. Residual Analysis of Weekday -- 99th-Percentile TTI -- Salt Lake City Table C.99.a. Read the entire page →
From page 155... ... 155 95th-Percentile TTI Model California Table C.100. Residual Analysis of Weekday -- 95th-Percentile TTI -- California Table C.100.a. Read the entire page →
From page 156... ... 156 Minnesota Table C.101. Residual Analysis of Weekday -- 95th-Percentile TTI -- Minnesota Table C.101.a. Read the entire page →
From page 157... ... 157 Salt Lake City Table C.102. Residual Analysis of Weekday -- 95th-Percentile TTI -- Salt Lake City Table C.102.a. Read the entire page →
From page 158... ... 158 80th-Percentile TTI Model California Table C.103. Residual Analysis of Weekday -- 80th-Percentile TTI -- California Table C.103.a. Read the entire page →
From page 159... ... 159 Minnesota Table C.104. Residual Analysis of Weekday -- 80th-Percentile TTI -- Minnesota Table C.104.a. Read the entire page →
From page 160... ... 160 Salt Lake City Table C.105. Residual Analysis of Weekday -- 80th-Percentile TTI -- Salt Lake City Table C.105.a. Read the entire page →
From page 161... ... 161 50th-Percentile TTI Model California Table C.106. Residual Analysis of Weekday -- 50th-Percentile TTI -- California Table C.106.a. Read the entire page →
From page 162... ... 162 Minnesota Table C.107. Residual Analysis of Weekday -- 50th-Percentile TTI -- Minnesota Table C.107.a. Read the entire page →
From page 163... ... 163 Salt Lake City Table C.108. Residual Analysis of Weekday -- 50th-Percentile TTI -- Salt Lake City Table C.108.a. Read the entire page →
From page 164... ... 164 10th-Percentile TTI Model California Table C.109. Residual Analysis of Weekday -- 10th-Percentile TTI -- California Table C.109.a. Read the entire page →
From page 165... ... 165 Minnesota Table C.110. Residual Analysis of Weekday -- 10th-Percentile TTI -- Minnesota Table C.110.a. Read the entire page →
From page 166... ... 166 Salt Lake City Table C.111.d. Tests for Normality Test Statistic p-Value Shapiro-Wilk W 0.7251 Pr < W <0.0001 Table C.111. Read the entire page →

From page 65...

... 65 Overview This appendix presents the validation analysis of the L03 datarich models. Pages 143–145 of the L03 report contain six model equations that predict the following travel time index (TTI)