Principles and Guidance for Presenting Active Traffic Management Information to Drivers (2021)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

48 Chapter 5: Empirical Studies This chapter describes the methods and finding from three research studies that were conducted to identify the specific research gap identified in earlier project activities. The studies consisted of two driving simulator experiments to examine some of the research gaps involving human factors issues about ATM message dissemination. The studies also included an outreach activity to identify current and best practices used by agencies to deploy and evaluate various ATM strategies. The goal of Experiment 1 was to investigate the effects of availability of ATM information on driver behavior and distraction. The two levels of information availability included real-time presentation versus discrete presentation for “in-vehicle” ATM applications. Experiment 2 focused on the “information modality” of in-vehicle ATM displays and its effects on drivers’ behavior. Furthermore, Experiment 2 examined the effect of information type, communicated by the dynamic speed limits. The objective of Experiment 3 was to identify current and best practices used by agencies to effectively deploy and evaluate the potential and realized benefits of various ATM strategies, as well as guidance available to support a transition to innovative, non-traditional media for presenting dynamic information. To make the most efficient use of project resources, multiple research gaps were addressed in the studies. Furthermore, each study incorporated “add-on” data collection in the form of surveys or supplementary research questions to obtain the most actionable information from participants. Table 13 shows which studies and study components were used to address each of the key research questions. Table 13: Overview of the three experiments and research gaps. Research Gap Studies Methods 1 Experiment 1 Driving simulator (add-on) 2 Experiment 1 Driving simulator (add-on) 3 Experiment 1 Driving simulator (main) 4 Experiment 1 and 2 Driving simulator (main) 5 Experiment 2 Sign comprehension test 6 Experiment 2 Sign comprehension test 717 Excluded N/A 8 Experiment 3 Interview/focus group 9 Experiment 3 Interview/focus group 10 Experiment 1 Pre-experiment survey 11 Experiment 2 Driving simulator (main) 12 Experiment 1 Driving simulator (main) 1318 Excluded N/A 17, 17 Questions were excluded because they received low ratings for relevance and usefulness in the research gap analysis. See Appendix C of the interim report for a full explanation of the screening procedure used for research gaps.

49 General Approach to Experiments 1 and 2 The two experiments conducted to examine human factors issues regarding ATM message dissemination involved data collection in a driving simulator. In both simulator studies, participants drove for about 8 miles in a pre-designed straight four-lane highway. The routes were fixed, and driving directions were presented by cell phone navigation applications. Drivers encountered several overhead gantry signs (19 signs for Experiment 1 and 20 signs for Experiment 2) along the routes. The overhead gantries were placed 0.5 miles apart based on Washington State DOT (WSDOT) ITS Design Requirements (WSDOT, 2018) for both experiments. A time-based legibility distance at 60 mph was implemented based on WSDOT ITS Design Requirements (nine seconds of reading time which provides approximately 800 ft of tangent sight distance). Participants Participants were recruited from the greater Seattle area in the United States. In total, 86 participants’ data were analyzed for Experiments 1 and 2 (n of Experiment 1 = 44, and n of Experiment 2 = 42). This sample size is consistent for both driving behavior data and survey data. The eye-tracking data analysis had a smaller sample size (n of Experiment 1 = 21, and n of Experiment 2 = 37) after filtering low-quality data. Participants needed to hold a valid United States driver’s license, drive at least 75 miles per week and/or take at least 10 trips per week, be able to read English, and complete written questionnaires. Prospective participants were also screened for susceptibility to simulator sickness. Ethics approval was obtained from the Institutional Review Board at Battelle. Gender and age group were study blocking factors. Participants were recruited from three age groups: younger drivers (18-25 years), middle-aged drivers (30 – 60 years), and older drivers (65+ years). These categories were a modified version of the age groups used by Bao & Boyle, 2009, where 9-year age gaps were used to provide clearer separation between groups. We used 4-year age gaps to provide separation, while at the same time reducing restrictions on participant recruitment. Summary statistics for the participants information are presented below: • Experiment 1 (driving behavior and survey data) o n = 44 o average age = 42.61 (SD = 19.58) o n of male participants = 23 o n of female participants = 21 • Experiment 2 (driving behavior and survey data) o n = 42 o average age = 42.88 (SD = 20.95) o n of male participants = 21 o n of female participants = 21 • Experiment 1 (eye-tracking data) o n = 21

50 o average age = 45.33 (SD = 21.23) o n of male participants = 13 o n of female participants = 8 • Experiment 2 (eye-tracking data) o n = 37 o average age = 40.95 (SD = 20.09) o n of male participants = 19 o n of female participants = 18 Apparatus: Battelle MiniSim™ Simulator The Battelle Driving Simulator was used for data collection. It uses the NADS MiniSim™ platform and a Ford Merkur chassis with the drive train, engine compartment, trunk, and wheels removed serves as the vehicle cab. The simulator includes the following features: automatic transmission, variable vehicle dynamics, simulated road noise (engine and drive train), tire squeal, and three-dimensional rendering of displayed objects. A participant can operate the controls of the cab just as he or she would on the road, and the visual representation of the virtual roadway changes appropriately in response to drivers’ actions. The virtual world was displayed on three screens, one in front of the car and two on each side. The screens were 100 inches wide and 75 inches tall; however, the size of the projected images was 100 inches by 56.5 inches. All three screens together subtended a visual angle of 135 degrees horizontally. The resolution of each projection was 1920 x 1080 for a combined resolution of 5760 x 1080 (projected in mosaic mode). The images themselves were updated 60 times per second using a MiniSim server, which parallel processed the images projected to each of the three screens using two high-end NVIDIA Quadro K5000 multimedia video processors/graphic cards. The numerous vehicle parameters and driving performance measures were also recorded at 60 Hz. The sound system for the simulator consisted of three Logitech 2.1 Surround Sound speakers, two located on the left and right sides of the car and one, a sub- woofer, located in the back of the car. Driving metrics were stored in a data acquisition file (.daq file). The data acquisition files were processed using MATLAB and exported as .csv files for analysis. Apparatus: In-Vehicle ATM Device’s Interface and Operation The smartphone navigation/ATM In-Vehicle Information System (IVIS) consisted of a custom software application installed on an Android smart phone. The application presented information similar to that of a typical navigation system, but with additional ATM messages. The display consisted of a navigation panel with a representation of the road being driven, an information panel that presented ATM messages, and two sections that provided speed and trip information. Figure 44 shows the IVIS display with the driver passing through a complex interchange. The top panel of the IVIS display showed navigation instructions and ATM information, including lane closure, merge, and through lane. The center section showed the vehicle location centered on the moving map. The bottom sections included speed limit, vehicle speed, estimated time remaining in the trip, distance to end of trip, and estimated time of arrival. The vehicle speed was

51 color-coded to indicate travel speed at or below the speed limit in blue, exceeding the speed limit by 10 mph or less in orange, and exceeding the speed limit by more than 10 mph in red. The application read a stream of data sent from the Battelle Driving Simulator. Control messages were generated by triggers embedded within the driving scenario and passed to the ATM device in the data stream. The control messages determined what information was displayed on the IVIS, including vehicle speed, vehicle location relative to the simulated world, ATM message state, number of lane indicators, and other relevant variables as needed. The IVIS device parsed the control messages and displays the ATM messages and/or specified imagery (see Figure 43. This methodology ensured accurate timing and control of ATM messages displayed on the IVIS. The navigation display consisted of an image derived from a “birds-eye” view of the scenario world, presented in the navigation panel viewport. A coordinate system corresponding to that of scenario world was mapped to the image. The simulator sent the coordinates of the ownship vehicle (virtual vehicle driven by participant) to the application as it moves through the scenario, and the application continually redrew the map image with the location of the simulated vehicle centered in the navigation panel viewport. Accurate spatial and temporal synchronization of the driver location with the map image were achieved because the coordinates of the ownship vehicle were known precisely with respect to the geometry in each simulator video frame. Figure 43. The smartphone was located to the right of the steering wheel (Battelle).

52 Figure 44. A screen capture from the smartphone application (Battelle). Eye Tracker Participants’ glance behaviors were collected using the Ergoneers Dikablis eye tracker (Figure 45), which used the D-Lab sensor-fusion software to record binocular eye-glance information at 60-Hz and synchronize with data from the MiniSim™ variables and multi-channel video streams. Each individual participant’s area of interest (AOI)—in this study, the smartphone— was manually defined (see Figure 46), and algorithms to eliminate blinks and “fly-throughs” (i.e., very short sequence of glances at an AOI) were applied before calculating glances at the AOI.

53 Figure 45. Ergoneers head-mounted eye tracker. Figure 46. Manually defined AOI (blue box) and drivers’ fixation point (red crossed circle) captured from the D-Lab software. Experimental Design Experiments 1 and 2 were within-subject designs, which allowed each participant to experience all conditions and minimize potential effects of individual differences. Experiment 1 was designed to test the effect of ATM information’s “presentation timing” on drivers’ behavior and their experience. Experiment 2 was designed to test the effect of ATM information’s “modality” and “message types” on drivers’ behavior and experience. Each participant was fully informed about different experimental conditions (e.g., “just-in-time” vs. “always-on” modes along with

54 two baseline conditions for Experiment 119) before the main drive and also had a short practice drive with the smartphone application across all experimental conditions. Presentation orders of the experimental conditions were counterbalanced. Both Experiment 1 and Experiment 2 implemented two types of baseline conditions: “smartphone-only” and “gantry-only” conditions. The smartphone-only baseline represents a deployment case where the ATM information is only disseminated from in-vehicle displays or a third-party mobile application; thus, the overhead gantries do not present any information. The gantry-only baseline represents present-day ATM applications where the ATM information is disseminated from infrastructure-based media such as overhead gantries. All other experimental conditions represent a deployment stage where drivers would receive the ATM information from both infrastructure-based media and third-party applications (e.g., smartphone applications). Comparisons between the baseline and experimental conditions were used to investigate behavioral impacts of using a smartphone as an ATM medium and participants’ subjective preferences. Procedure The entire driving simulator experiment was held within a two-hour session. Participants completed two simulated drives (practice and main drive) and two surveys (pre-/post-experiment survey) throughout the session. In total, participants completed approximately 30-35 minutes of simulated driving. Each participant was provided with a description of the experiment and asked to give their informed consent. Before a practice drive, the participant was instructed to complete a pre- experiment questionnaire. After completion of the pre-experiment questionnaire, the participant was introduced to the vehicle and asked to drive a short practice drive to acclimate themselves with the steering and braking of the simulator and for simulator sickness prevention. Before the practice drive, participants were informed that they would receive lane closure information (speed limit information in Experiment 2) either from overhead signs or smartphone application. The location of the smartphone and components/features of the smartphone application were explained. Full descriptions of each experimental conditions were given to participants along with description of two baseline conditions. The descriptions of experimental conditions and baseline conditions were provided again just before beginning of the main drive. To mitigate and monitor for simulator sickness, participants were required to take breaks between the practice and main drives and complete the Simulator Sickness Questionnaire after the practice drive (SSQ; Kennedy et al., 1993). The participant drove through all experimental conditions with ATM signs from the smartphone and/or infrastructure-based ATM signs. After the drive, a post-experiment questionnaire was provided to the participant. Participants were compensated $60 for their participation. 19 Originally, the two experimental conditions were known as “real-time” and “discrete” in the work plan. In the data collection stage, “discrete” was changed to “just-in-time” and “real-time” was changed to “always on” in order to minimize participant confusion.

55 Experiment 1: Evaluating the Effects of Information Availability of Dynamic Lane Control on Driver Behavior and Distraction Overview The literature review found that there has been little to no research involving systematic comparisons of ATM media (including alternative media) in a situation where multiple types of ATM media are deployed and available at the same time. Experiment 1 focused on research gaps 3 and 12, while secondarily addressing several other research gaps identified from the Task 4 gap analysis (see Table 14) In particular, it was unclear how drivers would utilize in-vehicle ATM media along with infrastructure-based ATM media, and what the potential effects of the in- vehicle ATM media might be on drivers’ behavior (e.g., lane choice, visual distraction). Thus, the overall goal of Experiment 1 was to investigate the effects of information availability of the ATM information on driver behavior and distraction. The objective was to compare the effects of two levels of information availability (real-time vs. discrete) for “in-vehicle” ATM applications on drivers’ behavior. Experiment 1 manipulated information availability as an independent variable. In the real-time mode (“always-on” mode), the most current ATM information was available in real-time and updated as soon as drivers passed the overhead signs. In the discrete mode (“just-in-time” mode), ATM information appeared on the smartphone when drivers’ vehicle position reached the legibility distance, then disappeared as drivers passed corresponding overhead signs. Experiment 1 also included an add-on to address other research gaps (see Table 14). This included gaps regarding the effectiveness of provisional signs (Research gap 1 and 2), which was examined by ending each driving scenario with a short period (around 5 minutes) of dynamic speed limit zones with a provisional sign. In addition, Research gap 10 was addressed using a pre-experiment questionnaire about drivers’ preferences regarding ATM information dissemination methods. Finally, to address Research gap 4, drivers’ glance behaviors were recorded and analyzed using the eye-tracking system and face videos. Specifically, glance data measured driver visual distraction and partially addressed a research gap regarding distraction. Dynamic lane control was selected for Experiment 1, because currently employed messages for dynamic lane control strategies can be easily implemented into in-vehicle displays without major modifications; we also assumed that dynamic lane control may have more benefits with increased information availability compared to other ATM strategies. A pre-experiment survey was developed in a questionnaire format with closed-ended questions (with a few open-ended questions) to assess participants’ subjective preferences regarding information type, ATM media, and information reception timing for a set of ATM strategies were asked. This pre-experiment survey was implemented using SurveyMonkey (online format) and participants’ responses were collected by a laptop located in the experiment room. Given the known limitations of the driving simulator study (e.g., speed perception in a simulator and on- road driving might not be the same), the post-experiment survey was implemented to provide valuable opportunities to collect subjective outcome measures. Due to the within-subject design, participants experienced all experimental conditions and provided their overall preference for each ATM information presentation condition, how effective each condition was, and how they utilized in-vehicle ATM displays along with infrastructure-based ATM.

56 Driver distraction was measured by applying existing vehicle-industry guidelines for assessing the distraction impacts of in-vehicle devices. Two sets of guidelines were available for this objective. The first is the Alliance of Automobile Manufacturers (AAM) (2006) guidelines, which describe distraction evaluations based on glances to task-related areas such as in-vehicle displays. The second guidelines were the National Highway Transportation Safety Administration’s (NHTSA) 2013 criteria, which are based on measuring eye glances away from the forward roadway (e.g., glances to the interface devices as well as mirrors). This study adopted the AAM approach because we were specifically focused on glances to the smartphone area. In particular, we examined if receiving ATM information using the smartphone application can be completed with sequential glances that are “brief enough” to not adversely affect driving. The AAM guidelines list two main criteria (A1: single glance durations generally should not exceed 2 seconds, and A2: task completion should require no more than 20 seconds of total glance time to task displays and controls). We only applied the first (A1) criterion to measure distraction since the continuous nature of ATM accessing information through the smartphone meant that there are no clear start and end points to demarcate the task. Therefore, the second criterion was not applicable for this study. Table 14. Data sources for addressing research gaps in Experiment 1. Study Objective Research Gaps Studies Objective Measures Subjective Measures Investigate drivers' preferences regarding ATM information (Res. Q #1) What, when, and how do drivers prefer to receive ATM information? (#10) Exp 1 (Pre- survey) NA Comparison of ATM applications Can provisional signs affect driver compliance dynamic speed limit messages? (#1) Exp 1 (Post survey) NA Effectiveness of provisional signs Examine drivers' information processing abilities (Res. Q #2) Do static, advance signs increase driver understanding of ATM information? (#2) Exp 1 (Post survey) NA Effectiveness of provisional signs Do ATM media differ in terms of distraction, message understanding, and usage? (#4) Exp 1 (Main) Glance measurements Comparison of presentation timing Examine effectiveness of alternative ATM media (Res. Q #3) What media is the most effective for disseminating dynamic? (#3) Exp 1 (Main) Lane choice behavior Preferred ATM medium, comparison of presentation timing Investigate effective ways to present ATM information (Res. Q #4) What will be the best way to harmonize multiple ATM media? (#12) Exp 1 (Main) Lane choice behavior, glance measurements Comparison of presentation timing

57 Methods While details of our general approach are described in the previous section, this section only includes methods specifically applied to Experiment 1. Independent Variable Experiment 1 included two baseline conditions. In the “smartphone-only” baseline condition, the smartphone functioned as a generic navigation application and disseminated ATM information as well (however the overhead gantries did not disseminate ATM information in this condition). In the “gantry-only” baseline condition, the smartphone functioned as a generic navigation application but did not disseminate any ATM information. Experiment 1 manipulated “information availability” (real-time vs. discrete) of ATM information as an independent variable. In the discrete mode (“just-in-time” mode), ATM information appeared on the smartphone when drivers’ position reached the legibility distance of the information on the gantries, then disappeared as drivers passed corresponding overhead signs. In the real-time mode (“always-on” mode), the most current ATM information was always visible in a dedicated part of the smartphone display and the information updated as soon as drivers passed the overhead signs. Table 15 summarizes available sources of ATM information for each condition. Table 15. Sources of ATM information for each experimental condition. Information from overhead gantry Information from smartphone Information availability Smartphone-only (baseline) Not available Available Discrete Gantry-only (baseline) Available Not available Not applicable Just-in-time condition Available Available Discrete Always-on condition Available Available Real-time Dependent Variables Various dependent variables were collected to address the research gaps. The key variables are listed below. • Lane choice behavior o Percentage of time in merge/close lane: A high-level measure to examine drivers’ overall compliance to the lane signals. The percentage was calculated based on time spent in merge or close lanes within each experimental condition. • Glance measurements o Total glance time to the smartphone o Average glance duration to the smartphone • Survey data o Preferred ATM medium

58 o Comparison of ATM applications o Comparison of presentation timing o Effectiveness of provisional signs Road Layout The simulated road layout was designed to require drivers to change lanes according to the lane closure information. For each experimental condition (referred to as a block), a total of four overhead ATM signs were presented (i.e., four gantries within one block). The distance between signs was set to 0.5 miles. Orange barrels were placed to indicate closed lanes. Participants experienced the same road layout throughout the experiment and only the order of the experimental conditions varied across drivers to avoid order effects. Figure 47 shows an example road layout for Experiment 1. In Figure 47, grey horizontal lines represent four lanes and black vertical lines with colored blocks represent overhead gantries and lane signals. Green blocks represent “Lane open” symbols; red blocks represent “Lane closed” symbols; yellow blocks represent “Merge left/right” symbols (see Figure 48). Figure 49 shows a sample image showing rendered overhead gantry with lane signal symbols from Experiment 1. After completing the roadway section with lane closure information (seeFigure 47), participants then entered a dynamic speed limit zone (“Extra” block in Figure 47) that lasted another two miles. About 0.5 miles upstream of the dynamic speed limit signs (a speed limit change from 60mph to 45mph), there was an overhead gantry indicating all lanes open with provisional signs showing “REDUCED SPEED ZONE” messages (see Figure 50). Figure 47. Road layout for Experiment 1.

59 Figure 48. Lane signal symbols. Figure 49. A sample image of overhead gantry with lane closure signs from Experiment 1 (Battelle). Figure 50. Provisional signs (highlighted in red boxes) from Experiment 1 (Battelle).

60 Results The results section consisted of three subsections: (a) driving measurements, (b) glance measurements, and (c) survey responses. Driving measurements focused on participants’ lane choice behavior across various experimental conditions. Glance measurements focused on glance behavior and potential visual distraction caused the smartphone application. Survey responses interpreted participants’ responses to various questionnaires regarding ATM media and experimental conditions. Driving Measurements The percentage of time in merge/close lanes was calculated by dividing time spent in merge/close lane by total time spent in each experimental block. In most cases, smaller number of the percentage of time in merge/close lanes represented that drivers changed their lane sooner (i.e., before entering lanes with merge signs), and represented better lane compliance. Overall, the percentage of time in merge/close lanes was low across all conditions (M = 1.5%). Among all conditions, the always-on mode, in which drivers can receive the ATM information from both overhead gantries and smartphone in real-time, led to the lowest percentage of time in merge/close lane (M = 0.5%). See Figure 51 and Figure 52. Figure 51: Percentage of time spent complying with lane closures. A linear mixed-effect model was applied to examine differences across the experimental conditions. The drivers were set to a random effect and the experimental conditions were set to a fixed effect. Two baseline conditions exist; thus, two models with the same fixed and random effects, but with different orders of levels of the conditions were tested. The first model compared results to the smartphone-only condition and the second model compared results to the gantry-only condition. • Compared to the smartphone-only condition (baseline 1), other conditions were not associated with significant differences in percentages of lane compliance. o A difference between the smartphone-only condition and the “just-in-time” mode was the presence of ATM information from gantries. Therefore, this result may indicate that when the lane closure information was disseminated from the smartphone, additional information from overhead gantries may not add benefits to increase lane compliance. • Compared to the gantry-only condition (baseline 2), both “just-in-time” and “always-on” modes led to significantly higher percentages of lane compliance (p < .05 and p < .01, respectively). o The gantry-only condition represents present-day ATM applications where the ATM information is disseminated from infrastructure-based media such as overhead gantries. Compared to the current application, disseminating the same

61 information from both the smartphone and the gantries significantly decreased percentage of time in merge/close lanes. • Post-hoc contrast testing showed that there was no significant difference between the always-on and just-in-time modes in lane compliance behavior (p = .83). o Drivers’ subjective preferences may differ, but their driving performance (lane compliance) was not significantly different between two different information availability modes. Figure 52. Percentage of time in merge/close lanes. Glance Measurements Overall, the smartphone-only mode led to longer total glance time and longer average glance duration compared to other conditions. However, all of the conditions were associated with relatively shorter average glance duration (around .5 seconds) to the smartphone compared to the AAM distraction criterion (2-seconds), and even the longest glance duration for each condition was shorter than 1.2 seconds. This may indicate that checking/receiving ATM information from the smartphone did not distract drivers at unacceptable levels. Also, long-duration glances (> 2 seconds) to the smartphone were rarely observed in the sample, and the average number of long- duration glances ranged from 0 to 0.2 across all conditions. See Figure 53.

62 Figure 53. Glance measurement summary statistics for Experiment 1. The same set of the linear mixed-effect models was applied to two glance measures: total glance time and average glance duration to the smartphone. In terms of total glance time (see Figure 54), • Compared to the smartphone-only mode (baseline 1), both gantry-only mode and just-in- time mode led to shorter total glance time to the smartphone (p < .01) • Compared to gantry-only mode, only the smartphone-only mode showed a significant difference (as shown in the previous comparison) • Post-hoc contrast testing showed that there was no significant difference between the always-on and just-in-time modes in total glance time (p = .3)

63 Figure 54. Total glance time to the smartphone in seconds. In terms of average glance duration to the smartphone (see Figure 55), • Compared to the smartphone-only mode (baseline 1), the just-in-time mode led to shorter average glance duration to the smartphone (p < .05) • Compared to gantry-only mode, none of the conditions led to significant differences in average glance duration • Post-hoc contrast test showed that there was no significant difference between the always-on and just-in-time mode in total glance time (p = .7)

64 Figure 55. Mean glance time to the smartphone in seconds. Survey Responses (Comparison of ATM Applications) The survey provided information about drivers’ subjective opinions on four current ATM applications: (a) variable speed limit, (b) ramp metering, (c) lane signaling, and (d) junction control. Definitions of each application and brief descriptions of how each operated were provided with sample images. After that, the survey asked participants to compare multiple aspects of the four ATM applications: • The most important application (see Figure 56) • Usefulness (see Figure 57) • Communicatability (see Figure 58) • Level of comprehension with signs/symbols used in each application (see Figure 59) Participants percieved the intended purposes of both lane signaling and variable speed limit applications as more important than the ramp metering and junction control applications. When combining ratings for “extremely important” and “moderately important,” around 92% of the participants perceived the intended purpose of the lane signaling as important, whereas only 57% of the participants rated the junction control as important. A similar pattern was observed from another question that asked participants to rank the four applications from the least useful to the most useful. The junction control application was most frequently ranked as the least useful of the applications. Overall, comprehension of three of the applications was rated as very good or excellent by most participants; however, the function of junction control application was less clear than the other applications. The next question about participants’ comprehension with signs/symbols used in each application may explain the reason why junction control was harder to understand. Only

65 42% of the participants thought they totally comprehended signs/symbols used in the junction control application. One possible explanation for this result is that participants exposure to hard shoulder and junction control was relatively low when they were asked how often they see individual ATM applications. Around 33% of the participants indicated that they had never seen dynamic junction control. The results showed that, in general, symbols and signs used in the ATM applications were well-understood by drivers. Figure 56. Ratings for importance of each ATM application’s purpose. Figure 57. Ratings for usefulness of each ATM application. Figure 58. Ratings for how well each ATM application communicates.

66 Figure 59. Ratings for level of comprehension of symbols used in each ATM application. Survey Responses (Preferred ATM Medium) In the beginning of the survey, brief descriptions of both overhead gantry signs and smartphone application were provided with sample images. This survey focused on how participants utilized multiple ATM media (overhead gantry signs and smartphone application) when ATM information was available on both media at the same time (see Figure 60). In this situation, around 50% of the participants peceived the overhead gantry signs as more useful than the smartphone. Similarly, 45% of the pariticipants indicated that they preferred the overhead gantry signs as a way to receive the lane control information (see Figure 61). A similar pattern was also observed in the next question, which asked what participants’ preferred way to receive ATM information for various applications was: around 43-64% of the participants preferred to receive the ATM information on infrastructure-based displays. However, for variable speed limit, lane signaling, and junction control, around 50% of the participants wanted the information to be available on a second source (either smartphone or in- vehicle displays) along with the infrastructure display (see Figure 62). Figure 60. Usefulness ratings for overhead mounted sign and smartphone ATM applications.

67 Figure 61. Preference ratings for overhead mounted signs and smartphone ATM applications. Figure 62. Preference ratings for all ATM applications. Survey Responses (Comparison of Information Availability) This survey provided supplementary infomation about effectiveness and degree of distraction for the two different types of information availability (“just-in-time” vs. “always-on”). Descriptions and characteristics of each presentation mode were provided at the beginning of this survey and then the survey asked participants’ opinions about each mode and how they compared. See Figure 63. The results showed that around 70% of the participants preferred the always-on mode over the just-in-time mode, and around 50% of the participants felt that the just-in-time mode took more mental effort to use. The just-in-time mode was also rated as more visually complicated than the always-on mode. One explanation for this result is that the just-in-time mode presented the ATM information for only a brief period while the overhead gantry signs were continually visible to participants, so they could not access this information whenever they wanted. Participants may perceive this characteristic (which mimicked a characteristic of the infrastructure-based ATM media) as a constraint that caused more mental effort and required more visual attention for timely visual scanning. Note that this rating is a relative rather than absolute rating. In the next

68 question, which asked about the appropriateness of the presentation duration for the just-in-time mode, around 64% of the participants indicated that timing was appropriate. The result may indicate that participants preferred the always-on mode because they received the ATM information sooner (see Figure 64). The information availability influenced participants’ experience using the smartphone as an ATM medium, and the capability to provide the ATM information before the overhead gantry signs can be one of the benefits to disseminate ATM information through the smartphone. Figure 63. Preference rating for each ATM presentation mode. Figure 64. Participants’ timing assessment of the smartphone application in just-in-time mode. As shown in the survey of the preferred ATM medium, participants generally preferred and relied on the overhead gantry signs when ATM information was available from multiple sources. In a question that asked how they used both information sources in the two different information availability modes, participants indicated that they used both equally. Unlike other questions, this question asked participant to move a sliding marker on a continuous scale to indicate their overall strategy when it came to viewing the information they needed to navigate the driving scenario. Participants were free to choose and value along the scale. Figure 65 visualizes

69 participants’ responses as boxplots with median, interquartile range, and 25th and 75th percentiles to represent/compare two response distributions. Figure 65. Boxplots of participants responses on their strategy to use both ATM media. Results of the glance analysis suggest that the smartphone application did not qualify as a direct visual distraction (see Figure 66). The survey questions about distraction supported this notion. Overall, only 23 % of the participants perceived that the smaprtphone application was somewhat distracting (no one indicated that the smartphone was extremely distracting). Participants did not think that one of the presentation modes (always-on versus just-in-time) was more distracting than the other. Similar results were found when participants were asked to rate how annoying the visual sign presentation was in the always-on versus the just-in-time modes (see Figure 67). Only 18% of participants rated the smartphone application as “somehwhat annoying” in just-in-time mode. Two percent rated this mode as “very annoying,” and 57% of participants rated it as “not at all annoying.” The always-on mode was rated as less annoying by participants overall, though the difference may be too small to have practical significance. Over 50% of the participants indicated that they would use the smartphone application as an ATM medium in the future. See Figure 69. Figure 66. Distraction ratings for sign presentation by smartphone application mode.

70 Figure 67. Annoyance ratings for sign presentation by smartphone application mode. Figure 68. Overall distraction ratings for smartphone application. Figure 69. Participants’ self-reported likeliness to use the smartphone application to receive ATM information in the future. Survey Responses (Effectiveness of Provisional Signs) The survey also provided infomation about drivers’ subjective opinions about provisional signs. A set of questions asked participants to provide their feedback on effectiveness of the provisional signs in Experiment 1. Around 80% of the participants indicated that they saw the provisional signs (see Figure 70), and around 72% of the participants perceived that the provisional signs increased effectiveness of the dynamic speed limit signs following the provisional signs (see Figure 71). Figure 70. Percentage of participants who noticed reduced speed zone sign.

71 Figure 71. Participants’ self-reported effectiveness of the provisional signs. Additional questions focused on other types of provisional signs. Examples of other provisional signs were presented before asking participants’ opinion (see Figure 72). After that, the survey asked participants to rate multiple aspects of the provisional signs: • Importance of the intended purpose (see Figure 73) • Effectiveness (Figure 74) • Ease of understanding (Figure 75) Figure 72. Examples of provisional signs presented to participants (Battelle).

72 Overall, participants perceived the provisional signs used to alert drivers to speed reductions as (a) moderately to extremely important (77%), moderately to extremely effective (91%), and easy to very easy to understand (84%). See Figure 73. Figure 73. Importance ratings for provisional signs. Figure 74. Effectiveness ratings for provisional signs. Figure 75. Ease of understanding ratings for provisional signs. Discussion Experiment 1 investigated the effects of information availability, specifically for the lane signaling application. Participants’ percentage of lane compliance was sigficantly higher when they received the lane closure information from both the smartphone and overhead gantry, compared to when they received the information only from the overhead gantries. Regarding research gap 1220, this may indicate that introducing alternative ATM media for the lane signaling application may lead drivers being willing to exit closing lanes sooner. In general, however, the differences observed in this study are probably too small to have practical significance. 20 Gap 12: In a situation where multiple ATM media are deployed (e.g., electronic signs and in-vehicle displays), should each medium need to play a specific role to compensate each other and to avoid redundancy? And what will be the best way to harmonize multiple ATM media (including traditional and alternative media) and how to evaluate the effectiveness/efficiency?

73 When the ATM information was available from both the smartphone and overhead gantry, there were no significant differences between the always-on and just-in-time mode in terms of percentage of lane compliance. However, in terms of participants’ preference measured as by the post-experiment survey, the majority (70%) preferred the always-on mode over the just-in-time mode. Participants indicated that the just-in-time mode required relatively more mental effort to use and the just-in-time mode was more visually complicated. Given that a majority of participants (around 64%) indicated that the presentating timing of the just-in-time mode was fine, we interpreted this result as indicating that the always-on mode’s information availability was preferred by drivers. The always-on mode had one unique feature compared to the just-in-time mode—participants received the lane closure information sooner than the just-in-time mode (or the ovehead gantry). It appears that the early access to the lane information and the ability to view the most updated lane information regardless of locations reduced mental effort. Regarding research gap 321, the survey data showed that, in general, participants preferred and relied on the overhead gantry signs over the smartphone when the ATM information was available from both sources. Disseminating the ATM information sooner (compared to infrastructure-based displays) may be one of the advantages of smartphone applications as an ATM medium, and this view is supported by survey data indicating that participants perceived this characteristic as an advantage. Therefore, in terms of harmonization for multiple ATM media (research gap 12), the results supported disseminating lane signal information from the smartphone application along with the infrastructure-based displays, and suggested to disseminate the information as soon as available. Regarding the research gap 422, the results from the glance analysis indicated that total glance time and average glance duration to the smartphone in this study were below the AAM’s distraction criterion for human-machine interface interaction. The average glance duration to the smartphone in this study was around 0.5 seconds, which is substantially lower than glances durations to in-vehicle displays found in previous empirical studies. For example, one study found an average glance duration (to the center stack) for radio tuning tasks of around 1.02 seconds, and for navigation tasks of around 0.91 seconds (Lee et al., 2017). The results showed that receiving the ATM information from a smartphone did not increase either total glance time or average glance duration to the smartphone compared to the gantry-only approach. Specifically, this may indicate that while drivers use the smartphone application as a navigation tool, disseminating the ATM information along with the navigation information did not cause additional visual distraction. However, when the ATM information was not available from the overhead gantry and only available from the smartphone (i.e., smartphone-only), this condition required longer total glance time and average glance duration to the smartphone compared to conditions, which disseminated the ATM information from both overhead gantry and smartphone. 21 Gap 3: What media is the best for disseminating dynamic information to satisfy driver wants and needs? 22 Gap 4: What media for disseminating dynamic information achieves the lowest driver distraction, highest driver understanding and usage, and largest safety and mobility benefits?

74 In terms of research gaps 123 and 224, the survey data supported that the provision of additional information before reduced speed zones was perceived as highly important and effective. Also, drivers’ comprehension of the messages used in the provisional signs was good. Regarding the research gap 1025, the survey asked participants’ preferred way to receive ATM information for various applications and around 43-64% of the participants preferred to receive the ATM information through infrastructure. However, especially for “variable speed limit,” “lane signaling,” and “junction control,” around 50% of the participants wanted to receive the information from either smartphone or in-vehicle displays along with the infrastructure display. 23 Gap 1: Does the provision of additional information as an explanation to justify and encourage travel at reduced speeds increase the effectiveness of a dynamic speed limit with better speed compliance and lower average speed? 24 Gap 2: Is there a benefit of increased driver understanding in providing static signage in advance of ATM deployments to explain symbols? 25 Gap 10: Given a situation where multiple ATM media are available, what information do drivers want/how do drivers want to receive the information/when do drivers want to receive the information?

75 Experiment 2: Evaluating the Effects of Information Modality and Information Type of Dynamic Speed Limit Displays on Driver Behavior and Distraction. Overview Alternative ATM media can leverage unique information display characteristics compared to infrastructure-based ATM media. For example, alternative media can deliver information via auditory messages along with visual aids, whereas infrastructure-based electronic messages rely only on visual presentation. Outside of the ATM research area, in-vehicle displays or mobile devices have been widely tested, especially in the field of driver-vehicle interfaces. For example, the effects of input/output modality of driver-vehicle interfaces on driver distraction have been studied thoroughly (e.g., Mehler et al., 2015; Reimer, Mehler et al., 2013). However, the modality effects of in-vehicle ATM displays on driving performance have not been systematically tested. In previous approaches, modalities were usually confounded with alert types or system characteristics (e.g., Sykes, 2016), and it was not clear whether differences in the outcome measures were caused by the modality or other factors (e.g., alert type or other characteristics of different systems). Experiment 2 focused on the “information modality” of in-vehicle ATM displays and its effects on drivers’ behavior. Furthermore, Experiment 2 examined the effect of information type, communicated by the dynamic speed limits. Specifically, speed limit information can be presented “descriptively” (e.g., showing/telling a current speed limit) or “prescriptively” (e.g., showing/telling drivers what they should do to comply with a current speed limit). Compared to the previous infrastructure-based speed control strategies, alternative ATM media, especially with auditory output options, may offer more opportunities to utilize prescriptive information in a beneficial manner. Therefore, Experiment 2 investigated both information modality (visual vs. auditory-visual) and information type (descriptive vs. prescriptive) for dynamic speed control scenarios. A goal of Experiment 2 was to investigate the effects of the modality (e.g., visual vs. auditory-visual) and information type (descriptive vs. prescriptive) of in-vehicle ATM displays on driver behavior and distraction. Also, a sign comprehension study was included in Experiment 2 to assess how drivers interpret various DMS configurations, and to qualitatively assess effectiveness of different symbols and messages. Table 16 shows study objectives and corresponding research questions and gaps for Experiment 2. A pre-experiment survey was also developed to assess how drivers interpret and expect to use messages from ATM devices. Three different formats (overhead gantry signs, DMS, and pictograms) were examined and compared in terms of their effectiveness and levels of comprehension. Methodological details of the survey were similar to the survey in Experiment 1. A post-experiment survey was implemented as well to provide supplementary information. Similar to Experiment 1, the survey asked participants’ subjective preference for each condition, effectiveness, and strategies to utilized multiple information sources. In addition, the degree to which the smartphone application was distraction was measured by following the AAM guidelines, as in Experiment 1.

76 Table 16. Data sources for addressing gaps in Experiment 2. Study Objective Research Gaps Studies Objective Measures Subjective Measures To examine drivers' information processing abilities (Res Q #2) With multiple ATM media what is the most efficient and the least distracting modality (or modality combination) to deliver ATM information? (#11) Exp 2 (Main) Speed compliance, glance measurements Survey responses Comparison of presentation styles Do ATM media differ in terms of distraction, message understanding and usage? (#4) Exp 2 (Main) Glance measurements Comparison of presentation styles To investigate effective ways to present ATM information (Res Q #4) How understandable and effective are lane closure pictograms on a DMS relative to overhead dynamic lane control signs? (#5) Exp 2 (sign test) N/A Comparison of DMS, overhead signs, and pictograms Can dynamic information on lane control signs with supplemental DMS be effectively presented on less signage? (#6) Exp 2 (sign test) N/A Comparison of DMS, overhead signs, and pictograms Methods Since details of the general approach are described in Experiment 1, this section only includes methods specifically applied to Experiment 2. Independent Variables Similar to Experiment 1, Experiment 2 included two baseline modes: “Smartphone-only” and “Gantry-only” baselines. Experiment 2 focused on (a) information modality, and (b) information type of the in-vehicle ATM media. The modality variable had two levels: (a) visual, and (b) auditory-visual. Information type had two levels: (a) descriptive information (e.g., presenting a current speed limit), and (b) prescriptive information (e.g., instructing what they should do to comply with current speed limits) and only the auditory-visual condition had two levels of information type. A visual-prescriptive mode was not implemented because it was not viable to implement with the gantry. Therefore, a total of three experimental conditions (“visual-descriptive mode,” “auditory- visual-descriptive mode,” and “auditory-visual-prescriptive mode”) were implemented. In Experiment 2, the smartphone application presented current speed limit in a scenario where the participants drove on a roadway with a dynamic speed limit. In the visual mode, speed limit information was presented only visually (i.e., displaying the current lane-by-lane speed limits on the screen or gantry). In the auditory-visual modes, speed limit information was visually

77 presented on the screen and also the application read out the speed limit (i.e., displaying the current speed limit with auditory message of the speed limit) when the speed limit information was updated. Auditory messages were presented in two different ways: descriptive mode and prescriptive mode. In the descriptive mode, the smartphone app simply read aloud the current speed limit without additional information or instruction. In the prescriptive mode, speed limit information presented additional information about speed exceedance (when drivers exceeded the speed limit) along with the current speed limit. In addition to the three primary conditions that included visual presentation on both the smartphone and the gantry together, separate baseline conditions were included with visual presentation on just the smartphone or the gantry- only. Table 17 summarizes features of each experimental condition. Table 17. Sources of ATM information, information modality, and availability of prescriptive warning for each experimental condition. Information from overhead gantry Information from smartphone Information modality Prescriptive warning Information availability Smartphone- only (baseline) Not available Available Visual Not available Discrete Gantry-only (baseline) Available Not available Visual Not available Not applicable Visual- descriptive Available Available Visual Not available Discrete AV (auditory- visual)- descriptive Available Available Visual and auditory Not available Discrete AV (auditory- visual)- prescriptive Available Available Visual and auditory Available Discrete Dependent Variables The following dependent variables were collected and analyzed in Experiment 2: • Speed choice behavior o Percentage of speed compliance: Since participants were instructed to comply with the posted speed limit, speeding was defined as driving over the posted speed. However, since speed cues are weak in the driving simulator, a +5mph buffer was included to account for normal variations in speed before marking behavior as non-compliant. Therefore, in this study, speed compliance was operationalized as driving no more than 5 mph above the posted speed limit. o Delta speed: The speed above the posted speed limit for the duration of the drive (excluding the beginning and end segments). • Glance measurements o Total glance time to the smartphone

78 o Average glance duration to the smartphone • Survey data o Comparison of DMS, overhead signs, and pictograms o Comparison of presentation styles o Preferred ATM medium Road Layout The simulated road layout was designed to require drivers to change their speed according to dynamic speed limit information. For each experimental condition, a total of four overhead ATM signs were presented (i.e., four gantries within one block). Distance between signs was set to 0.5 miles. Although speed limits were presented for each lane, the speed limit was the same across all lanes. Similar to Experiment 1, participants experienced the same road layout through the experiment and only the order of experimental condition varied across drivers to avoid order effects. Figure 76 shows the road layout for Experiment 2. In Figure 76, black vertical lines with numerical digits on the top represent overhead gantries and dynamic speed limits. Figure 77 shows a sample image of dynamic speed limit information presented by the overhead gantry signs. Figure 76. Road layout for Experiment 2 (Battelle).

79 Figure 77. Sample image of dynamic speed limit information presented on gantries for Experiment 2 (Battelle). Results The results section consisted of three subsections: (a) driving measurements, (b) glance measurements, and (c) survey responses. Driving measurements focused on participants’ speed choice behavior across various experimental conditions. Glance measurements focused on glance behavior and potential visual distraction caused the smartphone application. Survey responses interpreted participants’ responses to various questionnaires regarding ATM media and experimental conditions. Driving measurements Overall, percentage of speed compliance was high across all conditions (M = 94.6%). Among the conditions, the AV-prescriptive mode led to the highest speed compliance (M = 97.6%). The level of speed exceedance was also low across all conditions. The smartphone-only condition led to the highest speed exceedance, but the average speed exceedance was around 1 mph. See Figure 78-Figure 80. Figure 78. Percentage of speed compliance and average speed exceedance (delta speed) by experimental condition. A linear mixed-effect model was applied to examine differences across the experimental conditions. The drivers were set to a random effect and the experimental conditions were set to a fixed effect. Two baseline conditions exist; thus, the two models had the same fixed and random

80 effects, but with different orders of levels of the conditions were tested. The first model compared results to the smartphone-only condition and the second model compared results to the gantry-only condition. • Compared to the smartphone-only condition (baseline 1), all conditions (visual mode, AV mode-descriptive, and AV mode-prescriptive) were associated with significantly higher speed compliance percentage (p < .001). o For the dynamic speed limit application, disseminating the information from only the smartphone led to the lowest speed compliance. Specifically, the smartphone- only condition led lower speed compliance compared to the current application (i.e., gantry-only). • Compared to the gantry-only condition (baseline 2), only the smartphone-only condition showed a significant difference in speed compliance. Other conditions did not lead any significant differences. o This result may indicate that for the dynamic speed limit application, disseminating the information from the smartphone may not bring any behavioral benefits. • Post-hoc contrast testing showed that there was no significant difference in percentage of speed compliance between modalities (V vs. AV) and information types (descriptive vs. prescriptive) (it must be acknowledged that speed cues in the driving simulator are weak, thus speed perception and speed choice in the simulator may have limited validity). Figure 79. Percentage of speed compliance.

81 Figure 80. Speed exceedance. Glance Measurements Glance times for each condition are shown in Figure 81. Overall, the smartphone-only mode led to longer total glance time and longer average duration compared to other conditions. However, all of the conditions were associated with relatively shorter average glance duration (around .5 seconds) to the smartphone compared to the AAM distraction criterion (2-second). As observed in Experiment 1, this may indicate that checking/receiving ATM information from the smartphone did not distract drivers at unacceptable levels. Also, long-duration glances (> 2 seconds) to the smartphone were seldom observed from our sample, and the average number of long-duration glances per participant was ranged from 0.1 to 0.3 across all conditions. Figure 81. Summary glance statistics for Experiment 2.

82 The same set of the linear mixed-effect models was applied to two glance measures: total glance time and average duration of glances to the smartphone. In terms of total glance time (see Figure 82), • Compared to the smartphone-only mode (baseline 1), all other conditions led to shorter total glance time to the smartphone (p < .001) • Compared to the gantry-only mode, visual-descriptive mode led to longer total glance time to the smartphone (p < .05) o The difference between the gantry-only mode and the visual information mode was the addition of ATM information on the smartphone (i.e., in all conditions the smartphone provided at least navigation information). This may indicate that participants used the ATM information on the smartphones since the glance times are longer than when just navigation information was present. o However, when the ATM information was presented in AV modalities, there was no difference compared to the present-day application (i.e., gantry-only condition). This may indicate that the auditory message contributed to decrease total glance time to the smartphone. • Post-hoc contrast testing showed that there were no significant differences between the visual information mode and any of the AV modes (p = .6 and p = .9, respectively) o Through Experiment 1 and Experiment 2, it was observed that visual demands of the ATM smartphone application was not close to the level of distraction in other human-machine interface (HMI) research. Therefore, it can be interpreted that the modality effect is not significant due to the visual demand is low. o In addition, there was no significant difference between the two AV modes (p < .9).

83 Figure 82. Total glance time to the smartphone across experimental conditions. In terms of average glance duration to the smartphone (see Figure 83), • Compared to the smartphone-only mode (baseline 1), both AV modes were associated with shorter average glance duration to the smartphone (p < .05) • Compared to gantry-only mode, none of the conditions led to significant differences in average glance duration • Post-hoc contrast testing showed that there were no significant differences between the visual information mode and any of the AV modes (p = .9 and p = .6, respectively) • In addition, there was no significant difference between the two AV modes (p < .9)

84 Figure 83. Mean glance time to smartphone across experimental conditions. Survey Responses (Comparison of DMS, Overhead Signs, and Pictograms) The survey provided information about drivers’ subjective opinions on effectiveness and clarity of communication for (a) DMS, (b) overhead gantry signs, and (c) pictograms. Definitions of each sign presentation medium along with brief descriptions were provided with sample images. After that, the survey asked participants to rate effectiveness and clarity of communication for the sign images. Three sign images were provided for each medium, thus a total of nine sign images were rated. All signs used in the test images provided information about lane use, but situations differed across the sign images (see images in Figure 84 and Figure 85). Participants indicated that DMS and overhead gantry signs were almost equally effective (64% of the participants rated DMS as extremely/very effective and 66% of the participants rated overhead gantry signs as extremely/very effective), but pictograms were less effective (52% of the participants rated pictograms as extremely/very effective) compared to other ATM strategies (Figure 84).

85 Figure 84. Participants’ subjective ratings of effectiveness for three types of ATM media. A similar pattern was observed in participants’ ratings for how well the signs/symbols communicated what they should do. Participants indicated that DMS and overhead gantry signs were equally effective means of communication (67% of the participants rated DMS as excellent/very good communication and same percentage for the overhead gantry signs), but pictograms were less effective (49% of the participants rated pictograms as excellent/very good communication) compared to other ATM strategies (Figure 85).

86 Figure 85. Participants’ ratings for how well three types of ATM media communicate. A second part of this survey used a different approach for comparison of the three sign presentation methods. In this part, specific situations (4-lane highway and 8-lane highway) were given to participants with images to help their understanding of the situations. Then, examples of DMS, overhead gantry, and pictogram were provided while each medium delivered the same information. Participants were asked to pick the most effective way to disseminate the ATM information given situations.The same question was repeated two times by varying choices: (a) without options with combinations (such as “overhead gantry with DMS”), and (b) with options

87 with combinations. Results showed that participants rated the overhead gantry as the most effective way to disseminate the ATM information when combinations were not allowed (Figure 86). However, when combinations were allowed, participants rated “overhead gantry with DMS” as the most effective media (Figure 87). As observed in the first part of this survey, pictogram was rated as the least effective medium. Figure 86. Participants’ relative ratings of effectiveness for three ATM media by road type. Figure 87. Participants’ relative ratings of effectiveness for five ATM strategies by road type. Survey Responses (Preferred ATM Medium) Similar to the survey in Experiment 1, in the beginning of the survey, brief descriptions of both overhead gantry signs and smartphone application were provided with sample images. This survey focused on how participants used overhead gantry signs and smartphone application, when ATM information was available on both media at the same time. In this situation, around 71% of the participants peceived the overhead gantry signs as more useful than the smartphone. Similarly, 74% of the pariticipants indicated that they preferred the overhead gantry signs as a way to receive the speed limit information. See Figure 88.

88 Figure 88. Participants’ ratings for usefulness and preference between overhead signs and smartphone application as ATM media. Survey Responses (Comparison of Presentation Styles) This survey provided supplementary infomation about effectiveness and degree of distraction for the three different presentation styles (“visual information” vs. “AV-descriptive” vs. “AV- prescriptive”). Descriptions and characteristics of each presentation mode were provided at the beginning of this survey and then the survey asked participants’ opinions about each mode and how they compared. See Figure 89. Results showed that around 60% of the participants preferred the AV-descriptive mode. Around 60% of the participants found that the visual-descriptive mode took more mental effort to use than the other two modes. Around 67% of the participants rated the AV-prescriptive mode as more distracting than the other two modes. The visual-descriptive mode was rated as a more visually complicated modality by the greatest percentage of participants (50%), but the AV- prescriptive mode was also rated as a more visually complicated modality by a large proportion of participants (43%). A main difference between the AV-descriptive and AV-prescriptive modes was the additional prescriptive auditory warning, which was provided when drivers exceeded the speed limit, while visual information on the smartphone screen remained the same across two conditions. However, participants’ rating of distraction for each condition varied noticeably (14% vs. 67%). This suggests that participants perceived the AV-prescriptive mode to more cognitively distracting than visually distracting. Overall, the results supported that the audio message decreased the level of mental efforts to use the smartphone application. However, the additional prescriptive warning was perceived as more distracting.

89 Figure 89. Participants’ comparisons of the three ATM presentation modes in Experiment 2. Overall, a majority of the participants (over 64%) perceived that the smartphone application was not distracting or was slightly distracting. Among the three conditions, the AV-descriptive mode was preceived as less distracting compared to other conditions. See Figure 90. Figure 90. Distraction ratings for the three smartphone application modes in Experiment 2. The next question extended participants’ preferred modality question to a navigation application such as “WAZE.” Participants were asked to select their preferred way to receive navigation instruction on applications, and around 45% of the participants preferred audible instructions. On-screen instruction was preferred by 31% of the participants. This result was consistent with the survey result, which focused on the smartphone application in Experiment 2. See Figure 91.

90 Figure 91. Participants’ preferred way to receive navigation instructions. Similar to Experiment 1’s results, participants were asked their willingness to use a smartphone application as an ATM medium (Figure 92). Around 46% of the participants indicated that they would use the smartphone application as an ATM medium in the future. Figure 92. Participants’ self-reported likeliness to use the smartphone application to receive ATM information in the future. Discussion Experiment 2 investigated the effects of modality (visual vs. auditory-visual) and information type (descriptive vs. prescriptive) of in-vehicle ATM displays on driver behavior and distraction. Specifically, our objective was to examine the most effective information modality and information type of in-vehicle ATM displays for presenting dynamic speed limit information. Participants’ percentages of speed compliance were significantly lower when they received the dynamic speed limit information from only the smartphone (i.e., smartphone-only) compared the other conditions, including the gantry-only condition. This indicated that introducing alternative ATM media for the dynamic speed limit application may not lead to increased speed compliance compared to the present-day application. In fact, when the smartphone became the sole source of ATM information, the speed compliance decreased. Post-hoc contrast testing showed that there was no significant difference between the visual- descriptive mode and the AV modes, nor was there one between the AV-descriptive and the AV- prescriptive mode in terms of the percentage of speed compliance. However, the post-experiment survey showed that participants preferred the AV-descriptive mode the most and found the prescriptive warning to be more distracting. When the dynamic speed limit information was only

91 presented as visual information (i.e., the visual-descriptive mode), participants thought using the application was more visually complicated and required more mental effort compared to AV conditions. Overall, a majority of the participants preferred to receive auditory-visual information rather than only visual information. However, they did not prefer prescriptive warnings. Regarding the research gap # 426 and #1127, the results from the glance analysis indicated that total glance time and average glance duration to the smartphone in this study were below the AAM’s distraction criterion for HMI interaction, which was similar to the findings from Experiment 1. The smartphone-only mode led to longer total glance times and average glance duration to the smartphone compared to other conditions. Compared to the present-day application (i.e., gantry-only), the visual information mode led to longer total glance time to the smartphone. However, none of the AV modes significantly increased either total glance time or average glance duration compared to the gantry-only mode. While considering that both visual information mode and AV modes presented the same amount of information in terms of visual information, it seems that the auditory messages contributed to decrease total glance time to the smartphone. The survey data showed that around 50% of the participants felt that the AV-prescriptive mode was somewhat to extremely annoying. The difference between AV-descriptive and AV- prescriptive mode was the additional prescriptive warning, which was only available when drivers exceeded the speed limit over 10 mph. It is possible that the “slow down” warnings annoyed the participants and also contributed their rating of distraction (around 67% of participants rated AV-prescriptive mode as “distracting”). In terms of the research gap #528 and #629, the survey results showed that participants preferred overhead gantry signs and DMS over pictograms. Especially when the three media were compared while disseminating the same lane closure information, pictograms were rated as the least effective medium. One explanation is that drivers in Washinton state may not be familiar with pictograms as they are not commonly utilized. Participants preferred a combination of DMS and overhead lane signaling the most. 26 Gap 4: What media for disseminating dynamic information achieves the lowest driver distraction, highest driver understanding and usage, and largest safety and mobility benefits? 27 Gap 11: Given a situation where multiple ATM media are available, what would be the most efficient and the least distracting modality (or modality combination) to deliver ATM information for the alternative/innovative ATM media? 28 Gap 5: Are lane closure pictogram on a DMS understood by drivers, and is it equally or more effective than information presented via overhead dynamic lane control signs? 29 Gap 6: Can dynamic information traditionally presented on lane control signs with supplemental DMS be presented on less signage in an equally or more effective manner?

92 Experiment 3: Examining How Agencies Approach the Deployment and Evaluation of ATM Applications. Overview The goal of Experiment 3 was to identify current and best practices used by agencies to effectively deploy and evaluate the potential and realized benefits of various ATM strategies, as well as guidance available to support a transition to innovative, non-traditional media for presenting dynamic information. Specifically, the objectives were: 1. To determine how agencies accurately quantify mobility-related benefits of ATM deployments, as well as the safety-related benefits of a temporary ATM deployment. 2. To determine how agencies systematically trade-off the various criteria associated with mobility, safety, cost, and driver needs when considering the deployment of ATM strategies. To determine available guidelines that agencies use to facilitate a transition from traditional media to in-vehicle media, and the gaps between currently available guidelines and requirements for presenting, formatting, and prioritizing ATM messages on the alternative media. Experiment 3 was a stakeholder engagement study that relied on email communications, targeted web-based surveys, and virtual telephone interviews, and a focus group to gather information. Methods A web-based survey using Survey Monkey was developed to distribute to agency deployers of ATM strategies. Survey questions were structured to gain an understanding of current and planned ATM deployments, including questions about whether agencies have conducted a benefit-cost analysis for planned deployments or evaluated the ATM deployments. Follow-up interviews with survey respondents were conducted, as needed, to gather additional information. As available, the following documentation was requested from participating agency representatives: 1. Benefit-cost analysis documentation that the agency conducted for any temporary or permanent ATM deployments, including any evaluation that quantified safety, mobility, or other benefits. 2. Alternatives analysis documentation that the agency conducted that projected estimated costs and benefits of planned temporary or permanent ATM deployments. This approach was used to identify best practices and develop guidance to serve as a framework for recommended, consistent approaches to consider both the tradeoffs when planning an ATM deployment and also to evaluate the benefits of an ATM deployment. Additionally, gaps within currently available guidelines and requirements were identified. Finally, a focus group was conducted to allow for an interactive discussion of all project findings with the Project Working Group from Phase 1 in order to verify and improve recommendations for guidelines development.

93 Results The findings in Experiment 3 are compiled through interviews conducted in April-May 2017 and more widely distributed surveys with follow-up interviews conducted in July-August 2019. Individuals were identified from agencies that have diverse experiences in deploying ATM strategies. Individuals providing input to this research effort and their respective agency and ATM experience are shown in Table 18. In general, identifying an appropriate contact who is aware of an ATM deployment and its history can be a challenge, particularly for relatively smaller or legacy deployments that have been operational for many years. In these circumstances, the motivations for deploying ATM, the resources used, and knowledge of any evaluation may no longer be available. The Interviews and Full Survey Results for all responses are presented as Appendix A. Table 18. Stakeholders interviewed for Experiment 3. Name Agency ATM Experience Marco Ruano Caltrans Dynamic merge control, lane control signage, VSLs Juan Pava Illinois DOT Work zone applications including queue warning Elyse Morgan Illinois State Toll Highway Authority Dynamic lane use control, dynamic shoulder lanes, and queue warning Faisal Saleem Maricopa County DOT In-vehicle work zone advisory, alert, and warning messages Andrew Meese Metropolitan Washington Council of Governments N/A – the Metropolitan Planning Organization (MPO) has not taken an active role in the consideration and deployment of ATM systems or equipment Jennifer Foley Michigan DOT Dynamic lane use control, dynamic merge control, dynamic shoulder lanes, queue warning, VSLs Brian Kary Minnesota DOT Advisory variable speeds, dynamic lane use control, queue warning, priced dynamic shoulder lane, and adaptive ramp metering Natalie Bettger North Central Texas Council of Governments Dynamic lane reversal, dynamic lane use control, and temporary deployment of dynamic shoulder lanes Dennis Mitchell, Galen McGill Oregon DOT Advisory variable speeds, queue warning, and travel times, specifically for incident and road weather management Josh Van Jura Utah DOT Work zone applications including dynamic speed limits Michael Fontaine Virginia DOT VSLs in urban and rural (weather) areas, dynamic lane use control, dynamic shoulder lanes Vinh Dang Washington State DOT VSLs, dynamic lane use control, queue warning, dynamic shoulder lanes, merge control, and queue warning Vince Garcia Wyoming DOT Dynamic speed limits on rural corridors for road weather applications

94 Objectives for the Study Objective 1: Quantifying ATM Deployment Benefits The first objective of Experiment 3 was to determine how agencies accurately quantify mobility- related benefits of ATM deployments, as well as the safety-related benefits of a temporary ATM deployment. Note that quantifying safety benefits of permanent ATM deployments tends to be more straightforward and was therefore not an emphasis for this objective. Overall, interviewees had mixed responses about quantifying ATM deployment benefits as part of an evaluation or a formal return on investment. Several ATM deployment evaluations that quantify mobility benefits were identified and are described in this section. However, since safety is frequently the primary driver to secure funding for deployments, analyses often focus on changes in crashes. This is particularly true for systems installed for low visibility and other adverse weather conditions, such as VSL systems in Virginia and Wyoming during which times mobility is not the concern. Finally, sites deploying a relatively small or local ATM deployment do not always conduct a formal evaluation and have only anecdotal evidence in terms of mobility or safety benefits. For instance, California did not formally evaluate dynamic junction control on the SR 110 at I-5, but did observe reduced congestion and informally examined crash rates. Quantifying Mobility Benefits of ATM Deployments Most ATM deployment evaluations focus on travel time and delay measures to quantify mobility benefits. Evaluation approaches vary from a relatively simple before-after calculation of trends or statistical analysis of data to more complex modeling methodology or video analysis. Various evaluation approaches that successfully quantified mobility benefits of ATM deployments are described below, with additional information available in the linked evaluation reports: • An evaluation of advisory variable speeds in Minnesota, which were part of the larger, full-gantry ATM deployment on I-35W examined a variety of mobility impacts (Hourdos et al., 2013). Loop detector data and posted advisory speed records were used to visualize the evaluation data in speed contour plots showing congestion patterns with individual sign activation. Findings indicated that the advisory speeds positively impacted the most severe congestion, with the instances and spread of extreme congestion waves being reduced after the advisory speed activation, which helps to reduce rear-end crashes. Flow-occupancy plots and statistical analysis were also generated, which seemed to indicate that drivers consider the advisory speed to gauge downstream congestion even if they do not comply with it. • Oregon examined early trends to demonstrate mobility benefits of the OR 217 dynamic advisory speed deployment (Oregon DOT, 2015). These initial results were used to justify funding for deploying additional dynamic speed limit deployments, including on I- 5 north of downtown Portland. Peak period and mid-day travel times, travel time variability, and speeds were examined. The preliminary findings showed a nine percent reduction in peak period travel time, 50 percent reduction in travel time variability along the corridor, and increased throughput during the peak hour. The monetary value of this deployment was not calculated. • The Illinois Tollway conducted an evaluation of the full-gantry ATM system on I-90 that included dynamic lane use control, dynamic shoulder lanes, and queue warning (Illinois Tollway, 2018). The evaluation leveraged a 246-day period after construction was

95 completed but before ATM became operational to compare with operations after ATM was initiated. The evaluation included video analysis to identify changes for inadvisable and compliant maneuvers for both lane change behavior and Scott’s law, which requires drivers to slow down and move over one lane, if possible, when approaching a stopped emergency vehicle. The video analysis found improved behavior for all of these measures. The evaluation also examined average travel time index for the ATM corridor compared with three control corridors, finding negligible changes. • The Virginia DOT conducted an evaluation of the I-66 full-gantry ATM system impacts, including an examination of mobility benefits using INRIX travel time data and ATM activation logs (Chun & Fontaine, 2016). Mobility measures in this evaluation included total delay, as well as average travel times and travel time reliability, which were analyzed using paired t-tests to identify statistically significant before-after differences. Because the shoulder lane was open almost double the hours it was before the larger ATM improvement, mobility improvements occurred in this section and benefits were monetized, focusing primarily on travel time savings. Specifically, total operations benefits were calculated only for weekend improvements and estimated to be about $3.7 million per year. • VSLs on I-77 in Virginia were deployed primarily to reduce the number and severity of crashes during low-visibility conditions, but also to improve speed compliance. While an evaluation of the I-77 system did not seek to identify mobility benefits, it did measure mobility impacts (Gonzales & Fontaine, 2018). Virginia DOT used before-after data on speed, volume, visibility, and VSL posted speed logs to analyze mobility. Specifically, models were developed to show how mean speed varied before and after VSL activation. Stepwise regression was employed to describe speeds as a function of visibility, weather conditions, and/or VSL factors for each of the models considered. Additionally, speed data after the ATM deployment became operational for low-visibility conditions were examined as: posted speeds by MP, observed speeds versus posted speeds by milepost, and speed differentials between observed and posted speeds by milepost. Paired t-tests were performed on observed speeds to determine if mean speeds for a set posted speed at a milepost were significantly different. The before-after analysis showed statistically significant speed reductions of 2-5 mph for reduced visibilities, while regression models relating speeds as a function of visibility and showed positive impacts from the VSLs. Reductions in mean speeds indicate VSL effectiveness, with recommendations to improve results. Several individuals observed that some ATM locations measure only safety benefits given the nature and goals of the deployment, but that a reduction in crashes will inherently improve travel time, speed, and travel time reliability. For example, the Wyoming DOT conducted a formal return on investment analysis which showed that the safety-related benefits of the VSL systems deployed on rural interstates and highways for adverse road weather justify the cost after only one year of operations (Vince Garcia, 2017). In general, evaluation analysis findings are frequently mixed or unable to be determined given other changes in the corridor that may have occurred when ATM was deployed as part of a larger reconstruction project or other operational changes, which is the case for Minnesota and Washington state, for example. Although the National Evaluation Reports for the Seattle/Lake Washington Corridor Urban Partnership Agreement (FHWA, 2014) and the Minnesota Urban

96 Partnership Agreement (FHWA, 2013) included mobility, safety, and exogenous factors analyses on the ATM corridors, exogenous factors pertaining to a major reconstruction project in Minnesota, the initiation of tolling operations on an existing bridge with no change in capacity in Washington state that changed travel patterns, and technology improvements in both corridors make it difficult to isolate the relative benefits and impacts of the ATM strategies. Even so, the participants from Washington State DOT and Minnesota DOT that provided input through this research effort noted that given high costs for deployment, operations, and maintenance, as well as the difficulty in measuring quantifiable benefits, a large benefit relative to those costs is unlikely. Safety Benefits of Temporary ATM Deployments Respondents with temporary ATM deployments noted the difficulty of evaluating work zone deployments due to the short-term duration of such deployments and different conditions for each location (Utah DOT, 2018; Illinois DOT, 2014). It is challenging for agencies to analyze the relative benefit of the temporary ATM deployment, i.e. to measure the number of crashes that are not occurring. However, two evaluation approaches that quantified safety benefits of temporary ATM deployments are described below: • Utah DOT conducted an evaluation of the dynamic variable speed limit system in work zones that identified benefits using available speed data to analyze compliance (Utah DOT, 2018). Specifically, significant speed reductions were observed within the active workspace where workers are present, with average speeds near the active workspace being 15-25 mph lower than the original roadway posted speed limit, which improves worker safety. There is also anecdotal evidence for an improvement for worker safety based on a near-miss crash that may have resulted in a fatality if the vehicle had been traveling at a faster speed. Additionally, use of the VSL system reduces the mobility impact to drivers by limiting the distance that drivers have to reduce their speed, i.e. the maximum speed limit reduction is displayed only in the segment of the work zone in the active workspace. • Illinois DOT has only conducted rough calculations of crashes before and after queue warning systems were deployed within a work zone in 2010 and 2011 (Illinois DOT, 2014). Although work zone conditions changed before and after the queue warning systems were deployed, work zone exposure was estimated based on average daily traffic volume and the number of days with a lane closure to compare with crash rates to calculate a 14 percent reduction in rear-end crashes despite an over 25 percent increase in exposure. Objective 2: Balancing Driver Needs versus Safety, Mobility, and Costs for ATM Deployments The second objective of Experiment 3 was to determine how agencies systematically trade-off the various criteria associated with mobility, safety, cost, and driver needs when considering the deployment of ATM strategies. Survey respondents were asked whether their agency considered four specific criteria (driver needs, safety, mobility, and costs) when considering six different types of ATM deployments, as applicable. Survey respondents gave very different responses, as summarized in Table 19. For example, for their deployment(s) with multiple ATM strategies, Michigan DOT considered all four criteria, Oregon DOT considered three criteria (not driver needs), and Minnesota DOT

97 considered only mobility and safety impacts. Other agencies applied different criteria, depending on the ATM strategy. For example, Virginia DOT considered all four criteria for dynamic lane use control, dynamic shoulder lanes, and VSL, but only considered driver needs and safety impacts for queue warning. Similarly, North Central Texas Council of Governments considered all four criteria for dynamic lane reversal and dynamic lane use control, but applied just three criteria and not safety impacts for dynamic shoulder lanes. Finally, for the Illinois Tollway deployment with five ATM strategies, safety impacts were considered for all strategies, mobility impacts were considered for all strategies except VSL, and costs and driver needs were considered only for dynamic lane use control. Full 2019 survey responses are in Appendix A. Table 19. Deployment of ATM strategies among respondents’ agencies. ATM Strategy Total Respondents with ATM Strategy Percent Using Criteria Costs Driver Needs Mobility Impacts Safety Impacts Dynamic Lane Use Control 5 80% 80% 100% 100% Dynamic Shoulder Lanes 5 60% 60% 100% 80% Variable Speed Limits 5 60% 40% 80% 100% Queue Warning 4 50% 50% 75% 100% Dynamic Merge Control 2 50% 50% 100% 100% Dynamic Lane Reversal 1 100% 100% 100% 100% Respondents Using Criteria (for at least one type of ATM strategy) 6 83% 67% 100% 100% Most interview respondents had difficulty with this question, with some saying they were not sure that their agencies really worried about driver needs. Additional details from interviews are in Appendix A. • California: This is a struggle. Caltrans strives to provide as much information to drivers as possible, but there are many constraints, including funding, political, and environmental constraints. Public acceptance is the key to gauge the success and to help find this balance point. • Illinois DOT: This is a difficult question to answer for work zone deployments because the purpose is for incident mitigation, and it is difficult to gauge how many crashes have been prevented. A need of the driver is to have reliable travel times, so it is assumed that the cost of providing these systems is worth the theoretical benefit of deploying them, based on the improved mobility due to reduced crashes. • Minnesota: The ATM lane control signage comes with a high cost to install and maintain. Minnesota DOT is cutting back to fewer signs and longer spacing because other investments, such as incident response teams, demonstrate a higher return on investment that cannot be demonstrated by ATM, where we know the cost is very high and quantifiable benefits are hard to measure. • Oregon: The biggest driver is a reduction of crashes, which is the ultimate need of the driver. Although ATM systems are not cheap, they are cheaper than building new lanes on the freeway. Since infrastructure is already in place for communications and operations, ATM systems come with a comparatively small cost. Oregon DOT cannot eliminate congestion but can impact the crashes.

98 • Utah: Utah DOT is mobility based and relies heavily on user costs. Anything can be justified by user costs and delays: if the DOT can increase mobility for taxpayers and citizens, it is supported. • Virginia: Virginia DOT examines this on a case-by-case basis. Staff at the chief engineering level are concerned about installing infrastructure that will have to be maintained, knowing that in the next 10-20 years connected vehicles may not need this infrastructure. The question becomes: what can we do now, but not overbuild? Striking this balance and answering these questions is not a formal process, but something that comes through the systems engineering process and concept of operations. • Washington state: All ATM deployments are one-way communications to the driver, so the agency does not generally know driver needs. The DOT is looking for better ways to get driver feedback to complete the circle, by examining traffic data and seeing that drivers respond to the messages. The DOT is interested in implementing more dynamic routing and it would be helpful to know whether people respond to alternate route suggestions. • Wyoming: Wyoming DOT is deploying VSLs strategically where it makes sense, based on developed crash mitigation factors, since VSLs should not be deployed systemwide. Objective 3: Available Resources for ATM and Gaps The third objective of Experiment 3 was to determine available guidelines that agencies use to facilitate a transition from traditional media to in-vehicle media, and the gaps between currently available guidelines and requirements for presenting, formatting, and prioritizing ATM messages on the alternative media. Note that most dynamic ATM strategies that have been deployed employ roadside signing and messaging strategies that fall on a spectrum between traditional and in-vehicle media. Available Resources All stakeholder participants were asked about the resources their agency used (e.g., documents, guidance, peer deployments) to inform the ATM deployment design, including signage, display format of information, content, timing, and priority for displaying information. The following reflects the responses provided. Given the innovative nature, limited resources, and limited number of ATM deployments in the United States, peer exchanges and interactions with other agencies that have deployed ATM strategies are most frequently the primary resource for agencies in planning and designing a new ATM deployment. The MUTCD provides guidance in Chapter 4M on the use of lane control signals (MUTCD, 2009). Additionally, a March 2019 Memorandum provides clarification on required lane use control signal indications in freeway applications following experiments that concluded a steady Diagonal Downward Yellow Arrow indication do not improve on the standard steady yellow X recommended by the MUTCD (FHWA, 2019).

99 Resource Gaps Stakeholder participants were also asked about the topic areas for which guidelines and resources were lacking and would have been helpful for planning the design and operations of ATM deployments. Full responses can be found in Appendix A. The following reflects the responses provided from the six completed surveys in 2019: • Message display (graphics vs. text) (5 responses, 83 percent) • Sign placement (spacing, overhead vs. side-mount) (4 responses, 67 percent) • Software and algorithm development for automated operations (4 responses, 67 percent) • Priority of message to display (2 responses, 33 percent) • Other resources, including: o State of practice reports or case studies, including lessons learned (3 responses, 50 percent) o Best practices in standard operating procedures for ATM deployment operations (1 response) o Partners to engage in ATM deployment development, i.e. police, fire, tow truck operators, etc. (1 response) o Maintenance and communications costs (1 response) One respondent commented that resources for all of these areas are important, since ATM varies depending on location and need. Further, the best information available to agencies is examining what other states have deployed, what they have learned, and how they use ATM, to see if it applies to the application an agency is considering. This was reinforced by a second respondent who noted that it is hard to replace the value of talking directly with someone else that has experience with an ATM deployment. Yet another respondent emphasized that better guidance and information about lane control sign placement and the type of graphics that can be used on the signs would help to reduce design and operational strategy changes. A fourth respondent noted that effectiveness data is widely available, but there is sometimes a lack of data on exactly how systems were operated; since operations influences effectiveness, more data on algorithms would be very useful. At this time, very few agencies have deployed or tested any in-vehicle ATM messaging strategies. No respondents indicated that their agency was currently pursuing or seeking resources for in-vehicle ATM messaging. This may reflect concerns about liability or the role of the public sector in providing in-vehicle messages. To that end, some agencies rely on the private sector or university partners to develop and facilitate the provision of in-vehicle messages. For example, while Maricopa County DOT provides work zone data and information, the agency partnered with a private-sector entity to provide in-vehicle messages that inform, advise, alert, and warn drivers as they drove through the work zone, based on information received by the vehicle at the beginning of the work zone and the vehicle trajectory (Faisal Saleem, 2019). Similarly, Virginia DOT has worked with the VTTI to develop two mobile applications for presenting in-vehicle audio and visual messages for testing and demonstration purposes: one duplicated the information presented on I-66 ATM signage and the other presents work zone information like lane closures, worker presence, and truck entering notifications (Applications and Interfaces, 2019). Both of these applications present visual messages that are consistent with static or dynamic roadside signage and are being used to demonstrate the technology using a test

100 population. However, the DOT does not necessarily expect to be the provider of in-vehicle messages in the long-term as it would be a burden for drivers to download multiple apps from multiple agencies. Instead, the long-term vision for this effort is to demonstrate the value of the test apps so that third-party information providers like Google Maps, WAZE, or Drivewyze will access the public DOT data used by these test apps to integrate with their apps that provide such information nationally. It is expected that the third-party information providers would then use their own chosen message format to present in-vehicle messages to drivers (Michael Fontaine, 2019). Other Lessons Learned and Key Takeaways from Deployers Lessons learned and takeaways from deployers through the surveys and interviews in this study include the following: • Smaller-scale deployments: three early deployers of multipurpose overhead lane control signs all favor a scaled back approach for future deployments that may not involve signs for each lane, believing a comparable benefit can be achieved with lower capital and operations costs. • Evolution of deployments: an initial deployment provides a foundation for expanding to additional corridors; understanding what works and making modifications for things that do not; and enhancements to capabilities and software to make systems more efficient and responsive. • In-pavement lighting: the only two deployers with ATM systems that included in- pavement lighting experienced issues due to heavy traffic and salt corrosion that resulted in discontinued usage. • Relative effectiveness of dynamic signage in work zones: two deployers are interested in understanding the relative effectiveness of a fully or partially dynamic sign compared to an alternate approach; specifically, a dynamic speed limit strategy that is manually adjusted based on worker presence, and a queue warning system that uses a flashing beacon with “when flashing” text on a static sign. • Visibility: more than one deployer noted the importance of placing signage overhead or on each side of the highway to allow all drivers to see the display since passing trucks can block the view.