Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
69 Issues Encountered and Lessons Learned As an introduction to this chapter, it is important to note that the study center activities went remarkably well, especially considering the complexity of the project. VTTI, TRB SHRP 2 staff, and the study centers communicated weekly via tele- conferences throughout the program. At these conferences, project status and problems that may have arisen were dis- cussed. In the case of the latter, solutions or workarounds were identified. In general, the tools, procedures, equipment, and training provided by VTTI were effective in supporting study center activities. This chapter summarizes the issues encountered and les- sons learned during the S07 NDS data collection activities at the six study centers. These observations are presented as they relate to the following study center activities: IRB process, recruiting, participant consent and assessment, DAS equipment installation, participant management and fleet maintenance, DAS equipment deinstallations, equipment management, and program management. Many of the lessons learned have been cited in preceding chapters but are repeated here to pro- vide a consolidated list. IRB Process ⢠Establishment of good communication with IRB commit- tees, both in advance of application submission and during execution of project, is essential for smooth operation of a complex program such as the NDS. ⢠Allowing the Virginia Tech IRB to serve as the IRB of Record for two of the study centers resulted in significant time savings for those two centers, given the number of IRB amendments (18 total) that had to be processed during the 3-year study. ⢠The timing of IRB amendment submittals should be care- fully considered to ensure the least number of submittals, and thus delays, in executing the project. Bundling small amendments together could improve the efficiency of the process. ⢠Over the 3-year project performance, some study centers experienced changes in personnel in their IRB committee. This resulted in changes in IRB management styles and required additional time from project staff to manage the amendment process. Recruiting ⢠Recruiting participants for the NDS turned out to be one of the biggest challenges facing the program. It is recom- mended that recruitment of participants start early, at least 3 months before they are needed in the study. In general, the study centers found young (16â21 years of age) and old (76+) participants to be the most difficult to recruit. The establishment of an earlier focus on efforts to recruit those two age groups would have been beneficial to the program. ⢠For the highest recruiting returns, it is important to understand whom you are targeting with each recruiting method so that areas of exposure are selected based on the interests and underlying motivations of that popu- lation. Craigslist ads and posted flyers are more likely to recruit individuals already seeking opportunities to par- ticipate in research or to earn money. Groups not actively seeking these opportunities must be sold the project on other merits. ⢠Once a recruit registers, timely follow-up with that recruit (preferably within a week) is important. Study centers found that if the follow-up contact with a recruit extended beyond 2â3 weeks, the recruit frequently lost interest in participating. ⢠The compensation offered to recruits, as well as the study inclusion criteria, changed during the project. Once poten- tial participants were told they were ineligible, or they deter- mined that the compensation was inadequate, they rarely reconsidered participation (i.e., you only get one chance to make a first impression). C h A P t E R 4
70 ⢠In some cases, if one part of a test was unsuccessful, the entire test had to be repeated (e.g., the DHI suite). The pro- cess would be improved and more efficient if only the unsuccessful portion of the test had to be repeated. ⢠It would have been more convenient if the tests admin- istered by the researcher were grouped together and kept separate from the tests that required participant interac- tion on the touch screen. This would have expedited the process by not requiring the frequent switching of seats. ⢠Participants reported that some survey questions did not have relevant answer options, and they felt some survey questions contained obvious bias (e.g., regarding alternative lifestyles) which they found ridiculous or even offensive. ⢠Feedback from participants indicated they felt the ques- tionnaires were too long and some questions were unclear or did not have appropriate choices for responses for some classes of recruits. ⢠In some cases, conducting multiple computer-based assess- ment tests proved to be a challenge. The process was com- plicated by requiring the participant to close a window and go back to the original screen to launch a new window for the next test. Enabling the participant to step through all tests in one continuous process might have simplified the procedure, especially for the older participants. Installations and Associated tools ⢠It is important to keep participants informed about any delays that might occur during the installation process. In general, participants were cooperative and accommodating when problems arose. ⢠It is also important to accurately document the condition of the vehicle (as determined during the preinstallation vehicle inspection and the installation process). Proper doc- umentation made it easier to address vehicle problems that arose during the program by referring to the installation and inspection records. ⢠Reference sources are of great assistance during the installa- tion process. Commercially available maintenance manuals that encompass all vehicles proved very useful in determin- ing the routing of wires and locating power lines in vehicles. ⢠When planning a new installation, careful thought should go into how wires associated with the equipment are routed and secured, especially when hiding them behind interior trim panels and under seats and carpet. The deinstallation process can be greatly facilitated by strategically routing the cables and securing them to objects that are easily accessible or that do not require substantial disassembly of interior trim parts. This can reduce how much of the vehicle will need to be dismantled during the removal of the wires. ⢠Obtaining a spot on the local evening news reached thou- sands of people and also legitimized the project. (Some recruits initially thought project was a scam.) ⢠Radio ads targeting specific age groups and radio stations selected for the desired demographic were quite effective in urban study areas in recruiting both easy and hard-to-reach age groups. In rural areas, TV and radio ads were expensive and not very effective. Newspapers, flyers, posters/banner ads, Craigslist, and word of mouth appeared to be more effective in the rural areas. ⢠Craigslist ads were particularly effective for recruiting participants between the ages of 36 and 75. ⢠Promotional items (especially T-shirts worn around cam- pus and the community) further promoted the driving study by word of mouth. ⢠In some study centers, advertising was permitted on a state DOT traffic map website. When permitted, the website ban- ner ad was a very effective recruiting method. Exhibits, kiosks, and booths were effective recruitment tools at some of the study centers, especially if the venue was carefully chosen for the desired age group (e.g., a well-established, weekly open air market on a college campus successfully targeted 18â25-year-olds at one study site). ⢠For the highest recruiting returns, it is important to under- stand whom you are targeting with each recruiting method and message. One study center had significant recruiting success by employing the services of a professional market- ing agency to tailor the recruiting messages to the interests of the targeted population. Consent and Assessment ⢠It was helpful to send the consent form to recruits before they arrived for their installation appointment. This enabled them to review the forms ahead of time and not delay the intake/installation process. (However, sending the consent form did not guarantee they would read it.) ⢠The VTTI-provided 10-minute video explaining the main points of the consent form was very useful. The video was particularly helpful if a local IRB required that a partici- pant be questioned about the consent form to verify that they had read and understood it. ⢠Feedback obtained from participants suggested that the vision assessment test was very long, and some vision tests were difficult (and frustrating) for older (and even some younger) drivers. ⢠When the participant completed the tests, the study center needed to upload the test results to VTTI. Once uploaded the study center did not have the ability to reopen and check files that had been sent. The ability of the study cen- ter to check the uploaded files would have improved the process.
71 waiting area of the Buffalo center, a poster was put up to notify participants that their first payment would be made in the first week of the month following their start in the study. This notice reduced, but did not eliminate, the num- ber of calls regarding payments. (In the future, a simpler payment schedule would be beneficial.) ⢠Once equipment was installed, participants were generally very good at keeping their maintenance appointments. The percentage of participants not making their mainte- nance appointments (including SSD swaps) was low. One way to cope with the inefficiencies caused by no-shows was to have cross-trained staff that could perform another function if the participant didnât show up. ⢠Some maintenance was performed off-site at the partici- pantsâ homes or work places. Scheduling maintenance trips by considering the number of participants in neighboring towns increased the efficiency and number of maintenance actions that could be scheduled in a given day. This strategy reduced travel time between maintenance visits. ⢠Having an internal (local) database to track information (including participant call records, consent form versions, appointment history, participant contact information) proved to be an essential tool. ⢠Participants moved and changed phone numbers with surprising frequency. It was helpful to check on their cur- rent address and phone number whenever scheduling an appointment with them. ⢠Text messaging and e-mailing were very helpful for sched- uling younger participants (a dedicated study cell phone was used). ⢠Directions were sent to participants repeatedly throughout the study. ⢠Having appointments available on evenings and Saturdays was critical to scheduling some recruits. ⢠Contact information was always confirmed when speak- ing with participant or whoever answered, and additional contact options were requested if not already listedâcell phone, work and home numbers, e-mail address, and prefer- ences for contact. Besides best times for contact, also noted preferred method of communication (i.e., text? voicemail? letter?). ⢠If voicemail did not have a personally recorded outgoing message or recorded name, assumed that this participant does not check voicemail. Deinstallations ⢠The GPS/cellular antenna mounted in the rear window was destroyed in a high percentage of deinstallations. This was primarily due to the soft flexible base of the antenna being damaged in the process of removing it from the window. ⢠Some vehicles were very sensitive to power taps into cir- cuits that are tied to the vehicle computer. For example, even though the circuits that were used only operated a solenoid for a vent valve, several vehicles were encoun- tered that had issues with this process, while many others did not. As more and more circuits are controlled by the vehicle computer on new vehicles, careful thought should go into how to provide power to the equipment. ⢠The age of vehicles had to be considered when installing the DAS equipment. While the power taps provided by the program worked well with the mini-style fuses in most newer vehicles, they did not work well in vehicles that have the older, larger-style fuses. Using these power taps in a fuse box that has the larger-style fuses can result in a loose power connection that will affect DAS or RIB performance. ⢠The field data collection efforts were supported by a num- ber of computerized applications that supported all aspects of the study center activities. Of particular importance in this regard were the activities associated with the installa- tion and testing of the DAS equipment and the mainte- nance of study center equipment inventories. In general these applications performed well. The following sugges- tions and observations provide insight into the use of these applications in a project that involves working with the publicâthe expectations of seamless software functionality were encountered on a daily basis: C Software critical to the successful completion of installa- tions should be beta-tested at a limited number of study centers (or a single study center) before its release to all study centers. If beta-tested, any problems uncovered could have been addressed quickly by the developers and inconveniences to participants and study center staff could have been reduced. C On most occasions when problems arose during instal- lations, the required outside technical support was avail- able. However, there were some installations during which technical support was needed but not available. In some of these cases, the participant had to schedule another appointment and return to the study center to complete the installation. This inconvenience to the participant could have been avoided by ensuring technical support availability during all scheduled installations. Participant Management and Fleet Maintenance ⢠Participants were compensated for their time in the study. It was important for participants to be informed of the sched- ule for payments and the dates these payments were to be made. A large number of phone calls to the centers early in the program were from participants asking why they had not yet received their compensation. In the participant
72 ⢠In some cases there was an insufficient supply of extra kits/ parts to support participant vehicle maintenance activities. This resulted in the need to make multiple trips to service participant vehicles. Program Management ⢠The most useful information exchange strategy was weekly teleconference calls between the study centers, VTTI, and NAS. The wiki (a VTTI web-based information sharing site) worked for a while, but it seemed that after some point, the technicians either did not want to enter additional informa- tion or did not have the time to enter more information. ⢠The program management team had to be flexible to accom- modate the changing needs of the study. Challenges arose in dealing with some unhappy participants, even though every effort was made to accommodate their needs. ⢠Additional challenges arose when participants claimed that the installed equipment caused malfunctions or damage to their vehicles. Even when it was clear that the claims had no merit, it was easier to reimburse the costs of fixing the problem than try to prove that the equipment did not cause the damage. ⢠It was suggested that any future study provide incentives to engage participants who are not the type to sign up for a safe driving study. It is possible that the participants tended to be those who are more safety minded and more aware as drivers than the average population. ⢠Older car models were particularly difficult to deinstall. The plastic components such as kick panels and moldings were very brittle. Removing some of the panels broke their fastening elements. In many cases, replacement parts for these vehicles were difficult to acquire as they were no longer available. ⢠Several study centers (Buffalo, Seattle, State College, and Tampa) noted issues with corrosion between the provided license plate screws and the brass inserts in the license plate frame on the radar unit. This corrosion problem also extended to the radar alignment screws. ⢠Several study centers developed a protocol to check fea- tures of the vehicle before returning it to the participant. This included checks to ensure proper postdeinstallation functioning of features such as the instrument lights, elec- tronic and mechanical latches, lights, turn signals, radio, and air conditioner. Equipment Management Issues ⢠At the beginning of the study and several points throughout, the logistics of keeping a sufficient supply of DAS systems or individual parts was difficult. ⢠An inventory software tool was needed by study centers to keep track of equipment that was issued to the center and its status. If such a tool had been made available, it would have been very useful. (One site designed its own about halfway through the project.)
