Design of the In-Vehicle Driving Behavior and Crash Risk Study (2011)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

C H A P T E R 4 Quality ProcessesQuality Goals and Objectives The overall goal of SHRP 2 NDS data quality processes is to work within budgetary and scheduling constraints to obtain the most complete and highest-quality data set possible. More detailed quality objectives include gathering data from a high- quality sample, gathering the full complement of data expected, gathering intended data efficiently, storing intended data securely, and maintaining data privacy and security. The specific data quality processes have a threefold purpose, as follows: 1. To identify and preclude or reduce errors at their source; 2. To identify DAS system failures to initiate prompt main- tenance actions during data collection; and 3. To mitigate the effects of data collection errors by assur- ing that the data entering the database are as high-quality as possible. To implement the deceptively simple quality policy noted above, it will take the efforts of many individuals represent- ing at least the following entities: S06 contractor’s person- nel, S07 contractors’ personnel, DAS manufacturer(s), and the participants. Quality Activities: DAS Installation and Deinstallation Processes It is important to ensure the required throughput (an aver- age of two installations per day per bay) and quality in DAS installations. Training is the quality assurance process that will be used to ensure that DASs are installed in a timely and high-quality manner. All DAS installers will be trained at VTTI facilities by experienced personnel to make sure they have the ability to maintain such a schedule. DAS units must be installed without permanent destruction or defacing of the participant’s vehicle in any way. Once the DAS is fully installed,21its functionality will be verified with VTTI-provided custom software on the installer’s laptop computer (also provided and imaged by VTTI), and the installation is not considered com- plete until this verification has been accomplished. There are two other key aspects for ensuring a high-quality installation: the design of the DAS and the installation panel and lasers. The DAS is designed to minimize the chances of misinstallation because it can only be installed in the correct way. In addition, VTTI engineers have devised sophisticated yet easy-to-implement alignment protocols and laser hard- ware to ensure correct DAS installation (see Figure 4.1).Health Check The DAS is equipped with cellular communications technol- ogy facilitating the automatic health check. This is a process whereby the DAS will automatically send a small batch of key data back to S06 servers. Automated algorithms will be applied to this batch of data to detect potential problems with DAS functionality. The health checks will be generated and sent to the commu- nications server at VTTI on a weekly basis starting on the day of installation of the DAS unit, unless some subsystem failure warrants immediate action. As a message is received by the VTTI server, it will be added to the database of stored health checks. A formula will be applied to each of the subsystems to determine whether the system should be considered func- tional (good) or whether it requires further attention. For all health checks classified as “bad” by the server process looking at communications, an issue will be immediately and auto- matically generated in the issue tracking software, Request Tracker (RT). The issue will then make its way through the triage process until it is fully resolved. Training Training is a key responsibility of the S06 contractor to facil- itate quality and consistency in S07 processes.

22Training for Site Contractors Training programs and materials will be administered to the S07 contractors. Training sessions must be conducted for DAS installers, driver assessment administrators, and others handling participant data. Also, all designated project person- nel must be IRB certified. As the project is likely to experience attrition, a train-the-trainer approach will be employed. The initial set of trainees will travel to the S06 facility for a train- ing session. DAS installers will undergo a 3-day training ses- sion, and assessment researchers will undergo a single day of training. Data handling will require a relatively small amount of training and will be incorporated within the DAS installa- tion training sessions. Subsequently, SHRP 2 and S06 person- nel will travel to the S07 sites to make certain that their facilities are suitable and procedures are consistently being applied. For each element of the training program, training criteria will be established for the individuals involved and the S07 site as a whole to determine whether training has been suc- cessful and what remediation is prescribed if the training has not met the criteria of success. Participant Processing and Assessment Training Participant processing and assessment will be done consis- tently across S07 sites. Participant processing starts from the time the participant arrives at the S07 site and continues until the time that the instrumented vehicle is returned to him/her. It covers such requirements as the initial stages of the consent process (including reviewing and signing the consent form and other documents) and setting up the participant payment schedule described in Chapter 2. Driver assessment data must be collected using a consistent set of tools and protocols to avoid introducing additional variance into the data. Specificprotocols regarding how to properly administer each of the driver assessments (and the entire experimental flow from one assessment to another, including suggested breaks and data-recording techniques) will be included in the training program. This training program will be primarily classroom based, with some hands-on use of the test equipment. DAS Installation Training In terms of installation throughput, each S07 contractor must maintain installation throughput in accordance with the pre- determined schedule (e.g., two vehicle instrumentations per work day per 150 DASs managed). DAS data quality relies on a correct installation. Also, installation must be accomplished in a manner that leaves the vehicle in a state where the par- ticipant feels comfortable with the aesthetics, and where it can eventually be returned to its preinstall condition. Train- ing materials and a generalized training program instruct- ing technicians how to install the DAS into a vehicle will be developed by the S06 contractor. The training program will include classroom, as well as hands-on, training in the garage. In addition, specific wiring diagrams and installation proce- dures will be prepared for some vehicles in the fleet. A wiki site will also be available for the sharing of lessons learned to facil- itate ease and consistency of installations. Data-Handling Training Although the data-handling process will be automated to the greatest degree feasible, there are specific steps that the S07 contractors must accomplish. S07 technicians will be trained on how to remove the hard drive from the DAS, replace the DAS with a new/refurbished hard drive, test the DAS func- tioning in the field, and use the hard drive bays to upload data to the staging server that will then automatically upload data to the S06 contractor servers. The S07 technicians will be trained on how to insert the hard drive into the reader, how to know when it is finished being processed, and how to determine when the hard drive can be put back into rotation for future use in the field. The training program will include classroom as well as hands-on training. Institutional Review Board Training Each individual who interacts with a human participant, or may come into contact with personal information or items belonging to a participant, must provide evidence of success- ful completion of standard IRB training. This would apply to experimenters performing driver assessments, managers, and DAS installers as well as crash investigators who may have contact with participants. The S06 contractor will not provide this training but will provide recommended, low- or no-Figure 4.1. Alignment panel and driver-side lateral alignment laser apparatus with left-right-center laser spots.

23cost sources for this type of training. However, some specific instruction regarding IRB issues relating specifically to natu- ralistic data collection will be developed and given to both the S07 DAS technicians and the S07 participant processors. Site Contractor Inspections During the study, with a heavy emphasis on the early stages for establishment of conformity and integrity, the S06 contractor will visit each S07 facility to assess the following areas: • Observation of full DAS installation to verify installation/ calibration integrity. • Observation of participant assessments, where such would not affect or disrupt the assessments. • Observation of S07 contractors’ activities to  Note any S07 contractor-specific anomalies;  Determine protocol compliance;  Determine ability to continue operation per protocol requirements;  Identify issues or problem areas that require corrective actions;  Provide feedback to S07 contractors regarding perfor- mance; and  Seek input regarding any areas that the S07 contractors feel the S06 contractor could improve to enhance oper- ational quality and efficiency. The S06 contractor will periodically produce a report sum- marizing the results of these observations. Continual Project Assessment Mission Control Software (MCS) will allow S06 and S07 man- agers to track study progress in a variety of ways (e.g., DAS inventory management, participant/vehicle information, and recruitment summaries). The workflow processes of partici- pant intake and DAS tracking will provide the data reflected in the MCS application. More detailed information will be viewable for each individual participant or logically aggregated group (e.g., by S07 site). This additional information includes at least the following: • Where each DAS is located (e.g., in vehicle A, or at site B, or en route to site C). • How many of those participants with a DAS installed in their vehicles have completed assessments. • Data gathered to date in terms of miles, hours, aggregate months installed, and so forth. • Participant progress by age group—can be viewed per S07 site or overall. Shows the number of participants at each stage in the process.• Data collection progress by age group—can be viewed per S07 site or overall. • Data collection progress compared to plan. • DAS kit inventory types by S07 site. • HD status. • Variable costs compared to planned expenditures. • Participants installed compared with goals by  Age group;  Gender;  12 or 24 months; and  Advanced vehicle. Intraproject Communication Standing Meetings The management plan will entail, at a minimum, routinely scheduled standing 30- to 60-minute conference calls hosted by S06 and including S07 managers. The frequency of these meetings can be adjusted up or down as the project moves forward. Topics will include, at a minimum, communication from the S06 manager on overall study issues and status as well as the same from each S07 manager, and action item assignment and follow-up. Critical Issues Committee A daily meeting time will be established where a critical issues committee can be quickly and reliably pulled together to address critical issues, which may include unexpected or abnormal situations that pose a risk to either an individual or group of participants, an individual or group of data- collection sites, or the study in its entirety. This team will be composed of principal members of the S06 contractor and TRB staff. In cases where it may be relevant, an indi- vidual or group of S07 site contractors may also be invited, although it is not expected that the site contractors will routinely be required for this meeting. Critical issues com- mittee meetings will only be called to order when there is a critical issue that requires the committee’s attention. How- ever, the time slot should be kept available, as often as possi- ble, in the event it is required. Also, only committee members who need to address the issue at hand will be required to attend. A process will be used to determine whether or not to convene a Critical Issues Committee meeting and whom to invite (Figure 4.2). Issue-Tracking Software The issue-tracking software, Request Tracker (RT), will be in place for the NDS. This is necessary to successfully and con- sistently manage issues that arise during the course of the

24Anomalous situation comes to attention of S06 oversight contractor Does situation threaten the viability of the entire project? Situation classification process Does situation threaten the viability of a single site? Does situation qualify as an IRB adverse event? Does situation threaten the data? Example: Negative national publicity raises ire of Congressional leaders Example: Site loses all or most key employees in a short period of time Example: A si te contractor uses coercion to recruit family members Example: Catastrophic hacking event Meeting of Critical Issues Committee to enact/develop contingency plan to save project Notify relevant IR B(s) of adverse event; act on their directions Meeting of Critical Issues Committee to enact/develop contingency plan to save or terminate site Meeting of Critical Issues Committee and dB experts to determine extent of data exposure, develop strategy Yes Yes No No Yes No Handle with normal procedures (not an anomalous event) Still consider possibility of adverse reportable IRB event Yes No Figure 4.2. Critical issues management.study. The key elements are the ability to track, assign, and report on issues that otherwise may be lost in formats not eas- ily or systematically tracked, such as e-mail, phone calls, and messages, or Post-it notes. Issues can be created by any author- ized user (e.g., TRB staff, S06 and S07 personnel), and every issue will be assigned to an appropriate individual to resolve in a timely manner. Issue status will be tracked throughout its life cycle from creation through resolution. Via this approach, reports can be generated on issues by any combination of ele- ments, such as S07 site, issue category (e.g., DAS hardware problem), participant, issue status, and creation date. It is anticipated that the issue tracker will also yield valuable expe- rience that can be used to populate a knowledge base or wiki that will be available to and beneficial to all project personnel.