Click for next page ( 22


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 21
C H A P T E R 4 Quality Processes Quality Goals and Objectives its functionality will be verified with VTTI-provided custom software on the installer's laptop computer (also provided and The overall goal of SHRP 2 NDS data quality processes is to imaged by VTTI), and the installation is not considered com- work within budgetary and scheduling constraints to obtain plete until this verification has been accomplished. the most complete and highest-quality data set possible. More There are two other key aspects for ensuring a high-quality detailed quality objectives include gathering data from a high- installation: the design of the DAS and the installation panel quality sample, gathering the full complement of data expected, and lasers. The DAS is designed to minimize the chances of gathering intended data efficiently, storing intended data misinstallation because it can only be installed in the correct securely, and maintaining data privacy and security. way. In addition, VTTI engineers have devised sophisticated The specific data quality processes have a threefold purpose, yet easy-to-implement alignment protocols and laser hard- as follows: ware to ensure correct DAS installation (see Figure 4.1). 1. To identify and preclude or reduce errors at their source; 2. To identify DAS system failures to initiate prompt main- Health Check tenance actions during data collection; and The DAS is equipped with cellular communications technol- 3. To mitigate the effects of data collection errors by assur- ogy facilitating the automatic health check. This is a process ing that the data entering the database are as high-quality whereby the DAS will automatically send a small batch of key as possible. data back to S06 servers. Automated algorithms will be applied to this batch of data to detect potential problems with DAS To implement the deceptively simple quality policy noted functionality. above, it will take the efforts of many individuals represent- The health checks will be generated and sent to the commu- ing at least the following entities: S06 contractor's person- nications server at VTTI on a weekly basis starting on the day nel, S07 contractors' personnel, DAS manufacturer(s), and of installation of the DAS unit, unless some subsystem failure the participants. warrants immediate action. As a message is received by the VTTI server, it will be added to the database of stored health Quality Activities: checks. A formula will be applied to each of the subsystems to DAS Installation and determine whether the system should be considered func- Deinstallation Processes tional (good) or whether it requires further attention. For all health checks classified as "bad" by the server process looking It is important to ensure the required throughput (an aver- at communications, an issue will be immediately and auto- age of two installations per day per bay) and quality in DAS matically generated in the issue tracking software, Request installations. Training is the quality assurance process that Tracker (RT). The issue will then make its way through the will be used to ensure that DASs are installed in a timely and triage process until it is fully resolved. high-quality manner. All DAS installers will be trained at VTTI facilities by experienced personnel to make sure they Training have the ability to maintain such a schedule. DAS units must be installed without permanent destruction or defacing of the Training is a key responsibility of the S06 contractor to facil- participant's vehicle in any way. Once the DAS is fully installed, itate quality and consistency in S07 processes. 21

OCR for page 21
22 protocols regarding how to properly administer each of the driver assessments (and the entire experimental flow from one assessment to another, including suggested breaks and data-recording techniques) will be included in the training program. This training program will be primarily classroom based, with some hands-on use of the test equipment. DAS Installation Training In terms of installation throughput, each S07 contractor must maintain installation throughput in accordance with the pre- determined schedule (e.g., two vehicle instrumentations per work day per 150 DASs managed). DAS data quality relies on a correct installation. Also, installation must be accomplished Figure 4.1. Alignment panel and driver-side lateral in a manner that leaves the vehicle in a state where the par- alignment laser apparatus with left-right-center ticipant feels comfortable with the aesthetics, and where it laser spots. can eventually be returned to its preinstall condition. Train- ing materials and a generalized training program instruct- ing technicians how to install the DAS into a vehicle will be Training for Site Contractors developed by the S06 contractor. The training program will Training programs and materials will be administered to the include classroom, as well as hands-on, training in the garage. S07 contractors. Training sessions must be conducted for In addition, specific wiring diagrams and installation proce- DAS installers, driver assessment administrators, and others dures will be prepared for some vehicles in the fleet. A wiki site handling participant data. Also, all designated project person- will also be available for the sharing of lessons learned to facil- nel must be IRB certified. As the project is likely to experience itate ease and consistency of installations. attrition, a train-the-trainer approach will be employed. The initial set of trainees will travel to the S06 facility for a train- Data-Handling Training ing session. DAS installers will undergo a 3-day training ses- sion, and assessment researchers will undergo a single day of Although the data-handling process will be automated to the training. Data handling will require a relatively small amount greatest degree feasible, there are specific steps that the S07 of training and will be incorporated within the DAS installa- contractors must accomplish. S07 technicians will be trained tion training sessions. Subsequently, SHRP 2 and S06 person- on how to remove the hard drive from the DAS, replace the nel will travel to the S07 sites to make certain that their facilities DAS with a new/refurbished hard drive, test the DAS func- are suitable and procedures are consistently being applied. tioning in the field, and use the hard drive bays to upload data For each element of the training program, training criteria to the staging server that will then automatically upload data to will be established for the individuals involved and the S07 the S06 contractor servers. The S07 technicians will be trained site as a whole to determine whether training has been suc- on how to insert the hard drive into the reader, how to know cessful and what remediation is prescribed if the training has when it is finished being processed, and how to determine not met the criteria of success. when the hard drive can be put back into rotation for future use in the field. The training program will include classroom as well as hands-on training. Participant Processing and Assessment Training Participant processing and assessment will be done consis- Institutional Review Board Training tently across S07 sites. Participant processing starts from the time the participant arrives at the S07 site and continues until Each individual who interacts with a human participant, or the time that the instrumented vehicle is returned to him/her. may come into contact with personal information or items It covers such requirements as the initial stages of the consent belonging to a participant, must provide evidence of success- process (including reviewing and signing the consent form ful completion of standard IRB training. This would apply to and other documents) and setting up the participant payment experimenters performing driver assessments, managers, and schedule described in Chapter 2. Driver assessment data must DAS installers as well as crash investigators who may have be collected using a consistent set of tools and protocols to contact with participants. The S06 contractor will not provide avoid introducing additional variance into the data. Specific this training but will provide recommended, low- or no-

OCR for page 21
23 cost sources for this type of training. However, some specific Data collection progress by age group--can be viewed per instruction regarding IRB issues relating specifically to natu- S07 site or overall. ralistic data collection will be developed and given to both Data collection progress compared to plan. the S07 DAS technicians and the S07 participant processors. DAS kit inventory types by S07 site. HD status. Variable costs compared to planned expenditures. Site Contractor Inspections Participants installed compared with goals by During the study, with a heavy emphasis on the early stages for Age group; establishment of conformity and integrity, the S06 contractor Gender; will visit each S07 facility to assess the following areas: 12 or 24 months; and Advanced vehicle. Observation of full DAS installation to verify installation/ calibration integrity. Intraproject Communication Observation of participant assessments, where such would not affect or disrupt the assessments. Standing Meetings Observation of S07 contractors' activities to The management plan will entail, at a minimum, routinely Note any S07 contractor-specific anomalies; Determine protocol compliance; scheduled standing 30- to 60-minute conference calls hosted Determine ability to continue operation per protocol by S06 and including S07 managers. The frequency of these requirements; meetings can be adjusted up or down as the project moves Identify issues or problem areas that require corrective forward. Topics will include, at a minimum, communication actions; from the S06 manager on overall study issues and status as Provide feedback to S07 contractors regarding perfor- well as the same from each S07 manager, and action item mance; and assignment and follow-up. Seek input regarding any areas that the S07 contractors feel the S06 contractor could improve to enhance oper- Critical Issues Committee ational quality and efficiency. A daily meeting time will be established where a critical The S06 contractor will periodically produce a report sum- issues committee can be quickly and reliably pulled together marizing the results of these observations. to address critical issues, which may include unexpected or abnormal situations that pose a risk to either an individual or group of participants, an individual or group of data- Continual Project Assessment collection sites, or the study in its entirety. This team will Mission Control Software (MCS) will allow S06 and S07 man- be composed of principal members of the S06 contractor agers to track study progress in a variety of ways (e.g., DAS and TRB staff. In cases where it may be relevant, an indi- inventory management, participant/vehicle information, and vidual or group of S07 site contractors may also be invited, recruitment summaries). The workflow processes of partici- although it is not expected that the site contractors will pant intake and DAS tracking will provide the data reflected routinely be required for this meeting. Critical issues com- in the MCS application. More detailed information will be mittee meetings will only be called to order when there is a viewable for each individual participant or logically aggregated critical issue that requires the committee's attention. How- group (e.g., by S07 site). This additional information includes ever, the time slot should be kept available, as often as possi- at least the following: ble, in the event it is required. Also, only committee members who need to address the issue at hand will be required to Where each DAS is located (e.g., in vehicle A, or at site B, attend. A process will be used to determine whether or not or en route to site C). to convene a Critical Issues Committee meeting and whom How many of those participants with a DAS installed in to invite (Figure 4.2). their vehicles have completed assessments. Data gathered to date in terms of miles, hours, aggregate Issue-Tracking Software months installed, and so forth. Participant progress by age group--can be viewed per S07 The issue-tracking software, Request Tracker (RT), will be in site or overall. Shows the number of participants at each place for the NDS. This is necessary to successfully and con- stage in the process. sistently manage issues that arise during the course of the

OCR for page 21
24 Anomalous situation comes to attention of S06 oversight Handle with normal contractor procedures (not an anomalous event) Situation classification No process Does situation Does situation Does situation Does situation threaten the viability of No threaten the viability of No qualify as an IR B No threaten the data? the entire project? a single site? adverse event? Example: Negative Example: Site loses Ex ample: A si te national publicity Example: all or most key contractor uses raises ire of Catastrophic hacking employees in a short coercion to recruit Congressional event period of time family members leaders Yes Yes Yes Yes Meeting of Critical Meeting of Critical Meeting of Critical Issues Committee to Issues Committee to Notify relevant IR B(s) of Issues Committee and enact/develop enact/develop adverse event; act on dB experts to determine contingency plan to contingency plan to their directions extent of data exposure, save project save or terminate site develop strategy Still consider possibility of adverse reportable IR B event Figure 4.2. Critical issues management. study. The key elements are the ability to track, assign, and life cycle from creation through resolution. Via this approach, report on issues that otherwise may be lost in formats not eas- reports can be generated on issues by any combination of ele- ily or systematically tracked, such as e-mail, phone calls, and ments, such as S07 site, issue category (e.g., DAS hardware messages, or Post-it notes. Issues can be created by any author- problem), participant, issue status, and creation date. It is ized user (e.g., TRB staff, S06 and S07 personnel), and every anticipated that the issue tracker will also yield valuable expe- issue will be assigned to an appropriate individual to resolve rience that can be used to populate a knowledge base or wiki in a timely manner. Issue status will be tracked throughout its that will be available to and beneficial to all project personnel.