Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
1Executive Summary Introduction The Virginia Tech Transportation Institute (VTTI) served as the Coordination Contractor for the SHRP 2 Naturalistic Driving Study (NDS). In this role, VTTI led the implementation of the framework, which had been developed in the foregoing study design project described by Antin et al. (2011). The ambitious goal of the effort documented in this report was to collect and archive the largest store of naturalistic driving data ever attempted. The scope of this study ranks among the largest and most comprehensive of any driving-based research study conducted to date, facili- tated by recent advances in camera and sensor technologies along with similar advances in the collection, movement, and secure storage of âbigâ data. The overarching objective was to coordinate and oversee participant- and vehicle-based opera- tions managed by six different groups of site contractors at six unique and geographically distrib- uted data collection sites. The final set of selected sites and the manner in which the 1,950 data acquisition system (DAS) units were allocated to each one is illustrated in Figure ES.1. The sites were managed by the following site contractors: ⢠Buffalo, New York: CUBRC, Inc.; ⢠Tampa, Florida: Center for Urban Transportation ResearchâUniversity of South Florida (CUTR-USF); ⢠Seattle, Washington: Battelle Memorial Institute (Battelle); ⢠Durham, North Carolina: Westat; ⢠Bloomington: Indiana University; and ⢠State College: Pennsylvania State University. The desired results included the collection of not only the naturalistic driving data but a variety of associated participant, vehicle, and crash-related data as well. Collected data were stored securely in a manner that protected the rights and privacy of the more than 3,000 participants enrolled in the study. The major task efforts fell into six key categories: human subjects protection, DAS management, system integration, supporting activities for the site contractor efforts, quality control and over- sight, and reporting. The first two are detailed here; see Chapter 1 for more about all six categories. Human Subjects Protections Categories of Participants There were two types of participants: (1) Primary participants were the main focus of the study, and recruitment efforts focused on getting the targeted mix in the various age group and gender
2categories. Primary participants consented to have data collected from their main vehicle when- ever it was driven during their participation in the study. They also underwent a broad set of functional assessments. Note that primary participants who were minors at the time of study enrollment provided assent to participate, but consent for their participation was provided by a parent. (2) Secondary participants were other adults who regularly drove a primary participantâs instrumented vehicle and granted consent to have their data analyzed. These drivers were asked to provide a reference image for driver identification purposes, and they were asked to fill in two brief surveys. For example, a spouse may have chosen to grant consent to become a secondary participant, but a hotel valet would not qualify. Both categories of participants were compensated, with secondary participants receiving a more modest compensation for their less-demanding role in the study. Any other driver of the vehicle was not considered to be a participant, and any data collected during trips when such individuals were driving have been expunged from the data set. Certificate of Confidentiality A Certificate of Confidentiality was secured from the National Institute of Mental Health (NIMH) for the SHRP 2 NDS. The Certificate of Confidentiality protects personally identifying data col- lected during the approved data collection period (i.e., from fall 2010 through the end of 2013). The certificate helps researchers protect the privacy of participant data against compulsory legal demands (e.g., court orders and subpoenas) that seek the name or other identifying character- istics of a research subject. This protection was crucial in that it gave prospective participants confidence that the data collected would not be used against them; without such protection in place, it is felt that recruitment would have been a much more daunting exercise. The protections provided by the Certificate of Confidentiality extend for the life of the data. Design of the DAS To comply with the requirements of in-vehicle performance, ease of handling, and ease of installation, the DAS was a comprehensive custom design. The DAS incorporated six primary components: NextGen main unit, head unit (HU), network box, radar, radar interface box (RIB), and solid-state data drive (Figure ES.2). The NextGen main unit housed the computing engine for the system, the electronics for which were encased in a rugged plastic enclosure with room for the solid-state drive (SSD) on which data were initially stored. DAS components were installed as indicated in Figure ES.3. A total of 2,085 DAS kits was purchased for the SHRP 2 Figure ES.1. Site locations and nominal DAS allocations of 1,950 total units.
Figure ES.2. DAS kit components (from upper left and moving clockwise: NextGen main unit, head unit, network box, RIB, and radar assembly). Front Turn Signals Radar Interface Box Radar Unit PWR Bluetooth PWR DAS Main Unit OBD Connector Head Unit Sub-Head Unit GPRS Antenna Rear Camera Figure ES.3. Typical in-vehicle locations of DAS components. Figure ES.4. Quad image of video views. (Note: Driver in image is a nonparticipant employed by Coordination Contractor.)
4NDS; additional quantities of several strategically chosen parts were purchased for logistical or replacement purposes. DAS sensors and capabilities included the following: ⢠Multiple video and still views; ⢠Machine-vision-based applications; ⢠Accelerometers (x, y, and z axes); ⢠Rate sensors (x, y, and z axes); ⢠GPS; ⢠Forward radar; ⢠Illuminance sensor; ⢠Passive cabin alcohol presence sensor; ⢠Incident pushbutton; ⢠Turn signal state; and ⢠Vehicle network data (as available). Video output included a four-quadrant image of the video data. The upper left quadrant fea- tures a color view of the forward roadway. The upper right quadrant features a monochrome image of the driverâs face and driver-side views (rotated 90 degrees to maximize use of available pixels but seen in its correct orientation during analysis). The bottom right quadrant features a right-rear view, while the bottom left quadrant captures a view of the driverâs interactions with the steering wheel and the center stack. Figure ES.4 demonstrates the quad view of the video images. A cabin snapshot was also recorded once every 10 minutes during a drive. This snapshot was irrevocably blurred at the time of collection and is intended to be used to determine, as possible, the number and other basic characteristics of passengers (e.g., approximate age and/or gender; see Figure ES.5). Data Collection Site Facilitation The Coordination Contractor was responsible for technical coordination of the six data collection sites. The preparation activities needed to support ongoing coordination efforts included assess- ing and certifying each site for readiness to collect data, training site contractor personnel, and providing software tools to assist them in managing their DAS kit installations, maintenance, and deinstallation. Software was also provided to help the site contractors manage their inventory, participants, and vehicle fleets. Training To ensure consistency across the six data collection sites, the Coordination Contractor provided in-depth training to site contractor personnel on all study protocols, including human subjects protection standards, encompassing ethics and special situations; enrollment, including providing informed consent; the collection of participant functional assessment data; and the installation, maintenance, and deinstallation of DAS kits (Figure ES.6). Sample Design The high-level goal for the sampling plan was to recruit an equal number of male and female licensed drivers across the full breadth of the driver age spectrum. It was also a goal to over sample the youngest and oldest drivers, as these are the most interesting due to prior indications of
5Figure ES.6. Hands-on DAS installation training. Figure ES.5. Blurred cabin image. (Note: Driver in image is a nonparticipant employed by Coordination Contractor.)
6elevated crash risk. More than 18,000 individuals were recruited using a variety of approaches, and more than 3,000 ended up participating for 4 months or more. Network Data Classification Initial difficulty in recruiting younger and older drivers compelled the Coordination Contrac- tor, in cooperation with SHRP 2 program managers, to expand the vehicle fleet to include older vehicle-years for which a less robust set of vehicle network data was available. The enlargement of the vehicle fleet necessitated the creation of four distinct vehicle classes with three discrete hard- ware installation configurations: prime, subprime, legacy, and basic, as described in Table ES.1. The parameter identification (PID) refers to the codes used to interpret the network data. Reports A wide variety of reports was generated by the Coordination Contractor on weekly, monthly, or quarterly bases; many were produced on an ad hoc basis. These reports provided crucial infor- mation on the current and projected status of key progress metrics that continually provided the information needed to help leadership guide the study toward its ultimate goals. Operations Metrics An operations metrics report was prepared weekly and circulated among Coordination Contrac- tor staff to present a glimpse into current study operations. The report encompassed all aspects of the study, including data quality, data ingestion progress, fleet communications status, counts of participants and vehicles installed in the past week, solid-state drive status, outstanding main- tenance items, inventory counts, and statistics pertaining to fleet issues of particular interest. The counts dictated the work activities each week, and the associated spreadsheets provided a road map for the completion of that work, identifying vehicles with communications issues, video quality problems, or nearly full data drives. Data Volume Reports One metric of the total quantity of data collected was the number of vehicle-months. According to this concept, each participant contributed one vehicle-month of data for each full month of Table ES.1. SHRP 2 NDS Vehicle Classes Class Vehicles Included Vehicle Network Information Collected Vehicle Count Percentage of Fleet Prime Vehicles for which PIDs were available Speed, plus wiper activation, brake actuation, headlight activation, turn signal acti- vation, and steering data, as available 1,717 51% Subprime Generally vehicles manufactured after 2009 for which PIDs were not available Speed and accelerator position 488 15% Legacy Vehicles manufactured between 1996 and 2008 Speed and accelerator position 736 22% Basic Vehicles manufactured before 1996 without vehicle networks None 421 13% Total 3,362 100%
7participation in the study, regardless of the actual number of driving miles or hours represented during that time period. It is true to state, on the one hand, that driving distance and time are more accurate ways to determine overall data quantity, but these were also more difficult to accu- rately capture or estimate during the conduct of the study. The total number of vehicle-months collected, on the other hand, could be much more readily calculated at any given point in time for any desired subset of the data (e.g., by age group, gender, and/or data collection site). In addition, when aggregated over a large number of drivers, the vehicle-months metric closely approximates the accuracy of driving time or distance metrics. These reports provided weekly information on study progress and important feedback, which guided recruiting strategy. Data Ingestion and Protections Data Ingestion Process Data ingestion involved the movement of data from the vehicle to secure storage on Coordina- tion Contractor servers. First, data-filled solid-state drives were harvested from study vehicles by site contractor technicians when the health check indicated the drives were at 70% or more of capacity (or when a convenient opportunity otherwise arose). These drives (up to five at a time) were then physically placed into custom-designed drive bays connected to staging servers at each site contractorâs facility. At that point, the staging server commenced automatic upload of the data from each inserted drive. Once a driveâs data were uploaded to the staging server, the drive was provisioned so that it could be recirculated into the next vehicle needing a fresh drive at that site. Data on the staging server were then automatically transmitted via the Internet2 high-speed research network to servers at the Coordination Contractor facility. At that time, a copy of the data was made for processing, and the original encrypted files were sent to permanent archival storage for the duration of the lifetime of the data. The copied data remained in their original encrypted state until they were queued for processing in the workflow system. At that time, the data were decrypted to perform any transformations required before loading the data set into its ultimate repository. Figure ES.7 illustrates the transfer of data from the site contractor staging servers to servers at the Coordination Contractor facility. Data Protections SHRP 2 data were protected from the moment they were collected and throughout their migra- tion from the vehicle into the final research repository. In addition, data were stored âas collectedâ in a modern peta-scale hierarchical storage management (HSM) system where an archival copy was maintained in the HSMâs tape library. The first line of protection started on the DAS with a sophisticated data encryption process. Once data were transferred, decrypted, and ingested, they were protected by role-based security. That security limited usersâ access based on either their Institutional Review Board (IRB) approvalsâin the case of access to personally identifying information (PII)âor their need for access to data elements required to address research ques- tions. Additionally, multiple copies of SHRP 2 data were maintained at separate facilities within the same locality, in case one facility was to suffer a disaster of any sort. Data Quality Processes With any study, it is imperative to not only continually monitor but also work to ensure that the data in the database are as high quality as possible in terms of completeness and accuracy. To that end, data checks were applied to ensure data were being collected appropriately and meeting the expected high level of quality. These checks are described below.
8Sensor Data Once ingested into the database, data underwent a standardization process and a subsequent battery of automated quality checks as follows: ⢠Not present indicates whether at least one data point was captured for a variable within a particular file. If a variable was not present for an entire file, no other checks for that variable were necessary. ⢠Bounds indicates whether the values recorded for a given variable were within the bounds defined in the relevant data dictionary available at the SHRP 2 Data Access website (https://insight .shrp2nds.us). Boundary values (i.e., lower, upper, both) could be specified independently for each variable. ⢠Simple dependency indicates whether the dependent variable (i.e., the variable being checked) should be considered of questionable quality given that a âparentâ variable had failed one or more of its quality checks. These comparisons were made on a timestamp-by-timestamp basis. Each simple dependency consisted of only one dependent and one independent variable, but more than one simple dependency could be applied to a single dependent variable. For example, one of the quality metrics for the processed accelerometer values considered whether the cor- responding raw values exhibited good quality during the same time period. Site Contractor Staging Server AUTOCOPY FILE TRANSFER Coordination Contractor Servers Archival Storage (Encrypted files) High-speed Research Network (Internet) Copy for later processing (Encrypted) Figure ES.7. Data collection and ingestion workflow.
9⢠Complex dependency is similar to a simple dependency but with more complex conditions allowed. While a simple dependency was a function of the independent variable having âgoodâ quality when the dependent variable was collected, a complex dependency could further refine what values of the independent variable indicated âgoodâ quality for the dependent variable. Each complex dependency consisted of only one dependent and one independent variable, but multiple complex dependencies could be applied to a single dependent variable. For example, a check for any variable collected from the vehicle network modules required that the last reported status for that module indicated a ârecordingâ status to output a good quality score. ⢠Duplicates indicates whether a particular variable had two entries on the collected data under the same timestamp. If that was the case, the data quality for the timestamp in which this occurred was considered âbad.â ⢠Spike identification indicates whether a data point that was otherwise within the expected bounds for the variable should be considered experimental noise, typically due to sensor noise. This particular check was used for longitudinal and lateral accelerations. The code examined preceding and following values around the suspected spike and assessed whether the overall pattern was feasible based on the expected physics of the scenario. Multiple metrics were used in this assessment, including the derivative of acceleration, the variance in the sample, and measures from basic principles of motion. Video Data Part of these subsequent analyses entailed a manual review of images transmitted via Advanced Health Checks, with an eye toward identifying specific vehicles in need of camera adjustment or replacement. Health checks were provided via periodic transmissions from the DAS to the database with an accompanying notification and included gross performance metrics related to select sensors and cameras. Table ES.2 summarizes the standards to which each camera view was held for the quality review. Table ES.2. Camera ViewsâIdeal Descriptions and Purposes Camera View Ideal Purpose Face camera Complete, clear view of the driverâs face, including eyes and mouth. Camera should be positioned to exclude views of backseat passengers. A clear view of the face facilitates eye- glance analysis and evaluation of distraction associated with secondary tasks of talking, eating, and singing. Forward camera High-quality, color video of the forward roadway. Forward road and traffic, traffic lights, and cars in front should be visible, with roadway centered horizontally and with the horizon just above the center line. A clear view of the forward roadway facili- tates evaluation of traffic density, visibil- ity, road conditions, and time of day, as well as recognition of potential hazards posed by oncoming traffic and activities of drivers in surrounding vehicles. Instrument panel High-quality video of the distance from the driverâs door to the center console, featuring a complete view of both of the driverâs hands and steering wheel, radio/CD player/cigarette lighter, and center console. A clear view of the hands and center console facilitates analysis of distrac- tions resulting from secondary tasks such as adjusting cabin temperature or radio, using a cell phone, and reaching for objects. Rear camera High-quality video of the traveled road- way. Traveled roadway, following traffic, and traffic lights should be visible, with roadway centered horizontally and with horizon just above center vertically. A clear view of the traveled roadway facilitates analysis of traffic density and potential hazards posed by following traffic.
10 Video data review of a sample of 10 trip files per month per participant was undertaken by a team of trained data reductionists under a protocol that elicited a quality assessment for each of the four camera views: face, forward, instrument panel, and rear. The quality assessment for each view was selected from one of four options, defined as follows: ⢠Good quality: Video is clear, viewable, and correctly aligned. ⢠Misaligned video: Video is misaligned from target (i.e., pointing in the wrong direction). ⢠Distorted: Video is available but not usable for research purposes. ⢠Not available: Video is unavailable. Non-DAS Data In addition to driving- and vehicle-related data collected via the installed data acquisition equip- ment, a variety of non-DAS data were also procured, including ⢠Basic demographic information; ⢠Functional ability relative to driving safety and risk; ⢠Vision tests; ⢠Cognitive assessments; ⢠Physical ability metrics; ⢠Vehicle information; and ⢠Post hoc crash investigations. These non-DAS data were obtained through a variety of instruments, including questionnaires; assessments of physical acumen, cognitive capacity, and visual acuity; and participant interviews. Assuring the quality of the time series data and video collected via the DAS was a central focus of the overall quality efforts; but considerable effort was also devoted to assuring the quality of the many non-DAS sources of data (e.g., questionnaire and visual field data collected from each driver). Several approaches were used to identify outliers, including applying basic knowledge of the data when applicable (e.g., for male and female heights and weights). In the absence of such baseline knowledge, a statistical outliers approach was employed such that extreme values were distrusted and discarded, except when independent verification suggested otherwise. With this interquartile range (IQR) approach, any value ⤠[Q1 - (1.5 à IQR)] or ⥠[Q3 + (1.5 à IQR)] was considered an outlier, where Q1 = first quartile, Q3 = third quartile, and IQR = interquartile range or (Q3 - Q1) for the particular variable distribution in question. When excluding extreme values, it was decided to implement methods that would tend to eliminate only the most obviously extreme values. Study Metrics by Site Progress at each of the six data collection sites was measured by a variety of metrics, including the number of primary and secondary participants enrolled and vehicles instrumented. Participant periods of study participation varied from as little as a single day to as long as 3 years. For the purpose of participant counts provided here, a criterion of at least 4 months has been applied. Primary Participants Table ES.3 presents the number of primary participants with a minimum of 4 months in the study across the six data collection sites. Vehicle Installations Figure ES.8 depicts the growth of the SHRP 2 fleet over the course of 38 months of data collection.
11 Participant-Related Outcomes Primary Participants by Age Figure ES.9 presents the total number of primary participants across age groups, with the hori- zontal line indicating the original study targets for each age group as per the original sample design. Vehicle-Related Outcomes Vehicles constituting the SHRP 2 fleet were further classified according to a number of parameters, including vehicle type and manufacturer. Types Figure ES.10 depicts the vehicle fleet by type. Table ES.3. Primary Participants with 4+ Months in Study Site Primary Participant Count Buffalo, New York 719 Tampa, Florida 698 Seattle, Washington 676 Durham, North Carolina 504 Bloomington, Indiana 239 State College, Pennsylvania 256 Total 3,092 Figure ES.8. Number of installed vehicles over time.
12 Manufacturer Figure ES.11 presents a view of the SHRP 2 vehicle fleet broken down by manufacturer. Crashes Five hundred thirty-two possible crash events were identified in the data set. Potential crashes were discovered via a variety of means, including participant reports, automatic crash notifica- tion (ACN) algorithms on the DAS, and similar ACN algorithms run on ingested data. Once identified as possible events, classification as actual crashes was verified via video review. Each time it was established that a potential crash had occurred, the event was assigned to one of four severity categories decreasing in severity from Level 1 to Level 4: ⢠Level 1: Airbag/injury/rollover, high delta-V crashes (virtually all would be police reported); ⢠Level 2: Police-reportable crashes (including police-reported crashes, as well as others of similar severity which were not reported); ⢠Level 3: Crashes involving physical contact with another object; and ⢠Level 4: Tire strike; low-risk crashes. Figure ES.12 presents the number of verified crashes across severity categories. The pyramid shape is employed to reinforce the basic and fortunate truth that as crash severity increases, the Figure ES.9. Primary participants across age groups relative to original targets (horizontal line). Figure ES.10. SHRP 2 vehicle fleet by type.
13 frequency of occurrence diminishes. Thus far, more than 500 possible crashes have been identi- fied, so the numbers in Figure ES.12 are expected to increase as the verification process continues. Cell Phone Records Study The Cell Phone Records Study (CPRS) was commissioned as the first follow-on to the SHRP 2 NDS. Participants were asked when exiting the driving study whether or not they would agree to participate in future studies. Those who did agree were then immediately asked if they would like to participate in the CPRS by allowing researchers access to specific aspects of their calling and texting records for the duration of their participation in the driving study. In this way, the driving and cell phone records could be more easily matched based on universal time syncs, indicatingâwith video verificationâwhich trips might have included the use of cell phones. The data being collected were limited to the date, time, and duration of calls, origin of the call (participant or other), and the date, sent time, and origin of text messages (including picture or video messages, as available). In no case was the content of a call or text message captured Figure ES.11. SHRP 2 fleet distribution by manufacturer. Figure ES.12. Evaluated crash events by crash severity level.
14 (including picture or video text messages) nor was the identity or number of the other person engaged in the call with the participant. To be eligible for the CPRS, participants had to be 18 years old or older and able to access a minimum of 3 months of their cell phone records overlapping their participation in the driving study. Use of minors in a study requires parental consent in addition to the minorâs assent, both of which typically must be given in person to ensure freedom from parental coercion. The CPRS design called for consent to be provided remotely via mail. Therefore, including minors in the CPRS was deemed infeasible; however, younger participants expressing a willingness to be con- tacted regarding participation in follow-on studies were invited to participate once they reached the age of consent. Figure ES.13 presents the total number of participants, both primary and secondary, versus the number of participants who agreed to be contacted for future studies and the number of participants who agreed to participate in the Cell Phone Records Study, respectively. Conclusions The driving study resulted in the successful collection of two petabytes of real-world driving video and sensor data from more than 3,000 participants over a 3-year period between October 2010 and December 2013. This data set includes some 50 million miles of travel and well over a million hours of naturalistic driving data. The participant pool consisted of individuals aged 16 to 98, with an approximately equal mix of males and females. The overarching goal was to collect a very large, extremely rich, and detailed store of data, which is expected to be mined and analyzed by a generation of transportation safety researchers and others attempting to answer many of the key traffic safety-related questions of today and well into the future. Figure ES.13. Total participants versus participants who agreed to be contacted for future studies.
15 Lessons Learned The SHRP 2 NDS has already led to a number of operational observations that should prove helpful to research teams undertaking similar ventures in the future, specifically in the areas of securing necessary human subjects protections, site-based facilitation, equipment management, and participant recruitment and management. Securing initial IRB approvals of study protocols and materials, and the approvals of subsequent amendments, took a great deal more time, money, and effort than was initially anticipated. This was greatly exacerbated by the need to deal with the sometimes differing requirements of multiple IRBs. Allocating sufficient resources to this considerable task is essential before, during, and after the data collection period. Likewise, a plan for keeping the IRB amendment process in constant motion to address unforeseen events is crucial in maintaining fluidity in the conduct of the study. In short, expect the unexpected and plan accordingly. Adequate time must be built into the project timeline to allow for unavoidable delays in manu- facturing and delivery of components as well as for equipment repairs. Likewise, adequate time and resources must be devoted to design modifications, such as the one necessitated by the addi- tion of legacy vehicles to the vehicle fleet. Such design adjustments can result in kinks in the sup- ply chain that appreciably impede study progress. Ideally, the entity providing oversight for the project would have a sizable inventory from which to draw and a plan for necessary repairs and modifications in advance of the commencement of installations. Once installations are under way, a real-time mechanism for monitoring activities and inventory levels at remote sites is essential to adequately provision each and respond to inevitable fluctuations in demand. In terms of participant recruitment, researchers must be prepared to make adjustments to com- pensation schemes and recruitment strategies to successfully meet the challenges of recruiting for this type of study, especially if the goals include recruiting younger and older drivers or other special population subsets. The challenge of managing over 3,000 participants is by no means limited to recruitment. Managing a participant pool of this size, even employing a site-based model as was done here, requires equal measures of flexibility and creativity. Addressing prob- lems of participant recalcitrance (i.e., not responding to repeated communications), responding to participant requests for access to some portion of their video data, and securing consent and necessary reference images (especially from secondary drivers with whom study personnel have little, if any, actual contact) are issues researchers must thoughtfully prepare to confront. Another consideration when planning the project timeline is whether to conduct initial data analyses concurrent with data collection or after its conclusion. SHRP 2âs Safety Technical Coor- dinating Committee authorized initial analyses before the completion of data collection and pro- cessing because sufficient time was not available for SHRP 2 to conduct any analysis otherwise. Four analysis projects were funded under SHRP 2 Project S08. While some degree of analysis is necessary to ensure proper functioning of study equipment and to assure stakeholders that suitable data are being collected, SHRP 2âs experience suggests it is preferable to start providing data access for researchers only after the data set is complete. Data analysis on an incomplete and constantly changing data set is frustrating for the analyst and could possibly result in misleading conclusions. In addition, data sharing while the data set is being built is also disruptive to and inefficient for the data management process. In the end, researchers are not satisfied, the useful- ness of the data is misrepresented, and database completion is delayed.
