The Iowa Driving Simulator: Using Simulation for Human Performance Measurement
Ginger S. Watson1and Yiannis E. Papelis
This paper describes the Iowa Driving Simulator (IDS), a high-fidelity driving simulator, and relates its use to human performance measurement. This virtual reality device is currently in use for research in mechanical engineering, transportation, and medical disciplines. The IDS currently is considered one of the highest-fidelity driving simulators in the United States. Application of the device in the engineering field includes virtual prototyping, a process by which the design of new vehicles is accelerated through use of the simulator in lieu of a physical prototype. The device has been used to investigate human factors for new highway technologies, in-vehicle devices, and licensure issues, such as fitness to drive.
The paper is structured in two sections. The first section provides a technical overview of the device, along with specification for its hardware and soft-
ware components. The second section discusses some of the authors' experiences with the IDS.
Driving is one of the most complex tasks with which the majority of the population is familiar. Despite its commonality, driving is a rather complex task that involves quick decision making, significant motor skills, and the ability to quickly process information from a variety of sources. To simulate this task faithfully in a simulator requires extensive realism and fidelity with respect to several factors, including the hardware providing vibration, sound, and visual stimuli involved in driving a vehicle, the performance characteristics associated with the vehicle, and the external virtual environment in which a simulation takes place. In addition to simply reproducing these cues for the driver of the simulator, several other requirements are necessary for a simulator used to conduct scientific research. These requirements include an external environment that can be customized to meet study requirements; a realistic yet repeatable set of behaviors of the surrounding area; extensive data collection and analysis capabilities; and from an engineering standpoint, strict determinism and repeatability in the execution of the overall system.
IDS TECHNICAL OVERVIEW
The IDS (Freeman et al., 1995; Kuhl et al., 1995) is a high-fidelity driving simulator located at the University of Iowa's Center for Computer-Aided Design. The IDS cueing subsystems utilize state-of-the-art technology to provide visual, motion, audio, and tactile-instrument feedback. In addition, the IDS uses a sophisticated scenario control system (Cremer et al., 1994), which includes independent simulations of multiple robot drivers that comprise the external traffic participants. The wide range of uses that has been required of IDS imposes some surprisingly challenging technical requirements. Very often, it is not only necessary to model the environment faithfully, but the environment must be modeled differently from actual life in order to test some hypotheses. For example, in order to test a new highway design, the visual database must be constructed to reflect not traditional design rules but the new rules that are under study. To test and evaluate new in-vehicle devices requires the IDS to simulate systems that are not in existence, a task perfectly fit for a simulation that is nevertheless challenging, given the lack of engineering experience in the operation of the new devices. Similarly, to research reaction to specific traffic situations, the IDS must be able to reproduce traffic scenarios faithfully that may involve multiple other robot drivers coordinated to produce interactions (such as lane incursions and sudden braking), which may or may not be able to be reproduced in real life. To achieve this level of programmability and reconfigurability, the IDS utilizes several advanced technologies associated with its cueing systems. Requirements on these systems, along with details on their design, are given below.
The IDS uses an Evans and Sutherland ESIG 2000 dedicated Image Generator (IG). The IG is capable of displaying up to 1.8 million pixels per image at rates of up to 50 images per second. The vertical field of view is 190 degrees forward, with 60 degrees in the rear view. The forward view allows the driver to observe out of both the front and side driver and passenger windows. No images are provided in the normal blind spot. The rear view is projected on a dome-shaped panel located behind the vehicle and is reflected in all three rear-view mirrors. It is possible to redesign the visual configuration of the IDS and trade the refresh rate (the rate at which the views in the rearview mirrors are changed) with resolution and/or field of view. For example, the refresh rate can be reduced in order to gain higher pixel and polygon capacity. To date, the IDS has been reconfigured with smaller field of view and higher pixel density for experiments that require higher visual fidelity. The IG supports full-color textures that can be applied to any polygon and can increase dramatically the visual detail provided to the driver. The visual system supports several sophisticated functions typical of advanced image generators, including MIP (Multi Im Parvo, Latin for "many things in a small place") texture (eliminates the distracting "dancing" of textures by modifying in real time the texture images based on their distance from the simulator driver), anti-aliasing (a technique that causes lines to look smoother without the jagged edges typically associated with computer-generated images), level of detail management (displaying images of varying complexity based on their distance to the eyepoint to increase the effective display capacity for complex scenes), along with a variety of special effects, such as time-of-day variation, fog, and lights. Visual databases used in the IDS are constructed with particular attention to both functional and cosmetic details associated with driving. Functional details include proper road markings, road construction consistent with highway engineering standards, proper signage, and correctly functioning traffic lights. Cosmetic issues include detailed vehicle models for the traffic participants and cultural features, such as buildings, bridges, vegetation, and a variety of structures (e.g., poles and antennas). Depending on the requirements of research studies, it is possible to construct roadways with nonstandard parameters, place features that can change adaptively based on dynamic feedback obtained during the course of a given experiment (i.e., speed limit signs that reduce or increase the speed limit based on the subject's behavior), and create numerous other special effects and customized behavior. The IDS library currently consists of numerous areas populated with rural, interstate, and town roadways with standard and nonstandard widths, markings, and signage. In addition, exact replicas of actual world locations have been constructed and used in validation studies.
Motion feedback refers to the feeling of acceleration sensed in the process of driving. The IDS uses a hexapod (six-legged) motion base with 60-in stoke hydraulic actuators that can provide a maximum of 1.1 g (unit of acceleration based on that produced by the earth's gravitational attraction) of acceleration at a frequency of about 4 Hz. A motion base such as the one used in the IDS cannot reproduce faithfully the magnitude of accelerations involved even in typical driving situations. In fact, any attempt to reproduce 100 percent of the actual acceleration involved in driving will cause the motion base to extend the hydraulic legs to their maximum followed by a sudden stop when the hard motion limits are reached. To avoid such occurrences and maximize the fidelity of the motion feedback, the IDS uses sophisticated washout algorithms that filter the input acceleration signals generated by the vehicle dynamic model and make the best use of the motion envelope available in the hexapod. The algorithms used in the IDS allow prepositioning of the motion base, a technique that maximizes the motion envelop when the behavior of the simulated vehicle can be predicted with certainty, as is often the case in prescribed scenarios. For example, if it is known that a braking maneuver always will occur at a given location, the prepositioning algorithm can shift the motion base so as to maximize actuator travel for the upcoming maneuver. The shifting of the rest point of the motion base is done at a rate that is below the human perception threshold to avoid distracting the subject. The National Advanced Driving Simulator (NADS) utilizes a more sophisticated motion base that consists of a hexapod mounted on top of an X-Y track with 20 feet of travel in each direction (Stoll and Bourne, 1996).
The audio subsystem in IDS consists of a multichannel digital sampling workstation that reproduces sounds associated with the engine, wind, and tire noise, along with sound produced by the remainder of the vehicles in the scene. The audio system will reproduce a variety of sound effects, including the Doppler shift caused by sounds generated by sources moving towards each other (as when a honking vehicle drives by another). In addition, the audio system can replay a variety of specific sounds and actually record voices under the control of the scenario system.
Tactile and Instrument Feedback
Tactile feedback is provided by a high-accuracy motor mounted on the steering wheel and a variety of additional pumps and actuators connected to the various pedals and levers. All instruments within the vehicle cab operate as in
the normal car. Additional devices often are installed to provide newly designed capabilities.
A virtual driving environment such as IDS requires roadway traffic simulated at the microscopic level. A microscopic level traffic simulation is one in which individual vehicles are simulated on their own and interact with each other and with the driver of the simulator (Cremer et al., 1996). In the IDS, these vehicles are aware of the rules of the road and will exhibit a rich set of behaviors similar to those in real life, including randomness and unpredictability. Whereas this behavior is attractive for casual interactions, the unpredictability contradicts the scientific requirement of repeatability necessary in controlled studies with multiple subjects. Complicating the situation even more is the fact that there are great differences among the driving habits of different subjects. Without some means to ensure that a specific interaction (such as a rear-end collision situation) occurs with the same conditions for all subjects, the results of a study may be compromised. The scenario control subsystem in the IDS utilizes an architecture in which intelligent agents within the simulator's virtual environment coordinate events and ensure that specific interactions repeat with similar conditions, despite differences because of specific drivers' habits. The scenario control system utilizes the Hierarchical Concurrent State Machine (HCSM) formalism (Cremer et al., 1995) to simulate the traffic's autonomous behavior. The HCSM formalism allows modeling of simultaneous, multiple thinking processes taking place while driving. Scenes with up to about 25 other vehicles can be simulated, and a variety of traffic situations can be forced to occur. Examples of scenarios used in IDS studies include precrash behavior, following behavior, and effectiveness of antilock brakes. An additional unique aspect of the IDS scenario system is the existence of a special database that contains all traffic participants, along with a variety of logical information about the road network, including lane positioning, signs, intersection topologies, and traffic control devices (Papelis and Bahauddin, 1995). The existence of this information is important because it allows for the collection of performance data that would otherwise be impossible to collect. For example, a simple performance measure that is often used is lane deviation. In complicated road networks (as opposed to a straight road), computing lane deviation requires some representation of the lane. The visual databases used in the IG contain textured polygonal descriptions of the virtual environment that are not useful in deriving lane information. Similarly, information on signage and road markings opens the potential for automatic evaluation tools and detailed performance analysis. In fact, the software used for scenario control in the IDS is currently in use in a simulator used for training truck drivers and utilizes the extended capabilities of the road database to provide both continuous feedback to the driver on rules-of-the-road violation, as well as other performance suggestions.
Research Issues and Limitations
While simulators of this sort provide new and interesting technologies for the measurement of human performance, they are not free of challenges and limitations for their effective use. Some of the issues include choosing and sorting through the detailed and numerous quantifiable performance measures that are possible with this technology (Bloomfield and Carroll, 1996), gathering validation evidence for those events that are safe in a simulator but not safe to test in the real world, and ensuring efficient use of the device to maximize high usage costs. While the amount of realism or fidelity required for research is assumed to be high, the degree of fidelity and associated costs necessary for sound measurement in research settings such as this are not known. Further, the amount of fidelity may vary among research study, study population, task, and experience (Alessi and Watson, 1994). Little research has been conducted to understand fidelity needs, although theoretical models do serve to guide the simulator designer and user.
Finally, simulators are known for inducing something similar to motion sickness in certain individuals. The occurrence of this phenomenon varies with simulator design (fidelity, subsystem components, and cueing congruence), individual susceptibility (gender, age, and experience), and exposure (length, time of day, and scenario interaction). Simulator designers must consider the potential for sickness and attempt to design systems that minimize its occurrence. Researchers who use these devices must understand the phenomenon and attempt to design experiments that limit the likelihood of such effects (Watson, 1995).
CONDUCTING EXPERIMENTS WITH THE IDS
The IDS is a relatively new facility that began operation in 1990. The motion-base facility was not operational until about 1993, when a new motion base was obtained. There is also a fixed-based facility in which the same computer resources are used, but a different simulation bay is set up so that there is no motion. Motion is an expensive cue to simulate, but it adds some of the additional feedback necessary to get real-world matches of driving performance between a simulated environment and that in the real world. The visual system was upgraded in 1994, which has allowed much higher visual resolution.
The IDS utilizes a six-degrees-of-freedom motion base, capable of accelerations up to 1.1 g, which is adequate for some driving maneuvers but not for all. For instance, motion cues experienced during high-speed chases and vehicle handling for a police pursuit cannot be reproduced faithfully.
Visually, the simulator utilizes a 250-degree field of view and fully textured graphics, which help images appear much more realistic and give much better sight distances. The IDS has interchangeable cabs, and vehicle modeling also can be accomplished with an engineering level of detail.
The highly reconfigurable architecture allows IDS researchers to jump between an A model car, a Ford Taurus, a HMMWV (high-mobility multipurpose wheeled vehicle), and an M1A1 tank, so different components of different vehicles can be held with that engineering level of fidelity. Researchers know that they are getting the engineering response that they should. That helps to increase the validity of measure for drivers once they are put into that environment.
There are also high-fidelity roadway models and very realistic traffic models, which help with Department of Transportation work. On-line data collection and reduction are done, and they are probably the most important elements of using simulation. The work that is done with DARPA (Defense Advanced Research Projects Agency) primarily has dealt with vehicle design and prototyping, but when a human is placed in the system, he or she must be put into a simulator. A lot of that other work can be done in a workstation on a desk without ever putting the driver in the loop. It is when the driver is put in the loop that data collection becomes important and also where high-fidelity simulation becomes important because a lower-fidelity environment cannot be immersive for certain drivers, especially experienced ones. Thus, the drivers might accept it for the first few minutes, but after those few minutes have elapsed, they no longer believe the simulation is real.
In fact, there is a lower-fidelity environment where, if the experiment goes on for very long, people start looking around at the ceiling and other things in the room. Therefore, a high-fidelity environment is necessary for data collection.
From a vehicle dynamic standpoint, the simulator utilizes multibody dynamics in real time. The power train, tires, steering, braking, and aerodynamics are simulated with multiple points so that the individual components of the vehicle, as well as the general driver performance, can be tested. All of the vehicle dynamics are at an engineering level of fidelity and take into account all the driving surface properties and elevation. For power train modeling, engine torque and speed, throttle relations, the transmission gearing, and slip are taken into account, as well as differentials and gearing. From the engineering standpoint, modeling those aspects helps with the vehicle design.
The motion system moves as the driver moves throughout the environment so that if a person makes a left-hand turn, the simulator provides the adequate cues for that situation. There are accelerometers and rate gyros hooked to the simulator. Part of the DARPA exercise for 1995 consisted of examining the responses of that motion base in particular sorts of maneuvers. In the simulator, sophisticated motion washout algorithms exist to get the motion back into a central position, so it is ready to perform the next cue.
For all visual database construction, American Association of State Highway and Transportation Officials (AASHTO) standards are utilized, which are the civil engineering standards for roadway design. Correlated terrain generation also can be done, and that is what has been done on the Churchville and Munson test courses at Aberdeen Proving Ground. Parts of these courses have been
modeled with 3-in resolution. Less rough areas were modeled based on data that were gathered with profilometer.
As part of the redesign of the instrument cabs, the engine is replaced with electrical components. A Ford Taurus, a GM Saturn, and a HMMWV cab are available. A software reconfigurable cab exists for the tank simulation, which is simply a mock-up of the inside of a tank. There are several other vehicles that are not instrumented at this time, but all of the controls work, as mentioned above.
The researchers actually go out and gather audio data, which is then fed through a Musical Instrument Digital Interface (MIDI) system so that an attempt can be made to correlate it to the different engine RPMs to ensure that the audio cues are adequate. One of the things being studied right now is exaggerating the audio cues, because in the simulated environment, the sound is not 100 percent correct nor can it be. But people cannot judge their speed, and it is hoped that by exaggerating the audio cue, people might be helped to judge their speed a little bit more accurately.
The University of Iowa will host the NADS in 1999, which is sponsored by the National Highway Traffic Safety Administration. It will be much more powerful than the current IDS but will compete with the top simulator in the world, which is the Daimler-Benz driving simulator in Germany.
As of 1996, 41 studies are funded. Most funding comes from the Department of Transportation, although there is some Department of Defense (DoD) funding, and DARPA has been one of the major funding sources. Work has dealt with automated highway systems (Bloomfield et al., 1995a, b, 1996a, b) and vehicle collision warning and countermeasure devices (Hankey et al., 1996), such as "drowsy driver" countermeasures. In addition, certification of implants for intraocular lenses, examination of cognitive impairment such as Alzheimer's disease and age-related dementia (Rizzo et al., 1994a, b), and vehicle virtual prototyping (Grant and Watson, 1997) are being done.
A set of exercises was carried out last year in conjunction with the DoD project. DARPA funds two types of work at the IDS installation. One consists of pure computer-aided engineering and is performed entirely at workstations. All of the kinematically correct models are transferred to the simulation for that particular vehicle, and that is where the other part of that funding is directed.
The Aberdeen Proving Ground demonstration has been completed; the concurrent engineering is the workstation aspect of that demonstration. The Center for Computer-Aided Design is on a DIS (Distributed Interactive Simulation) network, one of the only academic nodes or sites on the network, so interactive simulation exercises can be done.
The purpose of both the 1994 and 1995 exercises is to examine vehicle virtual prototyping. The aim is to be able to establish an operator-in-the-loop design of a vehicle and redesign of the vehicle so that it is accomplished in the simulated environment prior to being mocked up. Feedback can be obtained
from the driver that may be pertinent to the actual design of the vehicle to reduce the time required in the design process and to lower associated costs.
Last summer, drivers were placed on both the Munson and Churchville courses at the Aberdeen site and then brought to the simulator in Iowa City. There they drove a number of times around the simulator with a certain predescribed behavior in an effort to gather validation data from both the vehicle and the driver performance standpoint because driver performance measures were being examined extensively. An HMMWV was used to gather considerable data. It is much easier to analyze data collected in a simulated environment than those from a test course. Some of the last test course data are still being analyzed because the analysis is time-consuming due to the presence of noise in the data.
Considering the performance measures used (e.g., the velocity in miles per hour at every part of the course), it is necessary to know how challenging the course is to really understand the match of the data (how well behavior in the simulator resembles that in a real-life driving situation). The course in question has a hill with a 29 percent grade, blind turns that require the driver to ''turn on a dime," and also other downhill grades; the rough part of the course has 24-in-high bumps to go over, which is difficult.
The velocity match is also very similar. People tend to overdrive a little bit in the simulator, accelerating on a few of the straight areas and maintaining high velocity in some of the areas. There is also a tendency to oversteer a bit, which could result from the lack of a motion cue; this is being examined again in the 1995 follow-up to the HMMWV study. Other parameters being measured include use of the accelerator pedal and the brake. With the accelerator pedal, there definitely is some mismatch. While a trend appears to develop in some of the areas, it is off in others. A followup study was done in October 1995 with nine HMMWV drivers.
Subjective data, workload data, and physiological data were collected from all drivers, and similar sorts of matches were obtained for all of those. A perceived realism rating also was collected in the simulator only, with a reliability of about 0.92, which is very good. Thus, an expert set of drivers, who are much harder to fool, were finding the simulator experience to be very realistic.
The IDS, as mentioned above, is DIS-compliant. This allows for the performance of interactive exercises for additional measurement sorts of settings. It allows interaction with other simulators at other installations, where synthetic battlefield exercises can be assembled or responses of the vehicles or the drivers can be examined.
The work to date has been performed primarily for realistic mobility testing and for the iterative vehicle design process2 in which operators are included in the loop (design and testing).
The IDS and the Research Community
Some of the challenges of using virtual reality for this sort of measurement include resource issues, such as billing the research community, looking at simulator effects or simulator sickness, and validation of virtual reality—in particular this driving simulator—as a measurement tool.
As part of a university, the IDS staff's first mission is to do research for the university and the Center for Computer-Aided Design at the University of Iowa. The expectation is that the IDS will be a precursor to the NADS, which is being built in Iowa City. The NADS is expected to be operational in 1999 and is funded by the National Highway Traffic Safety Administration for use by researchers throughout the nation.
With respect to educating the research community, it is not known exactly what parameters must be measured to study driving performance. Driving assessment primarily has been qualitative. It is now possible to get very precise data in the simulator; thus, attempts are being made to understand how the quantification of those data relates to the qualitative criteria that customarily are used to evaluate performance.
Also, attempts are being made to teach the research community how to use simulator technology efficiently and effectively. This technology is expensive, at least in the high-fidelity market, so not a minute is wasted on the simulator. Subjects are shuttled, and there cannot be one 30-sec down time, as a result. Experiments are run 12 hours a day. The other 12 hours of the day are used for development.
Also, attempts are being made to educate the research community on the fidelity requirements. High fidelity is not always necessary, and it is possible to use virtual reality environments that are lower in fidelity and still get very valid performance measures. However, it is necessary to understand what fidelity is required (Alessi and Watson, 1994).
The potential for motion sickness in simulator subjects is under investigation. Although it was assumed that much of the motion sickness literature from flight simulation would be applicable, that research has not been completely applicable to experiments done with driving instead of flight. The driver population is a much more generalized population than pilots, and the potential for motion sickness is therefore higher (Watson, 1995).
There is a real need for evidence regarding the validity of the simulator experience with respect to real driving, and this must be emphasized to the research community. The simulator has a very novel effect for people who have not used it before. Attempts have been made to look at experimental designs to get past that novel effect because it is necessary to know that the performance obtained is indeed real performance and that it transfers to the real world. The
challenges with simulator effects are to identify those susceptible groups and the simulation in order to take care of some of those parameters. It has been possible to tune out sickness fairly well in the IDS; although there is an attrition rate of about 2 percent, up to about 8 percent of the simulator subjects report at least some symptoms.
An attempt has been made by simulator researchers to design scenes and scenarios that try to reduce the simulator effects. Sickness is always analyzed and considered as a covariant in all experiments to ensure that it is not creating performance differences.
The research community has expressed an interest in using the IDS but asks whether it is validated. Validation is an ongoing process. There is a real need to gather information for every set of tests that is run. Different validity evidence must be given to different types of tests because something that would be used for the DARPA experiments could be very different from something that might be used for the automated highways experiments. Those variables always must be examined.
Gathering on-the-road data is also a continual process, and it is time-consuming and expensive. In fact, in several studies, it has challenged the simulator rate because it is so expensive. Seven people must be out on the road in order to gather data. If it rains, the work cannot be done.
To date, a number of validation studies have been completed in addition to the DARPA study (Grant and Watson, 1997). A comparison of different age and gender groups has been done in the simulator, looking at trends from past research only (the subjects were not taken out on the road) (Romano and Watson, 1994). A recognition and detection distance of signs also has been done, taking subjects out on the road in twilight night and glare conditions and putting them in the simulator to ensure that the recognition and detection distance of signs or the supply of sign information on the simulator is a match to that in the real world.
Validation is a very important issue, and it is hoped that the appropriate validation data can be gathered with the IDS. Many other devices exist, many less expensive, and they may have real potential. However, with regard to the virtual reality environment, it is necessary to gather the validity evidence to test whether the performance measures transfer to real-world performance regardless of the simulator fidelity.
It also is necessary to test whether the perceived realism is high enough so that subjects feel they are immersed in an environment, subjects react in a similar way, and they accept that environment over a long period of time or during a number of experiments. Confounding factors such as motion sickness must be
controllable, and it must be possible to take this technology to a platform3 on which it is very cost effective, where high-fidelity simulation can be obtained for a fraction of the cost.
Alessi, S.M., and G.S. Watson 1994 Driving Simulation Research at The University of Iowa. Proceedings of the 1994 Annual Meeting of the Association for the Development of Computer-Based Instructional Systems . 16–20 February 1994, Nashville, Tenn.
Bloomfield, J.R., and S.A. Carroll 1996 New measures of driving performance. Pp. 335–340 in Contemporary Ergonomics, S.A. Robertson, ed. London: Taylor and Francis.
Bloomfield, J.R., J.R. Buck, S.A. Carroll, M.W. Booth, R.A. Romano, D.V. McGehee, and R.A. North 1995a Human factors aspects of the transfer of control from the automated highway system to the driver. Technical Report No. FHWA-RD-94-114. Washington, D.C.: U.S. Department of Transportation, Federal Highway Administration.
Bloomfield, J.R., J.R. Buck, J.M. Christensen, and A. Yenamandra 1995b Human factors aspects of the transfer of control from the driver to the automated highway system. Technical Report No. FHWA-RD-94-173. Washington, D.C.: U.S. Department of Transportation, Federal Highway Administration.
Bloomfield, J.R., J.M. Christensen, S.A. Carroll, and G.S. Watson 1996a The driver's response to decreasing vehicle separations during transitions into the automated lane. Technical Report No. FHWA-RD-95-107. Washington, D.C.: U.S. Department of Transportation, Federal Highway Administration.
Bloomfield, J.R., J.M. Christensen, A.D. Peterson, J.M. Kjaer, and A. Gault 1996b Transferring control from the driver to the automated highway system with varying degrees of automation. Technical Report No. FHWA-RD-95-108. Washington, D.C.: U.S. Department of Transportation, Federal Highway Administration.
Cremer, J., Y.E. Papelis, J. Kearney, and R.A. Romano 1994 The software architecture for scenario control in the Iowa Driving Simulator. Pp. 373–381 in Proceedings of the 4th Annual Computer-Generated Forces and Behavioral Representation Conference. Orlando, Fla.: Institute for Simulation and Training.
Cremer, J., J. Kearney, and Y. Papelis 1995 HCSM: A framework for behavior and scenario control in virtual environments. ACM Trans. Modeling Comp. Simul. 5(3):242–267.
Cremer, J., J. Kearney, and Y.E. Papelis 1996 Driving simulation: Challenges for VR technology. IEEE Comp. Graphics Appl. 16(5):16–20.
Freeman, J.S., G.S. Watson, Y.E. Papelis, A. Tayyab, R.A. Romano, and J.G. Kuhl 1995 The Iowa Driving Simulator: An implementation and application overview. Pp. 81–90 in Proceedings of the Society for Automotive Engineers International Congress and Exposition . Detroit, Mich.: Society of Automotive Engineers.
Grant, P., and G.S. Watson 1997 Validation of a HMMWV in the Iowa Driving Simulator. Unpublished manuscript. Iowa City: The University of Iowa.
Hankey, J.M., D.V. McGehee, T.A. Dingus, E.N. Mazzae, and W.R. Garrott 1996 Initial driver avoidance behavior and reaction time to an unalerted intersection incursion. Pp. 896–899 in Proceedings of the 40th Annual Meeting of the Human Factors and Ergonomics Society. Santa Monica, Calif.: Human Factors and Ergonomics Society.
Kuhl, J.G., D. Evans, Y.E. Papelis, R.A. Romano, and G.S. Watson 1995 The Iowa Driving Simulator: An immersive environment for driving-related research and development. IEEE Comput. 28(7):35–41.
Papelis, Y.E., and S. Bahauddin 1995 Logical modeling of roadway environment to support real-time simulation of autonomous traffic. Pp. 62–71 in Proceedings of the 1st Workshop on Simulation and Interaction in Virtual Environments. Iowa City, Iowa: Department of Computer Science, University of Iowa.
Rizzo, M., G.S. Watson, D.V. McGehee, and T.A. Dingus 1994a Simulator driving and car crashes in aging and cognitively impaired drivers . J. Neurol. 241:526.
1994b Simulator driving and car crashes in Alzheimer's disease. Soc. Neurosci. Abstr. 20:439.
Romano, R.A., and G.S. Watson 1994 Assessment of capabilities—Iowa Driving Simulator. Technical Report No. FHWA-IR-94. Washington, D.C.: U.S. Department of Transportation Federal Highway Administration.
Stoll, D., and S. Bourne 1996 The National Advanced Driving Simulator: Potential applications to ITS and AHS research. Pp. 700–710 in Proceeding of the 6th Annual meeting of the Intelligent Transportation Society. Washington D.C.: ITS America.
Watson, G.S. 1995 Simulator effects in a high-fidelity driving simulator. Pp. 124–138 in Proceedings of the 1995 Driving Simulation Conference. Sophia-Antipolis, France: Neuf Associes.
JOHN VANDERVEEN: You mentioned the cost several times. What is the cost for this simulator, and what is it going to cost for the future device?
GINGER WATSON: This is part of the initial plans: It has to operate at no more than $1,000 an hour. We currently are operating loaded at the University of Iowa at $1,300, but that might drop every time computer technology gets less expensive. For instance, our previous image generator cost $1.5 million. We just bought a new one for a quarter of a million, and it has four times the capability. So every year we have to reassess those rates, and we will be doing that again in July.
BERNADETTE MARRIOTT: You mentioned physiological measures. What physiological measures do you take?
GINGER WATSON: We just took postrespiration blood pressure in that particular case. We can always correlate. We did not correlate for those experiments, and that is one of the reasons you do not see it. We did that only at the request of the PI on the project at that particular point in time. We are actually looking for some more robust measures to use and to correlate.
DAVID DINGES: That was a great presentation. I am very excited that you were candid about the validation issues and what you are doing with it. As you know probably better than anybody, this is another area where people have sold what they claim are simulators that are just utterly fraudulent. They are just nothing but a simple reaction time task.
What I wanted to ask you about was the fidelity of your crash. In NASA's aviation simulators, a crash is a serious event, and everybody works hard to avoid that. It is taken deadly seriously. What is the fidelity of the crash like in this?
GINGER WATSON: Well, we actually do not go through the full-motion simulation because we typically run people up to 85 years old. All Department of Transportation studies require that we run older populations, so we simulate only the visual aspects up to a certain point in the crash. We do not go through the full impact. But we could not do the full impact anyway with our motion base, to be very honest.
DAVID DINGES: To the extent that a crash is possible, and certain groups in the population have a tendency to drive more risky, will they take a crash event on the simulator as seriously as a real crash?
GINGER WATSON: Our sense is that they do take it very seriously in the higher-fidelity simulator, but we run a lot of studies on our lower-fidelity simulator, especially pilot work. I have to tell you, with certain populations they do not take it seriously in the less immersive environment.
Now, my question is, if you exposed somebody to the high-fidelity environment for a long period of time, would they still be as serious about it? We have actually had people cry or scream—very violent reactions.
DAVID DINGES: Finally, may I ask you statistically what is the reaction of risk-taking driving groups, such as males in their 20s. I mean, how do they behave on it?
GINGER WATSON: We have had a few that we have thrown out of studies, very few, though. We have run about 1,500 subjects now. I think we have thrown out two, and they were in the automated highways experiments.
It is a valid point because they were saying, "Oh, this is great, oh, yes." We said, "Stop; this is costing us $1,300 an hour, and you're out."
PARTICIPANT: What are your instructions to the subjects?
GINGER WATSON: We tell them to drive as realistically as possible, and we try to set up a very serious situation. One of the things that we do tell them is it is very expensive to run the study. We will not pay them the full amount if we do not get the full performance from them.
Really, the only group we have ever had problems with, to tell you the truth, is young, college-age males. They are computer literate to begin with, have been in some other virtual reality environments, and do not take it quite as seriously.
DAVID DINGES: That is right; they do not take real driving seriously, either.
GINGER WATSON: Yes, perhaps there is a correlation. I do not know.
PARTICIPANT: You could emphasize speed, or you could emphasize performance and you could emphasize safety, in going through the course. What is the message to the subject?
GINGER WATSON: We usually emphasize safety in subject preparation before we take them in. Then when we get them in the simulator, we always remind them what the speed limit is on a given roadway, and if it is a long drive, we will frequently remind them somewhere in the middle. We usually do not have to, though, but we do include that in the protocol quite frequently.
It has worked so far and, again, a lot of it is just anecdotal, but when we go back and review tapes, we think that people are really driving as they would. And we ask people, too—we have looked at a cross-correlation on this—what
speed they normally drive on an interstate and what speed they would normally drive on a rural road. We have found some pretty decent correlations with that.
PARTICIPANT: Is your test population just from Iowa?
GINGER WATSON: No. In fact, in the DARPA experiments our test population is from Maryland. We flew those people out, but that is expensive. We are also doing a study for a company that is seeking Food and Drug Administration approval for an intraocular lens and, because of that, they have a sample population throughout the United States that will be flying in from everywhere. That is an expensive consideration. They can afford to do that; I do now know if that is cost effective for all studies.
DAVID SCHNAKENBERG: What are examples of the type of experimental variables they are trying to look at in your simulator?
GINGER WATSON: Oh, they vary. Actually, I have given a half-day presentation talking about performance variables. If it is automated highway experiments, you are looking at reaction time …
DAVID SCHNAKENBERG: No, what are the factors they are looking at? Is it independent variables between the driver, is it the age, is it the gender?
GINGER WATSON: That is just as complex as the dependent variables. Just to give you an idea, usually it is age; looking at different sorts of driving scenes, especially in the DARPA situation, what are the different tests that are required, and is this something that is uphill or over a rough course?
In the automated highway experiments we look at things like entry into the automated lane, is it automated, is it manual, is it partially automated (Bloomfield et al., 1996a, b)? What happens in the case of a failure, and also, we look at exit from the systems, breaking it up into those pieces, but they are looking at things like age and automation. It really varies by experiment. Every single experiment that we do has different independent and dependent variables, which really adds to our validation complexity.
JOHANNA DWYER: Have you done any experiments where you have fasted, thirsted, or sleep-deprived people or a combination thereof?
GINGER WATSON: There is a graduate student who has done a sleep-deprivation study in our low-fidelity simulator, but that is it. We have had people approach us to do that sort of work, but we have not, because, to be very honest, this is where that fidelity issue comes in for certain sorts of experiments, especially when there is a secondary task or something, you might be able to do much more cost effectively on a lower-fidelity device.