Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 64
An Assessment of the National Highway Traffic Safety Administration's Rating System for Rollover Resistance: Special Report 265 4 Consumer Information The congressional mandate for this study requested an assessment of the practicality and utility to the public of the National Highway Traffic Safety Administration’s (NHTSA) consumer information on rollover. As discussed in Chapter 2, the static stability factor (SSF)—which forms the basis of NHTSA’s rating system for rollover resistance—is a useful metric in assessing a vehicle’s rollover propensity. However, there were deficiencies in NHTSA’s use of statistical analyses of crash data to develop its rating system (see Chapter 3). Investigation of consumer response to NHTSA’s rollover information requires empirical data on vehicle buying behavior in general and on consumers’ use of NHTSA’s rollover resistance rating system in particular. The committee was unable to obtain such data. Consequently, the committee addressed this part of its charge by extrapolating from research on other consumer products, using its judgment, and evaluating the process used by NHTSA to develop its rollover rating system. This chapter first presents the results of the committee’s evaluation of the practicality and utility of NHTSA’s rollover resistance ratings. Next is a discussion of the dissemination and use of NHTSA’s rollover information. Suggestions for NHTSA’s future development of consumer information on motor vehicle rollover in both the near and longer terms are then presented. The chapter concludes with the committee’s findings and recommendations in the area of consumer information. PRACTICALITY AND UTILITY OF NHTSA’s ROLLOVER RESISTANCE RATINGS Every day consumers make decisions about buying goods and services using processes and behaviors that researchers have studied for more than 30 years (see, for example, Bettman 1979; Bettman et al. 1998; Hansen 1972; Howard and Sheth 1969; Nicosia 1966; Simonson et al. 2001). Consumers obtain information about the attributes of products and their alternatives from a wide range of sources, and the difficulty of their choices increases with the amount of information they must consider (Bettman et al. 1991). Advances in information technology have now made it possible for Americans to access a wealth of product information far beyond what was previously available. In addition, the opportunity to purchase products over the Internet is changing shopping behavior. For most consumers, buying a vehicle represents a major financial decision—one of the larger purchases they will make in their lives in dollar
OCR for page 65
An Assessment of the National Highway Traffic Safety Administration's Rating System for Rollover Resistance: Special Report 265 terms. The decision is a difficult one given the numerous choices involved (e.g., vehicle cost, size, type, appearance, quality, reliability, safety, performance and handling, fuel economy), together with consumers’ preferences and constraints (e.g., budget, driving style, need to carry multiple passengers). Adding to the difficulty, each year automobile manufacturers introduce new vehicle makes, models, and features or options from which to select (e.g., side airbag curtains, electronic stability control, hybrid propulsion systems). Although manufacturers perform extensive market research and obtain proprietary information about the preferences of their customers, the open literature contains few reports of research on the behavior of buyers of the approximately 17 million light vehicles sold in the United States annually.1 Consumer Interest in and Use of Vehicle Safety Information Many consumers shopping for an automobile regard vehicle safety as important. In a survey for the Insurance Research Council (1999), 78 percent of respondents who had recently leased or purchased a vehicle stated that vehicle safety was important in their purchase decisions. In a 1999 study conducted by DaimlerChrysler, 84 percent of consumers said safety features were an extremely or very important reason for buying a vehicle (Automotive News 2000). General Motors reported that in 1994, consumers ranked safety sixth of 38 possible reasons for choosing the vehicle they purchased instead of their second-choice vehicle (General Motors Corporation 1994). In discussions with vehicle manufacturers (see Appendix B), members of the committee heard about the diversity of consumers: some place a great deal of weight on vehicle safety in their purchase decisions, whereas others place relatively little weight on safety relative to other factors. Within the group of consumers who regard vehicle safety as important, there are varying levels of interest in or capacity for dealing with the complexity of the information involved. Some consumers simply may want assurance that a vehicle is safe without any details about what “safe” means. Others may want to understand all the technical details of each vehicle and be able to make their own comparisons with other vehicles and safety determinations. Given this variation, it is impossible to use a “one size fits all” approach to the provision of consumer information without giving some consumers much less information than they want and others much more. Therefore, as recommended in an earlier study (TRB 1996), the development of easily accessed presentations of automotive safety information that increase hierarchically in detail is a logical approach to meeting the information needs of consumers. 1 This statistic reflects 1999 sales as reported by the University of Michigan (2001).
OCR for page 66
An Assessment of the National Highway Traffic Safety Administration's Rating System for Rollover Resistance: Special Report 265 Many different sources—including car dealers, newspapers, magazines, brochures, television, friends, family, and the Internet—provide vehicle safety information. A 1999 survey by Roper Starch for the Insurance Research Council revealed the following as popular sources of information for respondents: car salespeople (37 percent), Consumer Reports (29 percent), newspaper reports and car magazines (18 percent), automobile manufacturers (17 percent), friends (15 percent), and the Internet (14 percent) (Insurance Research Council 1999). These percentages are expected to change significantly as a result of the rapidly growing use of the Internet. At the same time, the amount and types of vehicle safety information obtained from these sources and stakeholders and the extent to which consumers seek and use vehicle safety information remain unknown, except anecdotally. Little is known regarding what consumers know and believe about the factors that can lead to rollover, the technologies available to improve a vehicle’s crash avoidance and crashworthiness capabilities, what can be done to reduce the risk of injury from a rollover, and how the available information should be used. Current knowledge suggests that vehicle purchase decisions occur in two stages: choice of vehicle class, followed by choice of specific make and model (TRB 1996). Thus, information on rollover resistance that distinguishes among vehicle classes could influence first-stage decisions, while information that distinguishes among vehicles within a class could influence second-stage decisions. Development of Vehicle Safety Information Given the complexity discussed above, the challenge of developing an effective consumer information rating system is clear. According to Shopping for Safety, a previous congressionally mandated report on consumer automotive safety information (TRB 1996, 3), “to be most effective, consumer safety information should be based on a systematic understanding of what consumers know about vehicle safety and how they go about obtaining and using information.” Although some relevant research has been documented since that report was published, the systematic understanding required for the development of effective consumer safety information is still lacking. Several researchers have identified good practices in the development of consumer information, including product rating systems. For example, Wogalter and colleagues (1999) suggest the following approach:2,3 In the initial phase of a project, investigate a number of candidate information systems, and select a few of the most promising for further study. 2 In a presentation to the committee on May 29, 2001, Mike Wogalter of North Carolina State University discussed means of presenting risk-based information to consumers. 3 Shopping for Safety (TRB 1996) describes similar elements of an approach to designing risk messages that are effective in supporting consumer decision making.
OCR for page 67
An Assessment of the National Highway Traffic Safety Administration's Rating System for Rollover Resistance: Special Report 265 Involve small samples of relevant consumer groups in formative research on the candidate systems, and conduct structured, open-ended interviews to determine relevant consumer beliefs. Engage experts in appropriate technical areas (e.g., automotive safety, communication, and graphic design). Evaluate candidate messages empirically, and use the findings to guide modifications to the candidate systems, with retesting as needed in an iterative design process (Morgan et al. 2001).4 Follow the preliminary testing with more extensive, formal testing of information products using large samples to validate the results from small samples, and to determine whether consumers understand the limits of the information and can apply it appropriately in a practical situation. Characteristics of human information processing, especially inherent limitations on people’s ability to process information, also need to be taken into account (e.g., Miller 1956). For example, side-by-side presentation of information comparing the ratings of different vehicles may reduce information processing needs and support more thorough consideration of vehicle attributes and alternatives. Providing a frame of reference is likely to affect the way information is perceived; thus, for example, preferences relative to risks presented in terms of potential losses may differ from preferences relative to the same risks presented in terms of gains (Fischhoff et al. 1978; Plous 1993; Levin et al. 1998). In the specific case of rollover, information presented in terms of rollover resistance could influence decisions differently from the same information presented in terms of rollover propensity (Bettman et al. 1998). Failing to provide sufficient context or to define concepts and terms clearly can undermine any communication. Consequently, the development of effective communication typically begins with a statement of objectives and creation of a sound process for achieving those objectives. NHTSA’s Objectives for Consumer Information on Rollover Consumer information on rollover is only a portion of the safety information offered by NHTSA. The agency has explicitly stated its objectives for providing this information based on SSF as follows (Federal Register 2000): Enable prospective purchasers to make choices about new vehicles based on differences in rollover risk. 4 When graphics are desired, the population stereotype method is one useful approach. This method, which entails asking users to draw graphics and using those graphics as a source of ideas for designers, has been shown to increase the likelihood that graphics will be understood by users (Green 1979; Eberhard and Green 1989; Mudd and Karsh 1961; Green 1993).
OCR for page 68
An Assessment of the National Highway Traffic Safety Administration's Rating System for Rollover Resistance: Special Report 265 Provide a market incentive to manufacturers to design their vehicles with greater rollover resistance. Inform drivers who choose vehicles with less rollover resistance that their risk of harm can be greatly reduced with seat belt use to avoid ejection. NHTSA’s Process for Developing Consumer Information on Rollover In 1997, NHTSA established a new department within its Plans and Policy Division—the Consumer Automotive Safety Information Division (Consumer Division)—and assigned this new division the task of developing and disseminating consumer safety materials. The Consumer Division has responsibility for the presentation of safety information in the New Car Assessment Program (NCAP), although other groups within NHTSA provide the technical basis for this information. Funding for both consumer research and publications comes from the NCAP budget, and totaled about $350,000 during the first 2 years of the division’s existence, rising to about $500,000 during fiscal year 2001. Each year, approximately $200,000 of this funding goes to printing costs, leaving limited funds for the research and evaluation activities needed to provide effective support for information campaigns on specific automotive safety topics, such as rollover. Few of the division’s staff of seven have formal qualifications in psychology or consumer research; only two engage in research, on a part-time basis. In the case of rollover information, the Consumer Division was provided with the rollover curve derived from crash data (see Chapter 3) and key items for inclusion in explanatory text. The division approached its task of identifying an effective means of communicating the relation between SSF and rollover risk to consumers using the results of two series of focus group studies (Equals Three Communications 1999b, 2000). NHTSA had previously used focus group studies in support of its communication efforts—most recently the rollover warning label currently found in sport utility vehicles (SUVs) and light trucks (Nancy Low and Associates 1996; Equals Three Communications 1999a). In the first phase of the development of the rollover rating system, six focus groups were conducted with new vehicle owners or lessees in Dallas, Texas; Overland Park, Kansas; and Richmond, Virginia (Equals Three Communications 1999b). Each focus group began with a discussion of awareness of the rollover problem. Participants attributed rollovers primarily to driver-related behaviors and said they believed that rollover was more of an issue for SUVs than for other vehicles. When asked where they would look for information on rollover, participants named Consumer Reports and insurance industry representatives. Participants stated repeatedly that rollover risk would be only one of several pieces of information they might take into ac-
OCR for page 69
An Assessment of the National Highway Traffic Safety Administration's Rating System for Rollover Resistance: Special Report 265 count when buying a new vehicle, and stated their belief that rollover information would likely influence only those who really care about safety. The second part of each focus group involved investigating the clarity of several different concepts for presenting vehicle rollover information. Participants were asked to react to five rollover information formats for both labels (see Figure 4-1) and brochures: Option 1—a star rating system with five categories, where one star indicates the least stable vehicle and five stars the most stable; Option 2—a line graph labeled with SSF values from 1.06 to 1.44; Option 3—a three-point tilted-vehicle graphic indicating whether a vehicle is “more stable,” “average,” or “less stable”; Option 4—a three-point letter grade (A, B, C), where A indicates “more stable,” B indicates “average,” and C indicates “less stable”; and Option 5—a number score on a scale of 90–150, with 90 being the least stable and 150 the most stable. The findings from the focus groups indicated that some participants did not have a good understanding of SSF and the rollover information presented. Even after the discussion moderator had provided an easy-to-understand description of SSF, participants tried to interpret this measure as a historical statistic on the relative frequency of rollover, rather than as a measure of rollover resistance in the event of a single-vehicle crash. In addition, participants made comments about alternatives to the five-star system and the specific scale ranges presented (e.g., lack of power to differentiate with a three-point scale, difficulty of using subdecimal differences in the 1.06–1.44 range, unfamiliarity of the 90–150 range). The study confounded graphical and scale attributes instead of varying them systematically; therefore, it was impossible to gauge the independent effects of varying the graphical representation (e.g., stars, line graph) and varying the scale (e.g., letter grade, number score). Although participants rated the star presentation as the best overall concept, they rated the tilted-vehicle icon as the most convincing graphic. Some participants proposed that this icon be incorporated into a presentation—modeled on the miles-per-gallon fuel pump label—that would show the tilted-vehicle icon enclosing a number between 1 and 10, where 1 would indicate the worst rollover performance and 10 the best. NHTSA subsequently decided to use a five-star rating system for rollover resistance and developed some explanatory text to accompany these ratings. A further investigation by Equals Three Communications (2000) explored consumer awareness and understanding of rollover problems and evaluated comprehension of two alternative texts for explaining the rollover resistance ratings. The study involved 12 focus groups in Chicago, Illinois, and Dallas,
OCR for page 70
An Assessment of the National Highway Traffic Safety Administration's Rating System for Rollover Resistance: Special Report 265 FIGURE 4-1 Options for rollover information labels. (SOURCE: Equals Three Communications 1999b.)
OCR for page 71
An Assessment of the National Highway Traffic Safety Administration's Rating System for Rollover Resistance: Special Report 265 Texas, comprising a total of 108 licensed drivers who owned or planned to purchase new or recent automobiles, minivans, pickup trucks, or SUVs. All participants had indicated during prescreening that safety was “somewhat” or “very” important to them when shopping for a personal vehicle. In general, as in previous focus group studies, participants expressed awareness of the rollover problem, commonly associated it with SUVs, and attributed it to driver behavior compounded by environmental factors. They generally understood the concept that in the rating system, more stars are better. However, some participants found the inverse relationship between rollover propensity and number of stars confusing, with more than 10 percent believing after reading the explanatory text (short or long version) that a vehicle with fewer stars is safer. Approximately two-thirds believed after reading the longer explanatory text that the rollover ratings described an “estimate of risk of rollover if run off road,” the answer the study designers considered correct for a multiple-choice question in the study. “Being in a single-vehicle crash” does not necessarily mean “if run off road,” and “risk of rollover” easily could be misinterpreted as a more comprehensive measure of risk than the likelihood of rollover. Some of the responses to open-ended questions were consistent with this misinterpretation. Also, the two texts appear to have been evaluated sequentially, which means their assessed effects were not independent. A preferable approach would involve asking half the participants to read the short text first and then the long version, and asking the other half to read the two in the reverse order. NHTSA’s process stopped with the focus groups. Once the rollover resistance ratings had been prepared for dissemination to the public, the agency did not conduct any large-scale formal testing of its information products. Furthermore, NHTSA has not surveyed consumers to determine whether the published ratings are influencing their vehicle purchase decisions, and the committee could not find analogous surveys by the industry. Comparison of NHTSA’s Process with Recommended Practices and Criteria of Demonstrated Validity Comparison of communications against standard evaluation criteria provides some basis for judgment of the communications’ quality, even though such assessments cannot replace appropriate user-based evaluation (Schriver 1996). In the absence of empirical data on consumer behaviors, therefore, the committee compared NHTSA’s process with the recommended practices discussed above and identified three important areas of concern: (1) over-reliance on focus groups, (2) the lack of an iterative design process, and (3) the absence of formal testing of consumer information. Evaluation of the agency’s rollover resistance ratings relative to the criteria for good summary measures identified in Shopping for Safety (TRB 1996, p. 110) also revealed some areas for improvement.
OCR for page 72
An Assessment of the National Highway Traffic Safety Administration's Rating System for Rollover Resistance: Special Report 265 Use of Focus Groups Researchers originally created focus groups as a means of fleshing out hypotheses (Merton 1987; Merton and Kendall 1946; Merton et al. 1956, 1990). While collecting focus group data is a first step in developing consumer information, such studies by themselves cannot reveal much more than group knowledge of an issue. Consequently, overreliance on focus groups can be problematic.5 Focus groups by themselves are not able to provide sufficient evidence to support reliable conclusions about what consumers know or believe about rollover risk, or whether the ratings are meaningful, easy to interpret and understand, and unambiguous for the public. According to recommended practices for developing consumer information, structured, open-ended interviews with a representative sample of consumers would be a better way to determine consumers’ beliefs about vehicle safety and rollover. Similarly, consumers’ understanding of proposed summary measures and supporting text might be assessed with equal or more effectiveness using one-on-one interviews or written questionnaires to determine individual responses, instead of gauging collective knowledge by means of focus groups. Additionally, as noted above, the focus group studies suffered from design flaws that further limited their usefulness (e.g., no systematic variation of variables, use of sequential evaluations susceptible to bias). Iterative Design Process An effective communication process requires iteration so that information can be tested, refined, and retested. Even though the star presentation emerged as the best overall concept in NHTSA’s first set of focus groups (Equals Three Communications 1999b), the participants also provided an alternative presentation (tilted-vehicle icon and numeric rating) and suggested that SSF and the rollover information presented were not well understood. NHTSA did not explore the alternative presentation or further iterate between design and evaluation. The second set of focus groups (Equals Three Communications 2000) examined the understandability of the five-star rating system selected by NHTSA, as well as two alternative explanatory texts. Almost all participants in these focus groups understood that “more stars are better,” but the consumer comments indicated that the explanatory text could have been improved. For example, many participants found the ratings not sufficiently comprehensive, lacking reference to driver behavior and driving conditions, 5 Group processes are known to affect individual responses and can shift the entire group’s judgment toward the position that first dominates in the group, as in group polarization (Isenberg 1986; Myers and Lamm 1976; Whyte 1993; see Plous 1993 for a general discussion).
OCR for page 73
An Assessment of the National Highway Traffic Safety Administration's Rating System for Rollover Resistance: Special Report 265 and some participants would have liked more information on how to avoid a rollover crash. Formal Testing of Consumer Information Once information products appear ready to be widely disseminated, more extensive testing is desirable using large samples to determine whether consumers are able to apply the information (see, for example, Wogalter et al. 1999). NHTSA assessed the extent to which consumers could use its proposed rollover information by asking participants in the second set of focus groups to complete a worksheet including two multiple-choice questions (one of which was discussed above). However, these focus groups involved only 108 drivers in two geographic markets. The report on these focus groups by Equals Three Communications (2000) cautions the reader against “making quantitative conclusions from the results of this study” and warns about possible problems with the small samples and the extent to which the results are representative of a larger population. Criteria in Shopping for Safety Revisiting the criteria for good summary measures of vehicle safety performance provided in Shopping for Safety (TRB 1996, 110) could help NHTSA improve its vehicle safety information. Three of these criteria are particularly relevant to efforts aimed at improving the rollover resistance rating system:6 Conveys the degree of uncertainty associated with current knowledge and expert judgment. The current information in NCAP provides no indication of the uncertainty resulting from the sampling, data averaging, and statistical modeling procedures used in deriving the ratings from crash data (see Chapter 3). The text simply states that “the ratings were found to relate very closely to the real-world rollover experience of vehicles,” leading the consumer to believe (incorrectly) that the star ratings can be assigned to individual vehicle models without ambiguity. Is transparent and flexible, allowing more-sophisticated users to understand how summaries are produced and to apply different judgments to obtain their own summaries as desired. NCAP notes that the ratings “relate very closely” to the real-world rollover experience of vehicles as characterized by “220,000 actual single-vehicle crashes,” but there is no attempt to explain in 6 Other sections of this chapter discuss the other three criteria: (a) related meaningfully to actual safety for the range of highway conditions in which the vehicle will be operated; (b) provides a summary whose use or interpretation does not require extensive manipulation or combination with other information; and (c) is unambiguous and easy to understand and use.
OCR for page 74
An Assessment of the National Highway Traffic Safety Administration's Rating System for Rollover Resistance: Special Report 265 any detail how the ratings were derived. Consequently, more-sophisticated users cannot easily understand how NHTSA produced the current summaries. Allows the consumer to place the information in context. NHTSA provides the rollover ratings in NCAP, together with other vehicle safety ratings. The absence of information on the relative importance of the different ratings and how to combine them remains a potential source of confusion and does not help the consumer determine which is the safest car. In addition, appropriate use of the ratings requires the ability to put the information in context (i.e., understanding that the ratings predict the likelihood of rollover given that a single-vehicle crash has occurred). AVAILABILITY AND USE OF NHTSA’s ROLLOVER RESISTANCE RATINGS As discussed earlier, the absence of empirical data on consumer behavior prevented the committee from making a definitive judgment about the usefulness of NHTSA’s rollover ratings to consumers. The ratings may be useful in helping consumers make informed purchase decisions, but the committee has no way of establishing this. However, some evidence suggests that the ratings are of interest to the public. As noted, NHTSA incorporated the rollover resistance ratings for a number of vehicles into its NCAP program, available on the agency’s website (www.nhtsa.dot.gov) (see Appendix D). Data provided to the committee by NHTSA provide some insights into Internet users’ interest in the ratings. Dissemination of the Ratings by NHTSA NHTSA’s strategy for dissemination of the star ratings for rollover resistance and associated information focuses primarily on the Internet—an important source of information on automobiles, at least for some consumer segments.7 As of August 2001, NHTSA had included three tiers of information on rollover on its website (see Appendix D): The star ratings themselves, along with the vehicle class, drive (front, rear, or 4 × 4), SSF, some vehicle details (including body style and trim, engine, transmission, tire size, and major options such as sunroof), and whether the vehicle has electronic stability control; A description of the rating system, along with frequently asked questions; and Graphics depicting rollover and crash trajectories, frequencies, and rates. 7 A 2000 proprietary survey by the Gartner Group indicated that 45 percent of vehicle purchasers researched online, and 3 percent bought their vehicles over the Internet (Macaluso 2001).
OCR for page 75
An Assessment of the National Highway Traffic Safety Administration's Rating System for Rollover Resistance: Special Report 265 NHTSA also has made the rollover ratings available in a somewhat different format in its Buying a Safer Car, 2001 brochure. As of September 2001, the agency had printed 125,000 copies of this brochure incorporating rollover resistance ratings. All have been distributed to various groups, including manufacturers and repair shops, and at auto shows, car races, and conferences. Partnerships with other organizations, such as the American Automobile Association, have increased distribution further. Use of NHTSA’s Rollover Information on the Internet NHTSA provided the committee with a Webtrends report8 detailing visits to the rollover section of its website (www.nhtsa.dot.gov/hot/rollover/) for a 6-week period from February 28 to May 14, 2001. During that period, almost 275,000 visits occurred to the entire site, or about 3,600 per day, representing approximately 54,600 unique visitors; the average session length was 2 minutes, 22 seconds. The number of visitors in a year is expected to be about an order of magnitude greater, but still a small fraction of all vehicle buyers. Table 4-1 lists some of the most requested pages and their mean viewing times. All of the times exceed 1 minute, with most being around 2 or 3 minutes, suggesting that users are finding the materials of some interest since they do not merely click rapidly through to other sites. Further, from the overall distribution statistics (provided to the committee but not shown in the table), just under TABLE 4-1 Statistics Detailing Visits to the Rollover Section of NHTSA’s Website, February 28 to May 14, 2001 Page Title URL Visitor Sessions Mean Time Viewed (minutes: seconds) Frequently Asked Questions About Rollover Resistance Ratings www.nhtsa.dot.gov/hot/rollover 36,107 1:03 Rollover Rated Vehicle Details www.nhtsa.dot.gov/hot/rollover/fullWebd.html 18,874 3:02 New Car Assessment Program Frontal and Side Crash Test Ratings www.nhtsa.dot.gov/hot/rollover/2001Rollover.html 17,997 3:18 NHTSA’s Rollover Resistance Ratings—Figures www.nhtsa.dot.gov/hot/rollover/figures.html 5,364 2:24 NHTSA’s Rollover Resistance Ratings—Q&A www.nhtsa.dot.gov/hot/rollover/QnA.html 663 1:56 8 wysiwyg://CONTENT.1/http://www.nhtsa.dot.gov/etc/reports/rollover_01_b.html
OCR for page 76
An Assessment of the National Highway Traffic Safety Administration's Rating System for Rollover Resistance: Special Report 265 half of the visits lasted less than 1 minute, 12 percent lasted 1–2 minutes, almost 10 percent lasted 2–3 minutes, almost 7 percent lasted 3–4 minutes, and more than 1 percent lasted up to 10–11 minutes. A review of the statistics also reveals that most visitors to the rollover section of the site (66.3 percent) came there from another part of the NHTSA website. NHTSA does not collect evidence of actual use by surveying visitors to its site or by obtaining direct observation of users accessing the site (e.g., in a field usability test). FUTURE APPROACHES Near Term The committee believes NHTSA could implement significant improvements in its consumer information on rollover in the near term. From a process perspective, the use of recommended practices in developing consumer information and more thorough evaluation of candidate consumer information materials would be beneficial. The committee also urges that in further developing and revising specific features of its rollover information (see Chapter 5), NHTSA reconsider alternatives to the use of stars, expand the level and quality of its hierarchical information, and assist consumers in placing the information in the appropriate context. Use of Recommended Practices Following recommended practices and proven techniques for communicating risk-based information to consumers can help ensure that the information will be effective. In particular, NHTSA would benefit from greater use of an iterative process that takes appropriate account of consumers’ reactions to candidate systems. Moreover, an appropriately constituted external advisory group could provide ongoing evaluation of the process, critique study designs, and recommend quality assurance steps. Evaluation NHTSA needs to test the effectiveness of consumer communications before putting them into widespread use, and continue to evaluate them once in use. The agency needs to evaluate rigorously how people are using the rollover resistance ratings in the context of the entire NCAP information program. Such evaluations could include experimental studies and protocol analysis of consumer understanding and use of the ratings, tracking of hotline calls, and periodic surveys of recent new car buyers to learn whether they were aware of the ratings when they purchased their vehicles and how they used this information in decision making.
OCR for page 77
An Assessment of the National Highway Traffic Safety Administration's Rating System for Rollover Resistance: Special Report 265 Use of Stars NHTSA would be well advised to reconsider whether stars are appropriate for conveying rollover ratings to consumers. One clear advantage to using stars is that many consumers are familiar with their use for other ratings.9 However, the limited evidence gathered from focus groups indicates some comprehension problems, perhaps arising from the superficial similarity of the star rollover ratings to the NCAP frontal and side test crash ratings. These latter ratings are based on crash tests and provide information about a vehicle’s crashworthiness, whereas the rollover ratings relate to a vehicle’s crash propensity. In response to the focus group findings, NHTSA explicitly addressed two potential misconceptions: the explanatory text accompanying the ratings notes that they do not predict the likelihood of a crash or predict directly the risk of death or injury. The current practice of having these corrective statements embedded in the description of the rating system—instead of being provided directly with the ratings—reduces the likelihood that consumers will read that information. One alternative to the star ratings would be to present SSF alone (or a rescaled version, for example, on a scale of 0–100), with a brief explanation of how it is calculated and a simple description of the relative stability of different vehicle classes (e.g., passenger cars, SUVs). Another option might be to present a rollover rating—instead of a rollover resistance rating—using a tilted vehicle or a similar graphic. Any candidate system would require appropriate testing to assess its effectiveness, using the procedures already discussed. Hierarchical Information Although some consumers may be content simply to know that “more stars means safer,” evidence from focus groups (Equals Three Communications 2000) suggests that others would like more information on how to avoid a rollover crash. An improved hierarchy of information, at increasing levels of detail, would reduce the need for information designers to make trade-offs to favor simplicity. Hierarchical organization within a page and within a website helps the reader keep track of where items are and how they are related. Hierarchical organization also facilitates searches for more details or for the bigger picture. For example, the top level of a “Buying a New Car” website could comprise a table of contents (or a site map), together with a description of the objectives of the site. This top level would set the context (see below) for more specific site information. The table of contents would provide links to items at the next level down in the hierarchy, such as vehicle reliability and vehicle 9 Many focus group participants noted the similarity of the stars to hotel ratings or to the five-star safety ratings in advertisements from automobile manufacturers (Equals Three Communications 2000).
OCR for page 78
An Assessment of the National Highway Traffic Safety Administration's Rating System for Rollover Resistance: Special Report 265 safety. The vehicle safety web page would be organized similarly, with a general overview and links to specific topics such as the vehicle, the driving environment, and the driver. Under the vehicle category, the reader would find information on vehicle safety ratings, including the rollover resistance ratings. The committee believes it would be desirable for NHTSA to expand both the levels and quality of its hierarchy of information on rollover, particularly since the Internet is the agency’s primary means of disseminating rollover information and is highly conducive to hierarchical presentation of information. For example, information on how the rollover resistance ratings are calculated and assigned, including the rollover curve, could be added to the site for consumers interested in this level of detail. Context Rollover is one of many crash risks consumers may consider when purchasing a vehicle. How this risk should be weighted relative to other crash risks or other safety information is contingent on driver behavior. NCAP does not address the issue of combining ratings for crashworthiness and rollover resistance, although the NHTSA website includes links to sites of other organizations that crash test vehicles and provide the former ratings.10 This feature may be helpful for consumers seeking to put ratings in context, but additional information from NHTSA on how to combine the different NCAP ratings would be helpful for consumers seeking to make informed car-buying decisions. The experience of the risk assessment community in combining risk measures could be helpful for putting different measures of risk in context (see, for example, Garrick and Kaplan 1995). Longer Term Comprehensive Rollover Rating One of the committee’s recommendations in the area of vehicle dynamics (see Chapter 2) is that NHTSA pursue the use of dynamic testing to supplement the information provided by SSF (see Chapter 5). Once appropriate dynamic testing has been selected, NHTSA will need to consider how best to communicate to consumers the combined information about static and dynamic factors related to rollover. Consumer Reports currently provides information on dynamic testing of rated vehicles, but the committee lacks empirical evidence about consumers’ use of this information. There is no evidence indicating whether consumers can or cannot understand and interpret infor- 10 Insurance Institute for Highway Safety (United States), National Organization for Automotive Safety and Victims’ Aid (Japan), National Roads and Motorists’ Association Crash Testing (Australia), European New Car Assessment Programme (Europe).
OCR for page 79
An Assessment of the National Highway Traffic Safety Administration's Rating System for Rollover Resistance: Special Report 265 mation from dynamic tests. Nevertheless, the use of established procedures to develop and evaluate such information should help ensure that consumers will be able to understand and use it. Single Summary Measures The committee’s discussions with representatives of Consumers Union and others indicated that many consumers simply want to know “how safe this vehicle is” and “how this vehicle compares with alternatives in terms of safety.” Currently, consumers cannot find simple answers to these questions. Safety depends on many factors, including the vehicle’s design, road conditions, weather, and driver behavior. Summary measures of vehicle safety can assist consumer decision making by simplifying comparisons of safety attributes among vehicles. A number of summary rating measures of vehicle safety are being developed and employed around the world, all using different data and different methods to combine the data. This year, for example, Consumers Union introduced for the first time an overall vehicle safety assessment that integrates key information from its own performance tests for crash avoidance with the crashworthiness information available from NHTSA and the Insurance Institute for Highway Safety (Consumer Reports 2001). Other summary rating systems are used in the United States (e.g., www.crashtest.com) and around the world (e.g., New Car Assessment Japan, Euro NCAP). The Safety Rating Advisory Committee (SARAC)—made up of international safety experts—has investigated the validity of various rating systems and the conflict that could occur when those systems result in different rankings for the same vehicle. SARAC found that different approaches can result in sizeable differences in vehicle rankings (Zeidler 2001), and that reaching consensus is difficult.11 There are also some drawbacks to providing a single summary measure. A major concern is that such a measure could conceal large variations in performance across different test types, with the possible result of masking some poor performers.12 Despite these concerns, the committee suggests that, in the context of its longer-term initiatives to improve consumer automotive safety information, NHTSA monitor efforts by groups in the United States and overseas to develop summary measures of vehicle safety performance and review the options for developing its own summary measure of overall motor vehicle safety. 11 In lieu of collaborative international efforts such as SARAC, countries may (and do) adopt differing approaches to vehicle safety ratings and information. What the effects of such differences are or might be is an empirical question beyond the scope of this study. 12 For example, consider two vehicles that have the same overall “acceptable” rating according to a system that categorizes vehicles as good, acceptable, marginal, or poor. Whereas one vehicle could have acceptable ratings across the board, the other could be good in some areas and poor in others. In general, these concerns could be addressed by making the measure transparent and providing a matrix with the ratings of individual components, for example.
OCR for page 80
An Assessment of the National Highway Traffic Safety Administration's Rating System for Rollover Resistance: Special Report 265 FINDINGS AND RECOMMENDATIONS Findings 4-1. There is a gap between recommended practices for the development of safety information and NHTSA’s current process for identifying and meeting consumer needs for such information. 4-2. The focus group studies NHTSA used to develop its star rating system for rollover resistance were limited in scope and inadequate in design. Furthermore, empirical studies have not been undertaken to evaluate consumers’ use of the ratings in judgments about vehicle safety or purchase decisions. 4-3. The information accompanying the rollover resistance ratings does not explain how to use them in the context of other safety ratings and information or provide specifics for the information-seeking consumer, such as how the ratings were derived. Recommendations 4-1. NHTSA should implement an ongoing process for developing and evaluating its consumer vehicle safety information. 4-2. NHTSA should give consumers more information that places motor vehicle risks in an overall context, and rollover risks specifically within that larger context. A hierarchical presentation of information could be beneficial in meeting varied consumer needs. 4-3. NHTSA should continue to investigate presentation metrics other than the current rollover resistance stars, given that the lack of resolution and context in the star rating system and the system’s superficial resemblance to the NCAP crashworthiness ratings could mislead consumers. 4-4. NHTSA should monitor efforts by groups in the United States and overseas to develop summary measures of vehicle safety performance, and review options for developing its own summary measure of overall motor vehicle safety. REFERENCES Abbreviation TRB Transportation Research Board Automotive News. 2000. Safety Steps into the Spotlight. March 6, p. 3. Bettman, J.R. 1979. An Information Processing Theory of Consumer Choice. Addison-Wesley Publishing Company, Mass.
OCR for page 81
An Assessment of the National Highway Traffic Safety Administration's Rating System for Rollover Resistance: Special Report 265 Bettman, J.R., E.J. Johnson, and J.W. Payne. 1991. Consumer Decision Making. In Handbook of Consumer Behavior (T.S. Robertson and H.H. Kassarjian, eds.), Prentice-Hall, Englewood Cliffs, N.J. Bettman, J.R., M.F. Luce, and J.W. Payne. 1998. Constructive Consumer Choice Processes. Journal of Consumer Research, Vol. 25, pp. 187–217. Consumer Reports. 2001. Which Cars Are Safer? Annual Auto Issue. pp. 16–19. Eberhard, J., and P. Green. 1989. The Development and Testing of Warnings for Automotive Lifts. Technical Report UMTRI-89-26. The University of Michigan Transportation Research Institute, Ann Arbor. Equals Three Communications. 1999a. Utility Vehicle Rollover Safety Label— Summary of Research Findings, Final. Prepared for NHTSA, Office of Planning and Consumer Programs, Consumer Automotive Safety Information Division, under contract No. DTOS59-95-00408. Bethesda, Md. Equals Three Communications. 1999b. Research Report to the National Highway Traffic Safety Administration, Consumer Automotive Safety Information Division. Results of focus groups in Dallas, Texas, Overland Park, Kansas, and Richmond, Virginia, regarding rollover labeling and information. Bethesda, Md., May 13. Equals Three Communications. 2000. Focus Group Report: Vehicle Rollover Focus Groups. Prepared for NHTSA. Bethesda, Md., Dec. 15. Federal Register. 2000. Consumer Information Regulations; Federal Motor Vehicle Safety Standards; Rollover Prevention; Request for Comments. Vol. 65, No. 106, June 1, pp. 34,998–35,024. Fischhoff, B., P. Slovic, and S. Lichtenstein. 1978. Fault Trees: Sensibility of Estimated Failure Probabilities to Problem Representation. Journal of Experimental Psychology: Human Perception and Performance, Vol. 4, pp. 330–344. Garrick, B.J., and S. Kaplan. 1995. Summary Report: A Risk-Based Approach to the Evaluation and Application of Disarmament Strategies and Technologies. North Atlantic Treaty Organization Advanced Workshop, Visegrád, Hungary, May 19–23. General Motors Corporation. 1994. Continuous Automotive Market Information Program Buyer Behavior. Green, P. 1979. Development of Pictographic Symbols for Vehicle Controls and Displays. SAE 790383, Society of Automotive Engineers, Warrendale, Pa. Green, P. 1993. Symbols for Controls and Displays. In Automotive Ergonomics, (B. Peacock and W. Karwowski, eds.), Taylor and Francis, London, United Kingdom, pp. 237–268. Hansen, F. 1972. Consumer Choice Behavior—A Cognitive Theory. The Free Press/Macmillan, New York and London. Howard, J.A., and J.N. Sheth. 1969. The Theory of Buyer Behavior. Wiley, New York.
OCR for page 82
An Assessment of the National Highway Traffic Safety Administration's Rating System for Rollover Resistance: Special Report 265 Insurance Research Council. 1999. Public Attitude Monitor 1999. Vehicle Safety Issues, Vol. 4, available from Insurance Research Council, American Institute for Chartered Property Casualty Underwriters, 718 Providence Road, P.O. Box 3025, Malvern, PA 19355-0725, telephone 610-644-2212, www.ircweb.org. Isenberg, D. 1986. Group Polarization: A Critical Review and Meta-Analysis. Journal of Personality and Social Psychology, Vol. 50, pp. 1141–1151. Levin, I.P., S.L. Schneider, and G.J. Gaeth. 1998. All Frames Are Not Created Equal: A Typology of Framing Effects. Organizational Behavior and Human Decision Processes, Vol. 76, pp. 149–188. Macaluso, N. 2001. Car Dealers Look For an Online Jump Start. E-Commerce Times, July 23. www.ecommercetimes.com/perl/story/12189.html. Merton, R.K. 1987. The Focused Interview and Focus Groups: Continuities and Discontinuities. Public Opinion Quarterly, Vol. 51, pp. 550–556. Merton R.K., and P.L. Kendall. 1946. The Focused Interview, American Journal of Sociology, Vol. 51, pp. 541–557. Merton, R.K., M. Fiske, and P.L. Kendall. 1956. The Focused Interview: A Manual of Problems and Procedures. The Free Press, Glencoe, Ill. Merton, R.K., M. Fiske, and P.L. Kendall. 1990. The Focused Interview: A Manual of Problems and Procedures. 2nd ed. The Free Press, New York. Miller, G. 1956. The Magical Number Seven, Plus or Minus Two: Some Limits on Our Capacity for Processing Information. The Psychological Review, Vol. 63, pp. 81–97. Morgan, M.G., B. Fischhoff, A. Bostrom, and C. Atman. 2001. Risk Communication: A Mental Models Approach. Cambridge University Press, Cambridge, U.K. Mudd, S.A., and R. Karsh. 1961. Design of a Picture Language to Identify Vehicle Controls: I General Methodology; II Investigation of Population Stereotypes. Technical Memorandum 22–61, U.S. Army Human Engineering Laboratory, Aberdeen Proving Ground, Md. Myers, D., and H. Lamm. 1976. The Group Polarization Phenomenon. Psychological Bulletin, Vol. 83, pp. 602–627. Nancy Low and Associates, Inc. 1996. Final Report on Focus Group Study of Rollover Research. Chevy Chase, Md. Nicosia, F.M. 1966. Consumer Decision Processes: Marketing and Advertising Implications. Prentice-Hall, Engelwood Cliffs, N.J. Plous, S. 1993. The Psychology of Judgment and Decision Making. McGraw-Hill, N.Y. Schriver, K.A. 1996. Dynamics in Document Design: Creating Text for Readers. John Wiley & Sons. Simonson, I., Z. Carmon, R. Dhar, A. Drolet, and S.M. Nowlis. 2001. Consumer Research: In Search of Identity. Annual Review of Psychology, Vol. 52, pp. 249–275.
OCR for page 83
An Assessment of the National Highway Traffic Safety Administration's Rating System for Rollover Resistance: Special Report 265 TRB. 1996. Special Report 248: Shopping for Safety: Providing Consumer Automotive Safety Information. National Research Council, Washington, D.C. University of Michigan. 2001. Contribution of the Automotive Industry to the U.S. Economy in 1998: The Nation and Its Fifty States. Prepared for the Alliance of Automobile Manufacturers, Inc., and the Association of International Automobile Manufacturers, Inc., by Institute of Labor and Industrial Relations, University of Michigan; Office for the Study of Automotive Transportation, University of Michigan Transportation Research Institute; and Center for Automotive Research, Environmental Research Institute of Mich. Whyte, G. 1993. Escalating Commitment in Individual and Group Decision Making: A Prospect Theory Approach. Organizational Behavior and Human Decision Processes, Vol. 54, pp. 430–455. Wogalter, S.M., V.C. Conzola, and W.J. Vigilante, Jr. 1999. Applying Usability Engineering Principles to the Design and Testing of Warning Messages. Proc., 43rd Annual Meeting of the Human Factors and Ergonomics Society, pp. 921–925. Zeidler, F. 2001. Problems in Defining Safety and Consequences for Consumer Information. Safety Rating Advisory Committee.
Representative terms from entire chapter: