Leveraging ITS Data for Transit Market Research: A Practitioner's Guidebook (2008)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

43 In 1997, TCRP convened a task group to explore strategic research initiatives that would guide fundamental change in the transit industry. In a future search conference, the task group developed a characterization of conditions in the tran- sit industry that, in its assessment, represented a crisis (TCRP 1998). The crisis condition was seen to be a consequence of the industry’s entrenched commitment to an “Efficient Tran- sit Performance” paradigm organized around a command and control philosophy bound to a dependence on subsidy and a perceived inability to influence revenues. In contrast, the task group developed a vision of an alternative para- digm—“Change, Growth, and Mobility”—organized around a philosophy of shared responsibilities bound to a reliance on information and a commitment to customer service. The outgrowth of the future search exercise was a commit- ment to fund a New Paradigms research initiative within TCRP. The first product of the initiative was TCRP Report 53, “New Paradigms for Local Public Transportation Organiza- tions,” (Cambridge Systematics 1999). Among other things, TCRP Report 53 concluded that the transit industry needed to move beyond performance measures focused solely on operat- ing efficiency to include measures “. . . that describe the attrib- utes of the product (service) as perceived by the user” (Cam- bridge Systematics 1999: 6-11, emphasis added). Implicit in this conclusion was the recognition that a greater focus on cus- tomers depended on a deeper commitment to understanding and acting on customer perceptions, preferences, opinions, and attitudes. In turn, this commitment would require a re- casting of marketing beyond its traditional focus on customer relations and promotion to a more expanded mission that also embraced customer and market research. Coinciding with the New Paradigms initiative, an accelerat- ing technological transformation in the transit industry began delivering a vast amount of information about service deliv- ery and customers. Although the industry’s transformation from “information poor” to “information rich” status was clearly anticipated in the New Paradigms initial future search exercise, perspectives on how this information would con- tribute to a customer-oriented approach to service changed. Initially, the New Paradigms focus centered on tapping infor- mation technologies to develop new services for customers, with Web and phone-based trip planning software and real time vehicle arrival information serving as examples. Subse- quently, the focus broadened to include operations informa- tion recovered from on-vehicle systems. These systems were clearly capable of producing information to support tracking service delivery performance, contributing to the efficiency objectives of the traditional paradigm. However, as was rec- ognized in TCRP Report 97, “Emerging New Paradigms: A Guide to Fundamental Change in Local Public Transportation Organizations” (Stanley et al. 2003), much of the information from on-vehicle systems was not only customer-relevant, it could also be coordinated (in some instances, linked) with customer and market information to produce a more com- plete picture of the quality of customers’ experiences. As envi- sioned in TCRP Report 97, this coordination would lead to performance assessment capabilities “. . . that bring into bal- ance the quality of the customer’s experience (the emerging strategic goal) and the efficiency with which resources are used (the production goal)” (Stanley et al. 2003: 2-4). The perspective on coordinated applications of customer and market information with ITS data developed in TCRP Report 97 represents the starting point for the present Guidebook. Overall, the Guidebook’s purpose has been to illustrate how ITS data can be used in tandem with infor- mation recovered by traditional market research tools. In some instances, this involves applications where ITS data leverage or facilitate traditional market research practices. In other instances, this involves coordinating information from market and customer research with information from ITS technologies. And in a few instances, it involves a substitution of information provided by ITS technologies for information that had been previously obtained through manual practices. C H A P T E R 5 Lessons Learned, Issues, and Concerns

The scope of the Guidebook goes beyond explaining how ITS data can be used to address specific customer and market research questions or cataloguing specific applications. As in the case of moving from the efficient performance to the cus- tomer-focused paradigm, the transition from traditional data collection to effective use of ITS data in customer and market research involves a number of intermediate steps and re- quires coordination across the agency. In the present case, the intermediate steps can be traced from technology acquisition to systems integration to data processing and management to the development of new reporting and analysis tools to staff development. This approach was considered necessary be- cause early experiences indicated that information from ITS technologies was generally being underutilized in the transit industry (Casey 2000, Kemp 2002). This chapter takes a technology life cycle approach to sum- marize the lessons learned from efforts to fold ITS data into customer and market research practices. In this approach, the life cycle is divided into four stages: (1) systems acquisition, (2) data management, (3) market research (or data analysis), and (4) decisionmaking. The four stages of the life cycle are recursive in the sense that conditions or limitations that arise in a given stage tend to carry over and have consequences for subsequent stages. Beginning at the systems acquisition stage of the ITS life cycle, the experiences of the case study properties and rec- ommendations of the Transit Standards Consortium (FTA 2005) suggest the following: • Involve key stakeholders on systems procurement teams to ensure that the data produced by each system is compatible with that produced by other systems and that duplication among systems is minimized. Think ahead. A “stovepipe” approach to procurement can result in integration and interoperability problems as new systems are added. Stovepiping can be avoided by adopting a “data-centric” approach to procurement rather than an “application- centric” approach. • Specify data integration and interface requirements in the procurement process. • Be sure to have complete documentation for each system. It is important to know exactly how data are produced. • Form an organizationwide data committee to ensure that the data recovered by ITS technologies are compatible with the needs of end users. At the data management stage of the life cycle, lessons from the experiences of the case study properties include the following: • The existence of standard industry templates for data mod- els, applications, and data management would have spared agencies the need to re-invent the wheel in developing information systems for ITS data. Standardization is fol- lowed in other industries and results in more cost-effective and quicker implementation. • Each agency should develop a comprehensive technology plan to document and prioritize technology strategies for the agency. The plan should cover all ITS technologies and include budget and staffing impacts. As an agencywide planning and budgeting tool, the technology plan would serve management by identifying and scheduling actions that must be taken through implementation. • The business units responsible for maintaining data in the agency’s database should monitor and ensure the validity and integrity of the data. As one person at a case study prop- erty observed, “Bad data can ruin trust.” Post processing is a necessary step to ensure data integrity. • Transit properties that are a part of city or county govern- ments generally benefit from having access to city or county-level IT resources, but IT staff at these levels often lack familiarity with ITS data. Coordination with IT can also be more difficult. • Invest in developing and maintaining meta-data and data dictionaries for the ITS databases. Researchers’ credibility is at stake when they use ITS data, and they need to under- stand limitations of the data. Also, researchers can’t tap the potential of ITS databases if they don’t know the details of the data. • Invest in training to support agencywide development of staff capabilities in using new enterprise applications for analyzing ITS data, such as GIS. At the market research stage of the life cycle, lessons can be drawn from the experiences of the case study properties and literature addressing the strategic role of marketing and mar- ket research (Cronin and Hightower 2004, Fielding 1987, Stanley et al. 2003) and workforce development (TCRP 2001). These lessons include the following: • The experiences of the case study properties and responses to the 2005 survey suggest that there is no “right” location of the market research function within the organization that opti- mizes the use of ITS data. At one case study property (CTA), market research resides within operations; at a second (TriMet), it is placed in the same division as IT; and at the third (Madison Metro), it exists as an independent entity. The New Paradigms approach, which emphasizes greater co- ordination between marketing and service development and delivery, would likely find advantages in the CTA alternative. The case studies also found synergistic spillover benefits from operations staff responsible for validating and maintaining ITS data from on-board systems to staff that analyze ITS data in monitoring and evaluating service delivery. 44

• Every property should prepare a marketing plan that identifies market research needs and establishes linkages to service de- velopment and operations plans. Ideally, marketing and oper- ations plans would be coordinated, as envisioned by Fielding (1987). Apart from being a logical thing to do, such coordina- tion would help to break down the traditional culture in the industry “. . . in which operational concerns have been viewed as of paramount strategic importance and customer concerns largely subordinated” (Stanley 2003: 3-14). • Reports on service delivery performance produced by ITS vendor developed software should be viewed as a starting point in using ITS data for evaluating service de- livery and leveraging market research. Staff at the case study properties have moved well beyond such reports, developing new performance indicators that are more closely aligned with customer satisfaction, defined through market research. • Peer exchange (e.g., Gross et al. 2003) holds great potential for diffusing state-of-the-art practices at this juncture of the transit industry’s ITS data transformation. Many of the innovative applications of ITS data that have been devel- oped at the case study properties have not been communi- cated to the rest of the industry. • Analysis of ITS data is currently limited to a few highly skilled persons who produce summary reports and do cus- tomized queries to address specific questions. They are somewhat concerned that the evolution toward wider access to ITS microdata could lead to misuse or misinterpretation and believe that general access should be limited to sum- mary data. • Apart from the difficulties of filling vacated positions, turnover of ITS data analysts interrupts the momentum of moving ITS data into research practice. At the case study properties, ITS data managers and analysts were investing considerable time building relationships with practitioners and decisionmakers to gain a better understanding of needs and opportunities. • Traditional practices in market research, service planning, and scheduling are resistant to change. New tools and re- ports using ITS data to support practices and decisions in these areas need to be “sold.” Reports or documents are not likely to be read. “Seminars” tend to be a more effec- tive way of engaging staff, providing an opportunity for staff to suggest improvements. • Transit properties commonly recognize the customer serv- ice benefits of ITS associated with trip planning software, automated stop announcements, real time vehicle arrival information, and using the AVL “playback” function to follow up on customer complaints. Surveys of riders and area populations have found that satisfaction and percep- tions of service quality have been positively affected by these services. • While there appears to be no practical limit to the number of service performance measures that can be derived from ITS data, this shouldn’t be interpreted as a license to over- whelm managers with information. Information overload is more likely to occur in the age of ITS data and is a symp- tom of inadequate communication between analysts and managers. • Market research departments commonly have few staff (the median full-time equivalent [FTE] in the 2005 survey was three) and survey projects are usually contracted out. Some of the leveraging opportunities involving ITS data are in supporting logistical aspects of survey research ac- tivity. Bringing contractors “up to speed” in understand- ing how to use the agency’s store of ITS information is an issue that market research staff will need to address. • One consequence of the ITS data transformation is that the skills needed among new hires are those that the industry is now having greatest difficulty recruiting and retaining (TCRP 2001). One strategy for getting ahead of the curve on this acute problem is to take a more aggressive approach with internships. The next generation of market re- searchers already has a positive attitude toward the transit industry, thanks to perceptions of its social and sustain- ability benefits. The industry would be more successful capitalizing on these perceptions now rather than at the point where it has greater difficulty competing with other industries in the full-time job market. Also, internships that are institutionalized through formal agreements, as is the case at CTA, have greater prospects for sustained suc- cess. The gap in compensation between the transit indus- try and private alternatives for persons in the marketing field, nevertheless, represents a serious barrier to hiring and retaining persons with the skills to analyze ITS data. The compensation gap appears to be greater for properties that are part of city or county governments, where pay is dictated by governmentwide pay scales (TCRP 2001). • Related to internships, the transit industry should not as- sume that the curricula of the disciplines that are educat- ing the next generation of market researchers are evolving to develop the skills that will be needed. The four largest “suppliers” of graduates to marketing programs in the transit industry—Marketing, Planning, Business, and Journalism (Cronin and Hightower 2004)—generally do not have a tradition of developing the skills that were demonstrated by staff that were the most active users of ITS data at the case study properties. Interns serve as a bridge between the transit workplace and education pro- grams, and internship programs provide a mechanism for students and the transit industry to communicate their skill needs to education programs. • Another avenue for supporting workforce development and the development of new practices using ITS data is 45

represented in the University Transportation Centers (UTC) program, administered by the Research and Inno- vative Technology Administration of the U.S. DOT. Many of the current 68 UTCs (located at 64 universities) have re- search and technology transfer themes that include ITS and transit. Projects that jointly engage market research staff and university faculty can produce cumulative bene- fits over time: (1) research can focus on developing new tools for analyzing ITS data, (2) UTC matching helps to leverage market research program resources, (3) involve- ment of graduate assistants helps to direct student interest toward careers in the transit industry, and (4) technology transfer activity can include training market research staff in ITS data applications. The experiences of two of the case study properties (CTA and TriMet) with UTCs indicate that the transit-university relationship has benefited both entities, especially when it is sustained over time and the partners are able to gain a better understanding of each other’s respective needs and expectations. At the fourth stage of the life cycle, the value of ITS data in market research is realized when the products of market research are used to inform management decisions. The “success stories” of the case study properties, in which ITS data are used to leverage or reinforce traditional market re- search practices, showcase outcomes that correspond very well to characteristics of effective market research programs presented in TCRP Report 37, “Integrating Market Research into Transit Management” (Elmore-Yalch 1998b). One sign of an effective market research program identi- fied in TCRP Report 37 is its ability to move the organization beyond stated commitments to being customer-oriented to coordinating practices across departments that demonstrate an ability to follow through. The coordination of market re- search and operations functions may be most important in this context because operations is responsible for developing and delivering the “product” to transit riders. Here, market research provides direction to analysts who are developing and monitoring service using ITS data. At TriMet, for exam- ple, customer satisfaction surveys were the catalyst that served to focus the attention of operations analysts on the root causes of unreliable service. In this instance, reliability prob- lems were traced to late departures from garages and termi- nals, and managers were able to clearly see a connection be- tween improving departure times and improving customer satisfaction. Operations managers at the CTA were similarly motivated when a connection was found between rider satis- faction surveys and the incidence of “bus bunching” (docu- mented through ITS data analysis). In this case, a “customer wait index” was developed from ITS data that allowed man- agers to track whether operations control practices were working from the customer’s perspective. A similar story can be told about market segmentation studies, where the identi- fication of latent demand through market research informs service planning. A second sign of an effective research program is its ability to reduce the uncertainty that a manager faces in making de- cisions. Even the best designed, executed, and presented mar- ket research studies can only reduce uncertainty, not eliminate it. From a manager’s perspective, the power of the informa- tion contained in a market research study is enhanced when that information is augmented or reinforced by information from other sources. Thus, when information from customer satisfaction surveys is coordinated with ITS information on service delivery, the power of the survey information is enhanced. It is something of a paradox that, at a time when transit data have never been more plentiful, managerial deci- sions often continue to be made on the basis of judgment and experience. Leveraging traditional market research informa- tion with ITS information will help to build the trust and con- fidence that managers need to make decisions based on re- search rather than judgment. A third sign of an effective market research program is its ability to provide assessment information to a manager after a decision is made. Managers need to know whether the con- sequences of their decisions play out as expected or play out in other ways. When monitoring and evaluation become sus- tained practices, managers will “learn” from their decisions and, with accumulated knowledge, will make subsequent decisions with greater confidence and trust. The traditional approach to evaluation has been to conduct “before and after” studies. When market research studies are coordinated with service delivery monitoring drawing on ITS data, follow- up evaluation can begin immediately and run continuously until the next market research study. Using ITS data to monitor service delivery should be con- sidered a supplement rather than a substitute for traditional market research. In a few instances, however, ITS data may provide more reliable information to support decisionmak- ing. An example is Madison Metro’s use of magnetic stripe card data to document pass program patronage, which in- forms the agency’s negotiation of pass program agreements with local institutions. The traditional alternative, where agreements would rely on information from self-report sur- veys of transit use, would be subject to unknown levels of self-selection and reporting bias. Looking across the four stages of the technology life cycle the most apparent overall lesson learned is that success in using ITS data for market research depends on agencywide coordination and communication. It is a rare instance where the responsibility for system deployment, data management, service delivery monitoring, and market research is confined to one division in an agency. Ensuring that all stages of the technology life cycle are coordinated is thus an executive 46

management responsibility. Shouldering this responsibility may place some executives in unfamiliar territory, especially at smaller properties, and the situation is further complicated by the insularity that often exists among agency divisions. Preparing a comprehensive technology plan provides a means of coordinating activities that are distributed across the agency. The most effective plans will look beyond capital and hardware issues to include resources needed to support changes in the data management infrastructure, as well as staffing and training needs in the end use departments. A comprehensive planning process will also force insular inter- ests to coordinate their approaches to system implementation, which helps to ensure that ITS data will be successfully recov- ered, validated, stored, and analyzed. 47