Multiagency Electronic Fare Payment Systems (2017)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

35 Fare payment systems, particularly ones that must accommodate multiple modes and agencies, differ from other large equipment-based procurement and deployment projects. This synthesis paper aligns the modern EFPS with information-driven systems that do not follow the typical construction or infra- structure models, arguing that a different procurement and development model should be followed: “Unlike rolling stock and heavy infrastructure projects . . . payment systems [live in the] continually evolving fields of telecommunications and information science . . . [and] often require systems integra- tion and need extensive coordination with supporting capital projects” (Smart Card Alliance 2010). Fare payment systems often span agency-wide business areas, covering all modes (as well as paratransit and demand-responsive services), financial management, performance, planning, operations, and customer services. In rolling out systems, agencies or regional organizations must consider the approach used to plan, procure, deploy, transition/replace, and phase out the old system. Planning for a new or expanded multiagency EFPS is evolving. TCRP Report 131: A Guidebook for the Evaluation of Project Delivery Methods (Touran et al. 2009) showed how procurement methods have evolved since 1972, when the design-bid-build (DBB) approach was enacted as part of the Brooks Act (Public Law 92-582); and later expanded to Design-Build-Operate-Maintain (DBOM) as part of the Intermodal Surface Trans- portation Efficiency Act of 1991. Since that time, agencies have tried numerous approaches to pro- cure a range of planning, development, operations, and maintenance approaches related to different services associated with the multiagency EFPS. Current procurement and deployment strategies try to address different challenges and attempt to avoid one or more of these following problems: • Conflicts among multiple contractors • Vendor lock-in • Adversarial relationships between agency and contractor • Funding cycles over the life of the project (which may last six to 10 years) • Multiple agency rollouts • New technologies testing • Technology obsolescence. Neither the literature nor this survey has identified a silver bullet approach to procurement deploy- ment. This section reviews the categories and tactics taken by different agencies to contract and deploy their services. FARE PAYMENT SYSTEMS AND SERVICES A multiagency EFPS is typically composed of a large variety of equipment and services. Systems and services may include: • Field equipment, such as point of entry readers and point of sales devices (ticket vending machines and sales equipment) • Customer sales services (websites, institutional sales site, IVR, customer service center) chapter five PROCUREMENT AND DEPLOYMENT STRATEGIES

36 • Retail and distribution network • Back office management systems • Inspection devices and app • Mobile fare or ticketing app • Revenue collection • System integration services. Equipment may include services such as design, development, and installation of equipment, operation, preventive/corrective maintenance, communications, data center, and software maintenance. Outsourced services may include elements of system integration, operations, and supply replenish- ment (e.g., managing and restocking cards in TVMs). Additionally, agencies may employ a program management consultant to support procurement and deployment. These services may be supported through multiple procurements. In the survey, respondents were asked about their equipment and if services were procured using multiple contracts or through a single contractor. The results are described in the discussion of staged procurement strategies. Procurement and Contract Models Multiagency EFPS can employ a variety of different procurement strategies. The approach taken by a region to deploy a multiagency EFPS may depend on state procurement rules and regulations. When federal funds are used, agencies must also follow federal rules. Guidelines developed by the SCA in 2010 define the dimensions of a procurement model (see Figure 17 and Appendix D: Procurement Dimensions for the definitions). Table 7 shows how the definitions of contracting alternatives in the SCA model relate to the types of procurement strategies asked of the survey respondents. Rollout The system rollout can make the multiagency EFPS a success or a public failure. TCRP Report 115 on smart card interoperability discusses two major aspects, the participant phase-in and capability phase-in: For example, are all rider categories deployed with all the functionality deployed at one time (“big bang”) or does a program deploy some of the functionality with limited participation (“limited” or “limited launch”). Sometimes even “limited launch” approaches contain too many functions to test incrementally (Figure 18). Contracng Alternaves Consulng Services Systems Manager Design/Build Task Order Method of Award Low Bid Negoaon Best Value Sole Source Contract Form Phased Contracts Task Order Purchase Orders Contract Type Fixed Price Cost Reimbursement Time and Materials Incenve Contracts FIGURE 17 Dimensions of a procurement model (Source: adapted from SCA 2010).

37 Contracting Alternatives Project Responsibilities. Survey Category Consulting Services The agency selects a consultant to design a system. The system design constitutes system requirements and specifications. The contract may include services to assist the agency during system implementation. N/A Systems Manager The agency hires a system manager through a consultant selection process. The manager participates in all phases of system implementation, including planning, design, development, and testing. System integrator (vendor) Design/Build A design/build agreement provides for design and construction of improvements by the contractor and is often preceded by preparation of a partial design (sometimes designated as a 30% design). Design/build Task Order Unlike the above options, task orders do not assign project responsibility but are used to acquire services or supplies as needed during the project. Task orders are used in conjunction with either the systems manager or design/build alternative. Each element in the DBOM Source: Word table, information from SCA (2010). TABLE 7 DIMENSIONS OF A PROCUREMENT MODEL—DEFINITIONS FIGURE 18 Rollout strategy and dimensions (Source: Acumen Building Enterprise 2006).

38 Some systems now use an agile methodology for software development. Agile software development is a method that develops software incrementally: A limited number of functions are developed, tested and released; and when these are nearly bug-free, a new set of functions is released in the same man- ner until all the functions have been thoroughly tested and approved. Mobile app development and legacy migration strategies tend to adopt this type of approach. Some recent procurement documents have requested an agile approach (in its 2014 RFP, DART asked respondents to address how they would use an agile approach to deploy its next generation system.) Rollout may occur over a long period of time. Large systems may need to incorporate participants many years after the initial deployment. Even after 15 years, the MTC Clipper program is still adding new agencies to their program. Participant phase-in may require added capabilities to accommodate different fare structures or modes. CTA and PACE deployed systems with similar fare structures—mostly flat fares and validation on entry. For Ventra, the fare structure drove the deployment strategy: According to the Ventra program manager, Metra waited several years until a mobile app was deployed before it joined the Ventra program (M. Gwinn, Ventra Program Manager, personal communication, Feb. 2016). Metra, a commuter rail service, employs a distance-based fare structure and proof-of-payment validation. Gwinn suggested that phased deployments may be rolled out by different capabilities such as: • Mode or service • Fare structure • User media (smart card vs. mobile app vs. limited use) • User group (regular, commuter, institutional, or other rider class) • Equipment type (e.g., turnstile, TVM, web, IVR) • Media sales distribution method. SURVEY RESPONSES: PROCUREMENT STRATEGIES Several questions were asked of respondents about their contracting and procurement strategies. The questions requested information on contracting alternatives; procurement and staging strategies; reasons for the agency’s approach; challenges agencies faced; and lessons learned. Contracting Alternatives The majority of these deployments began in the early 2000s. The results of the survey show that the contracting strategies selected in the early years of these regional systems are mostly turnkey (see Table 8). As fare payment projects require more flexibility and phased deployments, agencies are moving toward hiring system integrators and executing multiple contracts. Observations from the responses included the following: • Most agencies opted for a turnkey implementation which included DBOM. • Several organizations opted for a pre-qualification to limit the selection to only qualified bidders. • Several organizations acted as their own integrators (through multiple bids) whereas others hired vendors to act as system integrators (including building a team of multiple vendors and equipment suppliers). • Many organizations included a pilot stage with their deployment. • A few organizations developed a public-private partnership. These were limited to mobile fare payment and ticketing apps that were typically not integrated with their comprehensive fare payment program. In many procurement documents, agencies require the lead vendor to act as a system integrator (SI). In some cases, the SI is required to use equipment from a third-party supplier; DART and WMATA’s New Equipment Payment Program required a system integrator to propose third-party vendor equip- ment. A single SI consolidates the authority and accountability.

39 As mentioned earlier, many rollouts included a pilot implementation, which was approached in various ways. Some were set up to test equipment, while some were set up to test the feasibility of a new technology or to gauge customer acceptance. Some pilots were executed for a limited period or for a special event, whereas other agencies piloted and then integrated the system into operations. An open payment trial in the New York region, set up as a MasterCard and Citigroup pilot in 2006, tested the feasibility and customer acceptance of using bank-issued contactless credit cards as payment directly at the turnstile or point of entry. A second phase pilot was then deployed with three agencies: New York MTA, PATH train, and NJ Transit (SCA 2011). Many agencies, such as the MBTA, which set up a mobile ticketing program in 2012, transitioned it to a hardened system following its pilot. Most respondents indicated that an agency should clearly identify the reason, length, and success criteria for its pilot; however, the limited set of formal post-implementation analysis or return on investment (ROI) studies may indicate that the pilots are not being evaluated to their full potential. Table 8 shows the results of the the contract alternatives identified by several of the respondents. Contract Staging Strategy The order of procurement demonstrates the strategy by multiagency EFPS to stage their contracting approach. Table 9 shows the results of the survey by type of procurement: • All agencies included a project management procurement. • Most agencies procured all the elements of the system together (even if they hired a system integrator) including operations, maintenance, mobile app, and customer service center. Lead Organization Procuring Services Pr e- qu al ify D es ig n B ui ld O pe ra te M ai nt ai n Sy ste m In te gr at e (ag en cy ) Sy ste m In te gr at e (ve nd or) Tu rn ke y PP P Pi lo t Co m m en t Metropolitan Transportation Authority (NY) x DBOM LA Co. Metropolitan Transportation Authority x x x x x x x DBOM Milwaukee County Transit System x DBOM Bi State Development/Metro x x x x DB Capital Metro x x mobile WMATA x x DBOM PRESTO—A division of Metrolinx x x x x x x DBOM Sound Transit x Port Authority of Allegheny County x x x x x x x DBOM Metro Transit x x x Dallas Area Rapid Transit x x x x x x DBOM Metropolitan Transit Authority of Harris Co. Texas x DBOM TriMet* x We built the mobile application with a local startup. Chicago Transit Authority x DBOM Miami–Dade Transit x x x x x x x DBOM MassDOT x x x x BOM Source: Survey data, format is native Word table. *Refers to their mobile fare procurement TABLE 8 CONTRACT ALTERNATIVE APPROACHES

40 • Several agencies first hired a program management consultant and then procured field equipment, back office, and retail network. • Several agencies first deployed the system and then procured their mobile app and other customer services later. • Only one agency, DART, procured its mobile system (inspection and ticketing) before procuring its account-based payment system. Detailed staging by each fare program data are shown in Table 10. STAGING PARTNER AGENCY DEPLOYMENT The survey asked respondents about staging agency partner’s deployments. Many indicated that agencies delayed partner agencies’ implementation until “they were ready.” The reasons given included gaining confidence in the system or finding the right strategy or less expensive equipment. Some agencies delayed their deployment until equipment was ready or thoroughly tested. Responses included: • We launched first with each partner as they were ready. • Most . . . bus operators have delayed their participation, opting to wait for the app to be live and tested . . . and its functionality demonstrated for a few months. • Of the original group, none [was] delayed. There was a staggered rollout based on the imple- mentation plan. • Two partners were cautious of the initial deployment while we worked out the reporting and communications bugs. Their hesitation resulted in additional testing that further validated the accuracy of the system and resulted in [their] having confidence in the information collected and reported, resulting in a smooth reimbursement process. Another issue that emerged through the rollout process was that some agencies were not familiar with the technology, so they opted to delay their adoption until they understood how the system worked. Procurement Staging Approach First Procurement Second Procurement Third Procurement Additional Total Percent Count Percent Count Percent Count Percent Count Project management 100 11 0 0 0 0 0 0 11 Field equipment (TVM, readers, etc.) 75 9 25 3 25 3 0 0 12 Back office/data center services 76.9 10 15.4 2 15.4 2 0 0 13 Agency mobile device/app (inspection, sales) developer 77.8 7 11.1 1 11.1 1 0 0 9 Customer mobile ticket or service apps developers 66.7 4 16.7 1 16.7 1 0 0 6 Retail network and distribution 50 3 33.3 2 16.7 1 16.7 1 6 System integrator 66.7 4 16.7 1 16.7 1 0 0 6 Operator (including financial settlement) 100 6 0 0 0 0 0 0 6 Maintenance (equipment, software, and/or servers) 77.8 7 11.1 1 22.2 2 0 0 9 Customer service center/media inventory & distribution 60 3 20 1 20 1 0 0 5 Other 0 0 0 0 0 0 100 1 1 Source: Surveyresults, format Word table. Sample size = 13. TABLE 9 CONTRACT STAGING STRATEGY

41 Reason for Approach When asked why they selected the approach they did, agencies that identified turnkey or DBOM systems responded: • Deployment of [program] was accomplished under a broader, existing contract. • It was how fare collection systems were purchased at that time in 1990. • There was one procurement with change orders as needed. • The contract was a DBOM and deemed to be the most secure for the agencies. • Convenience and saved time. • Easier to manage with only one vendor. • Ten-year contract with a one-step solution with expertise to execute the complexity of a fare system across multiple transit agencies. Those agencies that selected to procure their systems using multiple contracts gave the following reasons: • First procurement was focused on scope, schedule, and cost considerations to permit evaluation of options. Elected to go with mobile ticketing next due to ease of procurement and deploy- ment. Elected to go with account-based fare collection with established system integrator once we were satisfied this was a viable approach. • Existing rail systems and the desire for replacement equipment made us go with independent contracts. • [Agency] used a contractor . . . to help gather requirements and facilitate the RFP. [Agency] then contracted with [vendor] to provide the hardware and software to operate the system. [Agency] does all system maintenance and customer service internally. • We had multiple procurements of different fare types; e.g., farebox, TVM, mobile app, that were procured similarly but [at] different times. Program No. Pr oje ct Ma na ge me nt Fi el d Eq ui pm en t B ac k O ffi ce /D at a Ce nt er M ob ile S al es / In sp ec tio n A pp M ob ile T ic ke t R et ai l N et w or k Sy ste m In te gr at or O pe ra to r M ai nt en an ce Cu sto m er S er vi ce C en te r O th er Metropolitan Transportation Authority (NY) 1 1, 2, 3 1 1 1 1 2 LA Co. Metropolitan Transportation Authority 1 1 1 1 1 1 Milwaukee County Transit System 1 2, 3 2, 3 2, 3 2, 3 * Bi State Development/Metro 1 1 1 1 1 1 Capital Metro 1 1 1 1 1 WMATA 1 1 1 1 1 1 1 PRESTO—A division of Metrolinx 1 1 1 1 1 1 1 Sound Transit 1 2 2 3 3 2 2 Port Authority of Allegheny County 1 3 3 2 2 3 3 3 Metro Transit 1 1 1 Dallas Area Rapid Transit 1 1 1 1 1 1 Metropolitan Transit Authority of Harris Co. Texas 1 1 1 1 1 1 1 1 1 1 TriMet* 1 1 1 1 1 *Comment: “We had back end and rail system, as well as open API for integration provided by [vendor 1], then Farebox hardware, bus back end software and integration to [vendor 2] services provided by [vendor 3]. Additionally, we contracted with [vendor 4] to integrate with the Farebox on board the bus. We also have an independent contract with [vendor 4] to get an IVR in place as well.” Source: Surveydata, format native word table. Sample size = 13. TABLE 10 FARE PROGRAM STAGED CONTRACTS CORRELATION

42 Challenges of Approach Survey respondents were asked to describe the challenges of their approach. Irrespective of whether they selected a turnkey or multiple contractors, most challenges were linked to vendor lock-in. On the other hand, agencies that went with a multiple contract approach faced integration and vendor coordination challenges. These can be seen by the seemingly conflicting comments made by survey respondents, such as the one who stated: “One vendor to some extent holds you hostage. Conversely, multiple vendors create finger pointing.” Challenges to procurement processes included: • No one to push the vendor to meet the deadlines imposed. • Non-competitive procurement. • Not having done it before made it difficult to know what you do not know. • Available and accessible TVMs, retail and mobile apps. • We have disparate system and that creates a problem with ongoing support. • The vendor provides the software, equipment, and media. Any changes to the system could only be made [by] the vendor, a non-competitive environment. • The contract was heavily weighted in favor of the agencies. After system was deployed, costs went up. • The ability to over time reduce the responsibility of the contractor was not built into the original contract. • Biggest challenge is managing multi stage procurement in a technology environment that is rapidly changing and with new vendors emerging. • Integrating various vendors with different agendas and timelines fell [to] the agency and has been a monumental challenge. Also, no liquidated damages in the contract means that accountability reduced and delays increased. Lessons Learned Although the majority of the multiagency EFPS deployment teams did not perform a post-implementation analysis or ROI evaluation (only 40% performed a study, whereas 60% did not), the respondents had recommendations for others who plan to procure a regional payment system. Respondents Lessons Learned 1 Challenges/issues are not all related to technology, management and people issues remain key: Project management capabilities: intra- and interagency coordination, spell out all details of operating/governance agreements, robust pilots and test phases, customer input into design, external alliances (law enforcement, financial institutions, payment processors, transit benefit providers, other transit properties, and retailers with similar operating needs). Training and new skill requirements for in-house teams: Employee buy-in, training tools for increased effectiveness, know rules/protocols of the business you're getting into, understand service providers' business perspective. Customer focus: Benefits to customers, enhanced customer experience, commercial and customer marketing. We would have benefited from having deeper understanding of banking, electronic payments, credit/debit operations earlier in our deployment. We needed to gain this understanding very quickly after implementation.

43 Respondents Lessons Learned 2 Try to get everyone to agree to senior age. Use many vendors [to have multiply suppliers that can provide] non-proprietary [equipment]. Have [the] contractor move in-house for better oversight, management, and “skin in the game.” Bring engineers in earlier to take over most if not all of the work and software development, change notices are very expensive. 3 Hire an outside project manager. 4 If using the same approach of the agency being the integration facilitator, we should have co-located the parties for several months until design, development, and testing was complete. 5 Pilot any system first to avoid a long implementation. Other customers may not have your unique challenges so like the system, but it may not work for you. Get Title 6 out of the way before award. 6 Design of the system (use of modular architecture) Getting agreement on a unified fare structure prior to development and deployment (i.e., make the system rules less complex) Single source for hardware [for example, vendor-] owned hardware interface [means that] any changes were high cost and time. 7 One vendor to some extent holds you hostage. Conversely, multiple vendors create finger-pointing. 8 The biggest opportunity for [agency] on this project was to have a dedicated and qualified project manager on the agency side for this implementation. [Agency] used one of their own engineers but he was a civil engineer and did not have the experience and knowledge necessary from a software and technology standpoint to be as effective as could have been within the agency in identifying the challenges and helping the agency plan ahead for their post-implementation needs. We have interim results linked to the mobile deployment. We are currently in the midst of the account-based deployment with revenue service scheduled for 2017, after which a post deployment assessment will be completed. 9 Would have used a different consulting firm 10 Engage employees first! 11 We underestimated the cost and time required to deploy our system. We launched the system prematurely, before base functionality was fully delivered. We did not give enough attention to requirements management throughout the system development life cycle.