Transit Enterprise Architecture and Planning Framework (2011)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

30 Evaluation Phase Goal The primary goal of the evaluation phase was to obtain transit agency involvement in reviewing, using, and helping to improve the Transit Enterprise Architecture and Planning (TEAP) Framework content on the TEAP wiki. Specific objectives of this phase were to: • Further improve the TEAP wiki’s usefulness based on tran- sit industry feedback. • Create a generalization or reference enterprise architec- ture (EA), with tools and templates, and publish it on the wiki. • Develop an approach for transit agencies (large and small) to apply the reference EA as a building block to expedite their particular EA process. • Identify other potential improvements to the wiki that can be incorporated in the future. Pilot Approach A multi-faceted approach was used to obtain transit input to further refine and add resources to the wiki site, and in par- ticular, create a Reference Enterprise Architecture for Tran- sit based on transit agency input. Although two agencies were specified at the panel meeting to participate in the pilot activ- ities, the team chose a multi-faceted approach because of early concerns about the availability of transit agencies with the capability to take on the additional workload of imple- menting an enterprise architecture planning (EAP) pilot. It was likely that participation in a targeted EAP development effort for the pilot, without having planned for it in the pre- vious budget cycle, would be difficult. Steps were taken to obtain broader input into the evaluation effort. The implemented approach consisted of the following key steps: • Develop improved materials and provide a training work- shop to facilitate transit learning and involvement. • Work with WMATA to review, assess, and enhance their EAP documents for transit review, generalization, and testing. Using transit-specific examples in the wiki and workshops would make it easier for the examples to be understood, rather than using general EAP industry examples. Pilot products and lessons learned could also be obtained from this step of working with WMATA. • Conduct a series of EAP workshops to obtain transit input on existing materials and potential new ones. This would allow the participation of a broader range of transit agen- cies in the review, testing, and development of transit EAP materials, reference architecture, and tools. • Develop a Reference Enterprise Architecture for Transit with one or more segments of a full EA. The areas were selected by the transit participants in the workshops. • Work with a smaller transit agency to do a more in-depth pilot (originally to be Cape Cod Regional Transit Author- ity [CCRTA], then recruitment efforts occurred with other agencies as well). • Conduct interviews with a set of transit agencies that reviewed and worked with the wiki and EAP materials to obtain additional feedback and ideas. (Core questions can be found in Appendix A). In order for the small-agency pilot to be a success, it was agreed at the TCRP panel meeting that the agency selected would need to fulfill certain roles and responsibilities. These responsibilities included: • Working with the TEAP project team to define a clear and concise set of objectives, scope, and schedule for the pilot. • Assigning resources sufficient to meet the TCRP 09-13 project schedule and making their staff available to the TEAP project team. C H A P T E R 6 Evaluation and Next Steps

• Allowing the pilot results to be published by the Trans- portation Research Board. Peer Review Webinars/Workshops Four webinars were conducted to obtain transit industry input in the development of a preliminary Reference Enter- prise Architecture for Transit for the TEAP wiki. The webi- nars included a training session and three workshops. The first workshop highlighted a presentation by the Chief Architect from WMATA, Jamey Harvey, on the WMATA EA. Harvey described the EA organization (metamodel), content, and general principles he used at WMATA. The focus of the peer review discussion was on the following three areas: • WMATA’s taxonomy for a Transit Reference Enterprise Architecture Business Process Model. • WMATA’s taxonomy for a Transit Reference Enterprise Architecture Data and Application Model. • A governance structure and organization as a set of roles and responsibilities that apply to a generic set of transit provider stakeholders. The second workshop was focused on how to make the architecture more generic and what segment to select for review and refinement. The result of this second workshop was the selection of the fare management area for review. Prior to the final workshop, the research team interviewed different agencies that were developing existing and new solu- tions for fare management. The models included the typical, closed systems that most agencies currently implement, open payment system, and the emerging mobile and regional branded (smartcard) payment system. In the final workshop, the discussion focused on how to rep- resent different fare management configurations, and how to generically represent applications and technology components. The results of these discussions and the workshop recommen- dations were posted on a private wiki site to which all partici- pants had writer-level access. Several transit agencies reviewed the resulting artifacts; some agencies applied their existing sys- tems to the model to validate it. The results of these pilots are described in the next section entitled “Pilot Agencies.” The final reference architecture, the four fare management solution architectures, updated streamlined TEAP, templates, guidance, and approach for incorporating solution architec- tures were included in the Phase I wiki site. Pilot Agencies Originally, the Phase II Project Plan specified the recruit- ment of at least one agency that would apply a segment of the reference EA. The project team contacted a range of agencies that had expressed interest in the pilot during Phase I. They then worked with a subset of agencies, including Cape Cod Regional Transit Authority (CCRTA), Dallas Area Rapid Transit (DART), Westchester County, and Chicago Transit Authority (CTA) to begin the pilot testing effort of a segment of the reference EA. None of the agencies were able to com- plete the pilot activities that they desired to do because of time and internal resources issues. A range of planning efforts and custom product developments went into either aborted starts or ongoing efforts. In the end, the research team met feed- back and product development goals through trading-off the depth of the agency pilot test for the broader involvement of more agencies. The pilot agencies that spent time and effort with the proj- ect team beyond the workshops to review, sometimes test, and to comment on the project’s wiki, EAP materials, and models are summarized below. Additional information about their involvement in the pilot effort and their contributions to the project follows. • WMATA: Feedback on wiki elements, review and modifi- cation of their implemented EAP materials to generate TEAP products, training and discussion with other transit staff, collaboration with the project team on material development, assistance with workshops • CCRTA: Review of wiki materials, provision and discus- sion of ARRA project documents for project, preliminary planning efforts • DART: Pilot planning, preliminary data collection for field location, technology and applications • CTA: Review of wiki content, preliminary application of Reference TEAP and mapping of Fare Management System, participation in the pilot interviews, provision of materials • King County Metro: Review of wiki content, active work- shop contributions, pilot interview, provision of materials • UTA: Provision of fare documents and an overview for the team in a teleconference, working on assessment of the “Open Payment Fare Management System” against the solution model, wiki content assessment and discussion, workshop and pilot interview participation • Avolution EAP Software: Validation of TEAP reference architecture using EA modeling software to ensure valid and accurate relationships Washington Metropolitan Area Transit Authority (WMATA) For the evaluation phase, WMATA agreed to have its EA undergo peer review and to help create a new Reference Enter- prise Architecture for Transit, which could be piloted by a 31

transit agency. Mr. Jamey Harvey, Chief Architect at WMATA, prepared and led a workshop laying out the details and moti- vation behind WMATA’s EA model. WMATA agreed to par- ticipate and put forward their architecture for several reasons, first and foremost because they wanted their efforts to be shared by the industry. A secondary reason is because they saw the value in transit experts reviewing and commenting on their approach. Moreover, they saw the benefit in creating a community of enterprise architecture experts in the transit industry to move the industry forward. As new technologies, applications, and business needs change and evolve, new solutions need to be developed, and consequently, models that represent these business needs will be developed and incorporated into the next generation “to-be” architecture models. Finally, he hoped that other agencies would work on different segments of the architecture and share their devel- opment efforts as well. It is too much work to create a transit EAP for just one agency. Cape Cod Regional Transit Authority (CCRTA) Initially, CCRTA staff volunteered to work with the TEAP project team to pilot the EAP related models at CCRTA. At first, we discussed that the TEAP Framework project could help CCRTA begin to develop their own EA using the tem- plates that are included in the EA/EAP Guidebook sections of the wiki, and the new ones developed as part of the Ref- erence TEAP. The plan was for the research team to follow-up with CCRTA to evaluate the successes and challenges encountered by the staff in using the reference enterprise architecture model and templates offered by the TEAP wiki. In the Phase II Work Plan, the original intent was to accomplish the following: . . . introduce the CCRTA staff to EA for transit and EAP methods. They will review the elements of the prototype Transit Reference Architecture with CCRTA and provide guidance on how they might customize the business architecture. In addition, the Project Team will train CCRTA staff how to populate data- bases for four EA models using a set of “templates” (technology, application, data and business process) that document the refer- ence architecture. (These activities may be changed in the Pilot Scope of Activities plan.) The research team reviewed the summaries of CCRTA’s projects and began the process of developing the scope of the pilot activities with CCRTA staff. Unfortunately, CCRTA decided that their project deadlines were too tight to add EAP activities related to the projects and withdrew from the pilot effort. Dallas Area Rapid Transit (DART) DART was very interested in piloting aspects of the EA model and spent a significant amount of time with the research team defining what would occur in the pilot. The team worked with DART’s Director of Technology Program Management to begin drafting a Memorandum of Under- standing (MOU) for the pilot. Several products were devel- oped for DART staff to use to collect information and insert it into a database management system with the connections among architecture layer components already defined. These included the following: • Spreadsheet/database (and model) of WMATA’s business processes, data components, applications • Preliminary traceability to TCIP application categories/ types • Draft metamodel and template descriptions that applied to DART • Changes to the metamodel and templates used that would be used to collect and insert the data collected into the database management system and EAP software • Guidance on how to apply the TEAP The research team began developing templates specifically targeted for DART’s scope. Several database models, tem- plates, and forms were developed for collecting information; the scope was focused such that the effort could be accom- plished in the 6 weeks allocated for the pilot. Ultimately, DART management had a higher priority for the key staff person who was working with the pilot over the project time period. The work had to be deferred or dropped. Lessons learned from DART: • It is difficult to start up an EAP project even with tools • There is a need for dedicated resources (especially time) to initiate the project • It is important to have a high-level managerial champion for the project (not just acquiescence as a “nice to have”) • It is critical to have a strong commitment to build EAP over the long run Chicago Transit Authority (CTA) CTA recognized the need for a comprehensive, enterprise- wide understanding of their technologies and where they’re going. As a result they hired Douglas Dalrymple, who has EAP expertise, as a Senior IT Solutions Project Manager in CTA’s Project Management Office (PMO). Doug was excited to be involved in the TEAP pilot effort. Although he did not participate in the workshops he was briefed by J. Harvey (WMATA) and tested Reference EAP materials and partici- 32

pated in the pilot interviews. We are still working with him by providing guidance on the templates and tools developed for the wiki. Key results of the pilot interview with Doug Dalrymple are included below. One of the primary reasons for wanting to develop an EA at CTA and participate in the pilot was to develop a good, comprehensive, updated technology strategic plan for CTA. The business and information layers of the EA are particularly important in developing a good strategic plan. Participation in the pilot and use of the wiki were also valuable for gaining more transit contacts and developing a network of transit staff that can share ideas, lessons learned, and work products. Mr. Dalrymple has already begun the process of incorporat- ing the WMATA governance model from the wiki and modi- fying it for CTA. The “swim lanes” will be unique to CTA. Further, the TEAP wiki materials have assisted him in starting to build CTA’s business layer of their EA. His preliminary results have already triggered valuable discussions and better understanding of complicated processes. He has indicated that the wiki was well structured for this. Other reasons cited for pursuing the development of an EA are the benefits of having business needs drive the technology investments. Further, an EA gives an enterprise view of what CTA has. The organization needs and wants to understand the costs of their business processes and systems to under- stand the true cost of ownership. A good EA provides critical documentation of systems and their relationships, so that inevitable retirements of senior staff don’t result in a harmful loss of key institutional knowledge. Also, if an agency has old systems, it needs clear documentation so issues and failures can be fixed quickly. Additional details of this effort may be found in Appendix C: Validation Report. King County Metro (KCM) and the Department of Transportation Business Solutions Group Supervisor, Stephen Bell, agreed to review and discuss EA materials within the context of his organization, the King County Department of Transporta- tion, which includes KCM. He was an active participant in the pilot workshops, reviewed materials, and participated in the pilot interviews. In addition, he provided some very clear and concise documentation that provides an overview of EAP, which should be integrated into the wiki. He and his organization are looking at EAP to improve the strategic alignment between business goals and technology. Also, the agency now wants the ability to link the cost of an investment with its performance effectiveness. He felt that given the increasingly complex technology and business envi- ronment at the organization, a tool like EAP can help man- age the increasing complexity. The EA would serve as a map to help get a handle on the complexity and it would help an agency gain speed and agility. King County doesn’t have an EA, although it strives for an enterprise-wide perspective. They are looking at options and prefer an approach that is conformant to a larger methodology. In reviewing the wiki, he felt that it provided a lot of knowl- edge and references, but EAP is a complicated topic and, in general, transit does not know too much about it. Most agen- cies will not be able to afford an Enterprise Architect. The wiki needs to assist transit in understanding the EAP value propo- sition and how to get help. In agencies where a lot of people are retiring, the risks from inadequate documentation can be high. EAP helps with the documentation and helps minimize risk. Further, the documentation that arises from an EAP process can help new staff, vendors and consultants to better understand the business. Some of the features of the wiki that he reviewed and found useful were as follows: • The enumeration of the transit business areas and common processes. • Discussion that related EAP, COBIT, and systems engineer- ing, although more could be said about them. • Seeing ABACUS® outputs that were in the transit domain. (For more information, see the section of this report titled “Testing within EA Modeling Software—ABACUS Enter- prise Architecture Software”). He also provided some additional comments on the wiki site and its topics, such as: • Make it clear why a transit EA is needed. It can be a hard sell because it’s complicated. • An EA improves the ability to see interconnections. Require- ments, connections and opportunities will be missed if a new project is done as a silo. Staff can miss issues when a sys- tem is complicated, and the EA helps them see the implica- tions, connections, and redundancies. • Larger agencies need a software tool to help maintain the EA components and relationships. Utah Transit Authority (UTA) UTA helped with the development of the TEAP fare man- agement solution model and tools. Toward that goal, UTA presented materials and an overview of their fare architecture and implementation, plus they were active workshop con- tributors and they participated in the pilot interviews. After being walked through UTA’s fare implementation, the TEAP team developed an “open payment” fare management solu- tion model including business, information, application, and technology layers and their interconnections. The model was returned to UTA for review. In addition, other transit agencies 33

were asked to review the model to ensure its correctness and validity for more than a single agency. Following the Reference TEAP (and solution model) development, Abraham Kololli, UTA’s Information Systems Manager, participated in the pilot interviews to obtain feed- back from their review of the process, the reference model, and wiki guidance materials. The interview results are high- lighted below. • For UTA, the goal is the outcome of EAP, which is good accessible documentation, not the EAP process itself. The wiki and the project workshops provided new source mate- rials and ideas for improving what is done at UTA. • The TEAP wiki can help transit agencies with a number of issues, such as understanding the importance of knowing their business needs before proceeding with technology investments. • Templates, like some on the wiki, can help you think of or remember critical things as you are planning, implement- ing, or maintaining systems. • EA is all about documentation. If “Joe” falls off the face of the earth, what are you able to do with him gone and how quickly can you recover from his absence? It depends on how good your documentation is. • We have lots of documentation, such as guidance to staff, source safe, source code, and flow charts. A new employee could spend weeks reading and learning before touching any code. We have documentation standards and our doc- umentation is heavily linked to our IT Disaster Recovery Plan, which fits with the overall organization’s Disaster Recovery Plan. • One of the benefits to transit from the EAP models and templates available on the wiki could be much better Dis- aster Recovery Plans. The tools and information can help an agency create better documentation in their Disaster Recovery Plan, which is crucial. • The wiki and EAP can help prompt the development of a comprehensive list of “what’s out there” and “what talks to what.” • Most agencies don’t have the resources that are needed to be devoted to an EAP process, so ways to share efforts and information, such as this wiki, are helpful. • Some of the additions or improvements to the wiki that were mentioned are: – Wish it were national. Wish there were contacts: agency names, staff names, phone numbers for people that are doing these things or are considering it. A possible downside is that people may talk off-line and forget to add to the wiki. – Have folders for each of the ITS areas so agencies can fig- ure out who’s implemented the type of system or is in development. Could have a place to volunteer one’s name. – Links to RITA resources and list, if they are not already included. – Can have online chats. – Some people don’t know what a wiki is and how it can be used. The open source crowd knows what it is. With diverse users, you need to determine how to allow more or less access and ability to change the wiki content. – Having some generic architecture examples on the wiki for things like Wireless Architectures may cut down on repeat questions to specific transit agencies. Testing within EA Modeling Software— ABACUS Enterprise Architecture Software In addition to the transit agencies that helped pilot test and comment on the TEAP wiki and EAP materials, one vendor heard of the project and volunteered time and software tools to help with a different type of pilot testing. The vendor worked in conjunction with WMATA staff and the research team to see if the TEAP Transit Reference EA model could successfully be input into general EAP industry modeling software. The EA tool, ABACUS, is designed for modeling, under- standing, and analyzing complex enterprises across people, processes, and technologies. At the vendor’s website, ABACUS is described as: . . . a flexible modeling tool that predicts the benefits, effec- tiveness and cost of alternative strategies. This is achieved through: • Analyzing an enterprise using metrics such as total cost of ownership, performance and reliability, and performing sophis- ticated trade-off analysis for guided decision making; • Uniting various levels of a complex enterprise into an inte- grated, hierarchical, single point of truth; and • The communication of an enterprise model and analysis using graphs, two dimensional pictures and advanced three dimen- sional visualisations. A critical element of any model is to ensure its logical con- sistency, completeness, and validity. Much of these attributes are testable through functions using a database management system or special-purpose tools. WMATA’s EAP is stored in a special-purpose tool (ABACUS) that models and stores architectures. A WMATA staff person and EAP vendor who supports WMATA’s EAP model (http://www.avolution.com. au/products.html) volunteered to implement the Reference TEAP into ABACUS. The implementation achieved several objectives: • Provided additional resources for transit agencies to imple- ment the reference EA. 34

• Validated the logical consistency of the Reference TEAP (relationships). • Provided for demonstration of “what if” scenarios and visualizations that were not possible for the project team to present (due to limited project resources). The TEAP wiki provides several outputs from ABACUS, including its native file format and hyperlinked report of the model. In addition, the TEAP MS Excel files are consistent with the ABACUS TEAP content and serve as templates that provide round-trip editing tools for any EA tool vendor that may wish to enter the transit market. Summary and Key Findings The evaluation phase was successful in achieving the pri- mary goal of obtaining transit agency involvement in review- ing, using, and helping to improve the TEAP Framework content on the TEAP wiki. Transit staff participated in the three EA workshops, provided documents, updated and tai- lored materials for the workshops and the wiki, began the process of testing and customizing EA materials provided via the wiki for their own agencies, participated in follow-up inter- views, and contributed ideas and materials for improving the TEAP wiki content. No single agency was able to do an in-depth pilot and test of the EA materials during the TCRP TEAP project time period. Unfortunately, the current economic troubles in the United States have added even greater stresses on transit agencies, including those that wanted to pilot test the materials and tools. All the transit agencies that agreed to participate in the pilot had less time than they desired to work on it or had to withdraw. Ironically, at times like these, documentation of “as-is” and “to-be,” such as EAP provides, is most helpful and valuable in helping maintain, enhance, and develop technolo- gies efficiently. Nonetheless, the time that the transit staff was able to contribute was highly valuable. The TEAP project’s multi-faceted approach that was developed for the pilot, given early concerns about transit agency time availability, worked well in obtaining much valuable transit feedback and developing new tools and reference materials. The pilot feedback was more diverse, coming from many agencies, rather than from one in-depth study. In the evaluation phase, the research team accomplished the following: • Conducted EA/EAP training workshops. • Obtained feedback and documents from transit staff. • Created a generalization of a transit reference enterprise architecture, with tools and templates, and published it on the wiki. • Developed an approach for transit agencies (large and small) to apply the reference EA as a building block to expedite the EAP. • Enhanced many parts of the wiki, including the front page. • Revised and improved the Guidance for Transit Managers. • Received the transit reference EA model information in ABACUS EA software for review via a donation by a ven- dor of time, effort, and access to their software tool. • Gathered many suggestions and other ideas for potential improvements to the wiki that can be incorporated in the future. The transit participants in the pilot phase felt that the wiki was very valuable and helped with a wide range of transit issues. Some of the benefits they mentioned are summarized as follows: • The TEAP wiki can help transit agencies with a number of issues, such as – developing effective strategic plans and – understanding the importance of knowing their business needs before proceeding with technology investments. • The wiki templates can help you identify critical items as you are planning, implementing, or maintaining systems. • The EAP models and templates on the wiki can be used to create more effective Disaster Recovery Plans. In addition, the transit participants gave many suggestions and ideas for potential improvements to the wiki that can be incorporated in the future, such as: • Develop generic briefing materials for executive managers and board members that help them understand the value of EAP and the need to support EAP efforts. • Develop information on how to build a business case for EAP, including some examples and key indicators. • Reinforce the point that the transit business areas have to be involved. • Include national contact information (agency names, staff names, and phone numbers) for transit agencies that are doing, or are considering doing, EAP or ITS applications. Have it organized by categories. • Consider having a place for vendors and consultants to list services, tools, and other related products. • Have some generic project architecture examples on the wiki for things like Wireless Architectures and some ITS applications. • Investigate the possibility of online chats. • Ensure that the wiki contains information on the benefits of forming an IT Steering Committee, including benefits to the organization as well as to the participants on the committee. 35

• Continue to add examples of how other transit agencies do things, such as how they use a good business plan to drive the technology plan. • Advertise the wiki and steer more agencies and transit staff to it. Conclusion and Final Comments The state of the practice survey and the pilot results demon- strated the transit industry’s overall difficulty in finding the needed resources and time to fully develop all the elements of the TEAP Framework within most transit agencies. Return on investment (ROI) data are still elusive with respect to EA efforts because there are so few agencies that have the resources to implement and measure the concrete benefits of implementing and maintaining an EA. All the transit staff that participated in this project wanted to make further progress in the TEAP Framework areas at their agencies and they gener- ously helped improve the relevance and quality of the prod- ucts and tools produced by this project. They recognized that transit EA information is valuable for many purposes. The creation of the solution architectures helped show the benefits of creating generic transit templates and providing examples. One of the benefits of having EA information about a transit agency’s technology and business environment is similar to a renovation/rehabilitation project having blueprints. Having an accurate building blueprint accelerates deployment and mitigates risk when building on to or fixing a problem in the facility. Finally, the transit staff felt that the wiki makes the informa- tion about the TEAP Framework more accessible and easier to update on an ongoing basis. Further, it enables the transit community to share ideas, approaches, and tools to become more successful. 36