Click for next page ( 31

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement

Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 30
30 CHAPTER 6 Evaluation and Next Steps Evaluation Phase Goal Develop improved materials and provide a training work- shop to facilitate transit learning and involvement. The primary goal of the evaluation phase was to obtain Work with WMATA to review, assess, and enhance their EAP transit agency involvement in reviewing, using, and helping documents for transit review, generalization, and testing. to improve the Transit Enterprise Architecture and Planning Using transit-specific examples in the wiki and workshops (TEAP) Framework content on the TEAP wiki. would make it easier for the examples to be understood, Specific objectives of this phase were to: rather than using general EAP industry examples. Pilot products and lessons learned could also be obtained from Further improve the TEAP wiki's usefulness based on tran- this step of working with WMATA. sit industry feedback. Conduct a series of EAP workshops to obtain transit input Create a generalization or reference enterprise architec- on existing materials and potential new ones. This would ture (EA), with tools and templates, and publish it on the allow the participation of a broader range of transit agen- wiki. cies in the review, testing, and development of transit EAP Develop an approach for transit agencies (large and small) materials, reference architecture, and tools. to apply the reference EA as a building block to expedite Develop a Reference Enterprise Architecture for Transit their particular EA process. with one or more segments of a full EA. The areas were Identify other potential improvements to the wiki that can selected by the transit participants in the workshops. be incorporated in the future. Work with a smaller transit agency to do a more in-depth pilot (originally to be Cape Cod Regional Transit Author- Pilot Approach ity [CCRTA], then recruitment efforts occurred with other agencies as well). A multi-faceted approach was used to obtain transit input Conduct interviews with a set of transit agencies that to further refine and add resources to the wiki site, and in par- reviewed and worked with the wiki and EAP materials to ticular, create a Reference Enterprise Architecture for Tran- obtain additional feedback and ideas. (Core questions can sit based on transit agency input. Although two agencies were be found in Appendix A). specified at the panel meeting to participate in the pilot activ- ities, the team chose a multi-faceted approach because of In order for the small-agency pilot to be a success, it was early concerns about the availability of transit agencies with agreed at the TCRP panel meeting that the agency selected the capability to take on the additional workload of imple- would need to fulfill certain roles and responsibilities. menting an enterprise architecture planning (EAP) pilot. It These responsibilities included: was likely that participation in a targeted EAP development effort for the pilot, without having planned for it in the pre- Working with the TEAP project team to define a clear and vious budget cycle, would be difficult. Steps were taken to concise set of objectives, scope, and schedule for the pilot. obtain broader input into the evaluation effort. Assigning resources sufficient to meet the TCRP 09-13 The implemented approach consisted of the following project schedule and making their staff available to the key steps: TEAP project team.

OCR for page 30
31 Allowing the pilot results to be published by the Trans- reference EA. The project team contacted a range of agencies portation Research Board. that had expressed interest in the pilot during Phase I. They then worked with a subset of agencies, including Cape Cod Regional Transit Authority (CCRTA), Dallas Area Rapid Peer Review Webinars/Workshops Transit (DART), Westchester County, and Chicago Transit Four webinars were conducted to obtain transit industry Authority (CTA) to begin the pilot testing effort of a segment input in the development of a preliminary Reference Enter- of the reference EA. None of the agencies were able to com- prise Architecture for Transit for the TEAP wiki. The webi- plete the pilot activities that they desired to do because of time nars included a training session and three workshops. and internal resources issues. A range of planning efforts and The first workshop highlighted a presentation by the Chief custom product developments went into either aborted starts Architect from WMATA, Jamey Harvey, on the WMATA EA. or ongoing efforts. In the end, the research team met feed- Harvey described the EA organization (metamodel), content, back and product development goals through trading-off the and general principles he used at WMATA. The focus of the depth of the agency pilot test for the broader involvement of peer review discussion was on the following three areas: more agencies. The pilot agencies that spent time and effort with the proj- WMATA's taxonomy for a Transit Reference Enterprise ect team beyond the workshops to review, sometimes test, and Architecture Business Process Model. to comment on the project's wiki, EAP materials, and models WMATA's taxonomy for a Transit Reference Enterprise are summarized below. Additional information about their Architecture Data and Application Model. involvement in the pilot effort and their contributions to the A governance structure and organization as a set of roles project follows. and responsibilities that apply to a generic set of transit provider stakeholders. WMATA: Feedback on wiki elements, review and modifi- cation of their implemented EAP materials to generate The second workshop was focused on how to make the TEAP products, training and discussion with other transit architecture more generic and what segment to select for staff, collaboration with the project team on material review and refinement. The result of this second workshop development, assistance with workshops was the selection of the fare management area for review. CCRTA: Review of wiki materials, provision and discus- Prior to the final workshop, the research team interviewed sion of ARRA project documents for project, preliminary different agencies that were developing existing and new solu- planning efforts tions for fare management. The models included the typical, DART: Pilot planning, preliminary data collection for field closed systems that most agencies currently implement, open location, technology and applications payment system, and the emerging mobile and regional branded CTA: Review of wiki content, preliminary application of (smartcard) payment system. Reference TEAP and mapping of Fare Management System, In the final workshop, the discussion focused on how to rep- participation in the pilot interviews, provision of materials resent different fare management configurations, and how to King County Metro: Review of wiki content, active work- generically represent applications and technology components. shop contributions, pilot interview, provision of materials The results of these discussions and the workshop recommen- UTA: Provision of fare documents and an overview for dations were posted on a private wiki site to which all partici- the team in a teleconference, working on assessment of pants had writer-level access. Several transit agencies reviewed the "Open Payment Fare Management System" against the the resulting artifacts; some agencies applied their existing sys- solution model, wiki content assessment and discussion, tems to the model to validate it. The results of these pilots are workshop and pilot interview participation described in the next section entitled "Pilot Agencies." Avolution EAP Software: Validation of TEAP reference The final reference architecture, the four fare management architecture using EA modeling software to ensure valid solution architectures, updated streamlined TEAP, templates, and accurate relationships guidance, and approach for incorporating solution architec- tures were included in the Phase I wiki site. Washington Metropolitan Area Transit Authority (WMATA) Pilot Agencies For the evaluation phase, WMATA agreed to have its EA Originally, the Phase II Project Plan specified the recruit- undergo peer review and to help create a new Reference Enter- ment of at least one agency that would apply a segment of the prise Architecture for Transit, which could be piloted by a

OCR for page 30
32 transit agency. Mr. Jamey Harvey, Chief Architect at WMATA, Dallas Area Rapid Transit (DART) prepared and led a workshop laying out the details and moti- DART was very interested in piloting aspects of the EA vation behind WMATA's EA model. WMATA agreed to par- model and spent a significant amount of time with the ticipate and put forward their architecture for several reasons, research team defining what would occur in the pilot. The first and foremost because they wanted their efforts to be team worked with DART's Director of Technology Program shared by the industry. A secondary reason is because they Management to begin drafting a Memorandum of Under- saw the value in transit experts reviewing and commenting on standing (MOU) for the pilot. Several products were devel- their approach. Moreover, they saw the benefit in creating a oped for DART staff to use to collect information and insert community of enterprise architecture experts in the transit it into a database management system with the connections industry to move the industry forward. As new technologies, among architecture layer components already defined. These applications, and business needs change and evolve, new included the following: solutions need to be developed, and consequently, models that represent these business needs will be developed and Spreadsheet/database (and model) of WMATA's business incorporated into the next generation "to-be" architecture processes, data components, applications models. Finally, he hoped that other agencies would work on Preliminary traceability to TCIP application categories/ different segments of the architecture and share their devel- types opment efforts as well. It is too much work to create a transit Draft metamodel and template descriptions that applied to EAP for just one agency. DART Changes to the metamodel and templates used that would Cape Cod Regional Transit be used to collect and insert the data collected into the Authority (CCRTA) database management system and EAP software Guidance on how to apply the TEAP Initially, CCRTA staff volunteered to work with the TEAP project team to pilot the EAP related models at CCRTA. At The research team began developing templates specifically first, we discussed that the TEAP Framework project could targeted for DART's scope. Several database models, tem- help CCRTA begin to develop their own EA using the tem- plates, and forms were developed for collecting information; plates that are included in the EA/EAP Guidebook sections the scope was focused such that the effort could be accom- of the wiki, and the new ones developed as part of the Ref- plished in the 6 weeks allocated for the pilot. erence TEAP. Ultimately, DART management had a higher priority for The plan was for the research team to follow-up with CCRTA the key staff person who was working with the pilot over the to evaluate the successes and challenges encountered by the project time period. The work had to be deferred or dropped. staff in using the reference enterprise architecture model and Lessons learned from DART: templates offered by the TEAP wiki. In the Phase II Work Plan, the original intent was to It is difficult to start up an EAP project even with tools accomplish the following: There is a need for dedicated resources (especially time) to initiate the project . . . introduce the CCRTA staff to EA for transit and EAP It is important to have a high-level managerial champion methods. They will review the elements of the prototype Transit for the project (not just acquiescence as a "nice to have") Reference Architecture with CCRTA and provide guidance on It is critical to have a strong commitment to build EAP how they might customize the business architecture. In addition, the Project Team will train CCRTA staff how to populate data- over the long run bases for four EA models using a set of "templates" (technology, application, data and business process) that document the refer- ence architecture. (These activities may be changed in the Pilot Chicago Transit Authority (CTA) Scope of Activities plan.) CTA recognized the need for a comprehensive, enterprise- wide understanding of their technologies and where they're The research team reviewed the summaries of CCRTA's going. As a result they hired Douglas Dalrymple, who has projects and began the process of developing the scope of the EAP expertise, as a Senior IT Solutions Project Manager in pilot activities with CCRTA staff. Unfortunately, CCRTA CTA's Project Management Office (PMO). Doug was excited decided that their project deadlines were too tight to add EAP to be involved in the TEAP pilot effort. Although he did not activities related to the projects and withdrew from the pilot participate in the workshops he was briefed by J. Harvey effort. (WMATA) and tested Reference EAP materials and partici-

OCR for page 30
33 pated in the pilot interviews. We are still working with him by agency gain speed and agility. King County doesn't have an providing guidance on the templates and tools developed for the EA, although it strives for an enterprise-wide perspective. wiki. Key results of the pilot interview with Doug Dalrymple They are looking at options and prefer an approach that is are included below. conformant to a larger methodology. One of the primary reasons for wanting to develop an EA In reviewing the wiki, he felt that it provided a lot of knowl- at CTA and participate in the pilot was to develop a good, edge and references, but EAP is a complicated topic and, in comprehensive, updated technology strategic plan for CTA. general, transit does not know too much about it. Most agen- The business and information layers of the EA are particularly cies will not be able to afford an Enterprise Architect. The wiki important in developing a good strategic plan. Participation needs to assist transit in understanding the EAP value propo- in the pilot and use of the wiki were also valuable for gaining sition and how to get help. In agencies where a lot of people more transit contacts and developing a network of transit are retiring, the risks from inadequate documentation can be staff that can share ideas, lessons learned, and work products. high. EAP helps with the documentation and helps minimize Mr. Dalrymple has already begun the process of incorporat- risk. Further, the documentation that arises from an EAP ing the WMATA governance model from the wiki and modi- process can help new staff, vendors and consultants to better fying it for CTA. The "swim lanes" will be unique to CTA. understand the business. Further, the TEAP wiki materials have assisted him in starting Some of the features of the wiki that he reviewed and found to build CTA's business layer of their EA. His preliminary useful were as follows: results have already triggered valuable discussions and better understanding of complicated processes. He has indicated that The enumeration of the transit business areas and common the wiki was well structured for this. processes. Other reasons cited for pursuing the development of an EA Discussion that related EAP, COBIT, and systems engineer- are the benefits of having business needs drive the technology ing, although more could be said about them. investments. Further, an EA gives an enterprise view of what Seeing ABACUS outputs that were in the transit domain. CTA has. The organization needs and wants to understand (For more information, see the section of this report titled the costs of their business processes and systems to under- "Testing within EA Modeling Software--ABACUS Enter- stand the true cost of ownership. A good EA provides critical prise Architecture Software"). documentation of systems and their relationships, so that inevitable retirements of senior staff don't result in a harmful He also provided some additional comments on the wiki site loss of key institutional knowledge. Also, if an agency has old and its topics, such as: systems, it needs clear documentation so issues and failures can be fixed quickly. Additional details of this effort may be Make it clear why a transit EA is needed. It can be a hard sell found in Appendix C: Validation Report. because it's complicated. An EA improves the ability to see interconnections. Require- ments, connections and opportunities will be missed if a King County Metro (KCM) and the new project is done as a silo. Staff can miss issues when a sys- Department of Transportation tem is complicated, and the EA helps them see the implica- Business Solutions Group Supervisor, Stephen Bell, agreed tions, connections, and redundancies. to review and discuss EA materials within the context of his Larger agencies need a software tool to help maintain the organization, the King County Department of Transporta- EA components and relationships. tion, which includes KCM. He was an active participant in the pilot workshops, reviewed materials, and participated in the Utah Transit Authority (UTA) pilot interviews. In addition, he provided some very clear and concise documentation that provides an overview of EAP, UTA helped with the development of the TEAP fare man- which should be integrated into the wiki. agement solution model and tools. Toward that goal, UTA He and his organization are looking at EAP to improve the presented materials and an overview of their fare architecture strategic alignment between business goals and technology. and implementation, plus they were active workshop con- Also, the agency now wants the ability to link the cost of an tributors and they participated in the pilot interviews. After investment with its performance effectiveness. He felt that being walked through UTA's fare implementation, the TEAP given the increasingly complex technology and business envi- team developed an "open payment" fare management solu- ronment at the organization, a tool like EAP can help man- tion model including business, information, application, and age the increasing complexity. The EA would serve as a map technology layers and their interconnections. The model was to help get a handle on the complexity and it would help an returned to UTA for review. In addition, other transit agencies

OCR for page 30
34 were asked to review the model to ensure its correctness and Links to RITA resources and list, if they are not already validity for more than a single agency. included. Following the Reference TEAP (and solution model) Can have online chats. development, Abraham Kololli, UTA's Information Systems Some people don't know what a wiki is and how it can Manager, participated in the pilot interviews to obtain feed- be used. The open source crowd knows what it is. With back from their review of the process, the reference model, diverse users, you need to determine how to allow more and wiki guidance materials. The interview results are high- or less access and ability to change the wiki content. lighted below. Having some generic architecture examples on the wiki for things like Wireless Architectures may cut down on For UTA, the goal is the outcome of EAP, which is good repeat questions to specific transit agencies. accessible documentation, not the EAP process itself. The wiki and the project workshops provided new source mate- rials and ideas for improving what is done at UTA. Testing within EA Modeling Software-- The TEAP wiki can help transit agencies with a number of ABACUS Enterprise Architecture Software issues, such as understanding the importance of knowing In addition to the transit agencies that helped pilot test and their business needs before proceeding with technology comment on the TEAP wiki and EAP materials, one vendor investments. heard of the project and volunteered time and software tools Templates, like some on the wiki, can help you think of or to help with a different type of pilot testing. The vendor remember critical things as you are planning, implement- worked in conjunction with WMATA staff and the research ing, or maintaining systems. team to see if the TEAP Transit Reference EA model could EA is all about documentation. If "Joe" falls off the face of successfully be input into general EAP industry modeling the earth, what are you able to do with him gone and how software. quickly can you recover from his absence? It depends on The EA tool, ABACUS, is designed for modeling, under- how good your documentation is. standing, and analyzing complex enterprises across people, We have lots of documentation, such as guidance to staff, source safe, source code, and flow charts. A new employee processes, and technologies. At the vendor's website, ABACUS could spend weeks reading and learning before touching is described as: any code. We have documentation standards and our doc- . . . a flexible modeling tool that predicts the benefits, effec- umentation is heavily linked to our IT Disaster Recovery tiveness and cost of alternative strategies. This is achieved Plan, which fits with the overall organization's Disaster through: Recovery Plan. One of the benefits to transit from the EAP models and Analyzing an enterprise using metrics such as total cost of templates available on the wiki could be much better Dis- ownership, performance and reliability, and performing sophis- ticated trade-off analysis for guided decision making; aster Recovery Plans. The tools and information can help Uniting various levels of a complex enterprise into an inte- an agency create better documentation in their Disaster grated, hierarchical, single point of truth; and Recovery Plan, which is crucial. The communication of an enterprise model and analysis using The wiki and EAP can help prompt the development of graphs, two dimensional pictures and advanced three dimen- a comprehensive list of "what's out there" and "what talks sional visualisations. to what." Most agencies don't have the resources that are needed to A critical element of any model is to ensure its logical con- be devoted to an EAP process, so ways to share efforts and sistency, completeness, and validity. Much of these attributes information, such as this wiki, are helpful. are testable through functions using a database management Some of the additions or improvements to the wiki that system or special-purpose tools. WMATA's EAP is stored in were mentioned are: a special-purpose tool (ABACUS) that models and stores Wish it were national. Wish there were contacts: agency architectures. A WMATA staff person and EAP vendor who names, staff names, phone numbers for people that are supports WMATA's EAP model ( doing these things or are considering it. A possible au/products.html) volunteered to implement the Reference downside is that people may talk off-line and forget to TEAP into ABACUS. The implementation achieved several add to the wiki. objectives: Have folders for each of the ITS areas so agencies can fig- ure out who's implemented the type of system or is in Provided additional resources for transit agencies to imple- development. Could have a place to volunteer one's name. ment the reference EA.