Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 30
30
CHAPTER 6
Evaluation and Next Steps
Evaluation Phase Goal · Develop improved materials and provide a training work-
shop to facilitate transit learning and involvement.
The primary goal of the evaluation phase was to obtain · Work with WMATA to review, assess, and enhance their EAP
transit agency involvement in reviewing, using, and helping documents for transit review, generalization, and testing.
to improve the Transit Enterprise Architecture and Planning Using transit-specific examples in the wiki and workshops
(TEAP) Framework content on the TEAP wiki. would make it easier for the examples to be understood,
Specific objectives of this phase were to: rather than using general EAP industry examples. Pilot
products and lessons learned could also be obtained from
· Further improve the TEAP wiki's usefulness based on tran- this step of working with WMATA.
sit industry feedback. · Conduct a series of EAP workshops to obtain transit input
· Create a generalization or reference enterprise architec- on existing materials and potential new ones. This would
ture (EA), with tools and templates, and publish it on the allow the participation of a broader range of transit agen-
wiki. cies in the review, testing, and development of transit EAP
· Develop an approach for transit agencies (large and small) materials, reference architecture, and tools.
to apply the reference EA as a building block to expedite · Develop a Reference Enterprise Architecture for Transit
their particular EA process. with one or more segments of a full EA. The areas were
· Identify other potential improvements to the wiki that can selected by the transit participants in the workshops.
be incorporated in the future. · Work with a smaller transit agency to do a more in-depth
pilot (originally to be Cape Cod Regional Transit Author-
Pilot Approach ity [CCRTA], then recruitment efforts occurred with other
agencies as well).
A multi-faceted approach was used to obtain transit input · Conduct interviews with a set of transit agencies that
to further refine and add resources to the wiki site, and in par- reviewed and worked with the wiki and EAP materials to
ticular, create a Reference Enterprise Architecture for Tran- obtain additional feedback and ideas. (Core questions can
sit based on transit agency input. Although two agencies were be found in Appendix A).
specified at the panel meeting to participate in the pilot activ-
ities, the team chose a multi-faceted approach because of In order for the small-agency pilot to be a success, it was
early concerns about the availability of transit agencies with agreed at the TCRP panel meeting that the agency selected
the capability to take on the additional workload of imple- would need to fulfill certain roles and responsibilities.
menting an enterprise architecture planning (EAP) pilot. It These responsibilities included:
was likely that participation in a targeted EAP development
effort for the pilot, without having planned for it in the pre- · Working with the TEAP project team to define a clear and
vious budget cycle, would be difficult. Steps were taken to concise set of objectives, scope, and schedule for the pilot.
obtain broader input into the evaluation effort. · Assigning resources sufficient to meet the TCRP 09-13
The implemented approach consisted of the following project schedule and making their staff available to the
key steps: TEAP project team.
OCR for page 31
31
· Allowing the pilot results to be published by the Trans- reference EA. The project team contacted a range of agencies
portation Research Board. that had expressed interest in the pilot during Phase I. They
then worked with a subset of agencies, including Cape Cod
Regional Transit Authority (CCRTA), Dallas Area Rapid
Peer Review Webinars/Workshops
Transit (DART), Westchester County, and Chicago Transit
Four webinars were conducted to obtain transit industry Authority (CTA) to begin the pilot testing effort of a segment
input in the development of a preliminary Reference Enter- of the reference EA. None of the agencies were able to com-
prise Architecture for Transit for the TEAP wiki. The webi- plete the pilot activities that they desired to do because of time
nars included a training session and three workshops. and internal resources issues. A range of planning efforts and
The first workshop highlighted a presentation by the Chief custom product developments went into either aborted starts
Architect from WMATA, Jamey Harvey, on the WMATA EA. or ongoing efforts. In the end, the research team met feed-
Harvey described the EA organization (metamodel), content, back and product development goals through trading-off the
and general principles he used at WMATA. The focus of the depth of the agency pilot test for the broader involvement of
peer review discussion was on the following three areas: more agencies.
The pilot agencies that spent time and effort with the proj-
· WMATA's taxonomy for a Transit Reference Enterprise ect team beyond the workshops to review, sometimes test, and
Architecture Business Process Model. to comment on the project's wiki, EAP materials, and models
· WMATA's taxonomy for a Transit Reference Enterprise are summarized below. Additional information about their
Architecture Data and Application Model. involvement in the pilot effort and their contributions to the
· A governance structure and organization as a set of roles project follows.
and responsibilities that apply to a generic set of transit
provider stakeholders. · WMATA: Feedback on wiki elements, review and modifi-
cation of their implemented EAP materials to generate
The second workshop was focused on how to make the TEAP products, training and discussion with other transit
architecture more generic and what segment to select for staff, collaboration with the project team on material
review and refinement. The result of this second workshop development, assistance with workshops
was the selection of the fare management area for review. · CCRTA: Review of wiki materials, provision and discus-
Prior to the final workshop, the research team interviewed sion of ARRA project documents for project, preliminary
different agencies that were developing existing and new solu- planning efforts
tions for fare management. The models included the typical, · DART: Pilot planning, preliminary data collection for field
closed systems that most agencies currently implement, open location, technology and applications
payment system, and the emerging mobile and regional branded · CTA: Review of wiki content, preliminary application of
(smartcard) payment system. Reference TEAP and mapping of Fare Management System,
In the final workshop, the discussion focused on how to rep- participation in the pilot interviews, provision of materials
resent different fare management configurations, and how to · King County Metro: Review of wiki content, active work-
generically represent applications and technology components. shop contributions, pilot interview, provision of materials
The results of these discussions and the workshop recommen- · UTA: Provision of fare documents and an overview for
dations were posted on a private wiki site to which all partici- the team in a teleconference, working on assessment of
pants had writer-level access. Several transit agencies reviewed the "Open Payment Fare Management System" against the
the resulting artifacts; some agencies applied their existing sys- solution model, wiki content assessment and discussion,
tems to the model to validate it. The results of these pilots are workshop and pilot interview participation
described in the next section entitled "Pilot Agencies." · Avolution EAP Software: Validation of TEAP reference
The final reference architecture, the four fare management architecture using EA modeling software to ensure valid
solution architectures, updated streamlined TEAP, templates, and accurate relationships
guidance, and approach for incorporating solution architec-
tures were included in the Phase I wiki site.
Washington Metropolitan Area Transit
Authority (WMATA)
Pilot Agencies
For the evaluation phase, WMATA agreed to have its EA
Originally, the Phase II Project Plan specified the recruit- undergo peer review and to help create a new Reference Enter-
ment of at least one agency that would apply a segment of the prise Architecture for Transit, which could be piloted by a
OCR for page 32
32
transit agency. Mr. Jamey Harvey, Chief Architect at WMATA, Dallas Area Rapid Transit (DART)
prepared and led a workshop laying out the details and moti-
DART was very interested in piloting aspects of the EA
vation behind WMATA's EA model. WMATA agreed to par-
model and spent a significant amount of time with the
ticipate and put forward their architecture for several reasons,
research team defining what would occur in the pilot. The
first and foremost because they wanted their efforts to be
team worked with DART's Director of Technology Program
shared by the industry. A secondary reason is because they
Management to begin drafting a Memorandum of Under-
saw the value in transit experts reviewing and commenting on
standing (MOU) for the pilot. Several products were devel-
their approach. Moreover, they saw the benefit in creating a
oped for DART staff to use to collect information and insert
community of enterprise architecture experts in the transit
it into a database management system with the connections
industry to move the industry forward. As new technologies,
among architecture layer components already defined. These
applications, and business needs change and evolve, new
included the following:
solutions need to be developed, and consequently, models
that represent these business needs will be developed and
· Spreadsheet/database (and model) of WMATA's business
incorporated into the next generation "to-be" architecture
processes, data components, applications
models. Finally, he hoped that other agencies would work on
· Preliminary traceability to TCIP application categories/
different segments of the architecture and share their devel-
types
opment efforts as well. It is too much work to create a transit
· Draft metamodel and template descriptions that applied to
EAP for just one agency.
DART
· Changes to the metamodel and templates used that would
Cape Cod Regional Transit be used to collect and insert the data collected into the
Authority (CCRTA) database management system and EAP software
· Guidance on how to apply the TEAP
Initially, CCRTA staff volunteered to work with the TEAP
project team to pilot the EAP related models at CCRTA. At The research team began developing templates specifically
first, we discussed that the TEAP Framework project could targeted for DART's scope. Several database models, tem-
help CCRTA begin to develop their own EA using the tem- plates, and forms were developed for collecting information;
plates that are included in the EA/EAP Guidebook sections the scope was focused such that the effort could be accom-
of the wiki, and the new ones developed as part of the Ref- plished in the 6 weeks allocated for the pilot.
erence TEAP. Ultimately, DART management had a higher priority for
The plan was for the research team to follow-up with CCRTA the key staff person who was working with the pilot over the
to evaluate the successes and challenges encountered by the project time period. The work had to be deferred or dropped.
staff in using the reference enterprise architecture model and Lessons learned from DART:
templates offered by the TEAP wiki.
In the Phase II Work Plan, the original intent was to · It is difficult to start up an EAP project even with tools
accomplish the following: · There is a need for dedicated resources (especially time) to
initiate the project
. . . introduce the CCRTA staff to EA for transit and EAP · It is important to have a high-level managerial champion
methods. They will review the elements of the prototype Transit for the project (not just acquiescence as a "nice to have")
Reference Architecture with CCRTA and provide guidance on
· It is critical to have a strong commitment to build EAP
how they might customize the business architecture. In addition,
the Project Team will train CCRTA staff how to populate data- over the long run
bases for four EA models using a set of "templates" (technology,
application, data and business process) that document the refer-
ence architecture. (These activities may be changed in the Pilot
Chicago Transit Authority (CTA)
Scope of Activities plan.) CTA recognized the need for a comprehensive, enterprise-
wide understanding of their technologies and where they're
The research team reviewed the summaries of CCRTA's going. As a result they hired Douglas Dalrymple, who has
projects and began the process of developing the scope of the EAP expertise, as a Senior IT Solutions Project Manager in
pilot activities with CCRTA staff. Unfortunately, CCRTA CTA's Project Management Office (PMO). Doug was excited
decided that their project deadlines were too tight to add EAP to be involved in the TEAP pilot effort. Although he did not
activities related to the projects and withdrew from the pilot participate in the workshops he was briefed by J. Harvey
effort. (WMATA) and tested Reference EAP materials and partici-
OCR for page 33
33
pated in the pilot interviews. We are still working with him by agency gain speed and agility. King County doesn't have an
providing guidance on the templates and tools developed for the EA, although it strives for an enterprise-wide perspective.
wiki. Key results of the pilot interview with Doug Dalrymple They are looking at options and prefer an approach that is
are included below. conformant to a larger methodology.
One of the primary reasons for wanting to develop an EA In reviewing the wiki, he felt that it provided a lot of knowl-
at CTA and participate in the pilot was to develop a good, edge and references, but EAP is a complicated topic and, in
comprehensive, updated technology strategic plan for CTA. general, transit does not know too much about it. Most agen-
The business and information layers of the EA are particularly cies will not be able to afford an Enterprise Architect. The wiki
important in developing a good strategic plan. Participation needs to assist transit in understanding the EAP value propo-
in the pilot and use of the wiki were also valuable for gaining sition and how to get help. In agencies where a lot of people
more transit contacts and developing a network of transit are retiring, the risks from inadequate documentation can be
staff that can share ideas, lessons learned, and work products. high. EAP helps with the documentation and helps minimize
Mr. Dalrymple has already begun the process of incorporat- risk. Further, the documentation that arises from an EAP
ing the WMATA governance model from the wiki and modi- process can help new staff, vendors and consultants to better
fying it for CTA. The "swim lanes" will be unique to CTA. understand the business.
Further, the TEAP wiki materials have assisted him in starting Some of the features of the wiki that he reviewed and found
to build CTA's business layer of their EA. His preliminary useful were as follows:
results have already triggered valuable discussions and better
understanding of complicated processes. He has indicated that · The enumeration of the transit business areas and common
the wiki was well structured for this. processes.
Other reasons cited for pursuing the development of an EA · Discussion that related EAP, COBIT, and systems engineer-
are the benefits of having business needs drive the technology ing, although more could be said about them.
investments. Further, an EA gives an enterprise view of what · Seeing ABACUS® outputs that were in the transit domain.
CTA has. The organization needs and wants to understand (For more information, see the section of this report titled
the costs of their business processes and systems to under- "Testing within EA Modeling Software--ABACUS Enter-
stand the true cost of ownership. A good EA provides critical prise Architecture Software").
documentation of systems and their relationships, so that
inevitable retirements of senior staff don't result in a harmful He also provided some additional comments on the wiki site
loss of key institutional knowledge. Also, if an agency has old and its topics, such as:
systems, it needs clear documentation so issues and failures
can be fixed quickly. Additional details of this effort may be · Make it clear why a transit EA is needed. It can be a hard sell
found in Appendix C: Validation Report. because it's complicated.
· An EA improves the ability to see interconnections. Require-
ments, connections and opportunities will be missed if a
King County Metro (KCM) and the
new project is done as a silo. Staff can miss issues when a sys-
Department of Transportation
tem is complicated, and the EA helps them see the implica-
Business Solutions Group Supervisor, Stephen Bell, agreed tions, connections, and redundancies.
to review and discuss EA materials within the context of his · Larger agencies need a software tool to help maintain the
organization, the King County Department of Transporta- EA components and relationships.
tion, which includes KCM. He was an active participant in the
pilot workshops, reviewed materials, and participated in the
Utah Transit Authority (UTA)
pilot interviews. In addition, he provided some very clear and
concise documentation that provides an overview of EAP, UTA helped with the development of the TEAP fare man-
which should be integrated into the wiki. agement solution model and tools. Toward that goal, UTA
He and his organization are looking at EAP to improve the presented materials and an overview of their fare architecture
strategic alignment between business goals and technology. and implementation, plus they were active workshop con-
Also, the agency now wants the ability to link the cost of an tributors and they participated in the pilot interviews. After
investment with its performance effectiveness. He felt that being walked through UTA's fare implementation, the TEAP
given the increasingly complex technology and business envi- team developed an "open payment" fare management solu-
ronment at the organization, a tool like EAP can help man- tion model including business, information, application, and
age the increasing complexity. The EA would serve as a map technology layers and their interconnections. The model was
to help get a handle on the complexity and it would help an returned to UTA for review. In addition, other transit agencies
OCR for page 34
34
were asked to review the model to ensure its correctness and Links to RITA resources and list, if they are not already
validity for more than a single agency. included.
Following the Reference TEAP (and solution model) Can have online chats.
development, Abraham Kololli, UTA's Information Systems Some people don't know what a wiki is and how it can
Manager, participated in the pilot interviews to obtain feed- be used. The open source crowd knows what it is. With
back from their review of the process, the reference model, diverse users, you need to determine how to allow more
and wiki guidance materials. The interview results are high- or less access and ability to change the wiki content.
lighted below. Having some generic architecture examples on the wiki
for things like Wireless Architectures may cut down on
· For UTA, the goal is the outcome of EAP, which is good repeat questions to specific transit agencies.
accessible documentation, not the EAP process itself. The
wiki and the project workshops provided new source mate-
rials and ideas for improving what is done at UTA. Testing within EA Modeling Software--
· The TEAP wiki can help transit agencies with a number of ABACUS Enterprise Architecture Software
issues, such as understanding the importance of knowing
In addition to the transit agencies that helped pilot test and
their business needs before proceeding with technology
comment on the TEAP wiki and EAP materials, one vendor
investments.
heard of the project and volunteered time and software tools
· Templates, like some on the wiki, can help you think of or
to help with a different type of pilot testing. The vendor
remember critical things as you are planning, implement-
worked in conjunction with WMATA staff and the research
ing, or maintaining systems.
team to see if the TEAP Transit Reference EA model could
· EA is all about documentation. If "Joe" falls off the face of
successfully be input into general EAP industry modeling
the earth, what are you able to do with him gone and how
software.
quickly can you recover from his absence? It depends on
The EA tool, ABACUS, is designed for modeling, under-
how good your documentation is.
·
standing, and analyzing complex enterprises across people,
We have lots of documentation, such as guidance to staff,
source safe, source code, and flow charts. A new employee processes, and technologies. At the vendor's website, ABACUS
could spend weeks reading and learning before touching is described as:
any code. We have documentation standards and our doc-
. . . a flexible modeling tool that predicts the benefits, effec-
umentation is heavily linked to our IT Disaster Recovery tiveness and cost of alternative strategies. This is achieved
Plan, which fits with the overall organization's Disaster through:
Recovery Plan.
· One of the benefits to transit from the EAP models and · Analyzing an enterprise using metrics such as total cost of
templates available on the wiki could be much better Dis- ownership, performance and reliability, and performing sophis-
ticated trade-off analysis for guided decision making;
aster Recovery Plans. The tools and information can help
· Uniting various levels of a complex enterprise into an inte-
an agency create better documentation in their Disaster grated, hierarchical, single point of truth; and
Recovery Plan, which is crucial. · The communication of an enterprise model and analysis using
· The wiki and EAP can help prompt the development of graphs, two dimensional pictures and advanced three dimen-
a comprehensive list of "what's out there" and "what talks sional visualisations.
to what."
· Most agencies don't have the resources that are needed to A critical element of any model is to ensure its logical con-
be devoted to an EAP process, so ways to share efforts and sistency, completeness, and validity. Much of these attributes
information, such as this wiki, are helpful. are testable through functions using a database management
· Some of the additions or improvements to the wiki that system or special-purpose tools. WMATA's EAP is stored in
were mentioned are: a special-purpose tool (ABACUS) that models and stores
Wish it were national. Wish there were contacts: agency architectures. A WMATA staff person and EAP vendor who
names, staff names, phone numbers for people that are supports WMATA's EAP model (http://www.avolution.com.
doing these things or are considering it. A possible au/products.html) volunteered to implement the Reference
downside is that people may talk off-line and forget to TEAP into ABACUS. The implementation achieved several
add to the wiki. objectives:
Have folders for each of the ITS areas so agencies can fig-
ure out who's implemented the type of system or is in · Provided additional resources for transit agencies to imple-
development. Could have a place to volunteer one's name. ment the reference EA.
