Transit Enterprise Architecture and Planning Framework (2011)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

49 State of the Practice Synthesis A P P E N D I X B

50 T A B L E O F C O N T E N T S 52 1 Introduction 52 1.1 Scope of State of the Practice Synthesis 52 1.2 Project Overview 52 1.3 Background 53 1.4 The Synthesis Topic Areas 53 1.4.1 Enterprise Architecture (EA) and Enterprise Architecture Planning (EAP) 53 1.4.2 ITS Implementation Funding 54 1.4.3 Business Case Methodology 54 1.4.4 Systems Engineering 54 1.4.5 Post-Implementation Evaluation 55 2 Synthesis Methodology and Industry Scan 55 3 Findings on Transit Enterprise Architecture Planning and Enterprise Architecture 56 3.1 Main Purpose of an Enterprise Architecture 58 3.2 General Approach to EAP/EA Used by Other Industries 58 3.1.1 Federal Enterprise Architecture (FEA) 60 3.1.2 The Open Group Architecture Framework (TOGAF) 60 3.1.3 Industry Implementation Approaches 61 3.1.4 Lessons Learned on EA/EAP from Other Industries 62 3.2 Transit Approaches to EAP/EA 62 3.2.1 Transit EAP/EA: Lessons Learned from the Literature 63 3.2.2 General State of Enterprise Architecture Adoption by Transit Agencies 63 3.2.3 Miami-Dade Transit 64 3.2.4 Washington Metropolitan Area Transit Authority (WMATA) 67 3.2.5 Other Transit Approaches to Enterprise Architecture Planning 72 3.3 Next Steps 72 Chapter 3 Appendix A: Enterprise Architecture Definitions from the Literature 72 Enterprise Architecture 73 Enterprise Architecture Planning 73 Framework 74 Chapter 3 Appendix B: FEA Segment Architecture Description 75 Chapter 3 Appendix C: Description of The Open Group Architecture Framework (TOGAF) 76 4 Findings on Transit IT/ITS Implementation Funding 76 4.1.1 General Findings from the Literature 76 4.1.2 General Approach to Implementation Funding 78 4.2 Transit Capital Investment Needs and Funding Approaches 78 4.2.1 Transit Funding Needs 78 4.2.2 Capital Funding Sources 80 4.2.3 Financing Mechanisms 81 4.2.4 Repayment Streams

81 4.3 Key Findings on Transit Agencies Implementing IT/ITS Funding 81 4.3.1 Salt Lake City Utah Transit Authority (UTA) 82 4.3.2 Washington Metropolitan Area Transit Authority (WMATA) 82 4.3.3 Metropolitan Atlanta Rapid Transportation Authority (MARTA) 82 4.4 Constraints on Funding Approaches: 82 4.4.1 Eligibility Requirements 83 4.4.2 Marketability 84 4.4.3 Risks and Uncertainties 85 5 Findings on Business Case Methodology in Transit Synthesis 85 5.1 What is a “Business Case”? 85 5.2 What is a “Business Case Methodology”? 85 5.3 Why is having a Business Case Methodology Valuable? 86 5.4 Differences Among Business Case Methodologies 86 5.5 Examples of Possible Topic Areas in a Business Case 87 5.6 Transit BCM Survey Results 89 5.7 Other Approaches to a BCM 91 5.8 Recommended Practices for BCM 92 5.9 Business Case and an Enterprise Architecture 92 5.10 Realizing Benefits 92 5.11 Have Buy-in and Get the Sign-offs 92 Chapter 5 Appendix A: Planning Report Template from TriMet 94 Chapter 5 Appendix B: WMATA’s Streamlined Form for the Business Plan Initiation (BPI) Review Process and Instructions for Completing it 103 Chapter 5 Appendix C: KCM’s Form for Determining Extent of Oversight Process 109 Chapter 5 Appendix D: King County Suggestions for Project Review Board Deliverables 113 6 Findings on Systems Engineering and Transit 115 6.1 Literature Review 117 6.2 Interview Findings 117 6.2.1 Use of the Systems Engineering Process by Transit Agencies 117 6.2.2 Benefits of Using the Process 118 6.3 Recommendations 118 7 Findings on Post-Implementation Analysis in Transit 118 7.1 Approach/Methodology 118 7.2 What is Post-Implementation Analysis? 119 7.3 PIR Benefits: Why is Post-Implementation Analysis Valuable? 119 7.4 Post-Implementation Review Process Overview 119 7.4.1 Planning the PIR 120 7.4.2 Preparing for the PIR 120 7.4.3 Conducting the PIR 120 7.4.4 Reporting/Follow-up 120 7.5 Transit Survey Findings 121 7.5.1 King County Metro (KCM) 122 7.5.2 Other Transit Discussion of Post-Implementation Analysis 123 7.6 Non-transit Approaches to Post-Implementation Analysis 123 7.7 Issues and Barriers Related to Post-Implementation Analysis 124 7.8 Recommended Practices for Post-Implementation Analysis 125 7.8.1 Checklist for Managers 125 7.8.2 Transit Manager’s Roles 126 8 References 51

1 Introduction 1.1 Scope of State of the Practice Synthesis This document, the state of the practice synthesis for the Transit Enterprise Architecture and Planning Framework project, consists of the deliverables for the following five project tasks. • Task 1: State of the Practice—Enterprise Architecture • Task 2: State of the Practice—IT/ITS Funding Implemen- tation • Task 3: Transit Agency Situational Analysis: State of the Practice—Business Case Methodology • Task 4: State of the Practice—Systems Engineering Imple- mentation • Task 5: State of the Practice—Post Implementation Analysis The five tasks relate to important disciplines that contribute to the successful planning, funding, development, and deploy- ment of transit Intelligent Transportation Systems (ITS) proj- ects. Chapter 2 includes an overview of the methodology used to develop the syntheses, which included a review of the liter- ature and interviews with transit agencies and the Depart- ment of Transportation (DOT) for a few States. The synthe- ses developed for the five tasks are included in Chapters 3 through 7. The purpose of the synthesis tasks was to obtain a better understanding of current industry knowledge and practice in the five topic areas, as well as to observe the state of readiness for transit to adopt industry best practices. Subsequent deliv- erables will address and recommend best practices within a framework that blends these five disciplines into a seamless, consistent Framework. 1.2 Project Overview The Transit Enterprise Architecture and Planning Frame- work project seeks to provide transit agencies with a roadmap, based on a Transit Enterprise Architecture and Planning (TEAP) Framework, to successfully implement Information Technology (IT) and ITS technologies that meet their business needs. The project includes a prelimi- nary assessment of the industry and tools available (the syn- thesis tasks) and the development of a framework and a process, supported by tools to assist agencies in implement- ing IT/ITS technologies. The Framework and tools will help transit professionals understand the financial, operational and management impacts of technologies, to help them bet- ter meet their enterprise business process needs and corpo- rate objectives. The Framework will also help guide an agency’s planning process, improve its understanding of risks, and validate and verify compliance with its needs, better manage the project implementation effort, and measure results and benefits. The project consists of two phases. During Phase I, the Research Team will complete the syntheses and develop the details of the framework for improving ITS project deploy- ments. As early Phase I deliverables, the syntheses describe current industry practice through a review of the literature and interviews with transit industry professionals, and identi- fies industry readiness for adopting best practices in the five specific disciplines associated with deploying ITS projects. In subsequent deliverables, the Transit Enterprise Architecture and Planning Framework will be described. It will include a high level overview for an executive management audience, details on how to develop an enterprise architecture that aligns technology investments with business needs, guidance on how to show the relationships among ITS business processes, per- formance, information, services and technology, and exam- ples and templates. During Phase II, key aspects of the TEAP framework will be field tested and demonstrated through the EA/EAP tool(s) implementation. 1.3 Background The five disciplines addressed in the syntheses, which will be included in a framework for successfully deploying transit ITS projects, are often poorly understood and executed in transit as well as other industries. This is due to several factors: • Lack of time and resources for training on the topics • Lack of time, resources and corporate support for imple- menting the disciplines • Lack of materials that tailor the topics for transit to make them relevant rather than complex and theoretical As competition for limited resources increases, the value and need for skills in building a good business case, arranging funding, using EAP to improve the value of the investment, managing projects with good systems engineering practices, and proving value with post-implementation analysis, will increase. Further, in transit as well as other industries, the relation- ships between the five disciplines are typically not well laid out and understood. In Task 4 of this project, the Framework and relationships will be described. An enterprise-wide framework approach to project planning better enables the identification of the impacts on people, systems and technologies over the lifecycle process, as well as the meeting of agency requirements. Specifically, a Framework guides transit in: • Planning how information, services and technology work together across an enterprise to support business processes, solve problems, and measure performance; 52

• Promoting information sharing across agency and institu- tional barriers; • Ensuring that IT/ITS projects are defined and staged in a way that ensures best value and supports the successful implementation, operations and maintenance; • Ensuring that the benefits and costs of proposed IT/ITS proj- ects are understood across the project’s lifecycle (including operations and maintenance), and resources are available to support the program; • Specifying IT/ITS projects to maximize the IT/ITS invest- ment decisions across the organization; • Ensuring that IT/ITS projects are described to meet stake- holder needs, requirements are explicitly described, risks are identified and mitigated, and the system development process is managed to ensure correct operations and requirements are met; and • Describing the leadership and organizational structures and processes that ensure that the organization’s IT sus- tains and extends corporate strategies and objectives (1). 1.4 The Synthesis Topic Areas The five synthesis topic areas provide tools for planning, developing, deploying, and evaluating the systems and tech- nologies that best meet an organization’s objectives. These topic areas, which will become part of the Framework, are: • Enterprise Architecture Planning (EAP) and Enterprise Architecture (EA) development process (developing the blueprints); • ITS Implementation Funding (how to pay for it); • Business Case Methodology (how well does this project fit into the your stated priorities; what are the risks, benefits and costs, and estimated return on investment [ROI]); • System Engineering for helping design and manage an ITS Project implementation; and • Post-Implementation Analysis for assessing system perform- ance (including reviewing the experience for lessons learned) and meaningful (estimated) ROI. These topic areas and concepts are described in Chapters 3 through 7. The following highlights the intent and scope of each chapter. 1.4.1 Enterprise Architecture (EA) and Enterprise Architecture Planning (EAP) The Enterprise Architecture Planning process is a set of activities used to develop the Enterprise Architecture mod- els, diagrams and descriptions. The process typically is stake- holder driven where the current performance measures, business processes, data, applications and technologies that are used in the organization are documented. The next step consists of documenting where the organization wants to be with respect to its business in the future, about four to five years. The organization consists of the corpo- rate mission, goals, objectives, and the business processes, data, applications and technologies that are needed to sup- port that vision. This is called the “to-be” architecture. A third step is a description of how to get there or a descrip- tion of the “gap” between the current (“as-is”) and the future (“to-be”). The Enterprise Architecture, both the “as-is” and “to-be” architectures, are composed of four or five models (depending on which Methodology is used— see Chapter 3 Appendix A) that are depicted in one or more diagrams, policy statements, procedures, inventories or other “artifact.” Chapter 3 on EAP/EA describes various IT industry approaches to EAP processes and EA artifacts. The transit industry does not have a track record in EAP (with only two agencies having documented their enterprise-wide EA); how- ever, some transit agencies are deploying segments of their architectures, through enterprise data, enterprise applica- tions, or during their business case and systems engineering processes. These examples will be discussed in greater detail in Chapter 3. 1.4.2 ITS Implementation Funding Chapter 4 on ITS Implementation Funding discusses guide- lines for obtaining, analyzing, and making use of various 53 Enterprise Architectur and EA e Planning (EAP) Post- Implementation Analysis Business Case Implementation Funding Strategies Methods Systems Engineering Figure 1. Synthesis topic areas.

sources of funding for IT/ITS projects. Like IT projects in gen- eral, transportation IT and ITS projects are delivered through public leveraging options like bond financing, public-private partnerships, comingled funding, and a variety of Federal, state and local funding sources. Transit agencies are using many of these financing mecha- nisms to access the various sources of capital for IT/ITS proj- ects. Historically, buy (pay-as-you-go), borrow (issue bonds) or lease were the primary financing mechanisms used by transit agencies. Since the 1990’s, creative use of these tradi- tional mechanisms and introduction of public-private part- nerships has occurred. Chapter 4 discusses financing mecha- nisms; in particular, the section describes four categories: debt mechanisms, capital leasing financing, equity and part- nerships, and credit enhancements. Chapter 4 discusses ITS implementation funding and ana- lyzes the best method for obtaining the necessary funds for the selected implementation. Based on the surveys conducted, no one financing method works for all situations, rather financ- ing decisions need to be tailored to the specific project, region and financial circumstance. 1.4.3 Business Case Methodology A Business Case Methodology (BCM) is a formal analysis used to justify and capture the reasoning for initiating a project. The business case typically reviews and verifies that (2): • The investment has value and importance • The project will be properly managed • The firm has the capability to deliver the benefits • The firm’s dedicated resources are working on the highest value opportunities • Projects with inter-dependencies are undertaken in the optimum sequence.” Chapter 5 discusses some of the different business case methodologies for justifying IT/ITS investments. Each of the methodologies use somewhat different techniques for build- ing the business case and determining return on investment, total cost of ownership, value of investments, risk factors, impacts, and opportunities. Some best practices and critical success factors associated with developing a good business case and business case methodology are included in the chapter. 1.4.4 Systems Engineering Systems Engineering is a discipline that attempts to ensure that customer needs are implemented in the system that is developed. Customer needs are defined by the stakeholders or people who have an interest in the system, as a user, a man- ager, or as someone impacted by the operations of the system (i.e., recipient of information or process coordination part- ner). Customer needs drive the system requirements, or what the system should do. For example, if there is a need to mea- sure ridership at stops (boardings and alightings) for each trip, then there is a corresponding requirement for the Automated Passenger Counting (APC) system to count boardings and alightings at each stop by trip identification. The systems engi- neering process must ensure that the requirement is described in the design, and consequently implemented in the software, data collected, stored and reported in a format that is consis- tent with its use as a performance measurement. The steps prescribed by the Systems Engineering process ensure a struc- tured approach to track the need throughout the development stages. U.S. DOT recognized the potential benefit of the systems engineering approach for ITS projects and included require- ments for the use of the systems engineering process in the FHWA Final Rule/FTA Final Policy on Architecture and Standards that was enacted on January 8, 2001. Chapter 6 dis- cusses the major steps that comprise the Systems Engineering analysis process and the results of the transit industry scan that shows the limited understanding and implementation of the policy by transit agencies. 1.4.5 Post-Implementation Evaluation Post-implementation analysis or Post Implementation Review (PIR), as it is commonly called in the IT field, is con- ducted after a project has been completed. “The purpose of the PIR is to evaluate how successfully the project objectives have been met and how effective the project management practices were in keeping the project on track (3).” Chapter 7 discusses what a PIR is and is not. The PIR is not the testing and verification activities that are typically per- formed in a project acceptance or closeout phase. As an exam- ple, the AVL system selected to meet the goal of increasing rid- ership may have to be accepted from a vendor if it performs according to the requirements in the Request for Proposal, it passes the test plan and the systems engineering verification process. The system, however, may not perform the way the users want. Perhaps the business changed or it was specified ambigu- ously and/or incorrectly. The PIR occurs after the IT/ITS system has been incorporated into the business and assesses how well the project meets the users’ needs, what needs to be done next, and how well the implementation process went. Developing and sharing lessons learned can continu- ously improve the agency’s project acquisition and manage- ment processes. The current practice in the industry, recom- mended practices, and a checklist for managers is included in Chapter 7. 54

2 Synthesis Methodology and Industry Scan To develop an assessment of the state of the practice, the research team reviewed available industry literature and conducted telephone interviews with a sample of transit agencies as well as several state DOTs. The literature search and interviews covered the five major elements to be included in the Transit Enterprise Architecture and Plan- ning Framework: • Enterprise Architecture Planning (EAP) and Enterprise Architecture (EA); • ITS Implementation Funding; • Business Case Methodology; • Systems Engineering for ITS Project implementation; • Evaluation for post-implementation measurement includ- ing assessing meaningful (estimated) ROI and performance. The literature search focused on innovative approaches in the IT and transportation industries, and in particular, tran- sit. Sources included professional journal articles, guidebooks, and tools that are available on the web, including several reports published by the Transit Cooperative Research Program (TCRP). To provide a reasonable sample of agencies for the telephone interviews, a group of 14 transit agencies and three DOTs was selected for interviews. Survey protocols were developed for the interviews. A standard set of interview questions was administrated to all the agencies. In addition, some agencies were asked more detailed questions on some Framework ele- ments, if the screening questions discovered areas to probe further, and if time was available. The first column in Table 1 shows the agencies and state DOTs that were interviewed. Several agencies were asked more detailed questions about their experience with the Framework topics. The checked columns in Table 1 represent the agencies that were surveyed in more detail on selected topics. 3 Findings on Transit Enterprise Architecture Planning and Enterprise Architecture This chapter presents methodologies for planning and documenting an Enterprise Architecture (EA) and discusses the transit survey findings. Developing an enterprise architec- ture is often perceived to be an arduous, expensive and lengthy process performed by outside consultants who spend many months at an organization and then provide a huge report with countless diagrams and, tables that ends up sit- ting on a shelf. The scan of the practice for Information Tech- nology (IT) and transit ITS shows that this was often the prac- tice in the past. Today, however, many industries have cre- ated useful templates for documenting an Enterprise Archi- tecture, including the business processes, data, performance measures and technology characteristics of their industry. The process of creating an EA has also been improved by new shortcuts and segmenting the enterprise into smaller, more manageable chunks. Industries are now starting to realizing substantial cost and time savings through Enterprise Archi- tecture Planning (EAP). The development of an enterprise architecture has shown significant benefits in planning for information technology programs and obtaining important information about the business. In “The Value of Enterprise Architecture,” (4) the author reported savings in several areas: • “Savings of 60% of the man-day efforts needed for collec- tion, processing, validation and reporting on the elements of the enterprise architecture—a task done continuously for reasons such as SOX, risk management, data protec- tion, user satisfaction etc. These savings alone can finance an EA program • “A 10% increase in deliverables from investments in IT projects by architecturally checking projects in the prepa- ration phase to ensure potential risks are identified and mitigated and also to avoid architectural conflicts during the execution of IT projects. • “A 10% reduction in yearly operating costs by the discov- ery of redundancies or excessive spending in the IT support to business and by standardizing the architecture, which not only leads to cost reduction but also to increased busi- ness and IT flexibility and agility.” For example, when a manager wants to save costs by elimi- nating an ancient data reporting system, it is important to know what parts of the agency access data it manages, which applica- tions depend on it, and which customers or users would be affected. This chapter is divided into two sections. First, it discusses the discipline of Enterprise Architecture Planning and the evolution of the leading methodologies. These are formal methods that describe detailed steps, products and building blocks that an organization uses to develop an Enterprise Architecture. The second part of the chapter discusses how transit cur- rently approaches Enterprise Architecture Planning (EAP) and the development of Enterprise Architecture. While few transit agencies have enterprise architecture planning processes in place or have developed an enterprise architecture in the formal sense, there are ways that organizations are employing “enterprise thinking” in making technology investment deci- sions. The brief scan shows that the majority of transit IT pro- fessionals are not making use of available industry tools to 55

develop segments of the enterprise architecture, and they miss capitalizing on the lessons and benefits of “enterprise thinking.” 3.1 Main Purpose of an Enterprise Architecture Enterprise Architecture is a structure that provides executive managers visibility into the overall relationships among their people, processes, technologies and performance. It enables executive managers to plan their technology investment deci- sions to better meet corporate business needs and processes. The Enterprise Architecture benefits organizations by: • Identifying where to reduce IT costs and complexity, by increasing the visibility of information flows and relation- ships between data, systems, technologies and business processes • Increasing business value and effectiveness through improved technology deployment The proliferation of servers and operating systems that started over two decades ago was one of the earliest drivers of enterprise architecture. Organizations started to have new problems as they accumulated different servers, operating system products and versions, database products, and vari- eties of personal computers with different variations of desk- top software. Soon IT could not keep up; IT staff often didn’t know all the software and hardware varieties used through- out the company. The lack of technology planning generated inefficiencies in the workforce and in the production of needed information. Agencies needed to hire experts to 56 Table 1. Transit agencies and state DOTs interviews for industry scan. ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓

maintain all of their equipment; they needed to send their people to training on all the platforms they supported; they needed to add specialists to their PC help desk to support the variety of applications. Most agencies now have hardware and software inventory lists that document every computer, applications, infrastruc- ture software and peripherals, as well as a set of systems specifications and standards for software/products used for databases, operating systems, web applications, application development and more. The inventories and standards are typical of the elements contained in a “Technology Enterprise Architecture” layer. Building the lists and standardizing the technology was driven by the need to align the technologies and future investments with the corporate resources (staff, skills) needed to support them. The current methods used to plan for and document enter- prise architectures take this concept further. In addition to a “Technology Enterprise Architecture” layer, a typical EA includes three to four other interrelated architecture “layers.” In a Government Technology article (5), the author states, “. . . one of the key things to know about enterprise architec- ture is that it is not ‘just an IT matter’—it involves the discus- sion and clarification of business processes and procedures. There is no sense building applications and an infrastructure that simply automate disorganized or inefficient processes, so defining and documenting business processes are key com- ponents of a full enterprise architecture undertaking.” The Business Architecture describes the business and includes details on business processes, work flows, and roles and responsibilities needed to meet the business goals and objectives of the organization. It describes the “who, what, where, why, when and how” business processes are accomplished. The Busi- ness Architecture helps project developers understand the busi- ness, identify stakeholders, find dependencies, and generally expedites the information gathering tasks needed to develop requirements. The Data Architecture describes the data and data struc- tures used by a business and its technology applications. It includes the meaning and relationship of information, infor- mation on data integration needed by the organization, and answers the questions of who, what, where, why, when and how the data is managed. A Data Architecture can help a tran- sit agency minimize ITS project delays due to missing or mis- understood data, such as when a trip planning system is pur- chased and its operation is delayed because accurate bus stop data is not available. The Services Architecture, which used to be called Appli- cations Architecture, describes the organization’s technology services and applications, such as web services, Automated Passenger Counters, customer information systems, inven- tory systems, Human Resource systems, etc. The Services Architecture contains other information about the applica- tions, such as the flow and delivery of information among subsystems, application versions, and restrictions on use. It helps identify integration opportunities and problems, sys- tem dependencies, gaps in functional coverage, the status of systems, and helps ensure that the development and enhance- ment of applications align with the business strategies of the organization. The Performance Architecture is a relatively newer part of an EA. It is a standardized framework to measure the perform- ance of major IT investments and their contribution to pro- gram performance. It includes Mission Goals and Objectives and Performance Measures. There are numerous examples in transit, where an EA would have eliminated ITS project delays and cost overruns. For example, there are instances where transit agencies procured automated passenger counting (APC) systems, only to real- ize that they are missing the people or processes to locate, track and update the data required as input to the system. Had an Enterprise Architecture been in place, it would have shown the connections among the business processes, data sets and technologies. It would have revealed the missing processes and data from the onset, allowing for better plan- ning and budgeting, and successful project delivery. Typically, the Enterprise Architecture (EA) is composed of three major parts: • Current or “As Is” EA • Future or “To Be” EA • Gap Analysis with transitional EA models The Enterprise Architecture begins by describing the peo- ple, processes and technologies in use today. It documents current processes, technologies and adopted standards and identifies problems, bottlenecks and missing linkages among the enterprise elements. Next, the Enterprise Architecture of the future identifies how to resolve these problems and struc- ture an organization where business goals are addressed by clearly defined processes, consistent data, easy to use applica- tions and “well oiled” technology solutions. The Gap Analy- sis defines a transitional program that identifies the stages necessary to move towards the future enterprise. The Enterprise Architecture Planning process is the work that is done to develop and update the Enterprise Architec- ture. The process is driven by many factors, including map- ping the corporate vision, customer expectations and stake- holder needs to the EA. The enterprise architecture planning process reveals the pieces and connections to all parts of the business, so that all the stakeholders along the activity chain see the flows of data, work and outcomes. By doing this, other groups within the organization may benefit from the processes being imple- mented, such as a process to manage bus stops, or they may 57

find a group already doing similar processes. The processes and technologies employed to manage bus stops may be elevated to an enterprise activity, thereby aggregating contributions from multiple departments, benefiting multiple users, eliminating redundancies by creating a single bus stop inventory source and improving corporate effectiveness through adoption of standard operating procedures. The main purpose of an Enterprise Architecture Planning Process (EAP) is: • To engage key stakeholders and IT staff in understanding the connections and dependencies among various parts of the business and work together to improve the Enterprise’s overall effectiveness by reducing redundancies, leveraging technology investments for multiple processes, and build- ing a seamless information infrastructure. • To prioritize enterprise information technology needs with respect to the organization’s strategic goals and objectives, particularly as the needs relate to technology investment decisions (build, operate and maintain). The key benefits of an EAP include: • Ensuring there is consensus among key decision makers about the organizational objectives, needs, priorities and business processes, and how they are served by the technol- ogy investments; • Ensuring that there is an awareness about how the deci- sions related to technology investments such as business processes that operate and maintain the technology invest- ment (including the lack of investment) impact the organ- ization, its people, objectives, needs, priorities, and assess- ment capabilities. Although the different Enterprise Architecture (EA) and Enterprise Architecture Planning (EAP) methods in the industry have similar categories and major processes, the industry is not consistent in the meaning of terms, classifi- cations, and scope of EA and EAP. A more detailed discus- sion of the definitions can be found in the Appendices of this chapter. In addition, the appendices include a discus- sion on how the term “framework” is used with respect to EAP/EA. 3.2 General Approach to EAP/EA Used by Other Industries This section describes two well-known, non-proprietary approaches to EAP/EA, the Federal Enterprise Architecture (FEA) and The Open Group Architecture Framework (TOGAF). The two approaches are introduced to show some similarities and differences, and to help the reader gain addi- tional familiarity with common EAP/EA concepts and terms. Some structure and content for this project’s Transit Enter- prise Architecture and Planning Framework will come from these two EAP/EA approaches. In addition, there are dozens of other hybrid approaches and even these two approaches have influenced each other over the past decade. The TOGAF, derived from the Department of Defense approach, is typically used as a set of templates with step-by- step instructions for developing, planning for and imple- menting a segment of an enterprise architecture. The FEA was originally a set of four reference architecture models and a process to build a plan to move from the current architec- ture representation to a future architecture vision; as it evolved, the reference architecture models grew into a tax- onomy that could be used as building blocks to describe seg- ments of the enterprise; the planning process became a prac- tical roadmap that involved developing a transition plan, incorporating core services, building a business case, and devel- oping an implementation plan based on a systems engineering process. 3.1.1 Federal Enterprise Architecture (FEA) The FEA has grown into a multi-faceted program to define methods, tools and assessment strategies for the Federal gov- ernment to develop Enterprise Architectures that describe business processes for improvements in key areas. In their own words: . . . the FEA is entirely business-driven. Its foundation is the Business Reference Model, which describes the government’s Lines of Business and its services [including financial, HRM, etc. and cross-cutting profiles like geospatial] . . . The outcome of this effort will be a more citizen-centered, customer-focused govern- ment that maximizes technology investments to better achieve mission outcomes. (6) Figure 2 shows the business-driven model. The agencies share similar functions (“lines of business”) such as Financial, Human Resources, and Homeland Security. They also are in need of integrated, cross cutting services (“profiles”) such as geospatial, security and records management. This three- dimensional model shows the inter-relationships and shared functions among the Federal government departments. When the effort began, the focus was on a development process. The result was an EAP process as shown in Figure 3. The EAP process for each Federal agency to develop Cur- rent and Future architectures, document their standards and policies, and develop transitional plans was daunting. To sup- port the planning process, the FEA working groups devel- oped three Reference Architectures (7) that described Perfor- mance Measures, high level Lines of Business, and a Data Reference Model (8) that could be used as a reference or tem- 58

Architecture Methodology (FSAM) Practice Guidance docu- ment (9). According to the FSAM Overview: A segment architecture is a detailed results-oriented architec- ture (baseline and target) and a transition strategy addressing a vertical or horizontal portion (or segment) of the enterprise. (9) Today, the FEA is composed of • Five-layer reference model (performance, business, infor- mation, services, technology) • Segment architecture process and guidance • Taxonomy for cataloging assets that are part of the EA • Process for creating an enterprise architecture • Transitional process for migrating from pre-EA (current) to post-EA (future) • Built in approach for measuring progress and success (through the performance model) • Self-assessment approach for determining success of using the EA to drive business value The approach seems to have paid off; several agencies have published detailed operational concepts and business processes, 59 Figure 2. FEA Business Reference Architecture. Source: FEA Guidance Figure 3. FEA Enterprise Architecture Framework from Version 1.1. Source: Version 1.1 FEA Enterprise Architecture Framework plate upon which the Government Departments could build their structures. By 2005, that effort was substituted with a new approach. The new emphasis was on scoping the devel- opment into smaller, manageable segments. The new approach using Segment Architectures was estab- lished in 2007, with the publication of the Federal Segment

requirements, data models and dictionaries, services and tools that are tied to their performance metrics. This approach may help the transit industry define a man- ageable process to allow it to reduce the complexity of the transit enterprise into parts that can address core processes and priority areas. However, the industry is missing a reference architecture (business, performance and data) which could serve as a template to define the “lines of business” and the cross-cutting functions. 3.1.2 The Open Group Architecture Framework (TOGAF) TOGAF is a set of resources and process guidance for devel- oping Architectures. The Framework is composed of three major parts • Architecture Development Method (ADM) Cycle • Enterprise Continuum • Resource Base The ADM is a set of guidelines that describes and guides developers through an enterprise architecture process that “meet[s] the business or IT needs of an organization.” (10) The Framework guidance emphasizes the need to scope the cycles through the architecture. The ADM, illustrated in Fig- ure 4, is cyclical, starting at the preliminary phase and cycling through steps A through H at ever increasing levels of detail. The cycle may be scoped by detail or process. The general Framework is comprised of a set of processes, tools, and building blocks for any industry, public or private, to use to develop an enterprise architecture. The cyclical nature enables a means of reducing the complexity, either by devel- oping increasing levels of detail for the four architecture mod- els specified by the method, or using a “segment” approach, similar to the one introduced by the FEA approach. The TOGAF approach depends on using a set of building blocks. It offers many resources to contribute to the Architec- ture development. TOGAF calls this part of the Framework the Enterprise Continuum. The Enterprise Continuum is described as a “virtual repository” of methods, patterns and solutions that help build an organization’s Architectures. The FEA reference architecture documents would be considered an industry representation of its core business, performance and data. The Resource Base is composed of standards, poli- cies and solutions (like open “web services”) that are available for the industry to procure off-the-shelf, or integrate into their industry applications. These ADM, Enterprise Contin- uum and Resource Base are explained in more detail in this Chapter’s Appendix C. The TOGAF model provides a framework for planning the Enterprise Architecture. Although many of the solutions, tem- plates and processes may be applied to transit, it is missing the tailored content, business and performance taxonomy that links the model to transit. 3.1.3 Industry Implementation Approaches There are hundreds of articles, workbooks, tools and guide- lines for developing Enterprise Architectures. The materials range from overviews to dense documents that could double as door-stops. The organizations described in this section docu- mented their approach, recording and organizing their vision, goals, and business practices. 3.1.3.1 National Association of State CIO (NASCIO) NASCIO develops many tools for State and local government information technology managers to deliver better services to their constituents. One initiative they undertook in 2004 was to publish an Enterprise Architecture Toolkit (11). The toolkit is a step by step process on how to build an Enterprise Architecture including identifying stakeholders, listing roles and responsibil- ities, collecting information on template forms for each archi- tecture domain and, finally, techniques for program manage- ment, EA lifecycle management and governance (see Figure 5). The Toolkit is organized into several books dealing with Architecture domains: Business, Information, and Technol- 60 Figure 4. TOGAF architecture development method cycle. Source: TOGAF Version 8.1.1

ogy. Similar to FEA and TOGAF, the NASCIO methodology includes a fourth domain, a Solutions Architecture, which describes requirements, design and approaches for imple- menting the conceptual architecture models. The EA Portfo- lio is the inventory documenting the characteristics of the four domains. The projects that derive from the domains are also contained in the Portfolio. The entire process is subject to a formal Architecture Governance process which falls under the purview of the Executive in Charge (e.g., CIO), wherein technology projects are prioritized and verified for confor- mance to the policies of the EA Program. Some states that developed an Enterprise Architecture require all project requests and business justifications to be tied to the business process, data and technology described in the Enterprise Archi- tecture. The business case serves as a check on the procurement process to ensure the consistency, integration and staging of the project with the state’s mission and goals. 3.1.3.2 Michigan Enterprise Architecture Framework Guidelines A concise document, the Michigan Enterprise Architecture Framework Work Plan Guidelines (12) describes the process for Michigan state agencies to develop a state- wide architecture. The Guidelines include three sections. The first section describes the process model; it includes “. . . eight process activities and coordination and integration requirements. The materials are based upon and in part excerpted from the Gartner EA Process Model, particularly ‘Gartner Enterprise Architecture Process: Evolution 2005’”. (12, p.4) The second section describes the EA Framework. The Michigan approach defines the term Framework as “structure.” The Framework is a description of the drivers, “requirements, enterprise architecture viewpoints (business, technology, information and services), enterprise solution approach and requirements, and governing, managing and accountability.” The framework section is derived from three sources: “Gartner Enterprise Architecture Framework: Evo- lution 2005”, “Architecture Frameworks: Some Options”, and “NASCIO EA Development Tool Kit,” Version 3.0. The final section describes the Work Plan and includes best prac- tices and guidelines for developing the EA. 3.1.4 Lessons Learned on EA/EAP from Other Industries Several lessons may be learned from the scan of other industry best practices with respect to developing a Frame- work that includes EA/EAP. The lessons include the following: • Reduce the complexity of EAP process; • The Enterprise Architecture process is cyclical, each cycle should focus on a limited scope of the business or level of detail; 61 Figure 5. NASCIO enterprise architecture process model. Source: [NASCIO Toolkit].

• Provide resources (templates and building blocks) that EAP facilitators and EA developers can use to expedite the process; • Provide a check on corporate projects to ensure that the EA (driven by corporate goals and core processes) was used in project scoping, development and deployment activities. The following activities or approaches support the imple- mentation of the four lessons: • Develop an overall, high level structure that may be used as a template to build an enterprise architecture. • Create a taxonomy (terms and definitions) of core business, performance and data that are related to the transit business. The taxonomy, like a reference architecture, may be used as a resource to help agencies build an enterprise architecture. • Develop a repository of enterprise “artifacts” that may be used as examples of procedures, guidelines, policies, and standards that apply to the different architecture models. • Build an enterprise architecture in manageable segments, tied to priority and core business processes. • Develop outreach materials to educate the industry on the benefits and uses of an enterprise architecture. • Develop a governance structure that reviews the proposed project’s role within the enterprise (performance, business, information, services and technology). 3.2 Transit Approaches to EAP/EA The scan of the transit industry revealed limited adoption and understanding of Enterprise Architecture Planning and Enterprise Architecture. Among the organizations we inter- viewed, most of the CIOs or IT managers were familiar with the concept, particularly if they came from other industries. How- ever, few had the resources or management support to under- take a comprehensive enterprise architecture planning process. Fewer were versed on the “segment architecture” approach currently applied by other industries. 3.2.1 Transit EAP/EA: Lessons Learned from the Literature TCRP Report 84 Volume 5: Concept for an e-Transit Reference Enterprise Architecture The TCRP J-09 Committee published a research paper [Report 84 Volume 5 (13)] that looked at how the disciplines of Enterprise Architecture and Systems Engineering work together to help the industry “quickly assess the impacts of potential opportunities of changes and new developments.” (13, p. 2) A recommendation was presented to develop a Transit Reference Enterprise Architecture. The report described a development process (13, p.4) that was based on similar paradigms of the architecture develop- ment methods: • Capture the “As Is ‘Transit Today’ ” • Describe the vision for the “To Be ‘Transit of the Future’ ” • Document the “typical sequence of actions and their impacts” or implementation plans for transitioning from current to future. Advanced Public Transportation Systems: The State of the Art Update 2006 Another study, sponsored by the US DOT, the State of the Art Report (14), described a few Enterprise Architecture devel- opment efforts in the transit industry. In particular, the report identified a specific challenge/lesson that was learned from organizations that performed a formal or informal internal enterprise architecture: Integration of technology cannot occur without the integration of business objectives and policies of the departments and/or agen- cies that are expected to cooperate in an ITS project. (14, p. 50) The statement reiterates the need to overlay a “governance” structure around the development of the enterprise architec- ture to ensure that executive manager and stakeholder partic- ipation and buy-in are incorporated into the development of the future architecture. Other Literature about Transit ITS Few lessons learned emerged through the industry scan because few organizations engage in planning and document- ing their enterprise architecture. Industry literature related to transit ITS technology deployment is rife with examples about how the lack of enterprise architecture planning is limiting success in system deployments. The transit literature identi- fies the issues, such as: Concentrate on the soft side [planning and business processes] of the system—this is where success is really achieved . . . Ensure that staff . . . understands how to use the data. Think long-term, and ensure that data structures can be integrated with down- stream applications. (15, p. 24) Underestimating the degree to which advance planning was needed . . . Ensuring support from IT, maintenance, and other parts of the organization . . . Adapting business practices and operating procedures. (16 p. 35) These studies do not explicitly point to a solution such as Enterprise Architecture Planning; this may be because there is a lack of understanding and guidance about how Transit EAP helps executive managers run their organizations more effectively. The 2006 State of the Art Report identifies key 62

obstacles to deploying Transit ITS that pinpoints areas where Enterprise Architecture or even “enterprise thinking” will directly benefit transit agencies: Key obstacles [to deploying Transit ITS] include: • The stand-alone nature of most individual technology deployments limits the benefits that could be provided by business-oriented, enterprise-wide technology strategies; • Most technology-based applications require continuous cooperation and coordination between and among many different departments, agencies, and jurisdictions that are often difficult to achieve; • Limited resources and gaps in education and training in the integration, use, and maintenance of technologies and the standards necessary for interoperability and data sharing make it difficult for transit professionals to keep up with technological developments and opportunities; • Fast-paced changes in technologies put deployment efforts at risk. (14, p. 6) 3.2.2 General State of Enterprise Architecture Adoption by Transit Agencies As reflected in the State of the Art Report Update 2006 (SOA), few organizations are following a formal method to develop an Enterprise Architecture. The industry scan reflected the same results. There are two areas that may pro- vide lessons for the industry. Progress toward enterprise development in transit, particularly Transit ITS, is occurring in Enterprise Data and GIS. The SOA Report discussed Enterprise Data as a key ingredi- ent towards integration. Several organizations have made sig- nificant strides in developing and implementing “enterprise data models” including TriMet, King County Metro, UTA, and other organizations that were not interviewed as part of this scan. Still, in the most recent publication Synthesis on AVL for Bus Transit (16), when asked, “What was the biggest way in which your bus AVL system has not met expectations the agency had when the decision was made to deploy?”, a signifi- cant number of responses cited the “[h]igh level of effort required for data management and reporting,” (16, p. 36) “data integrity,” and core data processing, exchange and archiving issues related to ITS bus systems. These problems emerge when information does not conform to a consistent set of standards and policies across the enterprise. Organizations that have developed data policies (e.g., quality, reporting and mainte- nance), data dictionaries and enterprise data models and have ensured that vendor products conform to their data standards have had a much easier time deploying, operating and main- taining ITS. The SOA Report also identified Geographic Information Systems (GIS) as an area that is supporting enterprise services and data architecture development. There are a number of potential reasons for the development of an enterprise GIS approach. Factors include: • Availability and adaptation of geospatial standards that promulgate an enterprise approach; most of these stan- dards incorporate transit (geospatial) feature descriptions and relationships, as well as location services needed by transit business processes. The standards and products that conform to these standards constitute the building blocks needed to describe the EA GIS segment. • Literature and tutorial materials directly relevant to tran- sit. These include vendor materials, case studies published by TCRP and an industry-developed Guidebook on Best Practices for Using Geographic Data in Transit: A Location Referencing Guidebook. The latter work includes a section that describes a taxonomy for the business processes, data and functions (services) that comprise the Transit Enter- prise GIS architecture domains. • Availability of training and conference opportunities that promote enterprise approaches for deploying GIS in transit. Many agencies see a single enterprise software tool such as Resource, Asset or Maintenance Management Systems (RMS, AMS, MMS) or a Customer Relations Management System (CRM) as an enterprise architecture. Certainly, these tools are solutions for critical business processes. However, they are no substitute for developing the four or five layered enterprise architecture, as exemplified by the difficulty that many agen- cies still encounter when deploying ITS such as Computer Aided Dispatch/Automated Vehicle Location (CAD/AVL), Automated Passenger Counters (APC), and Customer Infor- mation Systems/Trip Planners. 3.2.3 Miami-Dade Transit Miami-Dade Transit (MDT) initiated an enterprise archi- tecture planning process in 2002. The project lasted about 18 months including the development of the Transit Mission and Goals through the development of the IT/ITS Strategic Plan. Consultants were hired to develop the Enterprise Archi- tecture in coordination with IT staff. As illustrated in Figure 6, the project tasks were modeled after the FEAF method (see Chapter 3 Appendix B for a more detailed discussion of the FEAF), gathering informa- tion on the business environment, documenting the current “as is” architecture domains (business, data, applications and technology), developing stakeholder-driven target archi- tectures, understanding the gap between the current and target and describing how the IT organization needed to adapt to the changes. Finally, the IT 2003–2006 Strategic Plan was developed and a subset of the plan was extracted to define the IT/ITS Strategic Plan. 63

The Business Architecture was divided into several segments. Figure 7 shows how the six major corporate goals (Maximize Use & Efficiency, Educate Community, etc.) drive the business processes (Service Implementation, Service Management, etc.). The goals are traced from the business processes to the other architecture domains and eventually to the strategic plan, show- ing the impact, dependencies, and overlap among the projects in meeting those goals. Table 2 describes the six segments that were included in the architectures. In addition to linking the corporate vision to the business, the Environment described stakeholders (internal and external); their roles and responsi- bilities were described and mapped to the detailed process level. The Enterprise Architecture business process products used Unified Modeling Language (UML). Use Case and Activity dia- grams and were cataloged in tables. MDT developed some very detailed activity diagrams to model specific processes within their organization. The more detailed business processes that were related to ITS used the Transit Communications Interface Profiles (version 1.1) business areas and functional descriptions as a reference and taxonomy for more detailed segmentation of the business processes. The internal infrastructure business processes (e.g., finance, human resources, payroll and procure- ment) were modeled after the organization hierarchy. The current technology, applications and data sources were documented in lists. There was significant difficulty in develop- ing a detailed data architecture because many of the data sources were closed and subsumed by the proprietary applica- tions, and even the interfaces were covered by intellectual prop- erty restrictions. To that end, the data architecture identified core datasets and the development of a centralized data model was identified as a high priority project in the strategic plan. Where appropriate, linkages were made between the MDT EA (including the on-board/vehicle subsystems) and the Regional ITS Architecture. The business architecture and data flows were mapped to the Regional ITS Architec- ture MDT subsystems and architecture flows. The high- level mapping depicted in Figure 8 shows the transit ITS and back office subsystems assignments to the National ITS Architecture Subsystems: Center, Roadside, Vehicle and Traveler Systems. The gap analysis and corporate goals drove the sequencing and scope of the projects included in the strategic plan. The strategic plan is being used as a roadmap to help MDT and the IT staff build out the architecture. 3.2.4 Washington Metropolitan Area Transit Authority (WMATA) WMATA has the most history with developing Enterprise Architectures. In 2001, they initiated an effort which resulted 64 Project Start Scope Objectives Plan Business Environment Vision/Mission Current Organization Goals Strategies Key Processes Critical Success Factors Current “As-Is” System Resources* Organization Processes Applications Hardware/Software Systems Software Technology Trends Task 1: Initiation Task 2: Business Model Step 5 Functional Business Model Key Indicators Data Architecture Application Strategy Principles Technology Architecture Hardware/Network Systems Software On-board Integration Principals IT Organization ITS Proj Mgmt Approaches Issues & Recommendations Trade-Offs Skills Cu rre nt - Fu tu re G ap A na lys is Task 4: Target Data and Application Architecture Task 7: IT/ITS Strategic Plan Regional Coordination Participation Legend Executive Sponsor/Business Champion Signoff Business Units OPTM/MDT Mgrs. IT 2003-2008 Strategic Plan Customer-Centric Business Model Enterprise Architecture Plan ITS Strategic Plan Guiding Principles Implementation Phasing Task 3: Current IT Task 5: Target Technology Architecture Task 6: IT Organization ITS State/County Guidance Figure 6. MDT enterprise architecture planning process and project tasks. Source: MDT.

in the published report “Renewing Technology Infrastruc- ture at the Washington Metropolitan Area Transit Authority: Assessment of the Current Enterprise Architecture.” (18) The assessment was comprehensive, although it did not cover the architecture of on-board revenue and non-revenue vehicles and linkages between the back office and on-board systems. Some of the recommendations on the “administrative” side of the business were implemented, including payroll, finance and customer relations management software. Following this effort, there were small efforts to develop “project” architec- tures for technology areas, for example, on-board bus and customer communications segment architectures. In 2007, WMATA hired a chief Enterprise Architect, who restarted the comprehensive enterprise architecture develop- ment effort. Although currently under development, WMATA is applying a cyclical effort by first developing a structure that defines the segments of their business and then drilling into the details of each segment. Figure 9 shows the high level Enterprise Architecture. The Business Architecture is composed of three domains: • Enterprise Administration • Integration • Transit Management Each domain is composed of Functions, and Functions contain Processes; Table 3 shows the relationship between Domain and Function. The excerpt in Figure 10 shows the ITS Traveler processes: Remote Traveler Support Functions for Rail and Bus and Personal Information Access. These processes are derived from the National ITS Architecture. A process such as the Remote Traveler Rail Support Processes is a snapshot of the business, information, application and technology architecture views, as well as the organization that participates in the business processes. The Remote Traveler Rail Support Processes crosses organizational lines including groups supporting customer operations, marketing, police, 65 Figure 7. Relationship between MDT goals and primary processes. (Source: MDT Strategic Plan [17].)

and public relations. The process is defined as “support of patrons while using the WMATA Metrorail transit system.” The information view (IV) supports maps, schedules, fares, alerts, emergency voice communications, etc. The applica- tion view (AV) includes Channel M, Incident Management, Passenger information Display server, among other applica- tions. The technology that supports the business process includes signs, intercoms, and other technologies. Finally, there is a direct connection between the business process and the performance measures by which the services are evaluated. In the ITS Traveler Function area, the WMATA Scorecard measures customer satisfaction for the modes: bus, rail and vertical (elevators and escalators). At WMATA, the Enterprise Architecture is used to drive technology investments. In the corporate publication, “Pro- fessionals’ Guide to Information Architecture Standards and Services”, General Manager John Catoe wrote in the preface (19, p. 3): This guide . . . describes our architectural approach to devel- oping new technology systems . . . Standardizing our informa- tion technology infrastructure is our strategy to take Metro into the future . . . The guide includes the hardware, application and infra- structure software versions used throughout the organization; the guide also includes a scorecard (as depicted in Figure 11) showing the maturity of the IT Capabilities with respect to the organization’s goals (with the darker shaded boxes on the right being most mature and the shaded boxes on the left being least mature). The Information Technology Capability Pyramid shows the progress towards meeting the goals of the future Enterprise Architecture. In addition, the Guide pro- vides a corporate awareness about the transit enterprise, stan- dards and policies and “stewardship” over IT resources. The procurement process includes governance over the selection of technology standards to monitor conformance with the direction of the IT enterprise. 66 Table 2. MDT enterprise business process segments. Segment Description (17, Vol 1, p. 1-18 to 1-20) Service Implementation Understanding the service needs of current and potential customers, developing a service plan and translating the service plan into deliverable service are complex processes. Many of the functional areas in the transit industry, including rail, bus and paratransit, rely on the information generated in this process. In particular, schedule information is used in a wide range of operational, customer information and on-board applications. Service Management The transit industry is always looking for ways to gain efficiency improvements in the process of managing daily operations. In particular, bus and paratransit operations can benefit if system applications streamline or eliminate manual processes. A new generation of transit operator support systems can provide increased flexibility in the assignment of resources and improved reporting capabilities. Customer Information Many transit agencies spend significant resources in the process of providing customer information. Information is developed and distributed via paper timetables, bus stop signs, on-board signage, customer information operators, the Web, etc. Safety and Security Protecting the safety and security of employees and passengers is extremely important for many reasons. Public perceptions of safety and security at transit stops and on the transit vehicles affect the likelihood of attracting and retaining customers. Both passengers and operators like to know that help is close at hand in the event of an unsafe situation on a vehicle. Applications such as CAD/AVL, security cameras, emergency alarms and better communications will enhance actual, as well as perceived, safety and security. Safety and security incident tracking systems help prevent incidents and deploy resources more effectively. Asset Management Work order, facility management and inventory systems are crucial to maintenance efficiency and controlling costs. Effective management of asset replacement programs ensures that information, systems and infrastructure can be replaced at the end of their useful life without interrupting transit service and reliability. Internal Infrastructure The higher level Internal Infrastructure process includes functions such as human resources, finance, payroll, risk management, procurement and managing information technologies and core data.

3.2.5 Other Transit Approaches to Enterprise Architecture Planning Agencies use several approaches that support “enterprise thinking” either across an architecture level or between archi- tecture levels. • Bottoms up inventory • Segment Architecture (cross cutting or vertical) like GIS • Enterprise data • Project architectures 3.2.5.1 Current Application and Technology Architec- tures through Inventories A first step to developing an “as is” architecture is documenting a list of all the applications, both software and hardware (versions, models, maintenance schedules and license agreements). In addition, these agen- cies are defining application and technology standards for their organizations. For example, agencies are consolidating their databases around a certain manufacturer’s version (Oracle 11i, SQL Server 2008, PostGresSQL), running their critical infrastructure on a certain operating system, or turn- ing to Open Source Software as their first choice for infra- structure software. MARTA developed several Technology Architecture models from 1998 through to their current sys- tem. These standards and their current technology and appli- cations may be documented as a set of inventories. As seen in many of these inventories, a spreadsheet is used to identify the owner’s organizational unit, IT steward(s), related soft- ware or hardware, manufacturer and version. Additionally, other attributes related to technology component perform- 67 Legend To Be Systems External Subsystems that interface with MDT Remainder are MDT Process Figure 8. MDT current EA allocated to Regional ITS Architecture model. (Source: MDT Strategic Plan [17].)

Figure 9. WMATA Summary Business Architecture. (Source: Adapted from WMATA Enterprise Architecture, June 2009. Licensed under a Creative Commons Attribution-ShareAlike License [CC BY-SA].)

69 FunctionsDomain Enterprise Administration (based on the Gartner Framework) Enterprise Asset Management Human Capital Management Financial Management Operations Management and Supply Chain Integration (based on the WMATA Framework) Regional Safety Enterprise Management Customer Service Transit Management (based on the U.S. DOT National ITS Architecture Framework) ITS Vehicles ITS Field ITS Center ITS Traveler Table 3. Relationship between WMATA domains and functions. Figure 10. ITS traveler functions from WMATA business architecture. (Source: Adapted from WMATA Enterprise Architecture, June 2009. Licensed under a Creative Commons Attribution-ShareAlike License [CC BY-SA].)

ance may be included in the table. The attributes may include elements of behavior, cost, reliability and capacity. The next step would be to link the technology, software and applica- tions lists, relating them to each other and the organization, referencing OS software to servers, servers to their location and applications to the server and OS software (and version). Some methodologies describe the “as is” architecture as a “bottoms up” approach to architecture, meaning that it is the act of unearthing what exists. Using this approach, any docu- mentation collection of current capabilities, services, processes and existing standards describes part of the “as is” architecture. The documentation may be as simple as a spreadsheet that includes the agency standards and internal procedures. Most medium to large size agencies have a centralized list of their applications, software and infrastructure. Perhaps missing from the inventories are the connections among the infrastruc- ture elements, applications and software and the linkages to the business processes, objectives and goals. 3.2.5.2 A Cross-Cutting Segment Architecture Develop- ment: Geographic Information Systems (GIS) As described in the FEA, the Segment Architecture is a process to investigate a section of an overall EA that is either a vertical business area (e.g., line of business) or a cross-cutting area. Since the early 1990’s, the transit industry has been developing GIS enterprise architectures, an example of a cross-cutting area. In July, 1992, Seattle Metro (aka King County Metro) published a compre- hensive Phase I study (20) detailing the cross-cutting business processes, functions and supporting applications, geographi- cally related data elements and organizational impact on meet- ing those needs. A Phase I second study, published in 1993 (21), described the alternative hardware, software and applica- tions that support the business processes. The analysis segre- gated the analysis into “infrastructure” (technology), “GIS Software” and “Software” (applications) and “Data” (data), anticipating the divisions promoted by subsequent EAP methodologies. In hindsight, the early King County Metro Phase I studies followed the FEA Federal Segment Architecture Methodology through to its implementation, even as it contin- ues to evolve to today. There are several transit agencies that have implemented enterprise GIS architectures. The early work of Seattle Metro, which was widely disseminated, may have contributed to the early adoption of this architectural segment. The work was used to help develop a comprehensive Guidebook on Geo- spatial data management from planning through implemen- tation, operations and maintenance (22). Though not a refer- ence architecture or step by step guide to developing a segment architecture, the Guide provides building blocks to develop a business process model, identify geospatial and location ser- vices and identify core data needed to fully describe a GIS enterprise architecture. Furthermore, the enterprise approach to a Geospatial enterprise architecture has been aided by the Geospatial industry. The Industry promotes and vendor products support open specifications and standards, which drives standardization in all industries that use the tools; in 70 Figure 11. WMATA IT maturity capability pyramid [from Oct 30, 2007]. (Source: Professionals’ Guide [19].)

addition, Google has made access to map data (through KML) and geospatial functions free and available. 3.2.5.3 Enterprise Data: Centralized Database Many agencies aspire to develop a centralized database of core data (or a distributed set of databases with seamless sharing of information). An enterprise database, similar to ones developed by BART, TriMet, King County Metro, Long Island Rail Road and UTA, supports the development of IT/ITS applications as well as other downstream applications. These agencies can develop “homegrown” applications and services that better support their business processes. Some organizations, like MARTA, are developing robust data ware- house applications. MARTA’s warehouse application drives a “dashboard” that displays up-to-date performance infor- mation showing up-to-date operational performance data. Very few have actually developed an enterprise data architec- ture or centralized database. The development, operations and maintenance effort to manage a database of this sort requires staff with specific skills and resources to ensure data integrity. Several artifacts contribute to the development of a Data Architecture and ensure a comprehensive description of the data infrastructure. These include: • Data principles with respect to treating Data as a resource and asset. • Business and logical (physical) data model • Data dictionary for core data including naming conven- tions, unambiguous definitions, and unique identifiers • Data management process models • Data interoperability requirements • Data lifecycle needs • Data security needs • Data reporting requirements and aggregated data descrip- tions (particularly for performance measures) • Metadata needs (Note: this list is based on several data management and data enterprise architecture descriptions which include these and additional products.) In developing their data architecture, Miami Dade Transit defined six key Data Principles (17, Appendix A, p. A-3). “The six key principles identified by MDT staff are: 1. Avoid duplicating the development and maintenance of data and datasets 2. For a core data element, establish a single point of manage- ment, collection, “cleaning”, and define an authority or “a system of record” to ensure data consistency and com- pleteness throughout organization 3. Create enterprise awareness of critical data (understand processes that depend on data) 4. Create an awareness of the value and cost of data through- out the organization 5. When creating core data, determine priority order based on overall criticality to application, operations, and the num- ber of users 6. Consider security, including access requirements and restrictions.” Agencies that develop and implement a core database have fewer problems deploying data-driven ITS systems than most agencies. One reason is that their data is consistent and sup- ported by their business processes. Furthermore, they have developed tools to support their internal customers that help make their agencies more adaptable. Finally, they have con- trol over how and who can use critical data. 3.2.5.4 Project Architectures and Enterprise Solutions Transit agencies are migrating to enterprise-wide applications through open “services” or interfaces that help distribute key information in a timely manner. Some agencies are undertak- ing comprehensive studies of the needs and requirements for these applications (the sections on System Engineering and Business Case Methodology will discuss the processes related to these types of applications in more detail). The planning work related to developing these enterprise applications may be described as a vertical segment of the enterprise architecture process. Although the industry scan was limited, the related vertical architectures included: • Travel Management Coordination Centers (for regional and statewide human service and community transportation) • Computer Aided Dispatch and Automated Vehicle Loca- tion (CAD/AVL) systems • BusRapid Transit (limited to data, services and technology— communications and related project linkages) • Data and Service Architectures related to customer com- munications on transit service • Maintenance Management (to enhance predictive mainte- nance processes) • Human Resources The enterprise applications tend to be costly, which ham- pers smaller agencies that have limited resources. Many applications target areas outside the scope of transit ITS proj- ects, such as personnel (HR), finance and equipment main- tenance. The IT industry is providing solutions for smaller agencies through the Internet. For example, Hampton Roads Transit is migrating some of its applications to “cloud” com- puting or “software as a service” solutions, that is, Internet- provided services that are free or may be acquired for a min- imal fee. Other organizations, for example, Long Island Rail Road and TriMet, use open source software products. More open source software products are available for enterprise applications. In addition to the well-known operating system 71

(LINUX), web server (Apache) and web browser (Mozilla) software, new open source software tools like Customer Rela- tions Management (23), Enterprise Resource Planning, Mobile Computing and Communications (including Voice Over IP) are being offered by application service providers. 3.3 Next Steps The lesson learned from this review is that Enterprise Archi- tecture Planning is difficult without the necessary building blocks that other industries enjoy. Several approaches to plan- ning and implementing the Enterprise Architecture are avail- able, including NASCIO and TOGAF; it is the description of the high level business processes, data and services models and their relationships that are missing from the industry litera- ture. Agencies generate hundreds of pages of documents that may be shared, however, there are no tools or web sites to post these models, policies, examples and documents. There are many TCRP and FTA reports on Performance Measures, Transit ITS technologies and their cost/benefits that are not categorized by a formal taxonomy, and thus they are hard to access and use. The industry would benefit from a formal wiki site, with hyperlinks to show the relationship among the five FEA architecture models. Chapter 3 Appendix A: Enterprise Architecture Definitions from the Literature Although the major processes and categories are similar throughout the different Enterprise Architecture (EA) and Enterprise Architecture Planning (EAP) methods, the indus- try is not consistent in the meaning of terms, classifications, and scope of EA and EAP. This section describes definitions of Enterprise Architecture and Enterprise Architecture Plan- ning. In addition, this Research study asserts a “Transit Enterprise Architecture and Planning Framework”. Many of the definitions in the following sections discuss their method or model as a “framework.” To that end, this section defines the term framework to broaden our understanding of the term with respect to this research. Enterprise Architecture The scope of “enterprise architecture” varies by methodol- ogy. Across different definitions, the common factor is that the enterprise architecture is a formal description of the system (in this case the “enterprise” is not just a single application such as a resource management system). Although each EA Method- ology definition differs on what is implied by enterprise, each consistently includes people, processes and technology. In addition, the Enterprise Architecture is described from various viewpoints, from the business owner, operations manager to the developer and line staff. The Enterprise is typically composed of four to five intercon- nected architectures (these were presented in the introduction): • Performance Measures, • Business Processes, • Information, • Services (applications), and • Technology. For example The Open Group Architecture Framework (TOGAF) describes Business, Information, Application, and Technology (10). The Federal Enterprise Architecture describes five architecture views as described in the side bar (6). 72 The Federal Enterprise Architecture (FEA) Enterprise Architecture Models The Performance Architecture is a standardized framework to measure the performance of major IT investments and their contribution to program performance. It includes Mission Goals and Objectives and Performance Measures. The Business Architecture includes details of processes, work flows, roles & responsibilities needed to meet the business goals and objec- tives of the organization. It describes the “who, what, where, why, when and how” business processes are accomplished. The Data Architecture, a conceptual data model that describes the meaning and relationship of information, includes information on integra- tion issues associated with enterprise data; it also answers the questions of who, what, where, why, when and how the data is managed. The Services Architecture describes application req’ts, and the flow and delivery of information among subsystems. It records the application versions, restrictions on use and other informa- tion on applications and interfaces. The Technical Architecture describes technical information on security, communications, and infrastructure policies and standards associated with the deployment of technology. An agency may specify database, desktop and server equip- ment and software, network protocols, as well as, define security and privacy policies.

The relationships among these architectures are typically described as seamless and interconnected where the Corpo- rate Vision, mission and objectives (Performance Measures) drive the need to implement Business Processes; Business Processes are driven by decisions that people make using good Information; Information is generated from Applica- tions and Automated Processes (Services); Applications run on Technology. The Federal Enterprise Architecture (FEA) defines Enter- prise Architecture as a description of information, services and technology that work together to solve and measure busi- ness needs. [Enterprise Architecture is] . . . information, services and technology across an enterprise, that work together to solve and measure business needs and processes. (24) The National Association of State CIOs (NASCIO) describes enterprise architecture as a methodology for “designing gov- ernment processes.” Their description is closer to one used for Enterprise Architecture Planning. Enterprise Architecture is the management discipline for de- signing government processes and technology investments for success. (25) The IEEE definition for “architecture” is derived from Steven Spewak, who standardized an approach for enterprise architecture planning. It defines an architecture as a frame- work that includes people (environment), processes and technology. The fundamental organization of a system embodied in its com- ponents, their relationship to each other, and to the environment, and the principles guiding its design and evolution. (Definition for Architecture-26) The Open Group Architecture Framework (TOGAF) def- inition [TOGAF 8.1.1 Glossary] assumes the word architec- ture has two distinct meanings, both a system description as well as a plan for implementing it. Architecture (10, Glossary) has two meanings depending upon its contextual usage: 1. A formal description of a system, or a detailed plan of the system at component level to guide its implementation. 2. The structure of components, their inter-relationships, and the principles and guidelines governing their design and evolution over time. A Reference Enterprise Architecture has a different mean- ing then enterprise architecture. A Reference Enterprise Archi- tecture is a taxonomy and set of relationships described by an industry to define its core business. Many industries have developed a reference architecture to standardize functions, services, and critical performance measures, as well as to pro- mote interoperability and off-the-shelf tools that meet their business needs. The Federal Enterprise Architecture has three reference architectures—Performance, Business and Data. In addition, each Federal Department is tasked to define more detailed Lines of Business (e.g., business processes) which will add to this taxonomy. Additionally, some agencies are driv- ing the reference architecture to increasing levels of detail by developing Concept of Operations descriptions (for example, personnel management) or specifications for cross-cutting functions (such as geospatial services) that will ensure inter- operability across the Federal government. Enterprise Architecture Planning Methods that define Enterprise Architecture Planning are consistent in their understanding of the term’s meaning. It is a process to develop (1) a set of enterprise element descriptions, and (2) a plan to implement the systems that compose the architecture descriptions. Steven H. Spewak, in his ground- breaking book Enterprise Architecture Planning: Developing a Blueprint for Data, Applications, and Technology, lays out a method for implementing the planning process. Spewak defines EAP as follows: Enterprise Architecture Planning is the defining architectures for the use of information in support of the business and the plan for implementing those architectures. (27) The NASCIO also defines the term EAP similarly: The processes necessary to direct or guide initiatives, to ensure that performance aligns with the enterprise, to enable the enter- prise business by exploiting opportunities, and to ensure resources are used responsibly and architecture-related risks are managed appropriately. (11) The FEA and TOGAF (which uses the term Architecture Development Method (ADM) cycle) both apply Systems Engi- neering and Program Management planning processes to the EAP. The method used in the FEA and TOGAF frameworks will be discussed in the section below. Framework The word framework is used differently depending on which Enterprise Architecture methodology is described. Generally, framework is used to describe “a structure.” In the last decade it is evolving to mean a set of processes that are used to implement an Enterprise Architecture. The use of the term to mean structure is nowhere more evi- dent than in its earliest use by the father of Enterprise Archi- tecture, J. A. Zachman. Zachman defined the enterprise archi- tecture framework as a set of products and terms that are 73

associated with the cells in a matrix where the rows describe the perspectives of major players of the enterprise (planner, owner, designer, builder, subcontractor, and enterprise), and its columns refer to “what, how, where, who, when, why,” or “data, function, network, people, time and motivation.” Later in life, as described by Roger Sessions in [1], Zachman described his work “. . . as it applies to Enterprises . . . [as] a logical structure for classifying and organizing descriptive rep- resentations of an Enterprise.” (28, p. 11) In his paper from 2007, Sessions describes Architecture Framework as a structure that consists of artifacts (docu- ments, reports, analysis, models or other physical descrip- tion) or products that describe parts or perspectives of an Enterprise Architecture. He states that an Architecture Framework is: A skeletal structure that defines suggested architectural artifacts, describes how those artifacts are related to each other, and pro- vides generic definitions for what those artifacts might look like. (Definition for Enterprise Framework from 28) This definition is similar to the definition that the US DOT uses to define the National ITS Architecture, “reference archi- tecture framework” for the transportation system. The US DOT defines Architecture as: A framework within which a system can be built. Requirements dictate what functionality the architecture must satisfy. An archi- tecture functionally defines what the pieces of the system are and the information that is exchanged between them. An architecture is functionally oriented and not technology-specific which allows the architecture to remain effective over time. It defines “what must be done” not “how it will be done.” (29, Glossary) TOGAF defines a framework as a set of processes that sup- port the development of an architecture. The TOGAF method is a framework as described by its name. This defini- tion is closer to the one that describes this research Transit Enterprise Architecture and Planning Framework. A tool for assisting in the production of organization-specific architectures. An architecture framework consists of a Technical Reference Model, a method for architecture development, and a list of component standards, specifications, products, and their inter-relationships which can be used to build up architectures. Similar to TOGAF, when the Federal government launched the enterprise architecture development program in 1998, it named itself the Federal Enterprise Architecture Framework (FEAF). FEAF consisted of both the structure of the different models of the enterprise architecture based on the Zachman framework as well as an adaptation of the methods advocated by Spewak’s EAP methodology. In 2002, the Federal program renamed the FEAF methodology to Federal Enterprise Archi- tecture (FEA). Chapter 3 Appendix B: FEA Segment Architecture Description A Segment Architecture is a prioritized section of the Fed- eral Enterprise Architecture that contributes to describing a critical part of the Enterprise Architecture at a greater level of detail. For example, the Federal Geographic Data Committee (FGDC) describes its three lines of business (30): • Identify common geospatial requirements, responsibili- ties, and capabilities across [the] government • Allow for improved coordination of acquisition and oper- ations to government-wide benefit • Encourage the geo-enablement of appropriate government business processes to improve access to location-based data and services This Segment Architecture methodology (depicted in Figure 12) is defined by FSAM in the following five steps: 1. Determine Participants and Launch the Project: Project charter 2. Define the Segment Scope and Strategic Intent: Perfor- mance Architecture Model 3. Define Business and Information Requirements: Business and Information Architecture Models, Business Rules and high level Requirements (concept of operations) 4. Define the Conceptual Solution Architecture: The com- bined systems, services, and technology architectures that support the target performance, business, and data archi- tectures developed in the preceding process steps. 5. Author the Modernization Blueprint: The Sequencing and Transition Plans. The segment development steps are preceded by defining the strategic drivers (e.g., policy directives) and high level reference business architecture models and conforming to requirements defined by cross-cutting Segment Architectures. The FEA assumes that Government agencies that develop a Segment Architecture will use the Reference Architecture mod- els as a building block, taxonomy and checklist. The outcome of the FGDC initial cross-cutting segment architecture effort has been to develop a Profile (31), that is: . . . a tool for chief architects to determine how and where place-based approaches and associated geospatial resources fit into their enterprise architectures as they implement the FEA reference models. The document also lists standards and service components (e.g., web and location-based services) that should be sup- ported in Federal procurements across the agency to facilitate interoperability. 74

Chapter 3 Appendix C: Description of The Open Group Architecture Framework (TOGAF) Domain descriptions are performed in Phase B through D. In each of these Phases, the current and future architectures are described and validated with key stakeholders, and the model conforms to existing standards, policies and proce- dures. In addition, developers identify the “gap” between the current and future architectures. Additional contributions are made to the requirements and gap matrix at each phase. Gaps that may exist at the Business Architecture stage include (31, Part II, Phase B): • People gaps (e.g., cross-training requirements) • Process gaps (e.g., process inefficiencies) • Tools gaps (e.g., duplicate or missing tool functionality) • Information gaps • Measurement gaps • Financial gaps • Facilities gaps (buildings, office space, etc.) The ADM includes phases to move an organization towards transitional architectures. These are briefly described below. Phase E Opportunities and Solutions: This phase includes the exploration of funding implementation methods, alterna- tive analyses (at the four domain levels), an implementation and migration strategy and a detailed Implementation Plan. Phase F Migration Planning: This phase includes details related to prioritizing the projects that will form the detailed Implementation and Migration Plans. Phase G Implementation Governance: The objectives of Phase G are to: • Formulate recommendations for each implementation project. • Construct an Architecture Contract to govern the overall implementation and deployment process. • Perform appropriate governance functions while the sys- tem is being implemented and deployed. • Ensure conformance with the defined architecture by implementation projects and other projects. (10, Part II, Phase G) Phase H Architecture Change Management: This phase sup- ports the continual improvement and update of the architec- ture descriptions, policies and vision to meet the changing requirements of the enterprise. There are many resources available to contribute to the Architecture development. TOGAF calls this part of the Framework the Enterprise Continuum. The Enterprise Continuum is described as a “virtual repository” of methods, patterns and solutions that help build the Organization Architectures. The Enterprise Architecture is composed of a “conceptual” part called the Architecture Continuum (see Figure 13), and a product oriented part called the Solutions Continuum. 75 Figure 12. FSAM high-level overview. Source: FSAM.

The Architecture Continuum is a set of building blocks that may be used as a template to help build architectures. The Foundation and Common Systems Architectures are common to most organizations irrespective of market seg- ment. The Industry Architectures describe specific functions and processes of a vertical market or industry. The Retail industry’s “Active Store” architecture is an example of an Industry Architecture; the National ITS Architecture is an example of the ITS industry architecture (although it does not follow the TOGAF methodology). The Architecture Continuum also includes a Standard Information Base (SIB). The SIB is an inventory of standards that apply to the parts of the Architecture Continuum. The Solutions Continuum is a set of building blocks for the implementation of the architecture. It is composed of product and system solutions for IT, industry configurations and orga- nizational implementations. Finally, the Resource Base contains resources on gover- nance, boards, contracts, assessment and maturity models and examples of ADM products and required skill sets. 4 Findings on Transit IT/ITS Implementation Funding Prior to the 1980’s, pay-as-you-go was the primary way to fund transportation projects. State and local governments secured Federal transportation funding either through for- mula or discretionary grants. They would budget the non-fed- eral match with state and local funds. Local and state funds were derived from transportation fees or revenue from a vari- ety of broad-based taxes. The 1980’s marked an era of height- ened interest in private sector approaches applied to public transportation. Innovative financing became a dominant theme, where traditional financing approaches were used in new and creative ways. Public financing options with a variety of innovations arrived on the transportation scene in the 1990’s and have proliferated into the 21st Century. This chapter compiles information on current ways transit IT/ITS projects are funded. It summarizes literature obtained from a variety of sources (see references at the end of the chap- ter). Best practice is presented based on a survey conducted with a sample of transit agencies that are known for their progressive use of technology. Their application of technology occurs in electronic fare collection, passenger information, operation control centers, system surveillance, service scheduling, vehicle location, operator safety, substance detection and a variety of business management functions such as accounting, payroll, training, maintenance, purchasing and material storage. 4.1.1 General Findings from the Literature As we near the second decade of the 21st Century, financial engineering is alive and well in the transportation sector. Infra- structure banks exist at the Federal and State levels. Special pro- visions for financing technology can be found in Federal tax law. A variety of innovative financing mechanisms are being widely used. Several university business schools offer financial engi- neering programs to teach students principles and practices involved in structuring, analyzing and making decisions on eco- nomically efficient and effective financing approaches. Organi- zations like the National Association of State Chief Information Officers (NASCIO), Government Finance Officers Association (GFOA), Federal Transit Administration (FTA), Federal High- way Administration (FHWA), the Transportation Research Board (TRB) (through the FTA-funded Transit Cooperative Research Program and FHWA-funded Highway Cooperative Research Program) and the American Public Transportation Association (APTA) are documenting the state-of-the-practice and state-of-the-art in technology financing in general and pub- lic transportation technology financing in particular. 4.1.2 General Approach to Implementation Funding A summary of IT funding experience at the state level is identified in Table 4. As the table implies, IT investments are clearly becoming more competitive with other capital invest- 76 Figure 13. The architecture continuum. Source: TOGAF version 8.1.1.

ments. According to the NASCIO survey, few states were using user fees and grants to fund IT projects in 2003. However, by 2008, over 70% of the states participating in the survey indi- cated that they were employing user fees and grant funds to implement IT projects. More innovative and alternative fund- ing approaches appear in the same timeframe. Bond financing and different types of lease financings became more prevalent. In addition, a variety of public-private partnerships and public- public partnerships started commanding greater attention. An illustrative example of an emerging and innovative public-private partnership is the replacement of the IT sys- tems for the Commonwealth of Virginia Department of Taxation. In 2003, Virginia legislators amended the Public- Private Transportation Act to allow it to be applied to other government programs. The Department of Taxation (TAX) was the first area to make use of the amended authority. It ini- tiated and executed a partnership contract with CGI-AMS to design, implement and operate a new tax revenue collection system with online filing and payment capability. TAX used an enterprise architecture planning framework to develop the new tax collection system as shown in Figure 14. The private con- tractor raised $71 million to cover the cost of the new system. 77 Table 4. State funding of IT projects. Funding Approach Number of States Using Funding Approach, 2008, N=31 Number of States Using Funding Approach, 2003, N=23 User–Fee Revenue RN22 RN22Grant Funding Budgeting & Appropriations Strategies 19 18 Leveraged Financing 19 15 Outsourcing & Managed Services 19 16 Procurement Strategies 17 16 Notes: NR represents No Response. Leveraged financing includes leasing and various types of bond financing. Source: NASCIO’s 2008 and 2003 surveys of innovative funding for IT projects, 2008 Figure 14. Virginia TAX architecture planning framework. Source: Commonwealth of Virginia Department of Taxation, 2003.

In return, CGI-AMS received 3% of revenue collected from on-line returns over the life of the contract. Risks were shared. The contractor assumed systems integration and operating risks and Virginia assumed revenue risk. Absent the CGI- AMS annual payment, the Commonwealth receives an addi- tional $72 million of revenue. The Government Finance Officers Association (GFOA) recommends that finance officers achieve a full under- standing of the available options when determining if a public-private partnership agreement is a viable and pru- dent transaction for their jurisdiction. This includes devel- opment of an internal policy that defines the government’s criteria for making various contributions to or investments in “partnership” arrangements. Early in the process of ana- lyzing a proposed “partnership” transaction, the finance officer should also assess the nature and extent of any out- side consulting or financial analysis services that the govern- mental body requires for its analysis and negotiation of the transaction. As noted in the Recommended Practice (RP), The Role of the Finance Officer in Economic Development (32), finance officers are encouraged to participate in and provide essential information to the “partnership” process. This includes devel- oping the objectives for the partnership, analyzing financial aspects of proposed arrangements, making recommendations to elected officials, advising on procurement issues arising from the solicitation and engagement of non-governmental parties and participating in the negotiation of the develop- ment agreement. The finance officer must also determine the total value of the public contribution (participating jurisdic- tion and others) in the agreement, including non-cash items, to make sure that the public’s contributions to and invest- ments in the project are justified and properly compensated. The finance officer must also be mindful of any direct or indi- rect increased, ongoing public operating costs that may result from the project. The GFOA recommends that finance officers use the follow- ing list as a guide for preparing a comprehensive examination of issues that must be addressed before, during and after the project is determined to be viable and prudent. This list empha- sizes that a great deal of due diligence must be completed prior to entering into a contract, since these decisions may have sig- nificant and long-lasting ramifications. GFOA recommends taking the following actions when considering public-private partnerships: 1. Research private partners, their business and market; 2. Research the type of transactions being considered; 3. Consult with appropriate professionals about applicable federal and state tax laws; 4. Understand the rights and obligations of each party; 5. Set standards for public financial commitments; 6. Evaluate and disclose the financial and non-financial impacts of the proposals; and 7. Monitor the agreement. The finance and technology officers involved in a “partner- ship” should ensure full disclosure and make recommendations that the government’s participation in the venture does not bring excessive and unbalanced risk to the public. 4.2 Transit Capital Investment Needs and Funding Approaches 4.2.1 Transit Funding Needs Given the demand for transit funding, transit agencies are using all forms of funding approaches for state of good repair projects that maintain conditions and performance and for capacity enhancement and system expansion projects that improve conditions and performance. Like IT projects in gen- eral, transportation IT and ITS projects are delivered with public leveraging options such as bond and leasing financing, public-private partnerships, comingled funding and a variety of Federal, state and local funding sources. From the report, 2006 Status of the Nation’s Highways, Bridges, and Transit: Conditions and Performance Report to Congress, the replace- ment value of urban transit infrastructure in the United States was $402.7 billion in 2004 dollars (33). This is the cost estimate of replacing all of the transit assets using 2004 dollars. We know that price inflation for IT and ITS projects technology is advancing all the time. For example, it has been said that microprocessors, the heart of most systems, change nearly every 90 days. In order to maintain the conditions and performance of the Nation’s transit system, it is estimated to cost $9 billion per year, as shown in Figure 15. To enhance and expand capac- ity, an additional $12.8 billion is estimated for a total average annual capital investment of $21.8 billion through year 2024. Of that amount IT, ITS and other systems would require about $1.5 billion per year for maintaining condition and performance and nearly $2 billion per year to improve con- dition and performance. 4.2.2 Capital Funding Sources Figure 16 shows that transit agencies receive funding from a variety of sources. They not only receive technology funding from FTA but also from FHWA, state DOTS, local governments and other Federal agencies, such as the Depart- ment of Homeland Security (DHS). Transit agencies also make use of a variety of locally generated sources of funds. Since the 1990’s, Federal funding continues to account for 50% to slightly over 60% of transit funding. However, 78

79 Figure 15. Annual transit capital assets investment needs 2004–2024 ($ in 2004) (34). Source: Transit State of Good Repair: Beginning the Dialogue, October 2008, FTA. 0 2000 4000 6000 8000 10000 12000 14000 16000 Federal State Local Agency Total 19 95 19 96 19 97 19 98 19 99 20 00 20 01 20 02 20 03 20 04 20 05 20 06 Figure 16. Transit funding sources FY 1995–FY 2006. Source: Transit Statistics, Capital Funding, 2007, APTA. between 2001 and 2005, federal funding experienced a decline. During that period, growth in transit agency-gener- ated funds and state and local funding replaced the federal decline. Transit agency-generated funds led the way. The capital funding sources are defined more specifically as fol- lows: • Federal Funds are funding provided through a number of formula and discretionary programs. The Federal Transit Administration (FTA) distributes the bulk of the federal assistance through its Urbanized Area, Non-Urbanized Area and Fixed Guideway Formula Programs; Discre- tionary Bus and Bus Facilities, Major Capital Investment,

Planning and Research Programs; and Flexible Highway Funding. • Program and Homeland Security State and Local Gov- ernment Funding Programs. Under the Flexible Highway Funding Program, the Federal Highway Administration (FHWA) transfers funds to FTA from its Congestion Mit- igation and Air Quality Improvement (CMAQI), Surface Transportation (STP) and Interstate Highway Substitution Formula Programs and Research and Technology Discre- tionary Funds. IT/ITS projects are eligible under each of the FTA, FHWA and DHS formula programs and are funded through earmarks in the various FTA and FHWA discretionary programs. • State Government Funds are obtained from dedicated taxes, general funds, fuel taxes and toll revenue. This includes funding from bond proceeds and infrastructure banks. These state sources are typically used as a match for federal funding and for funding IT/ITS projects. From time to time, some states use specially appropriated funds to sup- port transportation initiatives like congestion manage- ment where ITS projects serve as primary tools. • Local Government Funds are derived from similar sources as the states and include sales and property taxes. Local funding sources are also used to match federal funding and can be used for IT/ITS projects. • Transit Agency Directly Generated Funds are revenues generated by or donated directly to the transit agency, includ- ing passenger fare revenues, advertising revenues and joint development revenue from leveraged assets, donations, self- imposed taxes, agency bond proceeds, and revenues from creative financing arrangements like cross-border leases and sales leaseback contracts. As observed in the project surveys, transit agencies are starting to make greater use of directly generated funds to pay for IT/ITS projects. We saw in Figure 15 above that transit agencies need to spend $21 billion per year to keep their systems in a state of good repair and to improve performance. In 2006, an $8.5 billion gap occurred. Consequently, competition for available fund- ing is extremely stiff. IT/ITS projects suffer in this competi- tive environment because of lingering misunderstanding of their benefits and higher priority capital projects like vehi- cle replacements, infrastructure rehabilitation and construc- tion of new lines. 4.2.3 Financing Mechanisms Transit agencies are using a variety of financing mechanisms to access the various sources of capital for IT/ITS projects. His- torically, buy (pay-as-you-go), borrow (issue bonds) or lease were the primary financing mechanisms that transit agencies used. Since the 1990’s, creative use of these traditional mecha- nisms and introduction of public-private partnerships have occurred. Today, financing mechanisms fall into four cate- gories: debt mechanisms, capital leasing financing, equity and partnerships and credit enhancements. More detailed defini- tions of these financing mechanisms are as follows (35): • Debt—These mechanisms are also known as leverage financ- ing and include long- and short-term issuances of bonds in the taxable and tax-exempt markets as well as direct loans from governmental and non-governmental sources. • Capital lease—Rather than purchasing an asset outright, the acquiring entity leases the asset over a number of years. While this is not always truly a mechanism to finance the acquisition of an asset, it most certainly is an alternative approach to gain use of the asset over a comparable period of time. Lease payments are made in lieu of payments of principal and interest. In many instances in which a lease- to-purchase arrangement is utilized, lease mechanisms do indeed result in full asset ownership. • Public-Private Partnerships (PPP)—These are contractual arrangements where an outside entity invests a certain amount of funds in a capital asset with the expectation of sharing in the profits of its operation or otherwise directly benefiting from its operation or is given access to agency owned assets in the context of more effective management of its assets. (36) In the transit arena, this can include IT, ITS, vehicle, joint development and infrastructure projects. • Credit enhancement—These mechanisms are designed to help manage financial risk and include bond insurance, let- ters and lines of credit, and governmental guarantees used not as stand-alone financing mechanisms but in support of the direct financing techniques. It is important to note that these financing options are not mutually exclusive and that the most innovative project delivery approaches tend to use them in combination. The TEA-21 authorized Trans- portation Infrastructure Finance and Innovation Act is an example of a federal credit enhancement program. • Pay-As-You-Go—This mechanism is used when a proj- ect’s schedule can be met with current sources of funds rather than by borrowing or leasing. This financing mech- anism is used more widely than the others for most transit capital investments in particular IT/ITS investments. • Comingling—Comingling of funds occurs when funding for one program is used in support of multi-program objec- tives. Transit agencies are starting to realize that enterprise approaches like comingling of funds, IT/ITS Architecture, and process management improvements are both efficient and effective ways to solve their financing, customer service, operations, and management problems. These mechanisms are very similar to those used by states to fund their IT projects, as found by the NASCIO surveys. 80

Technology in general has played an important role in these developments, as noted above. Lease financing and public- private partnerships involving technology projects received incentives from the tax code in the form of Qualified Tech- nological Equipment (QTE) depreciation allowances. Tran- sit technology in particular is helping to lead the way towards expanded and innovative use of these financing mechanisms. Table 5 summarizes how transit IT/ITS projects were financed by twelve transit agencies that participated in the project survey. Transit agencies are applying the full range of financing mechanisms to make IT/ITS investment from large enterprise technology replacement projects to small AVL projects. Pay- Go is the primary financing mechanism used by most transit agencies. However, comingling of funds and public-private partnerships (PPP) are starting to be used more. For exam- ple, in 1992 the Sacramento Regional Transit District comin- gled acquisition of 75 buses, a fare collection system and a radio system in a $32.4 million leveraged lease. Salt Lake City UTA comingled $12.3 million to acquire an account-based fare collection system and a performance reporting system. WMATA is pursuing a public-private partnership to finance, design, implement, operate, maintain and manage content of a streaming video advertisement and passenger information system called “The Metro Channel.” SEPTA is another tran- sit agency considering an ITS public private partnership, in their case to replace an antiquated fare collection system. Successful implementation of a public-private partnership requires state government legislation overriding low-bid pro- curement rules, as occurred in Virginia and demonstrated in projects participating in the FTA Turnkey Demonstration Program of 1993 and 2007, Public Private Partnership Pro- gram. (37) As of April 2007, 23 states and Puerto Rico have legislation in place that allows varying levels of private sector participation in several types of transportation projects. 4.2.4 Repayment Streams Payment streams include a mix of broad-based taxes, fare box revenue, non-fare box revenue from joint development, shared-use of assets and advertisements, and a mix of annual Federal formula and discretionary allocations. When debt and lease financing mechanisms are used, the repayment stream must be identified and reserved to meet the obliga- tions. The appropriateness of their use varies with their sta- bility and reliability. Economic and market conditions, polit- ical acceptability, federal regulations, revenue generation capacity and technology risks are some of the factors that would be used in deciding with which combination of these repayment streams to secure the financing of an IT/ITS proj- ect. For a more thorough treatment of the various sources of repayment see (38). 4.3 Key Findings on Transit Agencies Implementing IT/ITS Funding Transit agencies are using IT/ITS technologies as tools for improving management and operational functions across their enterprises. Decisions on technology investments are made within the context of the capital planning and pro- gramming process. Agencies are employing the whole range of funding sources, financing mechanisms and repayment streams. Comingling of funds is allowing transit agencies to more effectively leverage their IT/ITS investment. In the sur- veys, we saw where a performance reporting system was added to a radio system upgrade project or where broadband wireless communication was combined with an advanced passenger information system and a new electronic fare col- lection system. Pay-Go financing secured with federal, state, local and agency-generated sources of revenue remains the mostly widely used financing mechanism for all sizes of tran- sit agencies and for smaller technology projects that do not place extraordinary demands on capital. Bond financing is used by agencies with bond authority and large capital pro- grams that cannot be funded on a pay-as-you-go basis. With the inducements of the Government Accounting Standard Board’s rules, transit agencies are also seeking to optimize the value of their assets by leveraging them through public-private partnerships. Several cases were particularly illustrative of the trends in implementation funding of IT/ITS projects. 4.3.1 Salt Lake City Utah Transit Authority (UTA) UTA recently completed implementation of an upgrade radio system for $6 million, a reliability reporting system for $300,000 and an electronic fare collection system (EFC) for $12 million. The EFC System is an account-based bankcard system with 1400 bus validators and 30 rail station validators. 81 Table 5. Transit agency funding of IT/ITS projects. Funding Approach Number of Agencies Using Funding Approach, N=12 5Debt Financing 2 3 Capital Lease Financing Public-Private Partnerships 2Credit Enhancement Pay-Go 12 Co-mingling 12 Source: TCRP 09 Enterprise Architecture Planning, Funding Implementation Survey, January, 2009

The Reliability Reporting System was integrated with an existing AVL System and leveraged an extra $2 million dol- lars invested in the EFC System. These projects were funded with revenues from a dedicated UTA sales tax of 6.45 percent. The IT/ITS Program is allocated $5 million a year in sales tax revenue in accordance with the 30-year long-range plan. These funds are typically used on a pay-as-you-go basis but are also occasionally allocated from bond proceeds. 4.3.2 Washington Metropolitan Area Transit Authority (WMATA) In 2006, WMATA, through the Department of Planning and Joint Development, issued a request for expressions of interest for a public-private partnership to enhance the communication system; develop and implement a real time streaming video advertisement, passenger information and security alert system; and to upgrade the operations control center with a much-expanded customer information com- ponent. The system concept was called the Integrated Cus- tomer Communication System (ICCS). The ICCS concept of design and operations estimated funding being derived from a four-fold increase in revenue from dynamic advertise- ments. With 45 private sector teams submitting expressions of interest, WMATA initiated a procurement for a public- private partnership as an option to finance, design, build, operate and manage content for what is now called “The Metro Channel” (TMC). The TMC procurement does not include the operations control center upgrade or the com- munication system overhaul. A Neutral Host Communica- tion System was procured in a separate contract. WMATA is planning to implement the first TMC pilots in 10 stations by 2010 and to complete the entire rail and bus system in four to five years. 4.3.3 Metropolitan Atlanta Rapid Transportation Authority (MARTA) MARTA, like WMATA and many other transit agencies, has recognized that improving customer experience is a high return goal. MARTA decided to use a public-private partner- ship as a strategy for achieving this goal. MARTA entered into two 10-year public-private partnership contracts to install, operate and manage content of a passenger video dis- play system for its buses and rail cars. The passenger display system provides real time information on transit services, advertisements, news and weather. The bus agreement esti- mated the generation of $10 million per year, and the rail agreement estimated $20 million. More recently, MARTA implemented an $85 million Enterprise Resources Program of business systems with bond proceeds secured by its one- cent sales tax. 4.4 Constraints on Funding Approaches Funding approaches for transit IT/ITS projects are influ- enced by different eligibility requirements, market conditions and risk profiles. Federal funding requirements include archi- tecture planning, systems engineering and purchasing ITS based on full and open competitive procurements. In the sur- vey, the architecture planning and systems engineering require- ments were not replicated as conditions for state and local gov- ernment funding of transit IT/ITS projects. However, debt service caps, sunset provisions and credit worthiness were iden- tified as local and state government constraints driven by the marketplace. With rapid changes occurring in technology and in the marketplace, risk surfaced as a constraining factor in financing transit technology projects. Evidence of these con- straints is described below. 4.4.1 Eligibility Requirements In late 1989, the USDOT initiated development of a National ITS Architecture as a framework for integrating the flow of information between different systems (ITS). Figure 17 presents an overview of the current version of the National ITS Architecture. ITS technologies were categorized into four subsystem classes: Travelers, Centers, Vehicles and Field infra- structure systems. In sponsoring the National Architecture, the USDOT wanted to promote cost-effective technology decision-making, to facilitate seamless transportation across and between transportation modes and to encourage smart procurement of new technologies when using federal fund- ing. In essence, the federal government wanted its funds to be used to buy open, integrated and interoperable systems rather than customized, closed, proprietary systems. The Federal Transit Administration (FTA) and the Federal Highway Administration (FHWA) conducted town hall meet- ings in metropolitan areas around the nation. The feedback confirmed the technical need for architectures based on actual regional ITS systems that would use the single National ITS Architecture as a template. The National ITS Architecture was codified in the last three Transportation Authorization Bills: ISTEA, TEA-21 and SAFETEA-LU. Based on the legal require- ments, the FHWA responded by preparing a rule on ITS Architecture and Standards (Rule 940). Because of the differ- ent way that FTA operates they created a nearly identical pol- icy for implementing the authorization requirements. The smartcard experience provides a good illustration of the pitfalls of not using an enterprise architecture approach. FTA funded deployment of the first smartcard system at the Washington Metropolitan Area Transit Authority (WMATA) in 1996. The WMATA system was initially called the Go Card system, owned exclusively by Cubic Transportation, Inc. (Cubic). It evolved into what is now called the WMATA smart- card system. (39) 82

Transit smart card systems involve both proprietary soft- ware and hardware. As demonstrated at WMATA and other transit agencies, such closed systems are the antithesis of the National Architecture. These smart card projects have expe- rienced many of the problems that the National Architecture was designed to avoid, including lack of internal agency and regional integration, limited equipment and interagency interoperability, cost overruns, schedule overruns, difficulties changing fare policy and ineffective and unresponsive card management. While the FTA promoted systems integration in its action, the National ITS Architecture had from its outset limitations in its treatment of subway systems in general and transit fare collection in particular. Subway systems were not directly addressed in the National Architecture (although they were implicitly covered through several interfaces in the architec- ture). Fare collection was treated primarily from the viewpoint of the transit management subsystem to transit vehicle subsys- tem (buses and light rail vehicles only) interface, rather than a field or infrastructure system like highway toll collection. This limitation goes back to the very origins of the ITS Ini- tiative. The initiative itself was originally called the Intelligent Vehicle Highway Systems Initiative. It was changed to ITS at the urging of Gordon Linton, FTA Administrator from 1992–1998. Subway interests saw their systems as too vital and mission critical to be part of the National ITS Architecture and overall Initiative. Today, however, automatic fare collection systems for subway systems are treated as ITS and must com- ply with the same integration policies and regulations as sur- face systems like transit buses, light rail transit and highways. The Federal Highway Administration (FHWA) led the high- way industry towards an account-based, multi-state electronic toll collection system as illustrated by the multi-state I-95 EZ Pass. Also, in selecting the account-based approach, highway interests were able to take advantage of data standards and communication protocols developed, tested and certified out- side the transportation sector by the banking interest. Today, transit agencies around the world are starting to take notice of the significant benefits afforded by bankcard fare collection systems. Account-based fare collection systems are based on an open architecture, international standards, and certifications approved by the banking industry. They do not require the time, cost and funding needed for customized transit smart card standards. 4.4.2 Marketability The market for ITS continues to grow in the U.S., as shown in Figure 18. Since 1997, all ITS services experienced signifi- cant growth, with emergency management, transit manage- ment and electronic payment systems leading the way. Growth in transit management systems was fueled by larger increases in federal transportation funding in TEA-21, a large number of ITS projects in federal appropriations, and transit managers having a better appreciation of ITS benefits to transit opera- tions and management. 83 Figure 17. National ITS architecture. (Source: National ITS Architecture [29], “Physical Entities” Page.)

4.4.3 Risks and Uncertainties Risk is inherent in every investment we make. Consequently, it is extremely important that risk be considered carefully in conducting financial planning for technology projects. Ide- ally, a risk assessment would determine the likelihood that a technology will successfully integrate in accordance with the Enterprise Architecture, cost what the engineering estimate produced, be implemented in accordance with the schedule, be insulated from the vagaries of the economy and market- place, not experience support problems when things go awry and not end up in court over disputes with the vendor. Unfortunately, technology projects, like most other invest- ments, always bring a variety of risks that could adversely affect a project’s scope, schedule and budget. The different types of risks are: • Technology risk—The probability that the technology will not meet the technical requirements of the client and may not be flexible for future changes desired by a client. • Economic risk—The extent to which pledged revenues may not provide an adequate income stream to amortize debt, pay operating expenses and provide an adequate return to investors. • Completion risk—As a counterpart to the risk of revenue streams being insufficient, there are the risks that construc- tion costs will exceed initial cost and schedule estimates. • Legal risks—The potential to violate Federal and state statu- tory provisions relating to construction and operation of the system and relating to the taxable and/or tax-exempt financ- ing being applied. • Management risk—The ability of the private contractor and the sponsoring agency to successfully manage the project. • Financial risk—This refers to the probability that a debtor (the issuer of the bonds) is unable or unwilling to make timely payments of interest and principal (also known as default risk) and is addressed by the rating agencies in their assignment of bond ratings. • Interest rate risk—The probability that a bond will lose value because of a general rise in the level of interest rates (if interest rates rise, the value of a specific stream of bond payments falls; alternatively, if interest rates fall, there is a gain in value). 84 Figure 18. Market for ITS deployments.

• Reinvestment risk—Measuring the probability that an investor may buy a bond that yields a certain return (e.g., 10 percent) but may not actually get that return. • Liquidity risk—Capturing the possibility that a bond may not be quickly turned into cash at its fair market value. The level of risk increases with the complexity of the tech- nology project. As an illustration, a self-contained bus AVL is not as complicated as installation of a paratransit AVL and scheduling system. The latter takes more time, requires more systems integration work and costs more. Similarly, the Salt Lake City upgraded radio system is far less risky than imple- mentation of WMATA’s ICCS. An ICCS could hardly be implemented without an Enterprise Architecture, extensive coordination across all of the functions of a transit agency, a more advanced communication system, a more progressive operations control center and an enterprise database. Although these risks can be quite overwhelming, it is critical that they be identified, measured, allocated and mitigated. 5 Findings on Business Case Methodology in Transit Synthesis Objective of Business Case Methodology (BCM) Synthesis A goal of this Synthesis is to provide information that can improve the success of justifying and funding IT/ITS projects. The Synthesis will document the state of the practice in the transit industry with respect to the use of Business Case Methodologies to justify the development or procurement of Information Technology (IT) and Intelligent Transportation System (ITS) systems. The Synthesis will provide high level guidance on how to implement a practical and effective BCM. In addition, potential linkages between the BCM analysis and other project stages in the Transit Enterprise Architecture and Planning Framework will be highlighted. 5.1 What is a “Business Case”? A business case states, as of a certain point in time, the rea- sons for initiating a project. The IT Governance Institute says in its guidance document titled, “Enterprise Value: Governance of IT Investments: the Business Case,” (40) that the business case must provide the answers to the four main questions of: • Are we doing the right thing? (What is proposed? What are the intended business outcomes?) • Are we doing it the right way? (How will it be done? What is being done to have it fit with other current and future capabilities?) • Are we getting it done well? (What is the plan for doing the work? What resources and funds are needed?) • Are we getting the benefits? (How will the benefits be deliv- ered? What is the value of the project?) Formal business cases are evaluated to ensure that (41): • “The investment has value and importance • The project will be properly managed • The firm has the capability to deliver the benefits • The firm’s dedicated resources are working on the highest value opportunities • Projects with inter-dependencies are undertaken in the optimum sequence.” A business case is not a static document. It is a collection of assumptions, analyses and predictions of what will occur with a project at that given point in time. As the project moves through phases, additional information becomes available, assumptions get confirmed or disproven, and estimates can be updated. The business case should be updated at agreed upon project steps or phases. 5.2 What is a “Business Case Methodology”? A business case methodology is a formal analysis process used to develop a business case. It is used to justify and cap- ture the reasoning for initiating a project. For the purposes of this synthesis document, a BCM must have standardized ele- ments and be a documented process, even if it is a simple one. 5.3 Why is having a Business Case Methodology Valuable? A business case and a business case methodology are valu- able to the success of an IT/ITS project and the ability of an organization to demonstrate the benefits of a project. A busi- ness case can help to: • Determine if a project is financially viable before starting and running into trouble – What is the project likely to cost and is the funding and budget needed to see it to completion available? – Will you get the financial benefits that may be needed? • Helps decision makers understand and prioritize invest- ment options. • Demonstrate that the “thinking” and preliminary planning was done • Helps the various stakeholders understand and agree upon the project elements. Building stakeholder consensus in advance helps decrease project issues, delays, and cost increases during the implementation phase. • Drive a project to achieve its stated outcomes and perfor- mance measures. Specifying anticipated outcomes at the 85

beginning of a project helps assure that the project stays true to the initial purpose and priorities. The Information Systems Board (42) of Washington State believes that “[w]hat gets measured gets managed.” • Defining the desired outcomes and acceptance criteria at the beginning of the project also clarifies the project’s scope and provides a starting point for post-implementation demon- stration of the achieved benefits and for developing “lessons learned.” The most important role of the business case is to help the project obtain approval and funding. A funding organization, whether it is the Federal Transit Administration (FTA) or the transit agency’s budget office, prefers to fund projects with a good business case that makes the decision-makers feel com- fortable that the dollars will be well spent. Further, a good business case helps create a project that is more likely to be finished on-time, within budget, and meeting scope require- ments. By establishing credibility for successfully completing projects with demonstrable benefits, it increases the odds for obtaining funding on future projects. 5.4 Differences Among Business Case Methodologies Business case methodologies can be purchased, borrowed from an organization willing to share its methodology, cus- tomized from another methodology or developed from scratch. Some of the commercial processes vary in how easily their process steps and templates can be customized. BCMs vary in the number of analyses proposed as options and in the number of analyses that are required. The next subsection of this Synthesis chapter will include a list of some of the analyses and assumption areas that might be in a busi- ness case. In addition, the guidance and expectations for the level of detail in the analyses can vary greatly. Different BCMs grant different degrees of leeway to the developers of the business case in deciding how much to incorporate in the business case. For example, not all the BCMs include a feasibility analysis as a possible component, although it should be a consideration for some of the new ITS technologies. An example of a business case study conducted on an emerging technology, “Assessing the Business Case for Inte- grated Collision Avoidance Systems on Transit Buses,” (43) included steps that may not be required when implementing a more mature ITS related technology. The study conducted a technology evaluation, cost-benefit analysis, and “. . . assessed the receptiveness among transit operators to use IVBSS prod- ucts and the willingness of manufacturers to develop them.” Assessments of the technology as well as vendor, consultant and internal staff interests and capabilities are important ele- ments of the feasibility portion of a BCM when working with cutting edge technologies. Some agencies implement BCMs that focus on building a business case that may be approved in one step and then the project goes into the detail requirements, design and develop- ment phases. Other BCMs follow a more “gated” process, with a series of approvals and revisions to the business case docu- ments. For example, if a feasibility study provides favorable results, additional funding and resources may be provided to do high-level requirements development, alternatives analyses and complete other required aspects of a business case. 5.5 Examples of Possible Topic Areas in a Business Case The most common topic areas in a non-complex business case include the following: • Project Description • Statement of the problem to be fixed or business need to be met • Alternatives considered • Project costs • Anticipated benefits • Assessment of the costs and benefits • Risks and critical success factors • Recommended project approach • Anticipated outcomes that can be measured post- implementation Listed below are some additional areas that can be included in a business case. The list is far from exhaustive and there is some overlap in a few of the categories. Few business case methodologies were described in fewer than 20 pages. In later sections of this report, some comprehensive BCMs are identi- fied and referenced. • Executive Summary • Project Background • Project Description • Project sponsor, stakeholders and core team • Linkage to agency goals and objectives • Environmental Analysis • Assumptions, constraints and conditions, including critical success factors • Alternatives • Business and Operational Impacts • Technology Assessment • Project Risk Assessment • Anticipated funding approach • Lifecycle cost analysis • Cost/Benefit Analysis 86

• Return on Investment (ROI) • Project Schedule and analysis • Verification • Conclusions and Recommendations • Implementation Strategy • Review and Approval Process 5.6 Transit BCM Survey Results The screening survey included questions on the organiza- tion’s use of a business case methodology, verified terminol- ogy and asked about the use of a Return on Investment (ROI) analysis and other cost related analyses used in justifying an IT/ITS project. Additional follow-up questions were asked of a subset of respondents. Does your organization have a process for proposing, justifying and approving an IT or ITS investment (a business case methodology)? Approximately half the organizations had some sort of process, whether it was IT/ITS-specific or the general agency budget approval process, for proposing, justifying and approv- ing IT/ITS investments. Only a few of the agencies had an IT/ITS-specific process that provided templates and guidance for staff that needed to initiate and justify a project. Some respondents said their organization used consultants to build the justification for a project. Another said, “Nobody in our organization formally requires a BCM process, we have stan- dard budget justification forms, but no official BCM document or process. However, we end up doing some of a BCM’s steps to justify the project to the management and Board as part of the budget process, and because it’s helpful.” TriMet TriMet felt that the BCM should be simple, clear, flexible and understood by all the stakeholders. Flexibility was impor- tant, so the business case could be scaled based on the size and complexity of the project to ensure it would be used for all projects and not be skipped because of an onerous process. Basic templates are available for the Project Charter, the Plan- ning Report (which is shown in Appendix A), Alternatives Analysis and other aspects of justifying the project. They stated that the analysis should consider all the system life cycle stages, including feasibility, design, development, implementation, operation and maintenance and the end of the life cycle when the system is terminated or replaced. Further, TriMet has a Project Sponsor for each project, with “. . . responsibility for approving budget, schedule and scope changes, deciding the issues to be presented to other stakeholders and for accepting the final work product. The sponsor is typically the most senior person from the business unit needing the work who will stay informed of and involved in the project.” In their BCM, the Project Sponsor has a Quick Reference document with checklists to help them in their role of facilitating the project’s success. Examples of some of the Project Sponsor’s checklists, which help them do their job, are included in Table 6. WMATA WMATA is working on the development of an Enterprise Architecture and also has a project management methodology that it uses. As a result, their BCM includes a reference to the Enterprise Architecture. The project management methodol- ogy includes a Business Plan Initiation (BPI) process, although the process does not always require a justification for all proj- ects. The BPI feeds into the capital planning framework for all projects. Appendix B includes a streamlined form for the Business Plan Initiation Review process, plus instructions for completing the form. The form summarizes all the project justification documents. King County Metro (KCM) Over the last 15 years, King County Metro has used a couple of different processes for developing a business case. Currently, KCM must use King County government’s process for justify- ing and approving IT/ITS projects. The process is described in a 69-page document titled, “Project Manager Guide to PRB Reviews,” (44) which also references other documents for additional guidance. Appendix D contains two tables from the Guide which show the suggested deliverables for Phase I, called Project Planning and for Phase II, called Project Development, which in King County’s process includes the “business case.” The Project Planning phase is typically completed as a prelim- inary request for funding to further build the business case in Phase II. King County employs a gated process, with funding released by project phase. Does your organization use the term Business Case Methodology? Only one respondent said that the term Business Case Methodology was used in his or her organization. A few respon- dents wanted to know what the term meant before answering the question. Terms that were used for their agency’s process for approving IT/ITS projects included Business Case, QBC or Quantified Business Case, Phased Gate Review, and BPI or Business Plan Initiation. In a Phased Gate Review process, a management review event occurs between specified project phases to determine if the project should proceed “through the gate” to the next phase. Does your Business Case Methodology vary by type of system or IT/ITS project? If so, how? Of those agencies with a BCM, all allowed for lesser detail in describing the business case, depending on the size and 87

perceived risk of the project. Some skipped steps when they knew the project was required. Others were acutely aware of the costs of doing the analyses and wanted to keep the level of effort commensurate with the estimated project costs, com- plexity and risks. King County provided the only form for determining the level of oversight a project might require, which drives the num- ber and detail level of the forms to be submitted. The four cat- egories of factors used to determine project risk rating are Proj- ect size, Project manager experience, Team experience and Project type. The Project Size factor rates the project on size, pri- marily based upon onetime cost estimates and, secondarily, on project duration. The Project Type factor rates the technical complexity of the work being undertaken. An example of the Project Oversight Rating form is included in Appendix B. If yes, Does the BCM consider: (Operations and Maintenance costs and requirements, Agency architecture, Regional ITS architecture, Integration options, other enterprise-wide thinking)? All the BCMs took into consideration operations and maintenance costs. The BCMs also considered one or more aspects of the agency architecture and/or the regional ITS architecture. One of the King County BCM forms had a checklist of technical outcomes which included, “Leverages and/or extends integration architecture.” WMATA’s Busi- ness Plan Initiation form includes “Implement Authority- wide Integration” as an IT priority. In your organization, what have been the benefits and issues pertaining to completing a Business Case Methodology? TriMet felt that the BCM helps with ensuring a common understanding of the project and helps manage expectations. High-level documentation of the project from the BCM and project management process is available for stakeholders to access (they have it in a database). Standardization of the steps helped simplify training on the process, helped readers quickly find information, and helped somewhat with comparisons between projects. At MARTA, the head of IT said, “You are relating what you want to do to the business needs, costs, and impacts. You show why the project should be done, not just providing an opinion or gut feel.” 88 Sponsor Checklist Examples from TriMet Project Initiation Phase Is this project aligned with TriMet’s organizational goals? Is this project a priority for TriMet resources (within an agency-wide context) at this time? Have all stakeholders (especially those in other divisions) been identified? Does the project description describe the problem, need or opportunity … and not the solution? Should this project proceed to the Planning Phase? Immediately or at a future date (given resources and other priorities)? Planning Phase Have the business need, scope of the project, and expected outcomes from the project been verified by all stakeholders? Has cross-divisional input been obtained in identifying alternative solutions? Will this project comply with national, regional and agency standards … or will an exception need to be made? Have options been evaluated adequately? Should this project proceed to the Design Phase? Immediately or at a future date (given resources and other priorities)? Design Phase Have you reviewed and approved a project plan (work breakdown structure) for the project? Have those who will operate and maintain the resulting system participated in defining design requirements? Have ongoing operations and maintenance costs been estimated? Have impacts on users and business process/organizational changes been identified and change management planning begun? Does this project raise new safety and/or security issues? Have you communicated project plans, status and implications to all stakeholders? Should this project proceed to the Implementation Phase? Immediately or at a future date (given resources and other priorities)? Table 6. Project sponsor checklist example.

Issues pertaining to the BCM included finding the time and resources to do the analyses. Finding the data to do the ROI was also cited as an issue. A concern was stated that some- times, for some projects, the process can take so long that the user requirements and technology options change before the project is started. Does your organization usually perform a Return on Investment (ROI) analysis as part of the IT/ITS project justification process? A majority of the respondents said their agency had con- ducted an ROI analysis on one or more IT/ITS projects. More than one respondent was unclear on the difference between a cost-benefit analysis and an ROI analysis. “ROI analyses” were conducted on key projects at some agencies that did not have a BCM. Conversely, the existence of a BCM at an organization did not mean that an ROI analysis was always completed on a project, although some level of cost analysis was always done. Other cost related analyses completed when a new project is being justified. Many of the agencies completed some form of a cost-benefit analysis. For a subset, Total Cost of Ownership was calculated. King County has a process for completing a “Quantified Busi- ness Case.” Another said, “they consider if the overall cost exceeds the benefits.” Does your agency have a formal process for comparing and selecting among different proposed IT/ITS projects? If a respondent said their organization did not have a BCM, they were not asked this question. Mostly the answer to this question was “no,” although several said that having a standard form for proposing projects helped with the comparison process. TriMet said they had a three-category classification of projects, which are Mandatory, Highly Recommended and Discretionary. Others said that their organization had tried dif- ferent approaches but did not currently have a repeatable process in place. MARTA is pleased that the selection of projects is done through the IT Governance committees, which include tran- sit management. At their agency, Users prioritize all the IT projects. This relatively new process “ended the old user com- plaints about IT pushing them.” 5.7 Other Approaches to a BCM A search of the literature shows a diverse set of business case methodologies available for purchase or freely available on the websites of public organizations. Among the various BCMs, terminology varies somewhat, as do the number of steps, plus the line may shift that defines the starting and stopping points of project steps or phases. A number of the most accessible of the free BCMs are described below. Free templates and explanations are available from the New South Wales Government Chief Information Office. (45) The Business Case Template (46) includes a useful “3- Point Cost Estimate Table, incorporating the following three cost estimates, Most Likely Cost Alternative, Best Case Sce- nario (Minimum cost of alternative), and Worst Case Sce- nario (Maximum cost of alternative)” (Figure 19). On the South Carolina Division of the State Chief Infor- mation Officer website (47) there is a 2007 guide for building a business case methodology, titled Business Case Methodol- ogy Template. The report covers the following topics: 89 • When to Use this Template • Required Business Case Elements • Cover Page • Executive Summary • Project Background • Project Description • Environmental Analysis • Alternatives • Business and Operational Impacts • Technology Assessment • Project Risk Assessment • Cost/Benefit Analysis • Project Schedule • Verification • Conclusions and Recommendations • Implementation Strategy • Review and Approval Process The September 2006 guidebook titled Building a Business Case for Shared Geospatial Data and Services: A Practitioners Guide to Financial and Strategic Analysis for a Multi-participant Program (48), contains clear explanations and examples that are also useful for transit. Further, all transit systems deal with geographic data, so there are some secondary learning benefits. On its website (49), the state of Texas includes an easy to understand Texas Project Delivery Framework, which has hyperlinks to Instructions, Templates and an Excel Work- book for building a business case. The Excel Workbook (50) has many potential cost categories detailed. The Workbook also includes a useful Evaluation Factors Spreadsheet that allows the project to be rated on a wide range of categories, such as statutory fulfillment, strategic alignment, agency impact analysis, financial analysis, initial risk consideration and alternatives analysis. In addition, page 24 of the Business Case Instructions (51) contains rating categories so projects can be compared.

Another easy to navigate website that provides hyperlinks to guidance on the various parts of Business Case Analysis Report is the Federal Aviation Administration’s website at: http://fasteditapp.faa.gov/ams/do_action?do_action=List- TOC&contentUID=7. Figure 20 shows the initial web page for the Business Case Analysis Report. Other useful guidance on proposing and managing IT projects is included on the FAA web pages. The FAA also has links to documents on the “Investment Analysis Standard Guidance” webpage, (52) on the following topics: • Cost Basis of Estimate • Cost Estimation Methodology • Benefit Basis of Estimate • Benefit Analysis Methodology • Risk Analysis Basis of Estimate • Risk Metrics for Initial Investment Analysis • Risk Analysis Methodology for Initial Investment Decision • Risk Analysis Methodology for Final Investment Decision • Work Breakdown Structure The IT Governance Institute’s website at www.itgi.org pro- vides valuable information about building a business case. It is particularly valuable because it puts BCM in the context of IT governance and Control Objectives for Information and related Technology (CobiT), which is a set of best practices (framework) for IT management created by the Information Systems Audit and Control Association (ISACA), and the IT Governance Institute (ITGI). The website also includes a link to the informative document titled, “Enterprise Value: Gov- ernance of IT Investments: the Business Case.” Several useful documents address some of the issues that distinguish calculating an ROI for a public sector project, rather than for a “for profit” organization. A StateTech arti- cle, (53) titled Evaluating IT Investments, discusses the con- cept of a Public Return on Investment (PROI). Two articles from the Center for Technology in Government at the Uni- versity at Albany are also very helpful. The first article is titled, Public ROI: Advancing Return on Investment Analysis for Gov- ernment IT: A Public Value Framework, (54) which discusses, among other things, what kinds of public value are produced. They called it “a public value framework to emphasize the point of view of the public, not the government, as the basis for the assessment.” The second article is titled, Return on Investment In Information Technology: A Guide for Managers (55), which discusses business case and ROI development approaches and issues, including potential risk factors in the four categories: political, organizational, technology, and business process. Another document relevant to calculating ROI for a type of ITS project is the FTA sponsored report titled, Real-time Bus Arrival Information Systems Return-on-Investment Study (56), which discusses a methodology for determining the return on investment of real-time information systems for bus services and explores cost-benefit analysis issues. In the 90 Figure 19. Business case template. (Source: New South Wales Government [46].)

field of Geographic Information Systems (GIS), which sup- ports ITS, the 2006 guidebook titled, Building a Business Case for Shared Geospatial Data and Services: A Practitioners Guide to Financial and Strategic Analysis for a Multi-participant Pro- gram (57), discusses ROI and benefits from both an agency and regional perspective. The benefits, guidance and approaches are useful for other areas of ITS as well. 5.8 Recommended Practices for BCM A preliminary list of recommended practices pertaining to creating a BCM and developing a business case is included below. The list of best practices and recommendations will be expanded in Task 4, Transit Enterprise Architecture and Plan- ning Framework Guidance. Who Should “Own” the Business Case Methodology? The methodology, including guidelines and templates for conducting a business case analysis, is typically the responsibil- ity of an agency’s Project Management Office (PMO). If such an office does not exist, the BCM is either the responsibility of the IT Steering Committee, or, failing to have that committee, the Chief Information Officer. In some organizations, where the IT department has not had the staff or resources, to fully develop policies and procedures, the finance and budget office may impose a more generic BCM for major projects that are requesting funding. The transit executive management team should understand and own the BCM because it plays a critical role in investment decision making and how their proposed projects will be under- stood and judged. The standardized process and templates help decision-makers understand a project, its value and risks, how it impacts other parts or the organization, and how it might compare to other investment options. Ideally, the management team should review the process and ensure that it is unbiased and contains the information needed by the IT department, the transit business areas, finance and budget and other key stake- holders. Further, the transit management team should review and guide policy and practices concerning how flexible the BCM should be. For example, should the BCM be modified to have a simpler form for less expensive and risky projects? Selecting and Tailoring a BCM The BCM for the agency is usually selected or developed under the guidance of the Project Management Office or the IT Steering Committee. A number of methodologies, with their associated templates, are commercially available for purchase. They vary in how flexible they are and how easily they can be modified. In addition, there are quite a few processes available from the public sector that can be adopted as a starting framework and then tailored to the needs of a specific organization. By reviewing some of the BCMs listed in this report, agencies can select and customize 91 Figure 20. Business case analysis report. (Source: FAA.)

components that will best serve their organization and business culture. In building the business case methodology, use business case best practices and standardize the process, templates and tables of content for the documents. Provide examples. Train staff that quality business cases are living documents that should be updated in the successive project phases as new information becomes available and assumptions become obsolete. Create a collaborative environment to develop the standards. If stakeholders aren’t committed to the process, the quality and timeliness of the business case will suffer. If a Project Management Office is running the BCM, it needs to own the process, not the content. The business area manager needs to stay invested in the development of the business case content. Who Should “Own” the Business Case? Building a good business case requires the involvement of a wide range of stakeholders, such as future users of the sys- tem; transit managers; finance and budget staff; a range of IT staff with expertise in networks, infrastructure, data- bases, software development, operations and maintenance; and possibly regional stakeholders. The development of the business case is the responsibility of the assigned project manager, however, the business area manager for the pro- posed project (e.g., Manager of Operations, Manager of Customer Services, etc.) and the IT Manager may be jointly accountable for the validity of the assumptions and project approach. Transit agencies may assign responsibility and accountabil- ity differently, depending on the resources and abilities with- in the organization. However, the development of a RACI matrix, which assigns who will be Responsible, Accountable, Consulted and Informed to the key tasks in the process, will significantly increase understanding and effectiveness of the process. 5.9 Business Case and an Enterprise Architecture How well a technology investment aligns with the enter- prise’s overall business needs depends, in part, on how well the organization understands the impacts, linkages and oppor- tunities with respect to the businesses, performance goals, information, services and technologies of the organization. An Enterprise Architecture documents the linkages and enterprise architecture planning determines how to move from the current environment to a future one. The availabil- ity of Enterprise Architecture facilitates finding the answers to many of the business case questions as they relate to peo- ple, process and technology. 5.10 Realizing Benefits In an InformationWeek article, “Rules To Live By: Benefit Realization—Improving the Yield on IT” (58), the author stressed the “. . . importance of effective benefit realization practices. They can go a long way toward driving improved yields of IT investments. The bottom line, benefit realization practices are not cookie-cutter. Rather, they are a specific set of processes, methodology, a toolkit . . . Successful ones are very context-driven and take into account the organization’s culture and management style. Don’t wait to have everything thought through to perfection, it won’t ever be. And most important, make such capabilities part of the organizational DNA. . . .” The article also recommends that an organization be selec- tive with the benefit realization metrics. It suggests that: • An analysis should focus on a few, carefully selected per- formance measures rather than lose focus on the priorities and overwhelming staff with too many measures. • “Do not use the same level of measurement and process rigor on all initiatives, as you’ll risk killing the effort with bureaucracy.” • Use metrics that are business-relevant and matter to key stakeholders. 5.11 Have Buy-in and Get the Sign-offs A business case consists of a set of assumptions and predic- tions of what is likely to occur, pertaining to system function- ality, costs, schedule, risks and other factors. Having the appropriate stakeholders sign off on their portions of the busi- ness case builds awareness of the project details, builds “buy- in,” signifies commitment, and documents decision making, assumptions and agreements. Chapter 5 Appendix A: Planning Report Template from TriMet Describes the end product and/or outcomes of the proj- ect, the constraints within which the project must be imple- mented, and options for proceeding. Does not include ele- ments of design or descriptions of how the project outcomes will be accomplished. 1. Scope This is a high level project description, list of stakeholders (users and others), sponsors, team members, and level of effort (estimated total time and/or cost). Scope may differ from Proj- ect Charter after discussing these items with all stakeholders. Include the project goal statement. 92

2. High Level Functional Requirements A brief description of the current situation and why the new project is necessary. Technical, cultural and or business rea- sons may drive the change. Example: Our current MMIS sys- tem is over 20 years old, is written in Cobol, and resides on the Mainframe. The mainframe is being retired, users demand a GUI interface, and current IT skills sets have evolved past Cobol. Additionally, the MMIS Repair Ticket Processing can be made more efficient with a new system. This may be a paragraph, or refer to a Requirements Document for larger projects. 3. Assessment of Environmental Requirements A description of the constraints within which the project must operate, e.g. technical standards or architectural guidelines, policy/legal constraints, cultural issues, security requirements, required system interfaces, etc. If an exception to standards may be necessary, include explanation and/or justification. This may be a paragraph, or refer to a Requirements Document for larger projects. 4. Operation Scenarios and Summary of Impacts This section will detail the various business and IT opera- tion scenarios . . . may include both the “as is” and “to be.” It will describe how information or data flows through the system, who is involved in the management of the data and system and at what points. This will include both users and maintainers of the system. If additional IT or business staff are required or reduced, they will be identified in this section. Additionally, this section will identify the life cycle of hardware, required software/hardware licenses, and warranties. This may be a paragraph, or refer to a Business Use Cases document for larger projects. 5. Analysis of Options A description of the expected benefits and potential risks of alternatives proposed for the project. One of the options should be to “do nothing.” 6. Proposed Action a. Next Step. Describe the next step, e.g. Project Planning, conduct an RFI, defer further action until future date, etc. b. Project Team. Name the individuals to be involved in the next step (i.e. core team members only in cases of very large projects). c. Resources Required. Estimate the resources required for the next step (internal staff time, cost of contracted ser- vices, materials or other expenses). 7. Approval to Proceed granted by: _______________________________________ Date: ____________ 93

94 Business Plan Initiation (BPI) Form Please Complete Blank White Fields Only Link to Sample BPI OR Link to Instructions on How to Prepare a BPI CIO BPI-Approved Amount: Previous BPIs for this project?: No If Yes, enter the previous BPI number(s): BPI ID No.: Project Description Section Project Title: Project Manager: Phone #: BPI Type: [Reserved]: Department Supported: Performing Department: Funding Department: BPI Requested Amount: $ Funding Source: Chapter 5 Appendix B: WMATA’s Streamlined Form for the Business Plan Initiation (BPI) Review Process and Instructions for Completing it

95 Authority Priority: IT Transition Phase: Enter Executive Summary below. Use Alt + Enter for new line. Project Scope: BPI Scope: Expected Benefits: Total Project Development & Implementation Cost: $ Expected Annual Operations & Maintenance Cost: $ Approval Section Title Name Signature AGM-IT/CIO: Suzanne J. Peck IT PMO: Mary M. Bauer IT Program Manager:

96 Enterprise Architecture: Jamey Harvey Chief/Office Director Project Manager: Business Plan Initiation (BPI) Form CIO BPI-Approved Amount: BPI ID No.: Project Cost Section Development & Implementation Costs Key Tasks (Requirements, Design, Testing, etc.) To Date Enter date: ___/___/__ FY09 FY10 FY11 FY12 FY13 FY14 TOTAL TRAINING TOTAL DEV & IMPL. COSTS Operations and Maintenance Costs Operations and Maintenance Costs To Date Enter date: ___/___/__ FY09 FY10 FY11 FY12 FY13 FY14 TOTAL Software Costs Hardware Costs Staffing Costs Other Costs (ex. Data)

97 TRAINING TOTAL O&M COSTS TOTAL PROJECT COSTS Project Schedule Section Total Project Duration (Development & Implementation): Estimated Project Start Date: Estimated Project End Date: Key Milestones Section Major Milestones/Tasks (For items covered under this BPI Form) Duration (weeks)

4. 5.

103 These are instructions for completing a rating form used to assess the risk of IT projects. The four factors used to determine project risk rating are: 1) Project size 3) Team experience 2) Project manager experience 4) Project type To complete the Self-Rating form, determine the rating for each project evaluation factor. HIGH = 3 MEDIUM = 2 LOW = 1 Factor 1: Project Size This factor rates the project on size, primarily based upon onetime cost estimates and secondarily, upon project duration. Step 1: Rate the project by estimated one-time costs as follows: gnitaR stsoC emit-eno detamitsE hgiH 000,005$ naht retaerG muideM 000,005$ ot 000,05$ woL 000,05$ rednU Step 2: Adjust low and medium ratings in the above upward by one rating level if the estimated time period from project approval to “go live” is greater than twelve (12) months. Chapter 5 Appendix C: KCM’s Form for Determining Extent of Oversight Process

104 Factor 2: Project Manager Experience This factor rates the risk based on the project manager’s experience on similar efforts. gnitaR reganaM tcejorP Has not completed a like project in a project manager role. High Has successfully completed one like project in a project manager role. Medium Has successfully completed two or more like projects in a project manager role. Low Factor 3: Team Experience This factor rates the risk based on the experience of the project team key staff. The project team consists of all project staff reporting to the project manager, including contractor staff, if applicable. Step 1: Evaluate the experience of each key staff member, including contractor staff, for completion of like projects in key roles. gnitaR ffatS yeK fo %57 tsaeL ta yb detelpmoC stcejorP ekiL None High One Medium Two or more Low Factor 4: Project Type This factor rates the technical complexity of the work being undertaken. Technical complexity is only appropriate for projects that deliver a solution which impacts the current technical environment through new hardware and/or software. Solution Delivery projects should utilize the table below by performing steps 1 and 2 for table A.

105 For IT projects that don’t impact the technical environment, please use the LOW or 1 rating factor. [Examples of projects that don’t impact the technical environment include Plan/Document Delivery projects and Process Improvement projects] Step 1: Using Table A below, “Elements of Project Type,” circle the rating for each applicable element. Step 2: Assign the rating for this factor based upon the highest rating from among all of the elements circled in Step 1. Table A: Elements of Project Type gnitaR tnemelE detceffA yrogetaC ytivitcA tnenopmoC woL revreS / potkseD lacoLNew Install Distributed / Enterprise Server Medium woL revreS / potkseD lacoLUpdate / Upgrade Distributed / Enterprise Server Low woL gnilbaC / krowteN lacoL muideM krowteN detubirtsiD Hardware Infrastructure Data center / Network Operations Center/Wireless/Radio High woL revreS / potkseD lacoLCustom Development Distributed / Enterprise Server High woL revreS / potkseD lacoLCOTS Installation (new) Distributed / Enterprise Server High woL revreS / potkseD lacoLCustom Update / Upgrade Distributed / Enterprise Server High woL revreS / potkseD lacoL Software COTS Update / Upgrade Distributed / Enterprise Server Medium

106 gnitaR tnemelE detceffA yrogetaC ytivitcA tnenopmoC muideM erawelddiM muideM tcudorP dereyaL Infrastructure muideM SMBD woL A/N A/N lacinhcet-noN Computation of the Overall Project Rating After determining the rating for each evaluation factor, add the total ratings for factors 1-4, and divide by 4. The score will fall into one of two levels: • Level 1 – Project is subject to a single funding release and to provide monthly monitoring status. • Level 2 – Project is subject to phased funding releases as defined by the Project Review Board Process and to provide monthly monitoring status – this is the full oversight monitoring process. All IT projects in PRB oversight, regardless of their risk level, will need to request funding release from the PRB.

107 Project Oversight Rating Form

108

109 Chapter 5 Appendix D: King County Suggestions for Project Review Board Deliverables (60) PRB Deliverables Requirements for each Deliverable Suggested Information Project Managers May Wish to Cover under each Requirement Project Plan (Summary Version) How the Project will be Managed ▪ Brief description of the project’s: Charter Organization and management plan Communication and project reporting plan Issue and action item plan Risk management plan Quality management plan Change management plan epocStcejorP ▪ High level overview of: Project description What’s in scope What’s not in scope eludehcSyrammuS ▪ Gantt chart for the entire project with: Phases Major deliverables Major milestones Dates tegduBlevel-yrammuS ▪ Lifetime by year by account for the entire project with: Salaries and benefits Miscellaneous supplies Consulting Contract employees Travel Training Printing OIRM support Hardware/software Contingency Other (specify) ▪ Budget assumptions ksiRleveLhgiH Assessment ▪ High impact risks identified for the project

110 PRB Deliverables Requirements for each Deliverable Suggested Information Project Managers May Wish to Cover under each Requirement Possible Contract List List of Possible Contracts ▪ Brief description of each contract with:  Estimated amount of each contract  Estimated time period for each contract Work Plan for Phase II – Project Development One Page Summary Describing the Work of the Next Phase ▪ High level overview of:  Significant project activities  Approach and techniques  Major deliverables description  Major milestones description  Project dependencies  Budget release request for next phase phase  Begin and end schedule dates for next rof eludehcS deliateD Next Phase ▪ Resource loaded Gantt chart with:  Phases  Tasks  Resources (assigned to tasks)  Deliverables  Milestones  Dates rof tegduB deliateD Next Phase ▪ Budget detail (for each item of the budget) ▪ Spending plan ▪ Budget assumptions PRB Deliverables Requirements for each Deliverable Suggested Information Project Managers May Wish to Cover under each Requirement Business Case One Page Summary ▪ High level overview of:  Project objectives  Project description  Significant business needs and requirements  Solution recommendations  Summary costs  Significant quantifiable and non- quantifiable  Financial payback  Project schedule start and stop dates Typical Elements See business case web page http://kcweb.metrokc.gov/oirm/tools_templates/ business_case_tools.htm Business Needs Driving this Project Project Objectives ▪ Strategic goals and objectives ▪ Business goals and objectives ▪ System goals and objectives

111 PRB Deliverables Requirements for each Deliverable Suggested Information Project Managers May Wish to Cover under each Requirement Quantifiable Costs and Benefits for the County ▪ Total development costs by account and year ▪ Quantifiable benefits by year Hard dollar revenue Hard dollar reimbursements Hard dollar cost reductions Other hard dollar benefits Soft dollar cost avoidance Other soft dollar benefits ▪ Operating and maintenance costs by account and year ▪ Payback Break-even point in years Net present value Internal rate of return (IRR) Return on investment (ROI) Quantifiable Benefits for the Public ▪ Hard dollar reimbursements ▪ Hard dollar cost reductions ▪ Other hard dollar benefits ▪ Soft dollar cost avoidance ▪ Other soft dollar benefits Cost Benefit Analysis Worksheet ▪ Detailed quantifiable cost and benefit estimates Non-Quantifiable Benefits ▪ Project alignment with business strategy ▪ Competitive advantage provided by project for the county or the public ▪ Management information support provided by project ▪ Legislative directive or mandate ▪ Alignment with strategic IT architecture ▪ Other Project Plan (Detailed Version) How the Project will be Managed ▪ Description of the project’s: Charter Organization and management plan Communication and project reporting plan Issue and action item plan Risk management plan Quality management plan Change management plan Project Scope ▪ Project description ▪ What’s in scope ▪ What’s not in scope ▪ Constraints and Assumptions ▪ Management information support provided by project

112 PRB Deliverables Requirements for each Deliverable Suggested Information Project Managers May Wish to Cover under each Requirement Schedule ▪ Gantt chart for the entire project with:  Phases  Tasks  Resources  Deliverables  Milestones  Dates Budget ▪ Lifetime by year by account for the entire project with:  Salaries and benefits  Miscellaneous supplies  Consulting  Contract employees  Travel  Training  Printing  ITS support  Hardware/software  Contingency  Other (specify) ▪ Annual by account ▪ Spending plan ▪ Budget assumptions Project Control Plans ▪ Organization and staffing plan ▪ Risk Management Plan ▪ Issue and action item management plan ▪ Change (scope) management plan ▪ Communication plan ▪ Quality plan ▪ Vendor management plan ▪ Benefit Realization plan ▪ Summary level implementation plan ▪ Summary level architecture plan Updated Contract List Updated List and Description of Contracts ▪ Description of each contract with:  Estimated amount of each contract  Estimated time period for each contract  Possible vendors for each contract

113 PRB Deliverables Requirements for each Deliverable Suggested Information Project Managers May Wish to Cover under each Requirement Work Plan for Phase 3a - Implementation Planning & Solution Design One Page Summary Describing the Work of the Next Phase ▪ High level overview of:  Significant project activities  Approach and techniques  Major deliverables description  Major milestones description  Project dependencies  Budget release request for next phase  Begin and end schedule dates for next phase Detailed Schedule for Next Phase ▪ Resource loaded Gantt chart with:  Phases  Tasks  Resources (assigned to tasks)  Deliverables  Milestones  Dates Detailed Budget for Next Phase ▪ Budget detail (for each item of the budget) ▪ Spending plan ▪ Budget assumptions 6 Findings on Systems Engineering and Transit Systems Engineering as a process for system development was first described in the 1950s and was originally created to address the development of large-scale defense systems. Since then it has been broadened into a discipline that is used in all kinds of project developments. Systems engineering can be applied to any system development, whether you are devel- oping a household appliance, building an airplane, or imple- menting a sophisticated transit management system. As the International Council on Systems Engineering (INCOSE) defines it: Systems Engineering is an interdisciplinary approach and means to enable the realization of successful systems. It focuses on defin- ing customer needs and required functionality early in the devel- opment cycle, documenting requirements, then proceeding with design synthesis and system validation while considering the com- plete problem. Systems Engineering integrates all the disciplines and specialty groups into a team effort forming a structured development process that proceeds from concept to production to operation. Systems Engineering considers both the business and the techni- cal needs of all customers with the goal of providing a quality product that meets the user needs. Outside the realm of ITS, the Systems Engineering process is described by standards (such as ANSI/EIA 632-Processes for Engineering a System and ISO/IEC 15288: 2002(E)— Systems engineering—System life cycle processes) and hand- books (INCOSE SE Handbook and IEEE 1220). These docu- ments provide a general description of the Systems Engineer- ing Process and relate it to typical system life cycle phases. As such, these are excellent reference documents for the transit community to consider as they move toward the use of the systems engineering process in the development of ITS sys- tems. However, their generality make them less approachable and less understandable to many within the transit commu- nity, who may be better served with documentation that directly relates the basic systems engineering process to tran- sit ITS development. US DOT recognized the potential benefit of the systems engineering approach for ITS projects and included require- ments for the use of the systems engineering process in the FHWA Final Rule/FTA Final Policy on Architecture and Standards that was enacted on January 8, 2001. The Rule/Pol- icy requires a systems engineering analysis to be performed for ITS projects that use funds from the Highway Trust Fund, including the Mass Transit Account. Figure 21 shows an excerpt from the Final Policy that specifies the minimum requirements that the systems engineering analysis must include. Why has the USDOT instituted this policy? Because there are well documented benefits to using a Systems Engineering process for development of technology-based projects. What are some of the benefits? As described in the Introduction to Systems Engineering mentioned above, some of the benefits (along with references for these benefits) are:

“Systems engineering reduces the risk of schedule and cost overruns and increases the likelihood that the implementa- tion will meet the user’s needs. Other benefits include: • improved stakeholder participation • shorter project cycles • more adaptable and resilient systems • verified functionality and fewer defects • higher level of reuse • better documentation These assertions have been supported by several studies that have shown that good systems engineering results in better cost and schedule performance. Studies have been performed by the International Council of Systems Engineering (61), Boeing (62), and IBM (63), among others. Figure 22 shows the results of an INCOSE study that collected planned and actual project cost data and systems engineering cost data for 44 projects. The survey indicated that investing in systems engi- neering improved project cost performance. The responses indicated a 50% overrun on average without systems engineer- ing and a clear trend towards better cost performance results with systems engineering.” In order to provide a description of the Systems Engineer- ing process that is tailored to the transportation community, the USDOT developed two documents: • Systems Engineering for Intelligent Transportation Sys- tems: an Introduction for ITS Professionals (64), and • Systems Engineering Guidebook for ITS (65) The first of these is, as it says, an introduction that has the following purpose: This guide is intended to introduce you to systems engineering and provide a basic understanding of how systems engineering can be applied to intelligent transportation systems (ITS) projects. The guide leads you step-by-step through the project lifecycle and describes the systems engineering approach at each step in the life- cycle. It describes how to begin implementing the systems engi- neering approach on your next ITS project and incorporate it more broadly into your organization’s business practices. The second is a more detailed “text book” that provides not only details for each step in the process, but also templates for all of the Systems Engineering deliverables. An on-line version of the guidebook has been recently developed (http://www. fhwa.dot.gov/cadiv/segb/) that not only puts the guidebook information into a hyperlinked format but also provides exam- ples of systems engineering documentation drawn from actual ITS projects across the country. The following (taken from the first USDOT source above), gives a good description of how the Rule/Policy is applied to the agencies across the country that deploy ITS. The Rule/Policy allows each Project Sponsor to use a systems engineering approach that is tailored to fit the needs of each ITS project. As you will see in the following chapters, the systems engineering approach is actually broader than the seven specific requirements identified in the Rule/Policy. If you implement a good systems engineering process, you will meet or exceed the spe- cific systems engineering analysis requirements identified in the Rule/Policy. The FHWA Division and FTA Region offices determine how the systems engineering analysis requirements in the Final Rule/Policy should be applied to ITS projects in each region and how compli- 114 § 940.11 Project implementation. (a) All ITS projects funded with highway trust funds shall be based on a systems engineering analysis. (b) The analysis should be on a scale commensurate with the project scope. (c) The systems engineering analysis shall include, at a minimum: (1) Identification of portions of the regional ITS architecture being implemented (or if a regional ITS architecture does not exist, the applicable portions of the National ITS Architecture); (2) Identification of participating agencies roles and responsibilities; (3) Requirements definitions; (4) Analysis of alternative system configurations and technology options to meet requirements; (5) Procurement options; (6) Identification of applicable ITS standards and testing procedures; and (7) Procedures and resources necessary for operations and management of the system. Figure 21. FHWA/FTA systems engineer- ing analysis requirements. Systems engineering effort as % of project cost R at io o f a ct ua l t o pl an ne d pr oje ct co st Figure 22. Systems engineering improves project cost performance. (Source: INCOSE.)

ance should be demonstrated by each project sponsor. Federal oversight is provided based on oversight requirements defined in the stewardship agreements with each state. . . . . Contact the ITS specialist in your FHWA Division Office or FTA Regional Office for more information. These USDOT references describe the systems engineering life cycle through the mechanism of a “V” model (hereafter referred to as the Vee model) that is shown in Figure 23. This model describes the key steps in the overall process, beginning with the creation of a Regional ITS Architecture to define the integration of ITS deployments in a region, and continuing through the life cycle all the way to system retirement or replacement decision. Notice in the diagram that most steps on the Vee have a small oval at the conclusion of the step (identified as Document/Approval in the figure). At each of these steps there is some output of the process that must be reviewed and approved so that development can move to the next step. These “decision points” are one of the key attributes of the systems engineering process. The Vee diagram and system engineering steps, which sup- port a variety of technology project types, are similar to Sys- tem Development Life Cycle or Software Development Life Cycle (SDLC) methodologies that have been used in the IT industry for software development. A number of SDLC mod- els have been created, with some common ones being the “waterfall,” “spiral,” and “rapid prototyping” models. The Vee diagram can be tailored to suit any of these types of devel- opment models. How has the transit industry embraced the Systems Engi- neering process as described by the USDOT? That was one of the subjects of the interviews and will be discussed below. Before covering the results of the interviews, this section will consider how recent transit publications have discussed the use of Systems Engineering in transit ITS project development. 6.1 Literature Review The report Advanced Public Transportation Systems: State of the Art Update 2006 (66) covers a wide range of topics in transit ITS, including systems engineering. The report high- lights the FTA Policy on Systems Engineering (described above) and references the general systems engineering documenta- tion previously discussed. Some of the key points made by the report are: • The disciplined use of a systems engineering approach is a critical success factor for projects involving integration. • The systems engineering component to FTA’s Policy on the National ITS Architecture is extremely important when developing ITS projects. • Use a systems engineering approach in developing and inte- grating applications, and for the definition of stakeholders and their requirements. In addition the report considers lessons learned from a variety of deployments and identifies the following relative to systems engineering: • Avoid the tendency to quickly buy systems. Instead, initiate all steps of a systems engineering approach. Do not skip steps such as the definition of key stakeholders, functional requirements definition, alternatives analysis, detailed 115 Figure 23. Systems Engineering VEE Model. (Source: U.S. Department of Transportation.)

requirements definition, development of a thorough testing and acceptance criteria plan and development of an Opera- tions and Maintenance Plan. • Budget the time and resources needed for systems engi- neering. The report, AVL Systems for Bus Transit: Update, TCRP Syn- thesis 73 (67), “documents the state of the practice of com- puter-aided dispatch/automatic vehicle location (CAD/AVL) systems in fixed-route and demand-responsive services (referred to in this synthesis simply as bus AVL systems), as well as changes in agency practices related to the use of AVL systems.” The report is based on extensive interviews with tran- sit agencies and includes many lessons learned, including the following related to systems engineering: • Plan for delays in schedule; ensure the technology matches your current and future agency needs. Do not let the current technology limit your agency vision; use a good systems engineering approach to develop a concept of operations plan. In addition, the report includes an appendix that discusses the systems engineering process that agencies have used suc- cessfully to deploy technology such as bus AVL systems. Probably the most relevant previous review of the subject of Systems Engineering (and its relation to Enterprise Archi- tecture Planning) is TCRP Report 84-e-Transit: Electronic Business Strategies for Public Transportation-Volume 5 Con- cept for an e-Transit Reference Enterprise Architecture (68). This report, published in 2004, describes “the need for and uses of a reference enterprise architecture; the process for its development based on using systems engineering concepts and practices; the basic concepts behind systems engineer- ing and enterprise architecture; and the transit-specific task associated with creating an e-transit reference enterprise architecture.” The report provides a tutorial on the systems engineering process (which predates the FHWA/FTA doc- uments referenced earlier) and describes how to use the systems engineering process in the development of an enterprise architecture framework. Figure 24, taken from the document, provides an illustration of how using enter- prise architecture, along with the systems engineering pro- cess can be used to transform the operations of a transit agency. Lastly, a recent report dealing with technology and transit is another volume in the same TCRP series, TCRP Report 84- e-Transit: Electronic Business Strategies for Public Transportation- Volume 8 Improving Public Transportation Technology Imple- mentations and Anticipating Emerging Technologies (69). This volume presents the results of a study of technology in tran- sit. The study was performed to address the following two topics: “(1) to identify the steps that must be taken—by both individual transit agencies and the transit industry—to improve technology implementations; and (2) to promote consideration of emerging technologies by identifying several 116 Figure 24. A reference transit framework using systems engineering and enterprise architecture. (Source: TRB.)

developing technologies that have great potential for the tran- sit industry.” One of the key conclusions of the study is that improving transit technology implementation in public transportation requires the incorporation of the critical strategies of enter- prise architecture planning (EAP), systems engineering (SE) and change management (which is actually a part of the over- all systems engineering process but which the study authors broke out separately because of their view of its importance). The study reviewed the level of systems engineering knowl- edge and usage within transit agencies (primarily from inter- views and literature review performed prior to 2006) and finds a very uneven level of knowledge and usage. Some larger agencies have embraced the process in their technology proj- ects, but many others have little experience with or knowl- edge of the process. An overall summary of the previous literature on systems engineering in transit is that using a systems engineering process is one of the critical factors in successfully implementing tech- nology- (or integration-) based projects; however, only select agencies have incorporated the SE process into their develop- ment processes, with the majority of transit agencies either turning over responsibility for the process to their contractors or not using the process at all. 6.2 Interview Findings In order to determine where transit agencies stand on under- standing and use of Systems Engineering for ITS project development, a portion of each transit agency interview was devoted to the use of Systems Engineering. For several of the agencies that had recent experience with the systems engi- neering process, an additional set of interview questions was posed to assess whether the agencies had seen benefits from their use of the Systems Engineering process. The discussion below highlights the key findings from the interviews. 6.2.1 Use of the Systems Engineering Process by Transit Agencies Almost all of the agencies interviewed indicated they used some type of development process or did some aspects of the Systems Engineering process. Only two answered “no” or “not really” to the basic question, Do you use a Systems Engineering Process for project/system development? A closer examination of the interview responses shows that about half of the agencies could be described as having a development process, and of these only a couple are really using the Systems Engineering process. Why the discrepancy? There are several key reasons: • Low level of knowledge of the systems engineering process among agency personnel. In several cases, the agency response was that we do whatever parts of the process the contractor provides. It seems in some cases the agencies are content to rely solely on whatever level of expertise the con- tractor provides. In one or two of the agencies they specifi- cally hire a contractor to be their system engineer, providing the SE expertise that they lack. • Existing project management or system development processes. Several of the agencies that could be considered more advanced (based on the number and scope of their ITS deployments) have a definite process orientation, but in most cases this orientation is strong on project manage- ment (or in one case business management) but not strong in the technical development process that SE represents. Because of the project management focus, these agencies have a structured view of tracking the project’s progress against cost and schedule. They may also have detailed consideration of such cross-cutting activities as risk man- agement. However, what these processes lack is the techni- cal development process, with its Concept of Operations (focusing on the stakeholder needs and the operational scenarios of the systems), formal requirements definition, design tradeoffs and verification against requirements. They each cover parts of these activities (most often the requirements definition) but not all of them. • Transit Agencies have in general not been required to use the Systems Engineering process. Although FTA Pol- icy (on Architecture and Standards) requires a systems engineering analysis for each project using federal funds, the requirements do not cover the full range of the SE process, and can be met by cherry picking info from a far less systematic development process. Two of the agencies interviewed were required to closely follow the USDOT systems engineering process. They were developing sys- tems under the Mobility Services for All Americans (MSAA) Initiative. This effort, which in Phase I developed the concept of operations and functional requirements for the system, caused each agency to become knowledgeable of the USDOT SE process and to utilize it in the project development. As will be discussed below under benefits of the SE process, both agencies felt it was a worthwhile exer- cise and plan on using the Systems Engineering process for future efforts. 6.2.2 Benefits of Using the Process Have the agencies that have used the Systems Engineering process derived benefits from the effort they put into the process? The answer is a resounding yes. Some of the benefits they identified were: • Using the process helped the agency and the other stakehold- ers go through each step rather than jumping to the end. 117

• The SE process helps the agency keep the project on sched- ule and budget. It allows the agency to have better visibility into the contractor’s progress through the outputs. • Using the process saves the agency a lot of trouble at the backend of the project because the surprises are minimized. • The Concept of Operations made the agency and the rest of the stakeholders more aware of how the parts of the sys- tem will integrate and work together. 6.3 Recommendations Based on the observed level of usage of the Systems Engi- neering process, the following are recommendations for apply- ing the process to the development of Enterprise Architecture Frameworks. • Agencies should acquire a basic working knowledge of the Systems Engineering process as it would apply to their projects. Agency personnel are key participants in the proj- ect implementation process and must have an understand- ing of the process if they are to successfully deploy tech- nology-based systems. This knowledge is available through training, either in workshop or on-line settings, and agen- cies should put plans in place to obtain the necessary skills. • Any agency pursuing development of an Enterprise Architecture framework should use the Systems Engi- neering process to plan and perform the development. As mentioned in the TCRP reference above (and as discussed in the results of this current TCRP effort), the two disci- plines (SE and enterprise architecture) are inextricably linked and when pursuing enterprise architecture develop- ment it is critical to perform the development using a Sys- tems Engineering process. 7 Findings on Post-Implementation Analysis in Transit Objective of Post-Implementation Analysis Synthesis The purpose of this synthesis is to document the state of the practice in the transit industry with respect to post-implemen- tation analysis of IT/ITS deployments and to provide high-level guidance on how to approach post-implementation analysis. The synthesis will briefly address the difference between post- implementation analysis and project closeout activities. In addi- tion, potential linkages between the post-implementation analysis and other project stages in the Enterprise Architecture and Planning Framework will be highlighted. 7.1 Approach/Methodology To develop this synthesis, a review of the literature on Post- Implementation Analysis and Post Implementation Review (PIR) was completed. The literature review encompassed post- implementation analyses in transit ITS as well as other fields. To supplement the literature review, 14 transit agencies and two state DOTs were surveyed regarding their post-implementation efforts. Because post-implementation analysis is called different things in different organizations and methodologies, additional prompts and follow-up questions were needed to clarify what was being discussed. 7.2 What is Post-Implementation Analysis? Post-implementation analysis or Post Implementation Review (PIR), as it is commonly called in the field of Infor- mation Technology (IT), is conducted after a project has been completed. “The purpose of the PIR is to evaluate how suc- cessfully the project objectives have been met and how effec- tive the project management practices were in keeping the project on track.” (70) PIR is not the testing and verification activities that are typ- ically performed in a project acceptance or closeout phase. As an example, a system may have to be accepted from a vendor if it performs according to the requirements. It passes the test plan and the systems engineering verification process. The system verification step can include both “factory testing” and “on-site testing” that occur during the initial implemen- tation or deployment phase. The system, however, may not perform the way the users want, because either the business changed or it was specified ambiguously and/or incorrectly. The PIR occurs after the IT/ITS system has been incorporated into the business. PIR corresponds more closely to the Validation phase of the systems engineering process. In the Systems Engineering Guidebook for ITS (65), the difference between verification and validation is explained as follows: Verification is the process which makes sure that what was built matches the requirements. Was the system built the way the requirements and design specified? Was the system built “right”? Both the verification and validation processes are important and necessary. However, it is the validation which views the system from the system’s owner and stakeholder perspective. The veri- fication of the system is viewed from the development team’s perspective. Systems engineering’s goal is to unify these views. . . . System Validation by system owners and stakeholders com- pares against needs, goals, and expectations (this evaluation of validity directs the path to system upgrades and enhancements). The USDOT’s Research and Innovative Technology Admin- istration’s (RITA) web site provides guidance on how to com- plete comprehensive, independent evaluations of ITS projects. The ITS Evaluation Resource Guide at that site (71) expands and elaborates on recommended evaluation procedures out- lined in the SAFETEA-LU Reporting and Evaluation Guide- lines. If the funding is available for one of these more elaborate 118

evaluations, the PIR results would feed into the evaluation. It should be noted that a PIR can be very elaborate, but they also can be completed in relatively simple forms. Generally, the PIRs are designed to improve the performance of the system or the project management procedures at the particular organi- zation that has implemented the system, rather than provide comprehensive knowledge to the broader industry. Further detail about post-implementation analyses or PIRs is included in subsequent sections of this chapter. 7.3 PIR Benefits: Why is Post- Implementation Analysis Valuable? Post-implementation analysis is valuable for many rea- sons. The most commonly cited benefit of the PIR process is that it provides managers with critical information on how to modify and improve a recently implemented system to better meet the needs of the users and the staff responsi- ble for maintaining the system. The PIR process helps iden- tify system flaws, requirements that need to be changed, ways to improve performance and other potential future enhancements. The Information Systems Board (72) of Washington State believes that PIR process is valuable because “[w]hat gets measured gets managed. Determining performance measures and outcomes at the beginning of a project helps assure that the project stays true to the initial purpose and priorities. Defining the desired outcomes or acceptance criteria at the beginning of the project also clarifies the project’s scope. Using performance measures ascertains whether the project did indeed succeed, and provides a starting point for devel- oping future lessons learned.” Other benefits of post-implementation analysis include: • Identification of “lessons learned” about the technology • Identification of “lessons learned” about the project man- agement process • Documentation of “what went well” for: – awards and team building – developing and incorporating best practices into proj- ect management guidelines – sharing with other transit business areas and organiza- tions in the industry • Improved understanding of the client’s business needs • Improved effectiveness of the IT organization by incorpo- rating PIR lessons and, with time, enhancing its credibility • Increased knowledge on how to quantify benefits of ITS projects • Better investment decisions on future projects from using the PIR information • Ability to provide project sponsors and funding organiza- tions with evidence of costs and benefits • Provide stakeholders with measures of success to help val- idate their decisions and support if the project went well • Finally, information from ITS project evaluations can also help the industry with subsequent projects by helping understand the impacts of the technology on transit ser- vices and users, transit organizations and their staff. 7.4 Post-Implementation Review Process Overview Various organizations and vendors have slightly different PIR processes, but the core process consists of the following four steps: Planning, Preparing, Conducting and Reporting/ Following up. Generally, a cross-functional team consisting of key stakeholders, users and technical experts plans and con- ducts the PIR, unless the capability and resources are available for having an independent auditor or evaluator. In that case, the auditor or evaluator works with the team and ensures the data collection and analysis integrity. For very small projects, the PIR may consist of the business manager or IT manager surveying users and the technical staff and then reporting results. The FAA web site includes a moderate-sized list of what a post-implementation review covers (73): • “Perspective and insight of participants and users, cus- tomers, and stakeholders; • Original investment expectations including performance, investment and operating costs, schedules, benefits, and technical capability; • Actual investment results (e.g., operational performance; customer, user, and stakeholder satisfaction; investment and operating costs; technical capability; impact on mis- sion and program measures; unanticipated benefits); • Cost and schedule deviations, such as additional “hidden” costs related to investments that have been made to enable the primary investment; • Environmental changes that affected the investment (e.g., political, operational, economic, or technical conditions); • Original business case assumptions that justified the invest- ment program; • Expected next steps for the investment program; • Conclusions and learned lessons; and • Recommendations to senior management.” 7.4.1 Planning the PIR Planning begins during final investment analysis in con- junction with overall planning for implementation and life- cycle management of an investment program. Planning for the PIR is incorporated into the project plan and funding package. Goals, objectives and anticipated benefits that are detailed in the Business Case and ROI need to be considered 119

in conjunction with the PIR Plan. How will the objectives and benefits realization be measured and analyzed? Is it fea- sible? Can it be determined if the risks were adequately defined and mitigated? Are resources allocated to complete the detailed planning, data collection and analysis steps of the PIR? 7.4.2 Preparing for the PIR This step generally corresponds to the systems engineering process’s Validation strategy activity. It further defines the PIR Plan and prepares the survey forms, templates, analysis approaches and other resources that will be needed. It defines in more detail how the PIR or validation will take place and what resources will be needed. For example, if a before-and- after study design will be used, the “before” data will need to be collected prior to deployment of the system. 7.4.3 Conducting the PIR Most post-implementation interviews, surveys, “lessons learned” sessions and other data collection activities are con- ducted after the IT/ITS system has completed system verifi- cation and acceptance testing. Data collection may have to begin earlier to collect “pre-” or “before” data. 7.4.4 Reporting/Follow-up Data is analyzed, a gap analysis is performed, results are documented and they are reported to the project team, stake- holders, transit managers and individuals and organizations requesting the evaluation results. Ideally, appropriate bene- fits and lessons learned are reported to RITA for incorpora- tion into the RITA web site databases. Transit management and project managers should follow- up on the recommendations and implement changes. Action plans should be developed and implemented. If the results from multiple PIRs from different programs are reviewed together, the review may identify ways to improve an agency’s IT/ITS “investment planning and management control processes, enable more accurate estimates of invest- ment costs and benefits, and ultimately result in better investment decisions. Results from successive reviews on singular investment programs enable managers to deter- mine if actions to improve performance and benefits are working.” (74) 7.5 Transit Survey Findings The transit agencies that were surveyed had varying lev- els of understanding of post-implementation analysis or PIR. In addition, post-implementation analysis was called different things in the various agencies, so additional prompts and follow-up questions were needed to clarify what was being discussed. Does your agency have a post-implementation analysis or evaluation phase for IT/ITS projects? With the exception of a few of the transit agencies that were surveyed, most of the respondents described relatively little consistent post-implementation analysis activity. In a few cases, PIR was confused with system acceptance or project closeout activities. The majority of the agencies surveyed did not have a formal post-implementation analysis process. Of those that did, it was only sometimes or informally followed by a subset of those respondents. One respondent said their reports had varying levels of formality, but they usually included lessons learned, performance goals and compar- isons against initial model forecasts. Terms used to describe post-implementation analysis activ- ities or processes included Post Project Assessment, Benefits Realization Step, evaluation, Feedback, earned value manage- ment analysis and validation. When the transit agency’s post- implementation analysis had some form of proscribed proce- dures, it was generally because the organization’s central IT staff had a System Development Life Cycle (SDLC) method- ology that included a post-project-closeout analysis step. An interesting, related comment from MARTA was that they have hired staff to be an in-house, independent verifi- cation group that analyzes a new system prior to system acceptance (they complete the system engineering verifica- tion process step). This group and process have “paid off in dividends.” What is the time frame for measuring/evaluating the results of the IT/ITS project? The time frame for completing post-implementation analy- ses varied, but most were completed within one year of system acceptance. The Utah Transit Authority (UTA) has an interesting approach that includes two phases. First, it obtains feedback on the system from the customer within 30 days of system acceptance. UTA is striving for ISO (International Organiza- tion for Standardization) consistency, so this feedback is part of a regularly followed process. UTA strives to monitor, meas- ure and report on whether the project met the agreed upon quality, schedule and budget expectations defined in the scope, while acknowledging that all categories are subject to change requests that can modify expectations to the scope. UTA has another regular post-implementation practice, although there is no form for it. An IT supervisor or the project manager always checks back on the new system, generally after it’s running for three to six months (maximum one year) to see if anything else could have been done differently. They look for 120

lessons learned or needed system adjustments, as well as using it as an opportunity to keep up with changing business needs. The King County Metro Transit Signal Priority (TSP) team completes its Before and After data collection efforts imme- diately surrounding a new installation to have as similar as possible “before” and “after” operating conditions (usually two weeks before and two weeks after). Who or what is the driver for having a post-implementation analysis? A variety of reasons were given for doing post-implementa- tion analyses. Some agencies cited policy or practice for doing post-implementation analyses. Another said ISO standards and procedures, as well as it being critical for providing good cus- tomer service. Other answers included the following: • Federal requirements • “Usually we think it is the right thing to do.” • Grant requirements • When a project manager pushes for it • When it is a problematic project or one with lots of conflicts • When someone promised cost savings and now we have to find them • We had to justify why it cost so much • Want the lessons learned to improve practices and proce- dures • Want to know how to improve the system in the enhance- ment phase and if it is needed How are the results used? The most common answer was that the lessons learned were valued for improving future projects. The results were also used to guide the next set of enhancements for the new project or to identify new business requirements. The Utah Transit Authority used the PIR process for sev- eral purposes. Documenting PIR results from all of the IT/ITS projects “allows you to go back and see what you did and learn from errors.” From an IT perspective, “one of the best values is the alignment of the requirements and the deliverables (was it that the client changed their mind or that resources changed?). Feedback helps you clearly know what the clients think. It’s time consuming, but good. It just takes lots of time.” The TSP team at King County Metro uses the evaluation results in a number of different ways. They use the feedback for adjusting and fine-tuning the TSP system, for TSP staff training and education and for determining whether or not to shut down a location with poor performance. In addition, the analyses have helped them contribute to the industry’s knowledge about TSP in talks, papers and during the devel- opment of the TCIP standards. Finally, they use the evalua- tion data to help determine where to put the next TSP instal- lation, where to do improvements, to estimate how much time each vehicle spends on every block of the street and to provide the data to others in the organization who want it. One of the biggest benefits is that it helped build tools, such as the TSP Interactive Model (cost-benefit model), for creat- ing more effective installations. Does your agency apply the post-implementation analysis process to all or some of its IT/ITS projects? Three of the agencies said they did some post-implementa- tion analysis regularly after an IT/ITS project has passed sys- tems acceptance. Most said they would try to do more in the future. What are the biggest issues in completing the analyses? For those agencies that completed post-implementation analyses, time, money, gathering data, and motivation were issues in completing the work. For some, after the project was over, they felt pressure to either work on enhancements or move onto a new project. Another said that it is a struggle to obtain data for a good ROI analysis; they use the cost/benefit analysis portion of the ROI more as a planning tool for decid- ing between implementation options. 7.5.1 King County Metro (KCM) King County Metro has extensive, detailed documentation and requirements for how project managers will run their IT/ITS projects, interact with the King County Project Review Board and document their activities. The process is pro- scribed from the funding request phase through project close-out, plus a Benefits Realization Report that is due a year after project close-out. The Project Close-out Report is due within a month of the final monthly monitoring status report. It is supposed to include: Documentation of the project description, results, variance and a summary explanation of what the project accomplished— highlighting relevant project scope, schedule, and budget infor- mation from the close-out documentation. Also includes benefits measurements, lessons learned, records retention, and deliver- ables turnover. (44) Forms and templates are provided to help complete the report. Table 7 lists some of the project areas that are to be considered when developing lessons learned. The project teams are also encouraged to describe project practices that worked well and could be utilized by other projects through- out the county to improve their performance. The Benefits Realization Report is a relatively new require- ment that aims to identify the benefits and value of the proj- ect after it has been operating for up to a year. In addition, it 121

should provide a comparison of the benefits received to the value projected by the Business Case. The lessons learned and other data from the project closeout activities are part of the inputs to the Benefits Realization Report. The key questions addressed by the report are: • Did the project provide quantifiable value to the county or to the public? • Did the project provide non-quantifiable benefits to the county or to the public? • Did the project provide benefits comparable to those pro- jected by the Business Case? 7.5.2 Other Transit Discussion of Post-Implementation Analysis Transit agencies in the United States are not the only ones finding it difficult to complete PIRs on ITS projects. In 2004, Transport Canada published an article titled, Evaluation of the Intelligent Transportation Systems (ITS) Deployment and Integration Program, (75) which examined 12 Canadian ITS projects. The report indicated that “some recipients provided minimal information in their project evaluations.” In the rec- ommendations section, the report states that “In the future, Transport Canada should improve the measurement and reporting of results achieved by the ITS projects that it funds. . . . Transport Canada should work with ITS funding recipients to incorporate appropriate and cost-effective per- formance measurements and reporting methodologies to be able to evaluate the results of ITS projects.” In the ITS field of AVL, a 2008 Transit Cooperative Research Program report titled Synthesis 73, AVL Systems for Bus Tran- sit: Update (76) mentioned challenges with obtaining good post-implementation data. “In many AVL system implemen- tations, the implementing agency did not systematically evalu- ate aspects of benefits that might have been quantifiable as they did not see a need to undertake the additional evaluation.” The report further states, “Determining costs is complicated, since some could be attributed to other systems such as the Radio system, fare boxes, APC, WAN upgrades, new staff that work on more than one system.” The AVL Synthesis 73 report does list a number of lessons learned from the post-implementation follow-up, such as “staff resistance to accepting data as valid if it contradicts con- ventional understandings, . . . staff resistance to adopting needed changes in operational procedures, . . . a number of integration challenges, . . . the importance of securing partic- ipation from throughout the agency organization, carefully selecting the systems integrator, applying strong project man- agement for the implementation, and understanding the sub- stantial ongoing effort needed for system management once it is operational.” A paper from 1998, titled, ITS Benefits: Review of Evaluation Methods and Reported Benefits, (77) provides background infor- mation pertaining to ITS evaluations. It summarizes the reported benefits of a number of ITS systems that have been deployed and the evaluation methods used to quantify the ITS benefits. The report also presents several evaluation frameworks that have been used to evaluate and quantify ITS benefits. The US Department of Transportation provides information and tools that can help transit agencies with building a business case and designing the analyses for the PIR on its RITA website (78). For example, under the “ITS Resources” tab, information can be found in the benefits, lessons learned and cost databases that is helpful in planning a PIR. The evaluation support por- tion of the website (79) provides guidance on how to complete comprehensive, independent evaluations of ITS projects and provides links to a variety of sample evaluation strategies. Similarly, the IDAS (80) website may help with the design and analysis of some benefits of some ITS projects. “The ITS 122 Lessons Learned-Project Areas to Consider Project Planning Quality Management Implementation Budget Management Scope Management Schedule Management Issues Management Risk Management Change Management Communications Team Management Project Close-out Requirements Design Development Support Work Effort Estimating Transition to Production Testing Other Table 7. Lessons Learned.

Deployment Analysis System (IDAS) is software developed by the Federal Highway Administration that can be used in planning for Intelligent Transportation System (ITS) deploy- ments. State, regional and local planners can use IDAS to esti- mate the benefits and costs of ITS investments—which are either alternatives to or enhancements of traditional highway and transit infrastructure.” 7.6 Non-transit Approaches to Post-Implementation Analysis Other industries and countries have recognized the value of conducting Post-Implementation Analyses and Reviews. Pro- cedures and templates are available from a number of commer- cial vendors and consultants. In addition, various states and government organizations have instructions and tools available via the Internet. Despite all the available guidance and tem- plates, many organizations struggle with finding the knowledge, time, and resources to complete a PIR on a new system. In addi- tion, some of the barriers and issues cited in the next section conspire against the successful completions of PIRs. A study conducted in Australia on “The Politics of Post- Implementation Reviews” (81, Pages 307–319) indicated that “few organizations undertake any substantive form of ex post evaluation.” It is one of several studies that state that system- atic use of formal evaluation is relatively rare after a system has been accepted. Several of the more accessible web sites that provide guid- ance and templates on PIR are briefly discussed below. The Washington State Department of Information Ser- vices, Information Services Board, provides guidance and templates under the topic, “Project Management Frame- work, Closure-Post Implementation Review,” at the website: http://isb.wa.gov/tools/pmframework/projectclosure/postim plementation.aspx The Federal Aviation Administration also has easy to follow and understand guidance on PIRs on the web. The initial menu is located at: http://fast.faa.gov/post_implementation/ index.htm One of the initial menu items, PIR Standard Process Guid- ance, is expanded in Figure 25 to show how the website has nested the guidance and templates. As an example, the fol- lowing guidance is included under the item #1.5, Can We Tailor the Review? Post-implementation reviews are always tailored to the size, complexity, and importance of an investment program or set of programs. Activities and costs are scaled appropriately, and may range from periodic surveys or focus-group meetings with users of small, low-cost investment products to multiple site visits by a dedicated cross-functional team of users and stakeholders for large, complex, high-cost investment programs. In all cases, actual operational data from users must be gathered and assessed against performance targets. . . . As another example, the following guidance is included under the item #1.9, When do we Conduct Post Implemen- tation Reviews? We conduct post-implementation reviews 6 to 18 months after deployment at an operational site once initial problems are worked out and users are generally familiar with the new capability. Timing is crucial and dependent on the status of the investment program. A review conducted too soon may fail to capture full benefits, while a review conducted too late may lose institutional knowledge about the investment and recommendations may come too late to influence follow-on installations . . . Sample survey forms for the Project Manager, Sponsor, Stakeholders, and Team are available at: http://www.its.monash.edu/staff/projects/project-manage- ment/templates.html. These samples provide additional ideas for questions to include in a PIR process. Discussion of auditing guidelines for Post Implementation Review from the Information Systems Audit and Control Association (ISACA) can be found at the ISACA’s website. This discussion and website is geared more toward auditors or someone having to work with auditors that will look at an organization’s PIR process. It can be found at: http://www.isaca.org/Template.cfm?Section=home&Template =/ContentManagement/ContentDisplay.cfm&ContentID=18682 7.7 Issues and Barriers Related to Post- Implementation Analysis A number of issues that pose challenges to the successful completion of post-implementation analyses were cited both in the general literature and by the transit agency survey respon- dents. The issues include the following: • Lack of knowledge on how to complete a post-implemen- tation analysis • Lack of time and resources to complete the PIR • “Getting people to understand and care about it” • Failure to collect “before” data so that a comparison of per- formance can be made “pre-” and “post-” implementation • Prior post-implementation analysis efforts, which tried to “place blame” for aspects of the project that didn’t go well, discouraged staff from planning and participating in sub- sequent efforts • The difficulty of collecting accurate financial and operat- ing data • Common statistics related to transit may be gathered and defined in different ways, both internal to an organization and between transit organizations 123

• Collection and analysis of quantitative data for the purpose of rigorous evaluation is often very expensive. A Canadian study on its Intelligent Transportation Systems (ITS) Deploy- ment and Integration Program found that in some instances, “Quantitative data collection and analysis would require additional funding beyond that which was provided for the actual ITS deployments.” (82) • Managers who don’t want a discoverable record of system issues and failures • Major disincentives to further post-implementation analy- ses, and the future specification of budget-related benefits, are created when organizations promptly cut all dollar and staff savings from a work group that successfully imple- mented efficiencies through ITS improvements. Most staff that implement new systems hope to see some of the resources that were saved put to use in improving or expanding services in their business area. 7.8 Recommended Practices for Post- Implementation Analysis The FAA website, which provides distilled, straightforward guidance on how to conduct Post Implementation Reviews, (83) includes the following tips for a successful PIR: • “Build the review into program planning from the start during final investment analysis; • Conduct the review against expectations in the original busi- ness case and program baseline; • Don’t scrimp on resources or effort! This is the last best chance for taking corrective action when a program is not performing as intended; • Get close with the users; they live it every day and know best where we can improve; • Report both the good and the bad; there are always oppor- tunities for doing things better; • Ensure issues are handled effectively and that we have a plan for closure; • Identify next steps clearly; and • Follow recommendations and actions to completion.” Other recommendations for improving the success of post- implementation analysis efforts are: • TriMet felt it was important to have easy to understand procedures and templates for the PIR that require all the key steps to be completed but provide some flexibility and the ability to scale the effort proportionally. 124 Figure 25. FAA guidance on PIR. (Source: FAA.)

• At the beginning of the project, and again after project closeout, establish expectations and abate fears about the PIR process. It is very common for a project to meet requirements and pass the project verification phase, while still having issues during the project validation or PIR phase. This usually occurs because user needs can change over time and requirements may not have been initially specified optimally. • Involve and obtain input from the customers, the project team and other stakeholders. • Use a neutral facilitator during the lessons learned session and avoid placing blame. • Be clear on the project success measures prior to imple- mentation. • Use available resources, such as the ITS benefits and les- sons on the USDOT’s Research and Innovative Technol- ogy Administration web site, to help develop the project specific list of benefits to assess in the PIR. • Choose benefits and performance measures to assess for which elements data can reasonably be collected. • Remember to collect “before” data at the appropriate time, rather than remembering the task at the end of the project. The time period for collecting the “before” data may vary by data set. • Establish a data collection and analysis methodology for the project’s PIR that addresses what data will be collected, how it will be collected, what resources are needed to col- lect it, when it will be collected and how it will be analyzed or compared. • Selecting an assessment horizon depends on the ITS system being implemented, how stable the initial implementation was, available resources and the conditions of the operat- ing environment. Some of the assessment should be con- ducted after the new system and procedures have had ample time to be integrated into the business. Staff learn- ing curves and other project issues may dictate that a num- ber of the benefits be assessed a little later, after the system has “settled down.” • UTA felt it was important to link the PIR activity to an ongoing culture of continuous improvement and the reg- ular rechecking of customer needs and expectations. • Incorporate the lessons learned and best practices into the organization’s procedures. Share the newly gained knowl- edge that is valuable and constructive. 7.8.1 Checklist for Managers This Checklist for Managers Section will be used to stim- ulate discussion among transit staff participating in Task 4 to refine best practices for the Framework. The final Checklist for Managers is intended to assist transit managers in enabling their staff and transit organization to effectively acquire, assess and enhance IT/ITS systems and related busi- ness practices. This portion of the Checklist will focus on management activities that ensure the benefits of completing post- implementation analyses are realized. A number of the steps also improve the value and success of other phases of an IT/ITS implementation. 7.8.2 Transit Manager’s Roles Transit has become more and more dependent on the suc- cessful operation of its automated systems. Managers need both system performance and critical information from those IT/ITS systems for effective decision making and for the effi- cient provision of transit service. Transit business area man- agers need to play a key role in ensuring the success of these systems. In general, IT/ITS projects cannot be successfully implemented with only the attention of the IT manager. Manager’s Checklist Items Key roles of the transit management team are to: ▫ Ensure a common vision, communicate goals and priori- ties, be champions of integration, provide oversight and support staff. The transit General Manager and the head of Information Technology have particular responsibility for ensuring that an integrated, agency-wide approach is taken for developing data and information systems solutions. (84) ▫ Ensure that IT/ITS systems support the operational needs of the agency. The goals of the organization should be one of the drivers of the IT/ITS project’s goals, objectives and requirements. ▫ Ensure that a realistic evaluation plan, Post-Implementa- tion Review Plan or Project Validation Plan (depending on the terminology used by the agency) is developed before the systems development is started, so appropriate “before” data can be collected. ▫ Ensure that complete financial analyses, such as ROI with cost, benefit, and Total Cost of Ownership considerations are completed during the development of the Business Case. These analyses can be used to assess if the completed project met or exceeded the original expectations. ▫ Provide motivation, oversight, and the resources necessary to collect the data. ▫ Ensure that the project verification steps, which verify that requirements are met, are completed before system accept- ance and project closeout. ▫ After project closeout, ensure that the PIR data collection plan is underway, so the post-implementation analyses can be completed. ▫ Request and review the post-implementation analysis report. ▫ Follow-up to make sure appropriate system and process improvement recommendations are implemented. 125

8 References 1. Definition of IT Governance adapted from IT Governance Institute. 2. Description from http://en.wikipedia.org/wiki/Business_case [November 20, 2008] 3. From the Washington State Department of Information Services, Information Services Board, Project Management Framework, Closure-Post Implementation Review, http://isb.wa.gov/tools/ pmframework/projectclosure/postimplementation.aspx 4. Masling, Erik. “The Value of Enterprise Architecture.” http:// enterpriseinnovator.com/index.php?articleID=15148&sectionID=5, published 4/21/2008. Extracted 2/24/2009. 5. “Enterprise Architecture Demystified” by David Aden, Sep 24, 2008, published in Government Technology, http://www.govtech.com/ gt/articles/418008 6. “FEA Practice Guidance.” OMB. November 2007. [http://www. cio.gov/index.cfm?function=specdoc&id=FEA Practice Guidance, November 2007&structure=Enterprise Architecture&category= Enterprise Architecture] 7. Consolidated Reference Model Version 2.3 Issued By: OMB— Effective Date: 10.01.2007, 0.655K, PDF (http://www.cio.gov/ index.cfm?function=specdoc&id=Consolidated Reference Model Version 2.3&structure=Enterprise Architecture&category=Refer ence Models) 8. “Data Reference Model, Version 2.0,” November 17, 2005, is avail- able at http://www.whitehouse.gov/omb/egov/documents/DRM_ 2_0_Final.pdf 9. Federal Segment Architecture Methodology Overview—found at http://www.fsam.gov/federal-segment-architecture-methodology- toolkit/overview.php 10. The Open Group Architecture Framework. (http://www.open group.org/togaf/) 11. ENTERPRISE ARCHITECTURE DEVELOPMENT TOOL-KIT October 2004 v3.0. (http://www.nascio.org/resources/EAre- sources.cfm) 12. Michigan Enterprise Architecture Framework Work Plan Guide- lines. 2005 [www.michigan.gov/documents/Appendix_E_Archi tecture_145041_7.doc] 13. Mitretek Systems. TCRP Report 84 Volume 5 Concept for an e-Transit Reference Enterprise Architecture. Transportation Research Board, Washington, DC, 2004. 14. Hwang, M., J. Kemp, E. Lerner-Lam, N. Neuerburg, and P. Oku- nieff, Advanced Public Transportation Systems: The State of the Art Update 2006, Federal Transit Administration, Washington, D.C., 2006 [Online]. Available: http://www.fta.dot.gov/documents/ APTS_State_of_the_Art.pdf [accessed June 11, 2007]. 15. Boyle, Daniel. “TCRP Synthesis 77 Passenger Counting Systems”. Transportation Research Board, Washington, DC, 2008. 16. Parker, Doug. “TCRP Synthesis 73 AVL Systems for Bus Transit: Update.” Transportation Research Board, Washington, DC, 2008. 17. Miami-Dade Transit, “Information Technology and Intelligent Transportation Systems 2003–2008 Strategic Plan Summary.” (Volumes 1–2, Appendices A–B), January 19, 2004. 18. “Renewing Technology Infrastructure at the Washington Metro- politan Area Transit Authority: Assessment of the Current Enter- prise Architecture.” November 1, 2001. 19. “Professionals’ Guide to Information Technology Architecture Standards and Services.” [2009] 20. Seattle Metro Transit Department, Research & Market Strategy Divi- sion. “Geographic Information Systems Project Phase I Feasibility Study: User Needs Assessment Document”. Seattle Metro, July 1992. 21. Transit Department, Research & Market Strategy Division. “Alter- native Analysis and Recommendation.” Seattle Metro, March 1993. 22. Okunieff, P., N. Neuerburg, Teresa Adams. “Best Practices for Using Geographic Data in Transit: A Location Referencing Guide- book Defining Geographic Locations of Bus Stops, Routes and other Map Data for ITS, GIS and Operational Efficiencies.” FTA- NJ-26-7044-2003.1, US DOT FTA. April 2005. 23. “10 Areas Where Open Source is Open for Business.” From http://www.cioinsight.com/c/a/IT-Management/10-Areas-Where- Open-Source-is-Open-for-Business/?kc=CIOMINUTE02022009 CIO1 [2/2/2009] 24. Federal Enterprise Architecture Framework (v1.1) (http://www. gao.gov/bestpractices/bpeaguide.pdf) 25. NASCIO EA Brochure on Enterprise Architecture: Building Better Government through Enterprise Architecture. (www.immagic.com/ eLibrary/ARCHIVES/GENERAL/NASCIOUS/N070202B.pdf) 26. IEEE Standard 1471-2000 IEEE Recommended Practice for Archi- tectural Description of Software-Intensive Systems. Institute of Electronics and Electrical Engineers, NY, NY, Oct 09, 2000. 27. Spewak, Steven H. and Hill, Steven. Enterprise Architecture Plan- ning: Developing a Blueprint for Data, Applications, and Technol- ogy. John Wiley & Sons, NY, Oct 1, 1992. 28. Sessions, Roger. “Comparison of the Top Four Enterprise Archi- tecture Methodologies” [www.objectwatch.com] 29. National ITS Architecture. www.iteris.com/itsarch. 30. www.fgdc.gov/training/nsdi-training-program/materials/IntroGeo BusinessPlanning.ppt [extracted 1/29/09] 31. FEA Geospatial Profile, draft version 2.0. 32. The Role of the Finance Officer in Economic Development, GFOA, February, 2008 33. Status of the Nation’s Highways, Bridges, and Transit: Conditions and Performance Report to Congress, 2006 34. Transit State of Good Repair: Beginning the Dialogue, October 2008, FTA 35. Financing Capital Investment: A Primer for the Transit Practi- tioner, TCRP Report 89, 2003 36. GASB Statement No. 34: Basic Financial Statements—and Man- agement’s Discussion and Analysis—for State and Local Govern- ments, 1999 calling for management of public assets. 37. Introduction to Public Finance and Public Transit, FTA, 1993 and Report to Congress on the Costs, Benefits, and Efficiencies of Pub- lic-Private Partnerships for Fixed Guideway Capital Projects, FTA, 2007 38. TCRP Report 89: Financing Capital Investment: A Primer for the Transit Practitioner, TRB, 2003 39. TCRP Report 115 Smartcard Interoperability Issues for the Tran- sit Industry, TRB, 2006 40. IT Governance Institute, article http://www.itgi.org/AMTem plate.cfm?Section=Deliverables&Template=/ContentManagement/ ContentDisplay.cfm&ContentID=24261 41. From: http://en.wikipedia.org/wiki/Business_case, March 1, 2009. 42. From Washington State ISB at http://www.isb.wa.gov/tools/ pmframework/charter/performancemeasures.aspx 43. http://www.fta.dot.gov/documents/Transit_IVBSS_Business_Case_ Analysis_Final_Report_9-07.pdf 44. King County Office of Information Resource Management’s “Project Manager Guide to PRB Reviews”, Version 2.0, June 2008 45. http://www.gcio.nsw.gov.au/documents/ 46. From the New South Wales Office of Information and Communi- cations Technology, http://www.gcio.nsw.gov.au/documents/ Business_Case_Template.pdf?bcsi_scan_36B076C11B08DC10=0 &bcsi_scan_filename=Business_Case_Template.pdf 126

47. http://www.cio.sc.gov/NR/rdonlyres/FDC77967-F99D-4B6B-8119- B4B9A3D6C5CC/0/Business_Case_Template.doc 48. http://www.fgdc.gov/policyandplanning/future-directions/action- plans/draftroiworkbook 49. http://www.dir.state.tx.us/pubs/framework/index.htm 50. http://www.dir.state.tx.us/pubs/framework/gate1/businesscase/work book.xls 51. http://www.dir.state.tx.us/pubs/framework/gate1/businesscase/ instruction.pdf 52. http://fast.faa.gov/lifecycle/lpd/investab.htm 53. Article by Malcolm Huston at http://statetechmag.com/events/ updates/evaluating-it-investments.html 54. http://www.ctg.albany.edu/publications/reports/advancing_roi/ advancing_roi.pdf 55. http://www.ctg.albany.edu/publications/guides/roi/roi.pdf 56. http://www.fta.dot.gov/documents/Final_Report_-_Real-Time_ Systems_ROI_Study.doc 57. http://www.fgdc.gov/policyandplanning/future-directions/action- plans/draftroiworkbook 58. Article by Amir Hartman, InformationWeek, April 2, 2007, at: http://www.informationweek.com/news/showArticle.jhtml?article ID=198701922&pgno=1&queryText=&isPrev= 59. King County Office of Information Resource Management’s “Project Manager Guide to PRB Reviews”, Version 2.0, June 2008 60. The tables are from Appendix D of the King County Office of Infor- mation Resource Management’s “Project Manager Guide to PRB Reviews”, Version 2.0, June 2008 61. Honour, Eric. “Understanding the Value of Systems Engineering”, 2004 62. Vu, John D. “Software Process Improvement Journey: From Level 1 to Level 5” 63. Barker, Bruce. IBM Commercial Products, 2003 64. FHWA, Systems Engineering for Intelligent Transportation Sys- tems: an Introduction for ITS Professionals, September 2006 65. Systems Engineering Guidebook for ITS, Co-sponsored by FHWA and the California Department of Transportation, is located at: http://www.fhwa.dot.gov/cadiv/segb/ 66. FTA, Advanced Public Transportation Systems: State of the Art Update 2006, FTA-NJ-26-7062-06.1, March 30, 2006. 67. Parker, Doug. TCRP Synthesis 73, AVL Systems for Bus Transit: Update, 2008 68. Mitretek Systems. TCRP Report 84-e-Transit: Electronic Business Strategies for Public Transportation-Volume 5 Concept for an e- Transit Reference Enterprise Architecture, 2004 69. Burt, Matthew, et al. TCRP Report 84-e-Transit: Electronic Busi- ness Strategies for Public Transportation-Volume 8 Improving Public Transportation Technology Implementations and Antici- pating Emerging Technologies, 2008. 70. From the Washington State Department of Information Services, Information Services Board, Project Management Framework, Closure-Post Implementation Review, http://isb.wa.gov/tools/ pmframework/projectclosure/postimplementation.aspx 71. http://www.its.dot.gov/evaluation/eguide_resource.htm 72. Washington State ISB at http://www.isb.wa.gov/tools/pmframe work/charter/performancemeasures.aspx 73. From the FAA category, Standard Process Guidance, Item #1.10: What Does the Review Cover?, http://fasteditapp.faa.gov/ams/ do_action?do_action=ListTOC&contentUID=19&contentVersion UID=7381 74. From the FAA website on PIR, http://fasteditapp.faa.gov/ams/ do_action?do_action=ListTOC&contentUID=19 75. http://www.tc.gc.ca/corporate-services/des/reports/2004/its/menu. htm 76. http://onlinepubs.trb.org/Onlinepubs/tcrp/tcrp_syn_73.pdf 77. Turner, Shawn, Wm. R. Stockton, Scott James, Troy Rother, C. Michael, October 1998, http://citeseerx.ist.psu.edu/viewdoc/ summary?doi=10.1.1.40.9700 78. Information on the RITA website can be found at: http://www. its.dot.gov/about.htm 79. Evaluation guidance can be found at: http://www.its.dot.gov/ evaluation/eguide_resource.htm 80. http://idas.camsys.com/aboutComponents.htm 81. Gwillim, David, Ken Dovey & Bernhard Wieder, The Politics of Post- Implementation Reviews, Information Systems Journal, Volume 15 Issue 4 82. Evaluation of the Intelligent Transportation Systems (ITS) Deploy- ment and Integration Program, March 2004, mhttp://www.tc.gc.ca/ corporate-services/des/reports/2004/its/menu.htm 83. FAA site with guidance on PIR activities is http://fasteditapp. faa.gov/ams/do_action 84. Best Practices for Using Geographic Data in Transit: A Location Referencing Guidebook, Paula Okunieff, Nancy Neuerburg, and Teresa Adams, 2003 127