Guidance for Development and Management of Sustainable Enterprise Information Portals (2018)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

1 Guidance for Development and Management of Sustainable Enterprise Information Portals The emergence and maturity of the Internet has revolutionized how all people access, store, share, and manage information. This emergence also offers leaps in transparency, responsiveness, efficiency, and customization of services for Department of Transportation (DOT) information users through enterprise information portals (EIPs). An EIP is a frame- work for offering and integrating information through a web-based user interface using a unified access point. Virtually any resource can be made available via a web interface. This breadth of information challenges even the most robust content management systems and strategies that enable agencies to ensure that the information they provide remains acces- sible, timely, and relevant while ensuring security and privacy. NCHRP Project 20-103 was undertaken to develop guidance that supports DOTs in estab- lishing and maintaining more sustainable EIPs. Sustainability is the ability of the enterprise to meet its present needs without compromising its ability to meet future needs. Although sustainability is often discussed in the context of funding resources, for this guidance, sus- tainability is a far broader concept that spans technologies, policies, procedures, approaches, and decisions that support both the quality and effectiveness of services currently offered while proactively and efficiently accommodating future growth through replaceability and resiliency. This guidance document offers insights and practices to enhance agency personnel’s understanding of the value, uses, design, and maintenance of EIPs as well as the design principles, management practices, and performance characteristics that will ensure that a DOT’s EIPs effectively and sustainably serve its users and the agency’s mission. The guid- ance includes best practices in EIP development and maintenance through surveying of information technology (IT) experts from transportation and non-transportation domains, feedback from DOTs on challenges and success factors for implementing EIPs, and industry trends in the use of EIPs. EIPs for State DOTs The most common functional areas for transportation EIPs include asset management, engineering and maintenance services, knowledge management and data governance, geo- graphic information system (GIS) and mapping services, operations and performance man- agement services, documents and library services, bid and contract services, IT services, and environmental services. It is important to note that each of these functional areas vary in services offered, accessibility, frequency of use, and the types of internal and/or external users. EIP users may range from DOT employees, DOT contractors, and private real estate or engineering firms to emergency responders, researchers, news media, and the public. It is s u m m a r y

2 Guidance for Development and management of sustainable Enterprise Information Portals also important to note that the content and user needs of EIPs are dynamic; they vary based on factors such as the time of year (e.g., managing snowplow assets) and reflect changes in data and resources available to agencies. Every state DOT hosts one or more web-based portals through which internal and exter- nal users conduct business and offer or receive services. An agency may maintain one or multiple portals based on the business needs and structuring of the agency’s IT. Larger DOTs are more likely to host multiple loosely connected portals, while other DOTs may offer a single portal for the entire breadth of users and services. Fourteen technical considerations came to the forefront among agencies as they devel- oped their portals and designed the architecture to support internal and external users and promote EIP sustainability. These considerations are the following: • Search and Discovery • Data and Metadata • Reliability • Business Service Adequacy • Performance • Maintainability • Front-End System Flexibility • Availability • Security • Scalability of Back-End Systems • User Behaviors and Patterns • Technical Goals • Future Growth • Funding Most industries use a mix of cloud and on-premise infrastructure because it increases portal sustainability by reducing the risk of IT infrastructure obsolescence and reducing the duration and frequency of IT infrastructure maintenance down times. Furthermore, cloud services offer a scalable, flexible, and on-demand set of IT capabilities. Cloud infrastructure design differs from on-premise infrastructure design; consequently, agencies should recog- nize the opportunities and challenges in the use of cloud services. Eight key considerations for the development and operation of sustainable EIPs that leverage cloud infrastructure are summarized in Table S-1. As noted, the mixed infrastructure approach would permit increased scalability and reli- ability in the EIP applications while allowing sensitive data such as acquisition documents to be kept in house. The recommended approach to providing application deployment and server configuration should be either a cloud-based configuration management service or an open source configuration management service used in combination with a container- ized architecture. The use of either an open source continuous integration tool, a hosted continuous integration service (SaaS), or a proprietary continuous integration tool is rec- ommended for the implementation of sustainable DOT EIPs. Next Generation EIP: Microservices Architecture Sustainable DOT EIPs will need to move away from monolithic and heavily integrated portal applications to more distributed hardware and software portal applications. The microservices architecture is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms. Each service is independent and may be written in different programming languages or may use different data storage technologies. Services are built around busi- ness capabilities and are independently deployable, with the potential for applying fully automated deployment capability, and require a bare minimum of centralized management. Microservices architecture is a more granular evolution of the service-oriented architec- ture (SOA). Its goal is to decompose a software application into elementary software com- ponents to facilitate agile development and deployment. Large companies, such as Amazon,

benefits found in service-oriented architecture, such as increased levels of reusability, higher interoperability between systems, and improved scalability, but also allows for dynamic reconfiguration of business logic by the addition of new, or the replacement of existing, event subscribers. Leverage Open Source and Existing Software as a Service (SaaS) Using open source software on cloud infrastructure or a cloud providers’ SaaS solutions is preferred to the use of commercial software solutions implemented on on-premise or cloud infrastructure. The rationale behind this choice is that most commercial software does not offer licensing and pricing schemes adapted to cloud infrastructure flexibility. Maintain a Looser Governance IT governance typically is designed to help maintain tightly integrated systems and attempts to centralize and standardize a recommended set of technologies. This approach is not ideal for any computing system, whether monolithic or distributed, as it imposes technologies that can make specific software components more difficult to develop, inefficient in operations, and even more expensive to maintain. A preferable solution would be to allow the use of the most adequate technology for each component, taking advantage of the efficiencies of different languages and code libraries. Consideration Summary Reduce the Risk of IT Infrastructure Obsolescence On-premise infrastructure is often composed of reliable and expensive servers or server pairs that are treated as indispensable, that is, they can never be down (mainframes, database systems, and web servers). Unfortunately, this infrastructure can become obsolete very quickly, which is problematic because most owners of on-premise IT infrastructure do not have robust annual budgets. Cloud architecture was created primarily to circumvent these issues; this architecture uses inexpensive arrays of less- reliable but redundant servers and relies on automation that requires less human intervention, strategically “routing around failure.” Scalable, Flexible, and On-Demand IT Capabilities On-premise IT infrastructure is typically sized to satisfy an estimated maximum demand. However, if demand exceeds the planned maximum of on-premise infrastructure, no scaling is possible to accommodate the extreme demand, which impairs system users’ ability to access real-time data. Cloud infrastructures are built as a pay-as-you-go service and as such can be turned up or down as desired or even turned off when not needed. For example, a sudden request for traffic information from drivers or the processing of a large amount of GIS data can be run quickly and processed at higher speeds without risking crashing servers or losing data. Reducing IT Infrastructure Operation and Maintenance Time Cloud infrastructures eliminate the need to spend millions of dollars to acquire equipment and spare parts, hire IT infrastructure management experts, and allocate often expensive real estate to run an organization’s applications. Organizational applications can be developed and deployed directly on the already running cloud infrastructure, which frees the organization to focus its resources on its applications rather than the maintenance of its own IT infrastructure. Fears related to off-premise hosting are unfounded, particularly because cloud services offer greater resiliency, security, and reliability than most in-house hosting of data. Use Distributed Software Architecture A distributed computer system is one in which the computing power is distributed across several servers connected through a network, and each component performs a specific task. Consequently, failure of a component is localized and independent, allowing software to be able to recover from failure. In comparison, centralized software, such as the software running on monolithic computer systems, offers a single point of control, but exposes a single point of failure and often cannot deliver the recovery capabilities that distributed computer systems offer. Use Layered System Design To further optimize sustainability, a three-tier architecture is recommended so that software modifications can be performed on a specific layer without affecting the others. These layers are a user interface layer, a business service layer, and a data layer. System Use an Event-Driven Event-driven architecture (compared to time- or request-driven) provides the Table S-1. Considerations for the use of cloud infrastructure.

4 Guidance for Development and management of sustainable Enterprise Information Portals eBay, or Netflix, have already adopted microservices architecture as their main architec- ture. Microservices can be cloned or decommissioned rapidly to respond to fluctuating user requests in real time. Microservices architecture patterns are intended to be scalable in three different ways: • Scale by decomposition into simple functions (microservices) • Scale by cloning microservices on demand • Scale by partitioning data across a cluster A microservices architecture pattern is also a more platform-agnostic approach to appli- cation development. Microservices architecture relies heavily on containerization, which allows microservices to be developed completely independently from each other using dif- ferent languages and libraries. Only the external interfaces need to be standardized across all microservices. While the microservices architecture pattern has many advantages, it also greatly increases application complexity. The microservices architecture essentially trades code complex- ity for operational complexity. To alleviate this increased operational complexity, most microservices architecture pattern implementations rely on automated services to monitor, deploy, and manage their clusters. Scripting approaches have also been developed to pro- grammatically control the configuration and behavior of microservices clusters and allow them to be rapidly modified or corrected. Toward Successful Adoption of New IT Concepts In moving an agency toward the in-house and cloud infrastructure paradigm, effectively and proactively addressing cultural, knowledge, and technical obstacles will mean the dif- ference between success and failure. State governments and their DOTs vary considerably in their program area priorities, organizational structure, and general culture. Each DOT should consider its own characteristics in implementing recommendations. The key mecha- nisms for successful change are the following: • Developing Awareness, Understanding, Value, and Acceptance of New Concepts. A fundamental element to adopting new EIP practices is obtaining buy-in from various levels of the DOT organization. This means finding ways to share information among technologists and those in other domains and across different levels of management. This can be accomplished by developing content that effectively addresses the interests of different audiences and identifying ways to overcome resistance to change. A single delivery of content is not sufficient for support of new concepts. Outreach should be continuing and effectively linked with the activities of the business and the value derived from innovation. • Establish Consistent but Flexible Governance Principles. With the evolution of a service- based infrastructure, the underlying data and services are independent and redundant. Thus, the focus of governance shifts away from managing hardware and software resources; for instance, in this new environment, nothing sits on the same server. Instead, the new focus is on managing user access to data and protecting data from unauthorized access. Whereas traditional governance prescribes a centralized approach such as a common pro- gramming language or service, sustainable EIPs will require more decentralized strategies in governance with increasing autonomy. • Develop the Workforce. People are the backbone of any DOT. Understanding of their knowledge, skills, abilities, roles, and responsibilities is critical to understanding how changes might be adopted over time. With new technological environments, there should

summary 5 be more proactive approaches to recruitment, training, and retention that actively engage human resources expert staff. Innovative agencies recognize the costs for enterprise sys- tems expertise. Outsourcing may cost less; however, it comes with significant loss of con- trol and difficulty downstream in managing and evolving services. • Carefully Consider Procurement Options. Within state DOTs, procurement policies are typically developed with traditional procurement of goods and services in mind. The procurement of cloud-based services, maintenance components, and other types of services-oriented architecture requires flexibility in the procurement process. However, in most DOTs, procurement procedures have not kept up with trends in the IT market and technologies. Engaging and collaborating with procurement staff is important to ensuring that needs are well understood and that contract vehicles are designed to keep pace with innovation. • Don’t Underestimate Complex Legal Considerations. Legal issues are among the most challenging and complex barriers to IT implementation because these issues require the input of stakeholders from outside the DOT. In addition, as each state has its own laws regarding data retention, public records, and intellectual property, it is difficult to come up with one blanket approach to mitigating the challenges. However, it is important that within each state, the DOT works to gain a better understanding of its own legal precedents and challenges. Legal considerations are likely to include requirements for data sharing, data storage, intellectual property, and backup format. The strategies for promoting sustainable DOT EIPs are similar in many ways to the strategies implemented in the adoption of intelligent transportation systems. These include identifying stakeholders and champions within the leadership and technical domains; demonstrating the business need for and value of IT change; tackling institutional fac- tors that inhibit change by designing and conveying persuasive messages; establishing and adhering to IT policies, processes, and procedures; and directing improvements based on evaluation of performance. DOT agencies collectively will benefit from sharing success stories and lessons learned along the journey to make DOT EIPs sustainable and highly valued. Experiences from the Field and Looking Ahead The transition to cloud-hosted EIPs entails changing culture, policies, and processes, a change that may not be successful when first attempted. States look to peers within state or city agencies (e.g., other departments such as housing or health), counterpart agencies from other states, and commercial sectors such as finance. The experiences of groups within three state DOT agencies bring to the forefront these changes in culture, policies, and processes: • The Statewide Traffic Analysis and Reporting System (STARS II) within the Texas DOT (TxDOT) transformed, through a 5-year journey, processes, policies, and analyt- ics, and required re-baselining of state-level performance metrics all for the betterment of operations through more accurate, precise, and comprehensive access to data. This transformation was made possible by the adoption of the cloud for data and service hosting. • The Utah DOT Data Storage and Governance Evolution. The Utah DOT chose to follow the state lead by transitioning data storage to the cloud, and thereby reducing costs by as much as 50%. Inspired by state and private healthcare practices in Utah, the DOT also chose to overhaul its data governance by adopting a decentralized strategy weaving good data practices and a team of data experts into the DOT business units to improve business unit value through more efficient data management.

6 Guidance for Development and management of sustainable Enterprise Information Portals • Snowplow Information on Demand. An example of a service migrated to the cloud, albeit in a traditional fashion, is the Virginia DOT’s snowplow on demand, an applica- tion with highly variable and extreme user demand. The migration to the cloud offers the bandwidth to service the extreme demand and removes the responsibility to manage and maintain hardware; however, it currently does not scale for demand automatically. This means the application is sized for maximum usage. This application could quickly be transformed to leverage cloud autoscaling capabilities, adjusting demand automatically in real time, and thereby only bandwidth that is used would have to be paid for. Among the three case studies, what is noteworthy is the timeline for success, the cost savings afforded from these undertakings, and the changes in processes and policies that improve the agencies’ operations to better deliver on their unique missions. A DOT’s EIP will need to move to an architecture where each of the services offered by the EIP is more autonomous (less dependent on the EIP framework) and more extensible. A more modu- lar and scalable architecture is preferable to the current tightly integrated or centralized EIP architecture to satisfy this requirement. Giving up centralized processes and embracing modularity and scalability will greatly increase the complexity of EIP management unless alternative EIP management methods are developed and adhered to, ensuring that modular DOT EIPs will be not only able to adapt to change efficiently but will remain capable of maintaining the quality and reliability of services. Indeed, migration to EIPs that leverage resilient IT infrastructure, microservices architecture, and decentralized governance will position DOT agencies to better serve the needs of both employees and customers.