Development of Transactional Data Specifications for Demand-Responsive Transportation (2020)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

18 This chapter begins a more in-depth discussion of the general principles and considerations applicable to data specifications for software applications, as well as functional and technical factors relevant to a demand-responsive transportation transactional specification. The focus changes in this chapter from all stakeholders to those stakeholders who will actively use the specification and most likely be involved in some manner in the process of specification development, formal adoption, and subsequent governance. This focus on the stakeholders who will directly use the specification continues through Chapter 4. This chapter first describes core principles for designing the specification then discusses market considerations. Next, the specific functional requirements for a DRT transactional data specification are explained; a model of the trip lifecycle is presented to organize functional schema. Technical models for implementing a DRT transactional data flow are described next, and finally, a preferred technical approach for data communication of specification-compliant transactional messages is selected. Several key terms concerning transactional data specifications are frequently used. They are related but distinct. Whichever specific term is being used, readers should understand that the overall context is always one of data specifications that are relevant to transactional processes. • Specification—A set of documented requirements to be satisfied by a material, design, product or service; the act of identifying something precisely or stating a precise requirement. • Data specification—A set of requirements for the types of data and the format of the required data elements for a specific data domain, typically related to a specific functional area. • Transactional data specification—A data specification focused on enabling functional transactions that require the flow of data among two or more software systems. • Standard (specifically referencing technical standards)—An established norm or requirement, usually a formal document, that sets forth uniform technical criteria, methods, processes, and practices. Formal technical standards will typically have been ratified by an organization that represents the interests of the entities to which the standards apply. 2.1 Core Principles for Successful Specification Approaches The six core principles guiding the development of a successful specification are shown in Figure 2-1 and discussed in this section. • Sufficiency—Sufficiency means that (1) the specification covers all typical transactions that occur among parties; and (2) for each transaction covered, the specification incorporates Principles and Considerations in Developing the Transactional Data Specification C H A P T E R 2

Principles and Considerations in Developing the Transactional Data Specification 19 all the data elements needed for each party to accomplish that specific transaction. A speci- fication is sufficient if it is capable of handling all “use cases” that typically occur within its transactional domain; each “use case” describes a scenario in which two or more parties interact for purposes of transacting among themselves or exchanging information. • Simplicity—Simplicity in the context of a transactional data specification largely connotes the absence of complexity in the message typology and message contents. Ideally, message types are simplified so that no further subdividing of the message would also produce useful messages. At the same time, each fundamental step in the transactional process should be able to be accomplished via a single message (and when appropriate, followed by an acknowledging response). • Flexibility—A specification has flexibility when it can accommodate variations in transac- tional needs among participants. The same message types can support transactions in which only basic data elements are exchanged as well as those in which more detailed, albeit optional, data can be exchanged. • Adaptability—A specification has adaptability when it can be modified to meet new needs without being fundamentally altered. For example, if new message types are added to handle transactional situations that are new or different from prior practice, there should be no need to modify existing transactional interchanges that are not related to the new situations. In more technical terms, the changes should be “backwards compatible”: what currently works continues to work after the specification is modified. • Compatibility—A specification can be said to be compatible with existing practice when the types of data interactions it supports—encompassing data elements, types of data, functional reasons for exchanging data, and means by which parties exchange data— are already occurring and rely upon existing technical approaches and protocols broadly consistent with those defined in the specification. A specification is not compatible with existing practice if it includes a data transmission approach and/or many data elements not commonly used in current practice in the domain which is its focus. • Technical appropriateness—A specification is technically appropriate when it employs the computer languages, technical terminology, and technical approaches to data exchange that are in common use in the industry sector which is its focus, with particular reference to the software applications and computer systems of the major participants in that sector. 2.2 Market Considerations In addition to the importance of the specification’s fidelity to the core principles, it is impor- tant that specifications work effectively for the groups that will use them in their day-to-day activities and work well with existing technology. The research team identified two important market considerations that could affect the type of specification proposed: • Major DRT service providers and DRT technology suppliers must be supportive or at least neutral toward specifications. Sufficiency Simplicity Flexibility Adaptability Compatibility Technical appropriateness Figure 2-1. Core principles for the specification development.

20 Development of Transactional Data Specifications for Demand-Responsive Transportation • Specifications must build on trusted relationships and provide gatekeeping mechanisms to mediate collaborative system-to-system interactions among parties. Given their potential—and often actual—impact on technology suppliers and their customers, new specifications in particular are the opposite of business as usual. They require significant buy-in from the leading parties, particularly the technology organizations. They also must provide for the detailed information needed by diverse DRT providers so they are functional. The development process needs to include expertise and strong industry/sector understanding on the part of those directly involved. Market Is Supportive of or Neutral Toward Specifications Data specifications that are successfully developed and adopted must have the support of the key organizations that they directly affect. The organizations principally affected are: • DRT service providers that use the technology systems or components that are the focus of the specification (including public and private providers, private transit service contractors, taxi companies, and ride-hailing services); and • Technology suppliers that work directly with the technology systems, e.g., software applica- tions, hardware and other devices, computing infrastructure. Their products typically have functionality for booking, scheduling, vehicle routing, and/or dispatching. This also includes companies whose technologies are complementary to the core software systems, such as in-vehicle devices and fare-payment solutions. The public-sector organizations that fund public transit and human service transportation are also key organizations, but their overriding interest is that specifications exist rather than their specific form and details—hence, they are not a factor in this consideration. If service providers and/or key technology suppliers strongly support a proposed new speci- fication, this will significantly enhance the likelihood that agreement among participants can be reached on structure and details of the specifications. In interviewing key players, the research team found that the DRT technology industry is neutral about investing in development of specifications, with some in favor but also some large players that do not see the effort as in their interest. Providers, on the other hand, would generally like to have the benefits of transactional specifications but typically do not individually have the knowledge, resources, or organization to effect the change. Trusted Relationships and Gatekeeping Mechanisms In many industries, specification development and evolution are common and include indi- viduals from service and technology companies that are veterans of the process. This can greatly facilitate the achievement of a positive outcome. Unfortunately, that is not the case for the DRT sector, given the absence of prior experience with developing sector-wide specifications. When starting a new specification process with no prior history, the process itself will need to create the foundation of trust among participants. There may be an enhanced need for mechanisms that provide some assurances that technical and business collaboration will proceed in a smooth, business-like manner and lead to actual results. Third parties who are considered neutral and objective domain experts can be useful in launching the specification process. The use of such perceived neutral third parties and domain experts are examples of the gatekeeping mechanisms that are important to mediating collaborative interactions among parties. This will extend to the actual technical details of system-to-system interactions, and

Principles and Considerations in Developing the Transactional Data Specification 21 there will be a strong need to ensure that concerns such as data privacy and security are handled appropriately by the technical solution. In some cases, transactions can be routed through certified gateways, such as with standards-compliant payment processors. In other cases it is the trust among the parties themselves that serves as the gatekeeping mechanism. As such trust cannot be automatically assumed in a situation of developing new specifications, the ability to structure a process in which interactions among the parties create such trust becomes very important. Since a data specification typically requires changes to technology systems, the organizations whose systems are affected have an incentive to minimize resources required to achieve compli- ance with the specification. In practice, this means that compatibility with such factors as the vocabulary describing transactions; the data elements usually included in transactions; and the mechanisms for transmitting data among different applications/systems used by the major firms in the market represent the technical path of least resistance for advancing toward agreement on the many detailed elements of the specification. 2.3 Functional Requirements for DRT Data: The Trip Lifecycle In determining the scope of transactional data specifications, it is essential to consider functional requirements that must be supported by the software systems. The specification must support the trip from inception to completion—the trip lifecycle—and enable the different organizations involved in its booking, planning, and execution to share and view all relevant data about the trip during this process. The trip lifecycle is represented with five steps that are shown in Figure 2-2. Each of these five steps requires data to be exchanged by participants in the process. Some organizations require more data for the transactional process than others, but there is a core set of mandatory data elements needed to support each phase of the trip lifecycle. Step 1—Trip Booking (Reservation) In a trip reservation, data must be generated about the pickup and drop-off locations (addresses) of the traveler; when they wish to travel—which could be a requested pickup time or desired delivery time; if they have any special mobility needs (e.g., use a walker or wheelchair) or are using a mobility aid of some type; how many people are making the trip; and if there are any special instructions for the service provider or its driver, for example, “pick up customer at back door.” In addition to these mandatory data items, other data elements may be desirable or necessary for some organizations, such as the purpose of the trip or whether it is associated with a specific funding source. Step 2—Scheduling/Dispatching For trip scheduling, the organization responsible for delivering the trip will determine when the vehicle will arrive at the pickup location, generate an estimate of when it will arrive at the delivery location, and provide information on time windows associated with the pickup and Trip booking Scheduling/dispatching Cancellation (optional) Trip execution Data reporting Figure 2-2. Trip lifecycle steps.

22 Development of Transactional Data Specifications for Demand-Responsive Transportation delivery times. Later in the scheduling process, a specific vehicle will be assigned to the trip. The information about the trip will then be dispatched to the vehicle, or more specifically, some type of digital device in the vehicle capable of receiving data messages about the passenger trips assigned to the vehicle and displaying this information to the driver. Step 3—Cancellation (Optional) To cancel a trip, the only essential data elements are the time when the cancellation occurred and whether it is due to customer request or the inability of the operator to deliver the service. Step 4—Trip Execution In executing (delivering) a trip, the mandatory data elements are the identity of the trav- eler, the actual locations and times for pickup and drop off, how many passengers boarded the vehicle, what fare was paid, and the identity of the service provider, vehicle, and driver that handled the trip. It may also be important to know whether a wheelchair or other mobility aid is used. The real-time location of the vehicle (via GPS readings) during trip execution is also important information for the service provider, and can be used as an input to the functional ability to inform the customer of when the vehicle is likely to arrive at their pickup location. Step 5—Data Reporting In the final step, all pertinent data from the trip execution phase must be generated and sent to the entity that is responsible for the customer’s trip. In addition, the service provider must also communicate the cost (price) of the trip to the organization or individual responsible for the payment of the customer’s trip. The actual financial transactions associated with the trip, namely the payment to the service provider by the trip sponsor or individual responsible, are clearly important to participants in the system but extend beyond the operational elements of DRT services. As such, actual financial transactions are currently outside the scope of the specifications described in this report. 2.4 Models for Transactional Data Flow A transactional data specification is a document that contains the necessary information for software developers and technology providers to construct data messages that make it possible for two or more computer applications to interoperate. While the specification will typically have a precise syntax, with a defined message set and required and optional data elements for each message, there is more than one paradigm for how the applications/systems could interoperate. Although the specification syntax itself does not depend on which model is used, the technical approach to specification enforcement (i.e., validation of specification adherence) is strongly influenced by the application interaction paradigm used. There are several useful models for this type of application/system interaction. Three are explained briefly here, as they are the most directly relevant to a transactional data specification for DRT. Model 1—Request/Response In a request/response model, one computer software program—call it Application A—sends a data message to another computer software program—call this Application B—requesting

Principles and Considerations in Developing the Transactional Data Specification 23 that the latter perform some specific action(s) related to a business transaction or potential transaction. For example, on a stock exchange, the computer system of a financial firm— Application A—may send a request to the stock exchange’s computer system—Application B— to purchase 10,000 shares of Acme Widget stock at the current market price. Application B acts on the request—it attempts to match the potential buyer’s request with that of one or more sellers of Acme Widget stock to fulfill the order for 10,000 shares—and responds to Application A with confirmation that the request has been fulfilled. Alternatively, if there are no available sellers of Acme Widget stock, Application B will respond that the request has been unsuccessful. In this model, Application A cannot make any assumptions about the status of its request until it receives the response, which can be as simple as merely confirming that the request was received. For example, it might be the case that the protocol defined by the specification is for Application B to first acknowledge the request, namely that Application A’s buy order has been properly received by the stock exchange computer system. Application B would then in a separate message respond with the status of the request, for example, purchase request fulfilled or unsuccessful. Application A must wait until it receives acknowledgment that its message was properly received, and take appropriate action if it does not receive such an acknowledgment, until it can update the message status to “delivered” and then expect a response directly regarding the disposition of its request. In this model, each of the collaborating software applications must manage the dialog between them. The advantage of the request/response paradigm is that both applications can essentially guarantee that they have a consistent view of the status of messages. Only by receiving a response can the initiating application be sure that its message has been received and acted upon and what the disposition of its request was. The request/response approach does not necessarily imply a direct point-to-point data com- munication relationship between two applications. A message exchange or transaction broker could mediate between two applications and serve as the clearinghouse for message notifications and transmission; it would use the request/response paradigm to communicate with the endpoint applications. In this example, there could be a third application, Application C, which operates for another financial firm and which also communicates with Application B (the stock exchange). Application C might send a message asking if there are any current potential buyers for Acme Widget stock. Application B can then serve as the transactional intermediary (which is essentially what a stock exchange is) between Applications A and C, sending messages to each to enable them to bring the transaction to a conclusion; each interaction in the process involves a request/ response sequence of messages. An example relevant to DRT would be that of a trip exchange, such as has been developed in Denver and is discussed in Chapter 3. In the trip exchange situation, a DRT provider’s technology system—Application A—would send an unfulfilled trip request to the exchange— Application B—with the objective of having another DRT provider actually provide the service for the trip in question. Using a request/response approach, the trip exchange would inform Application A via a message that it had successfully received and processed Application A’s message, enabling the latter to know that there was no issue with the data transmission nor the contents of its message. If Application A did not receive a confirmation, it would be pro- grammed to send the message again. Another DRT provider’s technology system, Application C, would then be able to make a request to the trip exchange to send it any new trip postings since its previous request, and the trip exchange would respond with the trip posted by Application A (and any others that might have been posted). And this process would continue with requests and responses from the interacting software applications.

24 Development of Transactional Data Specifications for Demand-Responsive Transportation Model 2—Publish and Subscribe In a publish and subscribe model, an application registers with another system/application— similar to an individual signing up for an online service that requires them to provide an email address and mobile phone number as part of their account—and subscribes to a service that informs it of events of interest. The events can be of specific interest to the external application or more general in nature. For example, an individual with an online account for a national news service might wish to be notified only of the posting of new stories about the city they live in, whereas other account holders might be notified of every new posting. It is the responsibility of the external application to determine what to do when it is notified that an event of interest has occurred and it has received the data about that event. For example, consider a situation in which a trip exchange exists that enables service providers and other organizations that have DRT service requests they cannot fulfill themselves, to buy service from another service provider with available capacity. The external DRT software applications can post trip requests to the trip exchange that are “visible” to any other software applications capable of connecting to the trip exchange (this bears obvious similarities to the stock exchange situation described previously). Each trip request is specific to a service zone in the region in which the trip exchange operates. Using the publish and subscribe model, the trip exchange would send a message to all sub- scribers to a particular service zone, notifying them whenever a new trip request for that zone is posted to the exchange. (Additional criteria could also be used to qualify the entities to which the notification is sent.) Depending on the design of the notification service, it might simply notify the external systems of the existence of a new trip—alerting those systems that they should request the new trip information—or it might include the actual trip request message from the originating system—that message being in the transactional specification format. In the latter case, this broadcast functionality would be a core feature of the trip exchange’s publish and subscribe approach. The advantage of the publish and subscribe model is that each participating software appli- cation is only notified of, or sent the messages for, events in which it has interest. In the absence of such a notification function, the external applications would need to periodically interrogate the trip exchange (or other source of potential transactions) and determine if the trip requests are ones they are already aware of, or new requests. It can easily be imagined that this could become quite complex if there are a large number of trip requests whose status is frequently changing. With the publish and subscribe model, an application knows that only new or updated requests are published—and will be delineated as such—and it becomes simpler to manage what could be a large number of events of interest. The publish and subscribe model is robust and has been used in many large-scale data inter- change systems, such as for financial markets. It is particularly appropriate for exchange-type systems in which there are many transactions (or opportunities to transact) of potential interest, but different applications are focused on different subsets of those events. A brokerage approach— essentially an exchange-like hub which also makes transactional decisions such as which provider to purchase a trip from (as opposed to merely facilitating transactions in which the endpoint applications are the decision-making points)—could also make good use of the publish and subscribe model. Model 3—Open API Framework An application programming interface (API) is a software module that enables external applications or software systems to interoperate with the software application providing the API.

Principles and Considerations in Developing the Transactional Data Specification 25 Over the past 25 years, APIs have become the most common method for business applications to communicate with external software systems/applications. An open API is simply one in which the technical details of the API are published and available to a large community of other systems. Open APIs are not the same as open source software. There may be certain restrictions on what type of entities qualify to use the API, and those entities are likely to be required to agree to terms of service, but in general an open API provides the external world with the ability to transact in some fashion with the host system/application. With an open API approach, each external application is fully responsible for managing its interactions with the system/application that provides the API (i.e., the host system). The host system API makes possible the transactional data flow, typically via a published set of function calls (i.e., message requests) that external applications can invoke to initiate a specified action or to retrieve certain types of information. It bears emphasizing that while the host system API defines the available functionalities, it does not typically manage transactions for the external systems. A directly relevant example of an open API that is heavily used is the Uber platform for ride-hailing services. Uber publishes the specifications for its ride-hailing platform’s API services, which provide external systems with the capability to request and manage trips and obtain information about those trips. (Uber has recently restricted access to those that can use its API.) In essence, all of the transactional services directly available to a consumer via the Uber app on their smartphone are also available to third parties via their own software applications: trips can be requested, estimated vehicle arrival times can be obtained, the identities of the vehicle and driver assigned to the trip are available and can be sent to the customer, and so on. The major advantage of the open API approach is that a large ecosystem of applications can interoperate without any formal structure for coordination (beyond the simple permissions to use another software system’s API). The obvious disadvantage is that in the absence of a universal API for a particular industry or sector, technology providers will find it necessary to support many external APIs in order to interoperate with actual or potential transactional partners. In fact, this can make common data specifications difficult if not impossible except on the terms of an entity that dominates the field, such as Google with its Google Maps API. One further aspect of interest concerning open APIs is that they impose no formal structure on how each external application manages transactional processes. This is a strength, as it provides a great deal of flexibility for the external application in how it implements its own transactional approach, within the constraints of the functions and data made available via the host system’s API. However, each host system and external application may have a different transactional paradigm, meaning that across many such applications the view of transactions is unlikely to be identical. If more than a few applications are involved in the transactional ecosystem, this is likely to result in barriers to sector-level transactional specifications, as well as the necessity of complicated software solutions to enable transactional data exchanges among multiple applications. These three models are summarized briefly in Figure 2-3. 2.5 Recommended Approach for Specification Transactional Flow Given the relatively undeveloped status of transactional data interaction in the DRT sector, it is prudent to begin with a specification implementation approach that is both agnostic—that is, does not build on the transactional data practices of a dominant entity—and prescriptive.

26 Development of Transactional Data Specifications for Demand-Responsive Transportation The latter means that each step in the process is well-defined and that all entities using the specification must adopt the same approach in their external-facing transactional interactions. Moreover, given that this is the first step toward a common transactional data specification, the approach should be relatively granular and support data interchanges that are atomic, meaning each transactional step is self-contained. These considerations point toward the use of a request/response approach as the core par- adigm for managing transactional interactions at this stage of specification implementation. While this is a somewhat older style approach for system-to-system data interaction, it has the advantage that it enforces certainty in transactional data flow, verifying that each step in the process has in fact occurred. Moreover, the request/response approach has proven itself over many years in real-life conditions in the demand-responsive transportation sector in the Nordic countries. It is also the transactional data backbone of the most advanced multi-participant DRT system in the world, that of FlexDanmark in Denmark. Six regional public transit entities, more than 500 trans- portation service providers, and many local governments and healthcare organizations use this technology approach to provide up to 24,000 trips per day. The approach relies on the SUTI standard (introduced in Chapter 1 and discussed in some detail in Chapter 3) and embodies the request/response model of transactional data interchange. In addition, the request/response paradigm is flexible; it can coexist with the use of publish and subscribe models of transactional data flow. This means that both bilateral, direct data exchange between two applications, and multilateral, in which an exchange-type mechanism enables multiple applications belonging to DRT service providers and trip sponsors to partici- pate in transactional data flows, application interactions can be supported with an appropriate system design. The request/response model can be used for the former, and the publish and subscribe model can be used for the latter. In fact, the trip exchange that has been developed in Denver uses elements of both approaches, as is discussed in Chapter 3. 2.6 Summary This chapter described the core principles and considerations required for development of a transactional data specification. Data specifications should strive to be as parsimonious as possible (simplicity) while maintaining flexibility. Such flexibility is important, as the speci- fication must be compatible with existing data systems but also meet the needs of different Model 1: Request/Response •How it works: Request with response from receiver •Advantage: Clear sense of status of request for both sender and receiver •Disadvantage: Each party must manage every step in data exchange process Model 2: Publish and Subscribe •How it works: Applications subscribe to receive requests that meet certain criteria •Advantage: Helps manage large volume of potential requests •Disadvantage: No explicit model for transactions Model 3: Open API •How it works: Software module enables external applications to interoperate with the application providing the API •Advantage: Large number of applications can inter- operate •Disadvantage: Need to support many APIs Figure 2-3. Three models for transactional data flow.

Principles and Considerations in Developing the Transactional Data Specification 27 suppliers and users today and in the future. Market conditions and technical functionality that will affect the development of a transactional data specification for the DRT industry were also discussed; industry buy-in and governance considerations will be critical to the overall success of development and adoption of a specification for DRT transactional data. Finally, different models for supporting data interaction for transactional purposes were considered, and the request/response model selected as most appropriate for the initial implementation of a transactional data specification.