Designing the Archive for SHRP 2 Reliability and Reliability-Related Data (2014)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

1As part of Project L13A, the research team successfully developed, tested, and released a web- based, interactive archive system to store data and information from Reliability and Reliability- related projects from the second Strategic Highway Research Program (SHRP 2): http://www .shrp2archive.org. The SHRP 2 Reliability Archive is designed to provide an open and accessible data hub. This hub creates a foundation that encourages additional transportation data research. Without the SHRP 2 Archive system, valuable data artifacts—including measured traffic data by various types of sensing equipment, structured analytic databases, spreadsheets, research reports, and other valuable transportation data—would be stored in scattered locations in forms that might not be publicly accessible. The SHRP 2 Archive resolves the scattered nature of data by creating a hub of data storage that is organized with built-in visualization tools for online analysis as well as downloadable structured data sets to support further research. By storing data in a consolidated and public manner, SHRP 2 has increased the likelihood that future researchers will discover data, thus increasing the value of the data and decreasing the cost of data acquisition for the col- lective transportation community. This structure also saves valuable resources for additional research by reducing the acquisition costs of time and funding that are associated with additional data collection. As a part of the initial preparatory analysis, the L13A project team reviewed the findings of the SHRP 2 prototype L13 project report and past work on data archiving systems and technologies. This analysis resulted in the selection of WordPress as the core content management system (CMS), on which the Archive system was built. The selection of WordPress was based on its simplicity, flexibility, and extensibility. The development of the Archive itself was an iterative and two-sided process involving the stakeholders or users on one side and the developers or the project team on the other side. An agile software approach was used to develop the Archive; using the methodology allowed for needs and requirements to evolve through collaboration between both sides. Through a series of phases, the L13A project team established a collaborative effort in which a subject matter expert (SME) group—consisting of the Technical Expert Task Group (TETG) members, future users, and experts—worked together to help the software design team identify practical user require- ments and design a system that can be operational for more than 25 years. The main steps for developing the Archive consisted of the following: • Identify user needs, in collaboration with key stakeholders and potential users of the Archive; • Design, test, and deploy a cost-effective, interactive, scalable, and robust archive system to store artifacts from SHRP 2 Reliability or Reliability-related projects; • Develop processes and procedures to collect, upload, and use artifacts; Executive Summary

2• Define a guideline to prepare artifacts for upload; • Discuss challenges and issues regarding operations and maintenance of the Archive; and • Provide technical information on the design and architecture of the Archive system. The Archive is now operating on the Amazon cloud service (http://www.shrp2archive.org). The system has been developed based on open source web technologies. The system is expected to host more than 500 artifacts collected from more than 30 Reliability-related projects. The approximate size of the SHRP 2 Archive is anticipated to reach about 1 TB. A detailed Help sec- tion describing system features and providing a step-by-step guide on how to use the system is available online under the Archive Help pages. As mentioned earlier, the SHRP 2 Archive is more than a data repository. It is a system that enables uploading of artifacts, searching with results arranged by list or map, and bulk and subset downloading of artifacts. Also, the Archive is a user-friendly toolset that facilitates visualizations of user-selected data and collaborations between multiple researchers. A summary of the SHRP 2 Archive system’s functionalities is provided below and shown in Figure ES.1: • Upload. The SHRP 2 Reliability Archive’s ingestion wizard allows Reliability project leaders to upload artifacts along with their related metadata and data dictionaries. The system catego- rizes the artifacts into two general groups: data sets and non–data sets (e.g., documents, com- puter codes, video, pictures). Artifacts submitted under each category need to meet certain requirements. Therefore, the producer of an artifact has to preprocess it before submitting it to the system. • Search and download. The Archive provides faceted and text search tools to help users look for artifacts. The text search feature enables the users to conduct content searches within artifacts and metadata. In addition, the faceted search tool allows users to explore the Archive’s reposi- tory. Users can filter the search results by selecting various related criteria. The system provides the search results on a map and in a list. Users can also search by project within a focus area. • Map visualization. The system has the capability to map the geolocation information of the traffic detectors provided in a data set. Thus, the geofilter and map view capabilities can help users explore sensor locations. • Data visualization. Archive users can explore and filter the content of a data set. They can make various two-dimensional (2-D) plots (e.g., lines, points, bar, and column) of any fields in the data set or a subset of a data set. The system also has the capability to plot different series on a common plot. Furthermore, users can save and print the plots for future use. Figure ES.1. SHRP 2 Reliability Archive system.

3 • Collaboration. Registered Archive users can comment on project pages and artifact pages. They also can rate projects and artifacts. To comment on Archive pages and rate artifacts, users have to be registered. • Administration and user profile. The Archive administrator is capable of granting access to users as well as adding, editing, and deleting site content. Through the administrator interface, the Archive administrator can also monitor the artifacts being ingested and moderate the comments being submitted by users. The project team hopes the Archive developed out of this study will contribute to extending the state of the art in the use and development of data repositories in the transportation community.