Developing an ITS Technology Web Portal for Transit System Leaders (2016)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

21 Chapter 3 :  Findings and Applications (Phase 1)  This chapter discusses the findings for the following outputs of the Phase 1 effort:  Concept of Operations (ConOps) including system goals / objectives, stakeholder roles and responsibilities, and operational scenarios  Systems Requirements (SysRqts) including standards, templates, and procedures for content management  Technology Alternatives Analysis (TAA) / Selection of Tools  High-Level Design (HLD) including wireframes and workflow, data model for information cataloging and storage, and presentation methods  Test plan and procedures  Preliminary development of a formal transition process describing technology transfer methods (in collaboration with APTA)  Recommendations for Prototype Development Concept of Operations  Current Environment  This section describes the stakeholders who will use the Portal and the environment for which the System will be developed. The stakeholders will be exclusively APTA members or staff, and the System will be hosted on APTA’s web environment. The details of the current environment are described below. Stakeholders  The stakeholders who will interact with this System include APTA members and staff. Specific stakeholder categories are described in Table 3-1. Table 3-1. Current Stakeholder Category Descriptions Stakeholder Stakeholder Description APTA Member Person who is a registered APTA member. APTA Research and Technology (R&T) Committee Member “The committee’s objective is the advocacy of Research and Technology as a means of enhancing the performance of all aspects of the public transportation industry.” [2] APTA R&T Tech Committee Portal Subcommittee Member The R&T Committee Tech Portal Subcommittee members who are responsible for reviewing content posted to the System. APTA Information Technology (IT) staff Technical staff that are employed by APTA to manage their information technologies including their web sites. This also includes any support staff outsourced by APTA to host their system. May 2016

22 APTA SharePoint Environment   The web portal was designed and tested to operate in the APTA SharePoint environment. This environment was updated in 2012-13 and is currently composed of the following:  Cloud hosting of 2010 SharePoint Enterprise including backup, security and support services  SQL Server 2008  Expandable storage capacity  Web Server (for web site)  Use of multiple browser types compatible with Internet Explorer 7 to current IE version; and current versions of Chrome and Firefox  Coding capability: C++, SQL and Visual Basic  Adherence to APTA policies including Security and Intellectual Property  APTA User Security functionality (through APTA’s existing application – AMS) administered by APTA User Needs  The user needs provide an expression of the end users’ operational needs that can be met by the system functionality and information flow into and out of the system. The needs express what the system needs to do from the user perspective. These users are described by the stakeholder descriptions above. The user needs are:  Provide one-stop resource for APTA members to learn about new and emerging technologies applied to transit ITS.  Ensure that the resources stored in the System: o Are easily discoverable by users o Are current o Show their applicability to transit users including transit system leaders and non- technical transit professionals o Comply with APTA content policies (TBD)  Provide an interactive environment for industry discussion and exchange on new and emerging technologies  Provide guidance tools for APTA members to post content on ITS technologies in a standard format to the System. The tools should: o Support standard multimedia formats for posting to the System o Ensure the proper attribution and identification of the posting o Follow APTA policies on submittals hosted on APTA web sites o Provide a template for submitters to post their content  Enable the content to be managed by APTA R&T Committee Tech Portal Subcommittee (hereafter called the Review Committee) o Implement tools for the Review Committee to review submitted content. o Allow for the Review Committee to delegate posted content to industry experts. May 2016

23 o Provide event alerts to the Review Committee. Alerts may include when new or revised content is posted, when stored content should be reviewed periodically, and messages from users and reviewers. o Ensure that new Review Committee members can be assigned or removed from the Review Committee access privileges.  Develop a System that minimizes operations and maintenance costs and resources.  Ensure compliance with APTA Tech Portal Governance Processes (as described in Appendix A). Stakeholder Actor Roles & Responsibilities  The roles and responsibilities of the key stakeholders who interact with the portal are described below. The table includes a short description of the stakeholder roles and a bulleted list of their responsibilities. Table 3-2. ITS Tech Portal Actor Roles & Responsibilities Roles Responsibilities Content Contributor (CC) Portal user who creates and submits content to the portal.  APTA member  Submits content to System  Complies with and signs on with Portal Policies form  Owns or has the rights to submit content  Completes Portal “template”  Revises content per Review Committee request  Accesses content on the portal Review Committee Member (RCM) Members of the APTA R&T Committee Tech Portal Subcommittee who are responsible for approving or rejecting content (and may review content from time to time).  APTA R&T Tech Portal Subcommittee Member  Participates in review process per Governance Rules Content Reviewer (CR) Person designated by the Review Committee to review content on a technical and/or quality basis. Multiple reviewers may be assigned to review one piece of content.  APTA Member (may or may not be a member of the Review Committee)  Reviews and completes reviewer assessment form for designated post. Content Reader/ User (USR) Person who uses the portal to gain information. Searches for a given topic.  APTA member Content Administrator (CA) Individual with technical control over portal content.  Review Committee-designated content administrator (e.g., chair) or APTA IT Staff  Removes or changes the status of a posted resource  Provides (or removes) access/privileges to Content May 2016

24 Roles Responsibilities Reviewers and/or Review Committee Members. System Administrator (SA) APTA staff person(s) who administers the System.  Administers system control over the Portal including creating, revising and updating SharePoint configuration  Implements system lifecycle changes to SharePoint and associated presentation and back office systems. Each role is assigned certain privileges. The privileges are granted by the system and maintained by the system administrator. The privileges are defined as follows:  Content Reader: Read  Content Contributor: Edit  Content Administrator: Full Control For example a Content Administrator will have “full control” of the content management area whereas other roles would be limited privileges in the content management area as listed above. Operational Scenarios / Use Cases  The Portal needs will be driven by the lifecycle processes for sourcing, managing content, discovering, accessing, and collaborating on the content of the System. To this end, a series of Use Cases were developed to identify the key operational scenarios and high-level functional requirements of the System. A Use Case is a set of descriptive scenarios that cover the range of services in which a user interacts with the System. The operational scenarios are told in a narrative format in order for the reviewer to understand how a specific user might interact with the system. The lifecycle flow is illustrated in Figure 3-1 and the Use Cases that describe the lifecycle are listed in. These Use Cases are described below. Table 3-3. Use Case Descriptions Use Case Description Content Submission How content is created/contributed and uploaded Content Review and Acceptance How content is reviewed, including the conditionally approved process Content Publication How content is published Content Discovery How content is searched and used Content Access How content is accessed (presented and downloaded) from the system Content Administration How content is updated and removed or archived May 2016

25 Figure 3-1. Flow of Content Use Case: Content Submission  This use case describes how content is created/contributed and uploaded by the Content Contributor. Several alternative scenarios are described for this use case. They include content postings that are approved, conditionally approved, and rejected by the Review Committee. The scenario also includes a scenario where the committee has reviewed “dated” content that is still relevant, but may need updating (e.g., a paper on mobile technology). Table 3-4. Scenario List: Content Submission Scenario Title Description Approved Content Content which is approved for publishing Conditionally Approved Content Content which is only approved for publishing after changes are made Rejected Content Content which is not published Update Content Approved and published content that is dated and needs updating by its author or another contributor May 2016

26 Preconditions  Person accessing Content Post page must be an APTA member. Person posting content must own or have rights to submit content. Person logs on to APTA site and navigates to Portal pages. Scenario: Approved Content  Perspective: Content Contributor  Michael is a Senior Project Manager at a large transit agency who presented information to his senior managers on Connected Vehicle technology to support transit signal priority. Because of his success in clearly presenting the materials, the General Manager suggested that Michael submit the materials to the APTA ITS Tech Portal. Using his APTA logon, he accesses the Tech Portal by logging on to APTA’s web site. Navigating to the Portal content submission area, he follows the step-by-step instructions to submit his content. He fills out the Submitter Identification information (name, title, organization). Then he completes the Content Identification information (Topic Type, short summary, keywords). He then agrees to APTA’s terms and conditions for posting content, which includes that the submitter has the rights to distribute the content being submitted, and is told he will receive notification within 60 days if his content is approved. In 45 days, Michael receives an email saying the content was approved. Content is then published and can be discovered and accessed (e.g., downloaded). Scenario: Conditionally Approved Content  Perspective: Content Contributor Jessica works for a transit agency and is an expert on transit signal priority. She would like to post a brief paper on the topic. She creates a document without first visiting the Portal. Using her APTA logon, she accesses the Tech Portal by logging on to APTA’s web site. Navigating to the Portal content submission area, she follows the step-by-step instructions to submit her content. She fills out the Submitter Identification information (name, title, organization). Then she completes the Content Identification information (Topic Type, short summary, keywords). Finally she agrees to APTA’s terms and conditions for posting content, and is told she will receive notification within 60 days if her content is approved. Two weeks later, Jessica receives an email from the Review Committee saying that her content contained too many references endorsing a specific technology vendor. Per their instructions, she makes the necessary changes and resubmits her content via the same web process as before, with her original form data prefilled. One month later, she receives an email saying her content has been approved. Content is then published and can be discovered and viewed or downloaded (accessed). Scenario: Rejected Content  Perspective: Content Contributor Chuck works for a transit consulting firm that does systems engineering and he wants to post content about real-time information for transit to the Portal. Using the Portal template, he prepares a presentation that mostly discusses his Real Time application. With his APTA logon, he accesses the Tech Portal by logging onto APTA’s web site. Navigating to the Portal content submission area, he follows the step-by-step instructions to submit his content. He fills out the May 2016

27 Submitter Identification information (name, title, organization). Then he completes the Content Identification information (Topic Type, short summary, keywords). He then agrees to APTA’s terms and conditions for posting content, and is told he will receive notification within 60 days if his content is approved. In 45 days, Chuck receives an email saying his content was rejected and the reason for the rejection. Scenario: Update Content  Perspective: Content Contributor Jennifer has agreed to update a resource on Mobility Technology for the Portal. She is sent a link to the source information contained in the Portal on the subject, as well as related information that is referenced by the source information. She logs in with the access information provided by the Content Administrator, and although this was a requested assignment by the Review Committee, she must follow the same procedures as someone submitting new content. After completing her Identification and Content Identification information, including approving the APTA Terms and Conditions, Jennifer edits the content and submits the updated information. She receives the standard email indicating that she will receive notification within 60 days if the content is approved. In two weeks, Jennifer receives an email saying that the content was approved, instructions for updating the content, and a date when the update is due. Use Case: Content Review and Approval Processes  The Content Review and Approval Processes are outlined by the APTA R&T ITS Tech Portal Review Committee Governance procedures. The scenarios for the use case include:  Assign content and change status of content (provide access)  Content assessment review (new postings) Table 3-5. Scenario List: Content Review and Approval Process Scenario Title Description Assign Content Review The Content Administrator changes the status of content (e.g., review, accept, provisional accept, reject, update) and provides access of content to others (e.g., reviewers). Content Assessment Review The Content Reviewer receives a request to review Portal submittal, reviews and submits recommendations on the content. Preconditions  Content was submitted and is pending assessment. Scenario: Assign Content Review and Change Status of Content  Perspective: Content Administrator (and Review Committee) Linda, the Review Committee chair, convenes a meeting for the ITS Tech Portal Review Committee after receiving five alerts that people posted resources to the Portal. She logs into the Content Administration page and sees a list of five posts. After reviewing the identification and content information, and determining that all five submissions are complete, she provides access to her committee members. They receive an alert with a link to the documents that they will review at their next meeting. May 2016

28 During the meeting, the committee decides that three of the submittals are either not complete or too vendor specific. Allison, a Review Committee member, volunteers to review one of the documents, and the remaining document is assigned to Tracy, a current member of the R&T committee. The Committee determines the next meeting date and the date by which the assessments are due. After the meeting Linda logs onto her Administrator account and assigns reviewer status for the first paper to Allison and includes a due date to the assessment; next to the second paper, Linda browses through the email addresses of APTA members to assign to the paper. Linda finds and assigns Tracy’s email to the paper along with the proposed due date. An email is sent to Allison and Tracy with a link to the document and an assessment form that they need to complete. After the assessments are received, Linda, the Content Administrator, again receives alerts that the assessments are completed. She logs into the Content Administrator page and sees the completed assessment forms. She provides access to these forms to the Review Committee members to review for the next meeting. The first paper that Allison reviewed on Surveillance Technologies was deemed “brilliant.” Allison checked the box that the resource should be approved and also recommended (on the assessment form) that the resource should contain additional keywords and topic types (as listed in the Content Identification Information submitted by the Content Contributor). On the other hand, although the resource was technically relevant, Tracy only “provisionally accepted” the second paper on “Here I Am” aftermarket Connected Vehicle devices, indicating that it needed a quality review. After the meeting, Linda asks her deputy, Henry, to change the status of the documents. Henry also has content administrator status, so he logs on to the administrator’s page and pulls up the list with the pending submittals (Pending Content page). He checks the box that indicates the Surveillance resource has been approved. When he checks the “provisionally approved” box next to the Connect Vehicles paper, an email is generated to the Content Contributor. The email includes references to the paper and formal language on the status of the paper. Tracy’s comments on his recommendations for acceptance are automatically pasted into the email. In addition, he is provided instructions about how to update his paper as well as the due date by which the updates must be made. Henry has a chance to review the email before it is sent to the Content Contributor. He does so and then sends the email. Scenario: Content Assessment Review  Perspective: Content Reviewer [Approved] Linda, the chair of the review committee, asked Allison to review Michael’s submission. Allison receives a formal email inviting her to assess the paper with a link. She could also log on to APTA site and access the Content Assessment page. She logs in to the portal and navigates to the Assessment page. Although a committee member, Allison can only access Michael’s content for her assessment. She views the content, and completes the feedback form. The feedback form may be saved before it is completed, but it must be completed prior to it being submitted. The paper is brilliant, so she writes her assessment using the assessment criteria on the form and checks the approved radio button. The feedback form may be saved before it is completed, but it must be completed prior to it being submitted. The assessment form also has a section that asks whether additional keywords or topic types should be added to the ones checked by the Contributor. After reviewing the list, Allison determines that the May 2016

29 technology could be used for additional transit-related solutions and that additional keywords should be added to tag the technology. After completing the assessment, she submits it and it is available at the Administrator’s page. In addition, the Content Administrators are informed via email that an assessment was submitted. [Rejected] Linda, the chair of the review committee, asked Frank to review Gabe’s submission. Frank receives a formal email inviting him to assess the paper (a link to the paper is included in the email message). He could also log on to APTA site and access the assessment page. He logs in to the portal and navigates to the Assessment page. Even though he is a committee member, he can only access Gabe’s content for his assessment. He views the content and completes the feedback form. Frank doesn’t have time to complete the assessment form in one sitting, so he saves the form and later in the day logs back in to complete it. The content is not complete, nor is it well written. So Frank decides to reject the content. He completes the assessment form and flags it as a rejection. In addition, he writes a summary statement that will be sent to the Content Contributor. After completing the form, he submits it. An alert is sent to the Content Administrator that an assessment was submitted and the assessment is made available to the Content Administrator through the Administrator’s page. In addition, the Content Administrator is informed that an assessment via email was submitted. Use Case: Content Publication  The Content Publication Use Case describes how the content, once accepted, is stored and categorized to allow for user access. Table 3-6. Scenario List: Content Publication Scenario Title Description Publish Content How content is stored and included in a catalog (e.g., made ready for discovery) Preconditions  Content is assigned the status of “Approved” but does not yet have the status of “Published.” Content includes one or more “topic type(s)” and “keyword(s)” to allow for cataloging. Scenario: Publish Content  Perspective: Content Administrator Dave is designated a Content Administrator with the responsibility of publishing approved content. After the Review Committee votes to approve Michael’s content submission, Dave logs in to the portal using his APTA account. He pulls up the Pending Content page, and clicks on Michael’s submission. He clicks on the “publish content” button and is brought to a confirmation screen, where he has the option to set parameters including an expiration date when the content should be automatically archived, an alert date to remind the Committee to review the document, and a comments section for any comments the Committee may have made to be publically displayed. He then submits the form; keywords and topic types are processed and a concept map of the technology is now displayed. Dave has a chance to accept or edit the concept May 2016

30 map and links on the page. Once he reviews the resulting page, he can publish the results to the site. The resource is now discoverable and viewable by Content Readers (users). Use Case: Content Discovery  The discovery use case includes scenarios for Content Readers (users) to use a drop-down menu to access an alphabetical list of publications, browse a list of topics, or search for predefined keywords (or topics) that are included in the System. When submitted, a concept map with available resources is displayed which can be viewed by the Content Reader. Table 3-7. Scenario List: Content Discovery Scenario Title Description Browse Topic Displays a list of resources and publications available on the System. Search for Keyword Provides a search capability of the resources associated with a keyword available to view. Result displays the resources available to view related to the keyword. Since a resource may be assigned more than one keyword, multiple keywords may be included to drill down into finer detail. Preconditions  Content is “Published”. Content “topic type”(s) and “keyword(s)” are cataloged. Scenario: Browse Publication List  Perspective: Content Reader Kim just heard about the ITS Tech Portal and signs into to her APTA account, finds the Portal page, and quickly opens the Discovery page. She views two main ways of finding information – a search bar and a set of criteria to query available resources. She decides to look through the criteria and sees that she can refine the list of resources by several criteria, such as publication date, title, author, or a pull-down list of keywords. When she selects the topic “Connected Vehicles,” the screen displays one or more links to resources. Several resources identify Connected Vehicles as a topic. The information shows her the Portal content by topic. The result displayed on her screen is a list grouping links by connected vehicle technologies that are on a transit vehicle. When the User clicks on the “Here I Am” (HIA) Device link, information appears that explains the technology and its use by transit vehicles, transit solutions/systems that can use the technology and existing and emerging standards that apply to the technology. Kim sees a brief description in the technology frame on the website that summarizes the technology. In addition, there is a link to a page on HIA devices. The technology frame includes information that summarizes the standard developed for the HIA device. More links are provided to further explore related information that may include other devices (such as lateral, forward, and rear- facing detection sensors, GPS, odometer, and gyroscope) that are connected to the HIA on board the transit vehicle. Scenario: Search for Keyword (and requesting additional content)  Perspective: Content Reader May 2016

31 Tom logs onto the APTA web site because wants to know more about the next generation GPS and how it might improve transit operations. He goes directly to the search menu and searches topics for bus operations; he refines his search by listing GPS under keywords. No content is posted for the portal on this topic, but he sees that there is a link to an article that was posted on the site related to the next generation of GPS satellites. The article indicates that the next generation GPS accuracy and reception will improve. He submits a question to the site requesting information on how the improvement to next generation GPS will help transit operations. The question is posted to a “request” list for the R&T Committee Tech Portal Subcommittee to use to solicit content. Use Case: Content Administration  This use case describes how content is maintained over its lifecycle including archiving, updating and removing content. Table 3-8. Scenario List: Content Administration Scenario Title Description Content Archiving How dated content is marked as out of date while still remaining on the portal Content Removal How content is removed from the portal Content Updating How content that needs updating is updated Preconditions  Content is stored in the Portal and may become outdated. Scenario: Content Archiving  Perspective: Content Administrator Dave is the chair of the Review Committee. Every month his Committee receives notification of all content to be reviewed, typically one year after posting. A Content Administrator may view the flagged content from the Content Administrator’s page. An entry on AVL systems is flagged for review. At its next meeting, the Committee decides this content reflects old technology and decides to archive the content. Dave goes to the Content List and clicks the “archive” button. He is asked to confirm that he wishes to archive the item, which he does. The content is no longer viewable to Content Readers via Discovery, but can be viewed as archived material by the Content Administrators. Scenario: Content Removal  Perspective: Content Administrator APTA receives an email stating that some recently submitted material contains copyright violations, which they were unaware of during the review. The matter is forwarded to the Review Committee chair for the Committee to verify. Dave, the chair, uses the Content List to view the content. He clicks the “remove” button, which brings up a short form to confirm this. The content is deleted from the portal and a record of this transaction is logged. Scenario: Content Updating  Perspective: Content Administrator May 2016

32 Every month Dave receives notification of all older content to be reviewed, typically one year after posting. Content on Mesh Networks for Transit Signal Priority (TSP) systems is flagged for review, as it is now one year old. The content refers to a project that is being implemented; at this point, the project should be completed and the content updated to reflect the change. At the last meeting, the Committee directed Dave to request that the Contributor update the content. Dave opens the Content List, clicks on the “Email Contributor” button and generates an email on behalf of the committee requesting that he update the content. He also enables the content to be updated (e.g., view and write enabled) by the Contributor. By the next meeting, the contributor has made updates to the Content. Dave brings the content to the Review Committee, someone is assigned to review the content, and it is approved. When Dave goes to publish the content, first he archives the old content, and then he publishes the updated content. (Note: archived content is saved and may be viewed by a content administrator.) Perspective: Content Contributor Steve receives an email from the Review Committee requesting an update to content he posted on the Mesh Network TSP last year. He logs on to the portal, clicks on the “Submitted Content” menu, and clicks the article. He sees the “post an update” button is enabled, so he clicks on the button where he can edit the existing descriptions or upload new ones. He completes his update and submits the content. He receives an email confirming his submission. After six weeks (after it is reviewed by the Portal Committee), he receives an email that his content update was accepted. System Requirements  This section describes the detailed requirements for the ITS Technology Web Portal for Transit System Leaders General System Description  System Overview  The ITS Technology Web Portal is a one-stop environment that stores resources about new and emerging technologies applied to transit. Portal content is reviewed, approved, and managed by the Review Committee members. When approved, the content is published and made discoverable to any APTA member who can then view, comment on, or download the content. The operations of the portal will occur through four distinct functions, which are summarized in the table below. Table 3-9. Portal Function Overview Function Name Description Manage Storage The Storage function stores all the resources, artifacts, documents, and event lists that are used by the System. It stores provisional and approved content, resource status, resource identification, catalog services, user lists, and other key information needed to manage and categorize resources. The Storage function also contains functionality for Content Contributors to post resources and multimedia using a guided tool. May 2016

33 Function Name Description Manage Content Content Management functions enable the Review Committee to implement the content review processes. The processes include collecting, alerting, posting reviews (and delegating reviews to experts), and changing the status of content submitted. Manage Discovery Discovery Management provides features that enable Portal users to browse or search for information-specific topics. The functions also include context for categorizing and tagging the resources so they are more easily retrievable. Manage Access Functions for Managing Access to the portal enable control, roles, and privileges for downloading, interacting with the content, and users of the system. APTA Portal The APTA portal is controlled and underpinned by the features inherent in Microsoft SharePoint, which provides overarching administrative capability for portal management. The Portal provides the framework for the features described in the four primary functions. System Context  The portal will operate as part of the existing APTA website. Major System Capabilities, Conditions, and Constraints  SharePoint Environment  The current APTA SharePoint environment consists of a server farm – one or more back-end database servers and one or more front-end servers that provide Web services and Microsoft 2010 SharePoint Enterprise Server, including search, Excel Services, and indexing. Hardware and Software Infrastructure  Prior to installation and configuration of Microsoft SharePoint 2010, servers should have the recommended hardware and software. For a server farm, there must be at least two computers. One computer must serve as a web server and application server, and one computer must serve as a database server. The servers must meet the following hardware and software requirements: Hardware requirements  Front-end Web server and application server computers: a quad-processor computer with processor clock speeds of 2.5-gigahertz (GHz) or higher and a minimum of 8 gigabytes (GB) of RAM. Back-end database server: a quad-processor computer with processor clock speeds of 3.0 GHz or higher and a minimum of 8 GB of RAM. Software requirements for Web and Application Server  Microsoft Windows Server 2008 Enterprise (or later) Microsoft .Net Framework 3.0 May 2016

34 The Web server and application server computers must be configured as Web servers running Microsoft Internet Information Services (IIS) in IIS shared services mode. Back‐End Database Server  The back-end database server computer must be running Microsoft SQL Server 2008 or later. User Characteristics  This subsection describes the general characteristics of the intended Users of the Portal. There are six (6) types of primary users summarized in the table below. Table 3-10. Portal User Roles User Role Description / Responsibilities Content Contributor Portal user who creates/contributes and adds content to the portal (hereafter referred to as submittal or post).  APTA member  Submits content to System  Complies with and signs on with Portal Policies form  Owns or has the rights to submit content  Completes Portal “template”  Revises content per Review Committee request Review Committee Member Members of the APTA R&T Committee Tech Portal Subcommittee who are responsible for approving or rejecting content, (and may review content from time to time).  APTA R&T ITS Tech Portal Committee Member  Participates in review process per Governance Rules Content Reviewer Person designated by the Review Committee to review content on a technical and/or quality basis. Multiple reviewers may be assigned to review one piece of content.  APTA Member (may or may not be a member of the Review Committee)  Reviews and completes reviewer assessment form for designated submittal Content Reader (User) Person who uses the portal to seek answers to basic transit technology questions  APTA member Content Administrator Individual with technical control over portal content.  Review Committee-designated content administrator (e.g., chair) or APTA IT Staff  Removes or changes the status of a posting or resource  Provides (removes) access/privileges of Content Reviewers and/ or Review Committee Members System Administrator APTA staff person(s) who administers the System.  Administers system control over the Portal including creating, revising, and updating SharePoint configuration May 2016

35 User Role Description / Responsibilities  Implements system lifecycle changes to SharePoint and associated presentation and back office systems Assumptions  Prototype Implementation  This project will implement a working prototype for the ITS Technology Portal. Long-term operations and maintenance are outside the scope of this project. Relationship to Other APTA Programs  This project will serve as an extension of the APTA website, which uses SharePoint as part of its backend. The portal will be fully integrated into MyAPTA for use by members who already have access to this service. Policies and Regulations  The portal will follow all APTA polices for web content and the Governance Policy set forth by the APTA R&T ITS Tech Portal Subcommittee. System Requirements  Requirement 1: Functional Requirements  This section defines the functional requirements for the ITS Tech Portal. Requirement 1.1: Manage Access  The system shall provide the functional elements to control user access and permissions to ensure content is properly managed. Requirement 1.1.1: Manage Portal Login   The System shall grant access to users through MyAPTA. Users who do not have access to MyAPTA shall not have access to the system. Requirement 1.1.2: Manage Roles   The System shall provide the ability for a System Administrator to change the role of a user to any of the User Roles described in Table 3-11. SharePoint manages these roles as three separate groups: Table 3-11. User Roles Portal Role SharePoint Role Privilege Granted in SharePoint Content Reader Visitors Read Content Contributor Members Edit Content Owners Full Control May 2016

36 Administrator Content Reviewer Members Edit System Administrator Owners Full Control Requirement 1.1.2.1: Assign User Role   The system shall grant specific privileges to a user by assigning a user to a specific user role. Requirement 1.1.2.2: Assign User to Multiple Roles   The system shall permit a user to be assigned to multiple roles. The user shall have all the rights and privileges of all user roles to which the user is assigned. Requirement 1.1.2.3: Manage Role Privileges  The system shall provide the ability to manage roles created and the privileges associated with them in accordance with SharePoint security groups and permissions. Requirement 1.1.2.3.1: Manage System Administrator Privileges   The System shall include a role entitled System Administrator. The System Administrator shall control access to the system and its functions. The system administrator role will be a SharePoint administrator and have overall authority to create, update, or delete all other roles. Requirement 1.1.2.3.2: Manage Content Administrator Privileges   The System shall include a role entitled Content Administrator. The role shall have privileges as outlined in Appendix B. Requirement 1.1.2.3.3: Manage Content Reviewer Privileges   The system shall include a role entitled Content Reviewer. Requirement 1.1.2.3.4: Manage Content Contributor Privileges   The system shall include a role entitled Content Contributor Requirement 1.1.2.3.5: Manage Content Reader Privileges   The system shall include a role entitled Content Reader. Requirement 1.1.3: Automate Role Grant   The System shall automatically grant the roles of Content Contributor to any MyAPTA registered user Requirement 1.1.4: Manage Content Access  The system shall provide functions to ensure access to content is available for users with correct roles and permissions to download and view content stored in the system. Requirement 1.1.5: Manage User Access  All list data that supports the roles outlined in Table 3-11 must be editable and updatable by the system administrator. May 2016

37 Requirement 1.1.5.1: Manage Reviewer List  The system shall allow an authorized user the ability to view, add, remove, or edit users from the reviewer list. Designation on this list enables a person to be assigned to review content. Requirement 1.1.5.2: Manage Committee Member List  The system shall allow an authorized user the ability to view, add, remove, or edit users from the committee member list. Designation on this list bestows access rights to certain system lists as described in Appendix B. Requirement 1.2: Manage Storage  The Manage Storage function stores all the resources, artifacts, documents, and event lists that are used by the System. The function provides for the storage of provisional and approved content, resource status, resource identification, catalog services, user lists, and other key information needed to manage and categorize resources. The Manage Storage function also contains functionality for Content Contributors to post resources and multimedia using a guided tool or template. Requirement 1.2.1: Populate List  The system shall incorporate the results of a validated form to be inserted into an appropriate list. Requirement 1.2.2: Validate List Entry  The system shall validate a form submitted by a user based on the criteria set by the form (e.g., mandatory entries shall be completed, syntax shall be checked, etc.). Requirement 1.2.3: Manage Provisional Content  The system shall store content that has been posted but not yet approved in storage. Content not yet approved shall be designated as provisional. Requirement 1.2.3.1: Restrict Access to Provisional Content  The system shall restrict only authorized users to view provisional content. Requirement 1.2.4: Manage Content Catalog  The system shall assign only approved content to the Content Catalog. The Content Catalog will include only a subset of the Content List fields (Requirement 1.5.1.3). Requirement 1.3: Manage Content  Content management provides the functionality for the Review Committee to implement the content review processes. The processes include collecting, alerting, posting reviews (and delegating reviews to experts), and changing the status of content submitted. Requirement 1.3.1: Manage Content Status  The system shall provide the ability to designate status automatically as well as enable authorized users to manipulate status based on role permissions. May 2016

38 Requirement 1.3.1.1: Designate Status   The system shall assign a status designation to content as it flows from state to state in the Managed Content process. The designations shall be defined as in Table 3-12. May 2016

39 Table 3-12. Content Status Definitions Status Name Status Description Next Status (condition) Responsible Role Post-Complete Status assigned to submitted content. Pending Review Content Contributor Pending Review Status assigned to content that is assigned to a reviewer. Review Complete Content Administrator Review Complete Status assigned after a reviewer submits the content criteria record. To be reviewed by Review Committee: Approved (if approved) Rejected (if rejected) Revision Request (if provisionally accepted) Reviewer Approved Status assigned to content after it is approved by the Review Committee. Pending Re-review (after specific period of time) Content Administrator Revision- Request Status assigned to content after it is reviewed by the Review Committee with comments for revision. Pending Re-review Content Administrator Rejected Status assigned to content after the Review Committee rejects the content. <return to Landing Page> Content Administrator May 2016

May 2016 40 Requirement 1.3.2: Manage Status Change  The system shall alert appropriate users when a change of status occurs. The alert shall include the following information:  Type of Status Change  Trigger (person or automated process)  Date and time Requirement 1.3.2.1: Document Status Changes  The system shall log all status changes and the target user who receives the status change. Requirement 1.3.2.2: Publish Status Change  The system shall publish all status changes in a table on a targeted user’s personalized web page. Requirement 1.3.2.3: Manage Content Approval   The system shall allow the Content Administrator (and authorized users) to change a Content record in the Content List to Accepted, Rejected, or Provisionally Accepted. Requirement 1.3.3: Manage Content Registration   The system shall provide guided directions to register and post content. Requirement 1.3.3.1: Manage Registration Process  The system shall enable a Content Contributor to save and retrieve incomplete submittals that only they have submitted. Requirement 1.3.3.2: Validate and Store Content   Upon submission by the user, the system shall validate that the Content Contributor completes all mandatory fields (topic types and keyword associations, personal identification, content description) and checks the validity of each field. The Content List describes the fields (see Section 0). Requirement 1.3.3.3: Manage Content Upload  The system shall allow the user to upload a file and upon submission shall check that the file is a valid file format. Requirement 1.3.3.4: Manage Content Agreement  The system shall allow the user to accept or reject the terms and conditions related to the content. The terms and conditions are described in the APTA ITS Tech Portal Governance document. Requirement 1.3.3.4.1: View Terms and Conditions  The system shall allow a user to view the terms and conditions. Requirement 1.3.3.4.2: Print Terms and Conditions 

May 2016 41 The system shall allow a user to print the terms and conditions. Requirement 1.3.3.4.3: Reject Terms and Conditions  If the user rejects the terms and conditions, the system shall provide the user with a message confirming rejection. If the user rejects the terms and conditions, manage content registration process is stopped (i.e., the page reverts to the “landing page”). Requirement 1.3.3.5: Submit Content for Review  The system shall include a trigger for a Content Contributor to submit his content for review. The action shall trigger a change in status to post-complete and distribute an alert to respective authorized users. Requirement 1.3.3.6: Manage Incomplete Content Submission  The system shall allow a Content Contributor who starts to submit content to save his work and retrieve it at a later date and time. Note: this requirement should also be used for a Content Contributor who resubmits a revised file for review. Requirement 1.3.3.7: Manage Revised Content   The system shall provide a Content Contributor the ability to select an existing content submission that has a status of revision-request. For the revision process, the system will implement a streamlined version of the Manage Content Registration process. Requirement 1.3.4: Administer Content Review   The system shall provide the content administrator and review committee members with a page that shows the current status for all pending content, review assignments, committee review agendas, and pending approval items. Requirement 1.3.4.1: Manage Pending Content  The system shall display a summary of the content pending status items (post-complete) on the Content Administration page. The summary shall include a link to more detailed information associated with the summary data. The summary data shall include the following information from the Content List:  Content name  Posted date  Abstract or description  Multimedia format Requirement 1.3.4.2: Manage Content Review Assignment  The system shall display a summary of the content with pending status from the Review Scoring List on the Content Administration page. The summary shall include a link to more detailed information associated with the summary data. The summary data shall include the following mandatory information from the Review Scoring List:  Reviewer Name (link to list of reviewer)  Reviewer Assignment (link to posted content)  Reviewer Assignment date

May 2016 42  Reviewer Due date Requirement 1.3.4.3: Review Content by Selection Criteria  The System shall allow an authorized user to select one or more fields from the Content List (Requirement 1.5.1.3) and view the results. Requirement 1.3.4.4: Manage Content Committee Review   The system shall display a summary of the content to be reviewed at the next meeting. The summary shall include a link to more detailed information associated with the summary data. The content shall be grouped by status:  Post to be assigned to a reviewer (status): post-complete, post re-submitted  Posts pending, assigned to reviewer (status): pending-review, pending re-review  Posts to dispose of (status): review-complete  Outstanding Post (status): revision-requested The Posts to dispose of grouping shall link to appropriate Review Scoring List items. Requirement 1.3.4.4.1: Edit Review Agenda  The system shall allow an authorized user to edit, save, and submit a Committee Review Agenda. The action shall trigger an alert with the Review Meeting Agenda to be distributed to the Portal Review Committee. Requirement 1.3.4.5: Provisional Acceptance Alert  The system shall trigger an alert to the Content Contributor upon approval of the Content Administrator when content is provisionally accepted. The alert shall include information on the reason for revision and the recommendations for what should be revised. Requirement 1.3.5: Manage Content Quality Review  The Manage Content Quality Review describes the requirements associated with a reviewer evaluating and scoring content. Requirement 1.3.5.1: Reviewer Summary Page  The system shall provide a reviewer with a summary page showing his assigned pending, incomplete, and completed reviews. The summary shall include a link to more detailed information associated with the summary data. The summary data shall include the following mandatory information from the Review Scoring List:  Reviewer Assignment (link to posted content)  Reviewer Assignment date  Reviewer Due date  Reviewer Status (pending, incomplete, completed) Requirement 1.3.5.2: View Assigned Content  The system shall allow a reviewer to view assigned content. Requirement 1.3.5.3: Manage Review Scoring 

May 2016 43 Upon request, the system shall display a review scoring record that is associated with the content. The scoring record shall include at a minimum the following questions to be completed: Document Title    Review Name  Insert review name here Review Date  Assigned  e.g. 3/1/2014 Review Date Due  e3/30/2014 Review Date  Complete  3/1/2014 Review Type  ☐ technical ☐ quality Review Score  ☐0 (Unacceptable) ☒ 1 (Poor) ☒ 2 (Fair) ☒ 3 (Acceptable) Review  Recommendation  ☐ accept ☐ reject ☐ pending comments Review Comments  Submission  Insert review comments here Review Keywords  Insert any additional keyword inputs here Requirement 1.3.5.4: Manage Incomplete Scoring  The system shall allow a Reviewer who starts to review content to save his work and retrieve it at a later date and time. Requirement 1.4: Manage Discovery  The system shall allow a user to search for content by several methods. Methods include by free-text, predefined topics, keywords, and other methods such as metadata (author, publication date, etc.), and most viewed content. Requirement 1.4.1: Manage Content Availability  The system shall allow a user to search and discover only available published (approved) content. Requirement 1.4.2: Manage Free Text Search  The system shall allow a user to search for content by free text. Requirement 1.4.3: Manage Keyword Search   The system shall allow a user to search for content by predefined topics and refine his search by using a related set of keywords.

May 2016 44 Requirement 1.4.4: Manage Most Viewed Search  The system shall allow a user to search the most viewed content. Requirement 1.4.5: Manage New Content Search  The system shall allow a user to search the most recently uploaded published content. Requirement 1.4.6: Manage Content Catalog Search  The system shall allow a use to search content by reviewing a catalog of published content. Requirement 1.4.6.1: Sort Content Catalog  The system shall allow a user to sort the content catalog by any column. Requirement 1.4.7: View Multimedia Formats  The system shall offer services to view standard multimedia formats including:  Pdf, jpeg, mpeg Requirement 1.4.8: Print Multimedia Formats  The system shall allow a user to print the content in document formats including:  Pdf, Microsoft (MS) PowerPoint, MS Word, MS Excel Requirement 1.5: Administer the System  Administer the System describes functions provided by SharePoint to create lists, update list formats, and manage user access. Requirement 1.5.1: Create Lists  The system shall allow an authorized user to create a list and to associate these lists to each other. Requirement 1.5.1.1: Create Topic Type List  The System shall allow an authorized user to create a list of topic types. The initial set of topic types include:  Technology  Standard  Solution (System)  Business Process  Benefit  Application  Information  Human Resource Requirement 1.5.1.2: Create Keyword Lists  The System shall allow an authorized user to create a list of keywords. Requirement 1.5.1.3: Create Content List 

May 2016 45 The System shall allow an authorized user to create a list of files. The list shall include mandatory fields, including:  File name  File extension (multimedia format)  Posted date  Current Status [post-incomplete, post-complete, post-resubmitted, revision-requested, pending-review, pending-re-review, review-completed, approved]  Next Status [post-complete, post-resubmitted, revision-requested, pending-review, pending-re-review, review-completed, approved]  Next Status date  Content Description / Abstract  Content Author(s)  Content Contributor  Terms of Submission Agreement [yes/no]  Publication Status [provision, accepted, rejected] Optional fields may include:  Reviewer name  Last Review date  Approved date  Published date  Next Review date  Reposted date Requirement 1.5.1.4: Create a Reviewer List  The system shall allow for an authorized user to create a list of content reviewers. The list shall include the following mandatory information:  Name  Email address  Affiliation  Expertise (one or more keywords) Optional information may include:  Work phone  Mobile phone  Alternate email address  Other comment Requirement 1.5.1.5: Create a Review Scoring List  The system shall allow for an authorized user to create a list that reviews posted content by a reviewer. The mandatory information shall include:  Reviewer Name (link to list of reviewer)  Reviewer Assignment (link to posted content)  Reviewer Assignment date  Reviewer Due date  Status (pending, incomplete, complete)

May 2016 46  List of criteria for review  Quality Score  Recommendation for Acceptance (accept, reject, pending comments) Optional fields shall include:  List of additional keywords to associate with Content  Comments to Content Contributor  Comments to Review Committee Requirement 1.5.1.6: Create Supplementary Resources List  The System shall allow for an authorized user to create a list of supplementary resources. The resources may include a link to a stored document or a URL. Each supplementary resource shall include the following mandatory information:  Resource Name  Posted Date  Hyperlink or uploaded file (with an approved file extension) The following optional fields shall be inserted:  Author(s)  Person posting resource  Resource Content Description or Abstract  Publication date  Other pertinent information Requirement 1.5.1.7: Create Association between Lists  The System shall allow an authorized user to create a list that associates lists. In particular, the following types of associations shall be described:  Topic types and Keywords  Content and Topics (and Keywords)  Resource and Topics (and Keywords)  Topic Types to each other (see Appendix B for relationship descriptions) Requirement 1.5.1.8: Create a Lookup List  The System shall allow an authorized user to create a list of related terms for keywords. The list shall include the following fields:  Keyword  Related term Note: a keyword may have more than one related term. For example, AVL may have the following related terms: automatic vehicle location, automated vehicle location, location tracking. Requirement 1.5.2: Update List Format  The system shall allow authorized users to view, add, revise, and delete fields in a list. Requirement 2: System Security Requirements  Requirement 2.1: Security Policy Compliance  

May 2016 47 The system shall comply with all security policies of the APTA website. The system will use the default SharePoint Role permissions Requirement 3: Information Management Requirements  Requirement 3.1: User Interface   The system shall support several versions of web browsers, including:  Internet Explorer (version 8 and above)  Firefox (current version)  Chrome (current version) Requirement 3.2: System Performance and Life Cycle Requirements  The system shall follow APTA’s IT performance and lifecycle policies. Technology Alternatives Analysis  This section describes a Technology Analysis undertaken in 2012. Subsequent to this analysis APTA updated their system to SharePoint 2010. Alternatives Analysis Methodology and Tools  Several technologies are available to implement the ITS Technology Portal. Three technologies were originally proposed:  SharePoint 2007  SharePoint 2010, and  WordPress and Buddy Press SharePoint 2007 and SharePoint 2010 are products of Microsoft; APTA’s web site used Version 2007 with custom web parts. The alternative analysis was originally designed to measure the functionality of the various tools against the requirements versus the cost and risk factors. However, APTA has migrated to SharePoint 2010 and has stated that it does not have the resources to manage a technology other than SharePoint 2010. To that end, only one alternative will be analyzed, and the analysis will identify the estimated costs, schedule, and risks for implementing the high priority requirements. The analysis incorporated supporting technologies that are either used by the current APTA web site environment or that support the requirements in the analysis. A brief summary of these tools are listed below:  NetForum – manages access and authentication of APTA members (and MyAPTA functionality).  Web Pages and Branding – these include the presentation of information in a browser. The technologies will be developed using SharePoint in developing Custom Master Pages, Custom stylesheets, Web Parts, HTML, Javascript and other web development standards.  Browsers – these belong to users. The web pages will support the versions defined by requirement 3.1.  Web Analytics – tool to analyze how users navigate through the site.

May 2016 48  SharePoint FAST – a native SharePoint add-on that provides advanced search capabilities based on customer queries and analytics. Alternatives Analysis Summary  The System Requirements, driven by stakeholder needs, were developed and allocated to the various tools under consideration. The results of the analysis are listed in Table 3-13. The analysis assigns the requirements to the most appropriate, available tools. High priority requirements that are assigned to a current APTA-supported tool, such as NetForum and web analytics, will be implemented after the prototype demonstration by APTA, although the prototype may need to simulate these requirements. Other tools, such as SharePoint FAST, will be determined by this analysis. The results show that most of the requirements may be implemented using native tools already supported by SharePoint 2010. Access, Authentication, and Security Requirements  All access, authentication, and security requirements will be implemented by APTA using its NetForum application. In order to test the alerts, the prototype may need to simulate user roles using native SharePoint functionality. This functionality will be removed when the prototype is transferred to APTA. SharePoint Web Parts and Customization  All the functionality needed by the requirements may be met either by native SharePoint functions / web parts or software development tools supplied by SharePoint. However, even with the available SharePoint tools, any custom development will consume much of the available Phase 2 resources. In particular, in discussions with APTA IT staff, it was determined that a special template and work flow for Content Reviewers requires significant changes to the existing APTA SharePoint / NetForum configuration. To that end, a new workflow will be proposed that provides that an ITS Tech Portal committee member oversee the work of reviewers and insert the results into the Review Scoring Template. These changes will impact the operational scenarios and systems requirements. The updates will be developed and proposed to the Panel. Branding and Web Page Design  Branding of the site will be based on APTA Branding Policy with recommendations from TCRP staff on special branding solutions for the ITS Tech Portal implementation. The web page designs will be based on a simple design (Custom Master page and custom stylesheets) using standard best practices and tools. FAST  FAST is a specialized tool that provides the ability for the system to manage keywords used by the system, automatically analyze a document and extract their keywords, and direct the user to appropriate literature and multimedia. The tool has additional operations and maintenance costs and resource needs that will impact its long range use. However, APTA is also looking at this tool for needs beyond this project.

May 2016 49 Cost  Costs are divided into two categories – prototype development and impacts on APTA (transition, operations, and maintenance). The estimated prototype costs are in line with the costs included on the original proposal. The ongoing costs following the prototype are still under review by APTA, the contents of which have been provisionally reviewed and approved by APTA IT staff. Prototype Development Costs  Setup of SPS2010 and FAST for SharePoint - $750 vs. SPS2010 (without FAST) - $600  Monthly host of SPS2010 and FAST for SharePoint - $500/month  ISP Technical Support 20 support hours - $2700  Web Page/Branding and Custom SharePoint Development: approximately 100 hrs.  Content Development: approximately 60 hrs. Impacts on Transition, Operations, and Maintenance Costs  FAST license cost (Setup and Ongoing) Note that APTA currently does not have FAST implemented, so this cost will not figure in.  Transition costs (tbd)  Additional development costs: o Interfacing prototype with NetForum o Integration of web parts and web pages with APTA site o Analytics and analytics reporting o Migration costs to APTA’s web server  Operations / Maintenance Costs: other impacts will be noted in the Implementation and Sustainability Plans (see Phase 2) including user training. Risk  The following is a short description of anticipated risks based on the tradeoff review. Use of FAST FAST provides the taxonomy and keyword management necessary to enable the ontology and back-end structure that is desired. It can also enrich and enhance the experience of users, particularly those who are not technically knowledgeable. However, APTA does not have the experience with the tool and the ongoing license costs are not known. Although the prototype used this feature, currently APTA does not have it implemented. Manual vs. Automated Processes There are several aspects to applying manual versus automated processes. Several comments were raised during the requirements walkthrough about adopting manual processes versus the automated workflows described by the requirements document. The impacts fall upon system administration staff as well as special user roles – reviewers, review committee, content administrators. A brief discussion of these risks and tradeoffs follows. Impact on System Administration Staff Manual processes will require a layer of quality control and verification on the Administration staff for all transactions conducted by reviewers, review committee, and content administrative users. They will need to train the users about how SharePoint processes, lists, and tools work. The manual process will require resources from APTA staff.

May 2016 50 Impact on Reviewer, Review Committee and Content Administrator Automation guides the user through the work processes with only limited user documentation (already included in subsequent documentation). Although there is less programming work required for manual processes, users will need to be conversant in how to manage lists if a manual process is implemented, will need to undergo training by “super users” of SharePoint and the review processes, and will need to implement a step-by-step process to ensure that all related stakeholders receive the proper notification. This will add a significant burden on volunteer staff. Technology Alternatives Analysis Matrix Table 3-13 shows each requirement, its priority, and the proposed technology that is identified to implement the technology. The alternatives are driven by current APTA IT policies and resources. The “Requirements” column contains the requirements described in the System Requirements document. The “Priority” column represents whether the requirement is rated as High, Medium, or Low for the prototype. The final column, “TAA,” lists the tool that will implement the requirement. The tools are described above. The reference to SharePoint is further divided into functions that are configured for this effort from native SharePoint versus ones that must be custom programs. In addition, a lighter highlight is used to show the process that will be changed to comply with APTA’s security policy. Table 3-13. Requirements, Priorities, and Tools Reqmt ID Requirement Title Priority TAA Tool 1.1 Manage Access H netforum 1.1.1 Manage Portal Login H netforum 1.1.2 Manage Roles M SharePoint configured 1.1.2.1 Assign User Role M SharePoint configured 1.1.2.2 Assign User to Multiple Roles M SharePoint configured 1.1.2.3 Manage Role Privileges M SharePoint configured 1.1.2.3.1 Manage System Administrator Privileges M SharePoint configured 1.1.2.3.2 Manage Content Administrator Privileges M SharePoint configured 1.1.2.3.3 Manage Review Committee Member Privileges M SharePoint configured 1.1.2.3.4 Manage Content Contributor Privileges M SharePoint configured 1.1.2.3.5 Manage Content Reader Privileges M SharePoint configured 1.1.3 Automate Role Grant M SharePoint configured 1.1.4 Manage Content Access M SharePoint

May 2016 51 Reqmt ID Requirement Title Priority TAA Tool configured 1.1.5 Manage User Access M SharePoint configured 1.1.5.1 Manage Reviewer List M SharePoint configured 1.1.5.2 Manage Committee Member List M SharePoint configured 1.2 Manage Storage H SharePoint configured 1.2.1 Populate List H SharePoint configured 1.2.2 Validate List Entry H SharePoint configured 1.2.3 Manage Provisional Content H SharePoint configured 1.2.3.1 Restrict Access to Provisional Content H SharePoint configured 1.2.4 Manage Content Catalog H SharePoint custom 1.3 Manage Content 1.3.1 Manage Content Status H SharePoint custom 1.3.1.1 Designate Status H SharePoint custom 1.3.2 Manage Status Change H SharePoint custom 1.3.2.1 Document Status Change H SharePoint configured 1.3.2.2 Publish Status Change M SharePoint custom/ netforum 1.3.2.3 Manage Content Approval H SharePoint configured 1.3.3 Manage Content Registration H SharePoint configured 1.3.3.1 Manage Registration Process H SharePoint configured 1.3.3.2 Validate and Store Content H SharePoint configured 1.3.3.3 Manage Content Upload H SharePoint configured 1.3.3.4 Manage Content Agreement H SharePoint configured 1.3.3.4.1 View Terms and Conditions H SharePoint configured 1.3.3.4.2 Print Terms and Conditions L Browser native 1.3.3.4.3 Reject Terms and Conditions M Browser native 1.3.3.5 Submit Content for Review H SharePoint workflow

May 2016 52 Reqmt ID Requirement Title Priority TAA Tool 1.3.3.6 Manage Incomplete Content Submission H SharePoint workflow 1.3.3.7 Manage Revised Content H SharePoint workflow 1.3.4 Administer Content Review H SharePoint workflow 1.3.4.1 Manage Pending Content H SharePoint workflow 1.3.4.2 Manage Content Review Assignment H SharePoint workflow 1.3.4.3 Review Content by Selection Criteria H SharePoint configured 1.3.4.4 Manage Content Committee Review M SharePoint custom 1.3.4.4.1 Edit Review Agenda M SharePoint custom 1.3.4.5 Provisional Acceptance Alert M SharePoint workflow 1.3.5 Manage Content Quality Review H SharePoint configured 1.3.5.1 Reviewer Summary Page H SharePoint configured/ netforum 1.3.5.2 View Assigned Content H SharePoint configured/ netforum 1.3.5.3 Manage Review Scoring H SharePoint configured 1.3.5.4 Manage Incomplete Scoring H SharePoint configured 1.4 Manage Discovery H 1.4.1 Manage Content Availability H SharePoint configured 1.4.2 Manage Free Text Search H SharePoint configured 1.4.3 Manage Keyword Search H SharePoint FAST 1.4.4 Manage Most Viewed Search M SharePoint custom/ FAST 1.4.5 Manage New Content Search M SharePoint configured 1.4.6 Manage Content Catalog Search H SharePoint configured 1.4.6.1 Sort Catalog Content H SharePoint configured 1.4.7 View Multimedia Formats M Web page 1.4.8 Print Multimedia Formats L Web page

May 2016 53 Reqmt ID Requirement Title Priority TAA Tool 1.5 Administer the System H SharePoint configured 1.5.1 Create Lists H SharePoint configured 1.5.1.1 Create Topic Type List H SharePoint configured 1.5.1.2 Create Keyword Lists H SharePoint configured 1.5.1.3 Create Content List H SharePoint configured 1.5.1.4 Create a Reviewer List H SharePoint configured 1.5.1.5 Create a Review Scoring List H SharePoint configured 1.5.1.6 Create Supplementary Resources List H SharePoint configured 1.5.1.7 Create Association between Lists H SharePoint configured 1.5.1.8 Create a Lookup List H SharePoint configured 1.5.2 Update List Format H SharePoint configured High‐Level Design  This section describes the high-level design (HLD) of the ITS Technology Web Portal for Transit System Leaders. Introduction  The high-level design includes the functions, navigation, and data that will be included in the Portal. The design is driven by the user needs documented in the Concept of Operations and functional requirements driven by the System Requirements. The organization of this section is:  Portal Functions describe the primary functions of the portal – Access, Storage, Content, and Discovery. These primary functions are further described with functional flow diagrams that show the information and roles required to enable the functions.  Information and Data Model describes the informational elements and their relationship to each other in a data model.  Navigation and Wireframes describes the screens and navigation among screens in the Portal. Since the portal will include links to the main APTA web site, those links are included as “stubs” in the design. This HLD sets the framework for developing the ITS Technology Web Portal in Microsoft SharePoint 2010. From the design elements in this document, a SharePoint developer will be

May 2016 54 able to build the portal using SharePoint. Once the portal is built, it can be populated with content and list details. Limitation  Implementation of the design is subject to the environment configuration and the branding requirements of the host website, APTA’s SharePoint-based membership portal, which was recently updated from SharePoint 2007 to SharePoint 2010. Because of this, it is possible that the final configuration will differ slightly from that described in this section Portal Functions  Functional Flow  This section contains the functions that will be implemented in the portal. The functions are illustrated using functional flow diagrams and are further described by related tables. There are four major functions for the management of Access, Storage, Content, and Discovery. Figure 3-2. High-Level Portal Architecture Table 3-14. Primary Portal Function Descriptions Function Name Description APTA Portal The APTA portal is controlled and supported by the features inherent in Microsoft SharePoint, which provides capabilities in its overarching capacity for portal management. Manage Access Functions for managing access to the portal enable control, roles, and privileges for downloading and interacting with the content and users of the system. Manage Storage The Storage function handles all the resources, artifacts, documents, and event lists that are used by the System. It stores provisional and approved content, resource status, resource identification, catalog services, user lists, and other key information needed to manage and categorize resources.

May 2016 55 Function Name Description The Storage features also contain functionality for Content Contributors to post resources and multimedia. Manage Content Content Management functions enable the Review Committee to implement the content review processes. The processes include collecting, alerting, posting reviews, and changing the status of submitted content. Manage Discovery Discovery Management provides features that enable portal users to browse or search for information-specific topics. The functions also include context for categorizing and tagging the resources so they are more easily retrievable. Manage Access  Functions for Managing Access to the portal enable control, roles, and privileges for downloading, interacting with the content and users of the system. The relationship of the functions, and their descriptions, are shown below: Manage Portal  Login Manage Roles Manage  Content Access Assign User  Role Manage CA  Privileges  Manage User  Access  Manage  Reviewer List Manage  Committee  Member List Manage Role  Privileges  Manage RCM  Privileges  Manage CC  Privileges  Manage CR  Privileges  Signed into Portal Person  [access] Content Person [role] Function Data Figure 3-3. Manage Access Functional Flow Diagram

May 2016 56 Table 3-15. Manage Access Function Descriptions Function Description Manage Portal Login Access to the portal provide through the APTA Website Manage Content Access Controls access to content stored on the portal Manage User Access Controls assignment of portal users to lists of users getting special access privileges to the portal (e.g. committee members) Manage Committee Member List Manages the list of members of the review committee which oversees the portal Manage Reviewer List Manages the list of content reviewers Manage Roles Controls the assignment of user roles for the portal Manage Role Privileges Controls the privileges available to each of the roles defined for the portal Manage CA Privileges Controls the privileges available to Content Administrators Manage RCM Privileges Controls the privileges available to Review Committee Members Manage CC Privileges Controls the privileges available to Content Contributors Manage CR Privileges Controls the privileges available to Content Readers Assign User Role Manages assignment of a specific user to specific roles within the portal Manage Storage  The Storage function stores all the resources, artifacts, documents, and event lists that are used by the System. It stores provisional and approved content, resource status, resource identification, catalog services, user lists, and other key information needed to manage and categorize resources. The relationship of the functions, and descriptions of the functions, are shown below:

May 2016 57 Figure 3-4. Manage Storage Functional Flow Diagram Table 3-16. Manage Storage Function Descriptions Function Description Navigate to Content Provides access to the functionality for managing content Manage Provisional Content Provides tools to manage content which has been uploaded but not formally published Populate List Provides ability to add records to the content list Validate List Entry Validates entries to the content list Manage Content Catalog Provides ability to manage the content catalog Restrict Access Allows access to specific items of content to be restricted Manage Content   Content Management functions enable the Review Committee to implement the content review processes. The processes include collecting, alerting, posting reviews and changing the status of content submitted. The relationships between functions, and the descriptions of the functions, are below.

May 2016 58 Content  [uploaded] Manage  Content Status Manage  Content  Registration Administer  Content  Review Manage  Content  Quality Review Manage  Content Designate  Status Publish Status  Change Manage  Content  Approval  Validate and  Store Content Manage   Content  Upload Submit  Content for  Review Manage  Revised  Content  Manage  Content  Review  Assignment Manage  Pending  Content Manage  Content  Committee  Review  View Assigned  Content Manage  Review Scoring  Reviewer  Summary Page User Logs on to Portal Function Data Content  [scored] Content  [validated] Content  [approved] Content  [pending] Content  [reviewed] Figure 3-5. Manage Content Functional Flow Diagram Table 3-17. Manage Content Function Descriptions Function Description Manage Content Provides access to the content management functionality of the portal Manage Content Registration Controls the submission of new and revised content to the portal Manage Revised Content Controls the submission of content which has been revised and resubmitted Submit Content for Review Controls the initial submission of content Manage Content Upload Manages the process to upload content to the portal Validate and Store Content Validates the format of content and stores it within the portal Administer Content Review Manages the content review process Manage Pending Content Manages content which has been submitted to the portal but not yet approved Manage Content Review Assignment Manages the assignment of a Content Reviewer to review specific content Manage Content Committee Review Manages the process for approving content for publication Manage Content Status Manages the process for assigning publication status to content Designate Status Allows a specific status to be assigned to submitted content

May 2016 59 Function Description Manage Content Approval Manages the approval or rejection of content Publish Status Change Designates the official status of content Manage Content Quality Review Manages the process of reviewing content Reviewer Summary Page Allows Content Reviewers to receive a dashboard of assigned content View Assigned Content Allows Content Reviewers to view content which has been assigned to them Manage Review Scoring Allows Content Reviewers to formally assess submitted content Manage Discovery  The search functions will allow a user to discover the most recently uploaded published content. There will be two ways to search in addition to the default SharePoint search that the team is enabling for this effort. The relationships between discovery functions, and the descriptions of functions, are shown below. Figure 3-6. Manage Discovery Functional Flow Diagram Table 3-18. Manage Discovery Function Descriptions Function Description Manage Discovery Provides access to the content discovery features of the portal, and the library for viewing content by topic type Manage Available Content Controls content that is available to users Manage Content Catalog Search Allows a user to search for content Manage Most Viewed Search Allows a user to search for popular content Manage Free Text Search Allows a user to search for specific content

May 2016 60 Information and Data Model  SharePoint provides information management through a functional element called a list. A list is similar to a Microsoft Excel spreadsheet or a table in a Microsoft Access database. Unlike a spreadsheet that is blank when you first start working with it, SharePoint provides several predefined lists. These lists have columns and forms that make it possible for you to track everything from contacts to tasks. SharePoint provides three basic types of lists:  Communications lists are used to track announcements, contacts, and discussion boards.  Tracking lists are used to track information such as links, calendars, tasks, issues, and surveys.  Custom lists provide a starting template that you can build on to create a list with the exact columns you need. The portal will leverage all of these types of lists. Another special kind of SharePoint list is a “library.” This is used to store files, in addition to tracking them and the metadata about them. The portal content will be stored in a library and use the library features to manage the content provided therein. The list structure does not allow complex, relational connections among elements; however, list instances can be related through lookup columns. As such, the data model that will store information is simple, with a minimum number of relationships. The information and its relationships are depicted in the conceptual model below. This information model will provide the foundational elements to enable the functional inputs and outputs through the portal. Figure 3-7. Portal Conceptual Information Model An additional set of lists (represented as objects in the information model) that need to be developed to store information are listed in Table 3-19. The definition for standard transit business processes, application types,

May 2016 61 and other transit domain artifacts are derived from the TCRP Project 84 (volume 9): Transit Enterprise Architecture and Planning Framework (TEAP). Table 3-19. SharePoint Solutions Lists List Name Description Application Lists applications and software that are used to address transit business solutions. For example, a customer relationship management system and maintenance management systems are applications Benefit Lists benefits for transit Business Process Lists transit business processes (from TEAP) Human Resource Lists the human resources needed to support a solution or technology Information Lists the data and information needs (input or output data) to support a solution or technology Solution Lists systems, service packages, or collection of applications, technologies, information, and business processes used by transit to address a need Standard Lists formal standards developed by a standards development organization or recognized open specification that supports a technology Technology Lists specific technologies that are highlighted in the portal Figure 3-8. Portal Solution Information Model Content Keyword List  A keyword is used to catalog the multimedia content stored in the Portal. The keyword list is described below. Table 3-20. Content Keyword List

May 2016 62 Field Mandatory/ Optional Type Description Name M string The reference name used for the keyword. Description M memo The definition of the keyword. Topic Type List  A topic type is a classification of a keyword. The topic type is a high-level taxonomy that provides drill- down capability for the user. The set of topic types includes:  Technology  Standard  Transit Solution (or System)  Transit Business Process  Transit Benefit  Application  Transit Information  Human Resource The topic types are also contained in the Keyword List. Table 3-21. Topic Type List Field Mandatory/ Optional Type Description Topic Type M string Restricted reference to a Keyword –  Technology  Standard  Solution  Business Process  Benefit  Application  Information  Human Resource Related Keyword M string Reference to the Keyword List. For example, Technology is related to “communications,” “navigation,” and “Mobile Device” Business Rules: There is a one-to-many relationship between a topic type and keyword. In addition, the Topic Type and Related Keyword must form a unique pair. Content List  The Content List contains the fields that store information about all the content (pending and published) in the Portal. The fields in the Content List are described below. Table 3-22. Content List Field Name Mandatory/ Optional Type Description File name M string The title or name of the file. The Content Contributor inserts this field during the Registration Workflow. File extension M string The type of multimedia format assigned to the file. The Content Contributor inserts this field during the

May 2016 63 Field Name Mandatory/ Optional Type Description Registration Workflow. Posted date M date The initial date the file was uploaded to the portal. This date is automatically inserted when a new record is created. Current Status M string An automated designation related to the stage in the workflow assigned to the record (i.e., content). Valid values include: [pending-review, review-completed, revision requested, approved, rejected] The status definitions are described in Table 3-23 below. The workflow process automatically inserts this field. Next Status M string Based on the status of the “Current State,” this field is automatically loaded with the next status based on the workflow as illustrated in Figure 3-9. [pending-review, review-completed, revision requested, approved, rejected]. The workflow process automatically inserts this field. Next Status date O date The date at which the next possible change of status may occur. The next status date is inserted automatically based on the conditions identified in Table 3-23 by the provisions cited in “Duration until next status” column. Content Description M memo An abstract of the content in the file. The Content Contributor inserts this field during the Registration Workflow. Content Author(s) M memo One or more names of the authors of the content. A sublist should be included in this field that separates author (first, last names) and email for each author. The Content Contributor inserts this field. Content Contributor First Name M string The first and middle names of the person submitting the content is inserted here. The Content Contributor inserts this field. Content Contributor Last Name M string The last name of the person submitting the content is inserted here. The Content Contributor inserts this field. Content Contributor Email M string The email address of the person submitting the content is inserted here. The Content Contributor inserts this field. Terms of Submission Agreement M boolean This field indicates that the author agrees to the terms for submitting and posting content. The only state at which this field is not included is post-incomplete. The Content Contributor inserts this field. Publication Status M string The status assigned by the Review Committee to the document. Valid values include: [provision, accepted, rejected]. “Provision” is the default status. Provision is designated when the content is pending review or revision of the document.

May 2016 64 Field Name Mandatory/ Optional Type Description Acceptance is designated when the Review Committee agrees to post the document on the Portal. Rejection is designated after the Review Committee rejects the submission. An authorized Review Committee member inserts this field. Reviewer name O string A reference to a name in the Reviewer List. The review list includes all the credentials of each reviewer that may be assigned to review content. Review Assignment Date O date The date when the reviewer accepted the assignment. An authorized Review Committee member inserts this field. Last Review date O date The date when the reviewer submitted the review. An authorized Review Committee member inserts this field. Approved date O date The date when the Publication Status was changed to “accepted”. This field is automatically inserted when the publication status is changed to “accepted.” Published date O date Date when the content is published. An authorized review committee member inserts this field. Next Review date O date The date when the next possible review of content may occur. An authorized review committee member inserts this field. Reposted date O date Date when content is reposted. This field is automatically inserted when the status of previously rejected content is changed to “accepted.” The status definitions and designations are described below: Figure 3-9. Content Status Designation Table 3-23. Content Status Designations

May 2016 65 Status Name Status Description Next Status (condition) Duration Until Next Status (based on Content List fields) Responsible User Post- Complete Status assigned to submitted content. Pending Review Until Review Assigned Date is inserted Content Contributor Pending Review Status assigned to content that is assigned to a reviewer. Review Complete Until Last Review Date is inserted Content Administrator Review Complete Status assigned after a reviewer submits the content criteria record. To be reviewed by Review Committee: Approved (if approved) Rejected (if rejected) Revision-Request (if provisionally accepted) Until Publication Status is inserted Reviewer Approved Status assigned to content after it is approved by the Review Committee Pending-Re-review (after specific period of time) Content Administrator Revision Request Status assigned to content after review by the Review Committee generates comments for revision. Pending-Re-review Content Administrator Rejected Status assigned to content after the Review Committee rejects the content. <return to First Page> Content Administrator Person List  The system shall allow an authorized user to create a list of personnel that will include content reviewers, contacts and committee members. The list shall contain the following information as related in the table below. Table 3-24. Person List Field Name Mandatory- Optional Type Description Name Last M string Last name of the person

May 2016 66 Field Name Mandatory- Optional Type Description Name First M string First name of the person Name Nick M string Nickname of person Email Primary M string Primary email address of the person Email Secondary O string Secondary email address of the person Phone Work M string Primary phone number of the person Phone Mobile O string Primary mobile phone number of the person Phone Fax O string Primary fax number of the person Person Notes O string Additional notes on the person Person Expertise M memo One or many expertise keywords Committee Position M string Position from pick-list of committee member types Committee Date O date Date joined committee Reviewer M yes or No Is the person a reviewer? Yes or no Contributor M yes or No Is the person a Contributor? Yes or no Person Affiliation O string Person’s affiliation to or with Person Organization O string Person’s primary organization Review Assignment List  The system shall allow for an authorized user to create a list that reviews posted content by a reviewer. The list shall contain the following information as related in the table below. Table 3-25. Review Assignment List Field Name Mandatory- Optional Type Description Review Name M string Link to person list of reviewers Review Assignment M string Link to posted content Review Date Assigned M string Date review assigned Review Date Due M string Date review due Review Date Complete M string Date review completed Review Status M string Status of review as [pending, incomplete, complete] Review Criteria M string List of criteria for review Review Score M string Quality Score Review Recommendation M string Recommendation for Acceptance (accept, reject, pending comments) Review Comments Submission O string Comments to Content Contributor Review Comments Committee O string Comments to Review Committee Review Keywords O string List of additional keywords to associate with Content

May 2016 67 Resource List  The System shall allow for an authorized user to create a list of supplementary resources. The resources may include a link to a stored document or a URL. The list shall contain the following information as related in the table below. Table 3-26. Resource List Field Name Mandatory- Optional Type Description Resource Name M String Resource Name Resource Date M date Posted Date Resource Link M String Hyperlink or uploaded file (with an approved file extension) Resource Author O String Author(s) Resource Person Post O String Person posting resource Resource Content Description O String Resource Content Description or Abstract Resource Publication date O date Publication date Resource Notes O memo Other pertinent information Keyword List  The System shall allow an authorized user to create a list of related terms for keywords. The list shall include the following fields: Table 3-27. Keyword List Field Name Mandatory- Optional Type Description Keyword Name M String Keyword Name Keyword Related M string Keyword related term Keyword Notes O memo Other pertinent information Note: a keyword may have more than one related term. For example, AVL may have the following related terms: automatic vehicle location, automated vehicle location, location tracking.

May 2016 68 Navigation and Screen Drafts  SharePoint provides the core elements for navigation and the default navigation will be leveraged to the maximum extent possible. Wireframes are also provided as visual guides to represent the structural framework of the portal. Site Map  The primary site APTA.com will be the starting point for Navigation with a hyperlink to the portal for content management. Figure 3-10. Site Map Screen Drafts  The screen drafts provide a general draft concept and will likely change as the site is further refined from a prototype to a full-featured release.

May 2016 69 Manage Content Submittal   The Manage Content Submission draft concept provides the structure to support the submission of new content to the portal. Figure 3-11. Manage Content Submittal Manage Content Review  The Manage Content Review draft design provides the structure to support the content review.

May 2016 70 Figure 3-12. Manage Content Review

May 2016 71 Manage Discovery Wireframe  The Manage Discovery draft design provides the structure to support the navigation of topics and their related content elements as well as the ability for a user to perform a free-form search for content. Figure 3-13. Manage Discovery Wireframe

May 2016 72 Test Plan and Procedures  Introduction  This section describes the Test Plan and Procedures, for TCRP Project G-13, “Developing an ITS Technology Web Portal for Transit System Leaders.” The test plans and procedures are the key inputs to the Verification Stage of the Systems Engineering Process being used to develop the web portal. The verification stage is necessary, as it ensures that the web portal is built exactly as specified by the System Requirements document. Meeting each functional requirement is necessary to ensure that the user needs are met, and that the overall system purpose is achieved. Test Conduct  Test Items  The item to be tested is the ITS Technology Web Portal for Transit System Leaders. The features to be tested are all of those that are listed as functional requirements in the System Requirements document developed as part of this effort. All system requirements have been classified as High, Medium, or Low Priority in the System Requirements document. The specific requirements to be included in testing are as follows:  All system requirements identified as high priority.  All system requirements identified as medium or low priority that are selected for implementation. Prior to the commencement of testing, the medium and low priority requirements that have been selected for implemented shall be formally stated in the testing documentation. Test cases are included in the test procedures for all medium and low priority items, even if they are not currently identified for testing, in case they are ever designated for future implementation. Testing Roles and Responsibilities  The roles and responsibilities for the testing process are listed below. Note that one individual may serve in multiple roles.  Principal Investigator: The Principal Investigator is responsible for supervising the entire testing process, including preparation of all documentation (test plans/procedures and test reports), observing the testing procedures, and making the final decisions on acceptance of the test results.  Test Analyst: The Test Analyst is responsible for preparing the test plans and procedures, performing the tests in conjunction with the Lead Developer, and preparing the test report.  Lead Developer: The Lead Developer is responsible for reviewing the test plans and procedures, preparing the system for testing, providing assistance during

May 2016 73 testing, reviewing the test report, and suggesting resolutions to any testing failures. Testing Time and Location  Testing shall be conducted after the web portal is built and the lead developer considers it to be operational. Testing shall be concluded prior to any beta testing performed by individuals outside the development team. Since this is a web portal, testing need not be conducted in a specific physical location. Testing shall be conducted virtually, with the members of the identified testing team present via web conference. Testing Approach  Test Methods  For the purpose of testing the web portal, the team shall use “demonstration” as the test method. Demonstration is described as observing a requirement being explicitly met by the web portal. In case of any ambiguity, the Test Analyst shall apply judgment to determine if the requirement is met. In order to accomplish testing, several test cases are specified, each testing a defined set of system requirements. Each test has a set of procedures to be followed, which will ensure that all requirements specified by that test case are tested. Test cases are categorized by the four major areas of functionality for the web portal, plus the overlapping area involving System Administration:  Manage Access  Manage Storage  Manage Content  Manage Discovery  Administer the System Each test case includes a series of steps to be followed. Two types of steps will be used:  Configuration Steps: These steps set up the conditions required to test a requirement, or series of requirements, but do not explicitly test the ability of the system to meet a requirement.  Demonstration Steps: These steps will test explicit requirements. The results of each demonstration step will be recorded as follows:  Pass: The actual output of the step matches the output as stated in the test procedures, and the Test Analyst determines the stated requirement to be satisfactorily demonstrated.  Fail: The actual output of the step does not match the expected output of the step. A step that fails shall be classified in one of three ways: o Severe: This is a failure that, in the judgment of the Test Analyst, will impact overall operations of the web portal, and must be repaired in order for proper operation of the web portal to occur. o Warning: This is a failure that produces an error but does not fully impact portal operations in the judgment of the Test Analyst.

May 2016 74 o Enhancement: This is a failure that does not significantly impact portal operations, and requires additional implementation in order to resolve.  Not Applicable: This procedure being tested is for a feature that has not been selected for implementation and therefore does not impact the results of testing. Test Acceptance/Rejection  A test will be accepted if one of the following conditions is met:  All demonstration steps of the test are classified as either “Pass” OR “Not Applicable.”  All demonstration steps of the test are classified as either “Pass” OR “Fail” OR “Not Applicable,” AND all steps that are classified as “Fail” are further classified as an “Enhancement” Failure.  All demonstration steps of the test are classified as either “Pass” OR “Fail,” OR “Not Applicable” AND all demonstration steps that are classified as “Fail” are further classified as either a “Warning” OR “Enhancement” Failure, and the aggregate of all “Warning” Failures will not impact web portal operations at the same level as a “Severe” Failure. If one of these three conditions is not met for a given test case, the test is considered to be rejected. The full system is considered to be verified if, and only if, all tests are considered to be accepted. If any test has been rejected, retesting can occur once the Lead Developer has determined that all necessary repairs are made. Official acceptance of a test, and of system verification, shall be given only by the Principal Investigator, in consultation with the Test Analyst and, if necessary, the Lead Developer. Test Documentation/Reporting  The test analyst shall document the output of each step of testing in the format prescribed in the test methods section. At the conclusion of testing, the Test Analyst shall prepare a test report that states test acceptance or rejection results and whether the system is verified. In the event of failure of any procedure, or rejection of any test, the report shall provide recommendations necessary to rectify the problem. The report shall also state whether the full system must be retested or only affected aspects of the system must be retested in order for test acceptance to occur. Test Procedures  The test procedures contain 15 different test cases, or sets of tests, to test the complete set of functional requirements for the system. Table 3-28 below shows which test cases are used to test each requirement. Table 3-28. Test Case Summary New ID Requirement Title Group Test Case(s)

May 2016 75 New ID Requirement Title Group Test Case(s) 1.1 Manage Access Access 1.1, 1.2, 1.3, 1.4 1.1.1 Manage Portal Login Access 1.1 1.1.2 Manage Roles Access 1.2 1.1.2.1 Assign User Role Access 1.2 1.1.2.2 Assign User to Multiple Roles Access 1.2 1.1.2.3 Manage Role Privileges Access 1.3 1.1.2.3.1 Manage System Administrator Privileges Access 1.3 1.1.2.3.2 Manage Content Administrator Privileges Access 1.3 1.1.2.3.3 Manage Review Committee Member Privileges Access 1.3 1.1.2.3.4 Manage Content Contributor Privileges Access 1.3 1.1.2.3.5 Manage Content Reader Privileges Access 1.3 1.1.3 Automate Role Grant Access 1.2 1.1.4 Manage Content Access Access 1.4 1.1.5 Manage User Access Access 1.4 1.1.5.1 Manage Reviewer List Access 1.4 1.1.5.2 Manage Committee Member List Access 1.4 1.2 Manage Storage Storage 2.1, 2.2 1.2.1 Populate List Storage 2.1 1.2.2 Validate List Entry Storage 2.1 1.2.3 Manage Provisional Content Storage 2.1, 2.2 1.2.3.1 Restrict Access to Provisional Content Storage 2.2 1.2.4 Manage Content Catalog Storage 2.2 1.3 Manage Content Content 3.1, 3.2 1.3.1 Manage Content Status Content 3.1 1.3.1.1 Designate Status Content 3.1 1.3.2 Manage Status Change Content 3.1 1.3.2.1 Document Status Change Content 3.1 1.3.2.2 Publish Status Change Content 3.1 1.3.2.3 Manage Content Approval Content 3.2 1.3.3 Manage Content Registration Content 3.1 1.3.3.1 Manage Registration Process Content 3.1 1.3.3.2 Validate and Store Content Content 3.1 1.3.3.3 Manage Content Upload Content 3.1 1.3.3.4 Manage Content Agreement Content 3.1 1.3.3.4.1 View Terms and Conditions Content 3.1 1.3.3.4.2 Print Terms and Conditions Content 3.1

May 2016 76 New ID Requirement Title Group Test Case(s) 1.3.3.4.3 Reject Terms and Conditions Content 3.1 1.3.3.5 Submit Content for Review Content 3.1 1.3.3.6 Manage Incomplete Content Submission Content 3.1 1.3.3.7 Manage Revised Content Content 3.1 1.3.4 Administer Content Review Content 3.2 1.3.4.1 Manage Pending Content Content 3.2 1.3.4.2 Manage Content Review Assignment Content 3.1 1.3.4.3 Review Content by Selection Criteria Content 3.2 1.3.4.4 Manage Content Committee Review Content 3.2 1.3.4.4.1 Edit Review Agenda Content 3.2 1.3.4.5 Provisional Acceptance Alert Content 3.2 1.3.5 Manage Content Quality Review Content 3.3 1.3.5.1 Reviewer Summary Page Content 3.3 1.3.5.2 View Assigned Content Content 3.3 1.3.5.3 Manage Review Scoring Content 3.3 1.3.5.4 Manage Incomplete Scoring Content 3.3 1.4 Manage Discovery Discovery 4.1, 4.2, 4.3, 4.4, 4.5 1.4.1 Manage Content Availability Discovery 4.1 1.4.2 Manage Free Text Search Discovery 4.1 1.4.3 Manage Keyword Search Discovery 4.2 1.4.4 Manage Most Viewed Search Discovery 4.3 1.4.5 Manage New Content Search Discovery 4.4 1.4.6 Manage Content Catalog Search Discovery 4.5 1.4.6.1 Sort Catalog Content Discovery 4.5 1.4.7 View Multimedia Formats Discovery 4.5 1.4.8 Print Multimedia Formats Discovery 4.5 1.5 Administer the System Administration 5.1 1.5.1 Create Lists Administration 5.1 1.5.1.1 Create Topic Type List Administration 5.1 1.5.1.2 Create Keyword Lists Administration 5.1 1.5.1.3 Create Content List Administration 5.1 1.5.1.4 Create a Reviewer List Administration 5.1 1.5.1.5 Create a Review Scoring List Administration 5.1 1.5.1.6 Create Supplementary Resources List Administration 5.1 1.5.1.7 Create Association between Lists Administration 5.1 1.5.1.8 Create a Lookup List Administration 5.1 1.5.2 Update List Format Administration 5.1