Improved Models for Risk Assessment of Runway Safety Areas (2011)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

G-1 Objective The objective of this task was to test the software developed for risk analysis of runway safety areas (RSA) developed under ACRP 4-08. The feedback obtained helped improve the final software and mitigate any problems associated with its instal- lation, operation, and analysis of results. This plan describes the procedures used for testing the soft- ware developed in this project. It includes the identification of volunteer stakeholders that utilized a beta version to carry out analysis with sample data provided. Phase I During this phase, the software was evaluated during devel- opment before a final beta version was released to volunteers for testing. Software Development and Algorithm Tests During this phase the software algorithms and database management procedures were tested before a beta version was created. Input Data Quality Assurance (QA) Users may input incorrect information, use units that are not compatible, or enter the correct information in incorrect fields. Several help features were incorporated, including the checking of values to ensure the input was within allowable ranges. The analysis software includes features to check missing data and advise the user to make the necessary corrections. The software will not run if there are missing data or if the values are outside normal ranges. A P P E N D I X G Plan to Field Test Software Tool Portability Before the release of a beta version, the software was evaluated for portability using different computers with various operational systems (e.g., Windows 7, Windows XP). The objective was to search for possible conflicts with computer operational systems and supporting software versions. Installation The installation was tested on different computers to check for problems with installation of the files required to run the program and the supporting software that is required. The analysis software makes use of common Microsoft Office products, including Excel and Access. The user must have such software to run the risk assess- ment analysis. Access is necessary to handle the various databases, and Excel is used to characterize the RSA’s, the type of terrain, and the existing obstacles with their classification. Preliminary Testing The research team installed the software and ran some analy- sis using the guidance material prepared. Any problems detected were solved before the final beta release was provided to volunteers for testing. Phase II During this phase, a beta version of the software was tested by volunteers. Despite the attempts to make the software tool as user-friendly and practical as possible, the research team asked the volunteers that are familiar with airport planning

and the analysis methodology rationale to provide additional suggestions to improve software and to identify any software bugs they encountered. To facilitate the assessment, data for a couple of airports was prepared and provided to the volunteers to run the analysis. Table G1 presents the list of eight software beta testers. The research team proposed a small number of volunteers to facil- itate obtaining meaningful feedback and to ensure the research team could provide the necessary support to these volunteers during the beta testing period. Perform Tests Beta testers installed the software and ran analyses. A user manual was provided to the testers as well. Feed- back was requested through a basic questionnaire that so- licited comments on the use of the software, practicality, documentation, etc. G-2 Assist Volunteers A helpdesk was established to assist volunteers, answering questions and resolving software issues, particularly with instal- lation. Volunteers could ask for help by phone or by e-mail. The phone number and e-mail address was included in the beta ver- sion user manual. Track Problems/Bugs and Fixes The beta testers’ feedback was recorded. Bugs were fixed as soon as possible, and the updated software was distributed to the beta testers. Suggested improvements were considered and modifications made, as warranted, both during the beta test- ing phase and after. Retest After all bugs were fixed and improvements made, another round of internal tests to fix any new bugs was carried out. Name Stakeholder Organization Comments Doug Mansel Airport Operator Oakland International Airport Chair of ACI-NA Operations and Technical Affairs Committee Mike Hines Airport Operator Metropolitan Washington Airport Authority MWAA Planner Don Andrews Consultant Reynolds, Smith and Hill Airport planning Tom Cornell Consultant Landrum and Brown Consultant Amiy Varma Professor University of North Dakota Chair of TRB Committee of Aircraft/ Airport Compatibility Ernie Heymsfield Professor University of Arkansas Member of TRB Committee of Aircraft/ Airport Compatibility Michael A. Meyers Government FAA - AAS-100 Engineer in the Airport Engineering Division Ken Jacobs Government FAA – APP-400 FAA Liaison for ACRP 4- 01 Table G1. List of volunteers to test analysis software.

ACRP 4-08—Improved Models for Risk Assessment of Runway Safety Areas (RSA) Analysis Software Evaluation Questionnaire The purpose of this software beta testing effort is to test and help improve the software for analysis of runway safety areas. Although measuring software effectiveness is no easy task, the feedback provided will help identify the need for critical improve- ments to the software. Name:________________________________________________ Organization: __________________________________________ Position: ______________________________________________ 1. How easy was it to install the software? a. Difficult to install b. I had problems c. About right d. Easy to install Comments: _______________________________________________________________________________ _______________________________________________________________________________ _______________________________________________________________________________ _______________________________________________________________________________ 2. How easy was it to follow the user guide and documentation? a. Very difficult to follow b. It is necessary to understand risk assessment to use it c. Simple guidance but satisfactory for the purpose d. Easy to follow Comments: _______________________________________________________________________________ _______________________________________________________________________________ _______________________________________________________________________________ _______________________________________________________________________________ 3. Are the screens user-friendly and easy to understand? a. No b. I had a few problems (see my comments) c. Easy to follow Comments: _______________________________________________________________________________ _______________________________________________________________________________ _______________________________________________________________________________ _______________________________________________________________________________ 4. Was it easy to input operational data? a. No b. I had a few problems (see my comments) c. Yes G-3

Comments: _______________________________________________________________________________ _______________________________________________________________________________ _______________________________________________________________________________ _______________________________________________________________________________ 5. Was it easy to input weather data? a. No b. I had a few problems (see my comments) c. Yes Comments: _______________________________________________________________________________ _______________________________________________________________________________ _______________________________________________________________________________ _______________________________________________________________________________ 6. Was it easy to understand output results? a. No b. I had a few problems (see my comments) c. Yes Comments: _______________________________________________________________________________ _______________________________________________________________________________ _______________________________________________________________________________ _______________________________________________________________________________ 7. Please list the good and the bad points of the software. _______________________________________________________________________________ _______________________________________________________________________________ _______________________________________________________________________________ _______________________________________________________________________________ 8. Would you use this software again? a. Yes b. Possibly (see my comments) c. No (see my comments) _______________________________________________________________________________ _______________________________________________________________________________ _______________________________________________________________________________ _______________________________________________________________________________ 9. How long did it take to run the “Example 1” analysis?________ minutes 10. Any other comments that you care to offer. _______________________________________________________________________________ _______________________________________________________________________________ _______________________________________________________________________________ _______________________________________________________________________________ G-4