Simulation Options for Airport Planning (2019)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

6 At the beginning of a simulation study, analysts generally ask themselves certain questions that can help them frame the problem scope and determine analyses concepts. The types of questions that are generally asked include: • Which parts of the airfield, airspace, terminal structure, and roadways should be simulated? • What simulation tool fidelity is appropriate for the analysis, available resources, and overall project scope? • What input data are available and needed to build the models? • Which metrics will support the larger analyses and are meaningful? • What visualization and animation capabilities are desired for analysis and presentation? • Which tools and methods have other organizations used for similar projects and environments? The answers to these types of questions drive simulation tool choices. They were also used to guide the information contained in this report in terms of the overall selection framework and relevant information presented for each simulation tool. Simulation studies can vary significantly in terms of complexity and resource and data requirements, but the knowledge of a specific simulation tool needs to be combined with a fundamental understanding of simulation and analysis concepts to produce meaningful results. In addition to the ability to use a specific software or concept, the execution of a successful simulation and analysis project depends heavily on the understanding of mathematical, analyti- cal, and fast-time simulation concepts. This section of the report provides basic information on simulation processes, data gathering, and data requirements, as well as commonly used analysis metrics for the reader. Understanding some of these basic concepts is a key factor in choosing simulation tools for specific projects. With increased simulation tool complexity and functionality capabilities comes the need for more informed and structured simulation development and analysis methods. Whereas basic concepts of capacity, delay, and queueing theory may be sufficient to conduct macroscopic level studies, high-fidelity microscopic simulation tools need an increased level of expertise, resources, and input data. Their ability to model detailed objects as well as to incorporate dynamic demand patterns requires a detailed understanding of simulation methodologies, input and output data, metrics, and analysis techniques. This is particularly true in an increasingly connected world where simulation technology is used to address large-scale system-of-system problems at a complex level with competing objectives (Nelson, 2016). Regardless of the level of detail of analysis or how outputs from the simulation analysis are intended to be used, the methodologies need to be sound and the results defendable. This is par- ticularly true considering the size of capital investment decisions that are sometimes supported by the simulation results. C H A P T E R 2 Basic Simulation Concepts for Airport Planning and Design

Basic Simulation Concepts for Airport Planning and Design 7 Moreover, the use of simulation tools should be integrated into a collaborative process that involves all affected stakeholders. At every step in the process, educated and collaborative decisions need to be made, starting with the level of analysis required, data availability and requirements, resource availability, simulation tool selection, calibration capabilities and require- ments, design of experiments, and understanding of the limitations of the simulation tools themselves. Modeling Versus Simulation One of the most common misunderstandings about simulation studies is the difference between modeling and simulation. While many practitioners use these terms interchangeably and in sequence, they technically have different meanings. These differences should be under- stood and communicated to minimize misunderstandings across stakeholders and within the simulation study team. Models are representations of reality or concepts that can be useful for visualization pur- poses or to perform “static” analyses. This may include three-dimensional (3-D) or even four- dimensional (4-D) visualizations of aircraft or passenger operations. Simulations, on the other hand, provide a “dynamic” capability that allows the user to change system and environment variables and observe the perceived differences in operation. So, the purpose of simulation is to study system behavior through the manipulation of variables, that is, the addition of runways, changes in terminal layouts, etc., that cannot be, or are impractical to be, changed in real life. In essence, the model can be considered a product or an instance of an operation. A simula- tion is the process of using models for further analysis and decision-support purposes. Under- standing this difference can be crucial in the setting of expectations and the understanding of why and when simulation tools are used. Throughout a planning study, both static models and dynamic simulations may be used in the decision-making process. Typically, more static analyses using spreadsheet models and tables are used in the initial stages of a project to assess high-level capacity and delay con- straints. Results from this initial analysis then justify and feed higher-fidelity dynamic simu- lation studies. Although this needs to be evaluated on a project-by-project basis, certain types of projects suggest the use of simulation tools in addition to static models. These include: • Projects that focus on “what-if” studies to determine the best possible solution and that involve significant levels of uncertainty. • Projects of high cost or visibility. • Projects in which dynamic visualizations of objects are important. • Projects in which the likely outcomes result in environmental studies, regulatory changes, or other significant impacts on stakeholders. • Projects with multiple focus areas (i.e., airfield, ground, terminal) and multiple potential bottlenecks. • Projects for which it is infeasible or impractical to gather all operational data to construct static prediction models. • Projects for which mathematical simplifications are not feasible. Another important aspect of simulation studies that should also not be misconstrued is simulation-based optimization. Some airport planners consider simulation tools as a means for optimizing airside operations, passenger flows, and resource use. In fact, very few simulation tools—mainly the more generic tools—offer optimization modules where inputs are varied

8 Simulation Options for Airport Planning structurally until a specific objective is maximized. The vast majority of simulation tools simply run scenarios and environments that have been defined and developed by the simula- tion analyst with very limited sensitivity analyses based on specific randomized input variables. It is up to the simulation analyst to develop an experimental plan that explores alternative what-if scenarios sufficiently to be able to determine the “best” solution for a specific study. This by no means guarantees an optimal solution. Simulation Process An analysis of airport, passenger, or curbside operations can take many forms and be accom- plished using various analytical and simulation tools. From simple spreadsheet models to queue- ing theory analyses to highly detailed dynamic simulation tools, the general concepts and tools that support these studies include: • Spreadsheet Models based on prediction models or processes using historically observed data. • Queueing Theory, in which dynamic demand can be evaluated on a network of capacitated resources such as check-in counters and security check lines and overall delays and resource utilization are captured. • Optimization Techniques that use such concepts as dynamic programming to estimate maxi- mum throughputs, resource utilization, etc. • Monte Carlo Simulation, which uses repeated random sampling based on historical proba- bilistic distributions of such processes as security check times and passenger transit times to obtain statistically valid and more realistic results for the evaluation of operational processes. • Discrete-Event Simulation, which is the process of structuring and modeling the behavior of a complex system as a discrete sequence of events in time where object states and environmental variables change over time. All of these different concepts support airport planning and design decisions at specific levels and for very specific purposes. Some lend themselves more toward airfield or airspace studies, whereas others are really developed for passenger and process flows. All tools produce a plethora of output data and require the modelers to interpret, process, and display information. Some tools provide better model visualization capabilities than others. Airport planners today use different fidelity simulation tools to support decision making at various levels through the use and evaluation of what-if alternative scenarios. A basic simula- tion study process is illustrated in Figure 2. Although this process is logically laid out to be linear and sequential, most simulation studies will iterate through various phases multiple times to account for errors in data and interpretations and changes in requirements and to manage internal simulation variability that detracts from the study methodology and mission. An important aspect of simulation study project management is the development of a sound execution plan. This can be particularly valuable for simulation studies with large numbers of scenarios. The experimental design defines basic information about each simulation scenario and can be a valuable tool to depict simulation input changes in what-if scenarios and guide subsequent analyses. It can be used to demonstrate to team members which scenario variables are fixed and which are changing and thus subject to analysis. Another important factor to consider for simulation studies is the idea of multiruns, or the incorporation of statistical uncertainty into otherwise deterministic simulations. This practice has its roots in Monte Carlo simulation and adds a level of statistical validity to the output metrics based on random natural variations to input parameters such as demand profiles, entity characteristics or behavior, and resource processing times.

Basic Simulation Concepts for Airport Planning and Design 9 Simulation Calibration and Validation Before any what-if scenarios are tested and simulated, however, the baseline simulation models should be calibrated and validated to ensure that they reflect current reality, or intended reality, within reason. This can be accomplished visually for high-fidelity tools, as well as by using simulation output metrics. For instance, simulation models can be calibrated against historical busy-day observed throughputs and delays to ensure they represent a valid baseline for further study. Some metrics and resources that may be useful for calibrating baseline models include • ASPM/OpsNet Data for hourly throughputs, weather conditions, and runway configurations (FAA website access may be required). • Bureau of Transportation Statistics (BTS) data for taxi times and airline-reported delays. • System wide information management (SWIM) Traffic Flow Management (TFM) data for air- craft tracks, throughputs, and flight schedules (FAA SWIM data access required). • Airport Surface Detection Equipment, Model X (ASDE-X) data for ground tracks and arrival and departure records (FAA data access required). • Airline postoperational analysis data from tools such as POET (Post Operations Evaluation Tool). Simulation models can subsequently be validated operationally and visually by subject matter experts who are familiar with the given operation. These experts may include airport personnel Figure 2. Basic simulation study process.

10 Simulation Options for Airport Planning responsible for managing operations, air traffic controllers, or airline representatives. If simu- lation models are intended to be used for future studies, periodic recalibration may also be required to keep them up-to-date and ensure that infrastructure, environmental, operational, and procedural changes are reflected. What should be understood in this calibration and validation process is that a simulation of a specific operational environment will never exactly match—or replicate—the actual opera- tion. Rather than trying to exactly match historic operational characteristics, the analyst may be better off focusing on trend analyses and ranges that are aggregated for a specific configuration, weather pattern, or demand level. A calibration against historical data should also appreciate the variability in the system and that periods of irregular operations may skew historical data analyses. An experienced analyst will need to manage simulation project expectations and ensure that the important aspects of the models are “good enough” or reasonable from an operational reality perspective and subsequent what-if analyses. It should also be noted that FAA does not prescribe the use of specific simulation tools for airfield, airspace, and terminal planning studies. The choice of which simulation tool to use for a study is still very much left to the simulation analyst and sponsor of the analysis. However, to consider the simulation study as valid, FAA requires that studies using new simulation tools— those that have not previously been used on similar studies for FAA—contain a “simulation tool validation” process step. This step should not only validate that baseline simulations can be calibrated against real-world data but also that the simulation tool is internally verified and that output data is validated. Simulation Data Requirements Input data requirements for analytical and simulation tools change according to the level of fidelity. The level of detail required also varies depending on the area of applicability, the study methodology, and the intended analysis metrics. Generally, the types of simulation input data can be categorized into: • Published data such as airport and terminal layouts, procedures, flight schedules, and aircraft performance characteristics. • Collected data, which include information that is derived from historical data or manually collected such as process times, arrival or departure separations, etc. • Visual/observed data, which are based on the direct observation of specific processes and oper- ational procedures. The collected information is then transformed into simulation input data. The following datasets represent several common simulation inputs (a more complete listing can be found in Appendix B). • Flight, passenger, and vehicle demand schedules. • Terminal passenger characteristics and process time distributions. • Airport or airfield, airspace, terminal, and roadway designs and configuration. • Vehicle configurations and performance characteristics. • Operating rules and restrictions. As an example, a runway capacity study may simply require information on the expected busy arrival or departure demand period and fleet mix. This demand information can sim- ply reflect a single busy design hour to evaluate scenarios under high demand or load. However, a higher fidelity dynamic airport or airside simulation will need either a flight schedule based on a full historic high season, a busy day, or a detailed design-day flight schedule (DDFS) that is specially created to provide a clean and “pure” demand scenario based on known data and variables.

Basic Simulation Concepts for Airport Planning and Design 11 Care should be taken, however, to ensure the DDFS accurately reflects historic demand trends (Landrum & Brown et al., 2010). The collection, processing, and integration of data throughout the development of the simulation models also require care. A common problem that is repeated by many industry practitioners and the importance of which cannot be understated is that simulation study results and analyses are only as accurate as the data used to develop the models. Analysis and Metrics The effective use of simulation tools also requires analysts to have strong statistical analysis capabilities and an understanding of how the system works. A basic understanding of means, standard deviations, confidence intervals, histograms, and other basic statistics forms the basis of sound simulation analyses practices and produces results that are meaningful and defendable. Core metrics form the key evaluation criteria for simulation scenarios. These metrics are pri- marily numerical and based on mathematical derivations of simulation outputs, but they may also be in the form of derived visual representations of numerical data (e.g., maps, charts, and other infographics). A good analyst will usually find ways of extracting information from simu- lation tool output data to support the conclusions of a study. A description of several commonly produced metrics for airport planning and design studies follows. Throughput and Capacity From an airport or airfield perspective, capacity is a measure of the maximum number of aircraft operations that can be accommodated on the airport or by an airport component within a given period of time. Capacity calculations range from low-fidelity estimates based on simple assumptions about the runway layout to quick spreadsheet models to complex and high-fidelity simulation results (TransSolutions et al., 2014). Throughput and capacity can be analyzed in various ways. Lower fidelity analytical models will generally produce single summary statistics for capacity that can describe • Maximum capacity. • Maximum sustainable capacity. • Practical capacity. At the most detailed level, graphs and tables of daily or hourly throughput—based on the DDFS or other dynamic demand tables—can be used to demonstrate daily throughput varia- tions and provide a detailed comparison of how well a system can meet anticipated daily or hourly demand patterns. Generally, hourly throughputs are used as the basis for reports and as a metric for summary statistics of throughputs and capacity. Delay Delay can be associated with almost all objects on the airfield and within or near the terminal. Aircraft encounter delays in the airspace, at the departure runway, throughout taxiing, and at the gate. Passenger flow models will show delays at various queueing locations, including check-in counters, security checkpoints, and the like. Vehicles on roadways can experience congestion at various locations where they access or exit an airport.

12 Simulation Options for Airport Planning Capacity-driven delays can be accurately predicted using higher detail simulation software. Generally, to calculate delays, a nominal or unimpeded time to complete a specific process of function is calculated, and then any additional time is measured as “delay” (TransSolutions et al., 2014). However, the underlying simulation events for this delay calculation may differ slightly based on the simulation tool, and the differences need to be understood by the modeler and project sponsor. Some simulation tools may simply provide summations of individual process delays as a total delay for an aircraft, passenger, or vehicle without taking into consideration excess time experienced by longer travel paths and routine delays. For example, the Total Airspace and Airport Modeler (TAAM) does not natively count taxiway reroutes or pushback pauses in delay calculations. This can be counterintuitive to airline definitions of ground delay, which simply measure the differences in expected arrival, departure, or transit times versus actual times. Delays can be used to support cost–benefit analysis (CBA) by simply applying cost factors to individual delay types: aircraft operating cost (air and ground), aircraft nonoperating cost, and passenger time cost. A more detailed FAA description of guidance to support capacity-driven CBA studies can be found on the FAA’s regulations and policies website (FAA, 2017). Annualization Often delays and associated costs also need to be annualized to support CBAs. Care should be taken in the annualization of airspace and airfield simulation data if meteorological information is used as a basis for this scaling. It is common practice to use periodic airport meteorological weather information (METARs) in this analysis, but some airports are more affected by con- vective weather forecasts at specific times of the year, which are frequently ignored. Simulation-based daily metrics can also be scaled using annual service volume information— either aircraft, passengers, or vehicles—as well as proportion-of-time information for each configuration modeled. However, the analyst should take care in defining a sound methodol- ogy for this approach as it can be based on various factors, including weighted capacity and throughput information or aircraft and passenger delays. Other Metrics Other metrics that are frequently used in the analysis of simulation data include: • Airspace sector throughput and occupancy. • Resource usage (gates, check-in counters, security checkpoints, curb allocation). • Workload (primarily air traffic control [ATC]-related). In addition to detailed analysis statistics, other metrics are frequently used to visualize and quantify areas of interest or concern. These types of secondary metrics are geared toward high- fidelity simulation tools and can include: • Choke points where high localized delays are experienced. • Conflict points where airspace, airfield, or passenger flows frequently cross or conflict. • Heat maps displaying “busy” areas with high traffic volume. Quality Assurance and Control As with any simulation project, there are best practices and processes that should be con- sidered to ensure that a study produces high quality as well as dependable results. This is particularly important considering the capital costs and operational and safety impacts of the

Basic Simulation Concepts for Airport Planning and Design 13 types of projects that airport planners typically consider for simulation studies. In addition to a thorough understanding of the simulation tools that are to be used, as well as experience in managing simulation studies, some of the basic quality assurance (QA) concepts that should be incorporated into all simulation studies include: • The use of industry best practices for developing simulation scenarios, extracting metrics, and analyzing results. A specific example of best practices that is available to simulation analysts is the TAAM Best Practices Guideline. This document assembled a collection of industry best practices in the context of project planning and execution for applying TAAM to aviation systems simulations (Boesel et al., 2001). Similarly, Eurocontrol documented a TAAM Operational Evaluation project that also included many best practices for conducting fast-time airport and airspace simulation studies that extend beyond the use of the TAAM software (Sillard et al., 2000). Although some of these documents may be quite outdated in terms of simulation tool functionality and versions, the basic principles illustrated in these guidelines can still be very applicable. • A focus on the development of use cases and an understanding of variables and metrics at the beginning of the project rather than as the project evolves. The early identification of simulation project risks, data availability issues, and other potential shortcomings is key in maximizing the potential of a study and minimizing simulation study execution risks. • Basic systems engineering principles such as verification and validation of simulation scenarios against the study scope and requirements and use of a sound documentation and change control process. • A sound information and document version control system that ensures simulation input data integrity across numerous scenarios and tracks changes in simulation input assumptions and derived input data. • A detailed understanding of the roles and responsibilities across the simulation team, including simulation analysts, airport staff, airlines, and other aircraft operators. Seasoned simulation analysts will set and manage expectations at the beginning of a project to minimize the risk of failure.