Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
4 This chapter introduces users to performance metrics with an emphasis on those included in the Reference Guide and Smart Guide. Beginning with the definition of performance metrics, this chapter goes on to discuss uses and types of performance metrics. This chapter concludes with a discussion of the FAA harmonized performance metrics.1 2.1 Definition Simply put, a performance metric is a standard for measuring performance. Other related terms include performance indicator and performance measure. For example, the International Civil Aviation Organization (ICAO) refers to performance metrics as performance indicators and defines them as âa tool for quantitatively measuring current, past and expected future per- formance, as well as the degree to which performance objectives are being met and should be met.â2 For the purposes of this Reference Guide, performance metrics are considered synony- mous with performance indicators and performance measures. 2.2 Using Performance Metrics Performance metrics are used to measure the output of a process or procedure.3 By mea- suring output, an entity can determine how its performance has changed over time, how its performance compares to another entityâs performance, and whether performance goals are being met. Performance metrics can be used to evaluate performance at any level: organiza- tional, departmental, individual, or process. Regardless of level, measuring performance with metrics can be used to drive performance-based actions, enhance decision making, and facilitate communication. 2.2.1 Performance Measurement Systems Performance may be evaluated using performance metrics via a performance measurement system. The goal of performance measurement systems is to achieve targeted performance levels, often through graphical and numerical information such as a performance dashboard or scorecard.4 ACRP Report 19: Developing an Airport Performance-Measurement System and C H A P T E R 2 Introduction to Performance Metrics 1 U.S. Federal Aviation Administration (FAA). âOperational Metrics.â Accessed August 7, 2017. https://www.faa.gov/data_research/ aviation_data_statistics/operational_metrics/. 2 International Civil Aviation Organization (ICAO). Airport Economics Manual, 3rd ed. Montréal, Canada, 2013. https://www. icao.int/sustainability/Documents/Doc9562_en.pdf, p. APP 1â2. 3 ACRP Report 19: Developing an Airport Performance-Measurement System. Washington, D.C., 2010, p. 17. 4 Harbour, Jerry L. The Basics of Performance Measurement. New York, NY, 2009, p. 84.
Introduction to Performance Metrics 5 ICAOâs Airport Economics Manual provide guidance on developing airport-wide performance measurement systems. Performance metrics are an integral component of performance measurement systems. Development of a performance measurement system includes selecting metrics to be measured, evaluated, and reported. Metrics are selected by first identifying key performance indicators (KPIs). KPIs measure the performance of the factors most critical to an organizationâs success.5 To identify KPIs and select appropriate metrics, airports may refer to ACRP Report 19A: Resource Guide to Airport Performance Indicators, which provides an extensive list of airport performance indicators that support the development of an airport performance measurement system. 2.2.2 Benchmarking Performance metrics are also used to compare or benchmark performance. Internal bench- marking occurs when an organization compares its performance metrics results over time. External or peer benchmarking occurs when an organization compares its performance to that of another organization. By nature, external benchmarking is more complex than internal benchmarking. In order to compare âapples to apples,â the characteristics of the comparison organization must be considered. This is particularly true for airports. When comparing one airport to another, some of the typical factors that drive different results and should be considered in making comparisons include passenger volume, capacity constraints, mix of international and domestic traffic, mix of local and transfer passengers, mix of passenger carrier service (network, low cost, charter), mix of passenger versus cargo activity, degree of outsourcing, range of services provided by the airport, airport development program status, weather conditions, geographic location, urban versus rural location, physical size of the airport, public transportation access and usage, regulatory environment, local labor conditions, and ownership and governance structure.6 More information about external benchmarking is provided in Section 3.4, Benchmarking across Airports. 2.2.3 Using the Performance Metrics Database Thoughtful selection and use of the metrics in the Performance Metrics Database is necessary for multiple reasons. First, not all the metrics are applicable to all airports. Second, only some metrics are applicable to external benchmarking. Third, no one metric is the answer. Fourth, data are needed to derive metrics and thus it is important to consider the sources, parameters, and availability of data. Depending on the airportâs size and operating characteristics, some metrics in the Performance Metrics Database are more appropriate and/or useful than others when evaluating performance. Because not every airport is measuring the same processes or procedures, the performance metrics in the Performance Metrics Database are not universally appropriate and/or useful to all air- ports. For example, metrics such as Based Aircraft and Average Cost per Gallon Paid for Aviation Gasoline are obviously more useful for general aviation airports while Runway Occupancy Time and Average Gate Departure Delay are more useful and appropriate for congested commercial service airports. Some metrics in the Performance Metrics Database are not good candidates for external benchmarking. Metrics like Noise Exposure are not useful for external benchmarking because 5 Parmenter, David. Key Performance Indicators: Developing, Implementing, and Using Winning KPIs. Hoboken, NJ, 2015, p. 7. 6 Wyman, Oliver. ACI Guide to Airport Performance Measures, Quebec, Canada, 2012, p. 2.
6 Common Performance Metrics for Airport Infrastructure and Operational Planning each airport has unique circumstances when it comes to distances to noise sensitive resources. Metrics that are appropriate for external benchmarking are discussed in Section 3.4, Bench- marking across Airports. No one metric in the Performance Metrics Database is intended to address measurement of all of the issues in a focus area. A combination of metrics is likely required to understand the issue. For example, considering Average Out-to-Off taxi time alone might not tell the whole story. It may be important to also consider Runway Usage to understand whether a reduction in Average Out-to-Off taxi time is due to change in runway use and not Wake Recategorization (Wake ReCat). Data availability and applicability is an important factor when selecting metrics in the Perfor- mance Metrics Database. Data-related information including sources and guidance is provided in Chapter 4, Data Sources and Considerations. The Performance Metrics Database includes a number of metrics for each of the focus areas. To assist the user in selecting metrics within a focus area, the metrics in the Performance Metrics Database are identified as primary and secondary. The identification of metrics as primary and secondary is somewhat subjective and is intended only to assist users in initially selecting metrics. PrimaryâMetrics that are most useful in evaluating performance within a focus area. SecondaryâMetrics that may be useful in evaluating a certain aspect of a focus area. Second- ary metrics include metrics that the FAA uses to measure system-wide performance in the focus area. While these metrics might not be applicable to a specific airport, they are pro- vided to enhance user knowledge and awareness. Neither primary metrics nor secondary metrics are necessarily exclusive to one focus area. Also, a metric might be primary for one focus area and secondary for a different focus area. 2.3 Types of Performance Metrics Performance metrics can be classified in several ways. Understanding these classifications is useful in selecting metrics within a focus area. The classifications most relevant to the Reference Guide are discussed in the following paragraphs. Performance metrics may be classified as input, output, and outcome. InputâMeasures that describe the resources the airport has, such as the number of gates or jet bridges. While input measures are not in and of themselves measures of perfor- mance, they are important in providing context for other performance measures.7 Input measures may be used as a divisor when calculating a performance metric. For example, the number of gates is used to derive the turns per gate metric. Input measures may also be considered when comparing the performance of airports via external benchmarking. Input measures such as number of runways may be one of the input measures considered when selecting airports for benchmarking. OutputâMeasures that quantify the service provided.8 For airports, examples include number of enplanements and number of operations. OutcomeâMeasures that âprovide feedback on the quality and efficiency of services or on the intended performance of the organization.â9 Examples of airport outcome metrics include average delay per aircraft and aircraft operations per gate. 7 ACRP Report 19: Developing an Airport Performance-Measurement System. Washington, D.C., 2010, p. 21. 8 Ibid. 9 Ibid.
Introduction to Performance Metrics 7 Another way to classify metrics is by past, current, and future performance measures. These classifications align nicely with the ICAO definition of performance metrics provided in Section 2.1. PastâMeasures of past activities over months, quarters, or years. Examples of past airport metrics are annual enplaned passengers and annual operations. CurrentâMeasures that are monitored in real time or daily. Current aviation metrics include airport arrival rate and airport departure rate. FutureâMeasures of expected (planned) future performance. Potential aviation future metrics include calculated capacity and delay with forecasted aircraft operations. 2.4 FAA Harmonized Metrics The FAA uses a set of harmonized performance metrics to report performance on an agency-wide basis. These metrics show how the FAA is performing in the areas of safety, efficiency, capacity, environment, and cost-effectiveness. Harmonized metrics that apply to airports are included in the Performance Metrics Database. These metrics are included to provide airport personnel with insight about what the FAA measures and how they measure it. Table 1 shows the FAA harmonized metrics that are included in the Performance Metrics Database. Note that the scopes of these metrics are system-wide, such as the entire National Airspace System (NAS) or all of the FAAâs 30 Core Airports (e.g., Taxi-In Time is an average of carrier-reported taxi-in times at all Core Airports). Performance Area Metric Name Scope Included Safety Runway Incursions Rate (A&B) NAS Efficiency Taxi-In Time Core Airports* Taxi-Out Time Core Airports* Average Gate Arrival Delay Core Airports* Number of Arrival and Departure Delays Core Airports* Number of Operations Core Airports* NAS On-Time Arrivals Core Airports* Capacity Average Daily Capacity (ADC) Core Airports* and times of day relevant to the operations Average Daily Operations Core Airports* and times of day relevant to the operations Runway Pavement Condition Paved and open runways in the National Plan of Integrated Airport Systems (NPIAS) Environment Noise Exposure U.S. population Emissions Exposure (CO2 Emissions) NAS *Core Airports include Hartsfield-Jackson Atlanta International Airport (ATL), Baltimore/Washington International Thurgood Marshall Airport (BWI), Boston-General Edward Lawrence Logan Airport (BOS), Charlotte Douglas International Airport (CLT), Chicago Midway International Airport (MDW), Chicago OâHare International Airport (ORD), Dallas/Fort Worth International Airport (DFW), Denver International Airport (DEN), Detroit Metropolitan Wayne County Airport (DTW), Fort Lauderdale-Hollywood International Airport (FLL), HonoluluâDaniel K. Inouye International Airport (HNL), HoustonâGeorge Bush Intercontinental Airport (IAH), Las VegasâMcCarran International Airport (LAS), Los Angeles International Airport (LAX), Memphis International Airport (MEM), Miami International Airport (MIA), Minneapolis-St. Paul International Airport (MSP), New YorkâJohn F. Kennedy International Airport (JFK), New YorkâLaGuardia Airport (LGA), Newark Liberty International Airport (EWR), Orlando International Airport (MCO), Philadelphia International Airport (PHL), PhoenixâSky Harbor International Airport (PHX), Salt Lake City International Airport (SLC), San Diego International Airport (SAN), San Francisco International Airport (SFO), Seattle-Tacoma International Airport (SEA), Tampa International Airport (TPA), Ronald Reagan Washington National Airport (DCA), and Washington Dulles International Airport (IAD). Source: âFAA Operational Metrics,â https://www.faa.gov/data_research/aviation_data_statistics/operational_metrics/. Table 1. FAA harmonized metrics in the Performance Metrics Database.