National Academies Press: OpenBook
« Previous: Chapter Two - Resources Available to Airportss
Page 14
Suggested Citation:"Chapter Three - Survey Results ." National Academies of Sciences, Engineering, and Medicine. 2016. Tabletop and Full-Scale Emergency Exercises for General Aviation, Non-Hub, and Small Hub Airports. Washington, DC: The National Academies Press. doi: 10.17226/23584.
×
Page 14
Page 15
Suggested Citation:"Chapter Three - Survey Results ." National Academies of Sciences, Engineering, and Medicine. 2016. Tabletop and Full-Scale Emergency Exercises for General Aviation, Non-Hub, and Small Hub Airports. Washington, DC: The National Academies Press. doi: 10.17226/23584.
×
Page 15
Page 16
Suggested Citation:"Chapter Three - Survey Results ." National Academies of Sciences, Engineering, and Medicine. 2016. Tabletop and Full-Scale Emergency Exercises for General Aviation, Non-Hub, and Small Hub Airports. Washington, DC: The National Academies Press. doi: 10.17226/23584.
×
Page 16
Page 17
Suggested Citation:"Chapter Three - Survey Results ." National Academies of Sciences, Engineering, and Medicine. 2016. Tabletop and Full-Scale Emergency Exercises for General Aviation, Non-Hub, and Small Hub Airports. Washington, DC: The National Academies Press. doi: 10.17226/23584.
×
Page 17
Page 18
Suggested Citation:"Chapter Three - Survey Results ." National Academies of Sciences, Engineering, and Medicine. 2016. Tabletop and Full-Scale Emergency Exercises for General Aviation, Non-Hub, and Small Hub Airports. Washington, DC: The National Academies Press. doi: 10.17226/23584.
×
Page 18
Page 19
Suggested Citation:"Chapter Three - Survey Results ." National Academies of Sciences, Engineering, and Medicine. 2016. Tabletop and Full-Scale Emergency Exercises for General Aviation, Non-Hub, and Small Hub Airports. Washington, DC: The National Academies Press. doi: 10.17226/23584.
×
Page 19
Page 20
Suggested Citation:"Chapter Three - Survey Results ." National Academies of Sciences, Engineering, and Medicine. 2016. Tabletop and Full-Scale Emergency Exercises for General Aviation, Non-Hub, and Small Hub Airports. Washington, DC: The National Academies Press. doi: 10.17226/23584.
×
Page 20
Page 21
Suggested Citation:"Chapter Three - Survey Results ." National Academies of Sciences, Engineering, and Medicine. 2016. Tabletop and Full-Scale Emergency Exercises for General Aviation, Non-Hub, and Small Hub Airports. Washington, DC: The National Academies Press. doi: 10.17226/23584.
×
Page 21
Page 22
Suggested Citation:"Chapter Three - Survey Results ." National Academies of Sciences, Engineering, and Medicine. 2016. Tabletop and Full-Scale Emergency Exercises for General Aviation, Non-Hub, and Small Hub Airports. Washington, DC: The National Academies Press. doi: 10.17226/23584.
×
Page 22
Page 23
Suggested Citation:"Chapter Three - Survey Results ." National Academies of Sciences, Engineering, and Medicine. 2016. Tabletop and Full-Scale Emergency Exercises for General Aviation, Non-Hub, and Small Hub Airports. Washington, DC: The National Academies Press. doi: 10.17226/23584.
×
Page 23
Page 24
Suggested Citation:"Chapter Three - Survey Results ." National Academies of Sciences, Engineering, and Medicine. 2016. Tabletop and Full-Scale Emergency Exercises for General Aviation, Non-Hub, and Small Hub Airports. Washington, DC: The National Academies Press. doi: 10.17226/23584.
×
Page 24
Page 25
Suggested Citation:"Chapter Three - Survey Results ." National Academies of Sciences, Engineering, and Medicine. 2016. Tabletop and Full-Scale Emergency Exercises for General Aviation, Non-Hub, and Small Hub Airports. Washington, DC: The National Academies Press. doi: 10.17226/23584.
×
Page 25
Page 26
Suggested Citation:"Chapter Three - Survey Results ." National Academies of Sciences, Engineering, and Medicine. 2016. Tabletop and Full-Scale Emergency Exercises for General Aviation, Non-Hub, and Small Hub Airports. Washington, DC: The National Academies Press. doi: 10.17226/23584.
×
Page 26
Page 27
Suggested Citation:"Chapter Three - Survey Results ." National Academies of Sciences, Engineering, and Medicine. 2016. Tabletop and Full-Scale Emergency Exercises for General Aviation, Non-Hub, and Small Hub Airports. Washington, DC: The National Academies Press. doi: 10.17226/23584.
×
Page 27
Page 28
Suggested Citation:"Chapter Three - Survey Results ." National Academies of Sciences, Engineering, and Medicine. 2016. Tabletop and Full-Scale Emergency Exercises for General Aviation, Non-Hub, and Small Hub Airports. Washington, DC: The National Academies Press. doi: 10.17226/23584.
×
Page 28

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

14 As noted in chapter one, this study investigated current practices regarding emergency exercises at airports of all types and sizes. Furthermore, the study also identified concepts, tools, and procedures that GA, non-hub, and small hub airports can use. This chapter summarizes and interprets the infor- mation gained through the survey; the complete raw data set is presented in Appendix A. The dis- cussions in this chapter are keyed to the survey question numbers as they appear in that appendix. AIRPORT STRUCTURE Nearly half the airports are city departments, about two-fifths (42%) are authorities, 8% are county departments, one has a joint board, and one is privatized. See Question 8 in Appendix A. NATIONAL INCIDENT MANAGEMENT SYSTEM AND INCIDENT COMMAND SYSTEM IMPLEMENTATION Question 10 of the survey asked the airports which of their employees that would be involved in emergency exercises or in emergency communications, or both, had formal training in the imple- mentation of National Incident Management System (NIMS) or in Incident Command System (ICS). The positions addressed by this question were emergency manager, emergency planning, training officer, exercise designer, operational program planner, operational evaluator, risk/hazard manager, public information officer, or other position identified by the airport. Table 2 shows the results by NPIAS category. As the data indicate, most airports have at least one key position trained on NIMS and ICS, which is likely because of the importance of such training articulated by FEMA and the issuance of FAA Advisory Circular 150/5200-31C in 2009. This result holds across all seven NPIAS categories. A similar picture was seen across all governance structures. The result for smaller airports may be indicative of their close relationships with their community emergency response partners (Smith 2012, 2014). The survey did not directly ask whether the airports found NIMS or ICS to be important for the creation and maintenance of an effective emergency exercise program. However, in an examination of post-event recovery practices at 37 airports, ACRP Synthesis 60 (Smith et al. 2015) showed that the airports overwhelmingly found that application of NIMS and ICS aided their emergency man- agement efforts. The data in Table 2 show a major commitment by more than 80% of the airports in this study to NIMS and ICS training. Again, it should be noted that the airports in this study are not a random sample, but were selected as likely to have superior emergency exercise or emergency communications programs. AIRPORT STAFFING Survey Questions 9 and 10 addressed the sorts of specialized positions the airports have for emer- gency management, exercises, and public information; analysis of their responses is presented in Table 3. Thirty-two percent (32%) of the airports surveyed have emergency managers employed full time, and slightly more, 35%, have emergency management as part of an employee’s duties, so the chapter three SURVEY RESULTS

15 NPIAS Category No Position Has NIMS/ICS Training One or More Positions Have NIMS/ICS Training Do Not Know LH 0 0% 13 100% MH 1 17% 4 80% SH 0 0% 7 100% NH 1 14% 6 86% CS 1 33% 2 67% RL 1 25% 6 86% GA 2 25% 5 63% 1 12% Total 6 13% 43 83% 1 2% Source: Smith, Garcia, Sawyer, and Kenville data. TABLE 2 NIMS AND ICS TRAINING BY NPIAS CATEGORIES Positions Reported from 50 Airports Have FT on Airport Staff Have PT on Airport Staff Pay FT at Other Agency Pay PT at Other Agency FT Role Part of Current Duties PT Role Part of Current Duties Total Public Information Officer (PIO) 26 2 4 3 6 7 48 Emergency Manager 14 1 3 1 9 15 43 Emergency Planner 7 1 2 0 12 18 40 Training Officer 14 1 1 1 10 12 39 Exercise Designer 5 1 2 2 11 19 40 Exercise Evaluator 5 0 4 3 9 19 40 Operational Program Planner 7 2 3 1 9 16 38 Operational Evaluator 6 1 3 1 9 19 39 Risk/Hazard Manager or Equivalent Role 15 2 4 3 7 11 42 ARFF Training Officer 1 0 0 0 0 0 1 Planner, Training, and Exercise Designer and Evaluator all done by Emergency Manager 1 0 0 0 0 0 1 Source: Smith, Garcia, Sawyer, and Kenville data. TABLE 3 STAFFING CHARACTERISTICS OF SURVEYED AIRPORTS majority of the airports surveyed are dedicating human resources to the emergency management process at the nation’s airports either full- or part-time. These findings apply not only to large air- ports but to the full range of airports in the study, as detailed in the sidebar. Forty of the 50 (80%) airports giving full responses have an emergency planner and 43 of the 50 (86%) have some type of emergency manager. This greatly improves the development, implementation, and refinement of AEPs and airports’ certification requirements. This also signals the importance the airport manage- ment community has placed on emergency management and allocated resources appropriately. When looking at the emergency exercises, 48% have some sort of emergency scenario designer and evaluator as part of their job description. This has likely increased in the preceding decade because of the rewrite of FAR Part 139 and the Presidential Homeland Security Directive (White House 2004). All of the small hub airports, 84% of the non-hub primary airports, 67% of the commercial service air- ports, 78% of the reliever airports, and 75% of the general aviation airports have full-time or part-time emergency managers.

16 TYPES OF EXERCISES USED More than 80% of the responding airports reported utilizing tabletop and full-scale emergency exercises (Question 39 in Appendix B). This would follow the requirements of FAR Part 139 exactly; however, there are many other types of exercises and scenarios that can be utilized in an effort for ongoing self-improvement in emergency preparedness for airports. It is encourag- ing to see how many airports have created positions on their management teams for emergency management personnel; however, it appears that the airports are staying with the mandated types of exercises. Some airports indicate other methods used such as seminars/workshops, games, and simulations. STAFF TRAINING FOR EXERCISE DEVELOPMENT The surveyed airports reported a number of sources for exercise planning and development of their TTY and FSE personnel (Questions 37 and 38). When asked if the airport has undergone any type of specific training in the past 3 years to assist with the development and deployment of training exer- cises, over half (60%) responded “no” while 36% responded “yes,” and 4% indicated they did not know. If GA and RL airports are removed, “yes” responses represent 44% and “no” 47%. Of those who responded positively, the most frequent source is the various iterations of training provided by FEMA, especially HSEEP (DHS 2013). This has worked well for airports that can afford to train their existing personnel to become subject matter experts (SMEs) for the development and staging of their exercises. Edwards and Goodrich (2014) estimated that developing a fully HSEEP-qualified exercise designer requires a 2- or 3-year commitment on the part of the individual and the organization, but other sources disagree (D. Kann, personal communication, Sept. 22, 2015). Training for exercise development was heavily skewed towards the larger airports, as shown in Table 4. The FAR Part 139 airports (large, medium, small-hub, non-hub and commercial) that are required to perform annual TTX and FSE every 3 years appear much likelier to commit staff time and funding to training in exercise development. The results in Table 4 show the importance of having readily useful tools to help small airports conduct effective exercises where there may not be sufficient time needed to “pre-plan” the exer- cise as opposed merely to running it. Many airports consider the emergency manager a stand-alone position; however, at smaller airports it may be part of another position, which would likely limit pre-planning time. This could also be a reason why airports seem to rely heavily on the mandated exercise formats instead of trying new and different types of training. One airport described an internal training consortium consisting of fire, police, maintenance, operations, security, training, emergency management, information technology, and engineering. All members of the group have been through HSEEP training and the complete NIMS and ICS training available from FEMA. NPIAS Category Have Staff with Exercise Development Training LH 69% MH 33% SH 43% NH 20% CS 0% RL 10% GA 14% Source: Smith, Garcia, Sawyer, and Kenville data. TABLE 4 STAFF WITH EXERCISE DEVELOPMENT TRAINING BY NPIAS CATEGORIES Thirty-six percent (36%) of airports reported training to assist with the development and deployment of exercises; 60% had had no such training.

17 Those who do not have trained exercise SMEs can attempt to directly access sample exercise materials from other airports or turn to local government emergency exercise SMEs, such as county or city emergency managers, for assistance. EXERCISE DEVELOPMENT FAA Advisory Circular 150/5300-31C provides clear guidance with regard to whom FAR Part 139 airports should involve in their plan- ning for emergency exercises. Although many smaller airports are not required by FAA regulations to have AEPs, best practices and requirements for NIMS compliance have resulted in the vast major- ity of airport having some form of an airport-specific plan, and most have looked to FAA for guidance in the development of their plans. In addition, as noted in chapter two, the Minnesota Council of Air- ports, in conjunction with the University of Minnesota Center for Transportation Studies, has created a manual for general aviation airports to utilize for airport emergencies. In addition, FAR Part 139 provides a good reference in this area even if the airport is not bound to it by regulation. FAR 139 states that at a minimum airports should coordinate with the following to ensure that they are trained in the airport’s plan: law enforcement; rescue and firefighting; medi- cal personnel and organizations; principal tenants; facilities personnel and agencies; all personnel having duties and responsibilities under the plan; and all other agencies having responsibilities under the plan. EXERCISE DEVELOPMENT TOOLS AND PROCESSES USED When analyzed, the responses to Question 43, summarized in Figure 5, show that a majority of air- ports in the study use internal subject matter experts, or SMEs, to develop their exercises. This may be difficult for smaller airports with limited human and capital resources, a constraint that supports the need for pre-prepared exercise materials. Rochester International Airport (RST) differentiates between exercise planners and exercise players, bring- ing up the important concept that the same persons who plan an exercise may distort its effectiveness if they are also players in that exercise (K. Claussen, personal communication, Dec. 4, 2015). Airports of all sizes benefit from focusing on a wide spectrum of scenarios chosen by considering the factors of likelihood, severity, and impact of all possible events. None of the above The airport's FAA compliance inspector suggests them Subject matter expert in operations develops them Subject matter expert in law enforcement develops them Subject matter expert in emergency management develops them Subject matter expert in communications/media/public relations Subject matter expert in ARFF develops them Senior management (C-level) sets scenario and goals for exercise Senior management (C-level) develops scenario and materials for Scenarios are specifically designed to test corrections of Scenarios and materials from previous exercises at your airport are Scenarios and materials are taken or adapted from professional Scenarios and materials are borrowed or adapted from other airports Scenarios and materials are adapted from previous actual incidents Ready-made exercises are procured from a vendor Non-airport department of the airport sponsor develops them Mutual aid partner develops them Consulting firm or vendor provides exercise guidance, scenarios, Consulting firm on long-term contract or retainer provides exercise A standing committee of airport managers and employees develops 0% 10% 20% 30% 40% 50% 60% FIGURE 5 Processes and responsibilities for exercise design.

18 Airports reported involving a wide variety of outside agencies and partners in exercise develop- ment, including regional planning councils, mutual aid partners, military units on joint use airports, airlines, hospitals, local offices of emergency services, and voluntary organizations. CONTACT LISTS One of the crossover areas between this study and ACRP Synthesis S04-16, Emergency Communica- tions Planning for Airports, is emergency contact lists. The survey obtained data on how the airports keep their contact lists updated (Question 30); whether the airports were satisfied with their method of updating contact lists (Question 31); and whether the airports’ contact lists were consistent across all plans and documents that contain contact lists, such as AEPs (Question 32). A crucial yet often overlooked part of the emergency planning process is emergency contact list preparation. Even with today’s technology, telephones are still most often utilized as the tried and true medium to reach people in times of emergency, as it is reli- able and recordable. Nearly 86% of the airports surveyed rely on manual updating of their emergency contact lists, while only 8% use any type of electronic or technologically enhanced system. It would be useful to determine whether a more systematic method of creating electronic databases exists, one which is tied to a fixed schedule of modification or “updating,” much like cellular phones on a nightly basis, might be available. Thirty-eight percent (38%) of the airports surveyed say they are on a fixed schedule, but one which may be semi annual at best; 42% were not sure if their lists were current and consistent at the time of the survey, and would likely welcome a solution to ensure their lists are accurate and up-to-date at all times. Whether by manual or automated process, it is essential that call lists be kept up to date, so that the appropriate parties can be reached in time of peril. Half the airports said that they were satisfied with the method they use to maintain their contact lists; 32% said they were dissatisfied (Question 31). This topic is examined in detail in ACRP Syn- thesis S04-16. Exercises can be a successful way for airports to test the accuracy of their emergency contact lists. Functional exercises (e.g., call downs) are the most common test used, but exercises may not occur frequently enough to ensure that contact lists are adequate if an actual incident occurs. Problems with contact lists may be revealed during the response to actual emergencies (Smith et al. 2015). Inconsistency in contact lists may create issues during exercises as well as real emergency responses. This was suggested by a comparison of the airports’ level of satisfaction with their con- tact lists to whether airports’ contact lists are consistent across AEPs and other plans and documents (Question 32), which revealed a similarity of the “yes” and “no” percentages (50/54% and 32/24%, respectively). WHEN TABLETOP EXERCISES ARE USED When asked on what occasions the airports use TTX (Question 50), responses reflected the impor- tance of FAR Part 139 requirements. (Because airports could mark more than one answer to this question, it is important to examine the 10% that report never having done a TTX; these airports were all GA airports.) Of the 35% that reported doing TTX more frequently than once a year, some are on a monthly schedule, some use TTX to test new plans or procedures, and some use them to evaluate learning. The most interesting comment suggested using tabletop exercises as part of a building-block approach to preparation for a triennial full-scale exercise. Examination of the data indicates that LH airports are likelier than other sized airports to do TTX frequently, which may correlate with the number of employees dedicated to emergency management at the airport. A critical element of emergency planning is contact list preparations. The vast majority of respondents manu- ally update their lists. Airports may wish to consider whether a more exact, perhaps automated, method of updating electronic databases could be implemented.

19 SCENARIOS Fifty-two percent (52%) of the surveyed airports said that they use an SME to assist in developing the scenario for their exercise (see Figure 5 and refer to Question 43 in Appendix A). Only one respondent conducted a weather-related exercise, and it was in conjunction with an aircraft acci- dent scenario. Another used an earthquake scenario, and three others conducted active shooter exercises. Some airports have also drilled using scenarios that reflected current events and inci- dents at other airports. However, a majority of the airports surveyed use training scenarios based on aircraft crashes (Alert III). The overwhelming focus on aircraft accident exercises results from the FAR 139 regulatory com- pliance requirements for aircraft rescue and firefighting (Figure 6) or staffing and resource con- straints, or both—despite the fact that many of the surveyed airports are in areas subject to major natural disasters such as earthquakes, hurricanes, tornadoes, and wildfires, which would affect not only the airport but the whole community and possibly the state. This could be attributed to the fact that many airports have found it beneficial to participate in regional disaster exercises organized by local or state governments, regional organizations, or federal agencies that generally focus on non-aircraft related scenarios (Smith 2014). Part 139 airports can perform non-aircraft incident cer- tification exercises if they get prior approval from FAA certification inspectors (S. Demory, personal communication, Oct. 3, 2015; C. Stephens, personal communication, Oct. 29, 2015). Even though this may be a contributing factor in the narrow range of airport exercises, it also points to a significant gap within airport exercise environment. Airports report most success by prioritizing exercise scenarios and target capabilities: Likelihood × Severity × Operational Impact = Exercise Priority. This is possibly a result of having to respond to regulatory mandates for AEP exercises or time or staff constraints. The data from Questions 51 and 52 show that only 28% of the surveyed airports have ever con- ducted a full-scale exercise for any purpose other than FAR Part 139 recertification. The majority of additional FSEs were for active shooter incidents. Table 5 is an index to the scenarios reported as having been used in the airports’ most recent full- scale exercise, which most likely would have been a triennial recertification exercise at a FAR Part 139 airport but also include voluntary full-scale exercises at some GA and RA airports in the study. The scenarios are indexed according to the primary capabilities and secondary capabilities addressed in the full-scale exercise and the airport that used each scenario. As indicated in Table 5, typical full-scale exercise scenarios are complex, often very complex. Full-scale exercises now usually include tests of mass casualty procedures and often address other FIGURE 6 ARFF Equipment at Hector International Airport (FAR photo, used by permission).

20 Primary Capability Secondary Capabilities Co m m u n ic at io n s A le rt & w ar n in g Em er ge nc y pu bl ic in fo rm at io n Pr o te ct iv e ac tio n s La w en fo rc em en t Fi re & re sc u e H ea lth & m ed ic al R es o u rc e m an ag em en t O pe ra tio n s & m ai n te n an ce Se cu rit y Sa fe ty U til iti es Cr o w d Co n tr o l O th er (C o de s at bo tto m o f ta bl e) A irp o rt Active shooter X X X X X X X X X X X CRW X X X X X X X X X X X X X 5 NYL A irc ra ft A cc id en t A irc ra ft Si z e Ca te go ry in Ex er ci se > 20 0 pa ss en ge rs X X X X X X X X X 4 SEA X X X X X X X X X X X 4 FLL X X X X X X X X X 4 ATL X X X X X X X X X 4, 6, 7 SLC X X X X X X X X X X 4, 7 PHX X X X X X X X X X 4, 14 LAX X X X X X X X X X X X X 4, 16 MCO X X X X X X X X X X X 4 DEN X X X X X X X 4, 3, 18, 20 SFO A irc ra ft Si z e Ca te go ry in Ex er ci se 10 0- 20 0 pa ss en ge rs X X X X X X X X X X X X X LAL X X X X X X X X X X X X X 1, 2, 4, 12, 23 MIA X X X X X X X X X X 5, 4, 7, 9, 13, 14, 20 RSW X X X X X X X X X X 4, 20 MSP X X X X X X X X X 4 RDU X X X X X X X X X 4 YIP X X X X X X X X X X X 4 BUR X X X X X X X X X X X X X 5, 4, 8 JAX X X X X X 4 STL X X X X X X X X X X 4, 11, 20 DCA A irc ra ft Si z e Ca te go ry in Ex er ci se 30 - 49 pa ss en ge rs X X X X X X X X X 5, 4 APA 5, 9, 23 EGV X X X X X X 4, 17, 20 DVL A irc ra ft A cc id en t A irc ra ft Si z e Ca te go ry in Ex er ci se > 20 0 pa ss en ge rs TABLE 5 CAPABILITIES TESTED IN MOST RECENT FULL-SCALE EXERCISES BY AIRPORTS IN STUDY

21 (continued on next page) Primary Capability Secondary Capabilities Co m m u n ic at io n s A le rt & w ar n in g Em er ge nc y pu bl ic in fo rm at io n Pr o te ct iv e ac tio n s La w en fo rc em en t Fi re & re sc u e H ea lth & m ed ic al R es o u rc e m an ag em en t O pe ra tio n s & m ai n te n an ce Se cu rit y Sa fe ty U til iti es Cr o w d Co n tr o l O th er (C o de s at bo tto m o f ta bl e) A irp o rt X X X X X X X X X X X X BFF A irc ra ft Si z e Ca te go ry in Ex er ci se R eg io n al Je t X X X X X X X X X X X X 4, 9, 14, 20 RNO X X X X X X X X X X X X FAR X X X X X X X X X X 3, 23 OWA X X X X 8 MEM X X X X X X X SAV X X X X X X X X X X X X X 4, 6, 10, 20, 24 HIB X X X X X X X X X X X BIS X X X X X X X X X X X X 5, 12, 13 LEX X X X X X X X X X JLN X X X X ASE X X X X X X X X X X X X X BOI X X X X X X X X X X X X X 5 EUG A irc ra ft Si z e Ca te go ry in Ex er ci se < 9 pa x X X X X X X 9, 20 FOD X X X X X X X X X X DVT X X X MMU X X X X X X X OPF GA X X X X X X X X 4, 14, 20 WVI W in te r w ea th er o pe ra tio n s X X X X X X X X X X 4, 14, 19, 20, 21, 22 DFW Other Secondary Capabilities 1. CBP/ICE screening of passengers for criminal 2. Transport of injured criminal to medical facility 3. Airline response TABLE 5 (continued)

22 functions. The detailed scenario used by Southwest Florida International Airport (RSW) gives an idea of the complexity that an airport can put into its scenario; it is reproduced in Appendix H. A number of airports reported using the DHS Master Scenario Events List (MSEL) Package, which is based on HSEEP. It is a template for organizing the injects for an exercise based on the overall exercises objectives and the jurisdiction’s overarching objectives for an exercise. The tem- plate has pre-formatted fill-in-the-blanks guides for generating a summary MSEL and an expanded MSEL. Orlando International Airport (MCO) was the only airport to note using MSEL in its survey replies, but most of the airports that noted use of HSEEP tools probably use MSELs. An MSEL from Range Regional Airport (HIB) is reproduced as Appendix L. DESIGN OF EXERCISES FAR 139.325 Airport Emergency Plan, section G, 1,2,3,4, states that each certificate holder must: 1. Coordinate the plan with law enforcement agencies, rescue and firefighting agencies, medical personnel and organizations, the principal tenants at the airport, and all other persons who have responsibility under the plan; 2. To the extent practicable provide for participation by all facilities, agencies, and personnel specified earlier in the development of the plan; 3. Ensure that all airport personnel having duties and responsibilities under the plan are familiar with their assignments and are properly trained; and 4. At least once every 12 consecutive calendar months, review the plan with all parties with whom the plan is coordinated, as specified [previously], to ensure that all parties know their responsibilities and that all of the information in the plan is current. Primary Capability Secondary Capabilities Co m m u n ic at io n s A le rt & w ar n in g Em er ge nc y pu bl ic in fo rm at io n Pr o te ct iv e ac tio n s La w en fo rc em en t Fi re & re sc u e H ea lth & m ed ic al R es o u rc e m an ag em en t O pe ra tio n s & m ai n te n an ce Se cu rit y Sa fe ty U til iti es Cr o w d Co n tr o l O th er (C o de s at bo tto m o f ta bl e) A irp o rt 4. Emergency Operations/Coordination Center 5. Haz-Mat 6. Crime scene handling 7. Family & Friends Reunification Center; Survivor, Friends and Relatives, and Family Assistance Centers 8. Terrorism 9. ARFF off airport 10. Radio communications 11. River rescue/water rescue 12. Fuel spill 13. Post-accident investigation (NTSB, FBI) 14. Multi Agency Coordination 15. Volunteer airlift team for regional relief (pilots, planes, ground crews) 16. Regional medical transport and treatment surge capacity test 17. Test AEP revisions 18. Airport senior management roles coordinating with airline 19. Aircraft accident 20. Mutual Aid 21. Social media messaging 22. Stranded passengers 23. Structural Fire 24. Terminal Evacuation Source: Smith, Garcia, Sawyer, and Kenville data. TABLE 5 (continued)

23 FAA Advisory Circular 150/5200-31C lists 10 functions at an airport that the AEP and exercises at FAR Part 139 airports must address: command and control; communications; alert notification and warning; emergency public information; protective actions; law enforcement/security; firefighting and rescue; health and medical; resource management; and airport operations and maintenance (FAA 2009, pp. 37–38). FAA guidance also states that these functions are not all-inclusive; thus, each airport needs to assess its own needs, adding functions as applicable by its own emergency planning team. To these issues, this study added four: security, safety, utilities, and crowd control (survey questions 41 and 42). The survey also allowed airports to list other functions that they tested For tabletop exercises (Question 41), the proportion of airports testing each of the 10 required functions ranged from a low of 50% (for protective actions) to a high of 90% (for communications). The average for the 10 functions was 72%. If the GA airports are excluded, the average rises to 82% and the range becomes 60%– 100%. It is nevertheless clear that airports are making efforts to use TTX to test preparedness, procedures, and training results for most of the 10 functions and for safety and crowd control. Overall, the frequencies for the 10 required functions ranged between 50% for protective actions and 90% for communications. Most values were between 75% and 90%. For the four additional functions for all airports in the study, the range for TTXs was 32% (utilities) to 74% (security, safety). With regard to which of the 10 FAR Part 139 requirements were addressed in most recent full-scale exercises (Question 42), the range was from 40% (protective actions) to 82% (command and control, fire and rescue). The average for all 10 required functions was 68%. Removing the 10 GA airports pro- duced a range of 50% to 100% and an average of 77%. For all airports in the study, the results for the four additional functions for full-scale exercises ranged from 20% (utilities) to 68% (safety, security). As can be seen in the results for Questions 41 and 42, a small number of airports reported testing other functions. The most common of these were mass care and family assistance centers. Comparison of the functions tested in full-scale exercises (Question 42) with those tested in tabletop exercises (Question 41) showed very similar results. The greatest apparent difference is that communications are tested more often in TTX (90%) than in FSEs (76%). Utilities are also tested less often in FSEs (30% compared with 20%). PARTICIPANTS IN EXERCISES To Question 41, “Who participated in your most recent triennial/recertification or full-scale exer- cise?”, the surveyed airports gave the responses shown in Figure 7. The highest values of around 80% resulted from the number of GA and reliever airports that answered “N/A” because they were not required to have full-scale exercises. As noted in Table 5, some airports used an active shooter scenario for their most recent FSEs. In addition to the survey findings reflected in Figure 7, several case example and other airports in the study noted the impor- tance of having airport volunteers—airport ambassadors, information booth volunteers, tenant associations, AOPA chapters, and airport community emergency response teams (A-CERT)—participate in exercises. Although the exercises addressed in this study are all related to FAA requirements or airport operational needs and are not primarily The main deficiencies in typical exercise designs are insufficient attention to resource management and on utilities. This is important because many emergency management situations involve eventual reimburse- ment from an insurance company, through a lawsuit, or from FEMA in the case of Presidential declarations of emergencies. Reimbursement requires proper record- keeping and tracking of resource allocation and utili- zation. Utilities are important, as the most common disruptions of airport operations are failure of electri- cal supply (Griffith et al. 2015) and delays in restoring electricity (Smith et al. 2015). TSA participated in nearly all FSEs at FAR Part 139 airports.

FIGURE 7 Participants in most recent full-scale exercise.

25 FIGURE 8 Full-scale exercise at Rochester (Minnesota) International Airport (Peggy Gray photo, used by permission). aviation security (AVSEC) exercises, the high degree of participation by security-related agencies is worth noting. TSA participated in almost 60% of the TTX but in nearly all the full-scale exercises at FAR Part 139 airports. The Federal Bureau of Investigation and U.S. Marshals Service also par- ticipated, typically at large hub airports. SETTINGS AND LOCATIONS The surveyed airports indicated the use of more than 20 different settings and locations in their most recent full-scale exercises (Question 47). By far the most frequently used exercise setting was the aircraft operating area (AOA), which is consistent with the strong focus on aircraft crash scenarios. Some airports used off-airport crash scenarios. Other on-airport settings noted by surveyed airports included ARFF training facilities, Air National Guard bases, FBOs, museums, and law enforcement defensive driving areas. Other off- airport settings included community sports complex, hotels, local state park, friends and family reception centers, hospitals, military command centers, and radio facilities around the city. PROPS AND EQUIPMENT USED IN EXERCISES The most commonly used props and equipment used by the airports were make-up (moulage), in- service vehicles, mannequins, and physical simulators (Figure 8; Question 48). Communications systems such as telephones, cell phones, radios, and the Internet were not included in the survey, as they were assumed essential regardless of scenario. OTHER TYPES OF EXERCISES In addition to tabletop and full-scale exercises, the survey airports reported doing drills (58% of surveyed airports), functional exercises (52%), workshops (30%), seminars (22%), and games/ simulations (8%) (Question 39). Ten percent (10%) reported doing no exercises of any type; these were all GA or reliever airports. Exercise types other than tabletop and functional exercises lie out- side the score of this study except for noting that they are useful building-blocks for preparing for a full-scale exercise. EXERCISE EVALUATION Evaluating and assessing training initiatives is critical for a continuous learning organization and is a highly sought-after attribute. It is encour- aging to see the number of airports that have contributed resources to Exercise evaluation is probably the most critical component of an exercise program.

26 hiring emergency management professionals or have allowed time for personnel to develop sub- ject matter experts among their staff. Equally encouraging are those airports that have utilized outside SMEs to design scenarios and enhance their training efforts. As important as investment in staff and training is introspective assessment. Probably one of the more difficult pieces of the continuous learning loop is feedback and its incorporation into established airport procedures; this is addressed later in this chapter. Exercise evaluation, based on an airport’s AEP, is probably the most critical component of an exercise program, since it identifies best practices to replicate and deficiencies to correct. HSEEP, the AirTap Emergency Guidebook for General Aviation Airports, and the Mineta Transportation Institute Exercise Handbook (Edwards and Goodrich 2014) contain valuable guidance on exercise evaluation; however, these resources are either focused on first response institutions and emergency management agencies for HSEEP and overland transportation systems for the later two. FAA Advi- sory Circular 150/5200-31C, Appendix 3, provides a series of checklists to evaluate full-scale AEP exercises, which are tailor-made for the aviation sector, but the checklists only address full-scale triennial exercises. NFPA 424-13 provides an Emergency Exercise Critique Form filled out for an imaginary international airport (NFPA 2013, Figure A.15.4.2). The NFPA form is more compact than the HSEEP or FAA A/C forms for evaluating full-scale exercises. ACRP Report 19: Developing an Airport Performance-Measurement System makes an emphatic case for performance metrics at airports: “Measurement captures the quantitative and qualitative prog- ress of the strategies, initiatives, products, and services that position the organization to achieve its goals and make definitive progress towards a defined vision” (Infrastructure Management Group et al. 2010, p. 8). In other words, measuring outcomes is necessary for an airport’s success and continuous improvement efforts. Unfortunately, ACRP Report 19 does not address emergency management or emergency exercises. ACRP Guidebook 19A: Resource Guide to Airport Performance Indicators does not deal with methods to assess emergency training and exercises (Hazel et al. 2011). There are several ARFF metrics, but those are based on ARFF costs per enplanement, operations, and response time adher- ence. In the comments section following the ARFF metrics it states, “At present [2011], there are few widely used ARFF APIs” (p. 43). It appears that despite this recent effort to create widely accepted metrics for emergency management training, such metrics remain unavailable and warrant further research. FEMA’s HSEEP program has a series of guides and templates that are intended to improve the quality of evaluation of exercises and the likelihood that lessons learned will be used to improve plans (FEMA 2015). The evaluation guides appear to be primarily based on the ESF (Essential Support Function) concept; therefore, they are not easily applicable by most airports. The guides do not contain airport-specific materials. Only a few of the surveyed airports reported using the After-Action Report/Improvement Plan (AAR-IP) Template (FEMA 2015a). The FEMA HSEEP data collection guide (FEMA 2015b) grades exercises on how well the exercise’s capability targets are performed. Four ratings are used: P—Performed without challenge S—Performed with some challenges M—Performed with major challenges U—Unable to be performed. The guide does not consolidate the ratings for a single score, but keeps the target capabilities and the score for each as discrete items in the after-action report and in the improvement plan. The most frequently used evaluation tools used for the surveyed airports’ full-scale exercises (Question 49) are after-action reviews and reports, hot washes, and checklists. Although these are beneficial, it is important that quantifiable assessments are taken so the continuous loop of self- improvement can continue.

27 DEFINING A SUCCESSFUL EXERCISE The review of available literature did not find specific metrics to determine the success of an exercise. However, the participants in the survey identi- fied, from a provided list of nine statements, those that they though would identify a successful exercise (Question 44). The top four gauges for exer- cise success were reported as: 1. Were the major target capabilities and exercise objectives in the exercise plan achieved? (74% of responses) 2. Were the strengths and weaknesses of the AEP identified? (74%) 3. Was the exercise completed safely? (72%) 4. Were FAA certification requirements satisfied? (70%) APPLYING THE LESSONS LEARNED FROM EXERCISES Lessons learned from exercises must be captured during the evaluation phase of the exercise and reported in a manner that allows for follow-up. Action items and appropriate assignments must be made and tracked in order to ensure improvement. Use of schedules and action tracking can ensure that the action items are completed in a reasonable amount of time. As with the need to issue and track work orders in airport mainte- nance and construction, there needs to be a way to track the process of improvement or revision. None of the airports in this study seemed to do exercises purely to satisfy certification or other regu- latory purposes. To understand how airports promote the application of lessons learned, airports were specifically asked if they had a formal process for implementing lessons learned from exercises into their written plans and procedures such as AEPs, SOPs, or communications plans (Question 53). Nearly half (48%) of the surveyed airports have a formal system, and an equal number do not. About one-fifth of the airports (22%) have a written process for capturing and applying lessons learned. Ten of the surveyed airports—all among the 22% that reported having a written process for applying lessons learned—reported the use of one or more of five basic tools: • After action reviews and reports (AAR) • Improvement plans (IP) • Explicit provisions in AEP specifying process and individual responsibilities • HSEEP AAR/IR Improvement Matrix (DHS 2013) • Active tracking of the implementation of lessons learned, either by a committee or by assigned individuals. The survey results show that these tools are sometimes used in com- bination; this is also the recommendation of HSEEP (DHS 2013). It is important because of the resources that have been expended on planning that airports continue the final process of assessment with metrics that can be implemented and compliance gained in the next exercise, otherwise the effort could be viewed as futile. EXERCISE MATERIALS DESIRED BY AIRPORTS When asked to rate the desirability of having specific types of exercise materials and tools to aid the planning, execution, and evaluation of future exercises (Question 55), the airports ranked 17 exercise aids. The top six exercise aids that the airports would like to have are exercise Use of schedules and action tracking can ensure that the action items to apply lessons learned are completed in a reasonable amount of time. Identifiable and trackable means of providing feed- back to an emergency manager to ensure deliberate updating of the emergency response based on data gathered from exercises will fulfill the need for con- tinuous feedback. If the lessons learned from an airport’s emergency exercises are not applied to future behavior and invest- ments, the airport is wasting a major opportunity for self-improvement. Between 66% and 74% of the surveyed air- ports reported using checklists, hot wash, or after action reports as their main tools for assessing the exercise.

28 planning checklists, exercise evaluation forms, training and exercise event checklists, exam- ples of AARs, examples of exercise communications plans, and summaries of lessons learned (Table 6). Exercise Aid Rating Rank out of5 Exercise planning checklists 4.34 Exercise evaluation forms 4.06 Training and exercise event checklist 4.02 Examples of after action review reports 3.96 Examples of exercise communications plans 3.88 Summaries of lessons learned 3.66 Examples of full-scale exercise scenarios 3.62 List of functions typically exercised 3.54 Examples of TTX scenarios 3.52 Examples of exercise safety plans 3.50 Exercise “success” criteria 3.46 Budget and cost information about exercises 3.18 List of stakeholders involved in exercise execution 3.10 Examples of value of the exercise statements 3.08 Timelines for developing exercises 3.08 Outside training options 3.02 List of stakeholders involved in exercise design 2.98 Source: Smith, Garcia, Sawyer and Kenville data. TABLE 6 DESIRED EXERCISE PLANNING, EXECUTION, AND EVALUATION AIDS

Next: Chapter Four - Case Examples »
Tabletop and Full-Scale Emergency Exercises for General Aviation, Non-Hub, and Small Hub Airports Get This Book
×
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

ACRP Synthesis 72: Tabletop and Full-Scale Emergency Exercises for General Aviation, Non-Hub, and Small Hub Airports provides small airports with the tools and practices needed to practice emergency response. The report provides sample exercise tools and plans, a checklist of effective practices for tabletop and full-scale emergency exercises, and a road map for developing an effective exercise program.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!