Transportation Emergency Response Application (TERA) Support Materials for Airport EOC Exercises (2018)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Evaluator Handbook with Exercise Evaluation Guides P A R T I V

EXERCISE SENSITIVE EXERCISE SENSITIVE TERA Airport Exercise Evaluator Handbook Developed for use with Transportation Emergency Response Application (TERA) Exercises on the TERA Support Portal (https://tera.train-emst.com/)

EXERCISE SENSITIVE Evaluator Handbook ii EXERCISE SENSITIVE The Evaluator Handbook describes the roles and responsibilities of exercise Evaluators and the procedures they should follow. Because the Evaluator Handbook contains information about the scenario and about exercise administration, it is distributed to only those individuals specifically designated as Evaluators. It should not be provided to exercise Players.

EXERCISE SENSITIVE Evaluator Handbook iii EXERCISE SENSITIVE Table of Contents General Information ..................................................................................................................... 1 Exercise Objectives ................................................................................................................ 1 Participant Roles and Responsibilities.................................................................................... 1 Exercise Assumptions and Artificialities................................................................................. 2 Exercise Implementation and Logistics ....................................................................................... 4 Exercise Play ............................................................................................................................ 4 Safety Requirements ............................................................................................................... 5 Exercise Start ........................................................................................................................... 6 Evaluator Information and Guidance .......................................................................................... 7 Evaluator Responsibilities ........................................................................................................ 7 Documenting the Exercise ..................................................................................................... 7 Exercise Tools .......................................................................................................................... 8 Evaluator Instructions .............................................................................................................. 8 Evaluation Basics ..................................................................................................................... 9 Assessment and Analysis of the Exercise .................................................................................. 12 Debriefings ............................................................................................................................. 12 Evaluation .............................................................................................................................. 12 Improvement Planning ......................................................................................................... 13 Appendix A. Typical Exercise Schedule .................................................................................. A-1 Appendix B. Scenario Overviews ............................................................................................ B-1 Appendix C. Exercise Evaluation Guides ............................................................................... C-1 Appendix D. Acronyms and Abbreviations ............................................................................ D-1 Appendix E. Evaluation Form .................................................................................................... E-1

EXERCISE SENSITIVE Evaluator Handbook 1 EXERCISE SENSITIVE GENERAL INFORMATION EXERCISE OBJECTIVES The following objectives describe the expected outcomes for the Players in the exercise. Apply National Incident Management System (NIMS) Incident Command System (ICS) response protocols in airport emergency situations Determine the correct degree of response, at the correct time, during an emergency Differentiate important from unimportant information during a crisis Determine short-term and long-term recovery and restoration needs for the airport State how incoming and outgoing information should flow during a crisis Define the roles and duties of the personnel around them Describe the actions that promote effective planning during an emergency PARTICIPANT ROLES AND RESPONSIBILITIES The term participant encompasses many groups of people, not just those playing in the exercise. Groups of participants involved in the exercise, and their respective roles and responsibilities, are as follows: Players have an active role in responding to scenario events and execute their regular roles and responsibilities during the exercise. Players initiate actions that will respond to and recover from the scenario. The Controller is responsible for providing oversight to the exercise by moderating flow and Player engagement. In Transportation Emergency Response Application (TERA) exercises, the Controller may also serve as a Facilitator and may also be the TERA Simulation Cell (SimCell) operator. The terms “Controller” and “Facilitator” may be used interchangeably in this document. The Facilitator keeps Player discussions on track with the objectives for any discussion components of the exercise. They ensure all issues and objectives are explored as thoroughly as possible within time constraints. Subject-matter experts may also assist with facilitation during the exercise. Evaluators observe and document observations, capture unresolved issues, and analyze exercise results. Evaluators do not interfere with exercise flow. Simulators are control staff personnel who role play nonparticipating organizations or individuals. They most often operate out of the SimCell, but they may occasionally have face-to-face contact with Players. Simulators enact

EXERCISE SENSITIVE Evaluator Handbook 2 EXERCISE SENSITIVE roles (e.g., media reporters, federal officials, next of kin) in accordance with instructions provided in the Master Scenario Events List. All Simulators are ultimately accountable to the Controller. In TERA exercises, the SimCell may only consist of the TERA SimCell operator, who is controlling the TERA interface. Observers do not directly participate in the exercise and should not interfere in exercise play. They observe the exercise as it unfolds while remaining separated from Player activities. The Exercise Director is responsible for managing all exercise staff and the overall exercise effort. This may be the Lead Controller or another individual. EXERCISE ASSUMPTIONS AND ARTIFICIALITIES In all exercises, assumptions and artificialities are necessary to complete play in the time allotted and/or account for logistical limitations. Exercise participants should accept that assumptions and artificialities are inherent in any exercise and should not allow these considerations to negatively impact their participation. Assumptions Assumptions constitute the implied factual foundation for the exercise and, as such, are assumed to be present before the exercise starts. The following assumptions apply to this exercise: This is a no-fault exercise. This exercise will be conducted in a low-stress learning environment. There is no single right response to scenario events, and open discussion is encouraged. Varying viewpoints, even disagreements, are expected. Decisions are not precedent setting. Decisions made during the exercise may not reflect your department’s final position on a given issue. This exercise is an opportunity to present multiple options and possible solutions. Issue identification is not as valuable as suggestions and recommended actions. While a number of issues may be identified during the exercise, a goal is to develop suggestions or recommend actions that could improve the airport’s response to the scenario. Problem-solving efforts should be the focus. Players should use their knowledge of current capabilities. The scenario is realistic and plausible. It contains sufficient detail to elicit response activities. Some scenario aspects have been sped up or slowed down. Some decisions have been omitted or assumed in order to enhance exercise discussions. There are no “hidden agendas,” so please don’t fight the scenario.

EXERCISE SENSITIVE Evaluator Handbook 3 EXERCISE SENSITIVE Artificialities During this exercise, the following artificialities apply: Exercise communication and coordination will be limited to the exercise venue. Simulators and Controllers will represent all entities that are not represented at the exercise venue (i.e., “the rest of the world”) and can be contacted during the exercise to report information and make requests appropriate for responding to the scenario. Aside from face-to-face communications, which should be recorded in summary by participating Players in TERA, communications will be conducted using the TERA platform. In order to achieve the exercise objectives, the exercise may not be conducted in real time.

EXERCISE SENSITIVE Evaluator Handbook 4 EXERCISE SENSITIVE EXERCISE IMPLEMENTATION AND LOGISTICS EXERCISE PLAY Evaluators will report to <Insert Exercise Venue> at <Insert Time> to begin exercise setup. Registration for the Players will begin at <Insert Registration Time>. Prior to the start of the exercise, there will be a Players’ briefing provided by the Exercise Director. Evaluators will follow the progress of the exercise, make sure it remains on track, and provide key data/information to Players to prompt or initiate certain Player actions. The Evaluators will be positioned throughout the exercise venue to observe and record scenario progress and exercise events, including Player actions, in accordance with plans, policies, and procedures. Individual and team Player performances will be observed in order to make recommendations for future system- wide training. In keeping with the “no-fault” nature of this exercise, Evaluator observations will focus primarily on overall actions, rather than on individual Players. Exercise Rules The following are the general rules that govern exercise play: The Exercise Director will initiate exercise play by transmitting the start of the exercise (STARTEX) message. The decision to conclude the exercise will be determined by the Exercise Director and the Lead Controller based upon the completion of operations and attainment of the exercise objectives. Real-world emergency actions take priority over exercise actions. Exercise participants will comply with real-world response procedures unless otherwise directed by control staff. Players initiating communication with the SimCell must identify the organization, agency, office, or individual with whom they wish to speak. “REAL-WORLD EMERGENCY” will be the designated phrase that indicates there is an emergency requiring immediate attention that may or may not stop exercise play. This is a no-fault exercise. There is no single “right” response to scenario events. The exercise is an opportunity to discuss and present multiple options and possible solutions. Players, Observers, and Evaluators will not engage in casual conversation. The Controllers/Facilitators or Evaluators will provide Players only the information they are specifically designated to disseminate. Players are expected to obtain other necessary information through TERA or existing emergency information channels.

EXERCISE SENSITIVE Evaluator Handbook 5 EXERCISE SENSITIVE SAFETY REQUIREMENTS General Exercise participant safety takes priority over exercise events. Although participants involved in the exercise come from various agencies, they share the same basic responsibility for ensuring a safe environment for all personnel involved in the exercise. In addition, many aspects of an emergency response are inherently dangerous. Professional health and safety ethics should guide all participants to operate in their assigned roles in the safest manner possible. The following general requirements apply to the exercise: All Evaluators will serve as safety observers for exercise activities. The Exercise Director should be notified immediately of any emergency situation. If, during the conduct of the exercise, a real-world emergency (e.g., medical, operational) occurs, the exercise may be suspended or terminated at the discretion of the Lead Controller/Facilitator. All participants will comply with their respective environmental, health, and safety plans and procedures, as well as the appropriate federal and state health and safety regulations. Participants will be responsible for their own safety and each other’s safety during the exercise. It is the responsibility of all persons associated with the exercise to notify a safety officer if, in their opinion, a real safety problem exists. Once the problem is corrected, exercise play can be resumed. Non-emergency Interruptions to Exercise Play Only the Facilitator can announce exercise “holds” or “stops.” Evaluators should communicate with the Facilitator if they believe a hold/stop is required. Medical Emergency In the event of a medical emergency, emergency medical services should be called for assistance and transport as required. The Lead Controller and Exercise Director should be notified immediately and informed that there is a “REAL-WORLD EMERGENCY.” This will be communicated to all exercise staff, and exercise play will be stopped. Exercise play will be resumed when the emergency situation has been resolved. Fire Emergency In the event of a fire, the fire department will be called as required. All participants will follow the emergency procedures for the exercise location. The Facilitator and Evaluators should be notified immediately and be informed that there is a “REAL- WORLD EMERGENCY.” Exercise play will be resumed when the emergency has been resolved.

EXERCISE SENSITIVE Evaluator Handbook 6 EXERCISE SENSITIVE For any other type of emergency, notify and follow the direction of the Exercise Director. EXERCISE START Exercise Start, Suspension, and Termination Instructions The exercise is scheduled to run for <time> or until the Exercise Director determines that the exercise objectives have been met. The Exercise Director will announce the start of the exercise, exercise suspension, and exercise termination. TERA Check Before the start of the exercise, the Controllers/Facilitators and Evaluators will conduct a test of TERA to help ensure the uninterrupted flow of control information. Player Briefing The Exercise Director will give all Players a Player Briefing that will summarize key exercise instructions and safety issues.

EXERCISE SENSITIVE Evaluator Handbook 7 EXERCISE SENSITIVE EVALUATOR INFORMATION AND GUIDANCE EVALUATOR RESPONSIBILITIES Exercise evaluation assesses an organization’s capabilities to accomplish a mission, function, or objective. Evaluation provides an opportunity to assess if exercise objectives were met. Evaluation is accomplished by the following means: Observing the event and collecting supporting data Analyzing collected data to identify strengths and areas for improvement Reporting exercise outcomes in the After-Action Report (AAR) Evaluators document Player performance by using the evaluation tools and observations made during the exercise. The evaluations, documentation, and debrief discussion(s) provide important information that substantiates exercise conduct and performance. The AAR and Improvement Plan (IP) will summarize the overall results of the exercise and provide a comprehensive assessment of capabilities that were demonstrated. The evaluation results will provide an opportunity to identify ways to build on strengths and improve capabilities. The exercise is expected to result in multiple findings and recommendations for improvement. DOCUMENTING THE EXERCISE Evaluators must keep accurate records and notes because these records will form the basis for evaluation of Player performance. Evaluation is valuable because it provides constructive feedback (positive and negative) to improve the effectiveness of the airport’s response to emergencies. Accurate and detailed documentation is critical to facilitate a full record of all the events in the exercise and to understand Player actions. Evaluators will document the exercise using the notes and exercise evaluation guides (EEGs). Evaluators should document key activities and issues as well as analysis of root causes of challenges or problems that occurred during the exercise. Evaluators should also record any improvement recommendations identified by Players during the hotwash and those of the Evaluator based on experience, familiarity with the airport’s plans, and subject-matter expertise. Evaluators should review their notes immediately after the exercise to ensure an accurate reconstruction of events and activities for discussion at the Exercise Debrief. Evaluation materials, including notes, become part of the exercise documentation.

EXERCISE SENSITIVE Evaluator Handbook 8 EXERCISE SENSITIVE EXERCISE TOOLS Evaluator Handbook The Evaluator Handbook is designed to help exercise Evaluators evaluate the exercise. This handbook also enables Evaluators to understand their roles and responsibilities in exercise execution and evaluation. Should a Player, Observer, or media representative find an unattended handbook, it should be given to the Exercise Director. TERA TERA is a simulation-based exercise training system that allows individuals or teams to make decisions in realistic, scenario-driven situations. TERA is completely web based. Each exercise scenario presents the unfolding incident through realistic injects including email, news, videos, website articles, and simulated radio and phone calls. Users make decisions and mitigate the incident by responding to messages, communicating with teammates, and giving commands to simulated entities. EVALUATOR INSTRUCTIONS Evaluator Training This training will generally be conducted on the day prior to the exercise and will assist in preparing Evaluators for performance of their duties. It will include a detailed review of exercise events and activities. This training is the time for Evaluators to ask questions and ensure that they completely understand their roles and responsibilities. Questions will be addressed and information clarified so that Evaluators feel confident that they can perform their assignments effectively. Before the Exercise Review the appropriate scenario (see Appendix B. Scenario Overviews and review scenario documents in TERA). Review the appropriate airport plans, policies, and procedures. Review the Evaluator Handbook, including the exercise objectives and supporting appendices. Attend the Evaluator Training. Report to the exercise venue no later than 1 hour prior to the exercise start time to begin exercise setup. During the Exercise Wear your designated exercise badge. Stay in proximity to Player decision makers. Avoid personal conversations with exercise Players unless there is a safety issue.

EXERCISE SENSITIVE Evaluator Handbook 9 EXERCISE SENSITIVE Record all significant events you observe. Evaluate the actions of Players, not the Players themselves. Observe and record exercise artificialities that interfere with exercise realism. If an exercise artificiality interferes with exercise play, report it to the Exercise Director. Do not prompt Players regarding what a specific response should be. Do not give information to Players about the scenario or other participants’ methods of problem resolution. Players are expected to obtain information through their own resources. All questions that you cannot answer with the exercise material or from the training should be directed to the Exercise Director. After the Exercise Attend the hotwash which will be conducted after the exercise. Use EEGs to confirm that objectives are met. Summarize your notes and prepare for the Exercise Debrief. Complete the Evaluator Evaluation Form (see Appendix E. Evaluation Form). EVALUATION BASICS Recording Important Events Although numerous events may occur simultaneously, Evaluators do not need to record all the action. Knowing which events are important helps Evaluators eliminate superfluous data and provide the kind of information that is most useful for evaluation. Important events that Evaluators should record include the following: Actions of Players in relation to scenario events Key decisions made by Players Deviations from current plans, policies, and procedures Challenges or problems identified that need to be addressed through modification of current plans and procedures, additional training and exercises, and/or additional equipment needed Effectiveness of communications What to Look For Individuals who prepare the AAR/IP will analyze the results provided by all Evaluators to achieve an integrated evaluation of exercised capabilities. Their analysis will focus on the timing of events, decisions made, and actions taken. To assist in that analysis, you should focus on the following areas: Communication among Players and organizations

EXERCISE SENSITIVE Evaluator Handbook 10 EXERCISE SENSITIVE Direction and coordination of activities Creative Player problem-solving, potentially beyond current plans and implementation procedures Plans or procedures that affect Player efforts Equipment and technology issues in relation to Player efforts Placement and Monitoring Evaluators should be located so they can observe Player actions and hear conversations without interfering with activities. In certain conditions, more than one Evaluator may be needed in a particular setting or area. Using Exercise Evaluation Guides Terminology The EEGs (see Appendix C. Exercise Evaluation Guides) are structured to capture information specifically related to the evaluation of the exercise objectives. Documenting Observations For each EEG, Evaluators provide a target rating, observation notes and explanation of the target rating, and a final objective rating. To efficiently complete these sections of the EEG, Evaluators should focus their observations on the exercise objectives. Observation notes should include if and how the objectives were met. Additionally, observations should include: How the objective was or was not met Pertinent decisions made and information gathered to make decisions Requests made and how requests were handled Resources utilized Plans, policies, procedures, or legislative authorities used or implemented Any other factors contributed to the results Evaluators should also note if an obvious cause or underlying reason resulted in Players not meeting an objective. Evaluators should also include recommendations on how to improve identified areas for improvement. Assigning Ratings Based on their observations, Evaluators assign a target rating for each objective. Evaluators then consider all target ratings for the objective and assign an overall rating. The rating scale includes four ratings: Performed without Challenge (P) Performed with Some Challenges (S)

EXERCISE SENSITIVE Evaluator Handbook 11 EXERCISE SENSITIVE Performed with Major Challenges (M) Unable to be Performed (U) Definitions for each of these ratings are included in the EEG.

EXERCISE SENSITIVE Evaluator Handbook 12 EXERCISE SENSITIVE ASSESSMENT AND ANALYSIS OF THE EXERCISE DEBRIEFINGS Post-exercise debriefings aim to collect sufficient relevant data to support effective evaluation and improvement planning. Hotwash At the conclusion of exercise play, the Exercise Director will facilitate a hotwash to allow Players to discuss strengths and areas for improvement. Evaluators can also seek clarification regarding Player actions and decision-making processes. The hotwash should not exceed 30 minutes. Exercise Debrief Evaluators attend the Exercise Debrief immediately following the hotwash. During this debrief, Evaluators provide an overview of their observations and discuss strengths and areas for improvement. Participant Feedback Forms Participant Feedback Forms provide Players with the opportunity to comment candidly on exercise activities and exercise design. Participant Feedback Forms should be provided to a Controller at the conclusion of the hotwash. EVALUATION Exercise Evaluation Guides EEGs assist Evaluators in collecting relevant exercise observations. EEGs document exercise objectives. Each EEG provides Evaluators with information on what they should expect to see demonstrated in their functional area. The EEGs, coupled with the Controller/Facilitator Evaluation Forms, Participant Feedback Forms, and hotwash notes, are used to evaluate the exercise and compile the AAR. After-Action Report The AAR summarizes key information related to evaluation. The AAR primarily focuses on the analysis of objectives, including capability performance, strengths, and areas for improvement. AARs also include basic exercise information, including the exercise name, type of exercise, dates, location, participating organizations, a brief scenario description, and the name of the exercise sponsor and point of contact.

EXERCISE SENSITIVE Evaluator Handbook 13 EXERCISE SENSITIVE IMPROVEMENT PLANNING Improvement planning is the process by which the observations recorded in the AAR are resolved through development of concrete corrective actions, which are prioritized and tracked as a part of a continuous corrective action program. After-Action Meeting The After-Action Meeting (AAM) is a meeting held among decision makers from the airport, as well as members of the exercise planning team, to review and refine the draft AAR/IP. The AAM should be an interactive session, providing attendees the opportunity to discuss and validate the observations and corrective actions in the draft AAR/IP. Improvement Plan The IP identifies specific corrective actions, assigns them to responsible parties, and establishes target dates for their completion.

EXERCISE SENSITIVE Evaluator Handbook A-1 EXERCISE SENSITIVE APPENDIX A. TYPICAL EXERCISE SCHEDULE Time Exercise Event 8:30 am – 8:45 am Registration 8:45 am – 9:15 am Welcome, Opening Remarks, and Exercise Overview 9:15 am – 9:45 am TERA Sign-In and Familiarization 9:45 am – 1:15 pm Exercise Conduct 1:15 pm – 1:55 pm Hotwash and Feedback 1:55 pm – 2:00 pm Closing Comments 2:00 pm – 3:00 pm Exercise Debrief 3:00 pm Adjourn

EXERCISE SENSITIVE Evaluator Handbook B-1 EXERCISE SENSITIVE APPENDIX B. SCENARIO OVERVIEWS POWER OUTAGE Scenario Summary A VIP event, high heat, a summer storm, and damage to an off-site sub-station contribute to a power outage at the airport. SEVERE STORM Scenario Summary A local event is disrupted by a severe storm which causes a hard landing of an aircraft. BOMB THREAT Scenario Summary A bomb threat becomes reality when an explosive device is detonated in the airport at a security checkpoint. HAZARDOUS MATERIALS INCIDENT Scenario Summary An unknown substance discovered at a gate makes passengers ill. TERMINAL FIRE Scenario Summary A fire does extensive damage to a terminal and endangers passengers. SABOTAGE Scenario Summary An intentional act causes a crash, damaging an aircraft and the terminal building.

EXERCISE SENSITIVE Evaluator Handbook B-2 EXERCISE SENSITIVE FUEL FARM FIRE Scenario Summary A repair accident causes a devastating fire at the fuel farm. WATER RESCUE Scenario Summary Bad weather causes an aircraft to crash into the water beyond the runway. (use as appropriate) MASS CASUALTY INCIDENT Scenario Summary A hard landing results in a mass casualty incident aboard a large aircraft.

EXERCISE SENSITIVE Evaluator Handbook C-1 EXERCISE SENSITIVE APPENDIX C. EXERCISE EVALUATION GUIDES

EXERCISE SENSITIVE Evaluator Handbook C-2 EXERCISE SENSITIVE Evaluator Name: Exercise Objective Apply NIMS/ICS response protocols in airport emergency situations Critical Task Observation Notes and Explanation of Rating Target Rating Activate, staff, and organize the EOC in accordance with airport emergency plans and standard operating procedures Direct efforts to meet incident objectives Implement policies and procedures to ensure the provision and tracking of all necessary resources Develop all necessary components of the Incident Action Plan (IAP) and obtain approval Final Core Capability Rating Ratings Key P – Performed without Challenges S – Performed with Some Challenges M – Performed with Major Challenges U – Unable to be Performed

EXERCISE SENSITIVE Evaluator Handbook C-3 EXERCISE SENSITIVE Ratings Definitions Performed without Challenges (P) The critical tasks associated were completed in a manner that achieved the objective and did not negatively impact the performance of other activities. Performance of this task was conducted in accordance with applicable plans, policies, procedures, regulations, and laws. Performed with Some Challenges (S) The critical tasks were completed in a manner that achieved the objective and did not negatively impact the performance of other activities. Performance of this task was conducted in accordance with applicable plans, policies, procedures, regulations, and laws. However, opportunities to enhance effectiveness and/or efficiency were identified. Performed with Major Challenges (M) The critical tasks were completed in a manner that achieved the objective, but some or all of the following were observed: Demonstrated performance had a negative impact on the performance of other activities. Task was not conducted in accordance with applicable plans, policies, procedures, regulations, and laws. Unable to be Performed (U) The critical tasks were not performed in a manner that achieved the objective.

EXERCISE SENSITIVE Evaluator Handbook C-4 EXERCISE SENSITIVE Evaluator Name: Exercise Objective Determine the correct degree of response, at the correct time, during an emergency Critical Task Observation Notes and Explanation of Rating Target Rating In response to an incident, implement management, planning, and coordination of airport response operations Conduct initial assessment (size-up) and determine initial incident site perimeter Establish and maintain communications with airport departments and responding units Plan, direct, and coordinate information and activities internally within the EOC and externally with other coordination entities Upon receiving information, assess and identify current and anticipated resource shortages, technical support issues, and key policy decisions needed across all capabilities, and provide to the applicable agency for resolution

EXERCISE SENSITIVE Evaluator Handbook C-5 EXERCISE SENSITIVE Critical Task Observation Notes and Explanation of Rating Target Rating Upon identification of issues, establish priorities between Incident and EOC; provide strategic direction; coordinate and resolve policy issues, including the issuance of protective action recommendations and protective action decisions Arrange for shelter, housing, and feeding for responders and airport personnel supporting the incident Arrange for shelter, housing, and feeding for displaced airport passengers Final Core Capability Rating Ratings Key P – Performed without Challenges S – Performed with Some Challenges M – Performed with Major Challenges U – Unable to be Performed

EXERCISE SENSITIVE Evaluator Handbook C-6 EXERCISE SENSITIVE Ratings Definitions Performed without Challenges (P) The critical tasks associated were completed in a manner that achieved the objective and did not negatively impact the performance of other activities. Performance of this task was conducted in accordance with applicable plans, policies, procedures, regulations, and laws. Performed with Some Challenges (S) The critical tasks were completed in a manner that achieved the objective and did not negatively impact the performance of other activities. Performance of this task was conducted in accordance with applicable plans, policies, procedures, regulations, and laws. However, opportunities to enhance effectiveness and/or efficiency were identified. Performed with Major Challenges (M) The critical tasks were completed in a manner that achieved the objective, but some or all of the following were observed: Demonstrated performance had a negative impact on the performance of other activities. Task was not conducted in accordance with applicable plans, policies, procedures, regulations, and laws. Unable to be Performed (U) The critical tasks were not performed in a manner that achieved the objective.

EXERCISE SENSITIVE Evaluator Handbook C-7 EXERCISE SENSITIVE Evaluator Name: Exercise Objective Differentiate important from unimportant information during a crisis Critical Task Observation Notes and Explanation of Rating Target Rating Identify and share information with all pertinent stakeholders across all disciplines through a clearly defined information sharing system Gather, organize, and document incident situation and resource information from all sources to maintain situational awareness Share information vertically (up and down) with appropriate airport agencies and departments in a timely and effective manner Share information across disciplines (among fire departments, EMS units, law enforcement, etc.) in a timely and efficient manner

EXERCISE SENSITIVE Evaluator Handbook C-8 EXERCISE SENSITIVE Critical Task Observation Notes and Explanation of Rating Target Rating Final Core Capability Rating Ratings Key P – Performed without Challenges S – Performed with Some Challenges M – Performed with Major Challenges U – Unable to be Performed

EXERCISE SENSITIVE Evaluator Handbook C-9 EXERCISE SENSITIVE Ratings Definitions Performed without Challenges (P) The critical tasks associated were completed in a manner that achieved the objective and did not negatively impact the performance of other activities. Performance of this task was conducted in accordance with applicable plans, policies, procedures, regulations, and laws. Performed with Some Challenges (S) The critical tasks were completed in a manner that achieved the objective and did not negatively impact the performance of other activities. Performance of this task was conducted in accordance with applicable plans, policies, procedures, regulations, and laws. However, opportunities to enhance effectiveness and/or efficiency were identified. Performed with Major Challenges (M) The critical tasks were completed in a manner that achieved the objective, but some or all of the following were observed: Demonstrated performance had a negative impact on the performance of other activities. Task was not conducted in accordance with applicable plans, policies, procedures, regulations, and laws. Unable to be Performed (U) The critical tasks were not performed in a manner that achieved the objective.

EXERCISE SENSITIVE Evaluator Handbook C-10 EXERCISE SENSITIVE Evaluator Name: Exercise Objective Determine short-term and long-term recovery and restoration needs for the airport Critical Task Observation Notes and Explanation of Rating Target Rating Upon completion of response phase, terminate EOC activities, archive records, and restore systems, supplies, and staffing to a pre-incident ready state Conduct safety inspections to support the safety of first responders and to assess the habitability of the airport Support recovery personnel as they work to develop scopes of work and costs for restoring airport infrastructure. Participate in the identification of mitigation opportunities that may be factored into repair, restoration, and recovery efforts Assess and prioritize recovery needs in order to prioritize recovery operations

EXERCISE SENSITIVE Evaluator Handbook C-11 EXERCISE SENSITIVE Critical Task Observation Notes and Explanation of Rating Target Rating Final Core Capability Rating Ratings Key P – Performed without Challenges S – Performed with Some Challenges M – Performed with Major Challenges U – Unable to be Performed

EXERCISE SENSITIVE Evaluator Handbook C-12 EXERCISE SENSITIVE Ratings Definitions Performed without Challenges (P) The critical tasks associated were completed in a manner that achieved the objective and did not negatively impact the performance of other activities. Performance of this task was conducted in accordance with applicable plans, policies, procedures, regulations, and laws. Performed with Some Challenges (S) The critical tasks were completed in a manner that achieved the objective and did not negatively impact the performance of other activities. Performance of this task was conducted in accordance with applicable plans, policies, procedures, regulations, and laws. However, opportunities to enhance effectiveness and/or efficiency were identified. Performed with Major Challenges (M) The critical tasks were completed in a manner that achieved the objective, but some or all of the following were observed: Demonstrated performance had a negative impact on the performance of other activities. Task was not conducted in accordance with applicable plans, policies, procedures, regulations, and laws. Unable to be Performed (U) The critical tasks were not performed in a manner that achieved the objective.

EXERCISE SENSITIVE Evaluator Handbook C-13 EXERCISE SENSITIVE Evaluator Name: Exercise Objective State how incoming and outgoing information should flow during a crisis Critical Task Observation Notes and Explanation of Rating Target Rating Gather, organize, and document incident situation and resource information from all sources to maintain situational awareness within the EOC Manage and coordinate public information, alert/warning, and notification activities Upon activation of the Joint Information Center, monitor media and conduct press briefings Issue public information, alerts, warnings, and notifications through established systems to the public, coordinating officials, and incident managers and responders

EXERCISE SENSITIVE Evaluator Handbook C-14 EXERCISE SENSITIVE Critical Task Observation Notes and Explanation of Rating Target Rating Final Core Capability Rating Ratings Key P – Performed without Challenges S – Performed with Some Challenges M – Performed with Major Challenges U – Unable to be Performed

EXERCISE SENSITIVE Evaluator Handbook C-15 EXERCISE SENSITIVE Ratings Definitions Performed without Challenges (P) The critical tasks associated were completed in a manner that achieved the objective and did not negatively impact the performance of other activities. Performance of this task was conducted in accordance with applicable plans, policies, procedures, regulations, and laws. Performed with Some Challenges (S) The critical tasks were completed in a manner that achieved the objective and did not negatively impact the performance of other activities. Performance of this task was conducted in accordance with applicable plans, policies, procedures, regulations, and laws. However, opportunities to enhance effectiveness and/or efficiency were identified. Performed with Major Challenges (M) The critical tasks were completed in a manner that achieved the objective, but some or all of the following were observed: Demonstrated performance had a negative impact on the performance of other activities. Task was not conducted in accordance with applicable plans, policies, procedures, regulations, and laws. Unable to be Performed (U) The critical tasks were not performed in a manner that achieved the objective.

EXERCISE SENSITIVE Evaluator Handbook C-16 EXERCISE SENSITIVE Evaluator Name: Exercise Objective Define the roles and duties of the personnel around them Critical Task Observation Notes and Explanation of Rating Target Rating Establish branches, groups, and divisions needed to manage the incident and meet incident objectives Final Core Capability Rating Ratings Key P – Performed without Challenges S – Performed with Some Challenges M – Performed with Major Challenges U – Unable to be Performed

EXERCISE SENSITIVE Evaluator Handbook C-17 EXERCISE SENSITIVE Ratings Definitions Performed without Challenges (P) The critical tasks associated were completed in a manner that achieved the objective and did not negatively impact the performance of other activities. Performance of this task was conducted in accordance with applicable plans, policies, procedures, regulations, and laws. Performed with Some Challenges (S) The critical tasks were completed in a manner that achieved the objective and did not negatively impact the performance of other activities. Performance of this task was conducted in accordance with applicable plans, policies, procedures, regulations, and laws. However, opportunities to enhance effectiveness and/or efficiency were identified. Performed with Major Challenges (M) The critical tasks were completed in a manner that achieved the objective, but some or all of the following were observed: Demonstrated performance had a negative impact on the performance of other activities. Task was not conducted in accordance with applicable plans, policies, procedures, regulations, and laws. Unable to be Performed (U) The critical tasks were not performed in a manner that achieved the objective.

EXERCISE SENSITIVE Evaluator Handbook C-18 EXERCISE SENSITIVE Evaluator Name: Exercise Objective Describe the actions that promote effective planning during an emergency Critical Task Observation Notes and Explanation of Rating Target Rating Establish incident objectives, priorities, and operational periods Develop the incident action plan (IAP) to establish priorities, procedures, and actions to be accomplished to meet the incident objectives Obtain EOC Manager approval of IAP Final Core Capability Rating Ratings Key P – Performed without Challenges S – Performed with Some Challenges M – Performed with Major Challenges U – Unable to be Performed

EXERCISE SENSITIVE Evaluator Handbook C-19 EXERCISE SENSITIVE Ratings Definitions Performed without Challenges (P) The critical tasks associated were completed in a manner that achieved the objective and did not negatively impact the performance of other activities. Performance of this task was conducted in accordance with applicable plans, policies, procedures, regulations, and laws. Performed with Some Challenges (S) The critical tasks were completed in a manner that achieved the objective and did not negatively impact the performance of other activities. Performance of this task was conducted in accordance with applicable plans, policies, procedures, regulations, and laws. However, opportunities to enhance effectiveness and/or efficiency were identified. Performed with Major Challenges (M) The critical tasks were completed in a manner that achieved the objective, but some or all of the following were observed: Demonstrated performance had a negative impact on the performance of other activities. Task was not conducted in accordance with applicable plans, policies, procedures, regulations, and laws. Unable to be Performed (U) The critical tasks were not performed in a manner that achieved the objective.

EXERCISE SENSITIVE Evaluator Handbook D-1 EXERCISE SENSITIVE APPENDIX D. ACRONYMS AND ABBREVIATIONS These acronyms and abbreviations may be found in exercise documents and materials and are commonly used by Players. AC Advisory Circular ACRP Airport Cooperative Research Program ADFAA Aviation Disaster Family Assistance Act AEP Airport Emergency Plan ANG Air National Guard ARFF Aircraft Rescue and Firefighting CAP Civil Air Patrol CBRNE Chemical, Biological, Radiological, Nuclear, and Explosive CDC Centers for Disease Control CISM Critical Incident Stress Management COOP Continuity of Operations CP Command Post (see also ICP) DHS Department of Homeland Security DOD Department of Defense DOT Department of Transportation EAS Emergency Alert System EMT/S Emergency Medical Technician/Services EOC Emergency Operations Center EOD Emergency Ordinance Disposal EOP Emergency Operations Plan EPA Environmental Protection Agency ESF Emergency Support Function FAA Federal Aviation Administration FBO Fixed Base Operator FD Fire Department

EXERCISE SENSITIVE Evaluator Handbook D-2 EXERCISE SENSITIVE FEMA Federal Emergency Management Agency IC Incident Commander ICP Incident Command Post ICS Incident Command System JIC/S Joint Information Center/System MOU Memorandum of Understanding NIMS National Incident Management System NOTAM Notice to Airmen NTSB National Transportation Safety Board OEM Office of Emergency Management OSHA Occupational Safety and Health Administration PD Police Department SOE State of Emergency SOG/P Standard Operating Guideline/Procedure TSA Transportation Security Administration

EXERCISE SENSITIVE Evaluator Handbook E-1 EXERCISE SENSITIVE APPENDIX E. EVALUATION FORM Evaluator Name: 1. Please rate your opinion of the exercise: 1 2 3 4 5 6 7 8 9 10 Very Poor Very Good 2. Compared to previous exercises, this exercise was: 1 2 3 4 5 6 7 8 9 10 Very Poor Very Good Please provide additional details in the space below. 3. Please rate your opinion on the effectiveness of the TERA system: 1 2 3 4 5 6 7 8 9 10 Very Poor Very Good Please provide additional details in the space below. 4. List the top 3 strengths of the exercise. (1) (2) (3)

EXERCISE SENSITIVE Evaluator Handbook E-2 EXERCISE SENSITIVE 5. List the top 3 issues and/or areas where the exercise could have been improved. (1) (2) (3) Identify the action steps that should be taken to address the issues identified above. For each action step, indicate if it is a high, medium, or low priority. (1) (2) (3) 6. What other changes would you make to improve this exercise? Please provide any other recommendations on how this exercise or future exercises could be improved or enhanced. EXERCISE DESIGN AND CONDUCT ASSESSMENT Please rate, on a scale of 1 to 5, your overall assessment of the exercise relative to the statements provided below, with 1 indicating strong disagreement with the statement and 5 indicating strong agreement. Assessment Factor Strongly Disagree Strongly Agree a. The exercise was well structured and organized. 1 2 3 4 5 N/A b. The exercise scenario was plausible and realistic. 1 2 3 4 5 N/A c. The exercise documentation was useful. 1 2 3 4 5 N/A CONTROLLER/EVALUATOR FEEDBACK Please provide any recommendations on how this exercise or future exercises could be improved or enhanced.

Abbreviations and acronyms used without definitions in TRB publications: A4A Airlines for America AAAE American Association of Airport Executives AASHO American Association of State Highway Officials AASHTO American Association of State Highway and Transportation Officials ACI–NA Airports Council International–North America ACRP Airport Cooperative Research Program ADA Americans with Disabilities Act APTA American Public Transportation Association ASCE American Society of Civil Engineers ASME American Society of Mechanical Engineers ASTM American Society for Testing and Materials ATA American Trucking Associations CTAA Community Transportation Association of America CTBSSP Commercial Truck and Bus Safety Synthesis Program DHS Department of Homeland Security DOE Department of Energy EPA Environmental Protection Agency FAA Federal Aviation Administration FAST Fixing America’s Surface Transportation Act (2015) FHWA Federal Highway Administration FMCSA Federal Motor Carrier Safety Administration FRA Federal Railroad Administration FTA Federal Transit Administration HMCRP Hazardous Materials Cooperative Research Program IEEE Institute of Electrical and Electronics Engineers ISTEA Intermodal Surface Transportation Efficiency Act of 1991 ITE Institute of Transportation Engineers MAP-21 Moving Ahead for Progress in the 21st Century Act (2012) NASA National Aeronautics and Space Administration NASAO National Association of State Aviation Officials NCFRP National Cooperative Freight Research Program NCHRP National Cooperative Highway Research Program NHTSA National Highway Traffic Safety Administration NTSB National Transportation Safety Board PHMSA Pipeline and Hazardous Materials Safety Administration RITA Research and Innovative Technology Administration SAE Society of Automotive Engineers SAFETEA-LU Safe, Accountable, Flexible, Efficient Transportation Equity Act: A Legacy for Users (2005) TCRP Transit Cooperative Research Program TDC Transit Development Corporation TEA-21 Transportation Equity Act for the 21st Century (1998) TRB Transportation Research Board TSA Transportation Security Administration U.S.DOT United States Department of Transportation

TRA N SPO RTATIO N RESEA RCH BO A RD 500 Fifth Street, N W W ashington, D C 20001 A D D RESS SERV ICE REQ U ESTED N O N -PR O FIT O R G . U .S. PO STA G E PA ID C O LU M B IA , M D PER M IT N O . 88 ISBN 978-0-309-39053-8 9 7 8 0 3 0 9 3 9 0 5 3 8 9 0 0 0 0