National Academies Press: OpenBook

Guidelines for Transportation Emergency Training Exercises (2006)

Chapter: 5.0 Operations-Based Exercise Planning Process

« Previous: 4.0 Discussion-Based Exercises
Page 41
Suggested Citation:"5.0 Operations-Based Exercise Planning Process." National Academies of Sciences, Engineering, and Medicine. 2006. Guidelines for Transportation Emergency Training Exercises. Washington, DC: The National Academies Press. doi: 10.17226/13924.
×
Page 41
Page 42
Suggested Citation:"5.0 Operations-Based Exercise Planning Process." National Academies of Sciences, Engineering, and Medicine. 2006. Guidelines for Transportation Emergency Training Exercises. Washington, DC: The National Academies Press. doi: 10.17226/13924.
×
Page 42
Page 43
Suggested Citation:"5.0 Operations-Based Exercise Planning Process." National Academies of Sciences, Engineering, and Medicine. 2006. Guidelines for Transportation Emergency Training Exercises. Washington, DC: The National Academies Press. doi: 10.17226/13924.
×
Page 43
Page 44
Suggested Citation:"5.0 Operations-Based Exercise Planning Process." National Academies of Sciences, Engineering, and Medicine. 2006. Guidelines for Transportation Emergency Training Exercises. Washington, DC: The National Academies Press. doi: 10.17226/13924.
×
Page 44
Page 45
Suggested Citation:"5.0 Operations-Based Exercise Planning Process." National Academies of Sciences, Engineering, and Medicine. 2006. Guidelines for Transportation Emergency Training Exercises. Washington, DC: The National Academies Press. doi: 10.17226/13924.
×
Page 45
Page 46
Suggested Citation:"5.0 Operations-Based Exercise Planning Process." National Academies of Sciences, Engineering, and Medicine. 2006. Guidelines for Transportation Emergency Training Exercises. Washington, DC: The National Academies Press. doi: 10.17226/13924.
×
Page 46
Page 47
Suggested Citation:"5.0 Operations-Based Exercise Planning Process." National Academies of Sciences, Engineering, and Medicine. 2006. Guidelines for Transportation Emergency Training Exercises. Washington, DC: The National Academies Press. doi: 10.17226/13924.
×
Page 47
Page 48
Suggested Citation:"5.0 Operations-Based Exercise Planning Process." National Academies of Sciences, Engineering, and Medicine. 2006. Guidelines for Transportation Emergency Training Exercises. Washington, DC: The National Academies Press. doi: 10.17226/13924.
×
Page 48
Page 49
Suggested Citation:"5.0 Operations-Based Exercise Planning Process." National Academies of Sciences, Engineering, and Medicine. 2006. Guidelines for Transportation Emergency Training Exercises. Washington, DC: The National Academies Press. doi: 10.17226/13924.
×
Page 49
Page 50
Suggested Citation:"5.0 Operations-Based Exercise Planning Process." National Academies of Sciences, Engineering, and Medicine. 2006. Guidelines for Transportation Emergency Training Exercises. Washington, DC: The National Academies Press. doi: 10.17226/13924.
×
Page 50
Page 51
Suggested Citation:"5.0 Operations-Based Exercise Planning Process." National Academies of Sciences, Engineering, and Medicine. 2006. Guidelines for Transportation Emergency Training Exercises. Washington, DC: The National Academies Press. doi: 10.17226/13924.
×
Page 51
Page 52
Suggested Citation:"5.0 Operations-Based Exercise Planning Process." National Academies of Sciences, Engineering, and Medicine. 2006. Guidelines for Transportation Emergency Training Exercises. Washington, DC: The National Academies Press. doi: 10.17226/13924.
×
Page 52
Page 53
Suggested Citation:"5.0 Operations-Based Exercise Planning Process." National Academies of Sciences, Engineering, and Medicine. 2006. Guidelines for Transportation Emergency Training Exercises. Washington, DC: The National Academies Press. doi: 10.17226/13924.
×
Page 53
Page 54
Suggested Citation:"5.0 Operations-Based Exercise Planning Process." National Academies of Sciences, Engineering, and Medicine. 2006. Guidelines for Transportation Emergency Training Exercises. Washington, DC: The National Academies Press. doi: 10.17226/13924.
×
Page 54
Page 55
Suggested Citation:"5.0 Operations-Based Exercise Planning Process." National Academies of Sciences, Engineering, and Medicine. 2006. Guidelines for Transportation Emergency Training Exercises. Washington, DC: The National Academies Press. doi: 10.17226/13924.
×
Page 55
Page 56
Suggested Citation:"5.0 Operations-Based Exercise Planning Process." National Academies of Sciences, Engineering, and Medicine. 2006. Guidelines for Transportation Emergency Training Exercises. Washington, DC: The National Academies Press. doi: 10.17226/13924.
×
Page 56
Page 57
Suggested Citation:"5.0 Operations-Based Exercise Planning Process." National Academies of Sciences, Engineering, and Medicine. 2006. Guidelines for Transportation Emergency Training Exercises. Washington, DC: The National Academies Press. doi: 10.17226/13924.
×
Page 57
Page 58
Suggested Citation:"5.0 Operations-Based Exercise Planning Process." National Academies of Sciences, Engineering, and Medicine. 2006. Guidelines for Transportation Emergency Training Exercises. Washington, DC: The National Academies Press. doi: 10.17226/13924.
×
Page 58
Page 59
Suggested Citation:"5.0 Operations-Based Exercise Planning Process." National Academies of Sciences, Engineering, and Medicine. 2006. Guidelines for Transportation Emergency Training Exercises. Washington, DC: The National Academies Press. doi: 10.17226/13924.
×
Page 59
Page 60
Suggested Citation:"5.0 Operations-Based Exercise Planning Process." National Academies of Sciences, Engineering, and Medicine. 2006. Guidelines for Transportation Emergency Training Exercises. Washington, DC: The National Academies Press. doi: 10.17226/13924.
×
Page 60
Page 61
Suggested Citation:"5.0 Operations-Based Exercise Planning Process." National Academies of Sciences, Engineering, and Medicine. 2006. Guidelines for Transportation Emergency Training Exercises. Washington, DC: The National Academies Press. doi: 10.17226/13924.
×
Page 61

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Guidelines for Transportation Emergency Training Exercises Depending on the number of participating agencies, exercise planning teams for operations- based exercises can require 10 to 20 people. 41 5.0 OPERATIONS-BASED EXERCISE PLANNING PROCESS This section provides a brief overview of the exercise planning process for operations-based exer- cise and includes a description of common exercise planning events, exercise documents, and time- lines. A generic exercise cycle is depicted in Figure 8. EXERCISE PLANNING TEAM The exercise planning team for operations-based exercises should include representatives from each participating agency or functional area as well as from all necessary logistical support areas. Because input is needed from all of these disciplines, the exercise planning team will be larger than the one needed for a discussion-based exercise. In the transportation environment, exercise planning teams for full-scale exercises can range from 10 to 20 people. For operations-based exercises, the size of the group required to develop and manage the exercise is directly proportional to the complexity of the event being designed. For example, for a full-scale exer- cise, the planning and development may be done by organizing core transportation personnel and the supporting working group members into functional committees, as depicted in Figure 9. Within this sample structure, the management or executive planning committee may ▪ Develop or manage the exercise master schedule, ▪ Develop the scope, ▪ Identify objectives for the organizational exercise, ▪ Identify limitations, ▪ Develop guidelines to direct the preparation of exercise materials, ▪ Approve simulations, ▪ Develop a public information plan, ▪ Ensure quality control, ▪ Invite VIPs and evaluators, ▪ Ensure compliance with grant programs, ▪ Approve exercise packages, and ▪ Approve exercise evaluation reports. The scenario committee may ▪ Develop the scenario narrative, ▪ Develop the timeline of key scenario events, and ▪ Develop the message injects. The exercise control committee may ▪ Identify the control organization, ▪ Staff and train the control organization,

Guidelines for Transportation Emergency Training Exercises 42 FIGURE 8 GENERIC PROCESS FOR OPERATIONS-BASED EXERCISES Corrective Action Plan EVALUATE Exercise Develop Concept Exercise Purpose and Objectives Initial Planning Conferences Concept and Objectives After- Action Report Data Collection and Analysis EXERCISE CONDUCT Coordinate Objectives. Identify Broad Scenario. Scope Level of Protection Scenario Review. ID Key Events Exercise Details Master Scenario Events List Implementers Master Scenario Events List Planning DEVELOP Draft Exercise Plan Mid-Term Planning Conference Exercise Plan (Final) Develop Control and Evaluation Tools Final Planning Conference Control Staff Instructions and Evaluation Plan (Final) Conduct Participant Orientation FIGURE 9 POTENTIAL ORGANIZATION STRUCTURE FOR FULL-SCALE EXERCISE ▪ Develop the safety plan, ▪ Develop the security plan, ▪ Develop control communication requirements, and ▪ Identify simulation and prop requirements.

Guidelines for Transportation Emergency Training Exercises The exercise evaluation committee may ▪ Identify the evaluation organization; ▪ Staff and train the evaluation organization; ▪ Develop evaluator tools, such as evaluator modules; ▪ Set standards for rating performance; and ▪ Develop the exercise evaluation report. The administration and logistics committee may ▪ Set and manage exercise format and software; ▪ Maintain exercise document control; ▪ Develop and implement the VIP/observer control plan; ▪ Collect, document, and report expenditures; ▪ Schedule meeting and training facilities; ▪ Procure communications and simulation props and equipment; ▪ Identify and reserve hotel rooms and transportation for participants when required; ▪ Develop and implement event scene set-up and postevent cleanup; and ▪ Procure meals and water as required. EXERCISE PLANNING TIMELINES Operations-based exercises require a much more detailed and organized planning process than discussion-based exercises in order to ensure a high-quality product. A timeline that identifies key plan- ning meeting dates, milestones, and critical tasks should be established by the exercise planning team. A sample timeline for a full-scale exercise appears in Table 8. EXERCISE PARTICIPANTS Operations-based exercises have the following types of participants: ▪ Players: Individuals who actively participate in an exercise role by performing their regular roles and functions in response to the situations presented. ▪ Controllers: Individuals who ensure that objectives are sufficiently met to permit a valid evaluation, that the level of activity is sufficient to keep players occupied and challenged, and that the progress of the exercise is in accordance with the scenario. Controllers provide key data to players and may prompt or initiate certain player actions to ensure exercise continuity. Controllers are the only non- players who will provide information or direction to the players. ▪ Chief Controller: The individual responsible for the preparation of the control staff instructions and for coordination and oversight of the exercise control group, lead controllers, and all indi- vidual controllers. Exercise planning timelines organize the activities necessary to complete the exercise. 43

▪ Lead Controller: The individual designated to coordinate controller activities at a specific exer- cise location. ▪ Evaluators: Unbiased technical and functional experts tasked to document responder performance and the adequacy of facilities and equipment against established crisis and consequence man- agement plans and exercise objectives. An evaluator is assigned to one or more exercise functions or locations to document and evaluate individual, team, and organizational performance based on exercise objectives and performance criteria. Evaluators provide both positive and negative feed- back on how player performance supports the objectives. Evaluation has the aim of improving future efforts through recommendations arising from the exercise; its aim is not to judge. ▪ Chief Evaluator: The individual responsible for preparation of the evaluation plan, management of the overall evaluation or observation process, oversight of the evaluation team, and compila- tion of evaluation data for incorporation in an evaluation report. ▪ Lead Evaluator: The individual charged with supervising a group of evaluators at a given exer- cise site. Guidelines for Transportation Emergency Training Exercises 44 TABLE 8 FULL-SCALE EXERCISE ACTIVITY Ful l -Scale Exercise Act iv i ty Time Before and Af ter Exercise (E) Day Deve lop Concept and Se lec t Proposed Date o f Fu l l -Sca le Exerc ise E–365 days Coord ina te Date o f In i t ia l P lann ing Conference E–365 days Prepare and Ma i l In i t i a l P lann ing Confe rence Read-Ahead E–350 days Prepare In i t i a l P lann ing Confe rence Br ie f E–340 days Conduc t In i t ia l P lann ing Conference E–330 days D is t r ibu te Concept and Ob jec t i ve Paper to P lann ing Team E–320 days D is t r ibu te Dra f t Exerc ise P lan to Par t i c ipan ts E–240 days Rev iew P lan and Mater ia l fo r M id -Term P lann ing Confe rence E–200 days Conduc t M id-Term P lann ing Conference E–180 days Prepare and Approve Mid-Term P lann ing Conference Minu tes E–160 days Rev iew Dra f t Master Scenar io Events L is t , Cont ro l and Eva lua t ion P lan , Exerc ise T ime l ine , and Suppor t Requ i rement Sta tus E–120 days D issemina te F ina l Exerc ise P lan to P lann ing Team E–90 days Make F ina l Prepara t ions fo r F ina l P lann ing Conference E–65 days Conduc t F ina l P lann ing Confe rence E–60 days Prepare and Approve F ina l P lann ing Confe rence Minu tes E–53 days F ina l i ze Master Scenar io Events L is t and Imp lementers E–45 days Pub l i sh F ina l Rev iew o f Cont ro l le r /Eva lua to r Handbook E–30 days Pub l i sh Cont ro l le r /Eva lua tor Handbook E–25 days F ina l i ze Pre-Exerc ise Br ie f ings E–7 days Conduc t Pre-Exerc ise On-S i te Ac t i v i t ies E–1 day Conduct Fu l l -Sca le Exerc ise E day Co l lec t and Ana lyze Data (Par t i c ipan t , Observer , Eva lua to r ) E+30 days Forward Dra f t A f te r Ac t ion Repor t fo r Par t i c ipan t Rev iew E+52 days Rece ive Par t i c ipan t Comments on A f te r Ac t ion Repor t E+90 days D is t r ibu te F ina l A f te r Ac t ion Repor t to Par t i c ipan ts E+120 days Deve lop Improvement P lan E+150 days Imp lement Improvement P lan As needed

Guidelines for Transportation Emergency Training Exercises ▪ Simulators: Control staff personnel who simulate nonparticipating organizations or key nonpartic- ipating individuals. Simulators may come in face-to-face contact with players or perform their func- tions from a simulation cell (SIMCELL). They also may function semi-independently (e.g., as the mayor, media reporters, next-of-kin, or perpetrators). ▪ Actors: The volunteers who pretend to be victims of the emergency event. For realism, they may wear makeup and “act” injured, unconscious, hysterical, or dead, whatever is called for at the scene. ▪ Safety Officer: The person whose primary responsibility is to analyze the entire exercise from a safety perspective in both planning and operational roles. EXERCISE PLANNING CONFERENCES Operations-based exercises are typically planned using a minimum of three conferences: ▪ Initial planning conference (IPC); ▪ Mid-term planning conference (MPC); and ▪ Final planning conference (FPC). Because a large amount of information is needed to organize an operations-based exercise, other con- ferences are also recommended, including a concept and objectives meeting and a master scenario events list conference (MSEL). A brief description is provided for each type of planning conferences in the typical chronological order in which they would be held. Concept and Objectives (C&O) Meeting: Used to identify the type, scope, objectives, and purpose of the exercise. This meeting is typically run by the lead exercise planner and attended by the transporta- tion agency and senior officials from participating agencies. This meeting formally begins the exercise planning process. Specifically, the exercise planning team and other attendees agree on the exercise scope, determine an exercise location, define the overall objectives, determine the major participants, and select a date for the IPC. Exercise concept development is usually based on the transportation agency’s stated purpose in con- ducting the exercise, prior experience, operations, and historical precedence. Exercise objectives are used to establish the scope, specify the functions to be demonstrated, identify the extent of organization/ personnel participation, and identify the breadth and depth of activities to be accomplished or simulated. Participants in the meeting submit views on the proposed exercise concept, scenario, proposed objec- tives, recommended levels of participation, draft exercise responsibilities, potential planning milestones, and (if applicable) recommended changes to the host agency’s scenario to ensure that submitting agency interests are adequately reflected. The exercise concept guides the preparation of exercise doc- uments developed in subsequent exercise planning meetings. A C&O paper is prepared for dissemination to the exercise planning community and senior representa- tives of participating departments, agencies, jurisdictions and organizations. The C&O provides a syn- opsis of the IPC results and agreements and addresses the following: ▪ Exercise dates, ▪ Exercise purpose and type, ▪ Overall concept, ▪ Major exercise objectives, Operations-based exercises can have three to six planning conferences, depending on their complexity and the number of participating agencies. 45

▪ Exercise duration, ▪ Assumptions and artificialities, and ▪ Expected participant list. Initial Planning Conference: As for discussion-based exercises, the IPC provides a forum to obtain the planning team’s input on exercise location, schedule, duration, and other details required to develop exercise documentation. Planning team members should be assigned responsibility for the tasks out- lined in the meeting. The foundation of the IPC is the exercise sponsor’s proposed concept and over- arching objectives, and the goal is to reach consensus on the exercise concept, objectives, scope, and broad scenario so that exercise design and development can proceed. This consensus will provide the basis for the exercise planning team to: ▪ Refine draft exercise objectives and the scenario, ▪ Identify exercise assumptions, ▪ Confirm exercise dates, ▪ Coordinate levels of participation in the exercise, ▪ Disseminate current and specific planning guidance to exercise planners, ▪ Inform planners to provide their portions of the draft exercise plan, ▪ Finalize the C&O paper, and ▪ Prepare a draft exercise plan. Mid-Term Planning Conference (MPC): The MPC presents an additional opportunity in the planning timeline to settle logistical and organizational issues that arise during planning, such as staffing con- cepts, scenario and timeline development, scheduling, logistics, administrative requirements, and reviewing draft documentation. A Master Scenario Events List (MSEL) Conference: MSEL conferences are conducted for exercises with significant simulated and scripted play, typically functional and full-scale exercises. The MSEL pro- vides the script (or list of events) that both guides the unfolding of the scenario and identifies the antic- ipated responses of the players. The MSEL offers a synopsis of key events and expected responses. During the exercise, it is used to generate activity in specific functional areas to drive demonstration of objectives. There are generally two MSEL conferences scheduled, and they can be held in conjunction with the MPC and FPC or as separate events. The first conference focuses on the development of the MSEL and ensures that exercise planners from participating departments, agencies, jurisdictions, and organizations have identified activities that must occur during the exercise to enable achievement of exercise objectives. Exercise planners also define “injects” to stimulate players to perform tasks that address exercise objectives and coordinate event times in keeping with proposed scenario and expected responses. The result of the first conference is a chronological listing of exercise events and publication of the key event list. The ability to ensure that events occur, and to ensure that controllers are able to manage exercise flow, requires that certain information be injected into the exercise; this is accomplished through MSEL imple- menters. An implementer is the vehicle that places an MSEL item into exercise play. The second con- Guidelines for Transportation Emergency Training Exercises 46

Guidelines for Transportation Emergency Training Exercises ference typically is conducted to ensure that MSEL implementers developed by the exercise planners do the following (after which time the final MSEL is published): ▪ Reflect the intent of the event as described in the MSEL items, ▪ Accomplish intended results, ▪ Include the correct action and information addressees, and ▪ Provide a time sequence consistent with the flow of other exercise events. Final Planning Conference: A forum to review the processes and procedures for conducting the exer- cise, final drafts of all exercise materials, and all logistical requirements. There should be no major changes made either to the design or scope of the exercise or to any supporting documentation. Sample agendas and other materials for supporting the planning conferences for operations-based exercises are available on the G&T Secure Portal. HSEEP Volume IV describes these materials and is available at http://www.ojp.usdoj.gov/odp/docs/HSEEPv4.pdf. A sample IPC package is available from the United States Marine Corps at http://www.certip.org/ policies/6-14meeting.html. The Alaska Department of Homeland Security has prepared an extensive guide for the development of both discussion-based and operations-based exercises. This guide provides useful tips and recom- mendations for organizing and conducting planning conferences. The guide supports compliance with the G&T HSEEP and is available at http://www.ak-prepared.com/homelandsecurity/exercise/full-scale/ documents/ODP%20DesksideVol%20I.doc. DESIGN AND DEVELOPMENT Building on the exercise foundation, the design and development process should focus on identify- ing objectives, designing the scenario, creating documentation, developing policies, planning exer- cise conduct, and selecting an evaluation methodology. OBJECTIVES Exercise objectives are the foundation of design and development. Exercise objectives define specific goals, provide a framework for the development of the scenario, guide development of individual orga- nizational objectives, and provide evaluation focus for the exercise. Generally, the number of exercise objectives will be limited by planners to ▪ Enable timely execution of the exercise, ▪ Facilitate design of a reasonable scenario, and ▪ Adequately support the successful completion of exercise goals. Objectives are initially prepared during concept development. More complex exercises typically have both major and supporting objectives. A major objective contributes to development of general events to present to participants for action. Supporting objectives help planners to ensure that all participants receive an adequate opportunity to assess specific capabilities. The performance addressed by the objective should have observable and measurable indicators to aid in identifying evaluation criteria. FEMA’s SMART System, shown as Figure 10, is a good checklist for ensuring the completeness and accuracy of objectives. Objectives are perhaps the most important element of the exercise planning process. Subsequent evaluations and recommended improvements will be based on them. 47

SCENARIO A scenario provides the backdrop and storyline that drive an exercise. The first step in designing the scenario is determining the type of threat/hazard (e.g., chemical, biological, radiological, nuclear, explo- sive, cyber, or other). Thought should be given to creating a scenario that involves local incidents and local facilities and is based on exercise objectives derived from risk and vulnerability assessments con- ducted at the transportation agency. Each type of hazard presents its own strengths and weaknesses for evaluating different aspects of prevention, response, and recovery and is applicable to different exer- cise objectives. The next step is to determine the venue (i.e., the facility or site) that the scenario will affect. Venue selection should be based on the type of hazard used. For example, if a nonpersistent chemical agent (e.g., sarin) is selected, the venue should not be an open-air facility (e.g., outdoor station) because of the agent’s dissipating characteristics. Table 9 provides information on the characteristics of a good scenario. The Wisconsin Department of Emergency Management has released a set of 39 scenarios for use by local responders. This document provides an overview to Wisconsin’s recommended program for Guidelines for Transportation Emergency Training Exercises Many examples of scenarios are available. Transportation agencies should check with their local public safety and emergency management agencies for templates and examples tailored to their local communities. 48 FIGURE 10 SMART SYSTEM CHECKLIST Simple A good objective is simply and clearly phrased. It is brief and easy to understand. Measurable The objective sets the level of performance so that results are observable and you can tell when an objective has been reached. Achievable The objective is not too tough to achieve. It is reasonable in its commitment of resources. Realistic An objective is not only achievable, but also realistic for the exercise. Task Oriented The objective focuses on a behavior or a procedure. With respect to exercise design, each objective should focus on an individual emergency function. TABLE 9 WHAT DOES A GOOD SCENARIO LOOK LIKE? Character ist ic Descr ipt ion Appropr ia te The scenar io must f i t the Exercise objectives (for both the transportation organization and participating external agencies) Targeted incident location, in terms of geography and logical functioning of the location during transportation operations Emergency response organization's makeup and capabilities Rea l i s t i c The scenar io must be p laus ib le in te rms o f Design-basis coherence Threat and vulnerability assessment performed by the transportation agency Credible conditions and environmental challenges Robust The scenar io must be broad enough to Support all the envisioned exercise activities Provide the opportunity for all organizations to meet their objectives Engag ing The scenar io must Move participants to act with a high degree of involvement           Be challenging and raise responders' adrenaline level

Guidelines for Transportation Emergency Training Exercises conducting tabletops. It is available at http://emergencymanagement.wi.gov/docview.asp?docid= 738&locid=%2018. The Environmental Protection Agency has also made available a set of scenarios simulating radiological emergencies involving nuclear power plants, Department of Energy weapons and waste storage facili- ties, and military sites. These materials can be found at http://www.epa.gov/radiation/rert/exercises.htm. Michigan State University also offers a resource page to support the development of terrorism-based scenarios at http://www.cj.msu.edu/∼outreach/wmd/moduleresources.htm. Sample scenarios are also available to support operations-based exercises on the G&T Secure Portal. HSEEP Volume IV provides a description of these materials and is available at http://www.ojp.usdoj.gov/ odp/docs/HSEEPv4.pdf. DOCUMENTATION The list below briefly describes typical products for operations-based exercises. Documentation mate- rials such as meeting minutes, presentations, agendas, and news releases have been omitted because these documents typically are created while developing the primary products. The exercise plan (EXPLAN), typically used for operations-based exercises, provides an exercise synopsis and is published and distributed prior to the start of the exercise. In addition to addressing exercise objectives and scope, the EXPLAN assigns tasks and responsibilities for successful exer- cise execution. The EXPLAN should not contain detailed scenario information, such as the hazard to be employed. This document is generally intended for exercise players and observers. A sample exercise plan template is available from FEMA at http://training.fema.gov/emiweb/downloads/ EXERCISE%20%20PLAN1.doc. Sample exercise plans to support a variety of operations-based exercises are available on the G&T Secure Portal. HSEEP Volume IV provides a description of these materials and is available at http:// www.ojp.usdoj.gov/odp/docs/HSEEPv4.pdf. The controller and evaluator (C/E) handbook supplements the EXPLAN, containing more detailed information about the exercise scenario and describing exercise controllers’ and evaluators’ roles and responsibilities. Because the C/E handbook contains information on the scenario and exercise admin- istration, it should be distributed only to individuals specifically designated as controllers or evaluators. The C/E handbook is generally used on smaller or limited-scope exercises of short duration in lieu of control staff instructions (COSIN) and an EVALPLAN. Its specifics may include the following: ▪ Roles and responsibilities of functional or individual controllers; ▪ Evaluation aids and checklists; ▪ A schedule for training, site set-up, exercise conduct, and critiques; ▪ An exercise safety plan; and ▪ A controller communications plan. Larger, more complex exercises may use COSIN and an EVALPLAN in place of, or in addition to, the C/E handbook. A sample C/E handbook to support a variety of operations-based exercises is available on the G&T Secure Portal. HSEEP Volume IV provides a description of these materials and is available at http://www.ojp.usdoj.gov/odp/docs/HSEEPv4.pdf. The exercise plan resembles the situation manual in how it is applied during an operations-based exercise. 49

Control staff instructions (COSIN) contain the guidance that exercise controllers, simulators, and evaluators need concerning procedures and responsibilities for exercise control, simulation, and sup- port. The purpose of COSIN is to detail the scenario for the duration of the exercise, develop guide- lines for control and simulation support of the exercise, explain the exercise concept as it relates to controllers and simulators; and establish and define the control structures, communications, logistics, and administration. Its level of detail will vary and can include the following: ▪ Exercise overview, ▪ Exercise control organization and scheme, ▪ Controller roles and responsibilities, ▪ Control communications plan, ▪ VIP/observer management plan, ▪ List of key exercise events, ▪ Short MSEL and long MSEL, and ▪ Exercise safety plan. A template for a COSIN plan is available from FEMA at http://training.fema.gov/emiweb/downloads/ CONPLAN1.DOC. Sample COSIN to support a variety of operations-based exercises are available on the G&T Secure Portal. HSEEP Volume IV provides a description of these materials and is available at http://www. ojp.usdoj.gov/odp/docs/HSEEPv4.pdf. The master scenario events list (MSEL) is a chronological timeline of expected actions and scripted events to be injected into exercise play by controllers to generate or prompt player activity. Preparing the MSEL is probably the most complicated activity required for operations-based exercises. However, this activity enables the exercise planning team to identify expected actions and to document when and how the actions should be performed, based on existing plans, procedures, and training. Use of an MSEL ensures that necessary events happen so that all objectives are met. An MSEL contains a chrono- logical list of the events that drive exercise play. The MSEL links simulation to action, enhances the exer- cise experience for players, and reflects an incident or activity that will prompt players to implement the policy or procedure being tested. A sample MSEL format is presented in Table 10. In accordance with the recommendations in Table 7, each MSEL record identifies the ▪ Designated scenario time; ▪ Event synopsis; ▪ Controller responsible for delivering inject, with C/E special instructions (if applicable); ▪ Expected action (i.e., the player response expected after an MSEL inject is delivered); ▪ Intended player (i.e., the agency or individual player for whom the MSEL inject is intended); ▪ Objective to be demonstrated (if applicable); and ▪ Notes (for controllers and evaluators to track actual events against those listed in the MSEL, with special instructions for individual controllers and evaluators). Guidelines for Transportation Emergency Training Exercises Controllers and evaluators need handbooks and instructions to direct their activities during an operations-based exercise. 50

Guidelines for Transportation Emergency Training Exercises Times listed in an MSEL should reflect the time at which an inject should occur. These times should be as realistic as possible and should be based on input from functional area representatives. For exam- ple, to determine when triage and treatment should be established during the exercise, solicit input from emergency medical services (EMS) or a hospital representative. If the activity occurs sooner than antic- ipated, the time should be noted but play should not be interrupted. There are three types of injects: ▪ Contextual injects are introduced to a player by a controller to help build the contemporary oper- ating environment. For example, if the exercise objectives include information sharing, an MSEL inject can be developed to direct a controller to select an actor to portray a suspect. The inject could then instruct the controller to prompt another actor to approach a law enforcement officer and inform him or her that this person was behaving suspiciously. ▪ Expected action events are expected actions that would normally take place during this type of incident. For example, during an FSE involving a chemical agent, establishment of decon- tamination is an expected action. ▪ Contingency injects are events that should be verbally indicated to a player by a controller if the player does not discover them. For example, if a simulated secondary device is placed at an incident scene but is not discovered, a controller may want to prompt an actor to approach a player and say that he or she witnessed suspicious activity close to the device location. This should prompt the discovery of the device by the player and result in subsequent notification of law enforcement (specifically, the bomb squad). MSELs are typically produced in two formats: short and long. Short MSELs list the inject, the time, a short description, the responsible controller, and a player. These MSELs can be used as a quick 51 Time ( in minutes) Message No. Message Summary Expected Response -90 1 From senior controller to transportation supervisor: Permission to open exercise window. Exercise window is opened. -60 2 From senior controller to lead safety and security controllers: Implement security and safety plans. Event scene is isolated, and safety and security controllers begin to make final check of simulations. -10 3 From senior controller to all evaluators and controllers: Communication check and time check. Equipment is checked and time is noted. 00 4 5 6 From senior controller to event scene safety controller: Start smoke generator. From senior controller to event scene lead controller: Sound fire alarm. From senior controller to employee actor #1: Make 911 call "This is an exercise. We have just had an explosion at the store’s loading dock. The fire alarm is sounding. I see at least three injured persons. This is an exercise." Exercise start: • Fire department receives fire alarm. • Fire department receives 911 call. • Transportation control center and medical department monitor 911 call. TABLE 10 MSEL SAMPLE FORMAT

reference guide during exercise play. Long MSELs are used when greater detail is necessary. They include more detailed descriptions, exact quotes for injects by simulation cells (i.e., by groups acting out an element of the exercise, or SIMCELL), and descriptions of expected actions. Message injects are typically used in exercises that involve multiple simulated activities. These mes- sages are typically delivered via a SIMCELL and are used to simulate the actions, activities, and con- versations of an individual, agency, or organization that is not participating in the exercise but that would likely be actively involved during a real event. For example, in an exercise with limited scope, the state governor’s office may not be playing. To simulate the activities of the governor’s office during an emer- gency event, a message can be scripted to simulate notification of the mayor by the governor. That mes- sage can be delivered by phone through the SIMCELL. This script or message inject should be read by a simulator acting on behalf of the governor’s office. The Department of Energy’s Oak Ridge National Laboratory offers an extensive online training pro- gram for developing scenarios and MSELs. These materials, which include interactive forms and tables, can be accessed at http://www.orau.gov/emi/wbt/default.htm. As with the G&T Secure Portal, users must register to access these resources. Additional information on this registration process can be obtained by calling Oak Ridge at (865) 576-2007. Sample MSELs, supporting scenarios, and evaluation measures are also available on the G&T Secure Portal. HSEEP Volume IV provides a description of these materials and is available at http://www.ojp.usdoj.gov/odp/docs/HSEEPv4.pdf. Evaluation plans (EVALPLANs) provide evaluation staff with guidance and instructions on evalua- tion or observation methodology to be used as well as essential materials required to execute their specific functions. The EVALPLAN is a limited distribution document that evaluators use in conjunc- tion with the EXPLAN and the MSEL. Level of detail varies and can include the following: ▪ Exercise overview, ▪ Evaluation control organization, ▪ Evaluation methodology and observation techniques, ▪ Evaluator roles and responsibilities, and ▪ Evaluation communications plan. FEMA provides guidelines for the selection of evaluation methodologies at http://training.fema.gov/ emiweb/downloads/HMEEM%20R-VI%20UPDATE%20JULY%202000.DOC. FEMA provides a sample EVELPLAN that takes the user through all steps of the process. It is available at http://training.fema.gov/emiweb/downloads/evalplan.doc. ▪ Sample evaluation plans to support a variety of operations-based exercises are available on the G&T Secure Portal. HSEEP Volume IV provides a description of these materials and is available at http://www.ojp.usdoj.gov/odp/docs/HSEEPv4.pdf. POLICIES Exercise policies are developed to provide guidance or parameters of acceptable practices for designing, developing, conducting, and evaluating exercises. Policies are designed to prevent, or at least mitigate the impact of, an action that may cause bodily harm to participants, destruction of prop- erty, or embarrassment to the participants or affected community. State and local transportation agen- Guidelines for Transportation Emergency Training Exercises 52

Guidelines for Transportation Emergency Training Exercises cies should develop policies appropriate to the type of exercise that address safety, media, cancel- lation, and weather. A final activity that must occur during the development cycle is preparation of the protocols used to govern the exercise. Table 11 provides a sample list of protocols for responders, controllers, evalu- ators, and observers/VIPs. Conducting operations-based exercises is like producing theater. To create the final product of a stage play, there is both a script for actors and a supporting set of activities for the stage crew. In this analogy, existing plans, policies, and procedures (for the players) and the MSEL (for the controllers) serve as the script, and the message injects are like the supporting activities performed by the stage crew. 53 TABLE 11 EXERCISE PROTOCOLS Role Rules Responders • Move participants to act with a high degree of involvement. • Monitor your actions. Ensure you keep yourself safe. • Follow all instructions from controllers unless the instructions place you in danger. • Verbalize your thought processes. The evaluators are not mind readers, and you want credit for what you have accomplished. • Speak loudly so evaluators can hear what you are saying. • If you have questions on what you are observing or information that was provided, ask the controller. Evaluators are directed not to speak to you. • Always state "This is an exercise" when making radio and telephone calls. • If nonresponders get in your way, tell the controller. • Participate in the postexercise critique. • Ensure that you sign in so that you get credit for your participation. Controllers • Ensure exercise safety—this is your primary duty. • Do not give clues to responders. For example, "Who did you notify?" is a better question than "Did you notify the state?" The latter question tells the responder that the state has to be notified. • Be responsible for suspending, restarting, and terminating. Follow the instructions from the senior controller. • Inform the evaluator when you issue a contingency message. • Ensure that all responders sign in to get credit for their participation in the exercise. • Facilitate the responder critique immediately after the exercise. You may explain why things happened or the responder expectations, but do not comment on performance. Evaluators • Assist the controllers with monitoring safety. • Position yourself to observe and hear the responders’ performance, but do not get in the responders’ way. • If you have a question for responders, go through the controller. • Keep your timeline of observed performance during the exercise; evaluation of performance happens after the exercise. • Evaluate performance against the responder's plans and procedures, not what you believe the plans and procedures should be. • Attend the postexercise critique. Take notes regarding what is said, and use the debrief to understand the rationale for why specific actions were performed. • Do not comment on performance during the critique. Observers and VIPs • Do not ask questions of responders, evaluators, or controllers. All questions will be addressed through your escort. • Stay out of the way of responders. • Remain quiet. Do not critique or prompt responder performance. • Keep a professional bearing. Remember that the first people to be kicked out due to space restrictions are the observers and VIPs. • When in doubt, ask your escort. EXERCISE CONDUCT After design and development tasks are complete, the exercise takes place. Exercise conduct details include set-up, presentations/briefings, facilitation/control/evaluation, and call-off procedures. SET-UP The planning team should visit the exercise site on the day prior to the exercise to set up the site. On the day of the exercise, planning team members should arrive several hours before the scheduled start time to handle any remaining logistical or administrative items pertaining to set-up and to arrange for registration.

PRESENTATIONS/BRIEFINGS Presentations and briefings are important tools for delivering information. As described in Section 4, a discussion-based exercise generally includes a multimedia presentation to present the scenario and accompany the SITMAN. An operations-based exercise may include briefings for controllers, evalua- tors, actors, players, and observers. A briefing and/or presentation is an opportune time to distribute exercise documentation, provide necessary instructions and administrative information, and answer any questions. FACILITATION/CONTROL/EVALUATION In an operations-based exercise, controllers plan and manage exercise play, set up and operate the exercise incident site, and possibly take the roles of response individuals and agencies not actually par- ticipating in the exercise. Controllers give key data to players and may prompt or initiate certain player actions (as listed in the MSEL) to ensure that objectives are met and the exercise maintains continuity. Controllers are the only participants who should provide information or direction to the players. All con- trollers should be accountable to one senior controller. If conducting an exercise requires more con- trollers or evaluators than are available, a controller may serve as an evaluator; however, this dual role typically is discouraged. Evaluators are selected from various agencies to evaluate and comment on designated functional areas of the exercise. Evaluators are chosen based on their expertise in the functional areas that they will review. Evaluators have a passive role in the exercise and only note the actions of players; they do not interfere with exercise flow. Evaluators should use exercise evaluation guides (EEGs) to record obser- vations and notes. Controllers support the exercise by both conducting functional activities (e.g., setting up a simulation smoke machine) and inserting scripted event messages into play in accordance with the exercise sce- nario. Typically, two types of messages are used in transportation exercises: action messages and con- trol messages. Action messages provide event information to players during the exercise. These mes- sages are usually scripted in the MSEL. Control messages enable controllers to keep the exercise on track and to address situations in which players did not meet specific objectives or took actions that were not anticipated in the MSEL. Specific uses of messages are presented in Table 12. CALL-OFF PROCEDURES Controllers and evaluators should always be cognizant of safety issues that may present themselves during any exercise. If the safety of participants or bystanders is in any way compromised, it is the responsibility of those conducting the exercise to suspend or terminate the exercise. Appropriate atten- tion to that call-off procedure should be part of any safety plan in any operations-based exercise. Guidelines for Transportation Emergency Training Exercises 54 Message Type Uses Action Messages • Used to provide event information (e.g., 911-calls) • Used to provide earned information (e.g., description of damage when a reentry team enters a building) • Used to request information from responders (e.g., news requests via telephone) Control Messages • Used to start, suspend, and terminate the exercise • Used to force responder actions where expectations are not met • Used to insert time-sequenced data (e.g., change in predetermined weather conditions) • Used to start and end simulations (e.g., turning smoke generators on and off) • Used to provide instructions to controllers (e.g., remind responders to state "this is an exercise") TABLE 12 ACTION AND CONTROL MESSAGES

Guidelines for Transportation Emergency Training Exercises There is no standardized format for messages in transportation exercises. Messages may contain any combination of the following: ▪ Message item: Indicates the content or characteristic of the message. ▪ Message number: Indicates the message in the MSEL. ▪ From line: Indicates who issues the message. ▪ To line: Indicates the recipient(s) of the message. ▪ Time: Indicates the expected time at which the message will be issued. ▪ Method of delivery: Indicates how the information will be delivered. ▪ Subject line: Summarizes the message. This is often the exact wording in the MSEL. ▪ “This is an exercise”: Notifies readers that the message is for an exercise. This statement is located before and after the message text. ▪ Special instructions: Inform the person delivering the message of the special conditions for issu- ing the message. ▪ Script or instructions: Provides information to responders based on the MSEL. ▪ Instructions to controller or actor: Indicates activities to start and end simulations, provides attached data forms, or changes simulated conditions. ▪ Note area: Provides space for controllers to note responder performance. ▪ Time of message delivery: Indicates when the message was delivered. Controllers are in a unique position to view exercise play, grasp the dynamics of an action or activity as it unfolds, and comment on what they observe. Controllers should have extensive emergency pre- paredness experience, and they should have participated previously in tabletops, drills, and other exer- cises so that they know what to expect in the way of behavior and response. In addition, they should have observed exercises at other facilities in order to broaden their perspective and experience. They should keep current on updates to the emergency plan. PLAYER INTERACTION WITH CONTROLLERS AND SIMULATORS Controllers and simulators will have constant interaction with players throughout the exercise; however, each interacts differently. Controllers monitor and manage exercise activities to ensure that exercise objectives are being met, interact with players to determine the status of ongoing activities, and com- municate with players by following the MSEL and injecting implementer messages. Controllers must ensure that they do not disrupt play when communicating with players. Simulators, on the other hand, play the role of nonparticipating persons or organizations. The MSEL is the primary document used by exercise controllers to manage the exercise and to know when to insert event implementer messages into the exercise. The MSEL is restricted for use by con- trollers, simulators, and evaluators. Simulators communicate with players by responding to questions from the players directed to non- playing persons or organizations. Occasionally, a selected simulator may act as a surrogate for a senior official or decision-maker. While role playing as a senior official, the simulator may interact with players on a face-to-face basis. Controllers use the MSEL to direct communication with players and to ensure that the exercise stays on track. Simulators also communicate with players, representing an organization affected by the exercise that was not able to attend. Actors support players by playing specific roles, such as injured victims. 55

Players have an active role in responding to an incident by either discussing (in discussion-based exer- cises) or performing (in operations-based exercises) their regular roles and responsibilities. Each player is responsible for acting on exercise messages in accordance with established procedures and for coor- dinating actions in the same manner as for an actual event. Players also must ensure that exercise con- trol is informed of actions taken and completed through follow-up voice or message exchange initiated by the player. Actors are mock victims who simulate specific roles, including injuries from a disaster, to add realism to an exercise. Actors may be made up to more realistically reflect their injuries or symptoms. This practice is referred to as moulage. A sample checklist for managing the conduct of a full-scale exercise is presented in Table 13. EVALUATION Exercise evaluation refers to the act of observing and recording exercise activity or conduct; applying the behavior or activity against exercise objectives; and noting strengths, weaknesses, deficiencies, or other observations. As evaluated practice activities, operations-based exercises provide a process for continuous improvement. Evaluation is the cornerstone of exercises; it documents strengths and opportunities for improvement in a jurisdiction’s preparedness and is the first step in the improvement process. The evaluation process for all exercises includes a formal exercise evaluation, integrated analysis, and an AAR/improvement plan that should begin with exercise planning and end when improvements have been implemented and validated through subsequent exercises. The process recommended by HSEEP is presented in Figure 11. HSEEP Volume II provides extensive guidance for establishing evaluation programs and is available at http://www.ojp.usdoj.gov/odp/docs/HSEEPv2.pdf. As discussed earlier, Attachment 2 provides an exercise evaluation guide that can be used by trans- portation agencies to support evaluation of exercises. Both expected activities and performance mea- sures are identified for each of the eight mission outcomes specified by the DHS. HSEEP includes the following eight steps for evaluation: ▪ Plan/organize the evaluation. ▪ Observe the exercise and collect data. ▪ Analyze data. ▪ Develop AAR. ▪ Conduct debrief meeting. ▪ Identify improvements to be taken. ▪ Finalize AAR. ▪ Track implementation. As described in this process, exercise evaluation should address each exercise objective to answer the following questions: ▪ Was the exercise objective met? Guidelines for Transportation Emergency Training Exercises 56

Guidelines for Transportation Emergency Training Exercises ▪ If yes, what were the results? ▪ If no, what changes are necessary to achieve the objective? Determining how the exercise objectives were met allows evaluators to answer the following important questions about the transportation agency’s performance: ▪ Are parts of the plan in need of revision? ▪ Is current equipment adequate? 57 Full-Scale Exercise Checklist Participants: Controller(s)—sufficient to manage all event sites Actors (mock victims)—different age groups, body types, physical characteristics Players (most functions, all levels—policy, coordination, operation, field) Evaluators Simulators—to convey messages and actions for agencies or individuals who could not participate in the exercise Safety Officer Site Selection: Adequate space for number of victims, responders, and observers Space for vehicles and equipment As realistic as possible without interfering with normal traffic or safety Credible scenario and location Scene Management: Logistics (who, what, where, how, when) Believable simulation of emergency Realistic victims Preparation of simulators to realistically portray roles Number of victims consistent with type of emergency, history of past events Types of injuries consistent with type of emergency, history of past events Victim load compatible with local capacity to handle Props and materials to simulate injuries, damage, other effects Personnel and Resources: Number of participants Number of volunteers for scene set-up, victims, etc. Types and numbers of equipment Communications equipment Fuel for vehicles and equipment Materials and supplies Expenses identified (wages, overtime, fuel, materials and supplies) Response Capability Sufficient personnel kept in reserve to handle routine nonexercise events Safety Safety addressed through development Each design team member responsible for safety in own discipline Hazards identified and resolved Safety addressed in pre-exercise briefing, simulator, and evaluator packets Each field location examined for safety issues Safety officer designated, given authority Legal Liability Legal questions of liability researched by local attorney Emergency Call-Off Call-off procedure in place, including code word or phrase Call-off procedure tested                                              Media Role of media addressed in planning, used as a resource Media and observers considered in logistical planning TABLE 13 FULL-SCALE EXERCISE CHECKLIST

▪ Is additional training required? ▪ Are additional resources necessary? ▪ Are staffing levels adequate? ▪ Is the communication system vulnerable to overload? ▪ How effectively did independent agencies cooperate to resolve the problem? Evaluators record what they observe during the exercise, the hot wash sessions, and the after action review. Their objective is to describe what happened, compare it with what was supposed to happen (as scripted in the MSEL), and explain why any differences between the two occurred. Evaluators also take the lead in drafting recommendations and lessons learned from the exercise. Like controllers, eval- uators must receive special training for the exercise. For most operations-based exercises, evaluation involves the following activities: Guidelines for Transportation Emergency Training Exercises 58 FIGURE 11 HSEEP PROCESS

Guidelines for Transportation Emergency Training Exercises DEBRIEF/HOT WASH A debrief (for facilitators or controllers/evaluators) and/or hot wash (for players) should occur follow- ing both discussion- and operations-based exercises. The debrief is a forum for planners, facilitators, controllers, and evaluators to review and provide feedback on the exercise. It should be a facilitated discussion that gives each person an opportunity to provide an overview of the functional area that they observed and to document both strengths and areas for improvement. The debrief should be facilitated by the lead exercise planner or the exercise director; results should be captured for inclusion in the AAR. Other sessions, such as a separate meeting for specialized respon- ders during an operations-based exercise, may be held as necessary. A hot wash occurs immediately following an operations-based exercise and gives players the opportu- nity to provide immediate feedback. It enables controllers and evaluators to capture events while they remain fresh in players’ minds, to ascertain players’ level of satisfaction with the exercise, and to deter- mine any concerns and proposed improvement items. Each functional area (e.g., fire, law enforcement, and medical) should conduct a hot wash, and each hot wash should be facilitated by the lead controller for that area. The debrief and/or hot wash provides an ideal time for facilitators, controllers, evaluators, and players to complete and submit their completed EEGs and feedback forms. Information from these forms should be included in the AAR/improvement plan. Information on conducting debriefs and hot washes is available on the G&T Secure Portal, which is sum- marized at http://www.ojp.usdoj.gov/odp/docs/HSEEPv4.pdf. AFTER ACTION REPORT To prepare this report, the exercise evaluation team will analyze data collected from the hot wash and/or debrief, participant feedback forms, and other sources (e.g., plans and procedures) and compare the actual results with the intended outcome. The level of detail in an AAR reflects the exercise type and size. AARs describe the exercise scenario, player activities, preliminary observations, major issues, and recommendations for improvement. A sample outline for an AAR includes the following: ▪ Executive summary: — Strengths. — Areas for improvement. ▪ Exercise overview: — Exercise name. — Exercise duration. — Exercise date. — Exercise location. — Sponsoring agency. — Type of exercise. — Funding source. — Classification (i.e., sensitivity of information). The AAR is used to provide feedback to participating jurisdictions on their performance during the exercise. The AAR summarizes what happened and analyzes performance of the tasks identified through the planning process as critical and the demonstrated capacity to accomplish the overall exercise goal. The AAR includes recommendations for improvements based on the analysis, which will be addressed in the IPC. 59

— Scenario. — Participating agencies. — Number of participants. — Exercise overview. — Exercise evaluation. ▪ Exercise goals and objectives: — Goal #1. — Objectives for Goal #1. — Goal #2. — Objectives for Goal #2. — Goal #3. — Objectives for Goal #3. ▪ Exercise events synopsis: — Scenario. — Timeline. ▪ Analysis of outcomes for each participating agency: — Analysis of how well the transportation agency and other participating agencies and juris- dictions performed their functions during the exercise. Examples include a few paragraphs on how well each participating agency performed its functions in response to the exercise scenario. ▪ Analysis of critical task performance: — Analysis of how individual tasks were performed, as defined in the evaluation guides. Each task identified by the exercise planning team as critical to the response required by the sce- nario should be discussed in this section. Tasks that were performed as expected require only a short write-up that describes how the task was performed. These write-ups generally would not be followed by recommendations. For tasks that were not performed as expected, the write-up should include (1) an issue statement; (2) references to plans, procedures, and eval- uation guides; (3) a brief summary of the issue; (4) the effect of the issue on the response; and (5) an analysis of what happened or did not happen and the root causes for the variance from the expected outcome. Recommendations for improvement should also be presented to address identified issues. To facilitate tracking of recommendations and improvements, acronyms should be spelled out in each recommendation. — Documentation of a variance from expected performance that may have resulted in an improved response or innovative approaches that were used during the response. ▪ Conclusion ▪ Improvement plan matrix Guidelines for Transportation Emergency Training Exercises 60

Guidelines for Transportation Emergency Training Exercises A sample AAR is available in HSEEP Volume II at http://www.ojp.usdoj.gov/odp/docs/HSEEPv2.pdf. Templates and checklists, integrated with exercise evaluation plans, are located on the G&T’s Secure Portal and described in HSEEP Volume IV, which is available at http://www.ojp.usdoj.gov/odp/docs/ HSEEPv4.pdf. Other AAR resources for operations-based exercises are available at the following websites: ▪ http://www.dhs.gov/interweb/assetlibrary/T2_Report_Final_Public.doc. ▪ http://www.tahc.state.tx.us/emergency/exercises/Panhandle_Exercise_Report.pdf. IMPROVEMENT PLANNING Postexercise activities are essential to garnering the benefits of an exercise. Careful analysis and pri- oritization should go into developing the AAR recommendations and the improvement plan content. IMPROVEMENT PLAN The improvement plan converts lessons learned from the exercise into concrete, measurable steps that result in improved response capabilities. It is developed by the jurisdiction and identifies which actions will be taken to address each recommendation presented in the draft AAR, which agency will be responsible for taking the action, and what the timeline for completion is. This information should be derived from an after action conference conducted after the draft AAR is completed. IMPROVEMENT TRACKING AND PLANNING Once the improvement plan has identified recommendations and action items and responsibility and due dates have been assigned, the jurisdiction and/or agency should ensure that each action item is tracked to completion. Each state should review all exercise evaluation feedback and resulting improvement plans to assess progress on enhancing preparedness and incorporate the information into its planning process. This review process may identify needs for additional equipment, training, exercises, coordination, plans, and procedures that can be addressed through the state homeland security strategy or multiyear exercise plan. The G&T Secure Portal contains samples of improvement plans for tabletop exercises. HSEEP Vol- ume IV provides a description of these materials and is available at http://www.ojp.usdoj.gov/ odp/docs/HSEEPv4.pdf. FEMA also provides templates. They are available at http://training.fema. gov/emiweb/downloads/CorrectiveActionform.doc. TRANSPORTATION INCIDENT RESPONSE TYPOLOGY To support the activities of transportation agencies in preparing operations-based exercises, Attach- ment 6 provides additional information on transportation emergency response activities and possible emergency response organizations. This information can aid transportation exercise coordinators in developing scenarios and in designing evaluation measures for exercises. Improvement planning ensures that identified weaknesses in the response are adequately addressed and resolved through corrective action by the transportation agency. 61

Next: Appendices »
Guidelines for Transportation Emergency Training Exercises Get This Book
×
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

TRB’s National Cooperative Highway Research Program (NCHRP) Report 525: Surface Transportation Security and TRB’s Transit Cooperative Research Program (TCRP) Report 86: Public Transportation Security series publications have jointly published Guidelines for Transportation Emergency Training Exercises. The report is Volume 9 in each series. The report is designed to assist transportation agencies in developing drills and exercises in alignment with the National Incident Management System. The report describes the process of emergency exercise development, implementation, and evaluation. In addition, the available literature and materials to support transportation agencies such as state departments of transportation, traffic management centers, and public transportation systems are described.

NCHRP Report 525: Surface Transportation Security is a series in which relevant information is assembled into single, concise volumes—each pertaining to a specific security problem and closely related issues. The volumes focus on the concerns that transportation agencies are addressing when developing programs in response to the terrorist attacks of September 11, 2001, and the anthrax attacks that followed. Future volumes of the report will be issued as they are completed.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!