Uses of Simulators
Illustrative Case Studies
Research results reported in the literature and anecdotal evidence suggest that simulator-based mariner training can be used to improved the development of knowledge, skills, and abilities and that such training transfers to and improves the safety of actual operations. These results have not, however, been conclusively demonstrated through empirical research. Nevertheless, marine simulator-based training is considered of sufficient value by marine educators, some operating companies, and a growing number of marine pilot organizations to motivate their investment in the use of simulators to enhance professional development. The case studies presented in this appendix provide examples of various uses of marine simulators within the marine industry and the piloting profession. The studies provide practical insights on course content and operational scenarios and illustrate the value placed in simulators by the individuals and organizations. Case studies are presented for:
- a cadet watchkeeping course, which illustrates careful attention to instructional system design;
- the professional development of apprentice marine pilots;
- port familiarization for ship masters and pilots and the development of port-entry protocols;
- the combined use of simulator-based and follow-up, onboard training on coastwise towing vessels; and
- a recently approved combination training and testing course.
CASE STUDY ONE
SHIP-BRIDGE SIMULATOR-BASED CADET WATCHKEEPING COURSE, U.S. MERCHANT MARINE ACADEMY
This case study reports on the curricula for the U.S. Merchant Marine Academy's (USMMA) full-mission, ship-bridge simulator-based watchkeeping course (Meurn, 1990) and its implementation. The recorded grades for 31 courses involving 233 three- to four-person cadet watch teams (approximately 900 cadets) over a 10-year period (October 1985 to March 1994) were reviewed. The objectives are to examine the application of instructional systems design concepts,to quantitatively identify trends in cadet performance over the course of instruction, and to develop insights and lessons about the application of simulator-based training for third mate candidates with limited, prior nautical experience.
The data that were available were not collected as part of a research experiment. There were no control groups, nor was there follow-up monitoring of real-world performance. It was not possible, therefore, to compare course performance data for ship-bridge simulator-based training with the results of traditional training. Also, there were no data to assess transfer effectiveness for the individuals who participated in the training.
The analysis presented in this case study indicates that the development of watchstander knowledge, skills, and abilities can be significantly improved using simulators as a training medium. The data are not available to determine whether ship-bridge simulator-based training is more effective and efficient than traditional training. The analysis does suggest, however, that the ability to control the learning process (including the ability to design scenarios, monitor performance, and debrief cadet participants), in contrast to the lesser control over learning situations about ships at sea, leads to improvements in efficiency.
The Evolution of Simulator-Based Watchkeeping Training
The USMMA began using simulator-based watchkeeping training in 1980 to supplement other practical training opportunities. Simulation enabled the cadets to become actively involved in decision making on the bridge, in contrast to the observation status associated with most cadet sea-time training aboard commercial vessels.
The USMMA substituted ship-bridges simulators in form and function for a real ship's bridge. Basic operational scenarios were used. Little attention was given to creating scenario designs that would bring the trainees into specific learning situations with specific training objectives. Instructor-cadet interaction was similar to the relationship found aboard a school ship in which a licensed officer is responsible for safe vessel operation. During drill critiques, the instructor interacted as a lecturer rather than as a mentor or facilitator.
The training program was extensively revised in 1985 to incorporate an innovative instructional design process for watchkeeper training used in marine simulator-based courses at the Southampton Institute in England. The course was and continues to be offered to cadets in their senior year as final preparation for actual watches as third mates upon graduation. The course is given after training assignments aboard commercial ships. Separate departments oversee the watchkeeping training aboard the USMMA's training vessel and the simulator-based watchkeeping course curricula.
Specific objectives were established for each training course, and drill scenarios were carefully crafted to bring the cadet watch teams to the instructor-planned learning experience. The mariner instructors redefined their role as a mentors and, during the debriefings, as facilitators, to more effectively stimulate trainee intellectual analysis of the results. Each exercise was graded numerically, recorded, and maintained. The instructors reported that as training progressed, they observed a notable difference in the attitude and command (i.e., leadership) presence of the person in charge of the watch. For each task performance, the instructors reported a subjectively measurable increase in the level of confidence. On completion of the training, the instructors observed that professional maturity, attitude, and confidence were markedly improved.
The Simulator-Aided Watchkeeping Course
The simulator-aided watchkeeping course consisted of 10 preparatory classroom lecture sessions, 4 ungraded familiarization scenarios aboard a full-mission ship-bridge simulator at the Computer Aided Operations Research Facility (CAORF) in Kings Point, New York, followed by either 5 or 6 graded watchkeeping drills aboard the CAORF ship-bridge simulator. (The academic calendar and administration of the U.S Coast Guard's licensing examination for third mate preclude a sixth drill during courses held in the fourth quarter).
The classroom sessions consisted of case studies, familiarization with standard bridge team and procedure prescribed for use by the USMMA Nautical Sciences Division, advance preparation for the drill scenarios, and general briefings and technical discussions.
The simulated ship is the M/V Capella. She is a 40,000 deadweight ton tanker with a single propeller and rudder. Her bridge configurations are representative of this type of vessel. Navigation aids include radar, automatic radar plotting aids, Loran,and fathometer. Electronic charting and automated real-time, precision navigation systems are not available.
The familiarization scenarios consisted of daytime departure from the port of New York, a nighttime arrival at the port of New York and New Jersey, and both daytime and nighttime at-sea scenarios involving rules-of-the-road situations in unrestricted and restricted visibility. Rules-of-the-road scenarios are used because they are usually the first type of situations that third mates experience
and must respond to during their first underway watches. The familiarization simulations also focused on how to use the bridge equipment correctly, as well as how to communicate with the captain by the ship's internal communications and by voice radio with other vessels. By the commencement of the first graded drill, the preparatory elements of the course had exposed each watch team to arrival and departure procedures, change of watch, routine watchstanding procedures, and emergency procedures.
The graded simulations consisted of various operational scenarios with specific learning objectives. Each scenario was designed to bring the cadet watch team to the learning situation. Each succeeding simulation increased in difficulty by about 10 percent, as estimated by the mariner instructors who conducted the course. Scenario design took account of the instructor observation that cadets, as new learners, typically were capable of accomplishing a single task well, but experienced difficulty performing multiple tasks concurrently, especially in a stressful operating environment. The scenario designers also considered the fact, known to experienced mariners, that subtle factors that influence operations can quickly combine in their effect to create challenging and even untenable operating conditions. These factors contributed to the decision to use a single ship for all drills and were used as underlying themes in developing scenarios.
Many watchkeeping courses place trainees aboard a variety of ship types during succeeding lessons. Only the M/V Capella was used in the USMMA cadet watchkeeping course. The experience of the mariner instructors is that the use of the same platform facilitates cadet learning. The basic training objectives of the course were to build good watchkeeping practices and procedures that could be applied to all bridge situations and to prepare cadets to make timely and correct decisions in a multiple-task environment.
The mariner instructors agreed that changing the ship's operating characteristics for trainees with a limited nautical background would detract from course objectives. Changing the platform would have required that cadets to adjust not only to the changing operational scenarios, but also to changes in the ship and its maneuvering behavior. The use of the same ship contributed to progressive improvements in watch team performance reported in this case study. This approach is consistent with actual operations. Duty aboard the same ship for extended periods and continued service aboard the same category of ship are characteristic of marine operations in general.
Early graded scenarios in the drill sequence provided an opportunity for cadets to become familiar with the simulator environment, with the addition of actual responsibility for performance. Subsequent scenarios increased in complexity and difficulty, progressively combining multiple tasks and maneuvering situation to create mentally demanding and stressful operating conditions. A single scenario, for example, might combine an early morning approach, overtaking another ship, meeting another ship, maneuvering situation in which a ''bail-out" option was not conveniently available,and watch relief.
Each of the individual elements of the scenario posed no significant problem, and the overall scenario represented circumstances often faced during harbor approaches and departures. The combined effects of the maneuvering situation, however, with the inopportune timing of a watch relief and its distracting aspects, were used to create a challenging situation in which the timing and precision of navigation and conning tasks were critical.
The first three graded drills exposed cadets to basic fundamentals associated with arrival, departure, and watch relief. The final three drills repeated the basics of the first drills and added additional degrees of complexity, such as cross-currents and wind effects. The first drill was a straightforward port arrival without current and wind effects. The second drill was a departure from an anchorage for a berth. This scenario exposed cadets to the effects of shallow water with small underkeel clearances and congestion in the anchorage that necessitated them to back and fill.
The third drill included a watch relief, the addition of current, and night arrival at a foreign port. The fourth drill incorporated a watch relief with a combination of equipment casualties, man overboard situations, wind and current effects, and traffic ships. Drill five way a day arrival, and drill six was a night departure, each with increasing levels of complexity. The same scenarios have been used since they were introduced in 1985. The mariner instructors found that the combined effects of limited nautical backgrounds,interpersonal dynamics associated with working as a team, officer-of-the-watch responsibilities, and the complexity of watchkeeping effectively prevented cadets from the "programming" themselves in advance to do well in the course.
Different real and hybrid operating areas (that is, part real and part fictitious) were used for each exercise to guard against the masking of performance difficulties that might occur if trainees were to become familiar and at ease with the specific locale used in the scenario. In addition, the use of different operating areas exposed the trainees to differences in nautical charts and publications, the diversity of ports and approaches, and a range of operating conditions they could expect to encounter during actual service aboard merchant vessels.
The general type of graded scenarios and the operating areas simulated were:
- daytime arrival at the Panama Canal;
- daytime departure from Limon Bay, Panama;
- nighttime passage in Singapore Straits with a relief of the watch;
- daytime passage along the California coast with a relief of the watch:
- daytime arrival at Port International (fictitious port, San Clemente Island, California);and
- nighttime departure from Port International.
The use of real operating areas allowed the real nautical charts and publications. Hybrid operating areas (e.g., Port International) consisted of real operating areas with fictitious ports and fairways. These fictitious ports allowed use of real nautical charts and publications with a minimum of supplemental
information for the artificial port fairways. The instructors reported that the scenarios remained relatively constant in content over the 10-years period.
The Learning Environment
The cadets were formed into three- or four- person watch teams that remained together for the duration of the course. Team members alternated, playing the roles of officer of the watch, radar observer, helmsman, and navigator. The radar observer position was not filled in three person watch teams. The cadet officer of the watch was placed in complete charged of the watch. The instructor did not venture onto the bridge except when special role playing was required (e.g., to simulate the presence of a master or marine pilot). Otherwise, the instructor monitored the drill and provided stimulation from the simulator control room, which included closed-circuit television video and voice monitoring and recording. Virtually the entire responsibility for performance and the results of the simulation were in the hands of the cadets. Drill number one was the first time a cadet experienced this level of operational responsibility.
According to the course's mariner instructors, the practice of placing cadets in complete charge of the watch aboard the simulated ship provided more opportunity for broad professional growth than training assignments aboard an actual ship. This effect was attributed to the absence of operational risk, the instructor's control of the operational scenarios that were experienced, and fair treatment during grading (discussed in the next section). Unlike training assignments aboard actual ships, there was no need for the master or officer of the watch to step in before a situation got out of hand (the circumstance that occurs when the safety of the vessel must necessarily be placed above learning objectives). The simulation allowed the cadet watchstanders to experience the full range of cause and effect relationships, and the results of those interactions, and to gain professional maturity through the constructive debriefing. The instructors could stop a simulation instantly if errant decisions so compromised watch team performance that individual confidence would have been eroded. Situations did not arise that required use of this capability.
Watch Team Performance Evaluation Criteria
Most ship-bridge simulator-based courses in which trainee performance is evaluated rely on instructor, or instructional team, subjective evaluation as to whether performance was satisfactory or unsatisfactory. In contrast, the USMMA watchkeeping course was graded according to very specific criteria. Figures F-1 and F-2 are examples of the cadet watch team grading and evaluation sheets. The overall grade for the course has three parts. One-third of the grade is based on watch team performance, one-third on each cadet's performance as officer of the watch, and one-third on a subjective assessment of cadet attitude during the graded drills.
Classroom sessions were not graded. Evaluation criteria for the graded drills had four elements:
- appraisal and planning,
- monitoring and organization and,
Grading began in the fifth week of the course with the first drill. Grading was tailored for each scenario. During the 10-year period of the case study for this appendix, automatic deductions were consistently applied for watch team failure to comply with specific training objectives. The grades for watch team performance were averaged over the five or six drills for a composite grade. Each individual's drill was assessed subjectively by the lead instructor for that watch team. A composite grade for attitude was derived by averaging the scores for the complete series of drills.
Some of the instructors who conducted the course have left, although some have been involved in the course throughout the 10-year case study period. All instructors had prior experience as master and had received training in the use of simulators for instruction. Considerable care was taken by the instructional staff to ensure consistency and evenness in their grading of cadet watch team performance. Measures to ensure consistency and evenness across cadet watch teams and courses involved strict adherence to the course's overall aims and scenario-specific objectives.
Analysis of Courses Performance Data
The data tabulated and compared for each course. The average increase in absolute score was about 10 percent over 5 drills and 11 percent for 6 drills. Several courses resulted in slight decreases in absolute scores over the period. Two cadet teams received no points for one drill. These two scores were not included in the tabulated data because the drills were not conducted and the scores were assigned for reasons other than performance during simulation.
All data were then placed on a line chart to graphically convey the density of the grade distribution (Figure F-3).A modest upward trend in performance is suggested by the dark band at the top of the chart. The density graph, by visual interpretation. The observed variability in scores decreased markedly beginning with the third drill and continued a gradual but steady decline thereafter, as indicated by the calculated standard deviation (Figure F-4). The increase in assigned scores and decrease in variability among teams' performances coincided with increases in the degree of difficulty, observed improvements in watch team cohesion, increased familiarity with the "mechanics" of effective watchstanding, and acclimation to the high level of personal responsibility associated with duties as
officer of the watch. The instructors attributed the increased spread of scores in the second drill to the cadet watch team's lack of appreciation for the increase in the diameter of the turning circle associated with shallow water and small underkeel clearances and the implications of these effects to maneuvering, in this case, the need to back and fill the M/V Capella in order to safely depart a crowded anchorage.
Data were not available that might enable assessment of the degree to which the high variability in the scores for the first two simulation drills might have been influenced by differences in watchkeeping practices aboard the Academy's training vessel and merchant ships used during cadet training periods that preceded the simulator-based training. A plot of the standard deviation for each drill, without regard for the increasing degree of difficulty (Figure F-4) suggests that the overall improvement was gradual but steady after the second drill. The gap between the upper and lower grades suggest that several additional drills could potentially be used to optimize the training program for some cadet watch teams.
Figure F-5 is a plot of the average absolute scores per drill for all cadet watch teams and a weighted average of scores. The weighted average was obtained by multiplying each average absolute score by 110 percent to reflect the estimated 10 percent increase in difficulty with successive drills. The weighted average increase in score was about 54 percent over 5 drills and 67 percent for 6 drills.
Unlike actual operations, the instructors were able to control the operating environment to maximize the concurrent development of basic watchstanding procedures (i.e., the mechanics of standing an effective watch) bridge team coordination, navigation practices, and problem solving. The increase in both the average and weighted average scores does not mean that a similar result could not be obtained through the traditional development of watchstanding capabilities. The lack of control over the operational scenarios that are experienced would, in the committee's experience, require more time to obtain a similar result. The lack of scenario control would also be unlikely to provide either the full range of operational scenarios used in simulation or the exposure to the range of ports and operating conditions that were experienced during the CAORF simulations.
Reinforcement of Simulator-Based Training
No data were available to determine if overall performance could have been enhanced by a more systematic correlation of the simulator-based course curricula and school ship or merchant ship training criteria. A consistent and correlated approach would, however, progressively reinforce the best procedures and practices.
There was no formal program to monitor performance of trainees following the graduation or self-study programs to help reinforce lessons learned from the simulator-based training. Cadets were invited to correspond with the instructors following their first watches at sea as officer of the watch; however there is little anecdotal information of this type to enable an assessment of transfer effectiveness. The responses that were received are encouraging (see Box F-1). It appears from the responses that transfer has occurred in varying degrees, although the mariner instructors believe that optimal transfer has been inhibited by variability in bridge resource management practices aboard ship.
BOX F-1 Third Mate Observations of Value of Ship-Bridge Simulation Cadet Watchkeeping Course, U.S. Merchant Marine Academy
I'm sailing as relief third mate… and your teaching on Rules of the Road has made it easier to cope with the butterflies involved with a third mate's first watches. I find myself evaluating the situation systematically.
Jorge J. Viso, September 30, 1985
I did not assume my first actual bridge watch until we were well out to sea on our way to Valdez, Alaska. By this time, I had made good use of the opportunity to familiarize myself with the bridge, radars steering gear, etc. Still, I was a little apprehensive when I stepped on the bridge at 1930 sharp. After the Chief Mate gave me a few encouraging words, he left the bridge, and I had officially assumed my first bridge watch. The captain was not on the bridge, but he had told me that if I had any problems to call him. The first watch was uneventful. There was some traffic, but nothing came closer than six miles. I have to give CAORF a lot of credit for preparing me for bridge watchstanding. After some initial jitters, I felt very comfortable and confident. In fact, I was much more nervous standing my first watch on the Capella. After going through CAORF, I feel that I am ready for anything, steering failures, loss of plant, etc. Since that first watch, I have had three watches which required maneuvering for traffic situations. For all three situations, I spoke with the other ships over the VHF regarding the course of action we were going to follow. One was a starboard to starboard passing. On another occasion, we lost one of the boilers. The ship slowed from 80.2 revolutions per minute to 60. I was in the chart room filling out a weather report when I noticed the ever present rattling stop. The first place I looked was the RPM indicator which was steadily dropping. I had the helmsman go to hand steering, looked for traffic on both radars and visual, and was about to call the Captain when he came up to the bridge with a weather map. He had already known what was going on and said the engineers had been having problems with the boiler. The same type of situation had happened to me on the Capella.
Robert Lenahan, July 29, 1986
It is now only one and one-half months after graduation and I honestly feel comfortable "in control." I had the wonderful experience of passing through the Florida Straits in a "special circumstance" very similar to a CAORF simulation....I was amazed how smoothly I handled it. CAORF certainly does boost one's confidence.
Robert E. Munchbach, August 4, 1993
CASE STUDY TWO
SIMULATOR-AIDED MARINE PILOT DEVELOPMENT, PANAMA CANAL1
Marine simulation has become an important training aid in the development of Panama Canal pilots. This case study describes pilotage and pilot development and use of simulators.
Panama Canal Pilotage and Pilot Development
The Panama Canal is still operated by the U.S. government. Panama Canal pilots are currently federal employees, although this status is in transition under the Panama Canal Treaty. The Panama Canal Company, the operating arm of the Panama Canal Commission (PCC), employs 242 Panama Canal pilots and pilot trainees. The PCC, as a federal commission, established pilot development requirements that generally follow the federal pilotage program with respect to organization and licensing. A license as master (unlimited oceans) had been an entry-level requirement, but was reduced to aid in meeting requirements of the treaty, as discussed in this case study. There are eight steps (grades) of qualified pilots. After a candidate becomes a qualified step one pilot, advancement from smaller to larger ships is based on time served and other criteria. It takes about eight years to become a fully qualified step eight pilot. The only restriction at step eight is that a pilot needs an observation ride on a submarine before serving as a pilot aboard a submarine.
A structured pilot apprentice program is used for individuals qualified to enter the pilot development program. Qualified pilot examiners conduct check rides as part of initial licensing. Check rides are also used for upgrading purposes during the first two years of progressive advancement. The check rides are done by qualified Panama Canal pilots who serve as evaluators on a voluntary basis. In the past, check rides were qualitative, although some quantitative measures have begun to be applied. The program is technically guided by a substantial training manual, The Panama Canal Pilot (Markham, 1990). The program and specific pilot development criteria are described in The Panama Canal Pilot Training Programs (PCC, 1993).
Changes in Pilot Development Motivated by the Panama Canal Treaty
The pilot development program changed dramatically in response to the Panama Canal Treaty. The treaty requires that all pilots be of Panamanian nationality once the Panama Canal is returned to Panama. This requirement created
a significant training challenge when the treaty was signed, no Panamanians were qualified to enter the pilot training program. This approach to the problem included extensive use of ship-bridge simulators as part of a revised training program.
Anticipating the treaty, the PCC had begun a long-range pilot development program before the treaty was signed. Although the marine license requirement was lowered to enable entry by tugboat masters and ships' officers, this move did not resolve the deficit in pilot candidates of Panamanian nationality. The Panama Canal Company tugboats were not operated by Panamanian citizens.
To encourage Panamanians to enter the pilot development program and to expand the pool of individuals to choose from, the PCC established a preparatory program. Panamanian citizens were placed aboard PCC tugboats for on-the-job professional development. These individuals were progressively advanced to tugboat master over five years. After completing this preparatory training, qualified Panamanians were moved into the pilot training program. Even with this program in full operation, there were insufficient Panamanians available that were qualified to enter the pilot development program, so a pilot ''understudy" program was developed and implemented. The understudy program consists of nautical schooling and two years of licensed service as third mate aboard vessels of 1,600 gross tons.
In setting up the apprenticeship pilot-in-training (PIT) program, it was necessary to look at the specific tasks required of a pilot. Understanding what a pilot needs to know and be capable of doing is especially important in the Panama Canal because the pilot assumes a greater degree of control over vessel maneuvering than occurs under pilotage common to most seaports (NCR, 1994). English is taught as part of the PIT program to improve the English language skills of pilot apprentices. Training in the pilot understudy program is highly individualized. Between 20 and 25 individuals are in the program at any one time. No more than six individuals are accepted from any of the three avenues to enter the PIT program. This limitation was imposed to allow the training program to be tailored to each individual. Each candidate, however, is required to go through the same progression of check rides and series of examinations in qualifying. Candidates are limited to a maximum of six solo transits aboard small vessels during the PIT program. The rest of the transits must be with a qualified pilot.
Training consists of three segments, with a written exam at the end of each segment. The candidate must draw charts of the pilot routes. Each trainee must also present a graduation project, a type of thesis. Projects are assigned to help pilots. The PCC also has a "transit advisor" position for vessels under 65 feet. Trainees are placed aboard these small vessels to acquired additional practical experience, but their role is strictly that of an advisor. As a practical matter, the trainee handles the vessel, and the vessel's master will not take control back unless the trainee advisor experiences a problem.
Application of Simulators in Panama Canal Pilot Training
The PCC has had considerable experience with marine simulators. Computer-based shiphandling simulation was used extensively in the design of improvements for the Galliard Cut. A number of Panama Canal pilots participated in this applied research (NRC, 1992). Under labor-management contractual arrangement, the PCC is not able to to require the use of simulators for the training of licensed Panama Canal pilots or for performance assessments of licensed pilots. Voluntary training in shiphandling has, however, been used for many years. Before acquiring its own simulator capabilities, the PCC choose facilities that had former Panama Canal pilots on staff as instructors. The first simulator PCC installed was a small ship-bridge simulator used for training of licensed pilots. The PCC now also has a full-mission ship-bridge simulator, which has become an essential training resource to assist in qualifying Panamanian citizens as pilots. In using, simulators, the objective of the PCC is to enhance the training provided, not to replace hands-on experienced.
The PCC has established a series of workshops that use simulators. Hands-on training is emphasized. Trainees moderate the workshops. The small simulator is used in the workshops; the large simulator is used only for actual training. In general, the senior pilots have not been exposed to this training. Because of the compensation structure, pilots are generally unwilling to come in voluntarily to participate unless they receive credit for transits-the basis on which compensation is calculated. As a result, simulator-based training has primarily involved junior personnel.
Motivations for Increased Use of Simulators
In addition to treaty requirements, a number of other factors have placed pressure on the professional development of pilot apprentices. More pilots are being drawn from the pool inexperienced mariners who lack the extensive sea-service experience associated with mariners who came onto the program in previous years. In addition, the average age of pilots is going down. Age is an important consideration, because Panama Canal pilots needs a strong command presence.
Because of the great degree of control over vessel operation exercised by Panama Canal pilots under PCC rules, considerable professional maturity is needed. To aid in the development of command presence, professional maturity, and piloting knowledge skills, the PCC established a mentor program. A trainee piloting accompanies a senior pilot during an entire month. The PCC has also established a program in which two trainees accompany a lock approach pilot and participate in a large number of approaches and lockings. During most of these trips, a trainee is an observer. Pilots do not go into the docks as much as in the past at the terminal ends (the docks are now operated by the Panamanian government rather than by commission).
The trend toward larger ships has adversely affected the number of smaller ships available as training platforms. This professional development problem is significant and long term because the refinement of piloting skills has relied heavily on progressive advancement. The use of simulators is one way the commission has attempted to adjust to these changing trends and operating conditions. Each pilot candidate is required to participate in five days of training on the large simulator. Sea-time equivalency is granted on a 1 ratio. Simulation has become more important in developing command presence because, as the composition of the pilot cadre changes in response to treaty requirements, there are fewer senior pilots with whom to ride.
Simulator-Based Training Applications
A pilot has to be capable of spontaneously anticipating what will happen with respect to operational scenarios and ship behavior. Pilots must also be capable of mentally preplanning the transit. For example, the trainee should expect certain hydrodynamic effects under certain conditions. If these effects are not experienced or are different than anticipated, then the trainee should inquire why the effect did not appear or was different and adjust accordingly. Although ship-bridge simulators have limitations, it has been a most useful resource for developing anticipatory abilities.
Simulation is used to provide exposure to tasks and situations such as dockings, mooring, anchoring, and restricted visibility in Galliard Cut, which have not always been available in the canal. Simulation is also used to practice piloting in meeting situations and helps fill the gap created by the lesser number of suitably sized ships as training platforms. Simulation provides a capability for repetition of tasks and emergency response exercises. Scenarios from past canal accidents are used in the emergency response exercises. The training staff has not attempted to replicate the real-world for most aspects of pilot development. The observation opportunities in the Panama Canal are such that there is no need to fully replicate real-world activity.
The present training resources are limited compared to the number of pilot candidates undergoing training. There are a limited number of instructors, necessitating the use of the same instructor for much of the training. In the experience of the instructors, this situation reduces the training value somewhat. The training staff, however, is able to cover procedures that need to be followed to respond effectively. To establish a realistic master-pilot relationship, it is effective in the training program for a Panama Canal pilot to play the role of master during simulation. It also is important to have a pilot or someone familiar with the canal at the control station to observe and assess the simulation. the role-playing opportunity also helps build support among the licensed pilots for simulator-based training.
The training includes all of the tasks in simulation that would occur during a real transit. Close quarters maneuvering has not been emphasized in the training.
because of the difficulty in producing sufficiently accurate hydrodynamic interactions. The decision making needed in meetings and other ship-to-ship interactions are emphasized. The canal also offers substantial opportunities to observe close maneuvers through lock approaches. These opportunities,in effect, serve as real-time simulations for trainees. High-fidelity hydrodynamics are not considered important for many simulation exercises, such as those in which communications are the primary training objective. The training staff uses many short exercises with very specific objectives. Some communications exercises are conducted to address language difficulties.
The training staff has found videotapes of the simulation exercises to be very useful training, debriefing, and assessment aids. The exercise can be replayed in plain view or through the windows. The debriefing process is emphasized as part of the simulator-based training. Simulation has not been used as a testing tool, although there are plans to use simulation for testing for some aspects of training.
Presently, there are no measures of effectiveness for simulator-based training. Such measures have been very difficult to develop. Observations of trainee performance and capabilities are made over an extended period to develop a more complete profile of an individual's preparedness. Box F-2 summarizes a Panama pilot's observations on the value of ship-bridge simulation training. The professional credibility of the evaluator is also important to this process in the PCC program.
CASE STUDY THREE
PORT AND VESSEL FAMILIARIZATION, AMERICAN PRESIDENT LINES2
American President Lines (APL) is a U.S. shipping company. Its container ships presently fly the U.S. flag (although there are plans to reflag some ships). APL has 42 permanent masters (i.e., captains permanently assigned by their unions to specific APL ships). APL has been a major sponsor of ship-bridge simulator-based training and the use of simulators for assessing vessel operating criteria. The company also conducts onboard audits of individual performance. The company began sponsoring bridge resource management training to provide professional training between onboard audits. The company is considering bringing the entire bridge team in for training, rather than just the senior officers.
Perception of Simulator-Based Training
Simulation provides an opportunity to correct bridge team management deficiencies and problems in personal styles without interfering with and jeopardizing
BOX F-2 Observations of Panama Canal Pilots on the Value of Ship-Bridge Simulation Training
Getting the chance to practice maneuvers in Canal anchorage areas in the simulator has allowed me to develop more confidence while maneuvering in crowded anchorages, especially in Limon Bay anchorage." (Pilot understudy)
"My communications skills have improved since I have trained on the simulator. Communications with other vessels, marine traffic, and signal stations has become clearer." (Pilot understudy)
''Backing and filling maneuvers in closed quarters have become a lot smoother since I have been able to practice them in the simulator." (Pilot understudy)
"Maneuvering container vessels with strong winds abeam is one of the maneuvers that I have been able to practice in the simulator that has given me an edge when I have had to do it for real, especially in the approaches to Gutan Locks." (Licensed pilot, two years experience)
"I have been able to experience and better understand the interaction between vessels when passing each other." (Limited pilot)
"I had the opportunity to practice docking situations at dock 16B Critobal before I actually had to do it for the first time. The simulator helped me obtain a better understanding of the situation." (Limited pilot)
"The vessel I was piloting suffered an engine failure. Since I had practiced exercises in the simulator where this type of situation occurred, my response to the real event was quicker and more confident." (Panama Canal pilot)
"While transiting the narrowest part of the Canal, the Culebre Cut, heavy fog set down and the visibility was reduced to almost nil. Having practiced this very scenario in the simulator, I was able to handle this situation in a calm and relaxed manner." (Panama Canal pilot)
"The main disadvantage I have experienced in the simulator is the perspective looking from the bridge. It has taken me some time to get used to it, and it differs somewhat from what you see on board vessels." (Pilot understudy)
"Simulator training has helped me understand and set my priorities when making specific maneuvers." (Pilot in training)
real-world operations. For example, some masters overload themselves and do not effectively distribute tasks. This situation cannot be corrected on the real bridge because it is too dangerous. Simulation is not a replacement for shipboard experience, and especially not a replacement of unusual events.
Simulator-based training, in APL's experience, broadens the student's window of experience and accelerates his or her move up the learning curve. Multiple exposure, repetitions, and immediate feedback are key benefits. Simulation
provides the capability to experiment with alternatives, to see what works and what does not, and to do this without risk. Simulation is also considered to be a valuable medium for use in remedial training, skill development and enhancement, passage planning, port familiarization, shiphandling, familiarization with new vessels, and division of tasks in the ship-bridge operating environment.
Vessel and Port Familiarization
APL has used simulation to selectively introduce its personnel to new equipment and new ship types before the ships enter service. For example, the company sponsored simulated port entries of its new C10 container ships before they entered service in Oakland, California, and in Dutch Harbor, Alaska. In each case, a cadre of local pilots participated in the simulations at APL expense to assist the local pilots in preparing to handle these transits.
In the case of Dutch Harbor, the company had very little operational experience with extremely large container vessels at this new facility in a new location. In particular, the Dutch Harbor area is prone to catalatic wind conditions (known in the region as "williwaws") which occur during certain atmospheric conditions. The company used ship-bridge simulation to develop the operating criteria and standards to be used to guide actual operations. Because they had been prepared in advance for the conditions that were actually encountered, APL masters and local pilots were able to reduce the number of missed port calls and do more in actual operations. The simulation opportunities were particularly beneficial for the pilots.
A shift in state pilotage district boundaries resulted in pilotage services being provided by a new group who had less experience with large ships. Fifteen pilots trained during three training courses. The program was evaluated by the company and the pilots as extremely successful. A very fast learning curve was demonstrated, and the simulator-based training helped establish a working relationship among the pilots and APL's ships' officers.
CASE STUDY FOUR
COORDINATED TUG SIMULATOR-BASED ONBOARD DRILLS, MORANIA OIL TANKER CORPORATION3
The Training Program
Representatives of the Morania Oil Tanker Corporation provided an overview of Morania's training program. The company operates about two dozen towing vessels, with both licensed and unlicensed personnel. The unlicensed
personnel are important and are now part of the training process, and the classroom has been placed on board the vessel.
The company examined its operations and determined that it had significant training needs. The company was aware that a simulator-based training program for towing-vessel operations had been initiated by another towing industry company, Maritrans, a decade earlier. The results of the Maritrans program were encouraging and contributed to Morania's decision to sponsor simulator-based training for its licensed operators.
The company's initial objective was to find out what their operators knew and what they did not know, especially with respect to gaps in knowledge and situational awareness. The company decided that it would not provide simulator-based instruction in boathandling, because boathandling was already a fundamental component of learning by experience. Initially, the company sponsored training for its tug operators and for the operators of tugs it had chartered. A company representative attended the ship-bridge simulator-based training to gain a sense of operator receptivity to the training.
About a year after the initial simulator-based training, the company decided to continue with the training program. It was believed, however, that the vessel personnel were not ready for additional simulator-based training. The company, therefore, opted for an onboard audit of what had been learned and applied from the simulator. The onboard audit was conducted in the year following training. Initially, the company's port captains and senior engineer conducted the audits. These individuals, however, were too close to the operators, and the results were not sufficiently objective. The company decided there was a need for a credible third party to conduct the audits and contracted with a marine training consultant, who had also served as the simulation instructor for the computer-based training sessions. All vessels have now been audited. In the second year following training, the company decided to have their training consultant conduct onboard drills or "real-time" simulation.
On Board Training and Performance Evaluations
The integrated tug-barge units are about 600 feet long, with a speed of approximately 6 knots fully loaded and 10 knots light. The tug watch officer does not have excess power to maneuver the vessel. They use their experience to get out of tough situations.
As part of the instruction, the instructor discusses what needs to be done during different scenarios. After the drills, the instructor conducts an onboard debriefing. To instruct all crew members, since they are on port and starboard schedules, the riding instructor visits each unit twice. The company sends him anywhere the unit is located to complete the training. After the onboard training, the instructor files extensive reports on training results.
Onboard training is considered to be an outstanding way to find out what is going on aboard a vessel. Because the same instructor was used during simulation and onboard instruction, an instructor-student relationship had already been established. In addition, the instructor was aware of what to expect from each individual, based on his or her participation in the simulator-based training course. The instructor was not an board to instruct in or evaluate shiphandling. When on board, the instructor evaluated the performance of the vessel (and its crew) using his nautical expertise as a reference (the instructor's credentials included 50 years of maritime service as master and pilot). The instructor reported that in his experience, he found nothing better than looking over the operator's shoulder. (Of course, the operators were not always receptive to this situation.)
The testing and validation of simulations can be expensive. Once the data and adjustments are in the program, these aspects are basically not transportable. The price of simulation probably will not come down until economies of scale are achieved through more and wider use. Although insurance underwriters base rates on costs to the insurance industry, the company has been able to negotiate some rate reduction as a result of its documentation of training results and improved operator safety performance. Over the last three years, Morania has had a reduction in personnel and indemnity and hull insurance rates, which it attributes in large measure to improved safety performance as a result of the training program.
CASE STUDY FIVE
U.S. COAST GUARD-APPROVED SIMULATOR-BASED MASTER'S LEVEL PROFICIENCY COURSE4
U.S. Coast Guard Course Approval
In late 1994, for the first time, the U.S. Coast Guard (USCG) approved the use of a ship-bridge simulator-based course as an alternative to the standard multiple-choice examination. After several years of planning and development, SIMSHIP Corporation made an unsolicited proposal to the USCG requesting approval of simulator-based testing as a substitute for portions of the master (unlimited oceans) written examination for which ship-bridge simulation was suitable. A full-mission ship-bridge simulator was proposed as the testing platform.
The USCG is interested in encouraging professional training to improve the practical preparation of mariners for service. The agency did not accept a substitute
of a more costly testing platform for its license examination. It did, however, indicate that it was interested in an alternative that would combine training and performance testing. The agency ultimately accepted and approved a combined training and testing course using a full-mission ship-bridge simulator as a voluntary alternative to the entire master (unlimited oceans) license examination. The USCG believes that the course will go above and beyond what is currently required in the license examination and that the approach will increase individual abilities to effectively apply knowledge.
Approval of the training-testing course concept relied on the USCG's interpretation of existing, enabling authority for course approvals found in 46 CFR 10. According to the agency, to remain within the scope of enabling authority, testing must be an integral part of the course, In addition, the approval process must be applied to the curriculum, to the facility at which the course will be conducted, and to the instructors and evaluators.
The initial course approval was for a two-year period. If the approach is successful, the course would be regulated thereafter under the USCG's standard five year renewal policy. Since the course is approved as a substitute for the license examination, the USCG is not granting any credit for sea time. Participation in the course, which is being held at simulator facilities operated by the American Maritime Officers (AMO), is being subsidized by AMO for union members. License candidates not belonging to the union, but wishing to participate, would be required to pay all costs or the costs could be borne by their employers.
A significant difference from current practice, in which USCG license examiners conduct the testing, is that representatives of SIMSHIP Corporation conduct both the training and testing, with USCG oversight. A USCG representative is scheduled to attend at least two of the first three testing courses. The USCG has a goal of overseeing the course two out of every three offerings for the first two years. Thereafter, the frequency of USCG visits to spot check the training and testing processes will depend on the quality of the course. If high quality is obtained, the agency anticipates that its visits and checks would be on a part-time basis. Nevertheless, the USCG plans to conduct checks periodically.
The USCG, in approving the course, stipulated that SIMSHIP Corporation train a USCG representative (with either commercial or naval marine experience) for the oversight role. The objective of the training is to prepare the agency representative sufficiently so that the agency is capable of determining compliance with terms and conditions of the course-approval criteria. Additional stipulations are as follows:
- The course will be used as a substitute for the master (unlimited oceans) examination only.
- The lead instructor and evaluator must hold a master's (unlimited oceans)
- license and have served as master for one year on that license on a ship of 5,000 gross tons or greater
- Individuals with lesser qualifications could assist the lead instructor or evaluator, but a marine license is mandatory.
- Each instructor and evaluator is required to have training in the use of simulation and in basic teaching and evaluation techniques.
- No individual can serve as both an instructor and evaluator during any one course.
- Other students or license candidates may not role-play during performance evaluation components of the course.
- The training and testing course will be open at any and all times to USCG visitation, participation, and inspection.
There are no industrywide or USCG standards for the training of instructors or evaluators. The training may be provided "in-house" or obtained from other sources. An in-house "train-the-trainer" course has been developed by an expert in training systems and technology and is in use for training instructors in other courses offered by SIMSHIP Corporation.
The USCG did not require formal validation of the simulator or of the simulation as part of the course-approval process. The agency relied on SIMSHIP Corporation to self-validate the course. This self-validation was done through a structured series of live trials using volunteer test subjects, including chief mates, master mariners, and marine pilots. SIMSHIP's interdisciplinary staff, including a hydrodynamicist, master mariners, and individuals with extensive experience in the use of simulator-based mariner training, were also involved. A description of the validation process was not required prior to course approval. As part of the approval process, USCG representatives holding relevant marine licenses witnessed the dedicated trials and examination portions of the course.
The USCG has not established a monitoring program to determine the effectiveness of the training and evaluation course with respect to subsequent actual performance. As a quality control measure, the USCG will require that each candidate take one of the multiple-choice testing modules. The module will be selected at random and administered at a USCG regional examination center by a USCG license examiner. The candidate will be issued his or her license regardless of the outcome of the written examination. The USCG plans to retain and analyze the results of the random testing and will consider the results in determining whether to renew the course approval.
Course Description and Process
The approved training and testing consists of a two-week course. The approval documents stated that a maximum of four license candidates will participate in any given course. Each element of the current license examination is featured in the course curriculum. Heavy emphasis is placed on bridge resource
management and maneuvering ships in restricted waters. Every course topic is required to be at the master level. Student performance on examinations will be tracked through check-off lists during training and testing.
Training and testing will be conducted separately. The first week is devoted to training, which will address the subjects to be examined on the simulator. Students must also be prepared to take written examinations on topics that are not going to be examined, using the simulator as the evaluation platform. The second week is devoted to testing for all elements of the license examination. The instructors involved in training during the first week are precluded from serving as evaluators the second week. The USCG's stipulations do not preclude instructors from being involved in role playing.
The testing will include practical demonstrations for communications, chart work, bridge resource management, and situational awareness. Ship-bridge simulation will be used only for those elements of the testing for which it is suited (e.g., bridge resource management, rules of the road, shiphandling). A generic port is used for the simulations. Written tests will continue to be used for most other elements of the examination, including ship's business, and will be drawn from the existing pool of USCG-maintained, multiple-choice license examination questions.
The scoring of performance demonstrations will be done on a pass-fail basis and measured against specific, weighed criteria. This approach was adopted to remove subjective judgment as much as practicable from the assessment component of the course. The determination will rely on the professional expertise of the evaluators. The USCG has not established specific criteria for use as assessment benchmarks. The agency believes that course materials and any criteria developed and used by SIMSHIP Corporation are prorietary, even though the course is approved as a replacement for an official licensing examination.
On successful completion of the course, license candidates will be issued a certificate that can be presented to the responsible USCG licensing official. An individual who does not pass the training and testing course has the option of (1) retaking the course or (2) taking the USCG's standard multiple-choice examination. If retaking the course, the candidate would have to return at a later date. The length of time would vary, depending on the degree to which the individual's participation fell short of performance expectations. Under current criteria, the maximum number of times an individual may take the course is two. During the course, a participant can fail and retake one of the modules. Any trainee failing two of the modules is "washed out" of the course. Individuals who do not successfully complete the course can choose to take the written USCG examination without any time delay. In this situation, the USCG would view the course as nonrequired training.
Markham, G.A. 1990. The Panama Canal Pilot. Balboa, Panama: Panama Canal Commission.
Meurn, R.J. 1990. Watchstanding Guide for the Merchant Officer. Centreville, Maryland: Cornell Maritime Press.
NRC (National Research Council). 1992. Shiphandling Simulation: Application to Waterway Design. W. Webster, ed. Committee on Shiphandling Simulation, Marine Board. Washington, D.C.: National Academy Press.
NRC (National Research Council). 1994. Minding the Helm: Marine Navigation and Piloting. Committee on Advances in Navigation and Piloting, Marine Board. Washington, D.C.: National Academy Press.
PCC (Panama Canal Commission). 1993. The Panama Canal Pilot Training Programs. Unbound compilation of pilot training program materials. Balboa, Republic of Panama: PCC.