3
Effective Training with Simulation: The Instructional Design Process

Many current marine education and training approaches are outgrowths of an established profession with strong, traditional professional development practices. These practices reflect an approach to instruction in which instructors often served as both teacher and mentor, and ship's officers were trained to be generalists.

The use of ship-bridge simulators is becoming an accepted method of training in the international marine industry. Yet even as more simulators are being used, their use has not dovetailed smoothly into comprehensive training programs. Many simulator-based training courses were developed ad hoc, often designed to individual requirements of a shipping company or training establishment.

Among the experts the committee consulted, there was strong evidence that they had thought seriously about training needs and had organized their programs to address those needs. There were also diverse opinions on the exact character of those training needs. In some instances it appeared that training emphasis was guided by equipment capabilities such as simulators. The committee concluded that professional training in the maritime industry could be improved by effective advancement and systematic application of instructional design concepts.

DEVELOPING AN EFFECTIVE TRAINING PROGRAM

A simulator does not train; it is the way the simulator is used that yields the benefit. "It is easy to be impressed by the latest, largest full-mission simulator, but what is more important than the technology is how educational methodology



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 67
--> 3 Effective Training with Simulation: The Instructional Design Process Many current marine education and training approaches are outgrowths of an established profession with strong, traditional professional development practices. These practices reflect an approach to instruction in which instructors often served as both teacher and mentor, and ship's officers were trained to be generalists. The use of ship-bridge simulators is becoming an accepted method of training in the international marine industry. Yet even as more simulators are being used, their use has not dovetailed smoothly into comprehensive training programs. Many simulator-based training courses were developed ad hoc, often designed to individual requirements of a shipping company or training establishment. Among the experts the committee consulted, there was strong evidence that they had thought seriously about training needs and had organized their programs to address those needs. There were also diverse opinions on the exact character of those training needs. In some instances it appeared that training emphasis was guided by equipment capabilities such as simulators. The committee concluded that professional training in the maritime industry could be improved by effective advancement and systematic application of instructional design concepts. DEVELOPING AN EFFECTIVE TRAINING PROGRAM A simulator does not train; it is the way the simulator is used that yields the benefit. "It is easy to be impressed by the latest, largest full-mission simulator, but what is more important than the technology is how educational methodology

OCR for page 67
--> is applied and whether it increases training effectiveness significantly, incrementally, or at all" (Drown and Mercer, 1995). Trainees taking part in most simulator-based training courses can be divided loosely into two groups: unlicensed cadets who work through a series of structured courses; and fully qualified, licensed mariners who take stand-alone courses for updating, refreshing, and refining skills. Although instruction design theories can be applied to both groups, the training programs and corresponding instructional processes will vary by trainee level. An effective training program addresses the student's training needs with respect to knowledge, skills, and abilities. It exploits all media, from personal computer-based training to limited-task and full-mission simulators and applies the appropriate training tool to the specific level of training. For example, it would not be necessary to use a full-mission simulator for early instruction in rules-of-the-road training. Rather, a systematic approach to training promotes convergence toward full-mission expertise by developing basic modules of skills in several steps. This approach encourages the assembly of ever-larger skills modules until the trainee can exploit training on a full-mission simulator. The instructional process is central to the overall focus of this report. Instructional design is a relatively new process. It has not advanced sufficiently to the point where this committee can provide a complete vision of how it should be implemented. The committee can, however, provide guidance. Instructional design is an iterative process whereby training managers continually test innovations and improve training. It is an incremental approach that involves inserting new pieces developed by the instructional design process into existing training programs, assessing results, and then revising the program as necessary. This systematic application yields simulator-based training programs with clearly defined objectives, carefully designed training and evaluation scenarios, and qualified instructors. Figure 3-1 illustrates the iterative nature of this training process. There are several stages to implementing the instructional design process (see Box 3-1). A vital first stage is determining training needs. This stage is important because current national and international licensing and certification requirements and guidelines focus primarily on knowledge rather than skills and abilities needed to effectively apply knowledge (Froese, 1988). Training needs can be developed by identifying gaps or missing elements between the trainee's required and actual knowledge, skills, and abilities. The second stage is to determine specific training objectives (i.e., goals). Objectives identify each attitude, skills, and block of knowledge the trainee should have on successfully completing the course (Drown and Mercer, 1995). Once the course has started, these objectives should be clearly stated to orient the trainee. Development of training objectives should also include developing performance

OCR for page 67
--> FIGURE 3-1 The training process.

OCR for page 67
--> Box 3-1 Elements of Instructional Design Process characterizing training populations and identifying specific training needs; determining training objectives (i.e., goals); determining course content and course material requirements needed to meet training objectives; determining training methods; identifying training resource requirements and correlating them with training objectives; matching specific instructional techniques to curriculum content; • identifying student assessment requirements and developing assessment   methodologies; and establishing instructor qualification, selection, training, and certification requirements to ensure quality of instruction and successful curriculum implementation. measures for determining whether or to what degree trainees have achieved the training objectives. These important elements of instructional design have not been well addressed and have sometimes even been neglected in simulator-based courses. The third stage is to determine the training methods to be used. This stage includes an assessment of whether simulation use is relevant to achieving the training objectives. Assuming simulation is to be used, two things must be determined: the level of simulation (i.e., level of realism with respect to simulator components [see Chapter 4]) required to achieve training objectives (Hays and Singer, 1989) and the type of simulator—full-mission, multi-task, limited task, or special task (see Box 2-1)—that will be of most value. The development of the training approach should also consider factors such as: the total training program of which the simulator-based training is part (e.g., cadet training toward a first license), trainee experience, type of training media, instructor's qualifications and experience, and cost benefit and effectiveness of the training program. Once it has been determined that a simulator-based course is relevant to training needs, it is necessary to develop a detailed course outline. Finally, there

OCR for page 67
--> needs to be a validation of the simulator, the simulation, and the curriculum to ensure relevance and suitability. Anecdotal evidence, as well as experience and observations of the committee, suggests that differences in instructional techniques can result in a significant range of material that can be covered. The way material is covered also affects the relative value of the learning experience. These factors may be affected by simulator features and fidelity; however, limitations in these areas can be minimized or offset to a large extent for certain instructional objectives. For example, committee members found that, for bridge team and bridge resource management training, creative instructional design can be used to compensate for limitations in simulator capabilities. According to a group of mariner instructors, many of whom met with the committee, the practical considerations shown in Box 3-2 are particularly relevant when structuring a mariner training program. These observations generally correspond with the results of human performance research. APPLYING INSTRUCTIONAL DESIGN Defining Training Needs and Objectives The Trainee Population There is great diversity in the professional backgrounds and maturity of trainees. Group members may range from entry-level to mates and masters to marine pilots and shore-based management officials. This diversity can affect the development of effective training programs because of the possible range of training needs—from entry-level training; license upgrades and renewals; refresher training; and familiarization training for specific ports, routes, vessels, or vessel types. For these reasons, simulator-based courses will be most useful when developed systematically. Job-Task Analysis The systematic definition of training needs requires a detailed understanding of tasks and subtasks necessary to perform the function to be trained. Traditionally, professional development aboard commercial ships and tugs has relied pre-dominantly on "modeling the expert" for complex cognitive tasks—the person undergoing training watches and imitates the performance of senior professionals. This modeling is generally accomplished through on-the-job observation and hands-on experience aboard vessels. In the absence of comprehensive instructional design and attention to instructional abilities of the expert who is being modeled, this approach may have important limitations with respect to the quality of the learning experience.

OCR for page 67
--> BOX 3-2 Training Insights from Mariner Instructors The overall training program, not just one component, must be effective. Simulators should be selected and simulations designed to meet training needs rather than structuring training to fit the simulator. Training performance should be measured against predefined performance criteria. Training (or remedial training) should be continued until the required proficiency level is reached. No matter how well the training program is designed, refresher training may be needed to maintain a level of knowledge and skills. Careful, structured evaluation of trainee performance (a) prior to, (b) at the conclusion of, and (c) after the program (in the workplace) is necessary to monitor program effectiveness. Conditions and attitudes in the workplace must be conducive to transfer of training. Policies, practices, and attitudes of regulatory authorities, shipping companies, and ships' masters are vital to ensuring effective training and subsequent transfer to the workplace. The trainee's real-world performance, not the speed of acquisition of a task during training or the level of performance at the end of training, should be the measure of program effectiveness. There are two obvious difficulties in using direct modeling for complex cognitive tasks. First, the rationale for the performance of the tasks is not only opaque to observers, but may also be implicit for the experts: they may not be able to describe their own thought processes or the rationale for them, even though they can perform the tasks. Second, in order to properly coach a novice, an expert may have to formulate an accurate mental model of the novice's understanding of the task (sometimes called the student model). But a novice's understanding of a task is not always obvious to the expert. These two key problems… suggest that there might be a limitation on the extent to which direct modeling of complex cognitive skills can be done (NRC, 1991). The instructional design process offers an alternative to on-the-job or "modeling-the-expert" training methods. Without a more fully developed basis for quantifying actual training needs, the use of simulators in professional development and marine licensing will continue to be based on perceived needs and professional estimates. To apply instructional design, it is necessary to have detailed, relevant task and subtask analyses.1 As noted in Chapter 1, although 1   The U.S. Navy has used simulation to conduct shiphandling training since 1987. Its training syllabus was created by the instructional system design process, based on task analyses and personnel performance profile. This program may be a valuable source of information and data for the USCG and others in the development of simulator-based training programs.

OCR for page 67
--> there is a significant amount of literature on job-task analysis, much of that material is either dated and needs to be updated or has only limited application to the marine industry and to mariners' training needs. The task analyses needed to define training needs in instructional design should be detailed and include descriptions of steps required to complete identified subtasks. Applying Job-Task and Performance Analysis Instructional objectives, curricula, and, to some extent, instructional approaches are designed to satisfy perceived needs and expectations of training sponsors. Perhaps as a result, the instructional quality of the simulator experience at all ship-bridge simulator facilities is generally reported to be good by companies that have supported simulator-based training and mariners who have attended courses using ship-bridge simulators. However, because the committee could not find evidence of programs designed to measure and analyze resulting performance, it could not determine whether these courses are achieving optimal effects in improving mariner preparation and performance. To determine whether a training program meets its defined objectives, it is necessary to develop a system to measure and analyze resulting performance. A systematic approach to marine professional development necessitates improving understanding of job performance. This improved understanding could be accomplished by expanding existing job-task analyses to include dimensions that are generally missing with respect to behavioral elements and specific steps needed to execute each subtask. It is important to recognize that not all tasks contribute in the same way to overall performance of functions and duties of the job. There is little current task analysis work in the marine industry. One example of recent work (Sanquist et al., 1994) is a study done by Battelle Seattle Research Center under contract to the U.S. Coast Guard (USCG) to develop a systematic approach for determining the effects of new automation on mariner qualification and training. The program was undertaken in connection with the agency's responsibilities in marine licensing and its goal to ''determine the minimum standards of experience, physical ability, and knowledge to qualify individuals for each type of license or seaman's documentation." Although the report,Cognitive Analysis of Navigation Tasks: A Tool for Training and Assessment and Equipment Design (Sanquist et al., 1994), is targeted to automated systems aboard ships, the results should provide task analyses at a level that is detailed enough to effectively apply instructional design to relevant training program development. The following descriptions of the process are summarized from the report. To maintain the safety of our waterways, the U.S. Coast Guard needs to assess how a given automated system changes ship-board tasks and the knowledge and skills required of the crew.… Four different, but complementary, methods are being developed.

OCR for page 67
--> Task analysis. A task analysis technique has been devised that breaks down a ship-board function, such as collision avoidance, into a sequence of tasks… the current approach synthesizes various existing methods and adapts them to the marine operating environment. Cognitive analysis. This method looks at the mental demands (such as remembering other vessel positions, detecting a new contact on the radar, etc.) placed on the mariner while performing a given task. The cognitive analysis identifies the types of knowledge, skills, and abilities (KSAs) required to perform a task and highlights differences in mental demands as a result of automation. Cognitive analysis is applied to an operator function model (OFM) for task analysis. OFM provides a task analysis that is independent of the automation, i.e., OFM defines the (1) tasks, (2) information needed to perform the tasks, and (3) decisions that direct the sequence of tasks, regardless of whether they are performed by the mariner or the equipment. Skills assessment. This method evaluates the impact of automation. It takes the results of the task and cognitive analyses and determines the types of training required to instill the needed KSAs for performing the ship-board tasks. These will be compared with current training courses to highlight any new training needs. Comprehensive assessment. This method addresses the large number of problems resulting from an operator's misunderstanding of the capabilities and limitations of an automated system. For example, when the radar signal-to-noise ratio is poor, the ARPA [automatic radar plotting aids] may "swap" the labels of adjacent targets. If the mariner is not aware of this limitation, he may be navigating under false assumptions about the position of neighboring vessels, increasing the chances of a casualty. Comprehensive assessment will identify misconceptions about automated systems that could then be remedied through training or equipment redesign. The USCG intends to use this methodology to highlight necessary training and licensing changes. In licensing, for example, the agency's analysis found that use of automatic radar plotting aids in performing the collision avoidance function eliminated nearly all computational requirements on the mariner. Yet application of the cognitive analysis technique to a sample set of questions from the radar observer certification test found that approximately 75 percent of the questions tested computational skills. Given these results, the report concluded "it would appear that there needs to be a shift in emphasis from computational to interpretive questions on the radar observer certification exam." DETERMINING TRAINING METHODS Selecting the Training Media Mere possession of a ship simulator or other training device and the presence of licensed mariners as instructors do not guarantee the effective and credible

OCR for page 67
--> use of simulation or effective learning. The presence of specialized features, such as physical motion platforms and high-fidelity graphic images, do not in themselves guarantee a relevant and meaningful training experience. To be effective, simulator resources must be matched to instructional objectives. If simulation is relevant to training objectives, then the type (see discussion in Chapter 2) and level (see discussion in Chapter 4) of simulation, including fidelity requirements, need to be determined. The way a simulator is treated is important in creating a perception of reality among trainees. If the instructional staff treats a simulator as if it were a ship and the simulator environment as real operating conditions, then the trainees are more likely to treat the experience as "real." In creating the illusion of reality for a limited-task or higher-level simulator, attention needs to be given to accuracy requirements for the mathematical models that drive the simulation (Appendix D). Creation of the training environment is discussed in greater detail in Chapter 4. No quantitative research was identified by the committee that would establish the relative merits of different approaches to marine training and the types of training offered. There is, however, a research basis that supports the application of different levels of simulation to achieve certain training objectives for cadets, mates, masters, and pilots (Hammell et al., 1980, 1981a, 1981b, 1985; Gynther et al., 1982a, 1982b, 1985). The guides from this research are either preliminary or dated as a result of recent changes in manning and automation. Nevertheless, the guides remain a principal reference and could be used as a starting point for instructional design. The committee observed that few, if any, facilities appear to be using these materials for this purpose. Mariner instructors reported to the committee that the degree to which a participant is familiar with the training media affects the media's relative value with respect to individual learning. Media familiarity also influences individual performance during a simulation. Just as in real life, a mariner becomes more confident in his or her operating performance of a ship-bridge simulator, manned-model, or radar simulator as his or her familiarity with operating characteristics and specific operating conditions increases. The instructional and training value of all marine simulator-based training media are also affected by the nature and form of instruction, including the operational training scenarios (see discussion of type of scenarios in Chapter 4.) Defining the Training Program Curricular A complete ship-bridge simulator-based training course curriculum will typically include information on the following:

OCR for page 67
--> overall course objectives, characterization of course participants and numbers, characterization of participant educational and professional backgrounds, course structure, course timetable, individual simulation exercise planning sessions, individual simulator exercises and their content, individual exercise objectives, instructions to staff on the methodology of simulation exercises and subsequent debriefings, and method of evaluating participants, as applicable. Appendix E contains outlines of two simulator-based training courses. Exercise Scenarios Scenario Design. Once a simulator-based training program and its objectives have been defined, exercise scenarios should be developed. The following factors should be taken into account when designing these scenarios: type of simulator (e.g., special task, full-mission); geographical database; mathematical model ship type and, to the degree relevant to training objectives, the model's fidelity with respect to ship maneuverability in restricted shallow water with small underkeel clearances; type and structure of exercise scenario required to achieve the exercise objectives; exercise length; method of briefing and debriefing; cost effectiveness; level of fidelity and accuracy needed to support training objectives (e.g., quality, field of view, cues in the visual scene, and accuracy of trajectory prediction); and validation requirements. Scenario creation is crucial to optimizing the training value of individual exercises. Simply creating a realistic scenario does not necessarily result in operating conditions that will evoke desired student responses, create an effective illusion of reality, or create real life pressures (Edmonds, 1994). Developing situations intended to challenge or test trainees is sometimes accomplished through scenarios involving role playing. In one possible situation, assignments could be reversed, with seniors placed in subordinate positions and junior personnel in senior positions. The objective is to create a pressure situation in which it becomes apparent to participants that improved interpersonal

OCR for page 67
--> dynamics and communications are needed to reduce the potential for human and organizational error. This form of role playing seems to work. It should, however, be very carefully debriefed to avoid any negative impact on the confidence of junior personnel whose performance may have resulted in a failed solution during the exercise. In some bridge team management training, positional shifts in roles can be used to show junior officers the tasks of a master and to remind senior deck officers of the difficulties and limitations of watchkeeping. In the case of pilot training, the pilot and master roles can be reversed to increase the pilot's sensitivity to the master's concerns when the pilot is controlling ship movement. It also is possible to stimulate lessons through more subtle, yet equally effective, means. For example, a delegation from the committee participated in a simulation involving a crew change, a watch relief, two ports unfamiliar to the new watch officers, and a transit speed that was excessive for the situation but not readily apparent. As the scenario unfolded, bridge team members created enough pressures and problems for themselves without any assistance from the instructional staff. The need for more effective passage planning and improved communications among bridge team members was no less apparent than it might have been in a situation artificially influenced by role reversals or problems inserted by the instructor. Scenario Validation. Once designed, an exercise scenario must be validated. Validation is necessary to avoid variations in the scenarios that could adversely affect training objectives or provide inaccurate information or insights and therefore contribute to human error during real operations. Care must be taken to ensure that relevant cues are present. In cases where individuals are being prepared for shiphandling and piloting on specific waterways or vessels, higher levels of visual scene fidelity and trajectory prediction accuracy are indicated. These factors are especially important in operating conditions involving restricted shallow water with small underkeel clearances. It can be an advantage for instructors to visit and familiarize themselves with the real geographical area they are simulating. A visit and local knowledge also help instructors incorporate appropriate visual cues and local operating procedures, particularly if the instructor may have to play the role of a local pilot. Alternatively, a mariner with prior experience in the simulated area could serve as a design consultant. The instructor must be satisfied that the exercise can be concluded in a way that is relevant to exercise objectives and that the scenario can achieve exercise objectives enroute. The operational result may be successful or unsuccessful, as long as training objectives are satisfied. Only then can the scenario be used with confidence to effectively satisfy training objectives. There is no standard methodology for validating exercise scenarios. The instructor or instructional staff generally perform this function subjectively,

OCR for page 67
--> Instructor Knowledge and Expertise As a practical matter, the instructor's subject-matter expertise is essential to instructional design. The instructor's tasks, however, are multifaceted. Many of the instructor's tasks (Box 3-3) are in addition to and lie outside of the mariner's nautical expertise. There are limited opportunities for mariners to undertake instructional roles aboard modern ships. One possible exception is in piloting, where apprentices are developed under the supervision of marine pilots as a normal practice (NRC, 1994)3. Although some mariners develop good instructional capabilities during their seagoing service, effective application of the instructional design process requires specialized skills that must be developed or refined separately through specialized programs. The need for specialized skills is even more important in the use of ship-bridge simulators. As an instructional tool, simulation has evolved to a level of technical and instructional sophistication that often requires multidisciplinary expertise and technical support. In such cases, the instructor needs to be capable of working as a member or leader of an instruction team. The Instructor's Role In conducting training, the instructor lectures, role-plays, and facilitates. He or she is the intermediary for: creating a synergism among student, curricula, and simulator; and making the simulation believable and meaningful. Objectivity Although watchkeeping courses have not been mandatory for certification of officers in charge of a navigation watch, some national agencies (including the USCG) grant remission of sea time after completion of such courses. In these cases, the instructor, by virtue of the instructional role and student evaluations, is involved in the award of a completion certificate. The instructor has either moral or official responsibility, or both, for ensuring that each trainee's performance has been satisfactory. At the same time, care must be exercised to ensure that interpersonal relationships do not influence performance evaluations and that the 3   On-the-job training is also used in the professional development of docking masters and operators of uninspected towing vessels who pilot tug and barge flotillas on inland rivers and waterways. The pool of docking masters has not been a source of simulator instructors because simulation has not been used in training for their profession. There are only a few instructors with professional towing backgrounds because simulation has to date been used only to a limited extent in the coastwise towing industry (only one instructor as of 1994).

OCR for page 67
--> BOX 3-3 Instructional Tasks Development of a thorough professional knowledge of the subjectmatter (e.g., shiphandling, bridge team management techniques, radar operations). Maintenance of up-to-date knowledge about relevant developments in ship operations (e.g., navigation technology, marine safety policies, and procedures including marine traffic regulations). Development of a thorough knowledge of the functional operation of the simulator and its capabilities and limitations. Development and implementation of training courses, including objectives and, if appropriate, integration of these courses into the total training program. Development of simulation scenarios that best support instructional objectives. Communication with marine industry and piloting professionals regarding requirements and details of training courses (i.e., training needs). Preparation of all necessary course material and equipment. Validation of databases and scenarios. Validation of ship models and production of ship-model maneuvering data. Preparation of incoming courses and coordination of schedules and training strategy with other members of the instructional team. Conduct of courses in a professional manner, using proven and agreed-on teaching methods and skills. Supervision or conduct of debriefings. Preparation and development of trainee evaluation process. award of credit toward a certification requirement does not unduly influence training program integrity. Flexibility and Sensitivity The instructor must be capable of adjusting to trainees' different professional experiences. Experienced trainees are already a professional in their field and should be treated accordingly. In these cases, the instructor's role as a facilitator takes on more importance than it does with less-experienced deck officers or cadets. Currency The main demand on the instructor who teaches professional courses is one of stimulating the trainee to rethink his or her own performance objectively and

OCR for page 67
--> constructively. The instructor must not only know the subject-matter thoroughly, but be up to date on recent events, such as marine accidents and their proximate and underlying causes and developments in equipment and operating practices worldwide. Instructor Qualifications Professional Credentials There is a strongly held position among maritime instructors that all simulation instructors should possess the highest seagoing qualification awarded by a flag state, which is commonly understood to mean a master's license with no restrictions or an unlimited master's license. In principle, the marine license ensures subject-matter expertise and the institutional consideration of nautical credibility. Some people believe that knowledge of the course content and proficiency in instructional skills are paramount, and that possession of a senior marine license alone guarantees neither relevant nor recent content knowledge nor instructional skills. Instructors without formal instructional skills training are most likely applying instructional knowledge rooted in informal on-the-job and apprenticeship approaches to professional development. Although the insights that accompany this background are important for discerning and conveying the marine operations subtleties, practical experience does not by itself prepare an individual to apply modern learning concepts. The Need for Guidelines or Standards Instructor qualification must consider both instructional and institutional factors. From an instructional perspective, the instructor must possess the right content knowledge as well as instructional skills. From an institutional perspective, the instructor must be credible to trainees and sponsors. In addition, if marine licensing is involved, the appropriate form and level of instructor qualification is also important to the licensing authorities. The rapid evolution of simulator capabilities, from desktop computer-aided instruction and presentations to full-mission ship-bridge simulator, suggests that there should be more formal standards for qualification of simulator instructors. With a few notable exceptions (discussed below) there are no professional, industry, or national guidelines, standards, or requirements for certifying instructors, either through professional organizations, marine industry, education, training programs, or government agencies. Nor is there a specific professional code of ethics for instructors involved in mariner training. Generally, the instructional capabilities of instructors is determined by employers through job interviews and review of professional credentials.

OCR for page 67
--> BOX 3-4 Samples of Instructor Training Programs, Maritime Academy Simulator Committee (MASC): Draft "Train-the-Instructor" Course In pursuit of one of its goals as a committee, MASC has been working to develop standards for simulator instructors. MASC considers it mandatory that all simulator instructors be required to attend a course that covers the following subjects: generic laboratory teaching, development of a ship-bridge simulator-based learning system, instructional strategy for simulation via a design workshop, hydrodynamics, debriefing techniques, instructor attributes, exercise design, and grading. MASC is currently refining the course structure and curricula. Boxes 3-4, 3-5, and 3-6 summarize the focus of three different "train-the-trainer" programs. Box 3-4 summarizes an effort by the Maritime Academy Simulator Committee to develop a training program for simulator instructors at U.S. merchant marine academies. Box 3-5 gives samples of extensive courses at the Southampton Institute, Warsash Maritime Centre in the United Kingdom, for training instructors who teach on a full-mission ship-bridge simulator. Box 3-6 summarizes a government-required training program in the Netherlands. In the United States, a de facto certification of instructors occurs through the USCG's administration of course approvals for training programs used, in part or in whole, to satisfy certain federal marine licensing requirements. The agency has established criteria regarding instructor qualifications that must be met to receive course approvals. Evidence of training in instructional techniques is required, and a simulator facility must notify the agency of any changes in instructors, including the credentials of the individual who will be teaching the course in cases where some sea-time equivalency or licenses are awarded. In December 1994, the USCG began examining an internal proposal to establish a formal certification requirement that it would administer. The proposal envisioned three categories of certification: certified maritime instructor, designated simulator examiner, and designated practical examiner. The proposal also featured a requirement to use licensed mariners or individuals with comparable experience as instructors. The USCG tasked its Merchant Personnel Advisory Committee to review this proposal as an initial step in determining whether to seek implementation authority and resources. The agency's interest in formal

OCR for page 67
--> instructor certification is driven by its interest in encouraging quality training programs, concerns over instructor competency, and interest in using training and marine simulation actively within the marine licensing process. Teaching Methods Interdisciplinary Instructional Teams The instructor needs a wide range of maritime skills. Realistically, it is difficult to find all these skills and qualifications in one individual. Additional training can overcome deficiencies, and spreading of instructional skills over the entire staff can enable a training establishment to focus the required specialization to a particular training need. A full instructional team would consist of subject-matter experts supplemented by individuals with specialized instructional and technical capabilities in BOX 3-5 Samples of Instructor Training Programs The Southampton Institute, Warsash Maritime Centre, United Kingdom Full-Mission Ship-Bridge Simulator Key elements of the Maritime Simulation Instructor Training Program include the following: All candidate instructors are required to have a class one, masters or chief engineer's license and recent sea service. Training is provided in five subsections: (1) full-mission ship-bridge simulator, (2) radar and vessel traffic system simulators, (3) manned-model shiphandling, (4) machinery space simulator, and (5) cargo-handling simulator. For the full-mission bridge simulator there are five phases: 1.   Duration 5 days. Complete the Bridge Team Management (BTM) course as a student. 2.   Duration 20 days. Understudy senior lecturer. Goals for the student at the end of the period include: (1) understand operational philosophy of BTM program; (2) be familiar with bridge equipment; (3) continue to attend BTM lectures and casualty workshops; (4) understudy all BTM exercises, including exercise/planning briefings, simulator exercises, and exercise debriefings; (5) be familiar with all course administrative procedures; (6) receive instruction on the remote data station and on database and exercise construction; (7) receive instruction on the ship simulator instructor control station; (8) undertake pilotage duties on the ride of the simulator; and (9) understand all aspects of course administration.

OCR for page 67
--> 3.   Duration 15 days. Continue to understudy senior lecturer. Goals include: (1) be familiar with all aspects of exercise control in the instructor control station for the BTM course; (2) conduct all exercise briefings and debriefings for all BTM exercise; (3) control all BTM simulator exercises under supervision in the instructor control room; (4) continue familiarization in database and exercise construction; (5) be familiar with all BTM course lectures and casualty workshops; (6) begin giving selected lectures/workshops under supervision of a senior lecturer; and (7) be familiar with all aspects of course administration and undertake administrative duties. 4.   Duration ongoing through first year. (1) Independently conduct all BTM simulator exercises, (2) conduct exercise briefings and debriefings, (3) give selected lectures, (4) undertake various administrative tasks, and (5) undertake relevant pilotage duties on the bridge of the simulator. Under supervision (1) complete presentation of all lectures for the BTM course, (2) understand all aspects of the BTM course in all its versions, (3) undertake familiarization with the emergency procedures course conducted on the simulator, (4) undertake familiarization with pilot training courses on the simulator, (5) undertake familiarization with all special courses conducted on the simulator, (6) develop new exercises, and (7) assist in development of new databases. 5.   Duration ongoing updating. All full-mission ship simulator lecturers are expected to conduct at least one week's updating at sea each year. The Institute educational system allows five weeks "research and scholarly activity" per year for industrial updating, the presentation of papers at conferences, and other allied activities. In addition, all new staff members are required to complete a one-year course (at the rate of one day per week) on training techniques and the educational systems. the application of simulation and the setup and operation of computer-based simulators and manned models. In practice, the members of a simulator facility's staff are routinely called on at appropriate times during the course of instruction to support simulations through role playing, to observe student performance, and to provide specialized instruction or technical support. This practice satisfies multidisciplinary needs within the limits of the staffs' resident expertise. Sometimes specialized support is obtained from parties external to the simulator facility. For example, few facilities maintain a hydrodynamicist on staff unless the facility is also involved with channel design. There may be occasions when an expert in a nonmaritime field may have to be brought in to assist on a simulator-based course. The best example is that of bridge resource management training, where psychologists and specialist in human factors and stress and fatigue can contribute greatly to course content and presentation. The importance of highly qualified, trained, and motivated senior mariners as instructors cannot be overstated.

OCR for page 67
--> BOX 3-6 Samples of Instructor Training Programs MarineSafety International, Rotterdam The Netherlands government requires instructors to complete a formal course of instruction to prepare them for service at the new MarineSafety International Rotterdam simulator facility. Originally developed by FlightSafety International for commercial air carrier simulator instructors, the course was adapted to the marine simulator field. The one-week course was developed by the facility and was based on the parent company's earlier development of flight simulator instructor training. The course consists of 17 hours of classroom instruction in varying formats plus 2 days of training in the use of the facility's simulators. The classroom segment of the course includes lessons on: principles of teaching and learning, lesson planning, student-instructor relations, effective communication, oral questioning techniques, and the profession of instructing. NOTE: FlightSafety International is the parent company of Marine Safety International (MSI), which operates marine simulator facilities at Kings Point, New York, and Newport, Rhode Island. MSI is a partner with the Port of Rotterdam in MarineSafety International Rotterdam. Lead Instructors Use of Lead Instructors. Use of lead instructors has been possible because of small class sizes and the individual lesson content. As a practical matter, the content of each exercise or drill that can be effectively overseen by a single instructor more or less coincides with the level of detail and interaction that trainees can accommodate. On the other hand, the use of a single instructor rather than a dedicated, multidisciplinary instructional team is often influenced by cost. If a facility can afford only a single instructor or a small instructional staff, then the emphasis will be on nautical credibility rather than on staff instructional skills and proficiency. These factors may or may not result in optimization of either instruction or learning, depending on all factors present in a given simulation. Criteria for Ideal Instructor. The committee believes that the ideal lead instructor should have the following skills and qualifications: possession of an unlimited master's license or other high-level qualification for specialized training—for example, a license as a marine pilot for pilot training;

OCR for page 67
--> command experience; demonstrated effective teaching and communication skills; knowledge of the simulator capabilities; expert shiphandling skills; strong analytical capabilities; and current general knowledge of the industry and trainee sector in particular. Many trainees attending simulator courses are either serving masters in command or senior officers. It is desirable for nautical credibility that the lead instructor's professional qualification be at least the same as the highest qualification for which the trainees are being trained or examined. Perhaps more important, however, the instructor should possess appropriate subject-matter expertise (i.e., if the course is in pilotage, the instructor should be an expert in pilotage). Command experience would be an advantage and is desirable, but is not absolutely necessary. Many U.S. training establishments provide training for deck officers and vessel operators other than, or in addition to, masters and pilots. For example, some facilities provide training for coastwise tug and barge operations, and one provides training for operators of inland tug and barge flotillas. In these cases, the highest relevant mariner qualification is important. In addition to establishing credibility, the instructor and trainees must be able to comfortably relate to each other. FINDINGS Summary of Findings The current approaches to training and professional development in the marine industry are based on a tradition of "modeling-the-expert" and on-the-job training. Many courses that currently use simulation in their curricula have followed the approach of "inserting" the simulation into the training program rather than following a more structured approach to course development. Systematic application of the instructional design process offers a strong model for structuring new courses and continuously improving existing ones. The primary elements of the instructional design process include: determining training needs, including characterizing the trainee population and analyzing job-tasks and subtasks; determining specific training objectives, including performance measures to determine whether or to what degree the objectives have been met; determining training methods to be used, including assessing whether simulation is appropriate to the training objectives; developing a detailed course curriculum, including designing exercise scenarios (if simulation is used), determining the duration of the training program, and debriefing techniques;and validating the simulator, the simulation, and the curriculum.

OCR for page 67
--> Instructional design is an evolving concept. Application of the process should include periodic evaluation of the success or failure of course elements, periodic assessment of the program's overall effectiveness, and regular innovative modifications, as appropriate. Another issue of concern in the mariner training process is transfer and retention of the training. Use of simulators in training is based on subjective observation and anecdotal evidence that the training system is effective. Very little recent quantitative research has been conducted to determine whether or how effectively simulator training transfers to the work environment. One of the most critical elements in the application of instructional design is the effectiveness of the instructor. It is the instructor's responsibility to ensure that all training objectives are met. The instructor must possess both content knowledge and instructional skills, especially if he or she is responsible for teaching in a simulator environment. Standards or guidelines defining instructor qualifications are necessary to ensure instructional effectiveness. Research Needs In the course of its investigation of the uses of simulators in training and the instructional design process, the committee identified a number of areas where existing research and analysis did not provide sufficient information for the committee to extend its own analysis. Among the most significant areas identified was the need to update and expand relevant task and subtask analyses for application to the mariner's training needs. For the instructional design process to be effective, the course design should include the definition of training needs based on the steps required to complete identified tasks and subtasks for specific functions. This analysis should include dimensions that have been missing with respect to behavioral elements and specific steps needed to execute each subtask. This analysis is important for several reasons. First, not all tasks contribute in the same way to overall performance of functions and duties of the job. Second, task analysis is necessary in training course design and performance evaluation. Third, a clear understanding of the skills and abilities required for job performance is necessary for effective performance evaluation (Chapter 5). Fourth, task descriptions and related performance criteria are necessary to design an effective licensing program (Chapter 5). Other areas for research identified by the committee include: the need for a standard methodology for validating exercise scenarios; the need for guidelines or standards for qualifying or certifying training instructors; research on the optimum sequencing of simulator training; the effect of course duration (i.e., short courses that typically compress course content versus courses spread over weeks or months) on learning and transfer effectiveness by different categories of the training population;

OCR for page 67
--> a subset of the study of course duration—this would be an investigation of whether the effects differ in classroom versus simulator-based training for different categories of the training population; and a study on whether skills learned in a simulator can be employed aboard a ship. The study might employ a method such as comparing the shipboard performances of simulator-trained individuals to shipboard performances of a similar group with no simulator training. REFERENCES D'Amico, A.D., W.C. Miller, and C. Saxe. 1985. A Preliminary Evaluation of Transfer of Simulator Training to the Real-World. Report No. CAORF 50-8126-02. Kings Point, New York: Computer Aided Operations Research Facility, National Maritime Research Center. Douwsma, D.G. 1993. Using frameworks to produce cost-effective simulator training. Pp. 97–101 in MARSIM '93, International Conference on Maritime Simulation and Ship Maneuverability, St. Johns, Newfoundland, Canada, September 26–October 2. Drown, D.F., and R.M. Mercer. 1995. Applying marine simulation to improve mariner professional development. Pp. 597–608 in Proceedings of Ports '95. New York: American Society of Civil Engineers. Edmonds, D. 1994. Weighing the pros and cons of simulator training, computer-based training, and computer testing and assessment. Paper presented at IIR International Human Factors in Shipping Week 1994: Strategies for Achieving Effective Maritime Manning and Training, London, England, October 4. Flexman, R.E., S.N. Roscoe, A.C. Williams, Jr., and B.H. Williges. 1972. Studies in pilot training: the anatomy of transfer. Aviation Research Monographs 2(1). Champaign: Aviation Research Laboratory, University of Illinois. Froese, J. 1988. Can simulators be used to identify and specify training needs? Fifth International Conference on Maritime Education and Training. Sydney, Nova Scotia, Canada: The International Maritime Lecturers Association. Gynther, J.W., T.J. Hammell, J.A. Grasso, and V.M. Pittsley. 1982a. Simulators for Mariner Training and Licensing: Functional Specification and Training Program Guidelines for a Maritime Cadet Simulator. Report Nos. CAORF 50-8004-02 and USCG-D-8-83. Kings Point, New York: Computer Aided Operations Research Facility, National Maritime Research Center. Gynther, J.W., T.J. Hammell, J.A. Grasso, and V.M. Pittsley. 1982b. Simulators for Mariner Training and Licensing: Guidelines for Deck Officer Training Systems. Report Nos. CAORF 50-8004-03 and USCG-D-7-83. Kings Point, New York: Computer Aided Operations Research Facility, National Maritime Research Center. Gynther, J.W., T.J. Hammell, and V.M. Pittsley. 1985. Guidelines for Simulator-Based Marine Pilot Training Programs. Report Nos. CAORF-50-8313-02 and USCG-D-25-85. Kings Point, New York: Computer Aided Operations Research Facility, National Maritime Research Center. Hammell, T.J., K.E. Williams, J.A. Grasso, and W. Evans. 1980. Simulators for Mariner Training and Licensing. Phase 1: The Role of Simulators in the Mariner Training and Licensing Process (2 volumes). Report Nos. CAORF 50-7810-01 and USCG-D-12-80. Kings Point, New York: Computer Aided Operations Research Facility, National Maritime Research Center. Hammell, T.J., J.W. Gynther, J.A. Grasso, and M.E. Gaffney. 1981a. Simulators for Mariner Training and Licensing. Phase 2: Investigation of Simulator-Based Training for Maritime Cadets. Report Nos. CAORF 50-7915-01 and USCG-D-06-82. Kings Point, New York: Computer Aided Operations Research Facility, National Maritime Research Center. Hammell, T.J., J.W. Gynther, J.A. Grasso, and M.E. Gaffney. 1981b. Simulators for Mariner Training and Licensing. Phase 2: Investigation of Simulator Characteristics for Training Senior Mariners. Report Nos. CAORF 50-915-02 and USCG-D-08-82. Kings Point, New York: Computer Aided Operations Research Facility, National Maritime Research Center.

OCR for page 67
--> Hammell, T.J., J.W. Gynther, and V.M. Pittsley. 1985. Experimental Evaluation of Simulator-Based Training for Marine Pilots. Report No. CAORF 50-8318-03 and USCG-D-26-85. Kings Point, New York: Computer Aided Operations Research Facility, National Maritime Research Center. Hays, R.T., and M.J. Singer. 1989. Simulation Fidelity in Training System Design: Bridging the Gap Between Reality and Training. New York: Springer-Verlag. Hutchins, E. 1992. Learning to navigate. In Understanding Practice. S. Chaiklin and J. Lave, eds. New York: Cambridge University Press. Kayten, P., W.M. Korsoh, W.C. Miller, E.J. Kaufman, K.E. Williams, and T.C. King, Jr. 1982. Assessment of Simulator-Based Training for the Enhancement of Cadet Watch Officer. Kings Point, New York: National Maritime Research Center. Lave, J., and E. Wenger. 1991. Situated Learning: Legitimate Peripheral Participation. New York: Cambridge University Press. Lintern, G., and J.M. Koonce. 1992. Visual augmentation and scene detail effects in flight training. International Journal of Aviation Psychology 2:281–301. Lintern, G., K.E. Thomley-Yates, B.E. Nelson, and S.N. Roscoe. 1987. Content, variety, and augmentation of simulated visual scenes for teaching air-to-ground attack. Human Factors 29(1):45–51. Lintern, G., D.J. Sheppard, D.L. Parker, K.E. Yates, and M.D. Nolan. 1989. Simulator design and instructional features for air-to-ground attack: a transfer study. Human Factors 31(1):87–99. Lintern, G., S.N. Roscoe, and J.E. Sivier. 1990. Display principles, control dynamics, and environmental factors in pilot training and transfer. Human Factors 32:299–317. Miller, W.C., C. Saxe, and A.D. D'Amico. 1985. A Preliminary Evaluation of Transfer of Simulator Training to the Real-World. Report No. CAORF 50-8126-02. Kings Point, New York: Computer Aided Operations Research Facility, National Maritime Research Center. Multer, J., A.D. D'Amico, K. Williams, and C. Saxe. 1983. Efficiency of Simulation in the Acquisition of Shiphandling Knowledge as a Function of Previous Experience. Report No. CAORF 52-8102-02. Kings Point, New York: Computer Aided Operations Research Facility, National Maritime Research Center. NRC (National Research Council). 1991. In the Mind's Eye: Enhancing Human Performance. D. Druckman and R.A. Bjork, eds. Committee on Techniques for the Enhancement of Human Performance, Commission on Behavioral and Social Sciences and Education. Washington, D.C.: National Academy Press. NRC (National Research Council). 1994. Minding the Helm: Marine Navigation and Piloting. Committee on Advances in Navigation and Piloting. Marine Board. Washington, D.C.: National Academy Press. O'Hara, J.M., and C. Saxe. 1985. The Development, Retention, and Retraining of Deck Officer Watchstanding Skills in Maritime Cadets. Report No. CAORF 56-8418-01. Kings Point, New York: Computer Aided Operations Research Facility, National Maritime Research Center. Povenmire, H.K., and S.N. Roscoe. 1971. An evaluation of ground-based flight trainers in routine primary flight training. Human Factors 15:109–116. Povenmire, H.K., and S.N. Roscoe. 1973. Incremental transfer effectiveness of a ground-based general aviation trainer. Human Factors 15:534–542. Sanquist, T.F., J.D. Lee, and A.M. Rothblum. 1994. Cognitive Analysis of Navigation Tasks: A Tool for Training Assessment and Equipment Design. Report USCG-D-19-94. Washington, D.C.: U.S. Department of Transportation. Waag, W.L. 1981. Training Effectiveness of Visual and Motion Simulation. AFHRL-TR-79-72. Air Force Human Resources Laboratory, Brooks Air Force Base, Texas. Moffett Field, California: NASA Ames Research Center.