Conclusions and Recommendations
Simulation has been used to train marine since the l960s. Applied properly, simulators can go beyond the traditional test of knowledge to testing and assessing skills and abilities. Recent concerns about marine casualties and mariner competence and proficiency have lead the U.S. Coast Guard (USCG) to investigate the possibility of increased use of simulators in the programs under its jurisdiction.
The Committee on Ship-Bridge Simulation Training found that simulation can be an effective training tool, especially in bridge team management and bridge resource management, shiphandling, docking and undocking evolutions, bridge watchkeeping, rules of the road, and emergency procedures. Simulation offers the USCG an opportunity to determine whether mariners are competent in a much more comprehensive way.
Although there are not sufficient data to assess the full value or impact the use of simulators has had in altering or improving mariner performance, there is sufficient experience to warrant its continued and even expanded use. However, for the USCG to use simulation effectively for training and licensing, it is important that a stronger research base be developed and that the agency address issues of standardization and validation discussed in this report. The committee's conclusions and recommendations provide a technical framework for expanding the use of ship-bridge simulation for mariner training, evaluation, and licensing assessment.
USE OF SIMULATORS FOR TRAINING
Setting Standards for Simulator-Based Training Courses
The use of simulators as a tool for training mariners is increasingly accepted within the marine industry. In many cases, simulation has been added to existing training programs without substantial course redesign to ensure that the simulation contributes effectively to training course objectives. One result has been a lack of standardization in simulator-based courses.
The USCG plays a role in simulator-based mariner training, in part, through its incremental decisions to allow substitution of specific simulator-based training for sea-service requirements. To ensure that current mariner competency levels are maintained and improved, it is important that standards for simulator-based training courses used in licensing and remission of sea time be developed as soon as possible.
Recommendation 1: Marine simulation should be used in conjunction with other training methodologies during routine training, including cadet training at the maritime academies, for the development and qualification of professional mariner knowledge and skills.
Recommendation 2: The U.S. Coast Guard should oversee and guide the establishment of nationally applied standards for all simulator-based training courses within its jurisdiction. Standards development should include consultation with, and perhaps use of, outside expertise available in existing advisory committees, technical groups, forums, or special oversight boards. If the USGC relies on outside bodies, the process should be open and include interdisciplinary consultation with professional marine, trade, labor, and management organizations; federal advisory committees; professional marine pilot organizations; and marine educators, including state and federal maritime academies. Whatever process the USCG choose to use should be acceptable from a regulatory standpoint.
Specific elements of the training standards should address:
- identifying training objectives;
- developing standard course syllabi;
- establishing instructor qualifications and certification (see Recommendation 3);
- codifying procedures for teaching watchkeeping, bridge team and bridge resource management, shiphandling, emergency response, and other fundamental tasks and skills;
- creating student evaluation and assessment methodologies; and
- structuring industrywide research and training program effectiveness measures.
Training courses in the USCG licensing and sea-time remission program should be required to meet the resultant training course standards. These standards are urgently needed and should be developed without delay. To guide the rapidly evolving state of marine simulator practice effectively, these standards should be developed within two years.
Recommendation 3: U.S. licensing authorities should require that instructors of simulator-based training courses used for formal licensing assessment, licensing renewal, and training for required certifications (i.e., liquid natural gas carrier watchstander, offshore oil port mooring masters) be professionally competent with respect to relevant nautical expertise, the licensing process, and training methods. The professional qualifications of the lead instructor should be at least the same as the highest qualification for which trainees are being trained or examined. Criteria and standards for instructor qualification should be developed and procedures set in place for certifying and periodically recertifying instructors who conduct training.
Use of Simulators to Promote Continuing Professional Development
Continuing professional development is an important element in acquiring and retaining knowledge and skills. Simulators can be extremely useful to deck officers and pilots for renewing and refreshing existing skills and acquiring new skills through exposure to new technologies and operational scenarios. Ship-bridge simulators can be effective for both deck officer and pilot training in bridge team management, bridge resource management, and some shiphandling, although current computer-based simulators are limited in ability to simulate ships' maneuvering trajectories in shallow and restricted waterways and ship-to-ship interactions—capabilities important to pilot shiphandling training.
Because of the increased use of U.S. waters by foreign-flag vessels, the USCG should extend its concern for mariner professional development beyond the United States. The International Maritime Organization (IMO) is currently revising the Standards for Training, Certification, and Watchkeeping guidelines, including development of guidance for use of simulators. As a representative to the IMO, the USCG has a role in international professional development.
Recommendation 4: Marine pilotage authorities and companies retaining pilot services should encourage marine pilots, docking masters, and mooring masters who have not participated in an accredited ship-bridge simulator or manned-model course to do so as an element of continuing professional development. Marine pilot organizations, including the American Pilots' Association and state commissions, boards, and associations, should, in cooperation with companies retaining pilot services, establish programs to implement this recommendation.
Recommendation 5: Use of simulators for professional development should be implemented on an international scale to enhance the professional development of all mariners who operate vessels entering U.S. waters and to reduce the potential for accidents. The USCG should advocate this strategy in its representation of marine safety interests to the IMO and other appropriate international bodies.
There are a numbers of applications where special-task or microcomputer simulators could be used effectively in mariner training. One of the primary limitations affecting the widespread use of microcomputers is the limited availability of desktop simulations and interactive courseware. Lack of readily available interactive courseware constrains the training community's ability to provide comprehensive training using modern training media.
Recommendation 6: The U.S. Department of Transportation should selectively sponsor development of interactive courseware with embedded simulations that would facilitate the understanding of information and concepts that are difficult or costly to convey by conventional means.
USE OF SIMULATORS IN THE USCG LICENSING PROGRAM
Need for a Plan to Use Simulators Effectively
There is a need for more-comprehensive, performance-based assessments of professional qualifications in the licensing of mariners. Continual reliance on structured, objective, multiple-choice examinations for determining that an individual has achieved a defined level of professional competency has four significant weaknesses:
- The process does not include a mechanism for assessing the candidate's ability to apply knowledge (i.e.,no effective measurement of skills and abilities).
- The relevancy of the content of the multiple-choice examination to the real life duties and responsibilities of a deck officer is limited.
- The present testing methodology does not test the officer's ability to prioritize tasks in response to dynamic, real-world situations, which ships' officers must do in the normal course of work aboard ship.
- Written examinations test skills sequentially, one at a time, so the officer's ability to perform several tasks simultaneously, as is routinely required aboard ship, cannot be determined.
Full-mission ship-bridge simulators offer the USCG an opportunity to make significant improvements in its mariner licensing program. Simulation can
become an effective tool that goes beyond the traditional tests of knowledge to evaluate and assess mariner performance—competency, skills, and abilities. If simulation is used more in structured licensing programs, the need for a technical framework to incorporate it into licensing and other structured assessment programs grows more urgent.
Recommendation 7: The U.S. Coast Guard should develop a detailed plan to restructure its marine licensing program to incorporate simulation into the program and to use simulation as a basis of other structured assessments. Based on the results of the research detailed in Recommendation 16 (below), the program should include (1) identification of sections of marine licenses appropriate for assessment through simulators and (2) a methodology for regular assessment of program effectiveness in ensuring that mariner competency and safety levels are maintained. In developing the plan, the agency should consult with all parties of interest. Program objectives should include the following:
- improve the assessment of candidates for licenses,
- identify deficiencies in individual qualifications, and
- stimulate correction of deficiencies through supplemental preparation.
To establish the foundation for broad application of marine simulation in the licensing process, the USCG should:
- evaluate the examination subject-matter based on a complete job-task analysis (see Recommendation 16);
- develop performance measures that can be effectively measured through simulation;
- develop a methodology for assessing license candidates using simulation, including an indication of the level of simulation needed for each level of license;
- develop standards for the validation of simulators and simulation scenarios (see Recommendations 14 and 15);
- develop professional standards and procedures for certifying and recertifying license assessors (see Recommendation 8);
- establish a mechanism to document and evaluate results; and
- use results of the evaluations to improve the program.
Recommendation 8: Licensing authorities should require that license assessors of simulator-based licensing examinations be professionally competent with respect to relevant nautical expertise, the licensing process, and assessment methods. Assessors should hold a marine license at least equal to the highest qualification for which the candidate is being tested or should be a recognized expert in a specialized skill being trained. Specific criteria and standards for assessor qualification should be developed, and procedures should be set in place for certifying and periodically recertifying assessors who conduct licensing assessments
with simulators. The standards that result will require experience and credentials found only in the most seasoned members of the marine industry. It is not likely that people with that level of maritime experience will be found in the government. The USCG will probably need to look to professional groups and outside contractors for its license assessors.
Substitution of Simulator Training for Required Sea Service
The USCG program for granting sea-time remission has evolved ad hoc, not through a systematic technical analysis. An assessment of its sea-time remission policies should be included in the agency's plans for restructuring its licensing program. The previously noted need for standards for simulator-based training courses should be considered in developing a plan to allow substitution of simulator-based training for required sea time in cases where the committee finds such remission to be suitable.
The plan should include recognition that simulator-based training promotes the acquisition of skills and abilities in a safe environment using the repetition of scenarios to promote learning. In the case of third mate candidates, for example, structured simulator training can effectively replace some portion of the unlicensed sea-time requirement. For third mate candidates, the work on a modern ship undertaken during sea time often does not have much relevance to any of the work done by a third mate. For the cadet, that time might be better used to gain the requisite bridge expertise through a formalized program that combines structured simulator-based bridge team management and watchkeeping courses with an appropriate time at sea.
For license renewal without increase in grade, substituting simulator time for recent sea time can raise the standard of professional competence. For example, current USCG renewal practices grant credit for experience that is not directly relevant to the license. In some cases, that experience may have been gained in loosely affiliated industries and may not include any actual underway service or bridge watches. Substitution of simulator time would ensure that some minimum level of direct operational experience had been met at the time of renewal.
In addition, by its nature, reliance on ad hoc professional development during sea service does not ensure that an individual has received the necessary exposure or has learned from experience. Using an appropriately structured marine simulator-based training course that includes rules-of-the road training, bridge resource management, passage planning, emergency procedures, and watchkeeping practices could provide an alternative method to measure whether continued competence has been achieved in these areas prior to license renewal.
The use of simulation has the potential to provide higher-quality preparation than the existing recency requirements. Any course intended to provide recency
requirements needs to provide sufficient time, because learning occurs through repetition. A measurable understanding of retraining and refresher requirements necessary to maintain or restore professional competence to acceptable levels is needed.
The USCG currently grants remission of sea time for license upgrades for approved full-mission simulator-based training courses. Substitution of simultator-based training for upgrade to second mate, chief mate, or master, or for any type of pilot's license should not be considered until adequate research is available to demonstrate that simulator-based training can replace onboard experience and skills required for license advancement without degradation of mariner competency and the safety of the vessel.
Recommendation 9: The U.S. Coast Guard should grant remission of sea time for the third mate's license for graduates of an accredited, professional development program that includes bridge watchkeeping simulation. The ratio of simulator time to sea time should be determined on a course-by-course basis and should depend on the quality of the learning experience as it applies to prospective third mates, including the degree to which the learning transfers to actual operations. Research to establish a more formalized basis for these determinations should be implemented without delay (Recommendation 16).
Recommendation 10: The U.S. Coast Guard should establish standards for the use of marine simulation as an alternative to sea service for recency requirements for license renewal of deck officers and vessel operators. Remission of sea time should be granted for renewal purposes to individuals who have successfully completed an accredited and USCG-approved simulator-based training course designed for this purpose. The course should be of sufficient length and depth and include rules-of-the-road training, bridge team and bridge resource management, and passage planning. The ratio of simulator time to sea time should be determined on a course-by-course basis and should depend on the quality of the overall learning experience insofar as this learning transfers effectively to actual operations.
Use of Simulator-Based Training During License Renewal
Active mariners can benefit from structured training at each license level, and inactive marines can refresh vital skills in a relatively short time through simulator-based training. In both cases, simulations can play an important role by ensuring that applicants demonstrate some level of baseline competence at the time of license renewal.
Current operating practices in many segments of the U.S. merchant marine do not routinely provide adequate opportunity for chief mates to acquire essential shiphandling and bridge team and bridge resource management experience and expertise, much less proficiency. Development of these skills is very important
prior to receipt of a master's license and service as master. The use of manned-model and computer-based simulation can increase the chief mate's professional development by supplementing on-the-job experience and training with structured shiphandling and bridge resource management experience.
Recommendation 11: Deck officers and licensed operators of oceangoing and coastwise vessels who can demonstrate recent shipboard or related experience, but who have not complete an accredited simulator-based training course, should be encouraged to completed an accredited simulator-based bridge resource or bridge team management course before their license renewal. Those seeking to renew licenses who cannot demonstrate recent shipboard experience should be required to complete such a course before returning to sea service under that license.
Recommendation 12: Chief mates should be required to successfully complete an accredited hands-on shiphandling course prior to their first assignment as master. The license should be endorsed to certify that training has been successfully completed. Either a manned-model or a computer-based, accredited shiphandling simulations course should be established as the norm for this training.
Recommendation 13: Currently serving masters who have not completed an accredited shiphandling simulation course should be required to do so prior to their next license renewal. In addition, masters should be encouraged to attend an accredited shiphandling simulator-based training course periodically thereafter.
VALIDATION OF SIMULATIONS AND SIMULATORS
The effectiveness of ship-bridge simulation in training and evaluation students may be influenced by:
- the level and quality of courseware,
- the quality of simulation of the physical environment, and
- the ability of the simulator to effectively replicate vessel maneuvering behavior.
There are currently no standard criteria for accrediting simulators and simulator-based marine licensing programs. For simulation used in licensing and license-related training to be fully effective, it is important that there be industry-wide standards. It is currently common practice to make ad hoc adjustments to simulators and simulations. This practice leads to an absence of standardized ship models, simulators, and other simulation components. It is imperative that there be regular, critical analyses of all simulators and simulations intended to serve as examination or prerequisite training platforms for marine licenses to ensure that their degree of difficulty is equivalent.
Recommendation 14: The U.S. Coast Guard should enlist the assistance of standards-setting and other interested organizations and sponsor and support a structured process for validating and revalidating simulators, simulations, and assessment processes. In developing these standards, all parties of interest should
be consulted. The validation process should include objective, subjective, and behavioral assessments. The objective validation should include quantifiable factors; the subjective and behavioral validations should be performed by an impartial, multidisciplinary team consisting, for example, of (1) the simulation instructor, (2) a representative of the licensing authority, (3) a subject-matter expert, and (4) an expert to validate ship-model behavior.
Insofar as practical, the process should require the use of standardized mathematical ship models and operational scenarios and databases to ensure consistency within the process and across facility platforms. Once a simulator, simulation, or assessment process has been accredited for use in licensing, recertification should be required for any significant adjustments that are made.
Recommendation 15: Staff at simulator facilities should have objective knowledge of the capabilities and limitations of the hydrodynamic models on which their simulations are based. Modifications of the coefficients to address real or perceived deficiencies should only be performed based on competent oversight by a multidisciplinary team. Procedures should be developed to ensure that such changes are documented and that notification is given to the original vendors of the data and to the cognizant authorities.
RESEARCH NEEDED TO IMPROVE MARINER TRAINING, LICENSING, AND PROFESSIONAL DEVELOPMENT
Development of a Quantifiable Basis for Assessing Simulator Effectiveness
Understanding the equivalency of simulation to real life experience and on-the-job training is an important element in developing any comprehensive program for using simulators. Limited anecdotal evidence from the marine sector and from the commercial air carrier industry suggests that well-formed programs that combine instruction, simulation, team interaction, and debriefings can be used in a standard environment under quality-controlled conditions to develop theoretical and practical knowledge, as well as procedural and cognitive skills.
There are, however, insufficient data available that assess whether there has been effective application of the knowledge and skills derived from simulator-based training. There are also virtually no feedback mechanisms for assessing performance of past trainees, other than sporadic anecdotal reports.
The exact nature of the equivalency of simulation to real life has not been systematically investigated for several reasons:
- the existing job-task analyses are not adequate for this purpose,
- there has not been any systematic application of job-task analyses in either marine training or licensing for this purpose, and
- no systematic program currently exists to collect and analyze performance data for past participants in simulations.
Recommendation 16: The U.S. Coast Guard and the U.S. Maritime Administration, in consultation with maritime educators, the marine industry, and the piloting profession, should sponsor a cooperative research program to establish a quantifiable basis for measuring the effectiveness of simulator-based training. The research program should:
- document anecdotal reports of simulator-based training effectiveness to improve mariner performance and reduction of risk;
- acquire, update, and validate existing job-task analyses to be used in the establishment of experiential education and training requirements for professional development;
- analyze the degradation rate of specific knowledge and skills during actual service and while awaiting a billet;
- assess the transfer effectiveness of simulator-based training to actual operations;
- define the capabilities of full-mission, limited-task, and desktop simulators; and
- correlate the capabilities of each simulator type with job-task analyses to define the appropriate application of simulation to each job-task in training and licensing.
With respect to pilot development, the research program should take advantage of existing pilot development programs.
Simulation of the Physical Environment
Ship control and navigation are visually supported tasks, especially in confined areas. Learning visual skills is an important process in developing proficiency in control and navigation. The relative brightness and size of lights are particularly important in night simulations when they may be the only useable visual cue for judging distance, speed, and location relative to shoals, shore objects, and other vessels. Distance judgments are also important visual skill. In many simulators, the visual simulations are provided with a raster or projection technology. Raster systems have limited capabilities to represent bright lights, and projection systems result in distortion of distance perceptions as an observer moves around the simulated bridge.
In addition to visual systems, vibration, sound, and physical movement of the bridge in roll, heave, and pitch are ship operational characteristics that must be addressed by the bridge crew. The impact on training effectiveness of simulating these ship characteristics has not been verified.
Recommendation 17: The U.S. Department of Transportation (DOT) and the maritime industry should assess the impact on training effectiveness of apparent limitations in simulator visual systems. If these limitations have a negative
impact on training effectiveness, DOT should encourage development of visual systems that overcome or minimize the negative aspects of current systems.
Recommendation 18: The U.S. Department of Transportation should undertake structured assessments of the need for simulation of vibration, sound, and physical movement. These assessments should include consideration of the possibly differential value of these various sources of information in different types of training scenarios.
In contrast to ship-bridge and radar simulators, manned models are a means of simulating ship motions and shiphandling in fast time. Manned models are an effective training device for illustrating and emphasizing the shiphandling principles. They are particularly effective in providing hands-on ship maneuvering in confined waters, including berthing, unberthing, and channel work. Manned models can simulate more-realistic representations of bank effects, shallow water, and ship-to-ship interactions than electronic, computer-driven ship-bridge simulators.
There are currently three operating manned-model training facilities world-wide, one each in England, France, and Poland. The fourth, a U.S. Navy facility in the United States at Little Creek, Virginia, was recently closed. During its operating life, the U.S. Navy's manned-model facility provided opportunities for hands-on shiphandling training and instructor experience for some merchant mariners in conjunction with U.S. Naval Reserve service. Training at the facility also exposed some USCG junior officers to the maneuverability of tankers and cargo ships. Recent closure of this facility represents the loss to the United States of a unique training resource. Although the contribution of this facility to operational safety of commercial vessels was a side benefit, it nevertheless filled a gap in U.S.-based training resources for the development of merchant mariners.
Recommendation 19: Because there are no manned-model training facilities in the United States, and because of the usefulness of these models in familiarizing pilots and others with important aspects of shiphandling, DOT should study the feasibility of establishing or re-establishing a manned-model shiphandling training facility in the United States, to be operated on a user-fee basis.
Vessel Maneuvering Behavior
The ability of a simulator to closely replicate a ship's maneuvering trajectory is a strong measure of the usefulness and value of the simulator for training and licensing. At present, computer-based simulation of a ship's maneuvering trajectory is well developed in normal deep-water, open-ocean cases. In cases
involving shallow or restricted waterways, ship-to-ship interactions, and extreme maneuvers, fidelity may be significantly reduced.
Recommendation 20: The American Towing Tank Conference (ATTC) and International Towing Tank Conference (ITTC) should be advised of the needs to extend the database of ship maneuvering coefficients. ATTC and ITTC should be encouraged to investigate possible development of procedures that would allow the exploitation of existing proprietary data without source disclosure. Where data are not available from these sources, funds should be allocated to perform new tests, especially in very shallow water and in close proximity to channel boundaries or other vessels.
Improvement in Mathematical Models
The current practice for developing mathematical models for simulators is based on extrapolation of hydrodynamic coefficients from towing-tank tests for a restricted set of hull shapes. This practice may degrade the validity of the information when applied to conditions of loading and trim or to ships that differ from the model tests. In addition, the simulation of towed vessels is severely limited by the absence of systematic test data. Conducting full-scale real-ship experiments would significantly advance the state of practice in the development of mathematical models.
In general, computational methods for determining the pertinent hydrodynamic parameters based on theories offer the possibility of more general and accurate simulations, particularly for ship operations in restricted waters and ship-to-ship interactions.
Recommendation 21: The U.S. Department of Transportation should develop standards for the simulation of ship maneuvering. The fidelity of the models should be validated through a structured, objective process. Standard models should be selected and tested in towing tanks and the results compared to selected full-scale real-ship trials of the same ships to provide benchmark data for validation and testing of simulators.
Recommendation 22: The U.S. Department of Transportation should initiate research to integrate computational hydrodynamics analysis with simulators in real time.
FUNDING SIMULATOR-BASED TRAINING AND LICENSING
Specialized training on manned-model and computer-based simulators is not affordable to most individual mariners. Improvements in mariner competence and professional development possible through simulator-based training have been discussed throughout this report. Professional development is a shared
responsibility among the mariner, shipping companies, unions, port authorities and facility operators, and others. Each of these groups, as well as the general public, benefits from improved mariner competency and safety.
It is important, therefore, that any decisions to mandate training for licensing or renewal include full consideration of options and mechanisms available to ensure that the training is affordable and available to all effected mariners.
Recommendation 23: The U.S. Coast Guard and the U.S. Maritime Administration should assess the options for funding simulator-based training and licensing. The assessment should include the following options:
- allocation of some customs fees collected on marine cargos for maritime professional development,
- federal surcharges on port fees and pilotage fees,
- federal port fees,
- employer contributions,
- collective bargaining agreements,
- nonprofit foundations and other organizations, and
- individual cost sharing.