Two of these issues, evaluator and assessor qualifications and availability of simulators, could involve significant time. In the development of its licensing program framework, the USCG will need to include time to develop (or oversee development of) training courses and to train (or oversee training of) evaluators and assessors. The USCG will also need to factor in sufficient time to not only allow new simulator facilities to come on-line, but also to ensure that the simulators and simulations are properly validated.
One approach to developing the framework needed for the effective, widespread introduction of simulator-based licensing assessment would be to apply elements of the instructional design process discussed in the above section ''The Need for a Systematic Approach." Systematic application of this process would address many of the issues raised by the committee.
In the course of its investigation of the uses of simulation for training performance evaluation and licensing assessment, the committee identified a number of areas where existing research and analysis did not provide sufficient information to extend its analysis.
The "Research Needs" section of Chapter 3 discusses the need to expand and update specific task and subtask analyses, including data on dimensions related to behavioral elements and specific steps needed to execute each subtask. In addition to using these data in developing training courses, they are important in developing the licensing assessment program. In use of simulators as part of the licensing program, the skill and abilities to be measured must be defined.
There are also few data regarding specific requirements of a particular fleet or class of vessel. Such information would be useful in applying instructional design to the simulator-based licensing assessment process.
Another concern of the committee is that in the marine industry there is little or no research that would assist in developing performance-based criteria to be used as measurements for determining whether simulator-based licensing objective are being met.
Conducting licensing assessment using simulators at several different facilities would necessitate the development of industrywide standards for the assessment scenarios used. These standard scenarios should be based on cross-platform and cross-candidate research to ensure consistent, reproducible measures of performance.
There is also a need for cross-platform studies to determine the efficacy of specific for training evaluation or licensing performance assessment. Individual platforms may be limited in the range of performance they can simulate, and they may be more applicable to certain levels of licenses. These limitations could include such factors as number of other controllable vessels, degree of fog or reduced visibility, hydrodynamic forces (e.g., channel and bank, squat,