Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
26 Focus 3: Capacity for Training and Instructor Evaluation Focus Overview While it is commonplace for transit organizations to invest in workforce development pro- grams, only rarely do they attempt to discern whether the investment has been fruitful, if they even know how to do so. Successfully evaluating employee development programs requires structured evaluation designs, baseline data, a culture that supports evaluation, and a team that can support continuous improvements. Unfortunately, many organizations are met with obstacles. Following are some of the primary barriers transit agencies are likely to face in training evalua- tion, from difficulty developing measures that set the foundation for the evaluation to challenges resulting from a culture or leadership that may not actively value evaluation. Lack of Clear ROI Measures Organizational performance and ROI are often considered the most important metrics for training evaluation. Though financial indicators and the trainingâs impact on the bottom line can be difficult to measure directly, there are a few ways transit agencies can do so. If the focus of the training is on maintaining and/or repairing a particular type of equipment, one way to do so is to assess the amount spent replacing the equipment before and after the training. For example, the CTtransit pilot study course focused on battery system inspection and maintenance; CTtransit compared the number of batteries replaced before and after the training and the associated costs. As another example, a transit agency may choose to measure the mean distance between failures before and after the course and the resulting potential difference in revenue post-training. Although it is possible to measure financial outcomes related to training, it is still often difficult for many agencies to do so. It is more common to measure employee performance and engage- ment, which relate to these organizational outcomes. When attempting to assess training ROI, organizations should expand their focus beyond purely financial indicators to include those that enable bottom-line improvements such as quality, performance, satisfaction, and turnover. Exhibit 10 provides a number of sample evalu- ation metrics. Often, this is a major challenge for organizations, as it requires a culture shift from being interested in primarily clear bottom-line metrics that improve organizational financial measures to a more holistic focus on enabling measures. Regardless of the metric type, developing indicators that are clear and readily aligned to course and organizational goals can be challenging, whether because practitioners disagree on the focus of the measurement or because organizational goals themselves are unclear. Without clear metrics, it is difficult to attribute organizational impactâor any impact at allâto the training. Learning and development practitioners should be able to clearly link course learning objectives C H A P T E R 4 Barriers and Best Practices for Delivering Technical/ Frontline Training
Barriers and Best Practices for Delivering Technical/Frontline Training 27 to evaluation measures, and evaluation measures to intended results, including those at the organizational level. Exhibit 11 provides examples of how course objectives should align with training metrics and organizational goals and metrics. Further, consider the example of the CUTR at the University of South Florida. They measure post-training program accomplishments such as ânumber of work orders rejected for being less than acceptableâ and establishes organizational level objectives such as safety measured by âzero accidents, injuries, or safety violations.â APTA also provides guidelines for program evaluation criteria for a number of transit opera- tions and maintenance occupations. Recommended evaluation criteria typically include legal and regulatory requirements, industry standards and best practices, and local and community requirements and standards. Level 5 Training Evaluation Tool. The Level 5 Training Evaluation Tool, located in Appen- dix J, is intended to help agencies identify agency-level metrics and goals against which to mea- sure training outcomes in order to determine ROI of the training. It contains two forms transit agencies can use to do so. The first is a pre-evaluation planning tool, which enables agencies to establish evaluation metrics and identify data sources for each metric. Agencies can also use this tool to plan when data collections will take place for pre-training and post-training assessments. Exhibit 12 contains an example based on the Battery System Maintenance course developed by CTtransit for the pilot study. Metric Type Sample Training Quality Obtain participant satisfaction scores on training content of four or higher out of five within one year of training program start. Individual Performance Note number of desired behaviors observed in 3 months post-training. Organizational Performance Increase on-time performance of bus maintenance process by 10% within two years. Retention Decrease number of staff who quit or were terminated by 5% in one year. Maintain or exceed that figure for five years. Customer Service Increase customer satisfaction scores by 10% within one year. Sales Increase bus ridership by 10% over two years. Exhibit 10. Sample training evaluation metrics. Course Objective (Students should be able toâ¦) Training Evaluation Metric (Training outcomes will includeâ¦) Organizational Goal (Organization will achieveâ¦) Apply conflict resolution skills with customers dissatisfied in transit service. Reduction in number of customer complaints. Increase customer satisfaction scores by 10% within one year. Utilize problem-solving methods to overcome barriers to timely bus maintenance. Increase on-time performance of bus maintenance process by 10%. Increase bus ridership by 10% over two years. Identify colleagues who can serve as a developmental mentor or coach. Obtain participant satisfaction scores on training content of four or higher out of 5. Decrease number of staff who quit or were terminated by 5% in one year. Exhibit 11. Sample alignment of training evaluation metrics.
28 Transit Technical Training, Volume 2: Guide to Overcoming Barriers to Implementing Best and Innovative Training The second part of the Level 5 Training Evaluation Tool is a form that can be completed to carry out the actual evaluation. Exhibit 13 provides an example of how this form was adapted by CTtransit to align with the Battery System Maintenance course in the pilot study. The broad agency-level outcome of the Battery System Maintenance course was that fewer new batteries would need to be installed, as employees would have the knowledge and skills nec- essary to maintain and repair them. Therefore, evaluating the number of new batteries installed and the cost of these batteries both before and after training was one way to measure whether or not the course was effective. The Level 5 Training Evaluation Tool also includes step-by-step guidance for completing each of these tools and conducting the evaluation. Limited Engagement Post-Training To ensure successful training evaluation, it is critical to proactively create an evaluation plan that provides for the time, resources, and communications necessary to engage supervisors and employees after the training is complete. Exhibit 14 displays an evaluation plan that was developed for the CTtransit pilot, including when each will take place, how many questions rep- resenting each of the five evaluation levels will be included, and who will be required to complete it (trainee or CTtransit leadership). The most effective approach for evaluating training requires time for the employee to demonstrate new behaviors and the effect they have on an organizationâs success. Supervisors must also be engaged as they are the source that can typically best determineâwhile overseeing the employeeâs workâwhether the employee has demonstrated new skills after the training. As time goes on, however, these types of longitudinal evaluation designs naturally lose participation and evaluation becomes more difficult to complete. Data collection to support evaluation should be customized so that sufficient rigor is maintained in the data without fatiguing the participant and Exhibit 12. Level 5 pre-evaluation planning tool for CTtransit pilot study. Exhibit 13. Level 5 evaluation tool for CTtransit pilot study.
Barriers and Best Practices for Delivering Technical/Frontline Training 29 supervisor. Organizations should use a variety of techniques to obtain data post-training such as surveys, focus groups, observations, and inter- views to minimize participant time commitments while maximizing data collection opportunity. In addition, understanding the importance of evaluation and a culture of evaluation can strongly improve employee and supervisor engagement after a training session. Thus, creating a comprehensive evaluation plan prior to the administration of training, which accounts for assessment and engagement activities, is vital. Otherwise, these elements may be overlooked or underfunded. Lack of Resources for Training Evaluation With budgets regularly at risk, organizations rarely consider or accommodate evaluation needs when allocating funds for training. A perception that it is more important to deliver or offer a training than know if the training is worthwhile perpetuates this fact, though that perspective can end up putting a training program at risk. Those who use measures that demonstrate organizational results derived from training (e.g., improved retention) will help garner the support needed for evaluation. More importantly, this support, made possible from communicating evaluation results, can lead to the allocation of additional funds/resources for training programs. On the same hand, leadership support and the resources needed for training will be difficult to garner without some form of even simple or initial evaluation results that indicate positive improvement that can justify training. As such, organizations should be sure to allocate resources and establish evaluation in some capacity when first developing a training budget. A helpful Exhibit 14. CTtransit pilot study evaluation plan. Evaluation Planning Tools The Pre-Evaluation Planning Tools included as part of the Level 4 and Level 5 Training Evaluation Tools can be used to plan for post-training data collections. The tools can also be adapted to include who should be involved at which stages of evaluation (e.g., supervisors) and any communication information necessary for involvement.
30 Transit Technical Training, Volume 2: Guide to Overcoming Barriers to Implementing Best and Innovative Training method or complement to this is an evaluation plan that demonstrates how evaluationâand the resources required thereinâis or will be allocated and employed. Best practice suggests that putting an evaluation plan in place can help realize intended outcomes and ensure continuity of training. An evaluation plan is the formal, written documentation that outlines the project management (i.e., resource management) information, the questions to be answered (e.g., âCan training help improve bus maintenance efficiency?â), sources of data, and any other information regarding how the evaluation will be implemented and supported. Having these evaluation plans in place prior to beginning the training program can help practitioners keep track of scarce resources. In addition, organizations should use their evaluation plans to add credibility to the process for stakeholders such as leadership. Evaluation plans can then be expanded (and garner more resources) once justification and support are obtained from leadership. Lack of Leadership Support Somewhat unpredictably, actually having ample resources (such as personnel and time) can be less of a barrier to evaluation than not having leadership support and buy-in to evaluation. If the evaluation is demonstrating valuable insights for an organization, it is typically a cycle that can perpetuate itself. At a minimum, once evaluation supports such as metrics and forums for collecting and analyzing the data are established, maintaining evaluation programs can be done efficiently. Practitioners can re-use data collection and analytical methods and simply plug data points back into the structure without having to invest in much more. Though many organizations maintain an evaluation program like this, this approach can stunt evaluation by only providing shallow results and marginally improving training programs. A robust evaluation program that facilitates best-in-class and innovative training requires much more than funding; it requires a culture and leadership support. Aligning the training to organizational goals plays a large role in enabling this support. Practitioners should be prepared to provide evidence that the training programâand sub- sequent evaluationâwill be a key input for measuring organizational success. Leadership often is more supportive of initiatives that can help the organization reflect on where it can improve and make progress against those desired improvements. Practitioners should cascade the learn- ing objectives up to the strategies and goals of the enterprise and demonstrate how evaluation is the bridge between training and fulfillment of certain organizational goals. It is important to remember, however, that simply because training evaluation is conducted, even in preliminary forms to get an evaluation program started, positive results are not guaranteed. Practitioners should not shy away from communicating results that demonstrate gaps between the training and its intended outcomes to leadership. This is an indicator that evaluation is working and can fulfill an improvement need that otherwise would have remained unknown, which is still an outcome in which leaders should be interested. Once leadership understands the advantages that proper training evaluation can bring, they should actively work to cascade this belief throughout the organization. Leadership support and a culture that subscribes to evaluation practices are equal propellers of effective training evaluation. While leadership can provide resources, employees are the source of data and must similarly be proponents of evaluation, to help input the feedback necessary to provide the results leadership is looking for. Often, one can easily influence the other, but both are critical to establishing, maintaining, and/or expanding upon evaluation activities. Level 4 Training Evaluation Tool. The Level 4 Training Evaluation Tool, included in Appen- dix I, can help transit agencies tie training outcomes to larger organizational outcomes and, ultimately, communicate the need and anticipated benefits of the training to leadership and
Barriers and Best Practices for Delivering Technical/Frontline Training 31 stakeholders. In the event that results of the evaluation demonstrate a particular training was not effective, the results can be used to further communicate existing gaps and the need for training to improve organizational performance. If effective, the Level 4 evaluation results can help gain support for continuing or expanding training opportunities for employees in a given area. Similar to the Level 5 Training Evaluation Tool, the Level 4 tool contains a pre-evaluation planning tool to help agencies determine metrics and appropriate data collection sources and timeframes. The second part of the Level 4 Tool is a scorecard that enables agencies to carry out the actual evaluation using the metrics established. Exhibit 15 provides an example of how a transit agency might fill out this scorecard if evaluating how a training course has impacted on-time performance. The Level 4 Training Evaluation Tool also includes step-by-step guidance for conducting the evaluation. Effective and Innovative Practices While it is true there are many barriers to evaluation, often practitioners who are not familiar with best evaluation practices tend to overestimate the resources required to implement effec- tive evaluation when compared to those who use the techniques. The information contained within this section will provide workplace learning and performance professionals with greater insight into what evaluation techniques might work for their organization, the value they can potentially derive, and how they can be implemented. Armed with this insight, organizations will be able to make informed decisions on how to tailor training to address key organizational needs and desired outcomes and be able to determine the true impact of these programs on organizational success. Collaborate with Stakeholders Stakeholder buy-in and collaboration are important enablers to successful and innovative evaluation as they impact resources, organizational culture, and employee interest in evaluation. Practitioners should carefully identify partners within operations, training departments, and joint committees to assess important business measures and links to training programs. In addition, organizations should broaden their collaboration externally to seek potential partners for train- ing evaluation. For example, organizations should consider partnering with universities to help establish training evaluation and include them as part of the stakeholder group. In the case of CUTR, the center has partnered with other departments at the University of South Florida (such as their Industrial/Organizational Psychology department). In addition, others who have a stake in the outcome of the training should be considered, and their needs and interests mapped to training and evaluation design, implementation, and communications. These individuals could include supervisors, unions, and potential training participants. While they may not be collaborative sources, practitioners will equally benefit from including these groups in stakeholder buy-in activities. Exhibit 15. Level 4 Training Evaluation Tool scorecard.
32 Transit Technical Training, Volume 2: Guide to Overcoming Barriers to Implementing Best and Innovative Training Identify Sources of Measurement Data Organizations should identify the metrics to be assessed, where the data sources can be found when planning for evaluation, and whether they will be directly collected from participants and/or supervisors. Practitioners should also engage with data owners such as HR staff or technical staff with baseline data in order to enlist their support in data collection and analysis before and after training delivery. Sources of data will be highly dependent on the measurements chosen for evaluation. Possible data sources include the following: ⢠Annual performance score ratings and reviews (individual or aggregated at the departmental or organizational level) ⢠Employee and customer satisfaction ratings ⢠Human resources data such as retention rates, promotion numbers (existing and post-training), etc. ⢠Safety or compliance records ⢠Quality ratings ⢠Skill-gap analyses ⢠Supervisory behavior observations ⢠Participant self-ratings The pre-evaluation planning template included within the Level 4 and 5 Training Evaluation Tools was designed to support the identified data and data sources within CTtransit and was successfully applied during the pilot study. For example, CTtransit identified one of the Level 5 evaluation metrics for the Battery System Maintenance course as the cost per battery replaced. The pre-evaluation planning tool was then used to document where this data was located and when it would need to be collected before and after training delivery. Conduct Pre- and Post-Training Data Collection and Analysis Organizations should employ a combination of qualitative and quantitative metrics to best measure effectiveness, such as employee performance, customer satisfaction, and turnover rates. In addition, best class organizations often utilize metrics that demonstrate increased quality and may not focus on directly linking the results of training to bottom-line impacts such as profitability, market share, and sales, for example. While quantitative measures are fairly straightforward and easier to assess, qualitative measures should also be used, though they typically require a more creative evaluation design. For example, asking participants to partici- pate in observations to assess the successes and failures they encounter following training is con- sidered effective, although it is not often used. The combination of qualitative and quantitative metrics provides a fuller picture of the impact of the training. The presence of baseline metrics is an important factor in identifying the effects of the training and to legitimately claim that it created value. If learning professionals are not able to separate the effects of training and show clear evidence that reported results are due to the training, the ability to convince stakeholders that the training is a good investment becomes limited. For that reason, organizations should be sure to collect data against their metrics before training has begun, which will serve as the comparison dataset. If data infrastructure (such as data analysis expertise or data analysis procedures) does not exist within an organization, or baseline data is not present, creating such support is critical for demonstrating evidence as well as for creat- ing an evaluation culture. Organizations should first ensure this infrastructure is in place before conducting any type of data collection activities. Data collection activities should be comprised of evaluation questions or constructs that assess current skill, ability, or attitude. Evaluation questions should reflect the learning objectives of
Barriers and Best Practices for Delivering Technical/Frontline Training 33 the course and will be the source for the pre-test and post-test instruments. These questions reflect what is ultimately being tested in the evaluation. Once training is complete, organizations use the same pre-training evaluation questions and methods to collect comparison data. For example, before training begins, practitioners select a data collection technique that will test participants to determine their baseline knowledge, skills, and attitudes. Once training is com- plete, employees can be tested a second time to measure what they have learned. Practitioners may also apply alternative data collection techniques to formal tests such as interviews or verbal assessments to do the pre-test and post-test, or they may find that a combination of the two is most effective for their participants. Ideally, the pre-test and post-test occur shortly before and after training commencement and completion to more confidently assess changes due to the training. The time between testing and training might vary, though, depending on the change or measure being evaluated. In addition, practitioners may wish to assess how training concepts are applied over time, or whether the employment of newly learned behaviors changes over time. The Pre-Evaluation Planning Tools within the Level 4 and Level 5 Training Evaluation Tools can help transit agencies work through this process, as it helps agencies to determine metrics, data, data sources, data collection methods, and timeframes for data collection and analyses. Whichever data collection technique is used, organizations should center analysis of the results along the sample questions in Exhibit 16 below. Organizations should use even basic statistical evidence (such as correlations between improved employee performance and rises in departmental or organizational productivity) to demonstrate positive impact. Share and Promote Results For any evaluation to be successful, practitioners should place as much emphasis on facilitat- ing a culture around evaluation and obtaining support and buy-in from leadership as they do on the design and implementation of data collection and analysis. Strategically communicating results will help an organization to do this. Practitioners should develop and execute communications formally, in a manner that is visu- ally appealing, clear, and promotes consistency in conveying results. As part of the evaluation, organizations should develop an evaluation communication plan that includes the primary audiences, the frequency of communications, and any templates to be used. Sample Training Data Analysis Questions Is the training meeting expectations (e.g., a specific behavior change, retention improvement, etc.)? Is there evidence the training is working? Why/why not? How do I know? If the data is qualitative, are there themes evident in the data set? Can these themes be turned into recommendations for adjustments to the training? If the data is quantitative, what frequencies appear in the data? Does there seem to be a relationship between one measure and another? Does one measure seem to increase or decrease in relation to another measure increasing or decreasing (i.e., positively or negatively correlated)? Are the outlined measures still relevant to inform the evaluation questions? Exhibit 16. Sample training data analysis questions.
34 Transit Technical Training, Volume 2: Guide to Overcoming Barriers to Implementing Best and Innovative Training For outgoing promotional messages, best practice suggests that organizations should incor- porate data in communications to demonstrate the value of training to the organizationâs goals and bottom line. Practitioners should select the data points and results that are most compelling and best tell the narrative for the impact of the training. These selected results may change depending on the audience, to best achieve audience understanding and, where relevant, increase support for the training and/or evaluation. In addition, organizations should frequently update these data points so interested audiences are kept apprised of how the train- ing and evaluation are progressing. Organizations should continue to track desired behavioral indicators of improved organizational performance (such as employee retention, customer satisfaction, employee satisfaction, and individual development), and provide updates on a periodic basis. Organizations may also consider developing communication metrics to assess how effectively training results were communicated to the organization and its leadership. Metrics such as frequency of communications accessed or leadership and staff support/understanding of evaluation results, etc. should be considered. In addition, separate communications should be developed that prepare the employee and their supervisor for the expectations of the training and their role in evaluation. Before an employee begins training, practitioners should describe the method of evaluation and time required from both parties before and after training. The benefit of conducting the evaluation is also a critical element to communicate. Many employees will justifiably be concerned that the evaluation will be used to reflect upon their individual performance; if this is the case, practitioners should carefully explain how the data will be used. Organizations will be more likely to garner employee and supervisory support if it is clear that the purpose of the evaluation is primarily to ensure they both get what they seek out of training participation. For the participant, this will likely be individual growth in the depth or breadth of their skills or abilities that will improve their job experience or provide avenues for professional lateral or upward growth. Similar to the participant, the supervisor will likely want to see skill and ability improvement that will positively influence job performance. The key message for both audiences is that a training evaluation is the only way to know if the training content and delivery contribute to the desired change. Empower Supervisors Because they often have a stake in job performance improvement, supervisors play a critical role in any successful training evaluation and should be a primary source and champion of data collection before and after training. Organizations should consider supervisors as stakeholders in the evaluation, who will benefit from the evaluation results and improved training of their staff. As with other stakeholders, supervisors will have unique interests and concerns with the evaluation. If they are unfamiliar with training evaluation, they are more likely to overestimate the timing and barriers to assisting with training evaluation. Practitioners should closely engage with supervisors to demonstrate the value of evaluation and obtain their feedback and observance of their role in the training evaluation. Supervisors will play an important role in the successful evaluation and should be tasked with ⢠encouraging staff to partake in the evaluation activities associated with their training; ⢠monitoring and signing off on trained behaviors; ⢠participating in pre-training data collection as relevant; ⢠responding to staff concerns regarding evaluation such as performance review impacts; and ⢠communicating training results to leadership where appropriate. Organizations should supervisors as partners in the effort to gather data and ensure participation from employees.
Barriers and Best Practices for Delivering Technical/Frontline Training 35 Improve Instructor Selection Process Organizations should select instructors with a skillset that balances technical expertise with instruction or knowledge transfer capabilities. In terms of technical expertise, trainers or facilita- tors of the training should have a great deal of technical experience and background knowledge in the area they are training. At the same time, the instructor should have an understanding of adult learning principles and know how to best provide knowledge transfer. According to the Transportation Learning Center (2015), the most qualified trainer is someone with technical experience, college education in teaching methods, a training background, and extensive work experience. Instructors with experience who can provide real-world examples in the technical area can insert these anecdotes into the training to create a more realistic and relatable lesson. Other instructor qualities that should be considered include interpersonal skills and the ability to adapt to the needs of students. For example, the United States Army found that an instructor that has the ability to adjust the training based on the traineesâ knowledge and accomplishments and to fit the time constraints of the schedule is seen as effective because they are able to keep their students engaged. If a group of trainees is more advanced than other training groups have been in the past, a good instructor will alter the lesson by glossing over concepts and techniques that the group is already comfortable with and spend more time on new or more advanced topics. An effective trainer creates an agile environment to keep their students engaged (Cobb, Schaefer, Stallings, Blankenbeckler, & Wampler, 2014). Finally, the composition of an organizationâs instructors should reflect its workforce, as insufficient diversity for training delivery could be a barrier. For example, not including both men and women instructors could validate myths regarding women and transportation careers that deal with physical ability, demands, and earning potential (Shaffer, 2015). This issue could apply to other demographic variables as well. Provide Instructors with Feedback It is not sufficient, however, to simply have a cadre of well-balanced and training-experienced instructors. Feedback for those instructors is an important component of continually improv- ing training and better achieving training impact goals. The instructors themselves should be considered an element of the training design that will also contribute to its effectiveness. Level 1 training evaluation is the prominent source to gather instructor feedback from partici- pants. However, organizations should use multiple instructor evaluation methods (e.g., participant survey, test results) to assess instructor capabilities and identify areas of improvement. As stakeholders in the training evaluation, practitioners should partner with their instructors to identify opportunities for improvement. Part of the evaluation design should be communi- cating the results of the entire training evaluation, not solely the instructor component. Training design practitioners also should collaborate with the instructor to interpret the results and iden- tify what insights might be used for training modification. Organizations should engage with the instructor to appraise results by asking questions, such as any of the following: ⢠What associations or alignments can be made between the instructor evaluation and overall training evaluation? ⢠What specific changes should be made to training design or content? ⢠What specific changes should be made to training delivery? Using training evaluation results to continuously improve the training is an evaluation tech- nique that is frequently used and considered effective by best-in-class organizations; instructors can be effective partners to do so.
36 Transit Technical Training, Volume 2: Guide to Overcoming Barriers to Implementing Best and Innovative Training Focus 4: Access to Innovative Training Technology Focus Overview Technological development over the past two decades has enabled a revolution in the way that training is designed, developed, delivered, and evaluated. As reflected in TCRP Research Report 199, Transit Technical Training, Volume 1: Guide to Applying Best Practices and Sharing Resources, a wide array of innovative training technologies such as computer/web-based train- ing, simulations, virtual worlds, and video games can support more effective training at lower long-term cost and with high levels of trainee interaction, knowledge gain, and skill develop- ment. Although there are examples of cutting-edge and innovative training technologies in the transit industry, the industry, in general, has been slow overall to adopt, especially at small, suburban, or rural agencies. There are several reasons that adoption of advanced training technologies has been sluggish. Cost is certainly a major factor. Much of the cost for legacy training infrastructure and equipment has already been paid, whereas new training technologies can require significant upfront develop- ment and acquisition costs. Even methods designed to reduce infrastructure requirementsâlike CBTâcan be difficult to implement if agencies do not have sufficient computers for employees to use. Constrained budgets and risk-averse management can make it difficult to justify invest- ment in these technologies when the benefits may only be evident after many years. Industry NRTAPâeLearning and Classroom Materials The National Rural Transit Assistance Program recognizes that rural transit organizations do not always have access to high quality eLearning content nor the computer systems to support it. Thus, much of the training content is adaptable to both online and classroom learning environments. For example, an interactive version of their START (Safety Training and Rural Transit) can be taken for free online, or the course materials can be downloaded and printed for delivery in a classroom.
Barriers and Best Practices for Delivering Technical/Frontline Training 37 professionals are sometimes concerned that certain training technologies like computer- or web- based training are not conducive to the hands-on nature of technical and frontline transit jobs. Even those that are in favor of implementing such technologies find that the content is relatively low quality and not sufficiently engaging for their technical and frontline workforce. This may be the product of legacy CBT programs that do not practice modern instruction design principles and that simply involve reading or listening to instruction and clicking through slides. Although higher quality content exists in the marketplace, it is scattered across numerous different sources, and it can be difficult for training leaders to identify what is available, applicable to their workforce, of good quality, and of good value compared to other offerings or developing training in-house. Effective and Innovative Practices Blended Learning Blended learning represents an integration of computer- or web-based training methods and more traditional classroom-based, hands-on, and instructor-led training. It represents a viable approach for transit agencies to gradually and selectively incorporate more modern training technology without the significant upfront costs or other drawbacks associated with fully adapting their training programs to digital media. In this way, agencies can take advantage of digital media where it is most practical (e.g., presenting introductory material or videos on how equipment operates), while utilizing the limited classroom space and instructor/trainee time more efficiently by focusing on hands-on experience, skill demonstration, experiential learning (e.g., maneuvering the vehicle around a course), and feedback from the instructor to correct errors. It also addresses the challenges of scheduling several participants for lengthy classroom training as the computer and/or web-enabled portions of the course can be taken when it is most convenient for the individual employees, while the classroom time would be more limited and could be more easily scheduled around employee shifts or peak periods. Another example of how to selectively incorporate technology along with traditional train- ing methods is the use of simulations. Simulations can allow additional practice to replicate the experience of working on real equipment and can even engage learners in lifelike scenarios and situations, depending on the level of fidelity of the simulation. In transit, aviation, the military, and other contexts, simulations have been found to improve technical and functional expertise, problem-solving and decision-making skills, and interpersonal and communications skills or team-based competencies. For transit agencies, simulations could range from simulated electrical systems that allow trainees to troubleshoot different scenarios to virtual bus simulators that enable operators to practice emergency braking or maneuvering scenarios (among countless others). Although the upfront cost of simulations can be substantial, it can often be rolled into the cost of buying the vehicles and partially offset with federal funds. Moreover, simulations do not require the purchase of additional vehicles or for operating vehicles to be taken out of service for training purposes. In addition to delivering training in a blended fashion, agencies should consider the benefits of blended approaches to trainee assess- ment. For example, basic technical knowledge (e.g., names/functions/ diagrams of parts or systems) can be assessed through knowledge checks embedded in CBT, while skills and procedures (e.g., diagnosing a mechanical issue, operating a wheelchair lift) can be assessed by an instructor or supervisor while performed on the actual equipment. Blended Learning Checklist. The Blended Learning Check- list, located in Appendix D, is intended to assist organizations and WMATAâVehicle Maintenance Simulation Equipment WMATAâs training facility includes hands- on simulation equipment for virtually every system on their buses, and they operate almost identically to the real thing, with the option for different troubleshooting scenarios. This allows practice of activities such as repairing engines, identifying electrical faults, lo- cating HVAC leaks, and replacing brakes without having to take equipment out of service.
38 Transit Technical Training, Volume 2: Guide to Overcoming Barriers to Implementing Best and Innovative Training facilitators in the implementation of a blended course before, during, and after course delivery. The items in the checklist address the challenges and considerations for the organization to keep in mind when implementing blended learning. The organization should walk through every line of the checklist to ensure they can answer âyesâ to each. If they do not know the answer to one of the items, they need to stop and examine the requirements to see if they meet it before moving forward. If the organization answers ânoâ to any of the checklist items, this is an indicator that the blended course may not be ready to be implemented, it may not be as successful as the orga- nization desires, and/or they may not get the intended results. Once all items are checked off, the organization can feel confident in their implementation of the blended course. Utilize Shared/Centralized Transit Training Resources Unlike many businesses that compete in a national marketplace, transit is a public service and agencies are localized and thus do not compete with each other for the most part. Given this fact, it is often inefficient for each transit organization to develop technical/frontline training entirely on their own. Although circumstances differ, there are a limited number of equipment manu- facturers and the skills needed to operate that equipment are virtually the same, so it would be much more cost effective to purchase training off-the-shelf, customize or adapt existing training, or partner with other transit organizations to develop training based on common needs. The barriers and best practices related to shared-training models will be covered in more depth in the next chapter, but here it is important to recognize the specific benefits regarding incorporating technology. Specifically, eLearning solutions do not depend on having qualified instructors on hand or equipment to practice on. Thus, content can be stored in a centralized location with electronic access provided via the web. Given that the content can be used at virtually any agency regard- less of whether they have a large training facility, it is an affordable way for smaller agencies to access quality content without a large upfront investment. At the same time, if the training could be monetized and marketed, there is an incentive for larger agencies and training developers to pool resources to develop quality eLearning content. In short, the broader potential market for eLearning content makes it uniquely suited for inclusion in a central transit training resource.
