Leveraging Technology for Transportation Agency Workforce Development and Training (2017)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

37 ApproAch A review of the survey data yielded patterns and questions. For instance, what was the impetus for start- ing an ICT program, and what steps were taken to implement the program? The motivation and process used to implement the program provide guidance to other agencies seeking to establish their own pro- grams. What internal resources are used to create ICT-supported training in-house? Given the relatively small cost of these efforts ($1,000–$5,000), it was important to find out how agencies were utilizing their resources to get these projects developed and deployed. Partnership was another area of interest. Agencies frequently noted participation in partnerships to develop or deliver training. The structure of these partnerships was of interest, specifically the type of funds contributed to the partnerships, the level of involvement required, and the return that agencies received from participation. Finally, although the literature review and survey responses indicate that few agencies measure the impact of training on performance or the organization, the efforts of some agencies to start gathering such data should be explored (Laffey and Zimmerman 2015). Ultimately, it is intended that this information serve as a resource to other agencies as they consider, implement, or expand ICT-supported training. The case example information was obtained largely through e-mail and phone interviews. Representatives from a state DOT, LTAPs, the TCCC, and the Center for Training Transportation Professionals at the University of Arkansas answered a series of questions regarding their programs, which was drafted and provided back to them for review and approval. In several instances, case example participants provided additional resource information to help explain or expand upon infor- mation provided during the interview. That information is presented within this chapter. ImplementIng And developIng InformAtIon And communIcAtIon technologIes–supported trAInIng The survey results show that most agencies use ICT to deliver training. ICT is used to support train- ing not only because of the cost savings it provides (e.g., time, travel, and facilities) but also because it benefits learners. These benefits include improved learning experiences and greater acknowledg- ment of the need to balance work–life issues. However, the benefits of ICT methods versus traditional methods have yet to be proven, and most agencies in the past have favored delivery methods that promote instructor/learner interaction. That preference does not extend to ICT, for which the most common methods used are asynchronous, such as web-based and computer-based training. To find out why agencies are committed to using ICT and why they have opted for asynchronous delivery methods, representatives from the Ohio LTAP and Indiana LTAP were interviewed. In addition, because both of these agencies indicate that they use extensive in-house resources to develop training, both agencies were questioned on how they were utilizing those resources. ohio ltAp In the last couple of years, the Ohio LTAP has made a shift in how it delivers training. Previously, the LTAP offered predominantly classroom-based training. When tasked to develop training for project chapter four cAse exAmples

38 administrators, the agency decided to embark on a new approach. The new courses were developed using a combination of asynchronous web-based and classroom training. What prompted this shift? Ohio LTAP staff realized that through the use of ICT, the agency was better able to leverage its training budget. Instead of teaching knowledge-based information in the classroom, the agency moved that content to a web-based format. Learners attend shorter classroom training sessions focused on application activities, discussion, and mentoring sessions. A blended delivery model was also a better fit for the number of individuals to be trained and the time frame in which they need to be trained. To implement ICT-supported training, Ohio LTAP did several things. The agency trained target audiences on how to participate in web-based training and made sure they had sufficient techno- logical resources to complete the training. The agency made sure the training was accessible from a computer and mobile device. The agency evaluated rapid development e-learning authoring tools, eventually purchasing one to develop web-based training in-house. Finally, Ohio LTAP used stake- holder partnering (an Every Day Counts III initiative) to identify training opportunities. Ohio LTAP developed a standard process to create web-based training. First, a formal or informal needs assessment determines the need for training. A team is assembled consisting of SMEs and business owners/executive leadership to make sure there is buy-in on the project. The team out- lines and approves the content. A design plan is developed. Once approved, the training product is developed, reviewed, and piloted. Pilot audiences consist of the target audience and team members. Comments from the pilot study are used to finalize the course. Following this process, Ohio LTAP’s course developments typically take 3 to 9 months, depending on the complexity of the content. The agency is able to develop and deliver between 15 and 20 online instructional hours a year using in-house staff. In the future, Ohio LTAP plans to expand the blended delivery model to include additional audiences, such as technicians, and new content, such as equipment training. In addition, the agency plans to expand its mobile application offerings. Currently the agency has developed two smart phone applications with a third in production (http://www.dot.state.oh.us/Divisions/Planning/LocalPrograms/LTAP/Pages/ LTAP-Smart-Phone-Applications.aspx). These apps, which work in tandem with the web-based training offerings, have been well received by learners. This is yet another way for Ohio LTAP to meet learners’ needs and develop a trained workforce. Indiana ltAp Indiana LTAP delivers most of its training using traditional classroom methods. This means LTAP staff and resources were fully utilized supporting existing training and could not take on new training course deliveries without adding additional staff or hiring external trainers. This prompted the LTAP to explore alternative delivery methods to meet current training needs. The method selected was asynchronous, web-based trainings that met responsive design standards. By using this delivery method, learners are able to complete training anytime, anywhere, using any device. The first development project using this method will replace an instructor-led, classroom- based course. To develop the asynchronous, web-based products, Indiana LTAP relied on in-house and external resources. One product was purchased from a third party and customized for the agency. The other web-based training was developed by LTAP SMEs in conjunction with a Purdue University instruc- tional design graduate student. The LTAP also relied on Purdue University information technology staff, who have developed other web-based applications, to provide technical assistance and input on HTML design standards. The approximate cost for developing the training using a combination of in-house and graduate student staffers was $10,000. To support implementation of ICT delivery methods, Indiana LTAP needed to customize its personnel database to include a training module. Using the login authentication system, each local

39 employee was able to register and complete training. The system was also modified so administrators were able to track training progress and completion. Indiana LTAP anticipates it will be able to add one asynchronous, web-based course per year to its training catalog based on the current schedule and the assumption that the training budget and staff will remain as is. pArtnerIng Most state DOTs and at least a third of LTAP/TTAPs use partnering to support their training pro- grams. Partnering allows agencies to leverage limited training budgets more effectively, usually giving agencies greater purchasing power and better access to resources than could be achieved individually. The partners frequently used by agencies include the TCCC, community colleges, Clear Roads, the APWA, UTCs, FHWA, the National Transportation Training Directors organization, NHI, and the Transportation Learning Network. To explore how these partnerships work, this chapter explores interviews with a representative from the Nebraska Department of Roads (NDOR), the TCCC, and the Center for Training Transpor- tation Professionals at the University of Arkansas. The NDOR representative explains the agency’s partnership with the TCCC and how NDOR leveraged that relationship to create a blended Con- struction Inspection Certification program. The TCCC representative explains how membership is structured and the benefits members gain from participating. The representative from the Center for Training Transportation Professionals at the University of Arkansas explained how the agency’s relationship with the Arkansas State Highway and Transportation Department works. ndor Approximately 3 years ago, NDOR began development of a Construction Inspection Certification program. The program was intended to identify the construction inspection and materials testing proficiencies at each employment level so employees understood the duties associated with their role, how to develop the skills to perform those duties successfully, and what tasks needed to be mastered to advance. An in-house development team representing NDOR’s districts and Construction and Materials Departments designed the Construction Inspection Certification program to align with NDOR’s Construction and Specifications Manual. This manual is divided by discipline (e.g., administration). For each discipline, proficiencies are identified. The development team used the TCCC’s Construc- tion Matrix (http://tc3.transportation.org/training-resources/matrices/) to develop the proficiencies for each discipline. This information was further divided into four levels: prerequisite, basic, inter- mediate, and advanced. At each level for each discipline there is a list of proficiencies a technician must be able to meet to do the job. Once the proficiencies were fully documented, the development team identified training to develop and master each proficiency. NDOR’s deputy director requested that the team responsible for developing the program consider including the TCCC’s products to deliver elements of the train- ing. (NDOR was a member of the TCCC at the time.) The team reviewed the TCCC’s construction web-based training materials and aligned them with the proficiencies. They also identified which trainings were required and which were optional. Training needs that could not be met with TCCC products were sourced through other vendors or developed in-house. As a result, the Construction Inspection Certification program relies on two delivery methods: web-based training and classroom- based training. This approach provides the flexibility technicians need and the structure, application, and access to an instructor required for more complex content. At each level, there are some required TCCC courses that a technician must complete when hired or promoted. For example, “Ethics Awareness for the Transportation Industry” (https://training. transportation.org/item_details.aspx?ID=2494) is required at the basic level, whereas “Change

40 Orders, Claims, and Dispute Resolutions” (https://training.transportation.org/item_details.aspx? ID=2487) is an advanced level requirement. If a technician is promoted, he or she must complete all of the required courses at all of the earlier levels within a year if he or she has not done so already. NDOR tracks and monitors completion using the agency’s LMS. (NDOR uses the TCCC member benefit of loading courses directly to its LMS.) At this date, NDOR is almost at full implementation of the program (i.e., all technicians have fulfilled the requirements). NDOR obtains participant feedback through the LMS for each course but does not have a formal approach for evaluating content mastery. However, supervisors are encouraged to document completion of the certification program proficiencies on performance evaluations. The next phase to be implemented is a plan for continuing education. This would require construc- tion technicians to take at least three new training courses every year or re-enroll in a course taken more than 5 years ago. Technicians can retake training identified for proficiencies at their level or advanced levels. NDOR also plans to set a schedule for meetings between the Construction Inspec- tion Certification program development team and the districts to gather feedback on the courses and changes or additions to the program. In this way, the curriculum becomes a living document and always meets the needs of NDOR’s technicians. transportation curriculum coordination council The TCCC provides state highway agencies and the transportation industry with a partnering opportunity. Begun more than two decades ago as a federal pooled-fund effort and now an AASHTO Technical Service Program, the TCCC comprises member states working collaboratively to develop and deliver asynchronous, web-based technical training in maintenance, construction, and materials disciplines. Plans are in place to expand to other disciplines across AASHTO. The TCCC partnership provides a host of benefits to members. Through the combined efforts of its membership, the council offers access to a considerable catalog of training content: a state’s $20,000 investment helps to develop $300,000 worth of new web-based products annually. Membership pro- vides access to the council’s full curriculum, valued at more than $2.5 million. The 30 state highway agencies currently participating in the TCCC can: • Access more than 100 web-based training courses (https://training.transportation.org/browse_ bookstore.aspx). • Upload all TCCC courses to their state’s LMS. • Use travel scholarships to attend the TCCC annual meeting. • Propose new training course topics and help set priorities for the course development list. • Expect a minimum of 20 h of new content each year. The TCCC can develop and deliver a web-based course in 12 to 15 weeks following approval of the development project. That is a timeline that few state DOTs can achieve on their own. The council’s development process is fairly simple (see Table 16). The Curriculum Management Com- mittee identifies and approves a course for development. Base training materials are collected from member states. A volunteer team of SMEs is organized to develop the course content. The consultant formats the content into a course, tests the courseware, and loads it to the LMS. Thus, agencies have the needed training available in a timely manner. The TCCC partnership offers other benefits. The council maintains core skill matrices for maintenance, construction, traffic and safety, materials, pavement preservation, and employee development. It is currently aligning the training catalog and matrices to allow for better utiliza- tion of training products and identification of additional training needs. The council is adding job aids and other work support tools that can be deployed by the web or downloaded through a mobile application.

41 In addition, the TCCC has plans to enhance access to courses for LTAP/TTAPs and private indus- try. The council is working to establish a discount subscription service that LTAP/TTAPs and private industry can use to gain access cost-effectively to the council’s course catalog. center for training transportation professionals at the university of Arkansas The relationship between the Arkansas State Highway and Transportation Department (AHTD) and the University of Arkansas Transportation Center is about 20 years old. The Center for Training Transportation Professionals (www.cttp.org) operates under the general umbrella of the University transportation center but works independently, funded through a lump sum budget allocation from the AHTD to do training. The relationship between the training center and AHTD is built on a previous agreement between the University and AHTD to complete research and other contracts. The university proposed that it would conduct training outside of the AHTD that covered materials testing, quality control/quality assurance, aggregates, hot mix, concrete, and soils. Later, concrete strength, patching, and some NHI courses, such as on bridge scour, were added. The training center took over these responsibilities and added technology transfer training and LTAP training. The Center for Training Transportation Professionals develops online training for the AHTD. The training areas include math, specifications, and other supplemental training that help employees participate in the classroom training. Training may or may not be mandatory and require registration. The center also offers an online training course that is an alternative to the classroom-based course (concrete field testing). If a participant gets a high enough score on the quiz, he or she can bypass 2.5 days of in-class training and take the written and performance exams. The center also offers numerous web conferences for technology transfer courses. Steps Tasks Course selection Requests are sent to DOTs, AASHTO subcommittees, and regional groups for needed courses Course requests are prioritized Courses are chosen by priority and funding available Source materials/SME search DOTs, FHWA, regional groups are contacted for material availability Discussion and decision ensue on using available materials or developing via the Transportation Curriculum Coordination Council SMEs are identified for development assistance and reviews A technical panel is organized from SMEs, and a lead is identified Work plan/Design documents A work plan developed identifying tasks, resources, start and end dates, and dependencies Course outline developed as a starting point for the other design documents High-level design plan developed (approach, scope, outcomes) (optional) Detailed-level design plan (instructional strategies, media), prototype storyboards, and published prototype (1 lesson or designated module portion) developed SME reviews for all design documents Development/Testing Storyboards developed consisting of detailed PowerPoint slides, including audio scripts and media elements PowerPoint files for each module without narration published Pilot-ready PowerPoint files for each module with narration published SME reviews of content and functionality for all phases of development Implementation Final PowerPoint and audio files, media source files, resource files delivered Launch and test on LMS Final SME review and signoff for “go live” provided TABLE 16 TRANSPORTATION CURRICULUM COORDINATION COUNCIL WEB-BASED TRAINING DEvELOPMENT PROCESS

42 Every second or third year the Center for Training Transportation Professionals develops a new course. The center also develops three to four supplemental online modules per year that accompany classroom-based training and at least one new technology transfer course every year. The Center for Training Transportation Professionals follows a standard development process for identifying training needs. The center assembles a review panel made up of AHTD staff. The panel meets annually to identify content, including a review of customer input on training needs. If the train- ing need is state-driven, the center writes a proposal for development of a new class for AHTD to fund. Technology transfer web conferences (www.cttp.org/t2) and online modules are listed in an annual work plan, which lists the topics based on the budget. A design plan is created by the Center for Training Transportation Professionals with substantial input from AHTD stakeholders. In some cases, the center’s director develops the web conference through to pilot. Other times the center’s staff completes the work. During the development, the AHTD reviews the content and works with the center to finalize the training. Comments regarding the training are addressed after the pilot, although typically changes to the training are minimal after the pilot is developed. For required courses, the Center for Training Transportation Professionals administers online quizzes at the end of each training to verify the information has been mastered. Exam results indicate that ICT training content is mastered by most participants. However, satisfaction survey data indicate that certain segments of learners prefer classroom training, particularly those who do active field work. Engineers appear to be satisfied with ICT-supported training, particularly methods that are asynchronous. The partnership between the AHTD and the Center for Training Transportation Professionals provides several benefits. The AHTD has an organization focused on training employees and con- tractors. Trainings hosted by the center often are perceived as neutral environments where AHTD employees and contractors can interact and learn from one another. The information provided by the center is provided in a technically accurate way, without bias to either AHTD’s or the contractor’s way of approaching how to complete the work. This neutrality makes it easier for learners to accept and implement what is being taught. On the job, that translates into consistent practice and under- standing. This common knowledge allows employees and contractors who work together to attain a level of trust and confidence about the other’s abilities and performance that builds and sustains a positive working relationship. From the university’s perspective, it is a tremendous benefit to have this relationship with the AHTD. It connects staff to AHTD personnel and builds relationships. The relationship allows col- lege students the opportunity to participate in training and lab work and establish connections with AHTD employees and contractors. Students also get exposure to real issues in the industry. meAsurIng the ImpAct of InformAtIon And communIcAtIon technologIes–supported trAInIng Thirty-three state DOTS and 14 LTAP/TTAPs indicate that they administer some type of evaluation to monitor content mastery and learner satisfaction. Of that number, only one-third of state DOTs identified using performance-based evaluations. Although this number is relatively small, it is hoped that more agencies will adopt such practices as their ICT programs become established. With that in mind, this section explores the types of practices the example agencies or partners are using the measure the impact of ICT-supported training on employee performance and how they plan to use that information in the future. center for training transportation professionals at the university of Arkansas When the first online concrete field testing course went live, the Center for Training Transportation Professionals at the University of Arkansas’ transportation center monitored its results closely. The center staff compared performance exams between learners in the online and classroom-based ver-

43 sions of the course and noted that performance was the same. However, anecdotal observations from instructors noted that those who did not participate in the classroom training exhibited a completely different demeanor during performance exams (nervous, less confident) compared with classroom learners (less nervous, fairly confident). One of the reasons put forth by the Center for Training Transportation Professionals for the dif- ference in participant behavior was the training experience. Learners who attended the classroom- based version of the course were immersed in training. They focused on the class during the day and homework assignments in the evening. Because most were traveling and staying in a hotel, they were not experiencing day-to-day personal and job distractions. When learners worked indepen- dently, the number of distractions was not nullified by distance. The conclusion was that although the ability to start/stop training is valuable, integrating asynchronous training into a regular workday has challenges that can affect the learning experience. Learners should be prepared to address these challenges before participating to minimize the impact. ohio ltAp Ohio LTAP administers quizzes and postcourse evaluations through the agency’s LMS. The evalu- ation includes a question that prompts learners to identify their level of knowledge and efficiency before the training. For five to six courses a year, follow-up evaluations are performed to identify how the course has affected the participant’s performance and level of knowledge. The follow-up evaluations also measure the frequency with which training content is applied on the job. These data allow Ohio LTAP to estimate efficiency improvements resulting from training. The Ohio LTAP Post Course Survey Committee also provided specific recommendations to the agency’s administration about how the committee would like to further the current evaluation efforts (Ohio Department of Transportation 2010). Figure 31 outlines the other methods proposed to develop a comprehensive assessment of the impact of training on employee performance. In addition to measuring learner satisfaction and the impact of ICT-supported training on per- formance, Ohio LTAP plans to measure the impact of training on the agency. This will include FIGURE 31 Other methods for determining the impact of training on performance, as proposed by Ohio LTAP’s Post Course Survey Committee.

44 determining if the initial web-based trainings improved local project administration. To make this determination, the agency is tracking the number of issues identified per project. After another year of collecting data, the agency will run an analysis to see if the number of issues per project decreases after the completion of the training. Based on anecdotal data (FHWA no longer identifies local project administration and funding as a critical issue for Ohio), some improvements do appear to be evident. Other areas for which Ohio LTAP foresees measuring the impact of training on the agency are shown in Figure 32. FIGURE 32 Measuring the impact of training on the program areas, as proposed by Ohio LTAP’s Post Course Survey Committee.