Keeping What You Paid For—Retaining Essential Consultant-Developed Knowledge Within DOTs (2017)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

20 Case Studies A P P E N D I X A Case studies were developed for seven organizations (five state DOTs and two other organizations). As shown in Table A-1, a total of 11 case studies were produced. These provide broad coverage of the motivations and strategies for consultant knowledge capture and learning. Most of the case studies provide illustrations for multiple motivations and strategies. Table A-1. Case studies. Organization Topic Motivations Strategies Colorado DOT Colorado Flood Response Improving future practice based on experience Remembering what was done and why Developing seasoned program managers and project engineers Mentoring and Interaction Identifying and Learning Lessons Oregon DOT Delivering a Major Bridge Program Improving future practice based on experience Remembering what was done and why Developing seasoned program managers and project engineers Mentoring and Interaction Identifying and Learning Lessons Documentation and Handoff Processes Virginia DOT Outsourcing Bridge Engineering Improving future practice based on experience Developing seasoned program managers and project engineers Mentoring and Interaction Identifying and Learning Lessons Virginia DOT Design Quality Assurance Improving future practice based on experience Identifying and Learning Lessons Virginia DOT Construction Inspection Mentoring Program Developing seasoned program managers and project engineers Mentoring and Interaction Michigan DOT Paperless Construction Improving future practice based on experience Remembering what was done and why Documentation and Handoff Processes Identifying and Learning Lessons Michigan DOT Construction Project Scheduling Developing seasoned program managers and project engineers Mentoring and Interaction

Case Studies 21 Organization Topic Motivations Strategies Tennessee Alberta DOT Environmental Division Improving future practice based on experience Remembering what was done and why Mentoring and Interaction Documentation and Handoff Processes Transportation Highway Design Specifications Improving future practice based on experience Remembering what was done and why Mentoring and Interaction Identifying and Learning Lessons Documentation and Handoff Processes Philips Innovation Services Knowledge Sharing to Drive Innovation Improving future practice based on experience Developing seasoned program managers and project engineers Mentoring and Interaction Identifying and Learning Lessons Documentation and Handoff Processes Tennessee DOT Local Programs Office Improving future practice based on experience Remembering what was done and why Developing seasoned program managers and project engineers Mentoring and Interaction Identifying and Learning Lessons Documentation and Handoff Processes

22 Keeping What You Paid For—Retaining Essential Consultant-Developed Knowledge Within DOTs Colorado DOT Case: Colorado Flood Response Needs: Respond to major flood event impacting 27 state highways and 120 state bridges Leverage external emergency response and innovative delivery expertise Improve future agency emergency response capabilities Techniques: Use of blended teams with strong leadership and clear role definitions Daily stand up meetings Post project lessons-learned report based on survey and interviews with participants Management commitment to learning from experience to improve future practice The Context On September 11 and 12, 2013, much of northeast Colorado, including the cities of Boulder and Fort Collins, received nearly 20 inches of rain, the equivalent of one year’s average precipitation for that area. Twenty-seven state-owned roadways were closed with damage to 242 lane miles and 120 state bridges. A federal disaster declaration spanned 15 counties at an estimated impact to transportation infrastructure of $0.5 billion. While the CDOT workforce immediately mobilized to remove debris and tend to emergency road closures, the process of restoring transportation services by repairing and reconstructing the damaged highways and bridges required assistance from a number of construction companies and consultants. Governor John Hickenlooper and CDOT Executive Director Don Hunt had less than one year earlier announced the Responsible Acceleration of Maintenance and Partnerships (RAMP), a program that changed fiscal management in order to increase the department’s annual construction budget of approximately $550 million by nearly 50 percent, increasing the projects scheduled to be delivered. Despite this recent surge in agency project delivery, Governor Hickenlooper assured the public that newly damaged roads would be opened by December 2013, and Director Hunt and his staff developed a plan to maintain momentum on RAMP while achieving the Governor’s targets. CDOT opened the last of the 27 closed state highways on November 26, just before the Thanksgiving weekend and days ahead of the Governor’s aggressive schedule. This case study describes how the Colorado Department of Transportation (CDOT) used a blended team of consultants and agency staff to complete flood damage repairs within a highly ambitious timeline. Consultants brought experience with emergency response procedures, including proper record keeping methods required for Federal Emergency Management reimbursement. Adoption of standard roles and responsibilities defined by the National Incident Management System (NIMS) provided a proven structure for this blended team to operate productively and accomplish its mission. The tight timeframe for rebuilding damaged facilities created the impetus for sharing of lessons learned on a daily basis. Leadership strategy, clear definition of roles and responsibilities from the start, co-location of forces, and state/consultant collaboration techniques are highlighted.

Case Studies 23 Use of Consultants The entire response and recovery effort was led by CDOT Region 4 Director Johnny Olson, who assumed responsibility as Incident Commander while restructuring his own regional workforce to proceed with RAMP. A team of consultants were retained by CDOT before September had ended, providing both additional workload capacity and emergency response expertise to augment CDOT workforces. In all, 138 individuals, split 60/40 between consultants and staff, were co-located at a temporary Incident Command Center (ICC) in Loveland, Colorado. “Everyone's primary task was to get the job done,” recalls Olson. “We brought consultants and staff together as a single team. We set up teams based on disciplines, not based on employer or geography. This large group encompassed one collaborative network of subject matter experts from both CDOT and private firms. People soon forgot who they worked for." Critical Knowledge Areas Staff and consultants came individually with different areas of knowledge and experience, but nobody had an understanding of the complete picture. Consultants knew flood recovery but did not know CDOT. CDOT knew how to manage projects and administer contracts, but had to quickly learn a new set of rules under Federal Highway Administration (FHWA) and FEMA requirements. CDOT knew how to deliver projects under traditional design-bid-build methods but lacked experience using innovative delivery methods that could accelerate the process. Some members of the consulting team had recent National Incident Management System (NIMS) training as well as experience on Hurricanes Sandy, Irene, and Katrina. Nicole Boothman-Shepherd, Resilience Strategist & Senior Policy Advisor for AECOM, served with URS Corporation as Senior Policy Analyst for the CDOT Flood Project. Knowledge Capture and Learning Following elements of a prescribed ICC structure, Olson assigned four CDOT staff members to serve as discipline area chiefs: (1) Keith Sheaffer, P.E., Chief of Operations; (2) Steve Olson, P.E., Chief of Planning; (3) Kyle Lester, Chief of Logistics; and (4) Scott Young, Chief of Finance and Administration. According to FEMA definitions of the roles, the Chief of Operations manages tactical operations and allocates resources to operational assignments. The Chief of Planning oversees the planning section in collecting, evaluating, processes, and disseminating operational data. The Chief of Logistics is responsible for providing facilities, transportation, communications, supplies, equipment, maintenance, food, fueling, and medical services. The Chief of Finance and Administration is responsible for managing financial matters as needed.1 The chiefs leveraged the consulting team to fill immediate needs for knowledge in areas such as innovative project delivery practices, documentation and reporting, and federal emergency response regulations including reimbursement eligibility. Several knowledge capture and learning processes—both planned and improvised—were used throughout the emergency response. Boothman-Shepherd observed, “Some of the knowledge transfer was deliberate and proactive: ‘How do I solve this problem?’ Some was reactive.” Knowledge transfer 1Incident Command System Training, FEMA, May 2008. https://training.fema.gov/emiweb/is/icsresource/assets/reviewmaterials.pdf

24 Keeping What You Paid For—Retaining Essential Consultant-Developed Knowledge Within DOTs (KT) occurred from CDOT staff to consultants and from consultants to CDOT staff. “We should have formalized ‘KT’ early on,” reflects Boothman-Shepherd. “It would have accelerated the learning curve.” Mentoring and Interaction Onboarding for CDOT staff and consultants transitioning into the ICC was informal but effective. Olson greeted new arrivals by telling them, “What you learn in two weeks at the Incident Command Center is the equivalent of a one-year learning curve at CDOT.” Olson demanded open, frequent communication and held daily 9AM “stand up” meetings for the first several months to exchange staff and consultants praise, lessons learned from the prior day, and other critical knowledge. “We discussed the day's immediate issues and long-term processes and adjustments.” Throughout the early months—if not first two years—of the event’s aftermath, the work was so urgent that little formal mentoring occurred. Rather, continual interaction enabled rapid deployment of the most useful information. “Nicole (Boothman-Shepherd) brought an abundance of emergency response expertise,” observed Heather Paddock, CDOT Flood Program Manager. “She trained other consultants who then trained staff by working side by side.” Young, the full-time Business Manager for CDOT, noted that Colorado’s flood triggered the first use of new federal authorization (MAP-21) procedures for an event of this magnitude, necessitating retention of subject matter expertise. “Because URS had helped provide the written procedures for the emergency response program under MAP-21,” Young stated, “they were able to help determine precedents for CDOT staff, and staff could manage from there.” Working alongside staff, consultants helped speed the process of learning federal regulations, and several consultants drew upon their existing relationships with FHWA. Identifying and Learning Lessons Not only did the capture and learning of critical emergency response practices help open 27 roads in fewer than 90 days, it also positioned the agency to better respond to the next disaster. Olson speculates that if the same magnitude flood occurred today, CDOT would need the same level of consultant support. “The difference would be that we would stand up the ICC framework on day one and everyone would be told their roles immediately. The ramp up time would be reduced significantly,” in part due to the knowledge captured from consultants. As the agency enters its third year of permanent recovery, it is recognizing longer-term benefits of knowledge capture and learning methods. Permanent repair of U.S. 34 is being delivered as a construction manager/general contractor (CM/GC) project, and CDOT is relying on expertise brought to the agency by consultants for this project delivery method. The CM/GC delivery of U.S. 34 moves through its pre-construction phases leaning heavily on one former consultant in particular. Paddock began her work on the flood in September 2013 as a member of the URS team. By June 2014, Olson had hired her to CDOT to report directly to him as Flood Program Manager. She brought her design– build experience and has passed that on—mostly through on-the-job training—to the team she created. Steven Humphrey, a Colorado Flood Response & Recovery Manager from Mueller Engineering Company, Inc., points out that Paddock has taught her team to direct rather than manage the U.S. 34 project. “The CDOT Project Director is the owner, but has partners with a design consultant and a contractor who can help keep the Project Director out of the weeds.” Boothman-Shepherd led the capture and learn process and recently published CDOT’s Lessons Learned document for this entire effort. “CDOT wanted to gather data and institutionalize best practices early

Case Studies 25 on,” Boothman-Shepherd states. “Johnny (Olson) was very strong on this point. We need to learn from mistakes. We get new and better information and use it to course-correct now and in the future.” A web survey was conducted asking consultants and staff what went well and what did not. Consultants and staff that worked in the ICC generally did not complain about communication, long work days, or lack of information. “Their biggest complaint was about lack of lunch options,” jokes Boothman- Shepherd. In addition to the survey, Boothman-Shepherd also performed individual interviews to build the Lessons Learned report. The report, initially 40 pages, was distilled to 25 pages of action steps for use either in future emergency response situations or in CDOT’s institutionalized practices. An example of a lesson from the report is included below: “[There was a] significant logistics gap for the ICC related to technology (IT). In the next event, CDOT should consider dedicated, ICC IT technical support personnel and: . Procure a dedicated server for the response and recovery operation that can be backed up to the CDOT server or cloud . Network printers and plotters to the dedicated server . Provide immediate web access—both hard line and Wifi—for staff and consultants . Plan IT security/firewalls in advance . Provide storage capacity for thousands of photographs . Provide consultant staffs with access to the server, plotters, and printers.” When CDOT subsequently suffered another federally-declared disaster (substantially smaller than the size of the 2013 event) in the southeast portion of the state, Region 4 staff drove to Region 2 to offer assistance, drawing upon the lessons learned. Documentation and Handoff Processes The CDOT business office implemented new reporting for flood-related financial activity to demonstrate how much labor time was being spent by staff on flood activity. Budget-to-actual reporting was also developed in the business office. Boothman-Shepherd indicated that the consultants configured CDOT’s data warehouse and reporting tool to provide regularly updated financial and project-level information. CDOT staff also created a specific document retention plan to track flood items. CDOT staff pushed for the plan, and consultants led the implementation. The pace of activity during response was so torrid that information needed to be captured not for the next project but for the next decision. Michael Krochalis served as Flood Recovery Business Manager. “If you miss a meeting, you miss a lot of decisions. We considered this necessary knowledge management. We used the standard NIMS General Message Form (Form 213) to capture decision points and new processes,” recalls Krochalis. Following NIMS practices brought to the ICC by consultants, General Message Notifications helped keep interested parties current in their areas. An example of a NIMS General Message Form from the event is included in Appendix B. 2013 Colorado Flood: Keys to Successful Knowledge Capture & Learning 1. Clear vision 2. Well-defined roles and responsibilities 3. Co-location of consultants and staff 4. Two-way exchange of information and expertise 5. Streamlined processes for consistent learning and application

26 Keeping What You Paid For—Retaining Essential Consultant-Developed Knowledge Within DOTs As of early 2016, Olson was working with CDOT Headquarters to build a panel of staff and consultants to adopt emergency procedures for subsequent events. The experience gained by staff and retained consultants from the 2013 flood still plays a critical role in shaping how the next event will be governed. Those involved centrally with the 2013 flood will be working to achieve agency-wide buy-in on items such as a revised emergency response manual, pre-drafted contracts with standard FHWA- compliant language, assignments of emergency response duties to staff in all regions including revised job descriptions, and table tops in each region simulating an emergency response. Above all this, Olson underscores the importance of articulating a clear vision (e.g., “Open all roads by December 1”) and establishing a framework of consultants and staff to support it. Although the Colorado flood response was not without its challenges—some participants reported tension between staff and consultants—Johnny Olson was able to minimize many of the conflicts, and the response is considered a success.

Case Studies 27 Oregon DOT Case: Delivering a Major Bridge Program Needs: Successfully execute a $1.5 billion bridge program Provide transparency through comprehensive public involvement, outreach, and communications Leverage technology solutions to achieve efficient and coordinated program management Techniques: Establishment of expectations for knowledge transfer and handoff processes during project planning Close collaboration between DOT and contract staff during the engagement Implementation of technology solutions that institutionalize practices following the engagement The Context In 2002, routine inspections identified expanding cracks in bridges throughout Oregon, prompting Oregon officials to evaluate how deteriorating bridge conditions could impact mobility and the Oregon economy. Without a departure from its current funding levels for repair and replacement practices, they determined the situation would continue to worsen. Beginning in 2001, the Oregon Legislature passed a series of acts, referred to as the Oregon Transportation Investment Act (OTIA). In 2003, the OTIA III State Bridge Delivery Program was tasked to replace 149 bridges and repair 122 more across Oregon within a decade. Such a program would be the largest transportation investment in over 50 years, with a funding total of $1.3 billion in construction costs and $1.5 billion in overall program costs.2 OTIA III was successfully completed in 2015. The bridge program not only helped create and sustain 22,000 jobs, it also tested a number of innovations in Oregon. The program secured Federal Highway Administration (FHWA) funding to deploy rapid replacement on two bridges in the Oregon Coast Range, and earned FHWA recognition for 2 OTIA III State Bridge Delivery Program: Program Delivery, ODOT, April 2015 This case study describes how Oregon DOT outsourced delivery of a major bridge program under an accelerated timeframe. Consultants brought a new, more proactive approach to public outreach and introduced several new technology tools to increase efficiencies. Program management roles an d responsibilities were clearly delineated to ensure effective ongoing coordination and communication between DOT staff and the consulting team in contrast to a “pass the baton” approach. Knowledge transfer from consultants to DOT staff was achieved through (1) establishing clear expectations up front about how knowledge transfer would be done, (2) ensuring ongoing interaction between the consultants and the DOT staff during the engagement, (3) deliberate processes to involve DOT staff who would inherit ownership of new IT solutions in the implementation process, and (4) requirements for specific written deliverables from the consultant team.

28 Keeping What You Paid For—Retaining Essential Consultant-Developed Knowledge Within DOTs ODOT’s first use of construction manager/general contractor delivery on the Interstate 5 Willamette River Bridge project. In total, the program received 68 awards for all aspects of its work, including a 2014 Grand Award from the American Council of Engineering Companies and a 2015 Northwest Best Overall Project from the Engineering News Record for the Willamette River Bridge. Use of Consultants The Oregon legislature directed ODOT to utilize private-sector resources to partner with ODOT to develop innovative solutions for the OTIA III State Bridge Delivery Program. The Oregon Department of Transportation (ODOT) conceptualized and through a rigorous and competitive process hired the Oregon Bridge Delivery Partners (OBDP), a joint venture between Fluor Corp. and HDR Inc., to execute the State Bridge Delivery Program. As with program management for the bridge program, public involvement, outreach and communications were outsourced at the legislature’s direction, with ODOT retaining oversight.3 OBDP was contracted by ODOT to provide day-to-day support to the bridge program, ensure high quality projects at minimal cost, and manage engineering, environmental, financial, safety and other operational aspects of the program. The outsourced team was overseen by a few more than 20 ODOT staff and was expected to bring knowledge of scale and complexity for major projects that were typically seen as beyond the scope of traditional roles in a DOT. While ODOT maintained program responsibility, accountability and decision making, it also established a management hierarchy where the ODOT and OBDP teams mirrored positions so that peers could address and resolve issues efficiently. These included positions in ODOT’s Major Projects Branch. The OBDP Project Director met regularly with agency stakeholders to align on strategic issues, program change management, contract administration and areas of dispute. Clear roles and responsibilities, with minimal overlap, were critical to the success of the program. 3 OTIA III State Bridge Delivery Program: Communications and Community Engagement, ODOT, April 2015 “When the Legislature directed ODOT to outsource program management to the private sector, we could not know what a valuable ally we would have in Oregon Bridge Delivery Partners. With its approximately 400 team members, the joint venture of Fluor Corp. and HDR Inc. expanded the agency’s reach and vision. Each partner brought experience and perspective from previous infrastructure projects, and our collaborative solutions were much the better for it. The timeline for repair or replacement of so many bridges statewide meant that cost-effectiveness and efficiency were paramount. Our teammates from OBDP looked for ways to speed construction and save money. They fostered our crucial relationships with natural resource agencies so that changing regulations did not slow down delivery schedules. They designed information technology systems that coordinated and streamlined the work of many contributors in many locales. And they worked with us to negotiate advantageous contracts, avoid costly change orders and implement innovations.” Source: “Leaving a Legacy: Delivering the Oregon Department of Transportation’s OTIA III State Bridge Delivery Program,” Oregon Department of Transportation, September, 2014

Case Studies 29 With broad goals of stimulating Oregon’s economy, employing efficient and cost-effective delivery practices, maintaining freight and traffic mobility, building projects sensitive to communities and landscape, and capitalizing on funding opportunities, the ODOT and OBDP teams collaborated to successfully implement the bridge delivery program. Critical Knowledge Area 1: Communications A program that increases public spending is not always well received. It was vital that ODOT accurately and transparently report milestones and successes to legislators, citizens, freight haulers, and other stakeholders. ODOT and OBDP worked collaboratively to communicate program innovations, along with activities and benefits, such as delivering projects more cost-effectively and efficiently. ODBP consultants refined and adapted ODOT communication practices, proactively pursuing media coverage and securing presentation opportunities. Communications activities ranged from sharing insights with other state DOTs at national peer exchanges, to targeting proactive media placements, to actively engaging local communities such as Eugene and Springfield on the Willamette River Bridge and Ashland on two urban bridges that I-5 travelers first encounter on their northbound journey through the state. Kaylene Toews of Edelman Public Relations—a sub-consultant of OBDP—was with OTIA III from the beginning of the project to its completion in 2015, serving as Senior Account Supervisor at the conclusion. “We knew from the onset of the program that reporting on progress wouldn’t be enough. We needed to share the human impact of OTIA III, and we could use traditionally corporate reputation management techniques to engage with our audience.” From webinars on best practices to national award applications, OBDP took a public relations approach to the program based on corporate leadership strategies and focused on sharing ODOT’s learnings and stories. Toews remarks, “Communications on the bridge program were so successful because we had executive sponsors who saw communications as a vital part of the program.” Tom Lauer, ODOT’s Chief Engineer & Technical Services Manager, served as OTIA III bridge program manager. Lauer found that “the communication consultant brought a unifying voice to what we did. We wanted style and form of communication to be consistent.” Edelman and OBDP worked with staff to ensure a consistent message for both program- and project-level communication. “Digital presence was a key factor,” Toews notes. “We took public information to where people were. The integrated bridge program team organized ODOT’s first virtual open house. It worked because we had many digital channels to take advantage of the public’s access to the web, including live tweeting and video press releases. We got much more, and much richer, feedback through digital channels than through on-site meetings.” Speaking to the important aspects of consultant involvement on the bridge program, Toews observed that “there are many really talented public relations experts at ODOT, but by the nature of their job they are more focused on daily business. Consultants helped communicate proactively on the agency’s behalf.”

30 Keeping What You Paid For—Retaining Essential Consultant-Developed Knowledge Within DOTs Knowledge Capture and Learning: Communications Mentoring and Interaction Working side by side, OBDP consultants and ODOT staff helped the program better deliver a clear and consistent message. Karen Jones Jackley, Communications and Stakeholder Relations Manager, managed public information/involvement, outreach and communications for the entirety of the bridge program. Karen emphasizes the importance of partnerships in consulting-staff relationships. She notes that the project was “extremely collaborative, and working relationships were positive.” She also pointed out that, “We became a team and were able to work through solutions. As an agency, we wanted pushback. That is the advantage of having a consultant: show us ways to do things differently.” Jason Neil, OBDP Program Manager and current HDR Professional Services Director, worked closely with Jones Jackley and others to build a comprehensive outreach approach. “Once we realized that the communication would be overwhelming, we put together a communication strategy. Working together, we came up with a plan to manage all of the information and information sources.” Neil notes that both formal and informal training for the OTIA III bridge program were provided. This training did not vary based on whether audience was staff or consultant. Neil recalls, “The ODOT members that needed to interface with our program—bridge staff and others—participated and went through the same level of training as the consultant staff.” Jim Cox currently serves as ODOT Preservation & Operations Program Manager. For OTIA III, he wore many hats including project delivery manager and alternative contracting program manager, providing oversight of staff and consultants. He underscores the importance of consultants and staff working collaboratively, generating ideas and building on one another’s past experiences. With respect to public outreach, he recalls the contractors visiting schools during the day, to build model bridges with students. Parents would arrive at the school in the evening for a presentation on the OTIA program and see the work of their children that had been completed earlier in the day. Cox is aware that, “A lot of the success of the program was due to the number of bright people on both sides that figured out how to work together.” Because ODOT retained oversight and management of communications throughout the program, it gained valuable, third-party counsel to supplement its internal staff. In Cox’s many roles with the program, he worked collaboratively with consultants on an array of activities, from resource loading to communicating the return on investment of technology implementation. Agencies other than ODOT benefited from this cooperative effort. Cox reported that “the Bridge Reporting System developed by ODOT and OBDP was used by the state of Oregon for reporting American Recovery and Reinvestment Act requirements.” Jones Jackley stressed the benefits of interaction on the entire communications effort, as she noticed that “we couldn’t have done as robust and successful of a communications program with the outcomes we achieved without OBDP. We needed that support not only in terms of resources but in ideas. It pushed us to do new things. Without OBDP’s expertise and strategy, we probably would not have been as successful in receiving awards over the years. We learned from them and as a team we figured that out and put that to use.”

Case Studies 31 Critical Knowledge Area 2: Information Technology Solutions In addition to actively seeking solutions to establish new ways of managing infrastructure projects, and broadening public involvement and communication, the project invested heavily in information technology (IT) solutions. These solutions were intended to improve work flow, reduce cost, help coordinate program elements and provide timely, accurate information.4 Nine IT systems were implemented to support the bridge delivery program to help achieve the program goals and provide documentation and reporting to the public and the legislature: Geographic Information System (GIS) Infrastructure Construction Engineering Inspection (CEI) Tool Electronic Document Management System Work Zone Traffic Analysis (WZTA) Tool Bridge Reporting System Environmental Baseline Report Online Environmental Baseline Report Online (EBRO) Mobile Data Collection Engineering Drawing System Pre-Construction Assessment Tool In a 2010 study conducted jointly by ODOT and OBDP, the implementation of these tools was found to have a benefit-cost ratio of 2.1:1, producing more than a 100 percent return on investment.5 The GIS infrastructure was designed using best practices for the industry, and supported and interacted with other systems to provide a network of services including maps, databases and support for special studies. GIS data extracts were made available throughout the enterprise. The GIS infrastructure provided the foundation and flexibility to respond to changes over time. The CEI tool was implemented in combination with the Electronic Document Management, and helped to streamline the inspection process and reduce the need for paper reporting. This was a general purpose solution that had broader applicability beyond the OTIA project. The manner in which these systems were implemented also provided a model for using standardized naming conventions and structured metadata to facilitate shared access and retrieval of project information. Similarly, the GIS infrastructure used common metadata for bridge identification to link data sets and relate internal data sets to external data sources. 4 OTIA III State Bridge Delivery Program: Benefits of GIS to manage a major transportation program, ODOT, April 2015 5 Information Technology Benefit-Cost Evaluation Report, OBDP, January, 2011 The work zone traffic analysis tool (illustrated in Figure 4) was developed to support a program objective to minimize traffic interference due to construction. OBDP was able to replace a largely manual process involving compilation of data from multiple sources to an automated web service providing a single source of data. As stated by WD Baldwin of HDR Inc., “The GIS-based work zone traffic analysis tool saved a tremendous amount of time and provided ODOT the ability to adjust their strategies to changing needs of the organization.” This tool is still being used at ODOT.

32 Keeping What You Paid For—Retaining Essential Consultant-Developed Knowledge Within DOTs Figure 4. ODOT's Work Zone Traffic Analysis Tool The bridge reporting system (BRS) became the single, definitive source of bridge program information. As illustrated in Figure 5, BRS produced monthly progress reports supplying comprehensive data about the status of projects in the program. Tables and charts accompanied each report in a repeatable and predictable manner, earning the trust of the readers, legislators, media and the public. At peak construction, in 2009 and 2010, the monthly report was downloaded by hundreds of online users. Knowledge Capture and Learning: Information Technology Solutions Mentoring and Interaction ODOT and their OBDP partners collaborated extensively to find efficiencies and alignment through the organization to deliver scalable and useful technology solutions. Tangible benefits of this process were observed in both cost and time savings. The technology systems allowed a two-way flow of data Figure 5. OTIA III State Bridge Delivery Program Monthly Progress Report, October 2014

Case Studies 33 Identifying and Learning Lessons “Whether you’re planning to manage documents, or move wide loads through corridors under construction, you have to have a vision and build a strategy collaboratively,” says Lea Ann Hart- Chambers, Performance Manager within ODOT’s Major Projects Branch. “There was a lot of innovation brought by both sides.” Capturing that innovation has in some instances been deliberate. A robust change management process was developed within the program and has since been adopted by the agency regional offices. The development of the EBRO and Mobile Data Collection (MDC) effort, for example, highlighted the importance of sharing information and access to technology tools. Documentation and Handoffs The documentation and handoff process was, by most accounts, very successful in the bridge program. The expectation of knowledge transfer activities was set from the beginning, and both ODOT and ODBP were engaged. As Jason Neil from HDR stated, “Part of our charter was to document the process of project delivery and transfer that knowledge to the organization.” Three planning documents were, as between ODOT and OBDP, improving access to timely information, accountability between teams and consistency. ODOT and OBDP partnered closely to ensure the different technology solutions met the needs of the bridge program, aligned with agency goals, and could be sustained within the organization. While the OBDP team developed and maintained its own database, they leveraged ODOT existing efforts to better align with the agency and minimize operational costs. The OBDP application was built on the same system platform and Oregon OTIA III: Keys to Successful Knowledge Capture & Learning 1. Blended team 2. Joint training 3. Collaborative problem solving 4. Leadership engagement and encouragement 5. Common platforms, compatible technology and data standards was designed to be flexible and scalable to seamlessly integrate with ODOT GIS. Through close collaboration, ODOT and OBDP were able to successfully plan for seamless integration of systems to deliver an enterprise solution that enabled data sharing across functional levels and business applications using common platforms and standards. Jason Neil, a consultant with HDR, emphasized the importance of coordination and alignment when he stated, “The WZTA tool was developed in collaboration with OBDP and ODOT, which ensured not only its usefulness, but also engaged the ODOT team to participate in, and encourage, training throughout the organization.” examples, important in managing OBDP and maintaining alignment: Strategy Document, Responsibility Assignment Matrix, and the Staffing and Financial Plan. OBDP had a Document Management Team of four to five individuals. OBDP’s Document Management leader provided training for everyone who had a role in document management, with approximately two hours of training per individual. Templates provided information on how to create and store files, and the team provided support as needed through online guidance. The system offered browse and search capability for easy document location. This work has helped ODOT consider how to set up similar systems for other programs. Document management was not new to ODOT, but as Hart- Chambers mentioned, “Consultants brought expertise in how to manage data, naming conventions and

34 Keeping What You Paid For—Retaining Essential Consultant-Developed Knowledge Within DOTs Because the reporting system was implemented collaboratively, and was well documented, ODOT staff were able to carry on with their application. Due in large part to the sustained success of the bridge program, a state Radio Project was moved from another Oregon agency to the same Major Projects Branch at ODOT that had been leading the bridge delivery program. Lauer was asked by members of the legislature how the radio project would be managed and communicated. He told them it would be managed just as ODOT had managed the bridge program. They handed full control of the project to ODOT and began receiving monthly progress reports modeled after those to which they had become accustomed on the bridge program. Handoff processes for the various technology tools were also planned and managed so that ODOT could continue to use and benefit from these tools after the OTIA III consultant team had left. In many respects, the development and rollout processes for these tools were approached in a manner that ensured sustainability within the organization. For example, ODOT staff was heavily involved in the training for the WZTA tool. The OBDP development effort for the MDC was coordinated with ODOT’s existing GIS and designed to provide a general purpose resource for ODOT operations even after the conclusion of the bridge project. other critical aspects of Enterprise Content Management or Document Management. This provided a comfort level not only to ODBP but throughout the agency with regard to management of content.”

Case Studies 35 Virginia DOT Case 1: Outsourcing Bridge Engineering Needs: Utilize private consultants to handle bridge design and inspection workloads that exceed internal VDOT staff capacity Build and sustain internal agency bridge engineering expertise Take advantage of public/private collaboration to improve engineering practice Techniques: Joint public-/private-sector technical committee Lessons learned capture - vetted by communities of practice Informal brainstorming and collaboration sessions on issues of concern Mentoring The Context The Virginia Department of Transportation (VDOT) is responsible for maintaining and operating a 58,000-mile network of highways and bridges, which is the third largest state-maintained highway system in the country, behind North Carolina and Texas. VDOT inspects and tracks more than 20,900 bridges and structures. The VDOT Structure & Bridge Division has 130 people statewide doing design and project development. There are approximately 50 new bridge projects each year. VDOT has 34 bridge crews out in the districts. These crews do a minimum of 100 bridge replacement projects annually, but a majority of their time is spent on maintenance. Use of Consultants As state resources for VDOT have decreased, outsourcing has increased. Seventy percent of bridge design work and 20 percent of inspection work is outsourced to the private sector. From VDOT’s perspective there is some risk to high levels of outsourcing on bridge design. Even if VDOT were to stop doing design and play a purely project management role, they would still need to maintain internal staff design expertise. Otherwise VDOT staff will be unable to negotiate properly and knowledgably with private-sector consultants. When problems arise related to lack of needed expertise on the consulting side, VDOT staff must have the ability to properly evaluate consultant credentials and provide a substantive review of work products. VDOT currently outsources 70 percent of its bridge design work and uses multiple strategies to ensure that critical knowledge related to bridge engineering and construction is being captured and shared. VDOT has recognized the need for two-way knowledge sharing with their private-sector partners. A longstanding joint public/private technical committee has proved to be an effective mechanism for identifying and resolving complex issues. Work in this committee has tapped into the collective expertise of VDOT engineers, consultants and contractors to improve standard specificatio ns and practices.

36 Keeping What You Paid For—Retaining Essential Consultant-Developed Knowledge Within DOTs Critical Knowledge Areas VDOT’s assessment is that it takes 8–10 years for an engineer to become proficient at bridge design. VDOT makes a conscious effort to build and sustain bridge design expertise within its staff and to ensure that its consultants have this expertise as well. As part of VDOT’s quality control of consultant contracts, which are managed centrally, districts are asked to review design plans and evaluate their consultants every six months. Knowledge Capture and Learning VDOT utilizes a variety of methods to sustain and build staff expertise in bridge engineering: mentoring, capture of lessons learned, presentations, informal discussions, and focused brainstorming sessions. Many of these methods involve interactions with the consulting community, with an emphasis on two-way knowledge sharing and problem solving. The Joint VDOT and Virginia Transportation Construction Alliance (VTCA) Structure and Bridge Committee is an example of a longstanding mechanism that provides a stable channel for relationship building and knowledge sharing between VDOT and private consultants, supporting its other knowledge-sharing and capacity-building activities. Increased emphasis on strengthening connections between private and public sectors and increasing the interaction between these two entities has been advantageous for everyone involved. The end result has been minimal risk associated with the loss of knowledge associated with use of consultants. Mentoring and Interaction VDOT makes a conscious effort to have a structured, formal and informal, continuous learning program for bridge staff. Many of these activities involve the consulting and vendor communities. Examples include the following: Staff are assigned project management and design work on a continuing basis. VDOT creates mentoring relationships. New engineers conduct parallel designs of experienced designer’s. There is then a discussion/mentoring session to evaluate what worked and what did not, which allows the less experienced designer to learn by doing and by conferring with the expert designer. The state structure and bridge engineer meets quarterly with the districts individually to share information and address issues. VDOT has an open door policy for vendors which has been effective in establishing appropriate relationships in which VDOT staff are able to become familiar with available capabilities and vendors can learn about VDOT’s challenges and needs. VDOT held a “Vendor Demo Day” where vendors were invited to do demonstrations of their products. VDOT invited the maintenance crews, district engineers and other interested individuals. This event got rave reviews from attendees, who appreciated the opportunity to see various products in action rather than solely reading descriptive product information. Collaborative sessions are held on specific issues of concern—for example, VDOT has a slogan: “Corrosion is our nemesis.” The Structure & Bridge Division brought in corrosion consultants for a collaboration session to discuss available techniques for preventing and addressing concrete corrosion.

Case Studies 37 Identifying and Learning Lessons The Joint VDOT-VTCA Joint Structure and Bridge Committee provides a forum in which VDOT leaders and industry leaders who are consultants and contractors are able to identify and resolve complex issues and problems associated with bridge structure and design. The Joint Committee has operated effectively over the past 25 years. Meetings are held twice per year: in the spring, before the VTCA Spring Conference, and in the fall. Meetings generally last 2-4 hours. The subcommittee chair may call for additional meetings if needed to address any concerns/problems. Membership is drawn from VDOT, FHWA, and VTCA members. Kendal Walus, State Structure and Bridge Engineer, is the Chair of the Joint Committee and appoints the VDOT members. Tom Witt of VTCA is the VTCA leader and appoints the VTCA members. FHWA appoints its members. Currently there are eight VTCA members and eight VDOT and FHWA members. Each VTCA member serves a four-year term. VDOT and VTCA each provide a staff member to support the committee work. The Joint Committee has become an invaluable forum within which VDOT leaders and the private- sector design and construction firms can identify and resolve complex issues of building and construction standards. From the VDOT perspective, the Joint Committee can be used to solve issues with the contractors/consultants and address specification changes required by new technology. The following are a few of the issues the Joint Committee handled: Bridge contractors ran into difficulty setting girders in the field—anchor bolts were being inserted in the wrong place. The issue was taken to the Joint Committee. After research and discussion among the members, the problem was resolved by changing the specifications and design practice for casting anchor bolts. Industry was not keeping equipment on site for VDOT to do final inspection. The Joint Committee solved the issue by making the required equipment a pay item. An issue arose over the timing of backfilling around abutments. Based on requests from industry to have backfilling done earlier, VDOT made a change in the specification for backfilling. To address the cracking of bridge decks, a low cracking concrete specification was developed. The industry worked with VDOT to introduce specifications for non-corrosive reinforcement bars. Most recently, the Joint Committee provided the forum for discussing and vetting a revision of the specifications book. Thirty-one significant changes and additions were presented for discussion. VDOT and VTCA were able to come to agreement on the content for all of those changes. While VDOT remains the owner of the specifications, the deliberations with the VTCA committee substantially enhanced the final revised version. In addition, because VTCA was involved, there is both understanding and agreement between VDOT and the private-sector contractors and consultants who are hired to design and construct bridges in Virginia. In sum, The joint bridge committee has been used successfully by VDOT to resolve issues of concern, foster successful business relationships, and enable all parties to be kept informed. It provides a rich base of expertise and experience that can be tapped to resolve complex issues associated with bridge design and construction.

38 Keeping What You Paid For—Retaining Essential Consultant-Developed Knowledge Within DOTs Monthly “lunch-and-learn sessions” are used to share new technology and new information. These are sometimes in person and other times via video conferences so that techniques can be demonstrated. Presentations are made both by central office and district staff. Lessons Learned examples were developed, vetted by a community of practice (CoP) and published on VDOT’s internal portal. These have proved to be a valuable learning tool and have covered topics such as slope erosion, non-destructive testing, and hydro-milling. Other methods for identifying and learning lessons that have been used at VDOT include the following:

Case Studies 39 Case 2: Design Quality Assurance Needs: Continuous improvement in design quality Leverage consultants to supplement VDOT internal staff design resources Techniques: A standardized approach to capturing and documenting design issues and sharing lessons learned to improve design quality over time Use of lessons learned to update standard practices The Context VDOT’s Location and Design Division (L&D) develops and maintains procedures, technical guidance, and training for the agency’s highway design function. It provides oversight of policy compliance and quality assurance relevant to specific engineering functional areas. It also assists in the development and delivery of transportation projects within VDOT’s Six Year Improvement Program (SYIP). Projects are managed at the district level with support, as needed, from the central office. The L&D Division is staffed by approximately 100 employees in the central office. There are approximately 230 additional L&D employees throughout VDOT’s nine construction districts. In-house design work is conducted both within the central office and the districts. Use of Consultants VDOT underwent an organizational restructure in 2007 which saw a statewide reduction in L&D staff by approximately 25 percent. This reduction resulted in changes to the level of work generally outsourced to consultants. L&D utilizes consultant design contracts to assist in the development and delivery of VDOT’s SYIP. L&D strives to optimize the capabilities of in-house design staff along with those of the private sector to deliver the SYIP. Due to a fluctuation in workload over the past few years, the amount of design work outsourced has been approximately 50 percent. Design contracts are procured by VDOT’s Consultant Procurement Office and the L&D Division is responsible for their oversight and implementation. There is a large degree of synergy generated between L&D staff and the private sector. The state L&D engineer estimates that in the mid-1990s a very small percentage of internal L&D staff had private- sector experience. Currently, existing estimates place this number at approximately 50 percent. These increased levels of diversity serve to enhance the capabilities of all design staff (both public and private) by providing a more open forum for discussion of ideas, design flexibility, and opinions Over the past 15 years, VDOT has used a standard form for evaluating design quality following construction. This form is used for identifying and learning lessons about future improvements to standards or processes. The same process is used regardless of whether work is performed by consultants or by internal staff. Notably, the results of the evaluation are solely focused on continuous improvements to design practices. They do not result in an assessment of consultant performance and do not impact future consultant selection processes.

40 Keeping What You Paid For—Retaining Essential Consultant-Developed Knowledge Within DOTs regarding design fundamentals. The interaction between these two parties provides mutual benefits to both such as helping to decrease the amount of time and money required by consultants to “get up to speed” with the department’s policies and procedures. Critical Knowledge Areas L&D staffing includes experts in roadway geometric design, hydraulics and river mechanics, survey and photogrammetric work, project management, consultant services, stormwater management, alternate project delivery (Design–Build), traffic engineering design, landscape architecture, CADD support, safety hardware and the corresponding development documentation and interpretation of standards, policies and procedures. Knowledge Capture and Sharing Identifying and Learning Lessons For approximately 15 years, the L&D Division has utilized the Form LD-433 Design Quality Evaluation Report to provide a quality assurance review on how engineering designs are constructed in the field. The form provides a lessons-learned opportunity on VDOT- and consultant-designed projects for both design and construction disciplines to provide input to the design and construction processes, identify issues needing modification, discuss project activities that worked well and those that did not, and interject modifications for potential improvements back into the design plan development and construction processes. The results are not used to evaluate performance of individual consultants. VDOT has not found any difference in the ability to obtain honest assessments of issues for a consultant-designed project than for an in-house designed project. Up until 2015, the form contained a numeric score that allowed critical items to be assessed quantitatively based upon quality and constructability. The intent of the overall derived score was to provide VDOT and contractor staff a high level summary of critical issues, problem areas as well as beneficial practices. In November 2015, the scoring section was removed in favor of more qualitative discussions and exchange to help foster greater collaboration between the design and construction functions. At the same time, VDOT took the opportunity to shorten the form and create a fillable PDF with built in pick lists to make it easier to complete. It is hoped that the new form will foster more time spent on lessons learned and long-term quality assurance of the process. The LD-433 is completed at the end of each project with input provided by the project manager, project engineer, and various sponsors. Much of the information contained in the LD-433 is provided via the inspector’s field notebook, project staff discussions, and communication with the contractor. Such information is then used by all team members to provide a final assessment of the project. This assessment provides an opportunity for all parties to discuss positive issues about the project as well as those areas needing improvement. The form itself does a very good job in assessing the quality control of the project and also provides a platform to assess the overall quality assurance of both the design and construction processes. The LD-433 form has proved to be a useful tool in evaluating and modifying VDOT’s internal policies and procedures regarding the design and construction of projects. Early indications are that eliminating the quantitative scoring section is leading to more collaboration and discussion of lessons learned. The quality assurance (QA) process is used to identify things that could be improved as well as things that went well. This information is used by the individual responsible for VDOT’s training and

Case Studies 41 standards. Results of the process have been used to add to VDOT’s Lessons Learned repository and to change policies and practices included in the L&D manuals. Potential changes are discussed at division and district meetings. The following examples are based on feedback from the QA reviews: VDOT has taken steps to reduce clutter on construction plans by using Right of Way Plan Sheets in lieu of Right of Way Plats. Starting in April, 2016, VDOT Manuals and Polices were revised to use an additional sheet in the project development process. This sheet would show the utilities color coded to enable the utility companies to better identify their utilities. The sheet is created specifically for use by utility companies and is provided for informational purposes (not part of the official construction plans). Case 3: VDOT Construction Inspection Mentoring Program Needs: Developing seasoned construction inspectors Techniques: Use of consultant mentors to build staff capabilities Careful matching of mentors to mentees Classroom training using a standard curriculum combined with experiential learning Standardization in the approach, outcomes, and documentation of experiences to ensure sustainability and quality of a centrally planned yet regionally administered mentoring program The Context Like many DOTs, VDOT has faced a decrease in the number of experienced construction inspectors on staff as seasoned inspectors have retired or moved to jobs in the private sector. In recent years, VDOT has increased its use of consultants for both design and construction engineering tasks. Many of these consultants build up a rich knowledge base with the agency. Some move back and forth between the agency and consulting firms throughout their career. VDOT has established a mentoring program that taps in to some of this rich expertise to train new or less experienced inspectors. VDOT's mentoring program has involved mentors from both public and private sectors. This expands VDOT's options as they seek the best person to place in a particular situation. This case study illustrates implementation of a mentoring program for development of construction inspectors that involves both VDOT staff and consultants as mentors. This allows the agency to tap into a richer resource base in order to build internal expertise, particularly given the fact that there are a number of consultants that formerly worked at VDOT as inspectors and are closely familiar with agency work requirements and standards. It is also notable that VDOT is not overly concerned about newly trained internal inspectors leaving for private sector jobs. They recognize that this will happen given inspector salary levels and understand that the individuals that choose to leave will likely be available as consultant inspectors (or mentors) in the future.

42 Keeping What You Paid For—Retaining Essential Consultant-Developed Knowledge Within DOTs Use of Consultants VDOT makes extensive use of consultants to serve as construction inspectors—of the 190 total construction inspectors at VDOT, 70 percent are consultants.6 The construction inspection mentoring program also makes use of consultants. On average, 75 percent of mentees are under VDOT mentors and 25 percent are under consultant inspector mentors. The share of consultant inspector mentors is expected to grow given the agency’s plans to almost double the number of trainees. If from the private sector, the mentor is a senior inspector. The time spent in mentoring activities is not handled as a separate construction pay item. The decision to identify a VDOT inspector or a consultant as the mentor is based on which mentor can provide the best transfer of knowledge to the mentee. VDOT picks mentors who are the “right people” for the job regardless of whether they are VDOT employees or consultants. Many of the more experienced consultants are former VDOT employees. Consultants who are selected typically have a reputation in the field for doing good work. Critical Knowledge Areas Construction inspectors provide front line oversight of construction projects, ensuring conformity of work with the contract, plans, standards, and specifications. They receive technical training in surveying, plan reading, roadway and drainage construction, guardrail installation, paving, stormwater, erosion and sediment control, traffic control, documentation procedures, and safety procedures. In addition to engineering and procedural knowledge, inspectors need to be able to establish good working relationships with contractors to both ensure strict adherence to specifications and facilitate expeditious and fair resolution of issues that arise. Knowledge Capture and Learning Mentoring and Interaction The mentoring program is managed by a Technical Training Manager and is managed within the agency’s construction division. The program currently enjoys strong support from the executive leadership of VDOT. There are 42 individuals enrolled in the trainee program, and the mentors—senior inspectors—generally volunteer for the program. The program staff encourage suitable candidates to participate. Mentoring occurs primarily in the district offices of VDOT where the mentees are located, but each mentee first receives an orientation from the VDOT central office where the mentorship is controlled. Each mentor is provided a copy of the mentee training book that outlines the goals trainees are expected to achieve. This training book contains all of the information needed to manage and document mentee progress. Each mentee is expected to complete a standard curriculum consisting of four courses. Multiple mentees will participate in a given course. Course content includes plan reading and construction math, basic survey skills, documentation and record keeping, roadway design and drainage, structure and bridge, and general computer skills. As mentees complete each requirement, their mentor reviews their practical and classroom learning requirements and certifies that each 6 These figures were current at the time of the interviews; VDOT was planning at the time to hire additional in-house inspectors.

Case Studies 43 individual has met the required elements and knowledge content of those elements. As mentees complete the program, they are asked to evaluate their mentor. The average length of the program is 18-24 months. The exact amount of time is dependent on the mentee completing the required courses and the practical experience components of the program. While the core requirements must be completed successfully by each individual in the program, the program is sufficiently flexible to allow for individual learning styles. As mentees graduate from the program to assume inspector positions, the program managers select another individual to enter the program. The structure and content of the VDOT Construction Inspection Training Program is consistent with the principles and practices associated with successful mentor-mentee programs. Mentors are carefully selected to assure a proper fit between the mentor and the mentee. If there is not a good match between mentor and mentee then changes are made. The inspector training leaders stay informed on the formal training and the applied experience components of the training program through detailed conversations and evaluations from both mentors and mentees. If issues do arise, they are addressed immediately. A goal for program improvement is to develop a more structured process for identifying and assessing candidates for the Construction Inspector Trainee Program. The VDOT program managers are focused on addressing this issue. Currently mentee candidate sources include individuals who have served in the military, have worked in other positions in VDOT and want to become inspectors, and/or began their career in related work in the private sector. Trainees who are mentored by consultants are treated just like trainees with VDOT mentors with respect to learning standards, formal classroom training, and practical experience training requirements. One benefit of this is that mentees are exposed to the private-sector consultants and their ways of carrying out their responsibilities. This helps assure that the mentees understand the importance of knowledge transfer when private-sector individuals or firms are hired under contract to perform transportation road and bridge work for the state. They are thus much more likely to know how to deal with the private-sector contractors and consultants than if they did not have this experience as part of the mentoring program. While each mentor structures his/her activities with assigned mentees to the mentee’s experience and learning style, a typical situation includes assignments designed to ensure that the mentee gets the needed range of practical experience in the field. As one mentor explained, learning to read and understand contracts is essential knowledge for successful inspectors. Therefore, his mentee was given an example contract to analyze and then discuss with the mentor. Similar assignments were identified in each of the areas of experience needed, balancing learning the document components of the inspector position and then testing that learning through inspection assignments that allowed the mentee to apply his/her learning to a real inspection assignment. Depending on available projects, each trainee will have different experiences. For example, one mentee may end up having more roadway/paving experience; another may be exposed to more bridge work. VDOT does announce specialized types of work and tries to assign trainees to these projects when possible. An example of a typical mentee development is the experience of an individual who applied and was selected for the Inspector Trainee program in 2014. His assignments gradually increased in complexity and span of inspection responsibility. His first area of focus for practical experience was construction management. Three months later he was assigned to work for an experienced inspector. His assignments initially involved records management, then expanded to grading, serving as lead

44 Keeping What You Paid For—Retaining Essential Consultant-Developed Knowledge Within DOTs inspector on a sub-component of a larger project, followed by a bridge project where his inspection duties were focused on grading and landscaping. The required formal classes were intermingled with the practical experience. From this individual’s perspective, the classroom learning was helpful, but it was the application of that learning in the field that solidified his understanding of the intricacies of inspection work. He also liked the fact that as a person who did his best learning with the “hands-on” experience, the field experience provided this opportunity. He rated his mentor nine on a 10 point scale. An important success factor for the VDOT Inspector Trainee program is the substantial volume and variety of road and bridge work in the state. This provides a broad spectrum of opportunities for mentees to get a variety of experience in the various components of transportation construction. There is no requirement for the mentee to work as a VDOT inspector for a minimum amount of time for the agency after completing the training program. Inspectors do not make a significant salary. Therefore, the VDOT view is that if an individual can get a position that pays more money, or is closer to his or her home town, they are free to take advantage of that opportunity. If they become a private- sector inspector, VDOT still knows that they will be able to tap into these individuals as needed, and they will be well trained and familiar with VDOT work requirements and standards for inspections. The Virginia Department of Labor and Industry has adopted the VDOT program into an apprenticeship program for private employers. VDOT’s longer-term plan is to have the state’s community college system as the primary provider of courses to develop inspector skills. They are supported in this effort by the VTCA.

Case Studies 45 Michigan DOT Case 1: Paperless Construction Needs: Improved coordination with consultants and contractors Common access to a shared pool of construction documentation Improved efficiency through streamlining of documentation and reporting processes Reduced litigation risk Techniques: Transition to electronic documents for survey, design, contract development and construction phases Implementation of a document management platform that enables convenient document access by MDOT staff, consultants, and contractors anytime and anywhere, including from the field Standardization of document naming and filing conventions Automation of workflow for electronic bidding and construction management (including contracts, invoices, and digital signatures) Collaboration with industry groups to continue improvements in e-Construction practices The Context Michigan Department of Transportation (MDOT) underwent an organizational reinvention in 2010. The DOT downsized and refocused on core competencies. MDOT made a decision to outsource non-core competencies to consultants. Partly as a result of the reorganization, DOT staff had increased project management workloads involving coordination with consultants and contractors and provision of project oversight. MDOT implemented e-Construction to improve the efficiency of construction projects and to minimize the challenges of outsourcing. E-Construction began with three pilot projects in 2012 and was expanded statewide in 2014. At MDOT, e-Construction has involved transitioning away from paper- based processes, substituting online capabilities for activities such as bidding, project management, and documentation. The benefits of e-Construction include time savings, increased document accessibility and findability, reduced document loss, and greater coordination both internally at the DOT and also externally with consultants and construction companies. These benefits yield cost savings and also support knowledge capture. This case study illustrates the value of transitioning away from paper-based design and construction processes, automating workflows for review and approvals, and utilizing document management software that provides convenient access to project documents for consultants and contractors outside of the agency firewall. While tools for sharing electronic content between DOT staff and vendors are valuable in a supporting role, they don’t take the place of active strategies for ensuring that lessons are identified and learned.

46 Keeping What You Paid For—Retaining Essential Consultant-Developed Knowledge Within DOTs Use of Consultants In the five years since the reorganization, outsourcing at MDOT has been stable. MDOT currently outsources design for roughly one-third of construction projects. More complex and expensive projects tend to be outsourced more frequently. Critical Knowledge Areas The construction project delivery life-cycle includes survey, design, contract development, and construction. Each phase produces a variety of documentation including measurements, design calculations, plans, specifications, meeting notes, agreements, approvals, legal documents, field notes, change orders, etc. Project documentation is produced collaboratively and shared across multiple participants—including MDOT staff and consultants/contractors—within each phase. Documentation produced in each phase provides valuable reference information for subsequent phases and a historical record of information gathered and decisions made following project completion. Knowledge Capture and Learning Documentation and Handoffs MDOT views its e-Construction initiative as a highly successful technique for capturing construction project documentation and facilitating handoffs from one phase to another, particularly when a mix of MDOT staff and consultants are involved in various aspects of a project. Prior to the implementation of e-Construction, survey, design, contracts, and construction were distinct and separate phases. Consultants who worked on design had minimal involvement with the project past the design phase, and construction contractors were not involved in design. E-Construction has linked the project processes from design to construction, allowing for easy sharing of documents. For example, contractors provide electronic markups to design documents to show DOT and design consultants what needs to be changed in the field. These revisions are transferred and stored online. MDOT began its e-Construction effort in October 2012 by running three pilot construction contracts totaling $100 million. One piece of MDOT’s e-Construction platform is a commercial off-the-shelf engineering document management (EDM) solution. The DOT had been using this solution for 10 years for project design and development before applying it to construction. Beginning with pilots, workflow for electronic bidding and construction management (including contracts, invoices and digital signatures) was set up in the EDM system. The DOT estimated total savings of $300,000 for one pilot based on reduced paper, printing, mail, and staff costs. The pilots gave the DOT an opportunity to define file naming conventions and management protocols for documents ahead of time. This allowed them to simplify pre-existing methods involving multiple subfolders for document filing. The naming conventions and standards were important for achieving consistency and avoiding confusion and operator error when the system was expanded. Following the success the initial pilot projects, the system was rolled out statewide. 100 percent of MDOT construction projects initiated since October 2014 are paperless and a majority of all other projects (initiated prior to October 2014) are paperless as well. MDOT is currently rolling out e-Construction for local projects. Currently $1.5 billion of paperless construction projects are ongoing at

Case Studies 47 MDOT; the only paper documents still used in construction projects are delivery tickets for concrete, aggregate, and hot mix asphalt. E-Construction was a success at MDOT because of strong leadership support. The COO at the time, Greg Johnson, issued an edict to implement e-Construction within one year. This prioritized e-Construction and cut through the bureaucratic process. MDOT describes going paperless as “a game changer,” speeding up the construction process, reducing costs, and increasing transparency and accountability. Previously, documents had to be faxed or sent to many locations for signatures with many time-intensive steps. Use of digital signatures reduces document approval times significantly; what used to take weeks can now be done in a single day. Additionally, rather than having multiple paper documents in disparate physical locations with no version control, electronic documents are in one location and audit controls are in place. MDOT has deployed engineering document management as an enterprise solution across divisional lines rather than fragmenting each division with individual siloed approaches. The system benefits MDOT, consultants, and contractors. Each of the three groups can access documents anytime and anywhere, including from the field, allowing for better allocation of staff time. E-Construction also saves document storage space and improves findability. Previously with paper documentation, materials and payments files were sometimes lost. Keeping electronic files reduces this risk and improves the transparency and accountability of all users. E-Construction has improved access to files, maintaining online version control and minimizing the chance of lost files. Document storage in the EDM system allows for unimpeded flow of data and facilitates information flow between collaborating entities. The online file management system also helps ensure that documents are current and available. These benefits reduce the risk of document loss and potential legal liability. Implementing e-Construction was a drastic change, and MDOT faced a few challenges, including providing access to documents beyond the MDOT firewall and getting contractors to adopt digitally encrypted signatures. In addition, there was a learning curve for office technicians charged with database administration. Identifying and Learning Lessons While e-Construction provides valuable support for information transfer, it does not address all issues related to knowledge transfer. For example, in one $50 million reconstruction project, the MDOT design specification indicated that an item—underdrain removal—was included in the construction bid. However, MDOT had paid the construction firm separately for underdrain removal in three other projects. This type of inconsistency from MDOT exposes the agency to possible litigation and associated costs. One suggestion for reducing this risk from inconsistency is to expand the MDOT practice of holding post-project debriefings. MDOT is more likely to engage in knowledge-capture activities for bigger, more complex projects. For typical projects, knowledge is usually not shared beyond the project team due to time constraints. Occasionally there are post-construction or post-design reviews. During post-construction meetings that include consultants and other stakeholders, the agency tries to capture what went well and what went poorly during the project. The process is defined and outlined in an MDOT memo, which includes a meeting checklist. However, post-construction meetings are only required for MDOT trunkline projects, leaving other MDOT projects without a review meeting protocol.

48 Keeping What You Paid For—Retaining Essential Consultant-Developed Knowledge Within DOTs Prior practice at MDOT was to hold a meeting at the end of every project to review and evaluate the job. The meetings were distinct from the current practice of post-construction meetings and included staff from DOT, consultants, and contractors. Topics included what worked, what didn’t work, and what problems were resolved. The meetings, which no longer occur, had no defined product. In the future, MDOT would like to hold post-project debriefings to document lessons learned. These meetings and the resulting documentation could be a way to share lessons learned. MDOT has a robust relationship with its consulting partners, and that relationship has been formalized through a partnership agreement the DOT signed with the American Council of Engineering Companies of Michigan (ACEC). The charter spells out how the agency and consultants work together, communicate, and develop teams and work groups. The document was most recently updated in January 2013 and includes a mission statement, goals, and teaming efforts. The DOT and ACEC operate several committees: Contracts and Finance, Training, and Program and Procedure. The committees are composed of four members from MDOT and four members from ACEC. The committees seek to find common ground for the consultants and DOT. MDOT is currently discussing putting together an ACEC user group and a Michigan Infrastructure and Transportation Association user group to continue to improve e-Construction practices. Sources http://mdotwiki.state.mi.us/construction/index.php/E-Construction http://www.michigan.gov/documents/aero/2014-03-20_e-Construction_for_STC_- _Cliff_Farr_452732_7.pdf

Case Studies 49 Case 2: Construction Project Scheduling Needs: Building internal expertise in construction scheduling Techniques: Use of consultants to develop staff expertise Combining classroom training and on-the-job training and mentoring The Context Until recently, MDOT did not have internal construction scheduling expertise. MDOT decided to add a critical path method (CPM) specialist to their staff in response to FHWA’s increased emphasis on scheduling. However, the DOT encountered difficult filling the position in part because it required a very specialized and uncommon skillset and also because private-sector jobs generally pay more for the same specialty. After failing to attract a deeply experienced staff member, MDOT decided to hire an employee that had excellent construction field-service experience but was not previously focused on CPM network scheduling. A national consultant specializing in scheduling was hired to assist with the development of the new engineer. Brad Daavettila joined MDOT in 2015 as a Construction Staff Specialist in the Construction Field Services Division. As the scheduling specialist, his role is to provide oversight and coordinate scheduling for complex MDOT projects. Developing this internal scheduling capability within MDOT is expected to reduce the risk of knowledge loss associated with consultant turnover. Use of Consultants Michigan DOT has long relied on consultants to provide construction project scheduling services via an on-call contract. In February 2016, around the time that the decision was made to hire the scheduling specialist, MDOT’s contractor, HNTB, began work under a new contract to provide training to the new specialist, which includes both formal instruction as well as on-the-job mentoring. The training covers processes, software, reviews, and more. Critical Knowledge Areas Construction scheduling, specifically CPM scheduling, is used to make projects more efficient, trim costs, prevent budget overruns, and improve on-time delivery. CPM scheduling is a difficult skill to learn and requires mastery of the industry standard CPM scheduling software package This case study illustrates how MDOT used consultants to build internal construction project scheduling expertise. When it proved difficult to hire a highly skilled sche duling expert, MDOT decided to hire someone without in-depth scheduling expertise and train them. One of the consultants providing scheduling services to the agency was engaged to provide training and mentoring. The consultant also provides basic training on scheduling to construction engineers across the state to further develop and broaden the agency’s capabilities.

50 Keeping What You Paid For—Retaining Essential Consultant-Developed Knowledge Within DOTs Knowledge Capture and Transfer Mentoring and Interaction The training and mentoring provided by the consultant offer a mix of passive and active learning as well as transfer of explicit (documented) and tacit knowledge. Classroom instruction and associated documentation covers transfer of procedural “how to” knowledge of scheduling workflow and software use. Exercises provide opportunities for application of this knowledge. During the course of the one-on-one training, the consultant is also imparting tacit knowledge about how to approach scheduling for different types of projects and how to deal with specific situations. Finally, the mentoring elements of this engagement provide the opportunity for Mr. Daavettila to further cement his knowledge by observing and participating in the consultant’s scheduling activities. This combination of methods—combining classroom training, exercises, and ongoing mentoring—provide a complete learning experience. Formal training is delivered in half-day sessions, twice a week, for seven weeks. The lesson plan includes instruction on use of the scheduling software with exercises involving simulated project examples. During the course of the training, the consultant, who has more than 15 years of experience with CPM scheduling, answers questions drawing from personal experience in project scheduling. Course materials provided include a 200-page binder of reference materials related to the scheduling software. This resource functions as a custom software manual and contains screenshots and specific “how to” directions. The formal training is supplemented with on-the-job mentoring. Mr. Daavettila is engaged in the scheduling services provided by HNTB and learns observationally from their work. As a project progresses, the consultants point out good and bad parts of schedules, allowing Mr. Daavettila to learn from their experience. Through the mentoring, Mr. Daavettila is being trained as if he were a new hire at HNTB. The training and experience he gains through this process will allow him to provide scheduling services across Michigan. The MDOT central office provides scheduling services for rural areas throughout Michigan. Because a rural area might only need a schedule for a big project every few years, consultant-provided scheduling creates the risk that the central office will not have historical knowledge due to consultant turnover. Having an MDOT staff member such as Mr. Daavettila provide scheduling services aids continuity and prevents knowledge loss. MDOT employs another scheduling specialist, Aaron Raymond, in the Metro Region office. The Metro Region includes Detroit and is host to many of the large and complex projects that require CPM scheduling. Mr. Raymond is responsible for providing scheduling services within the Metro Region, while Mr. Daavettila works across all of Michigan’s regions. Maintaining two CPM specialists offers a degree of knowledge security to MDOT. If one of the two specialists were to leave the agency, MDOT would maintain its internal scheduling capabilities. This redundancy is necessary because MDOT has limited contractual mechanisms available to obligate or entice talented or skilled staff to stay at the agency. Michigan DOT is also strengthening core competencies by using a consultant to provide basic education on scheduling for construction engineers across the state. The agency holds regular one-day scheduling training sessions. The sessions are held for northern and southern Michigan and are led by a consultant with CPM expertise. The one-day sessions are not comprehensive training classes but are intended to elevate the general competency and comfortability with CPM for MDOT construction engineers.

Case Studies 51 Tennessee DOT Case 1: Local Programs Office Needs: Transitioning functions from substantial outsourcing back to agency employees Building staff capabilities Ensuring that agency staff can access critical information about prior (outsourced) activities Techniques: Use of consultants on a temporary basis for technical and professional positions for maintaining work continuity during a hiring freeze Maintaining notes on “what was done and why” so that in the future others can revisit those notes and understand the basis for actions taken Use of project logs to ensure continued access to information about decisions made by the consultants once they are gone Retaining consultant email archives to enable future research about past activities once consultants have completed their engagement with the agency The Context In 2009, the Tennessee Department of Transportation (TDOT) Local Programs Office was feeling the effects of a TDOT hiring freeze brought on by the recession. The American Recovery and Reinvestment Act (ARRA) came into effect the same year and included over $48 billion in federal funds designated for transportation projects. The money funded many local projects across the state, straining the human resources of the Local Programs Office. Overwhelmed by the surge in funded projects but unable to hire new staff, TDOT began using on-site consultants to help with the overflow workload. As the economy strengthened and the recession faded, TDOT lifted its hiring freeze and encouraged offices to replace consultants with full-time staff. The Local Programs Office, which had been one of the offices to rely heavily on outsourcing during the hiring freeze, began hiring staff to replace the consultants in 2014. Use of Consultants The Local Programs Office began hiring consultants in 2009 to supplement TDOT staff. Three companies were contracted to provide on-site services to the office: Ragan-Smith provided two consultants, Fisher Arnold provided one consultant, and TranSystems provided one consultant. The consultants were selected based on project management experience and knowledge of locally-owned This case study is the first of two examples showing how Tennessee DOT transitioned a temporarily outsourced function back to the DOT. While this example may not represent a common situation, it does illustrate several simple and practical techniques for record keeping by the consultants. Application of these techniques provided the continuity and knowledge base needed for DOT staff to understand what was done on several different projects and why.

52 Keeping What You Paid For—Retaining Essential Consultant-Developed Knowledge Within DOTs projects. The consultants, who worked on-site at TDOT, were expected to have limited responsibilities at the Local Programs Office. However, as the office’s workload continued to grow and TDOT’s hiring freeze persisted, the responsibilities of the on-site consultants grew to include project management, grant management, communication, invoice review, and document exchange in both design and project construction. The on-site consultants were treated as though they were TDOT employees: working in TDOT office cubicles, attending daily meetings, and given full access to the project management database and the accounting system. In 2010, one year after the surge in outsourcing began, 13 percent of the Local Programs Office comprised on-site consultants. TDOT used two manuals to ensure consistency in documentation with the consultants. One, the Local Government Guidelines Manual, is updated by the Local Programs Office every two years and is a handbook for local governments. The second, The Planner Manual, is an internal manual that includes screenshots of the programs used by TDOT Local Programs Office staff. The on-site consultants used both manuals frequently. Hiring four consultants into Local Programs at TDOT worked well, despite the fact that consultants reported both to TDOT management and their parent companies. There was minor tension in the office due to the feeling on the part of TDOT employees that consultants might be taking state jobs away from staff members. However, staff understood that there was a hiring freeze, and the complete integration of the on-site consultants resulted in positive and productive work relationships. One consultant broke the ice by regularly bringing donuts to the office. Using consultants in this situation gave TDOT a measure of flexibility, allowing the agency to absorb extra workflow on relatively short notice. The ability of consultants to work for variable contract lengths and prices also provided financial flexibility. The consultants could “hit the ground running” because they brought the requisite skills and experience to perform needed functions. In contrast, new TDOT hires typically have minimal experience or background knowledge and cost time and money to be trained. Critical Knowledge Areas The Local Programs Development Office manages and provides oversight for state and federal aid for local projects. The projects administered by the Local Programs Office include bridge replacement, intersection lighting, safety improvements, sidewalk improvements, and road construction or reconstruction. The Local Programs Office has more than 600 active projects, an increase of 100 projects over the past three years. Knowledge Capture and Transfer Mentoring and Interaction/Identifying and Learning Lessons The on-site consultants remained under contract during the hiring of new TDOT staff. The consultants were tasked with easing the transition by mentoring the new employees for two months. New staff were seated at the same desk as their respective mentors and the new TDOT employee and the consultant would go through plans, projects materials, and emails together. This arrangement continued until the TDOT employee was able to work independently. The consultants and new TDOT staff also participated in lessons learned/best practices meetings where consultants passed on their knowledge and experience. By 2015, all on-site consultants had been phased out at the Local Programs

Case Studies 53 Office. However, TDOT continues to tap into the knowledge and experience of one of the previous on- site consultants when needed. Documentation and Handoffs During their engagement, the consultants tracked all of their work activities on project management spreadsheets and their emails were maintained in an archive for future reference. The spreadsheets contained data on all of the Local Programs Office’s projects. Data fields included budget, project dates, milestones, and a field for comments. The on-site consultant from TranSystems used the comment fields frequently, taking notes on project discussions. In summarizing the daily conversations about the projects, she created a written log of major project decisions complete with reasoning and alternatives. Five years later, the comments are still being used to understand TDOT projects and the decision making that shaped them. The comments have proved to be invaluable when questions have come up about a particular decision. The spreadsheet notes present the decision-making process, translating project knowledge and experience of “what was done and why” into written form. TDOT Local Programs staff have adopted some of the documentation methods practiced by the consultants. For example, state employees now track milestones in a TDOT spreadsheet program, which was not a standard practice previously. While these knowledge transfer practices were not included in the consultants’ contracts, they did help ease the transition of new staff, preserve a clear record of project decisions, and minimize the chance of repeating mistakes in the future. References https://www.tn.gov/tdot/section/local-programs http://www.gao.gov/products/GAO-11-600

54 Keeping What You Paid For—Retaining Essential Consultant-Developed Knowledge Within DOTs Case 2: Environmental Division Needs: Transitioning functions from substantial outsourcing back to agency employees Building staff capabilities Techniques: Utilizing consultant services to codify complex knowledge in the form of comprehensive training materials that can be maintained and updated by the agency Comprehensive, multi-dimensional, immersive training combining classroom instruction, independent assignments, individual mentoring, group evaluation, and hands-on experience working on NEPA documents. The Context The Tennessee Department of Transportation (TDOT) Environmental Division had a large number of staff who started their careers together in the 1980s. As that cohort began to retire in the 2000s, the division faced a significant loss of staff, experience, and institutional knowledge. The wave of retirements coincided with the recession, which prompted a hiring freeze. In response to the recession, the federal government passed the American Recovery and Reinvestment Act (ARRA) of 2009, a stimulus bill that included over $48 billion designated for transportation projects. The increase in funding due to ARRA and new projects strained the human resources of the Environmental Division. Within the Environmental Division, TDOT’s NEPA office sees a wide variety of projects, including small projects such as road sign placement or larger projects such as major Interstate widening. Each new project needed oversight to ensure compliance with NEPA regulations. Use of Consultants Facing a growing pipeline of projects and restricted by a hiring freeze, the Environmental Division hired consultants to supplement their staff. TDOT experienced multiple years of the hiring freeze and the resultant growth of outsourcing. By 2012, Environmental Division staffing levels were down 40 percent from normal; consultants made up the deficit. The division has six technical offices with specialized staff: Archeology, Historic Preservation, NEPA, Hazardous Materials, Air and Noise, and Ecology. Following the election of a new Governor in 2011, TDOT underwent a top-to-bottom review of potential process improvements. Based on this review, a decision was made to restore the NEPA office staff but use the current consultants to train the incoming TDOT staff. This is the second of two case studies that illustrates how Tennessee DOT was able to facilitate an orderly transition back from outsourcing a key DOT function. This example concerns the agency ’s National Environmental Policy Act (NEPA) office and highlights how the consultants performing the NEPA function were used to develop and deliver a comprehensive training program for the incoming DOT staff. This was a valuable strategy given the complexity of this area, the steep learning curve, and the fact that it is becoming more difficult to recruit and retain staff with this type of expertise.

Case Studies 55 When the Environmental Division began hiring again, TDOT staff were filling only 2 of 14 positions in the NEPA office and consultants carried the rest of the workload. Skilled and experienced NEPA staff proved hard to recruit, necessitating the DOT to bring on untrained staff. The Environmental Division hired a large group of NEPA staff in a short period of time and tasked the consultants with developing a training manual for the new hires. The new staff members had technical degrees but minimal levels of NEPA knowledge. The DOT, operating under an on-call contract with Parsons Brinckerhoff, issued a task order for the development of NEPA training materials for the new employees. By November 2015, the NEPA office was fully staffed with 14 employees. Two are supervisors responsible for assigning projects, managing workload, and reviewing NEPA documents. The other 12 are responsible for writing documents. The current head of the NEPA office was previously a consultant to the NEPA office. While there are still consultants at the Environmental Division helping mentor the new hires, TDOT anticipates that staff will soon gain the necessary experience to operate independently. Critical Knowledge Areas The Environmental Division is responsible for ensuring compliance with environmental legislation for transportation projects. The National Environmental Policy Act of 1969 requires the preparation of Environmental Assessments and Environmental Impact Statements for all projects proposed by federal agencies. The TDOT NEPA office is responsible for producing all required documentation related to environmental impact. Knowledge Capture and Transfer Documentation and Handoffs The consultants developed an extensive set of NEPA training materials based on the knowledge and experience of existing NEPA staff, supplemented by the consultants’ expertise. The training materials consisted of presentation slides, a 287-page NEPA guide, example spreadsheets, FHWA documents, and other supporting files. The presentation slides were designed to be used as manuals after the training as well. TDOT intends to update the training materials as needed in the future. TDOT’s approach to working with in-house consultants also implicitly supports smooth handover of documentation gained during the course of the engagement. In-house consultants at TDOT work as if they are TDOT employees. They are plugged into all TDOT systems, keep documents on TDOT databases, and adhere to TDOT file structure policies. Consultants who work out of branch offices have to provide weekly spreadsheets documenting project progress. This standard approach was supplemented and strengthened through a project closeout process introduced by one of the consultants. This process ensured the DOT would have full access to both formal and informal documentation produced during a project. Upon completion of a project, the consultants submit a master file with emails and work products to the client. (This was the consultant’s policy and was not required by their TDOT contract.) This documentation handoff process worked well at TDOT and was particularly useful given that agency staff emails are deleted after six months if they are not saved to a shared drive. In fact, part of the NEPA training involves teaching trainees to properly manage their email inboxes.

56 Keeping What You Paid For—Retaining Essential Consultant-Developed Knowledge Within DOTs Mentoring and Interaction The transition from outsourced to TDOT staff-led services was facilitated by an extensive training and mentoring program. The consultants (Parsons Brinckerhoff) developed and led a two-week training program based on the NEPA services they had been delivering for TDOT. The program was a comprehensive NEPA walkthrough and consisted of classroom instruction and mentored document training. Every new hire in the NEPA office participated in the two-week training course. The training was modeled after the organization of NEPA and broken down into technical areas. There was a full day of training focused on each technical area: the substance of the work, relevant regulations, how to complete the work, and how to produce the required documents. The training program was an immersive two-week experience led by the consultants. The mornings were spent in classroom style instruction, using a series of presentations to walk through NEPA documentation. The afternoons were spent on assignments for which trainees worked on sections of active documents and submitted their work to the instructors for evaluation. Prior to the training, the NEPA office staff had made the necessary technical requests and gathered data for the NEPA documents. This allowed the trainees to work on documents in a compressed time frame without waiting for data. (Trainees also learned to request data, but completed those documents after the training.) By the end of the two weeks, each trainee had completed a NEPA document for TDOT. The projects addressed in training were simple categorical exclusions, which represent the majority of work for the TDOT NEPA office. Environmental Assessment and Environmental Impact Statement work is handled by a separate office. The group also participated in a review meeting every morning. The instructors would cover the previous day’s assignment, highlighting areas that were difficult for the trainees and offering suggestions for improvement. The feedback delivered during the recap meetings was beneficial for the trainees. The recap meetings were not formally documented. While the training was very helpful, both the consultants and the staff acknowledge that a two-week training course cannot replace the in-depth understanding of the organization that is gained through experience. One consultant noted, “After dealing with federal funding issues for five years, a person internalizes the nuances of FTA or FHWA rules. It is difficult to transfer that kind of knowledge. On-the- job training is the best solution.” The training was successful because it was multi-dimensional, combining classroom instruction, independent assignments, individual mentoring, group evaluation, and hands-on experience working on NEPA documents. Instructors taught using examples drawn from their own NEPA experience. The daily review meetings helped foster knowledge transfer by having the instructors provide feedback on the trainees’ work and explain best practices. The consultant who developed the training noted two lessons for the future. One is that additional individual (as opposed to classroom) instruction would be beneficial given the difficult nature of the NEPA materials. Second, the consultant noticed that two weeks of full-time training was draining on the employees. Trainees began to get overwhelmed in the second week. Expanding the training to four weeks and reducing the weekly workload could improve knowledge retention.

Case Studies 57 Alberta Transportation Case: Highway Design Specifications Needs: Maintain standardization and continuity while outsourcing design and construction oversight Improve specifications over time to reflect experience gained Techniques: Detailed project documentation requirements at the design and project closeout stages to provide a record of key decision points that impacted final design and cost Detailed tracking of design exceptions Extensive face-to-face project team interaction involving the agency, designer, and contractor Use of topical workshops involving public- and private-sector participants for collaborative problem solving on technical issues of concern Project closeout meetings to identify what went well and what could be improved Periodic updates to specifications to update standards based on project experience; review of proposed specification changes by both internal staff and external consultants and contractors Context Alberta Transportation (AT) is the transportation ministry for the province of Alberta in Canada and is responsible for transportation assets including a 31,000+ kilometers highway network. In 1995 and 1996, a political decision was made to downsize the agency. Seventy-five percent of the staff was laid off—the agency was reduced from a staff of approximately 3,600 to 600. AT lost a significant amount of internal capacity and transitioned to an operating model that is heavily reliant on outsourcing. The agency maintains a small team and relies on consultants for design, construction oversight, maintenance, and operations. The agency has retained a strong central function [the Technical Standards Branch (TSB)] for development and maintenance of technical standards. The agency’s evaluation and application of High Tension Cable Barrier (HTCB) over the past decade provides an illustration of how AT leverages consultant and contractor expertise to develop and refine its standards. AT first began exploring the idea of using HTCB in 2004. Its first substantial HTCB project was on a 10.75 km section of Deerfoot Trail, which is a six-lane divided urban freeway with a depressed median in Calgary, Alberta, carrying 150,000 vehicles per day. This project was initiated in response to the high number of fatal cross median collisions in this section of highway. It was completed in 2007. A This case study illustrates how a transportation agency that has adopted a highly outsourced operating model has established a variety of mechanisms to capture and leverage consultant and contractor experience for continued improvements to agency practices. These techniques combine a strong centralized standards function, detailed documentation requirements, clearly established roles and responsibilities, emphasis on collaboration, and use of consultant expertise to inform updates to standards.

58 Keeping What You Paid For—Retaining Essential Consultant-Developed Knowledge Within DOTs second, much larger project was initiated in 2008 and completed in 2010 on a 133 km section of Highway 2. Highway 2 is the primary north-south highway in the Province that connects Edmonton to Calgary, which carries close to 200,000 vehicles per day. Use of Consultants EBA, a Tetra Tech Company, was selected to provide engineering services on both HTCB projects. These services consisted of preliminary engineering, conducting research on available products, design, preparation of the tender package, contract administration, construction supervision, and completion of final details. Through a competitive bid process, Volker Stevin was selected contractor for the first project; Alberta Highway Services was the construction contractor for the second, larger project. Critical Knowledge Areas HTCB systems were developed in Europe in the 1980s but not applied in North America until the 1990s. Interest in HTCB as a means of controlling severity of median cross-over crashes and run-off-the-road crashes grew in the 2000s. Over the past 15 years, highway agencies and the consulting and contracting communities have been learning more about identifying appropriate locations for HTCB, proper placement, design elements, performance characteristics, maintenance considerations, and cost-effectiveness. AT’s design guidelines cover a variety of topics including face-to-face maximum deflection and post spacing, run length considerations, placement of HTCB in the median and on the roadside, placement on curved alignments, end treatment or anchors, overlapping barrier systems, and geotechnical investigations. Knowledge Capture and Learning Mentoring and Interaction AT project delivery staff in the regions have regular interaction with consultants and contractors to monitor progress and ensure adherence to policies and standards. AT considers face to face interaction essential for a successful project. During the design phase of a project, there are multiple review meetings and AT sign offs on the final design and tender package prior to advertisement. During the construction phase, there are biweekly construction meetings with the contractor and AT sponsor. Where issues arise, they are addressed collaboratively. For example, AT raised a question about optimal placement of the barrier for the Highway 2 project. In response, the consultant and AT convened a workshop to discuss placement issues. Following the workshop, the consultant drafted a memo summarizing the key concerns and recommendations. As part of the Highway 2 project, the consultants developed a training program for first responders along the corridor and delivered the training through a collaborative effort with AT. The training covered the basics of HTCB, how it works, what to expect when the barrier is struck, and recommended strategies for barrier repair. For example, the consultants explained that in most cases barriers that have been hit are still functioning effectively, so they should not be cut following an incident.

Case Studies 59 Documentation and Handoffs Standardization of project documentation ensures that pertinent details are recorded and available to AT staff as well as other consultants and contractors. Design Reports. AT receives a design report from the consultant at the end of the design process. Due to budget fluctuations, projects that have completed the design phase are occasionally put on hold for years until funding is approved. This potential delay makes it important to transfer and store design decisions so that critical knowledge is not lost between design and construction. Project Summary Reports. A project summary report is produced at the close of the construction phase. This report includes a financial statement detailing project engineering and construction costs, a breakdown of pay item quantities placed, a discussion of design issues faced and changes made, a chronology of construction activities, a listing of construction change orders, and a discussion of traffic and safety experience during construction. The report for the Highway 2 HTCB project included several detailed appendices with the traffic accommodation strategy, the environmental construction operation plan, site maps, weekly and monthly reports, meeting notes, correspondence and photographs. Identifying and Learning Lessons At project closeout, a tri-party meeting with AT staff, the engineering consultant and the contractor is held. These meetings provide an opportunity to review what went well and what could be improved. Specific topics include project team effectiveness; AT, consultant and contractor relationships; communications, contract administration, workmanship, and quantities/change orders. During the closeout meeting for the HTCB project on Highway 2, several suggestions for future improvement were made. For example, the contractor suggested increasing length of work zones to allow more flexibility on lane closure times. In addition, adding a grading component to future contracts was discussed in order to improve anchor locations and drainage. While lessons learned are intrinsically discussed, AT reports that the agenda template for these meetings is being revised to include a more explicit focus on lessons learned. AT’s project delivery unit is currently developing a website for lessons learned. The intent is that lessons will be captured and vetted by project delivery staff that work on individual projects. Design Specifications. In order to shape projects and properly direct consultants, AT has developed a robust design guide and set of engineering specifications. These specifications serve as a key knowledge base for the agency and are updated over time as experience is gained. For HTCB, AT relied heavily on their consultants to draft specifications. The interactive process of specification development provided an opportunity for AT staff to learn from the consultants. Now that these specifications are in place, AT has what they need to manage future HTCB projects, although they can continue to draw upon available consultant expertise as needed. At the time of the initial project design and installation, there were no AT guidelines for HTCB. The consultants on the HTCB projects worked with AT to develop comprehensive engineering specifications to guide future similar projects. These specifications were developed through a collaborative and iterative process involving input from the consultant, internal review within AT from both design and maintenance staff, and review/comment from the broader stakeholder community—including consultants and contractors. Throughout this process, participants were encouraged to identify issues

60 Keeping What You Paid For—Retaining Essential Consultant-Developed Knowledge Within DOTs Changes to design specifications at AT are issued in the form of design bulletins and are subsequently incorporated into master specifications documents (the Roadside Design Guide, in the case of HTCB). Once design specifications are approved by the TSB, they are posted to AT’s website and distributed widely to the stakeholder community. The distribution list includes AT staff management committees (Construction, Operations, Bridges, Infrastructure) and industry groups representing consultants (Consulting Engineers of Alberta) and contractors (Alberta Roadbuilders and Heavy Road Construction Association). Studies. Because of AT’s limited experience with HTCB, they commissioned the consultant to conduct studies of both projects to assess the safety performance of the barriers installed, based on before and after analysis of crash data. These studies also evaluated construction and maintenance costs and estimated benefit/cost ratios. They included input from maintenance personnel on HTCB performance and impacts on maintenance needs and activities. Results of these studies informed development of the design and installation guidance. Design Exceptions. AT views their design exception process as an important mechanism to document situations where projects have deviated from standard specifications. It is also viewed as a way to identify situations where updates to specifications are needed. The current design exception process was formalized in 2010. Design exceptions must be approved by the project sponsor, the head of project delivery, and the director of the TSB. Design exception requests must include a summary of the project, a description of the exception being requested, the applicable specification or guideline, and the justification or rationale (risk assessment, benefit/cost, options considered, mitigations proposed). If the exception request is based on another existing standard (e.g., from AASHTO or another province), a reference to that standard is provided. Approximately 20-30 design exceptions have been accepted in the past 5 years (fewer than 5 percent of projects have design exceptions). When accepted, the full documentation of design exception is maintained in the design files or a planning report so that it can be retrieved in the event that a legal claim is made on the project. The full documentation of design exception protects against that liability by explaining the decision making for that design. AT maintains a list (with links) to current design exceptions on their web site. References https://www.transportation.alberta.ca/5479.htm https://www.transportation.alberta.ca/Content/docType5479/Production/Summary_Report.pdf https://www.transportation.alberta.ca/Content/docType5479/Production/High_Tension_Cable_Barrier Performance_Evaluation%20_Summary.pdf https://www.transportation.alberta.ca/Content/docType233/Production/DB75.pdf http://www.transportation.alberta.ca/4921.htm and challenge proposed language. The specification development process provided an opportunity for AT staff to engage in active learning about HTCB design and installation considerations. Initial specifications were developed in 2012 and were subsequently updated in October 2016 to reflect new experience and input from contractors. For example, the 2016 specifications refined the list of design criteria, included a new section on placement of HTCB on undivided highways to accommodate wide loads, and clarified situations in which geotechnical testing and site-specific engineering design was required. (The 2012 guidelines had required these investigations for all projects.)

Case Studies 61 Philips Innovation Services Case: Knowledge Sharing to Drive Innovation Needs: Build and sustain knowledge to drive innovation Reduce risks associated with over-reliance on individual experts Techniques: Structured processes for knowledge capture and learning that can be adopted for use at state DOTs, including the following: o Conducting a workshop at the start of a project to learn from previous mistakes and prepare a team for potential issues before beginning the project o Conducting exit interviews with departing experts—both consultant and staff—to capture essential knowledge o Using a “knowledge drive” to create training material The Context Philips is a Netherlands-based company offering a diversified set of products and services in the healthcare, personal health (consumer products), and lighting industries. Philips Innovation Services (PInS) is an organization at Philips that provides engineering and consultancy services and specializes in complex electronic systems. PInS offers its services within Philips as well as to external clients. There is a 60-40 split between work for internal Philips business units and external industrial customers. PInS employs 1,000 people and has an annual budget of $200 million. Typical project engagements range in length from 6 weeks to 18 months. One of the services provided by PInS is assistance with knowledge capture and transfer. PInS helps its clients implement these techniques to support innovation and reduce dependence on “aging knowledge holders.” PInS also applies knowledge management methods internally to capture knowledge from departing experts—both internal staff and consultants. Internal application of knowledge management techniques helps PInS develop and maintain the skills they need to satisfy client demands. This case study from the private sector illustrates several structured techniques for knowledge capture and learning that are applicable within a DOT environment. The same techniques are used to capture knowledge from departing experts who have been engaged as consultants and those who have been employed by the organization. Knowledge capture and learning techniques are not highly sophisticated but do require discipline, a standardized process, and designation of one or more staff members who are responsible for applying them. *This case study is not an endorsement of this company by the Transportation Research Board or by the National Academies of Sciences, Engineering, and Medicine.

62 Keeping What You Paid For—Retaining Essential Consultant-Developed Knowledge Within DOTs Use of Consultants PInS provides consulting services to Philips. In addition, PInS itself utilizes consultants to supplement its own staff. PInS Use of Consultants. PInS outsources 10-15 percent of its work, relying on external consultants to address staffing shortages or to cover specialized competence gaps. Recently, consultants have been engaged to work in new business areas in which PInS does not have established expertise. The consultants report to senior directors of specific departments at PInS. Consultant contracts are overseen by Yacht, an external placement agency. Philips Use of PInS Consulting Services. Philips draws upon the consulting services offered by PInS to support the R&D process in their healthcare business. Critical Knowledge Areas PInS supports the research and development process in several areas, including the following: Medical devices & equipment High-precision equipment Connected digital products & systems MEMS devices & micro-assembly Manufacturing processes & systems Design for reliability solutions Industry consulting Environment & safety Because of the focus on innovation, PInS needs to maintain (1) highly specialized technical expertise in a diverse set of industries and technologies; (2) practical experience in product and process development; and (3) advisory, consulting, and strategic planning skills. Knowledge Capture and Learning PInS has identified three types of knowledge as important for transfer: Expertise (which enables you to do your job professionally) Organization (“Frank knows about these kinds of problems”) Network (knowledge about one’s environment, business contacts, etc.) PInS identifies essential knowledge for transfer based on three criteria: Necessity (indispensability, added value for the organization and its customers) Particularity (distinctiveness when compared to competitors, the difficulty for others to copy) Sustainability (is the knowledge future-proof) A variety of knowledge transfer techniques are applied, depending on conditions such as the cooperation and preferences of the expert(s), the willingness of the knowledge recipient(s), and the time and opportunities available for knowledge transfer. Techniques may involve knowledge transfer from one individual to another (e.g., a mentoring or apprenticeship arrangement), from an individual to several people in a work group, or across multiple individuals.

Case Studies 63 Documentation and Handoffs/Mentoring and Interaction PInS has developed a structured process for knowledge capture and learning involving the following techniques: Mentoring: This involves pairing up experts and engineers for tacit knowledge transfer. This technique is used for “tacit to tacit” knowledge sharing. It is recognized as having a limited effect because, at best, it creates a second single expert rather than a broad, workgroup-wide understanding of a knowledge area. Knowledge Drive: Sprints: Tacit knowledge for a particular set of topic areas is captured in a series of “sprints.” Each sprint involves one or more experts working with a knowledge engineer who interviews them and records different knowledge elements in detail. Depending on the knowledge area and element being addressed, a sprint might take 3-6 weeks to complete with a time commitment of two hours per week. Following completion of the sprint, the knowledge engineer provides a draft document with the results for review by the expert(s). As appropriate, the document is reviewed by others and then finalized. A sample template for documenting knowledge elements is provided in chapter 5. Knowledge Drive: Masterpieces: Results of multiple sprints are integrated into a “masterpiece” in which the material is packaged in a manner to enable learning. Once finalized and approved, the masterpieces become the backbone of a training curriculum for new hires. The masterpieces may include multiple formats such as documents with hyperlinks, videos, and formulas. Departing Expert Interview: This is a 2-4 hour workshop designed to capture pertinent knowledge of an employee expert leaving the company or transferring to a different business unit or a consultant who is concluding their engagement. The line manager responsible for the specific knowledge area is responsible for planning and conducting the interview. In addition to the line manager and the expert, participants include the expert’s successor and other members of their work group or project team. The interviews follow a formalized procedure, beginning with a discussion of the position/role and associated tasks. Following this introduction, specific questions are asked about how the work is performed, what information and knowledge sources are used, and where work products are stored. The results of the interviews are written up and archived for future reference. A sample Departing Expert Interview Report format is provided in Appendix B. Communities of Practice. PInS also uses communities of practice (CoPs) to foster knowledge sharing. Some CoPs are established through company-sponsored initiatives (top-down) while others develop organically (bottom-up). There are 20 company-sponsored CoPs and 60-80 others. Communities may form around a particular technology or component (e.g., magnets), process (e.g., product architecture), or technique (e.g., creating value propositions). Construction and logistics contractors may be invited to participate in the communities as appropriate. Wiki: An internal wiki is used to store the captured knowledge documents for ease of access and updating. The sprint-masterpiece technique was applied by PInS consultants within the Philips Magnetic Resonance (MR) unit. This unit had a number of lifelong experts nearing retirement and needed a way to capture the expertise so that it could be retained and shared. The sprint-masterpiece approach was particularly helpful given the high degree of complexity in MR systems, which is comparable to that of

64 Keeping What You Paid For—Retaining Essential Consultant-Developed Knowledge Within DOTs an airplane. With multiple integrated subsystems requiring specialized knowledge, no single individual had an understanding of the entire system. Thus documenting MR knowledge required breaking down the whole into multiple component parts. Identifying and Learning Lessons Several techniques are used to identify and learn lessons—these involve both internal staff as well as consultants: Opportunities for Improvement Workshop: The project team is assembled at the close of a project to review what happened and what could have gone better. The focus is not on finger- pointing or finding fault with individuals but on identifying ideas for future improvement. Strong facilitation is important to encourage a frank and open discussion. PInS has trained several facilitators and has developed a rule book on how to conduct these workshops. Results are posted on an internal wiki for easy access. Project Start Challenge Workshop: The project team is assembled at the start of a project to identify the likely issues and complexities that may be encountered and discuss how they will be approached. Material from prior similar engagements (including what was identified at the Opportunities for Improvement Workshops) is used where applicable. Change Logs. During larger projects, change request logging is used to document deviations from original plans. The project baseline and milestones are checked every two weeks, and changes are registered. Various systems are used to store the change logs (e.g., Microsoft SharePoint). If an external consultant is involved in the project, they work with a Philips liaison to register relevant information.