Strategic Program Delivery Methods (2017)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

25 INTRODUCTION This chapter discusses project and program delivery performance in the transportation industry. The information is taken from a national survey of state DOTs and a review of their guidelines and process documents. The chapter describes project and program performance measures, and then discusses the benefits of strategic program delivery and of various project delivery methods. The chapter concludes with a discussion of the challenges inherent in implementing program delivery in the transportation industry and strategies to overcome these challenges. Minnesota DOT’s Performance Measures MnDOT uses performance criteria to guide capital investments and annual operational budgets. MnDOT management regularly reviews transportation system and agency performance reports. The ongoing measurement and review process assists MnDOT in evaluating the efficiency of service delivery and assessing the effectiveness of program activities. This objective-based approach increases transparency and encourages innovation by focusing on outcomes. Accountability and transparency are a foundation of success for public agencies. The benefit is increased trust, which can help elected officials view MnDOT as a trusted partner and work with the agency on challenging issues. Confidence in agency decision-making processes and results is important to legislators considering new investments. http://www.dot.state.mn.us/measures, accessed April 2016 PROJECT AND PROGRAM DELIVERY PERFORMANCE Performance measurement is an essential element to improve project and program delivery methods. Tracking perfor- mance measures during project and program delivery helps state DOTs quantify performance and identify opportunities for improvement. In addition, AASHTO (2002) indicated that delivery measurement is a critical tool to communicate program delivery status to all stakeholders. Some state DOTs have performance measurement tools in place to help them improve their programs. Others are beginning the process of developing such tools or at least recognize that they need them. There are numerous ways to measure transportation project and program performance; performance measures can focus on cost, scheduling, or quality metrics. For example, if final construction costs consistently exceed initial budgets, estimating tech- niques need to be revised. Table 3 provides examples of program delivery performance measures. Of the 41 state DOTs that responded to the survey, 23 DOTs (56%) stated that they track the performance of the projects selected for a program and 17 (42%) reported that they have a process to measure the performance of program delivery. When asked what types of performance measures they have used, almost all state DOTs reported that cost and schedul- ing metrics are typical performance metrics for their transportation projects and programs. Some state DOTs measure the performance of their projects and programs with regard to their strategic goals. The following sections describe some typical performance measures. The Utah DOT (UDOT) uses an executive dashboard that includes typical performance metrics to conduct a regional program update, which provides an internal look at where individual regions stand on their projects. For each region, three main metrics are used in the dashboard: scope, schedule, and budget. The scope metric involves the measurement of contract payments, change orders, and final expenditures in comparison with the original contract. Figure 17 shows an example of UDOT’s scope performance metrics. CHAPTER FOUR PROGRAM DELIVERY PERFORMANCE: BENEFITS AND CHALLENGES

26 TABLE 3 EXAMPLES OF PROGRAM DELIVERY PERFORMANCE MEASURES Category Example Measures Cost Project within budget (yes/no) Activity unit cost Percentage cost increase/decrease Scheduling Contract milestones (e.g., completion date) Project on schedule (yes/no) Percentage schedule overrun Scope Number of change orders Activities performed versus planned Value of projects programmed versus delivered Number of projects programmed versus delivered Quality Performance specifications for capital improvements Levels of service (LOS) for maintenance and operations activities Number of noncompliance reports Source: AASHTO (2002). FIGURE 17 Scope performance metrics (UDOT 2016a). The schedule performance metric involves measuring conformance to the schedule and on-time delivery. It can be applied to active projects approaching their advertising date as well as projects in the scoping or conceptual phases. The metric provides information related to early, on-time, and late delivery. Figure 18 shows an example of UDOT’s schedule performance metrics. The budget performance metric compares the total project cost estimate with the approved project value in the precon- struction phase and measures calculated total projected expenses as a percentage of project value in the construction phase. The result is a summary of current program funding, totaled by month. Figure 19 shows an example of UDOT’s budget per- formance metric. North Carolina DOT (NCDOT) has an online performance dashboard that gauges performance based on actual and real- time statistics. The dashboard is directly tied to the department’s success in fulfilling its mission and achieving its goals.

27 NCDOT’s organizational performance dashboard includes five main areas aligning with the department’s strategic goals. One of these areas is the delivery rate, which is accompanied by indicators of how well NCDOT is delivering its planning, design, construction, and maintenance activities. The delivery rate is defined as the average completion percentage of the following four project delivery-related metrics: • Project development letting success rate (percentage of projects let on schedule) • Right-of-way plan delivery rate (percentage of right-of-way plans delivered on schedule) • Construction schedule completion delivery rate (percentage of construction projects completed on schedule) • Construction budget completion delivery rate (percentage of construction projects completed under budget) (NCDOT 2010). FIGURE 18 Schedule performance metrics (UDOT 2016a). FIGURE 19 Budget performance metrics (UDOT 2016a). Note: Committed advertise date is the date that the agency commits to advertise the project.

28 Anyone can access the dashboard from the NCDOT website. By clicking on any of the gauges, a user will see detailed information on how NCDOT determines the rating of that particular metric gauge. (See the example in Figure 20.) To be more transparent, NCDOT also maintains a web-based construction progress report on all active construction contracts. The report is fully searchable and provides detailed project tracking information, including monthly progress, financial reporting, and highway division and county location. Figure 20 shows that 86% of NCDOT’s transportation improvement program (TIP) projects in the state fiscal year were advertised for bid on schedule. FIGURE 20 NCDOT’s organization performance dashboard (NCDOT 2016). Figure 21 shows NCDOT’s delivery rate performance metrics in more detail. The dashboard shows that 81% of statewide transportation improvement program (STIP) projects and 90% of non-STIP projects in the state fiscal year were advertised for bid on schedule. The dashboard also shows the environmental compliance score—the calendar-year-to-date average score for all construction and maintenance projects statewide graded by the Sedimentation and Erosion Control Program. NCDOT has established a target range of 7.5–8.8. FIGURE 21 NCDOT’s delivery rate performance (NCDOT 2016). The Wisconsin DOT uses a design quality index (DQI) and a construction quality index (CQI) to measure project and program performance. The DQI is determined by evaluating 16 elements of a design plan. At the completion of a project, the project leader, prime contractor, and designer meet to fill out the DQI form. The project leader then enters the agreed-upon ratings into the Field Information Tracking System and submits them to office personnel. The CQI includes 21 elements on the maintainability of the constructed facility. The project manager and maintenance representative perform the CQI for the project. The project manager then loads the ratings into the Project Tracking System (WisDOT 2015). Figure 22 shows an example of the DQI system.

29 FIGURE 22 Wisconsin DOT performance measures (Wisconsin DOT 2015). The Missouri DOT’s (MoDOT) Tracker (see chapter two) is published quarterly to help decision makers. The January 2016 Tracker (MoDOT Tracker 2016) lists 60 measures spread out over the following seven tangible results: • Keep customers and ourselves safe, • Keep roads and bridges in good condition, • Provide outstanding customer service, • Deliver transportation solutions of great value, • Operate a reliable and convenient transportation system, • Use resources wisely, and • Advance economic development. Tracker reports on the status of each measure for each tangible result. For example, under the “deliver transportation solu- tions of great value” tangible result, MoDOT stated that it has used innovative contracting methods to improve efficiency, increase flexibility, mitigate limited resources, meet each project’s unique challenges, and maximize collaboration with the public and private sectors (MoDOT Tracker 2016). Table 4 shows the level of detail measured and monitored on a quarterly basis for each of six measures under this tangible result. As a final note on performance measures, NCHRP Report 662: Accelerating Transportation Project and Program Deliv- ery: Conception to Completion concluded the following: Performance measures are important because they showcase the tremendous needs state agencies face and build confidence that the agencies are spending tax dollars wisely. They provide a customer-based focus that helps state DOTs address public concerns and build public trust. They help ensure cost-effective use of limited funds; provide a tool to improve areas where progress needs to be made; and serve as a barometer on internal performance, delivery, and overall effectiveness. (Keck et al. 2010) BENEFITS OF PROGRAM DELIVERY State DOTs continually seek ways to improve their managerial, operational, and organizational effectiveness in project and program delivery. Effective program delivery does not just happen; it is a conscious decision made by the agency. A detailed discussion of the benefits and challenges of program delivery is presented in the next chapter through seven case examples. This section discusses findings from the national survey.

30 TABLE 4 MEASURES OF PROJECT AND PROGRAM DELIVERY Measure Purpose of Measure Results Percentage of programmed project cost compared to final project cost This measure determines how close total project costs are to the programmed costs. The programmed cost is consid- ered to be the project budget. 214 road and bridge projects were completed in fis- cal year 2016 at a cost of $483 million. This repre- sents a deviation of 2.08% (or $10 million) less than the programmed cost of $493 million. Percentage of projects completed on time This measure tracks the percentage of projects completed by the commitment date established in the contract. This includes road, bridge, local public agency and multimodal projects—rail, aviation, waterway, and transit. To date in fiscal year 2016, 67% of the closed-out projects were completed on or ahead of schedule. Percentage of change for finalized contracts This measure tracks the percentage difference between total construction payouts and original contract award amounts. This indicates how many changes are made on projects after they are awarded to the contractor for road, bridge, local public agency, and multimodal projects— rail, aviation, waterway, and transit. MoDOT’s performance in fiscal year 2016 is 0.9% so far. ($561 million worth of projects completed $5.0 million over the award amount.) Innovative contracting methods This measure tracks the use of innovative contracting meth- ods on MoDOT projects, including A+B contracts, alterna- tive technical concept contracts, and design-build contracts. In fiscal year 2015, the four projects delivered using innovative contracting methods accounted for $113.2 million of the $767.77 million program. Value engineering This measure tracks the use of value engineering during design and construction on traditional MoDOT projects, including value analysis during the design phase, con- struction value engineering (VE) proposals, and imple- mentation of best practices into standards and policies. So far for fiscal year 2016, 10 VE proposals were approved, resulting in MoDOT savings of $337,000. Average highway lane-mile and bridge construction costs This measure tracks the costs to construct a variety of common highway and bridge projects, including the costs for equipment, labor and fringe benefits, and materials to construct a project. Minor road asphalt resurfacing costs have increased due to a combination of fluctuating fuel and oil prices and increased material costs. Source: MoDOT Tracker. Measure of Departmental Performance, January 2016. Figure 23 shows the main benefits of program delivery from the national survey. The most common benefits are acceler- ated project delivery; better risk management, including flexibility in reassessing/reassigning risk; flexibility in delivery scheduling; increased control of scope, schedule, and cost; and flexibility in innovation. FIGURE 23 Benefits of program delivery (n = 41). In addition to identifying the benefits of program delivery as a whole, the survey attempted to examine the benefits of using alternative contracting methods. Figure 24 summarizes the responses to this question. The most common reported benefits of using ACMs in program delivery are flexibility in innovation and accelerated project schedule.

31 FIGURE 24 Benefits of ACMs to program delivery (n = 41). CHALLENGES AND LESSONS LEARNED IN PROGRAM DELIVERY State DOTs face numerous challenges in implementing program delivery. The transportation industry is experiencing sub- stantial organizational changes, such as teaming approaches to program management, decentralization of project develop- ment and delivery functions, and outsourcing of administrative functions and construction management (see discussion in chapter two). Henkin (2009) identified the following six challenges specific to program delivery: • Program expansion: Many state DOTs have expanded their programs significantly, requiring them to deliver more projects in a given period of time. • Increasing project complexity: Because of the increasing complexity of transportation projects—including mega-proj- ects and multimodal projects—state DOTs must consider changing their organizational structures to better coordinate project development and construction activities, such as National Environmental Policy Act documentation, utilities, and right-of-way acquisition. • Changing management focus: Some state DOTs are moving from a focus on the best engineering solution to a focus on the best business solution. This requires organizational and culture changes. • Workforce issues: To keep up with changing program delivery approaches, state DOTs must provide adequate, experi- enced staff. • Private sector capabilities: The increased capacity of the private sector to manage transportation programs and projects is enabling state DOTs to consider outsourcing additional areas in innovative delivery approaches. However, pressure exists to change internal management approaches and understand risk management and risk sharing in order to success- fully shift responsibilities to private firms. • Outsourcing project delivery and program management functions: As state DOTs face the need to downsize their inter- nal staffs, they find themselves relying more heavily on private firms for both individual project delivery and overall program delivery. State DOTs need an enhanced decision-making framework to help them choose between in-house and outsourced management. In the national survey, the respondents were asked to list the biggest challenges they face in implementing program deliv- ery. The following is a summary of responses from 26 state DOTs that are currently implementing or considering the use of program delivery methods: • Funding issues • Required culture changes from traditional points of view • Need to establish trust between regions and headquarters • Required training and education of staff, industry, and legislature • Obtaining buy-in from stakeholders • Staff to administer alternative delivery programs • Greater levels of coordination and collaboration among various parts of the organization • Identifying the right group of projects and programs

32 • Developing and maintaining consistency between the program and the rest of the agency • Time required to analyze the entire program • Understanding benefits and applicability of program delivery • Resource demands. The survey also attempted to identify the lessons learned by the 26 state DOTs that are currently implementing or consider- ing the use of program delivery methods. The respondents mentioned the following core lessons learned: • It is essential to educate the agency and all stakeholders on the process of choosing program delivery methods. • A great deal of training is required to ensure that agencies can take advantage of ACMs in a programmatic way. • The agency must be flexible and take a step back to ensure that the path chosen is the correct one. • Having skilled, trained, and effective program managers within the agency is essential to coordinate the whole process with both internal and external stakeholders. • The central office must be completely transparent and able to effectively communicate statewide goals to the regions. • Management of consultant contracts and program costs must be a priority. • Clear and consistent policies and procedures must be set at the beginning for how programs are to be delivered. • Consultation with the industry is necessary to optimize a program. • Stakeholders must be involved in developing and implementing program delivery. • Extensive time and resources are required to choose program delivery methods. SUMMARY This chapter describes the performance and benefits of program delivery through a summary of responses to a national survey of highway agencies and a review of their guidelines and process documents. These data provide a rich description of issues that influence the success of program delivery methods in the agencies. The survey found that more than half of the responding DOTs track the performance of the projects selected for a program, and nearly half of them have a process to measure program delivery performance. Cost, schedule, and scope metrics are typical in transportation project and program delivery. Some state DOTs also measure the performance of their projects and programs with regard to their strategic goals. State DOTs often use a performance dashboard and publish a performance report to monitor and improve their transportation programs. The most common benefits of program delivery are accelerated project delivery; better risk management, including flexibility in reassessing/reassigning risk; flexibility in delivery scheduling; increased control of scope, schedule, and cost; and flexibility in innovation.