Managing State Transportation Research Programs (2019)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

55 Case example interviews were conducted with transportation research programs of five agencies: District DOT, Louisiana DOTD, Minnesota DOT, Ohio DOT, and Utah DOT. These states were chosen on recommendation by state DOT survey participants, the NCHRP Synthesis Panel, and by the study team in order to gain a diverse range of perspectives on how agencies of different sizes, budgets, research models, and geographical locations manage their research programs for program quality and value. 3.1 Overview Among many current or recent initiatives, four overarching processes are being changed in full or in part at these research programs to help the overall departments be successful in their roles, and to increase the overall value proposition of these research programs: 1. Identifying research needs, 2. Conducting or managing product evaluations, 3. Implementing research with dedicated responsibilities, and/or 4. Engaging more proactively across the department. Figure 25 provides a short summary of each of the five agencies and their unique features. Figure 26 presents a distribution of budget levels for the five case example research programs. District DOT has the smallest agency budget at approximately $1 million annually, and Minnesota DOT had the largest agency budget at $14 million during fiscal year 2017. The following pages summarize each research program with background information and current initiatives before providing information on each of the four research program dimensions of program capability, program management, program quality, and program value. 3.2 District (of Columbia) Department of Transportation The District Department of Transportation (DDOT) has one of the smallest agency research programs, yet it is one of the few to be implementing a strategic plan for continuous improve- ment. The program sees itself as a driver for data-driven evaluation and innovation within the agency. One of its major initiatives has been to start a process of peer review for one project to enhance the quality of program research outcomes. The DDOT research program also fosters a culture of research by sponsoring interns who act as ambassadors of the program to promote research engagement across the agency. The program also promotes innovation by working with other parts of the agency such as intelligent transportation system (ITS) and policy to vet and prioritize pilots and meet agency objectives. C H A P T E R 3 Case Examples

56 Managing State Transportation Research Programs Agency Summary and Unique Features District (of Columbia) Department of Transportation The District Department of Transportation has one of the smallest agency research programs, yet it is one of the few that is implementing a strategic plan for continuous improvement. The program sees itself as a driver for data- driven evaluation and innovation within the agency. One of its major initiatives has been to initiate a process of peer- review for all projects to enhance the quality of program research outcomes. The DDOT research program also fosters a culture of research by sponsoring interns who act as ambassadors of the program to promote research engagement across the agency. The program also promotes innovation by working with other parts of the agency such as intelligent transportation system (ITS) and policy to vet and prioritize pilots and meet agency objectives. Louisiana Department of Transportation and Development Louisiana Transportation Research Center has a large research program with more than 90 staff members and 30 students. LTRC is unique in its structure because the department combines its resources with universities across the state to conduct research and is physically located on a university campus, with roughly half of research projects conducted by DOTD staff members and half of research projects conducted by university faculty. In recent years, LTRC developed a Local Technical Assistance program to evaluate issues in the field and provide recommendations. New project evaluations are also a new responsibility of the center. The program reports program-wide quantitative and qualitative performance measures as well to provide the wider DOTD with information on program value and outcomes. LTRC also sends out an annual survey to the wider DOTD and has found that a majority of DOTD employees view the center positively. Minnesota Department of Transportation The Minnesota DOT’s research program is roughly the same size as LTRC, with most research projects conducted by university and non-university contractors. The agency administers a unique local road research program—the Local Road Research Board. Both the MnDOT Research Figure 25. Case example research program highlights.

Case Examples 57 Agency Summary and Unique Features Services Division and the Local Road Research Board use research strategic plans to set the direction of their programs and to contribute to the overall State Multimodal Transportation Plan to fulfill the Minnesota GO Vision. Ohio Department of Transportation The Ohio DOT’s research program has gone through significant structural and procedural changes over the last 5 years to maximize its program value. Following a peer exchange, the Ohio DOT began the Ohio Research Initiative for Locals (ORIL) modeled after Minnesota and Iowa programs in order to better cater to research needs of locals. The program also terminated its external solicitations of research needs except for limited student projects in order to focus on the strategic needs of ODOT. These changes have had positive effects on the program’s ability to undertake research and increase its value proposition to the wider DOT. Utah Department of Transportation The Utah DOT’s research program is mid-sized and has focused on implementation and research needs solicitation and prioritization in recent years. A new innovation and implementation group was created to coordinate and communicate implementation and technology transfer initiatives because of persistent challenges in meeting goals for implementing research results. Utah has also revamped its research needs solicitation and prioritization processes by creating an annual research workshop where professors, department staff, and consultants come together to discuss agency needs before researchers submit problem statements. LTRC has a similar workshop approach. Utah also began using multicriteria, decision-making software to rank problem statements by specific criteria that they are working to refine. Although Utah does not have a strategic research plan, the program is working to better partner with leadership to develop its research vision, goals, and objectives. Figure 25. (Continued).

58 Managing State Transportation Research Programs Background DDOT’s Research Program is located in the Planning and Sustainability Division, reporting to the Associate Director for the Planning and Sustainability Division. The research program administers research funding, manages research projects, and promotes agency innovation. The program also manages library services. Research projects are managed by project managers from anywhere within the DOT and largely conducted by academic and private-sector consultants. Figure 27 indicates the breakdown of who conducts research for DDOT’s research program. Current Initiatives DDOT relies on its 5-year Research Strategic Plan developed in 2013 to continue to stream- line a number of activities. DDOT is in the process of updating its research manual, operating Figure 26. Where the case example research programs lie, based on fiscal year budget level. Universities 65% Non-University Contractors 25% Within the Research Program 5% Elsewhere in the DOT 5% Figure 27. Researchers conducting DDOT research.

Case Examples 59 procedures, and process documents. To improve the quality of its research projects, the agency has decided to incorporate peer reviewers (outside experts) in its research project management process. This approach has been piloted in a recent Freight Trip Generation Study and will be incorporated into other studies going forward. DDOT receives many requests for testing new technologies and piloting new solutions. The DDOT research program is working with other parts of the agency (information technology services, policy) on a process for vetting and prioritizing pilots to best meet agency objectives. DDOT accesses academic research expertise through a university consortium contract led by Howard University, which also provides administrative support for the research program. The agency is trying to broaden outreach to gain the right skill sets and university programs to support projects and the peer-review process. The DDOT research program currently does not have an integrated management system specifically for research, instead relying on broader agency systems (such as for invoicing and spend tracking) and spreadsheets to track program funding. Program Capacity Research Direction The research program’s vision, goals, and objectives are set through the development of a research strategic plan. The Research Program Manager provides oversight of the strategic direc- tion for research but relies on the Research Advisory Committee and senior agency leadership to help identify that strategic direction. The program hopes to improve this process by better engaging agency leadership in the goal-setting process, as well as incorporating the broader research community (university partners) to inform these goals. The research program is a direct participant in the State Transportation Innovation Council, alongside universities, industry partners, consultants, non-research program staff at DDOT, and the region’s metro- politan planning organization. DDOT’s Research Strategic Plan for 2013–2017 was developed out of a peer exchange. Prior to the peer exchange, program staff went to all internal branches to ask what their research needs were to focus research strategic areas. However, this process was difficult due to the time and effort required to determine concrete research needs at each division. Now, the research pro- gram is looking into helping divisions do strategic research plans in order to better coordinate and sequence research efforts to avoid duplication and to improve research value. The Research Strategic Plan’s recommendations remain very strong, and the program is working on determin- ing new goals and high-level statements of objectives to continue the momentum. The research program will also host a peer exchange this fall on some of these research management questions in order to continue to deliver and improve program value. Research Expertise The Research Division has two staff members and one librarian. Because of the small size of the program, most technical research management support such as IT, finance, and accounting are sourced from the DOT’s broader support offices. However, the research program does spend extensive time managing the research contract with its university partners. Research Funding District DOT’s total research budget is approximately $1 million annually for its research projects and staff. Reserve and discretionary funds are maintained for immediate research needs (Figure 28). Staff also have responsibilities for managing pilots and pilot evaluation, though these projects do not consume the research project budget themselves.

60 Managing State Transportation Research Programs Although the size of the research program is small, the program feels that it is well staffed relative to its research funding. There is generally enough capacity to oversee its projects with a typical load of three to seven projects per staff member. The research division perceives funding compliance requirements to be of moderate difficulty to fulfill as a whole. Program Management DDOT uses an open submission process to solicit for research projects and identify research areas or topics. The agency then assembles its Research Advisory Committee to prioritize research projects for funding. The committee includes senior leadership from across the DOT. This allows DDOT to benefit from bottom-up prioritization through the open submission pro- cess, as well as a top-down prioritization method through ultimate project selection by top DOT decision makers. DDOT’s research program struggles with the requirements for competitively bidding proj- ects and contracts when projects require specialized skills. On-call indefinite delivery, indefinite quantity contracts help to speed up the contracting process but can limit the researchers avail- able. Open solicitation processes also take a long time (7 months or more) with many steps involved, which makes it difficult to navigate contracting processes. Given these challenges, the agency has opted not to sponsor pooled-fund projects as a lead state. The agency does not have independent procurement or contracting authority. Research dissemination is currently not tracked at DDOT, because there is uncertainty with how to effectively track it at this time. The main recipient of disseminated research is DDOT in general, and the main methods used for disseminating findings include research reports, journal articles, and to a moderate extent, through the website, research presentations, and social media. Research is rarely disseminated through public meetings or research briefs. The research program does coordinate the agency’s booth at the TRB Annual Meeting and attempts to showcase the program and the agency there but does not participate in other trade show booths. Operations 10-20% Planning 20-30% Safety 53% Asset Management 12% Materials 5% Environmental 4% Figure 28. Approximate funding by research area.

Case Examples 61 Program Quality Oversight for project execution, budgeting, and implementation is the responsibility of the Research Program Manager at DDOT. The agency does not withhold payment for poor-quality research, and there is currently no consistent method for evaluating completed research projects through performance measures. DDOT is currently working on instituting peer-review panels for all research projects, partly due to some poor research projects in the past. Program Value DDOT does not currently have a formal definition of research implementation, but it sees implementation as the application of research results ideally beyond a pilot stage. The agency does not currently have designated staff at the research program to track implementation, because it sees implementation to be the responsibility of technical staff at the receiving program. There usually are no DDOT staff specifically dedicated to support implementation. The program does track implementation through an Excel sheet with categories for whether implementation is based on a DDOT research project, best-practice scan, and NCHRP, as well as the amount of funding secured for that work (MDT 2017). At the same time, the program perceives that the main reasons that research is not implemented is not due to a lack of designated personnel, but to the poor qual- ity of research products and changing agency priorities. Additionally, research funding almost never depends on successful research implementation, as with the majority of state agencies. The research program did secure $500,000 a few years ago to support the implementation of research results. This implementation funding is mainly used on projects that involve refining previous work done in other states or through NCHRP. DDOT also intends to develop a process to sort through NCHRP results to find research relevant to the agency. Implementation deci- sions are typically made as the project comes to completion and is handed off for other funding from the receiving group. DDOT as a whole has an orientation toward research and innovation. Leadership places great emphasis on getting better and stepping up as an agency, to be more data-driven and evaluation- driven, and ultimately being responsive to its citizens. The agency compares itself with other cities in its size range (San Francisco, Portland, and Seattle) and prides itself on being innovative. The research program is able to market itself widely outside of the DOT through the Annual TRB Meeting because of its location in Washington, D.C. The program also markets itself within the DOT through a unique Research Internship Pro- gram. Each year, 10 summer interns and 3 to 5 fall/spring interns are hired and distributed across DDOT to work on smaller research projects. Howard University recruits interns and manages the program for DDOT. These interns effectively serve as ambassadors for research while also helping to meet pressing agency needs and cultivating a future workforce for the agency. Several interns have gone on to be hired by DDOT after graduation. Research projects may arise from different divisions of the DOT. The research program has not pushed to be a clearinghouse and is not a single centralizing force for innovation. Rather, the research program provides value to the wider agency by being a “connector to resources” and bringing methodology and sometimes funding to enrich innovative projects wherever they arise. 3.3 Louisiana Department of Transportation and Development Louisiana Transportation Research Center has a large research program with over 90 staff members and 30 students. LTRC is unique in its structure because the department com- bines its resources with universities across the state to conduct research and is physically

62 Managing State Transportation Research Programs located on a university campus, with roughly half of research projects conducted by DOTD staff members and half of research projects conducted by university faculty. In recent years, LTRC developed a Local Technical Assistance program to evaluate issues in the field and provide recommendations. New project evaluations are also a new responsibility of the center. The program reports program-wide quantitative and qualitative performance mea- sures as well provides the wider DOTD with information on program value and outcomes. LTRC also sends out an annual survey to the wider DOTD and has found that a majority of DOTD employees view the center positively. Background The Louisiana Department of Transportation and Development research program is located in the Office of Engineering where it reports to the Chief Engineer. The actual research and technology transfer, training, and education program is named the Louisiana Transportation Research Center, located on Louisiana State University’s Baton Rouge campus where it merges the resources of state government and universities. The program administers research fund- ing, manages research projects, performs in-house research, promotes agency innovation, monitors implementation, and provides technical assistance in solving field problems (see Figure 29 and Figure 30 for more information). Although LTRC is located on the Louisiana State University campus, LTRC also has working relationships with all other universities’ engineering programs in Louisiana. LTRC has more than 100 employees and 30 student researchers. Half of the employees are DOTD staff and half are university faculty. LTRC is funded by SPR, other federal pro- grams (Innovative Bridge Research and Deployment, Surface Transportation Programs) and external contracts and grants (NCHRP, other federal grants, and the National Science Foundation). 5% 48% 48% 95% Non-University Contractors University Faculty DOT Staff “For over 30 years, the staff of LTRC has strived to merge resources of state government and universities to help identify, develop, and implement new technology to improve the state’s transportation system…. The center conducts short-term and long-term research and provides technology assistance, engineering training and continuing education, technology transfer, and problem-solving services to DOTD and others in the transportation community. LTRC’s goal is to merge the resources of state government and universities to identify, develop, and implement new technology to improve the state’s transportation system” (LTRC Website). Figure 29. Who conducts actual research at LTRC?

Case Examples 63 Current Initiatives The research program has a technology transfer engineer who proofreads, edits reports for technical issues, and creates technical summaries and capsules. Project capsules are short research briefs published when a research study begins, outlining the need for the research, the study’s objectives and methodology, and the potential for future implementation. However, the program perceives its implementation process to be of high quality and wishes to gain a discretion- ary implementation fund. A hindrance to implementation is currently resistance to change, which the program hopes to alleviate through a dedicated fund with implementation directives. A technical assistance program is also hosted by the research center as an internal engineering consulting practice for transportation projects. For example, when things go wrong in a con- struction project and the team is unable to diagnose the reason, LTRC experts are called in to inspect the roadway, bridge, or structure and write a technical assistance report complete with recommendations. Source: LTRC 2016b. Figure 30. LTRC functional relationships.

64 Managing State Transportation Research Programs LTRC is expending significant effort to reach out to the consulting community and DOT employees at large to include more of them in the research project identification process. The program hopes to gain research engagement from underrepresented offices that do not typically do research. LTRC has had some success with this by showcasing its connections with federal resources and assisting different groups with obtaining federal grants. By being proactive and inclusive, LTRC is creating success in these areas. Program Capability Research Direction LTRC sets its research direction through an advisory board consisting of executive leader- ship. Oversight for the program’s strategic direction is maintained by the RAC, DOTD top leadership, and the Research Program Manager. The research program also participates on the Louisiana State Transportation Innovation Council, which consists of industry partners, consultants, Local Technical Assistance Program staff, DOTD research program staff, DOTD non-research program staff, and metropolitan or regional planning organizations. Research Expertise Research project managers are employees of DOTD, both within the research program and outside of it but within DOTD. The most important professional experience for project managers is that they work in the specialized area of the research project. It is also important that they have experience managing research projects in general. Support services for research management are all located within the research program—IT, administration, library services, finance, accounting, and contracting and procurement. Because of the integration between the department and Louisiana universities, non- university contractors only conduct 5% of total Louisiana DOTD research. Universities are seen as a much more important partner through the University Transportation Center and through faculty participating as research contractors for the state. Research Funding DOTD’s total research budget is approximately $9.5 million annually and comes almost exclusively from SPR Part B funds (95%), with other federal funds and multistate pooled funds comprising the rest. A majority of these funds were spent on LTRC research (86%), with some spent on research program administration (9%), 3% on university transportation centers, 1% on research implementation, and 1% on other activities such as knowledge management and training. The DOTD sometimes maintains reserve or discretionary funds for immediate research needs. The program conducts approximately 85 research projects each year. To fulfill compliance and reporting requirements for various funding sources, there are dedicated DOT staff for all funds, specific accounting systems, procurement procedures, and legal support. The research division perceives compliance requirements to be very easy to fulfill as a whole. Program Management LTRC has a series of committees that oversee various aspects of the research lifecycle. 1. The Research Identification Committee sets broad research directives. 2. The Research Advisory Committee prioritizes research areas and topics. 3. The Project Review Committee awards projects.

Case Examples 65 4. The Policy Committee has broad oversight over LTRC activities and is made up of deans of all degree-granting engineering schools as well as DOTD’s division directors. The committee is also chaired by the department’s Chief Engineer. Every 2 years, a call is given to all DOTD sections and industry partners (consultants, aca- demics, paving associations, contracting associations, etc.) to solicit research needs. Workshops are also scheduled between academics and DOTD staff to match up research needs with research expertise of university faculty. The output of this biannual process is one-page problem state- ments (Figure 31) which include implementation benefits of the proposed research study, jointly written by DOTD and academic staff. This solicitation process occurs between October 1 and January 15, generating 95–150 prob- lem statements. From there, the LTRC organizes the statements by subject matter and brings in subject matter experts (80% DOTD staff, with the rest comprising consultants and academics) to rank problem statements based on implementation potential and need. Out of this ranking pro- cess, the top three to four problem statements for each subject move onto the Research Advisory Committee. This RAC is chaired by the Chief Engineer and includes section heads of the DOTD and the subject matter committee heads. These problem statements will then be ranked by the RAC based on importance and need. LTRC also has a robust research manual (LTRC 2016b) complete with its research identification process, proposals and ranking procedures, contracting requirements, and final reporting guidelines. Figure 32 provides an approximate breakdown of where LTRC’s research funding is spent by research area. Research dissemination is tracked by the research program. Research dissemination occurs most frequently through the website, research reports, research presentations, research news- letters, social media, and YouTube videos. Journal articles are also used frequently. Infrequently or never used are public meetings, research briefs, and trade show booths for disseminating research. Research is intended to be disseminated to DOTD leadership, the broader state DOTD, industry partners, consultants, and the federal government. Implementation oversight is the responsibility of the Research Program Manager and research project managers. Implementation tracking is done by following the progress of implemented projects for a determined amount of time and by assessing the benefits or returns on investment for implementation. A majority of DOTD research is implemented. LTRC categorizes imple- mentation status into five levels of classification: 1. Project/implementation in progress 2. Implementation recommended 3. Implementation complete 4. Not implemented 5. No implementation expected Research projects can be selected for implementation at different points during the research lifecycle. Program Quality The research program maintains a quality control method through Manual of Research Pro- cedures (LTRC 2016b) and set procedures for monitoring quality as seen in Figure 33. Research oversight for project execution is done by the Research Program Manager and project managers. Budget oversight is jointly done by the Research Program Manager and project managers. Performance measures for maintaining research project quality include completion of the project on schedule and within budget, post-project tracking for implementation, and fulfill- ment of research objectives. Research evaluation allows the research program to share lessons

66 Managing State Transportation Research Programs Figure 31. LTRC problem statement solicitation form.

Case Examples 67 learned as well as to evaluate individual researcher performance. Most of the research conducted is peer reviewed, a unique attribute of the program. Program Value LTRC’s research implementation usually does not fail on the basis of funding barriers, poor research quality, changing agency priorities, inconsistent research findings, or intellectual prop- erty constraints. Although most research is implemented, reasons that implementation might fail tend to be cultural: lack of acceptance within and external to the agency and preexisting per- formance standards that conflict with change. Sometimes, the lack of designated personnel for implementation and the difficulty of marketing research results inhibit implementation as well. The Louisiana DOTD places a high priority on LTRC’s research. The center is integrated well with construction, design, bridge design, road design, pavement design, and materials. An internal department survey showed that DOTD staff members view the center positively. As an example of LTRC’s reputation, the center has attracted three visiting scholars in the last 4 years to do sabbaticals, including those from other countries. LTRC assesses its program value and outcomes through program-wide quantitative and qualitative measures. As stated in LTRC’s Research Impacts brochure, an estimate of $500 million in savings has been achieved through the center’s research activities (LTRC 2017). 3.4 Minnesota Department of Transportation Minnesota DOT’s research program is roughly the same size as the LTRC, with most research projects conducted by university and non-university contractors. The agency administers a unique local road research program. Both the MnDOT Research Services Division and the Local Road Research Board use research strategic plans to set the direction of their programs and to contribute to the overall State Multimodal Transportation Plan to fulfill the Minnesota GO Vision. Minnesota also sets aside approximately a third of research funding for implementation efforts. Pavements 21% Planning 12% Safety 12%Traffic Operations 13% Geotechnical 13% Asphalt Materials 13% Bridges and Structures 4% Environment 4% Finance 4% Asset Management 4% Figure 32. Approximate LTRC funding by research area.

68 Managing State Transportation Research Programs Source: LTRC 2016b. Figure 33. LTRC publication review and approval process.

Case Examples 69 Background MnDOT Research Services and Library are located within the Modal Planning and Project Management (MPPM) Division and report to the Office Director of Transportation System Management. The mission is to help transportation practitioners solve problems through research and innovation. With funding from the state, local and federal research programs, the research program administer[s] more than 200 research projects annually. This includes research we manage for the Minnesota Local Road Research Board (LRRB) on behalf of Minnesota city and county engineers. Our goal is to create a better transportation network that moves Minnesota residents more safely and efficiently with less environmental impact. The research program administers research funding, manages research projects, promotes agency innovation, monitors implementation, and administratively supports the activities of the Minnesota LRRB. The program also manages library services, finance and contracting, and marketing and outreach activities. Minnesota DOT-sponsored research is conducted primarily by universities, some by contractors, and occasionally within the state DOT (Figure 34). MnDOT Research Services staff manage most projects. In FY2017 (Figure 35) MnDOT had 206 active research contracts, published 51 research reports and TRSs (short-turnaround research report), had 18,703 visits to the state research blog, supported 21 active pooled funds, led 6 pooled funds, had 735 Technical Advisory Panel members, and had 150 active Technical Liaisons. Universities 62.9% Contractors 36.5% Within the DOT 0.5% The Local Road Research Board (LRRB) was established in 1959 through state legislation and is administered by MnDOT Research Services. LRRB has sponsored more than 200 individual research projects over the last 15 years. LRRB makes it easy for transportation practitioners who are responsible for county highways and city streets to participate in setting the research agenda. Current LRRB-funded research falls primarily into the following categories: design, construction, maintenance/operations, environmental compatibility, administration, and implementation. Figure 34. Who conducts MnDOT-sponsored research? Source: MnDOT 2017a. Figure 35. FY2017 by the numbers.

70 Managing State Transportation Research Programs Program Capability Research Direction The Research Services and Library set their research direction through both the develop- ment of a DOT-wide strategic plan as well as through the development of a research strategic plan. Executive leadership, mid-level managers, and the Research Advisory Committee are all involved in the development of these strategies. The division also has external visibility as a par- ticipant of the Minnesota State Transportation Innovation Council, along with MnDOT office directors, district engineers, and the State Aid Engineer. MnDOT’s research strategic plan is connected to MnDOT’s 50-year Minnesota GO Vision and the Statewide Multimodal Transportation Plan (SMTP). These plans collectively address five main trends identified by Minnesota stakeholders: aging infrastructure, urban and rural popu- lation trends, climate change, environmental quality, and transportation behavior. Figure 36 illustrates the MnDOT’s framework for fine-tuning strategic research priorities through a matrix of trends and indicators. Research Expertise The research program’s in-house research staff come from a blend of technical and manage- ment backgrounds. Research project managers are mostly all staff in the research program. Professional experience most valued is management of research projects, followed by experience working in specialized research areas. Project managers’ familiarity with the geographical context of the research area is valued to a moderate extent. MnDOT’s research advisors are senior engineers with a Professional Engineer (PE) and a background in civil engineering. The project coordinators have backgrounds in project management and analytical skills. MnDOT uses contractors for 36% of its research projects, and contractors are most important for implementation and deployment of research results. Universities are also heavily relied on Source: MnDOT 2017b. Figure 36. MnDOT framework for fine-tuning strategic research priorities.

Case Examples 71 and conduct 62% of MnDOT-sponsored research projects. MnDOT research staff also princi- pally rely on other external partners such as AASHTO for committee participation; TRB for its core services such as webinars, annual conference, and publications; local associations for LRRB program administration through city and county representatives; and technology vendors for videos, marketing materials, website development, research database support, and development and report generation. Research Funding In FY2017, research funds of approximately $14 million came from three main sources: 25% State Planning and Research Part B funds, 34% state funds, and 38% from the Local Road Research Board. The federal funds were spent as follows: 29% on federal program support, 30% on single-state SP&R projects, and 41% on multistate pooled funds (Figure 37). To fulfill compliance and reporting requirements for various funding sources, there are dedicated DOT staff for all funds, accounting systems, and set procurement procedures. The research division perceives compliance requirements to be easy to fulfill as a whole. Program Management MnDOT’s Research Advisory Committee helps inform the development of its research stra- tegic plan and also helps to prioritize the agency’s research areas and topics. The agency allows open submission of research ideas through an idea submission system in order to be responsive to grassroots and front-line employee needs. The process of selection and awarding of projects is done by the Transportation Research Innovation Group (TRIG), a governance board com- prising 14 voting representatives from some of MnDOT’s offices and districts. Two non-voting members and Research Services staff also participate. These office representatives on TRIG are also responsible for the discretionary research in their offices which does not go through the research program. Selected projects fall into 10 research areas, comprising about a $10.6 million research project budget (Figure 38). The research budget is overseen by TRIG, Director of Research Services & Library project managers, and university faculty contractors. Source: MnDOT 2017b. Figure 37. MnDOT FY2017 research funds by source and SPR part II funding distribution.

72 Managing State Transportation Research Programs MnDOT Research Services uses its own Automated Research Tracking System (ARTS) data- base to manage research administration and track proposals, project progress, and project outcomes. This supports the work of research project managers by tracking research projects throughout their lifecycle through a remote and online system and helps the Research Services group become more efficient at managing research. Research dissemination is tracked by the research program, the research library, and com- munications personnel at MnDOT. Research dissemination occurs most frequently through the website, research briefs, research reports, research newsletters, and social media. This research is intended to be disseminated to state DOT leadership, the broader state DOT, other agencies in the state, other state DOTs, the federal government, and other researchers. A practitioner or an expert in the topic area is typically responsible for the implementa- tion of research results (MDT 2017). MnDOT dedicates approximately $1 million per year to implement research results. An implementation engineer coordinates these efforts with the statewide program as well as the Local Road Research Board’s Research Implementation Committee. Implementation is tracked by maintaining a list of completed research that is implementable and by assessing potential benefits of implementation. A moderate amount of research is imple- mented, and the selection process for implementation may occur at any point in the research lifecycle—during research project selection, while research is ongoing, when nearing project completion, or when the project is completed. The TRIG governing board selects implementa- tion projects. Program Quality Research contracts are paid by task, and final reports must conform to publication guide- lines. The technical advisory panel for each project will review and approve each task in a mandated work plan. Once reviewed, the draft will be updated by the principal investigator and approved by the technical liaison. Next, the report goes through editorial review prior to publication (Ohio DOT 2015c). Research Services and Library have set quality control methods in place that include moni- toring and tracking contract amendments, evaluating projects, and keeping up to date on tasks coming due or overdue. Source: MnDOT 2017a. Figure 38. MnDOT FY2017 research areas by funding.

Case Examples 73 Research is overseen by a combination of the two governing boards (LRRB and TRIG), the Research Program Manager, project managers, and university faculty contractors. At the same time, very little of the research is peer reviewed by another researcher working in the same field. Research project quality performance measures include completion of the project on schedule and in line with the budget, assessing potential benefits prior to the beginning the project, post- project tracking for implementation, and alignment to MnDOT’s strategic initiatives. Research evaluation helps direct funding sources, inform new research project selection, evaluate indi- vidual research performance, share lessons learned from the research program, and improve research program processes and effectiveness. Program Value Both state transportation agencies in Minnesota—MnDOT and the LRRB—benefit from the joint support and management structure offered by MnDOT Research Services. This increases the opportunities for collaboration, streamlines contract management and procurement, and minimizes redundancies in research topics and facilities. Program value and outcome assessment are currently measured on a project level but not on a program level. The program recently completed a research strategic plan in which one of the main recommendations is to move toward program-level assessment. While return on investment is traditionally a key consideration, the program is also beginning to work toward knowledge-based outcomes at a program level, such as “What have we learned through research? What do we know that we did not know some years ago? How has this improved our business? How has this changed our investments? Given an additional $1 million, what could we demonstrate or learn?” Additionally, the program is considering reporting benefits of research through topic or program-level measures. MnDOT completed the design of a systematic process to quantify the potential benefits of research in 2015 through a spreadsheet-based estimation tool. The program would like to systematically apply this process to new proposals and calculate realized benefits in ongoing projects, as well as retroactively assess the results of some past research projects. A task is added to the work plan to quantify benefits, with an initial assessment within 90 days of project start and updated at the end of the project. 3.5 Ohio Department of Transportation Ohio DOT’s research program has gone through significant structural and procedural changes over the last 5 years to maximize its program value. Following a peer exchange, Ohio DOT began the Ohio Research Initiative for Locals (ORIL), modeled after Minnesota and Iowa programs, in order to better cater to research needs of locals. The program also terminated its exter- nal solicitations of research needs except for limited student projects in order to focus on the strategic needs of ODOT. These changes have had positive effects on the program’s ability to undertake research and increase its value proposition to the wider DOT. Background Ohio DOT’s Research Program is located in the Planning Division and reports to the Admin- istrator for Statewide Planning and Research. The program “provides decision makers with the information and tools they require to ensure [that] Ohio’s transportation system meets the evolving needs of [its] residents and the traveling public. The program works to anticipate and address transportation concerns before they become critical problems.”

74 Managing State Transportation Research Programs The research program administers research funding, manages research projects, promotes agency innovation, and monitors implementation. The program also manages library services. Ohio DOT–sponsored research is conducted primarily by universities (85%), by contractors (9%), and within the DOT (4%), and most research project management is done by the DOT staff and technical panels. Current Initiatives In the last 5 years, Ohio has implemented its ORIL, which dedicates $0.5 million in SPR funds—this brings together four counties with one dedicated research engineer, and ODOT is finding that research is more effective for locals. ORIL advertises and solicits its own project proposals, promoting good will. This has been well received and has enhanced collaborations between county and state, where there were natural tensions. ODOT created this in 2013 after a 2011 peer exchange to learn how it is currently being done in other states, becoming the third state to form this type of local research program (Ohio DOT 2011). At end of 2014, the DOT started in-house research projects for workforce development. In 2014, the program also started research on-call through small, short projects, allowing them to be more responsive to research needs. The agency also stopped soliciting research needs from the wider research community, because it found that the community was not as responsive to agency needs. Ohio DOT also had two research cycles with an internal solicitation process in August and an external process in October. The back-and-forth logistics of the two cycles created additional work; by removing external solicitation, the agency was able to remove the second cycle. This has resulted in a more effective, streamlined process, and Ohio DOT is proud of its series of changes. The Research Program Manager stated that “there is not enough money to address curiosity when there are many projects that are concrete and implementable.” Program Capability Research Direction Ohio DOT’s Research Program sets its research direction through the development of a research strategic plan. Mid-level managers are the main contributors to determining the pro- gram’s vision, goals, and objectives. Research program staff are not participants on the Ohio State Transportation Innovation Council, although non-research program staff at ODOT are, alongside industry partners, consultants, LTAP staff, MPOs and RPOs, City and County staff, U.S. Army Corps of Engineers, U.S. Fish and Wildlife Service, and the Ohio History Connection. Research Expertise Ohio DOT’s research program contains two project managers and one contract manager managing 60 active projects. The program does not rely on other departments for research sup- port services other than invoicing and legal support. To keep track of costs to run the research program, Ohio DOT uses an Inventory Management System. Research project managers can be employed anywhere within Ohio DOT, and professional experience most valued in these staff are their experiences with managing research projects. Expe- rience working in a specialized research area or familiarity with the geographical context of the research area is not as important as management experience. Universities support Ohio DOT’s research activities through the University Transportation Center and principally as research contractors, with 85% of research projects conducted by uni- versities. Non-university contractors conduct about 9% of research projects. Ohio DOT believes that non-contractors are most important for providing capacity to the organization because of insufficient internal labor resources to conduct the full research program. Additionally, contrac- tors offer specialized skill sets that the DOT may not have available internally. Ohio DOT also

Case Examples 75 relies on AASHTO for surveys to support ongoing research, TRB for NCHRP research and annual field visits in topical areas, as well as local associations to provide board members for and propose research ideas to ORIL. Research Funding The program’s total research budget for FY2017 was $11,152,110 and came from three main sources: 78% State Planning and Research Part B funds, 19% state funds (including the 20% match for SPR funds), and 3% from multistate pooled funds. Figure 39 depicts Ohio DOT’s FY2017 research budget spending broken down by function. There is no predetermined allocation of funds for specific research areas on a year-to-year basis other than $500,000 designated to the ORIL initiative. Additionally, ODOT uses two on-call master contracts as “reserve funds” in the amount of $300,000 per year for immediate research needs. To fulfill compliance and reporting requirements for various funding sources, there are dedi- cated DOT staff for all funds and set procurement procedures. The research division perceives compliance requirements to be of moderate difficulty to fulfill as a whole. Program Management Ohio’s research program is proud that it has no master contracts. Requests for proposals are issued for all research projects, and it is expected that creativity and competition will ensue. Ohio DOT prioritizes research areas and topics through an open submission and voting process. However, research topic solicitation is dominated by internal DOT staff. External research topic solicitation is allowed only from students, with a few projects funded each year. Once research topics are determined, a project selection committee is used to award projects. The project selection committee ranks research topics in order of importance to their work on a 1–5 scale. This process allows the program to prioritize research in order of greatest agency need. No more than two projects are issued to one principal investigator to ensure that the researcher can perform quality work. Research projects fall into several main research areas (Figure 40), with small amounts also dedicated to Operations, Roadway, and Traffic subject areas, comprising about a $10 million research project budget (Ohio DOT 2017b). DOT Research 76% Multi-State Pooled Funds 14% National Programs Source: Ohio DOT 2017b. 5% LTAP 3% Research Implementation 2% Figure 39. Ohio DOT FY2017 research areas by funding.

76 Managing State Transportation Research Programs Program Quality The research program maintains quality through research report reviews by the research technical panel and the project manager. Ohio DOT’s payment policy also supports quality control by withholding 20% of each invoice payment during the project that is not paid until final deliverables have been received. The DOT stated that “If our technical liaisons determine that the report is poorly written, they will go through a series of revisions with the researcher during the last 4 months of the project that are reserved for report editing. Despite our efforts, we are sometimes not satisfied with the final version, though we have never denied payment” (Ohio DOT 2015c). A moderate amount of research is peer reviewed, meaning that it is evaluated by another researcher working in the same field. Ohio DOT has a clear delineation of oversight responsibilities: executive leadership at Ohio DOT provides strategic research direction and project selection oversight. The Research Program Manager oversees budgeting. Project execution oversight is the responsibility of research proj- ect managers and university faculty. Finally, implementation is overseen by the same research project managers. Research project quality performance measures include completion of the project on sched- ule and in line with the budget, assessing potential benefits prior to the beginning the project, post-project tracking for implementation, and fulfillment of research objectives. Research evalu- ation helps direct funding sources, inform new research project selection, evaluate individual research performance, share lessons learned from the research program, and improve research program processes and effectiveness. A multiple-phase approach to projects has worked out well at Ohio DOT for more realistic implementation with a more accurate projection of ROI. For example, an initial phase might cost $40,000–$50,000. Once the feasibility of the project has been determined through Phase 1, Phase 2 may be initiated. This helps the program acquire new knowledge, with the understand- ing that some projects will likely not undergo a second phase. Source: Ohio DOT. Maintenance 31% Hydraulics 24% Environmental 11% Safety 10% Aerial 7% Materials 5% Structures 3% Geotechnical 3% Construction 2% Planning 2% Pavements 2% Figure 40. ODOT’s FY2017 research areas by funding.

Case Examples 77 Program Value Research dissemination is tracked by the research program, but there is no designated position for tracking this. Research dissemination occurs most frequently through the website, research reports, research briefs, and sometimes through journal articles or research presentations. This research is intended to be disseminated to Ohio DOT leadership, the broader state DOT, other agencies in the state, and other state DOTs. Implementation is tracked by maintaining a list of completed research that is implementable and by assessing benefits and return on investment for implementation. Most research is imple- mented, and the selection process for implementation may occur at any point in the research lifecycle—during research project selection, while research is ongoing, while nearing project completion, or when the project is completed. Since most research is implemented, there is not a selection process to decide which projects will be implemented. As with most other agencies, funding almost never depends on successful research implementation. DOT staff typically are dedicated to supporting implementation: the technical expert responsible for that type of work and the project managers are involved in the implementation processes. The DOT’s research culture has changed in the last few years, but there remains a desire for the research program to be valued for more than the amount of funding it has. “Team Up ODOT” is an annual sharing session where at least half of research projects are showcased by various divisions to sell their ideas to leaders. Some divisions continue to believe that “research is a waste of time and money” and do not show up to meetings routinely. Ohio DOT’s research program assesses its program value and outcomes through both quan- titative and qualitative measures on a program-wide level. Ohio DOT is working on measuring its return on investment, performance improvements, savings, and safety, which will be inputted into its Annual Report. From a qualitative perspective, the agency evaluates whether its highest profile “super projects” are talked about by leadership. 3.6 Utah Department of Transportation Utah DOT’s research program is mid-sized and has focused on implementation and research needs solicitation and prioritization in recent years. A new innovation and implementation group was created to coordinate and communicate implementation and technology transfer initiatives due to persistent challenges with meeting goals for implementing research results. Utah has also revamped its research needs solicitation and prioritization processes by creating an annual research workshop where professors, department staff, and consultants come together to discuss agency needs before researchers submit problem statements. LTRC has a similar work- shop approach. Utah also began using multicriteria, decision-making software to rank problem statements by specific criteria, which they are working to refine. Although Utah does not have a strategic research plan, the program is working to better partner with leadership to develop its research vision, goals, and objectives. Background Utah DOT’s Research and Innovation Division is a part of the Technology and Innovation Group. Research reports directly to the Director of Technology and Innovation. The division “strives to be responsive to UDOT’s transportation needs. Our services and resources are accessible to our customers, and our work focuses on relevant issues facing our industry” (Utah DOT).

78 Managing State Transportation Research Programs The research program administers research funding, manages research projects, performs in-house research, promotes agency innovation, and monitors implementation. The program also manages library services. UDOT-sponsored research is conducted by universities, non- university contractors, and a limited amount within the Research and Innovation Division or broader state DOT, as indicated by Figure 41. Current Initiatives There is a new innovation and implementation group in the Research and Innovation Division. This group helps with technology transfer of innovative ideas and completed research through a dedicated Innovation and Implementation Manager (or Implementation Engineer), intern, and consultant, who help coordinate and communicate responsibilities. This group is helping to rewrite the charter for the State Transportation Innovation Council, in order to simplify the charter. The research program’s involvement with the STIC is an evolving process. Utah DOT has held an annual research workshop in most of the past 25 years to priori- tize research needs. This UTRAC Workshop is named after the Utah Transportation Research Advisory Council, a small group of UDOT leaders. In the past 2 years, UDOT has also sought input from professors, consultants, and UDOT prior to the submission of problem statements. Through this workshop, participants assess needs, challenges, and areas for improvement. The preceding input has resulted in more-targeted research problem statements. For the UTRAC Workshop, UDOT has also recently adopted a multicriteria, decision-making software tool for ranking criteria on problem statements. This was a partially successful effort, with the recognition that defining better criteria is needed. UDOT’s program is innovative and has implemented the Accelerated Bridge Construction efforts, Asphalt Pavement Performance Specifications, Bridge Deck Preservation Guide, and innovative pedestrian and cycling safety programs. Utah DOT also prepared its own pilot study on Road Usage Charge following pilot testing done through a western states pooled fund. Universities 70% Non-University Contractors 20% Research Division 5% Broader State DOT 5% Figure 41. Where UDOT-sponsored research is conducted.

Case Examples 79 Program Capability Research Direction The Research and Innovation Division sets its vision, goals, and objectives through an advisory board but not through a formal strategic plan. The UTRAC annual workshop helps identify components of the program’s research strategic direction. The overall strategic direc- tion of research is provided through the input of the Research Advisory Council, top UDOT leadership, and the Research and Innovation Division Director. The division has expressed wishes to improve this process by being engaged and developing recommendations to pre- sent to the UTRAC council, or UDOT senior leaders, for approval. The division has external visibility as a participant of the Utah State Transportation Innovation Council, alongside universities, industry partners, consultants, LTAP staff, and non-research UDOT leaders and staff members. Research Expertise The Research and Innovation Division’s staff members come from varied backgrounds. Research project managers may come from the research program or from other divisions of UDOT; their specialized research experiences are viewed as moderately important, more impor- tant than their experiences managing research projects. Research administration, library services, and contracting and procurement expertise are all housed within the Research and Innovation Division. IT, finance, accounting, and supple- mental contracting and procurement expertise for research is obtained from other UDOT divisions. UDOT uses non-university contractors for 20% of its research projects, and contractors are most important for providing capacity due to insufficient labor resources for UDOT to conduct the full research program. To some extent, contractors offer specialized skill sets that UDOT may not have readily available internally. Universities are more heavily relied on as research contractors with 70% of research projects being conducted by them. Additionally, universities participate widely in UDOT research activities through Technical Advisory Committee participation on research projects, Project Selection Committee participation at the UTRAC Workshop, State Transportation Innovation Council participation, and through university transportation centers. Outside of universities and other contractors, AASHTO, TRB, local associations, and tech- nology vendors are also main external partners relied on by UDOT’s Research and Innova- tion Division. The division is involved in AASHTO committees and the NCHRP process, TRB’s cooperative research programs, TRB committees, and states that TRB allows for sharing of research results. Local associations provide research ideas and needs to UDOT. Additionally, technology vendors provide research and deployment ideas and partnerships for test installations. Research Funding UDOT’s total research budget is approximately $3.5 million annually for its research projects and staff. In recent years, UDOT’s Research and Innovation Division has given between $100,000 and $150,000 to each of eight subject areas, depending on what problem statements are submit- ted and prioritized at the UTRAC Workshop. Figures 42 and 43 provide breakdowns of where UDOT obtains its research funding and where the agency spends its funds, respectively. The same support functions are used for all funding sources through set accounting systems, procurement procedures, legal support, and dedicated UDOT staff for managing specific funds. The Research and Innovation Division perceives compliance requirements to be of moderate difficulty to fulfill as a whole.

80 Managing State Transportation Research Programs State Planning and Research Funds 65% State Funds 20% Multi-State Pooled Funds 10% Other Federal Funds 5% Figure 42. UDOT research funding sources. UDOT Research 55% Research Program Administration 25% National Programs (AASHTO, TRB, etc.) 10% Multi-State Pooled Funds 5% UDOT Research Implementation 5% Figure 43. UDOT research funding areas.

Case Examples 81 Program Management UDOT uses a multifaceted method for prioritizing research areas and topics, through the UTRAC Workshop and council, open submission and voting, and focus groups. The UTRAC council and priorities defined at the annual workshop help inform the development of the divi- sion’s vision, goals, and objectives. To award projects, a project selection process is used, includ- ing prioritization by group discussion and voting at the annual workshop, recommendation of funded projects to the UTRAC council, approval by the council and the FHWA Utah Division, and involving the UDOT champion and technical advisory committee for each project. For prob- lem statements selected for funding that were co-submitted by a qualified researcher, UDOT typi- cally awards the research project to that researcher. Otherwise, a consultant or university is selected by the UDOT champion or technical advisory committee for the project. Sometimes, requests for qualifications or requests for letters of interest are sent to multiple firms and researchers. Selected projects and staff expenses comprise a $3.5 million research project budget, which covers a wide range of research programs such as Pavements, Bridges and Structures, Environ- ment, Planning, Safety, Traffic Operations, Finance, Geotechnical, Asset Management, Mainte- nance, Preconstruction/Design, and Public Transportation programs. Documenting, tracking, and sharing research broadly have been a challenging process that UDOT is working to improve. Research dissemination is tracked by the research program and the research library and is most frequently done through the website, research final reports, and the research newsletter as seen in Figure 44. Many research presentations and research briefs are also created to support research dissemination efforts. Public meetings sometimes occur. Less frequently used methods include journal articles, social media, and YouTube videos. This research is intended to be disseminated to state DOT leadership, the broader state DOT, other state DOTs, the federal government, and other researchers. Implementation responsibilities at Utah usually fall to the Innovation and Implementation Manager and/or the Research Implementation Engineer at the Research and Innovation Divi- sion (MDT 2017). Typically, there are UDOT staff specifically dedicated to support implemen- tation, but research funding usually does not depend on successful research implementation. Implementation is tracked by maintaining a list of completed research that is implementable and by assessing benefits and return on investment for implementation. A moderate amount of research is implemented, and the selection process for implementation may occur at any point in the research lifecycle—during research project selection, while nearing project completion, or when the project is completed. The project champions, research program, and funding con- straints determine what projects will be implemented. Program Quality UDOT has a quality control method in place for research activities—the quality of research deliverables is overseen by each research project manager, which mainly involves contract and schedule management and adherence to the final report guidelines, along with a technical review by the UDOT project panel. Most research is not peer reviewed, meaning that it is not evaluated by another researcher working in the same field. Oversight for project execution is done principally by the project managers and by research contractors. The research program manager and the project managers also have research budgeting responsibilities. Withholding payment for poor-quality research has only been considered a few times (Ohio DOT 2015c). On some projects, a delay in the final payment is made until the final report quality

82 Managing State Transportation Research Programs Source: UDOT Research Newsletter, Spring 2018, Page 1. Figure 44. UDOT research newsletter.

Case Examples 83 is improved, as long as the researcher is willing to improve and re-deliver it. However, even in cases of a poor-quality final report, payment for the work completed and the final deliverable is typically made to the researcher. Before the final payment, UDOT works with researchers to try to get revisions and improvements made to the report. This is based on technical feedback from the project’s technical advisory committee and review by the research project manager for elements not in compliance with the report guidelines or the research contract itself. Research project quality performance measures include completion of the project on sched- ule and in line with the budget, post-project tracking for implementation, and fulfillment of research objectives. Performance measures currently not used include assessing potential benefits prior to the beginning of the project and alignment to strategic objectives. Research evaluation is used in a myriad of ways including to help direct funding sources, inform new research project selection, share lessons learned from the research program, and improve research program processes and effectiveness. Research evaluation is not a factor in individual performance reviews. Program Value UDOT has a healthy research culture, spurred by the annual workshop which allows researchers to get involved with the Research and Innovation Division. Different DOT divisions can submit research needs, but UDOT has struggled with getting local regions involved. The Innovation and Implementation Group will help with broader research stakeholder engagement and collecting and supporting innovative ideas in UDOT. Research is well supported by leaders and various DOT divisions. Materials, maintenance, traffic management and safety, and planning depart- ments are especially champions of the research program. Utah DOT’s research program value and outcome assessment are measured qualitatively on a program-wide basis, such as improved processes, new standards, and lessons learned as a result of research. UDOT also periodically coordinates a value (cost-benefit) assessment of several recently completed research projects, based on performance and implementation feedback from research project champions, about every 4 years. This helps to provide a quantitative measure to support continuing the UDOT research program.