Managing State Transportation Research Programs (2019)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

6 Purpose of This Synthesis This synthesis takes a broad look at how state DOT research programs with differences in agency needs, resources, and constraints are able to produce programs of high quality and value. The study addresses the question “How do transportation agencies organize and manage their research programs to strive for quality and positive impacts on the transportation system over time (value)?” What Is Transportation Research? Transportation research is a broad field of intellectual and technical activities with the ultimate aim of enhancing the transportation system. To this end, research is defined as an endeavour that “comprise[s] creative work undertaken on a systematic basis in order to increase the stock of knowledge, including knowledge of man, culture and society, and the use of this stock of knowl- edge to devise new applications” (OMB 1999). Transportation research can then be defined as the set of activities that contributes specifically to the stock of knowledge and the application of knowledge to enhance the transportation system. There are two broad types of research—basic and applied—defined by the kind of knowl- edge being pursued and the degree to which that knowledge immediately addresses a practi- cal problem. Basic research targets fundamental knowledge dealing with physical, chemical, material, or other behavioral and natural phenomena. The goal of basic research is to improve collective understanding of nature and such phenomena (National Science Foundation 1953). It usually takes a long time, possibly decades, for various types of knowledge gained through basic research to converge in a manner that addresses a practical problem. On the other hand, some knowledge has already advanced to a stage where, with a little additional effort, it can be combined with other known facts to address a problem. Such research is called applied research because of the focus on near-term application, within a period of months or a few years. Applied transportation research strives to integrate knowledge across diverse disciplines to solve practical problems facing the transportation system. The differences in focus and nature of basic and applied research imply that different capabilities and processes are needed to pursue each kind of research. Universities whose primary functions are fundamental research and education are usually well equipped to pursue basic research and some types of applied research. Private companies such as auto- motive and industrial firms focus primarily on applied research to develop new or improved products and services. Government agencies such as state DOTs engage in applied transpor- tation research to advance policy goals, improve system performance, and address persistent challenges facing the system. Applied research leads to positive impact and an enhanced Introduction

Introduction 7 system only when the knowledge gained through research is translated into practice, through knowledge and technology transfer. Agency research is undertaken to solve specific agency problems. Research, development, and technology transfer tend to be used interchangeably in fields of industrial activity. However, technology transfer often implies the application of research findings through technology deployment. State DOTs typically refer to this type of activity as research implementation, which is to “demonstrate, pilot, or evaluate an R&D output, and/or facilitate the transfer of an R&D output to an adoption-ready state” (U.S. DOT 2017). It follows that agency research and implementation are closely related. Research is challenging because of some of its fundamental attributes. Research is time intensive, often integrates diverse disciplines, requires deep technical or policy knowledge, exhibits uncertain outcomes, and must be assessed using a mix of quantitative and qualita- tive assessments. For these reasons, transportation research is very different from many other state transportation agency functions such as planning, programming, construction, main- tenance, and operations. State DOTs that choose to pursue research therefore intentionally organize for research. How Do State DOTs Organize for Research? Agencies must develop their research and related knowledge building and training in a manner suited to their agency-wide mission and capabilities. There should be a meaningful link between an agency’s research and its other strategic objectives. Since every state DOT’s goals vary with the unique attributes of their systems, great variation in how state DOT research programs are organized can be expected. State DOTs organize their transportation research programs in different ways. Some of these differences manifest in program capabilities (what is the program designed and struc- tured to do?), whereas other nuances are observed in program management processes (how does the program accomplish its functions?). These two related ideas of program capability and program management can be thought of as the inputs or enabling requirements of a state DOT research program. The outputs and outcomes of a program are then captured through the two ideas of program quality and program value. The inputs and outputs are naturally related in that a DOT must coordinate program capability and management to not only accomplish research but to also maintain program quality and program value over time. These four core ideas—program capability, program management, program quality, and pro- gram value—are the dimensions of this synthesis framework. Figure 2 defines them in detail. It also describes some fundamental building blocks of each definition; for example, research direc- tion, research expertise, and funding are all aspects of program capability. Finally, “knowledge area” questions are used to review the literature and assess what is already known about state DOT research programs. Synthesis Methodology and Outline Literature Review This study began with a wide-ranging literature review: the study team reviewed most of the available literature and published surveys addressing state DOT programs spanning the last 15 years. Knowing that a variety of forums and databases feature reports and publications on

8 Managing State Transportation Research Programs transportation research, the study team identified sources in two ways: through structured key- word searches and website exploration. Structured Keyword Searches The study team also searched bibliographic databases using combinations of the following keywords, listed in Figure 3. The study team conducted searches in the Transportation Research International Database (TRID). TRID is an integrated database that combines the records from TRB’s Transportation Research Information Services (TRIS) Database and the OECD’s Joint Transport Research Figure 2. Framework for understanding state transportation research programs.

Introduction 9 Centre’s International Transport Research Documentation (ITRD) Database. TRID provides access to more than one million records of transportation research worldwide. Google Scholar was used as secondary database source to ensure that no salient or relevant sources were missed, although overlap between the two sources was significant. Using these two databases resulted in saturation for cataloged sources, i.e., those publications that were tagged or associ- ated by keyword. Website Exploration Since many reports and publications (especially those published prior to widespread digi- tization) are not cataloged as above, the study team also inspected the websites of a variety of organizations for completeness: • TRB’s Cooperative Research Program publications • AASHTO • State DOT websites All information obtained from published reports and documents has been systematically cited in the discussion below. The main categories of sources and key publication titles are provided in Figure 4. List 1 List 2 List 3 List 4 Transportation Agency Research Program Innovation Implementation Capability Management Quality Value Culture Direction STIC RAC Leadership Strategic Plan Funding Budget Compliance Expertise Recruiting Contracting Prioritization Project Selection Procurement Award Dissemination Return on Investment Figure 3. Keyword combinations for structured keyword searches.

10 Managing State Transportation Research Programs Source Type Report Annual Peer Exchange Reports (2007–2017) 2007 Minnesota Research Peer Exchange 2013 Florida Research Peer Exchange 2013 Wisconsin Research Peer Exchange 2014 Washington Research Peer Exchange 2015 Four-States Virtual Research Peer Exchange 2015 Ohio Research Peer Exchange 2016 Vermont Research Peer Exchange 2016 California Research Peer Exchange 2016 New Hampshire Research Peer Exchange 2017 Maine Research Peer Exchange 2017 Kansas and Missouri Peer Exchange AASHTO RAC Reports and Surveys (2008–2017) 2008 Washington: Research Project Management Database 2011 RAC-CUTC Liaison Group: State University Partnerships and Agreements 2011 Oregon: Research Project Selection Survey 2013 Arizona: Employee Mentoring Programs 2013 Louisiana Transportation Research Center: Research Project Management Software 2014 RAC Value of Research Task Force: Performance Measurement Survey 2015 RAC Value of Research Task Force: Marketing Survey 2015 Ohio: Quality of Final Research Reports 2015 Montana: Contracting Research Projects—Payments by Deliverables and Incentives/Disincentives 2016 Connecticut: Peer Exchange to Improve Implementation of Research 2017 RAC Region 1: AASHTO RAC Member Involvement 2017 Connecticut: Survey on Data Warehouses 2017 Kansas: Request for Overhead Rates Previous NCHRP Studies (1999–2018) NCHRP Synthesis 280: Seven Keys to Building a Robust Research Program (1999) NCHRP Synthesis 300: Performance Measures for Research, Development, and Technology Programs (2001) NCHRP Handbook 610: Communication Matters: Communicating the Value of Transportation Research (2009) NCHRP Synthesis 127: Performance Measurement Tool Box and Reporting System for Research Programs and Projects (2011) NCHRP Synthesis 461: Accelerating Implementation of Transportation Research Results (2014) NCHRP Research Report 874: The Transportation Research Thesaurus: Capabilities and Enhancements (2018) Figure 4. Main publication categories and publication/source titles.

Introduction 11 Online Survey This literature review identified knowledge gaps. For example, there are few published statistics on the range of agency research budgets across state DOTs, the distribution of amounts from various funding sources, and how agencies allocate these to different streams of research. There have been few attempts to connect these resource aspects with program management choices, or with program outcomes such as the degree of implementation. Other outstand- ing questions include how agencies assess the quality of their research, and which means of dissemination they deem most valuable. Although there are many variations in program structures and processes, it becomes clear that these can be better characterized through primary data collection. An online survey was created based on the knowledge gaps identified in the literature review. The survey consisted of 46 questions covering all four dimensions of research programs— program capability, program management, program quality, and program value. A copy of the survey is included in Appendix A. A total of 44 unique state DOTs participated in this survey—a response rate of 88%. Survey respondents are listed in Appendix B, and comprehensive responses to individual survey questions are in Appendix C. The survey questions consisted mainly of closed-ended or quick-response questions, which enabled cross-tabulation of responses to understand how different types of agencies encounter different types of challenges and methods of organization and management. Case Examples To gain an in-depth understanding of research program management, the team then conducted case example interviews with five states—District (of Columbia) Department of Transporta- tion (DDOT), Louisiana Department of Transportation and Development (Louisiana DOTD), Source Type Report Other Research Studies (2013–2018) MnDOT TRS 1307: Cost Sharing Policies of State Transportation Agencies (2013) TRB SR313: Framing Surface Transportation Research for the Nation’s Future (2014) LTRC Report 512: STC Synthesis of Best Practices for Determining Value of Research Results (2014) GDOT Research Project 16-07: “Transportation Research Implementation Management: Development of Performance-Based Processes, Metrics, and Tools” (2018) Strategic Research Plans, where available Vermont Agency of Transportation, District (of Columbia) Department of Transportation, Minnesota Department of Transportation, California Department of Transportation Agency Annual Reports, where U.S. Department of Transportation, Minnesota Department of Transportation, California Department of Transportation, Ohio Department of Transportation, available Washington Note: RAC = Research Advisory Committee. Department of Transportation Figure 4. (Continued).

12 Managing State Transportation Research Programs Minnesota DOT, Ohio DOT, and Utah DOT. These states were chosen on recommendation by state DOT survey participants, the NCHRP Synthesis Panel, and by the study team in order to gain a diverse range of perspectives as to how agencies of different sizes, budgets, research models, and geographical locations manage their research programs for program quality and value. Synthesis Outline The rest of this report is structured as follows: • Literature Review: The next section synthesizes the literature on state transportation research programs and informs the framework dimensions of program capability, program manage- ment, program quality, and program value. • Survey Findings: A section on survey findings is then presented, which helps illuminate the literature blind spots (analysis of responses to each survey question can be found in Appendix C). • Case Examples: The report then offers five research program case examples to concretely link program practices, successes, and opportunities using the four dimensions of the framework, and to add color and texture to the survey findings. • Conclusion: The report concludes with the main findings and suggestions on possible areas for further research. • Appendices: A copy of the survey is included in Appendix A. The full list of survey respon- dents is in Appendix B, and the comprehensive responses to each survey question can be found in Appendix C.