Review of the Federal Railroad Administration’s Research and Development Program (2020)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

11 1 Introduction The Federal Railroad Administration’s (FRA’s) Office of Research, Devel- opment, and Technology (RD&T) asked for this National Academies of Sciences, Engineering, and Medicine (the National Academies) review of the products and services that RD&T provides to other divisions of FRA and the railroad industry in accordance with its mission to ensure the safe, efficient, and reliable movement of people and goods by rail through ap- plied research and the development of innovations and other solutions. Spe- cifically, RD&T asked the National Academies’ Transportation Research Board (TRB) to convene a committee of experts to review the quality and relevance of RD&T’s current and planned research portfolio and to provide advice on strategies to better identify research needs, conduct high-quality research, and ensure that research products contribute to FRA’s primary goal of improving railroad safety. FRA oversees the safety of the nation’s commuter and intercity pas- senger railroads, which have carried about 680 million passengers per year, and freight railroad system, which has transported about 1.4 billion tons of freight per year on more than 135,000 miles of track (Bureau of Trans- portation Statistics 2020a,b; Federal Railroad Administration 2020a). To support FRA’s mission, RD&T seeks to deliver high-quality and impactful research results covering four main areas: track and structures, human fac- tors, rolling stock, and train control and communications. This research, which is carried out by contractors that include railroad operators, con- sultants, university researchers, and technical experts from other research institutions, is guided by a 5-year strategic plan.

12 REVIEW OF FRA’S R&D PROGRAM RD&T requested this review specifically to provide strategic feed- back, including feedback on the effectiveness of its research R&D support functions (i.e., planning, communications, and evaluation). The full study charge, or Statement of Task, is shown in Box 1-1. It contains a series of questions pertaining to the quality, relevance, and impact of the program. Some of the questions concern the performance of RD&T’s divisions re- sponsible for each of the four main areas of research. They ask whether the division has • Excelled in engaging, maintaining communication with, and using inputs from the full range of stakeholder groups (Question 1); • Excelled in conducting and using results from needs assessments and diagnostic studies to prioritize, focus, and plan projects and programs (Question 2); • Demonstrated sufficient flexibility and responsiveness to address changing economic, political, social, and technological contexts (Question 4); • Produced high quality work that is appropriate and feasible for implementation (Question 7); • Ensured that its R&D services and products are being used by FRA and the railroad industry (Question 8); and • Helped the railroad industry improve safety and reduce fatalities (Question 9). Other questions concerning R&D support functions ask whether RD&T • Engages in planning that is guided by a well-defined mission with associated goals and priorities that reflect safety needs in the rail- road industry (Question 3); • Takes steps to evaluate and ensure the usability and likelihood of adoption of research results by the railroad industry (Question 10); • Assesses the overall impact of the research and communicates the results to key stakeholders using means such as summative evalua- tion reports, technical reports, and conference presentations (Ques- tion 11); and • Sets budgets and staffing levels that are suited to addressing estab- lished goals and priorities (Question 6). These questions are similar to those asked by RD&T in commission- ing several previous TRB reviews. The most recent review, completed in 2015, contained several recommendations about communications, priority

INTRODUCTION 13 BOX 1-1 Statement of Task An ad hoc committee will conduct an expert review for the Federal Railroad Administration (FRA) Office of Research and Development (R&D)a of the U.S. Department of Transportation (U.S. DOT) to assess its R&D products and ser- vices to the agency and railroad industry. The goal of the review will be to provide strategic feedback to the Office of R&D for program improvement and planning purposes with specific emphasis on (1) validating FRA’s process to identify new priorities for addressing emerging safety issues and trends, and (2) evaluating the feasibility, usefulness, effectiveness, and impact of R&D products and ser- vices in railroad safety. The committee will evaluate each of the four major divi- sion areas (Track, Rolling Stock, Train Control and Communications, and Human Factors), including cross division efforts, and R&D support functions (planning, evaluation, and management). In conducting the review, the committee will consider the following questions: 1. To what extent has the Office of R&D excelled in engaging, maintaining communication with, and using inputs from the full range of stakeholder groups? 2. To what extent does R&D excel in conducting and using results from needs assessments and diagnostic studies to prioritize, focus, and plan projects and programs? 3. To what extent has R&D’s planning support function defined a sound mission and associated goals and priorities that reflect assessed safety needs in the railroad industry? 4. To what extent is R&D sufficiently flexible and responsive in addressing changing economic, political, social, and technological contexts? 5. To what extent does R&D’s current and planned portfolio and budget appropriately address its defined mission, goals, and priorities? 6. Do the Office of R&D’s defined mission and priorities align with its staff- ing and budget levels? 7. To what extent is the R&D Office’s science and engineering work of excellent technical merit and quality, and appropriate and feasible for implementation? 8. To what extent are R&D services and products being, or planned to be used by the railroad industry both internal and external to FRA? 9. How effectively have R&D services and products helped the railroad industry improve safety and reduce fatalities? 10. To what extent does the Office of R&D evaluate its services and prod- ucts prior to or during an implementation to help improve their usability and likelihood of adoption by industry? 11. To what extent is the Office of R&D effective in providing its key stake- holders with summative evaluation reports, technical reports, confer- ence presentations, and other communications that validly assess R&D efforts, impacts, and cost-benefits? continued

14 REVIEW OF FRA’S R&D PROGRAM setting, strategic planning, and evaluation (National Academies of Sciences, Engineering, and Medicine 2015). Notably that report recommended that RD&T • Increase its interaction with the engineering, mechanical, opera- tions, and research staffs of the railroads and industry suppliers to improve research dissemination and increase awareness of industry priorities; • Formalize communication with FRA’s Office of Railroad Safety (RRS) on projects of mutual interest and engage with this of- fice to formally evaluate research intended to support safety rulemaking; • Carry out project selection with the assistance of a software-based decision-support tool and by prioritizing projects that have an implementation partner at a railroad or industry supplier; • Regularly monitor its progress toward the goals in its strategic plan and use its observations on progress periodically to adjust resources as needed to achieve strategic goals; • Identify long-term needs in domain knowledge and talent among its personnel and key contractors and implement a plan to meet those needs; and • Establish evaluation processes for each project, including the use of review panels of external experts. RD&T’s progress in responding to these earlier recommendations was considered as part of this review, which was conducted using the methods discussed next. BOX 1-1 Continued The committee will deliver two letter reports on its findings and recommen- dations. The first letter report will include descriptive assessments and action- able comments and advice on the evaluation questions. It will present a holistic assessment of the Office of R&D, the individual divisions, and the R&D support functions. The second letter report will provide an evaluation of the content and delivery of a 2-day, public meeting in which FRA will present its R&D program to stakeholders. a The Office of Research, Development, and Technology is the current name.

INTRODUCTION 15 APPROACH TO CONDUCTING THE REVIEW To conduct the review, the National Academies appointed a committee of 14 experts in fields relevant to the study charge, including railroad safety, track and bridge infrastructure, train control and communications, rolling stock, human factors, railroad management and labor, and research pro- gram evaluation. As required by the Statement of Task, the 14 members were each assigned to one of four subcommittees charged with conducting more detailed reviews of the work of the four RD&T divisions (see Figure 1-1). The subcommittees reported back to the full committee, which re- viewed RD&T’s overall support functions. While the subcommittees’ work informed the majority of the findings and recommendations in the report, they were all reviewed and accepted by the full committee. The committee commenced its work by meeting with RD&T’s senior management, division chiefs from each of the four research areas, and pro- gram managers responsible for specific research projects. These meetings consisted of high-level presentations describing RD&T’s mission and goals and strategic planning and other support functions, as well as overviews of the portfolios, budgets, and staffing of the four divisions. During these meetings, the committee reviewed RD&T planning documents, such as the Annual Modal Research Plan (Federal Railroad Administration Office of Research, Development, and Technology 2018b), and queried senior managers, division chiefs, and program managers on project selection and prioritization processes, including their use of decision-support software. The management team was asked to explain how RD&T uses information on safety trends in the railroad industry to inform its program planning, research needs prioritizations, and budget allocations. Managers were also asked to explain how they engage with internal and external interests such as the safety regulators in RRS, other modal administrations within the U.S. Department of Transportation, freight and commuter railroads, railroad labor organizations, and the broader research and technical communities. The committee also met with officials from RRS, including those having responsibilities specific to the subject matter of the four RD&T divisions. They were asked to explain their research needs and reliance on RD&T products and services. These discussions were deemed important because many of RD&T’s products and services are intended to help RRS develop regulations and enforcement tools and methods. In consulting with RRS, the committee recognized the inherent tensions that can exist in an agency that has both regulatory and research responsibilities. It is essential that RD&T be responsive to the agency’s need to improve its safety regulations and enforcement tools, while also engaging with industry for information on and insight into safety problems that research can help address. One

16 REVIEW OF FRA’S R&D PROGRAM way to facilitate the candid sharing of sensitive safety information is the Railroad Information Sharing Environment, which is under development by RD&T and builds on FRA’s existing Confidential Close Call Reporting Sys- tem. The committee took the view that an intangible benefit of the RD&T program is that it can provide this important nexus of open dialogue for the pursuit of safety improvements. To conduct their work, the four subcommittees met separately with the leadership and program managers from each of the four divisions. During these in-depth sessions, which included in-person meetings and conference calls, division staff were asked to explain their research priorities, provide an overview of their research portfolios, and offer a more detailed descrip- tion of a small sample of recent and ongoing projects. Although too few in number to be representative of all of the work of each division, the sampled projects provided the subcommittees with additional insight into the meth- ods used by the divisions to identify needed research, procure projects, and disseminate and communicate results. To provide insight into how the four divisions set priorities for allo- cating budget, selecting projects, and managing the two research program areas, the committee asked FRA for data on derailments, their causes, and their severity and then to explain if and how such data are used for bud- geting and project programming. The subcommittees then examined the extent to which budget allocations and project selections appear to align with what the incident data suggest are the most significant safety concerns and incident causes. In their review of the data and its presentation in the report, the subcommittees opted against normalizing the frequency data on incident causes (e.g., Chapter 2, Table 2-1), which would typically be an important step in data analysis. They did so for various reasons, including the lack of a clearly appropriate metric with which to normalize, especially one that could be associated with the frequency of any given incident causal factor. FIGURE 1-1 Organizational chart, RD&T in FRA. Track Research Division Rolling Stock Research Division Train Control and Communications Research Division Office of Research, Development, and Technology (Director) Human Factors Research Division

INTRODUCTION 17 The purpose of requesting incident data was to determine if the re- search divisions appeared to be focusing their research on the most salient problems. It is important to emphasize that the subcommittees did not try to use the incident data to assess whether the research was successfully solving those problems or leading to increased safety outcomes. The com- mittee was not charged with conducting such research project and program effectiveness evaluations, nor can they be done simply by examining inci- dent data. However, the committee was asked to assess RD&T’s means for evaluating the effectiveness of its projects and programs, which are the subject of the study’s recommendations. In addition, the subcommittees consulted a range of railroad industry professionals with backgrounds in freight and commuter railroading, aca- demic and private sector research, and consulting. These professionals were selected from a pool of researchers and industry research partners who had worked on FRA projects and based on suggestions from the Association of American Railroads. Although the interviews were not highly structured or accompanied by a formal questionnaire, these individuals were asked for candid feedback on the quality and relevance of RD&T’s research as well as its methods of outreach in identifying research needs and in communi- cating, disseminating, and evaluating results. The individuals consulted are included in the Preface and in Appendix B. Following the completion of the subcommittees’ reviews, the committee as a whole convened to discuss their findings and recommendations, which are contained in each of the chapters of this report. There was considerable commonality in subcommittee observations and the recommendations that stem from them. This overlap could be expected because the divisions fol- low many of the same processes for identifying research needs, procuring projects, and disseminating and communicating results. It merits noting that the Statement of Task calls for the committee to conduct its review in two phases, each producing a short letter report. The first phase was supposed to contain descriptive assessments and ad- vice on the specific questions in the Statement of Task. The second phase was supposed to entail a 2-day, public meeting (or workshop) in which RD&T would present its R&D program to stakeholders, after which the committee would deliver a second letter report that summarizes and assess the discussions from the workshop. In concurrence with RD&T, the study committee chose to forego the plan for a workshop and focus instead on consulting with individual stakeholders in conjunction with the four divi- sion reviews. This decision to amend the study work plan, a prerogative of the committee in deciding how best to respond to the substantive elements of the Statement of Task, stemmed from concern that a short letter report could neither adequately describe the content and functioning of the four research divisions nor convey the committee’s review. While the summaries

18 REVIEW OF FRA’S R&D PROGRAM that are provided in this longer report of the feedback from consulted stakeholders is not a substitute for the feedback that could be anticipated from a public workshop, the committee concluded that the planning and organizing needed to design, arrange, and execute a workshop that would be sufficiently insightful—and not an exercise aimed mainly at demonstrat- ing outreach—was beyond its capabilities. However, the importance to RD&T of convening such workshops on a regular basis was recognized by the committee, as conveyed in its recommendations. REPORT ORGANIZATION The results of the committee’s review are presented in the five chapters that follow. Chapters 2 through 5 examine the work of the four research divi- sions in some depth. The chapters are organized similarly, starting with a more detailed review of the safety issues and problems relevant to each divi- sion—again by reviewing FRA data on incidents and their reported causes. In each case, consideration is given to how research needs are identified and priorities set both on the basis of these data and other inputs, such as by consulting with RRS, industry, and experts from academia and other research institutions. Each chapter contains a synopsis of consultations with external parties, and three to five sampled projects are discussed in most chapters with an eye to how the research topic was identified and selected, how the work was conducted (e.g., any partnering with industry), and how the research results have been, or expect to be, communicated and used in the field. Informed by this review, the subcommittees responsible for each chapter offer observations and recommendations with the concurrence of the full committee. The report concludes with Chapter 6, which evaluates and provides advice on RD&T’s research support functions, focusing on those related to strategic planning, communications, and impact evaluation. The chapter concludes with observations and recommendations pertaining to these sup- port functions, informed by the subcommittees’ reviews of the four RD&T divisions. Appendix A contains data on FRA incident reports from 2015 to 2019. These data are referenced in the individual chapters when considering the extent to which research portfolios reflect safety problems observed in the field. Appendix B lists all meeting dates and study participants. Appendix C includes the biographic information of the committee members.