Management Strategy and Priority Setting
As part of its Phase 3 review of the 21st Century Truck Partnership (21CTP), the committee received presentations from the four participating agencies—Department of Energy (DOE), Department of Transportation (DOT), Department of Defense (DOD), and Environmental Protection Agency (EPA)—and the 21CTP industrial partners. These presentations included detailed responses to the concerns about the program’s overall effectiveness, funding mechanisms, priority setting, Partnership coordination and performance, and other 21CTP issues raised in the National Research Council’s Phase 1 (NRC, 2008) and Phase 2 (NRC, 2012) reports. The committee also collected information by reviewing documents and formulating questions to which the 21CTP provided detailed responses and by making site visits to several of the key partners. In addition, the 21CTP provided responses to the recommendations in the Phase 2 report (please see Appendix C).
In this chapter the committee reviews each of these areas of concern and reports its findings and recommendations. For background on the structure of the Partnership, the chapter also includes and summarizes relevant information from the NRC Phase 2 report (NRC, 2012).
As noted in Chapter 1, overall management of the Partnership currently resides with the DOE’s Vehicle Technologies Office—VTO, formerly the Office of Freedom CAR and Vehicle Technologies (FCVT)—in the Office of Energy Efficiency and Renewable Energy (EERE).
DOE personnel publish Partnership goals through white papers and roadmaps (21CTP, 2006, 2013), maintain the information-flow infrastructure, and organize meetings and conference calls. The management of individual projects under the 21CTP umbrella rests with the individual agencies that have funded the work. These agencies communicate with one another through the 21CTP information-sharing infrastructure in an attempt to coordinate their efforts and to ensure that valuable research results are shared and that any overlap of activities among their respective efforts is minimized.
Figure 2-1 illustrates the relations among the key participants in developing and conducting 21CTP research programs. Government agencies request funding from Congress through the administration and work with the industrial partners and research organizations, including universities and government laboratories, to establish research programs that meet national priorities and the interests of industry. However, final funding levels are determined by congressional appropriations, with each agency overseen by different
FIGURE 2-1 Relations between 21CTP participants. SOURCE: Submitted to the Committee on Review of the 21st Century Truck Partnership, Phase 2, by the DOE Office of Vehicle Technologies (January 29, 2011).
congressional committees. This makes prioritization of all of the 21CTP projects across the four agencies extremely difficult, if not impossible.
This limitation was discussed in the NRC Phase 1 and Phase 2 reports, and there are signs of some improvement in the coordination of projects in the subsequent 3 years, but the overall structure and funding of the government agency side of the Partnership remains a limitation to efficient program management.
In the NRC Phase 1 and 2 reviews, the previous committees asked for a specific list of projects within each agency deemed to fall under the 21CTP umbrella, the associated line-item funding, and overall agency budgets for 21CTP. While DOE was able to provide this information for its own projects, neither the Phase 1 nor the Phase 2 review was able to secure this information from DOT, DOD, or EPA.
Even in the case of DOE, light-duty and heavy-duty vehicle work often overlaps, in such areas as combustion or lightweight materials, for example. Accordingly, there is difficulty in parsing exactly which projects are part of 21CTP, although leveraging appropriate research activity across the entire vehicle spectrum, regardless of where the research program resides, is desirable wherever possible.
In addition, with the exception of the SuperTruck projects, it was difficult for the committee to ascertain the level of resources contributed by the private sector participants. Note, however, that DOE 21CTP activity (which accounts for 78 percent of total 21CTP funding) falls under the Energy Policy Act of 2005, which requires a 20 percent cost share for R&D projects and a 50 percent cost share for demonstration and commercial application projects (such as SuperTruck), so that the private sector’s contribution is at least 20 percent in all such activities (DOE, n.d.).
The situation improved somewhat by the time of this Phase 3 review: Led by DOE, the Partnership provided an inventory of projects categorized as falling under 21CTP (although detailed EPA information was still missing1). It also provided the estimated associated funding levels (see Appendix D) and a partial summary of federal funding (excluding American Recovery and Reinvestment Act [ARRA] funds) contributing to 21CTP goals (see Figure 1-1 in Chapter 1).
In DOE vehicle research, which specifically addresses the national issue of energy security and the increasing pressures from the rising global consumption of oil, the VTO has involved the affected industries in planning the research agenda and identifying technical goals that, if met, will provide the basis for commercialization decisions. The government’s approach is intended to allow industry-wide collaboration in precompetitive research, which is then followed by competition in the marketplace.
The Partnership provides a forum for the exchange of technical information among the industry and government partners involved in heavy-duty transportation. While this exchange of information provides an opportunity for coordination of relevant initiatives, what it does not, and cannot, do is provide single-point management or direction of such activities, or enable a “combined portfolio approach” to managing collaborative research efforts.
The four agencies involved in 21CTP have many areas of common interest, illustrated in Figure 2-2.
The Partnership points to examples of successful collaboration between agencies, such as the Advanced Vehicle Power Technology Alliance (AVPTA) between DOD and DOE, work between DOT, EPA, and DOE on the relationship between heavy-truck fuel efficiency regulations and 21CTP research, and collaboration between EPA and DOE on hydraulic hybrid research (since dropped) and the EPA SmartWay Transport Partnership.
The overall management structure of the Partnership remains largely unchanged since its inception in 2001 as a virtual network (see Figure 2-3). The industry side is led by an executive committee comprising one representative from each of the three major industry sectors involved: engines, truck original equipment manufacturers (OEMs), and hybrids. At the time of this review, those representatives were from Cummins, Volvo, and Eaton.
In Finding S-1 of the NRC Phase 2 report, it was noted that the Partnership needs to review whether additional partners—such as major truck and component manufacturers that are not currently members but that could contribute to the R&D program—should be recruited. No changes have been
FIGURE 2-2 Some areas of common interest among government agencies participating in 21CTP. SOURCE: K. Howden, DOE, “21CTP Overview,” Presentation to the committee on May 14, 2014.
________________
1 EPA data were presented as simply two line items, of which the larger, approximately $10 million, was that portion of the EPA budget estimated to relate to heavy-duty vehicle testing and certification, and not R&D.
FIGURE 2-3 Partnership organization. SOURCE: K. Howden, DOE, “21CTP Overview,” Presentation to the committee on May 14, 2014.
made to the Partnership structure since 2001, and meanwhile at least two industrial members have exited the on-road truck business, and consolidation in the industry has reduced six of the members to two separate corporate entities.
The executive committee meets by teleconference once every month and is chaired by the 21CTP director. The purpose of these meetings is to discuss 21CPT activities for the month and make any appropriate planning decisions that are best facilitated by a small representative group.
The Partnership also holds regular teleconference meetings of its full membership once a month. The purpose of these meetings is to facilitate open communication among the federal and industry partners about research activities, industry business, and technical accomplishments related to the Partnership’s goals and objectives. The meetings are chaired by the 21CTP director and separated into three distinct activities: a government-only discussion, a government-industry discussion, and an industry-only discussion. Participation in these calls involves all of the federal agency and industry partner organizations, and attendance ranges from 20 to 40 people per call. In addition to the regular teleconferences, the Partnership holds at least one, and usually two, in-person meetings a year at a partner location.
Agendas and minutes of these meetings are archived on the Partnership’s internal website. The committee has reviewed examples of these meeting minutes and found the meetings to be comprehensive, well attended, and productive.
The descriptions of the overall program management process, originally published in the NRC Phase 1 and Phase 2 reports, have been updated in the present report to reflect current Partnership practices. They reflect the Partnership’s responses to questions from the committee during this Phase 3 review, dated August 28, 2014.
The original partnership structure was judged in the Phase 1 and Phase 2 reviews to be far from ideal. Accepting in the Phase 2 review that the form of centralized management and control preferred by the committee was simply not feasible within the prevailing government and congressional structure, the committee recommended (Appendix C, Recommendation 2-1) as follows:
DOE is urged to continue to improve the functioning of the 21CTP “virtual” management structure in every way possible. Such improved functioning would include strengthening interagency collaboration…and documenting and publishing specific 21CTP activity within all four agencies.
In its response, the Partnership argued that its informal virtual organization offers some advantages but also noted it is exploring new communication methods and continuing its efforts to strengthen interagency partnerships, among other initiatives.
Overall, the committee accepts that the Partnership is striving to operate as effectively as possible despite a less than ideal organizational structure. In this regard, illustrated in Figure 2-4, the DOE has a dedicated leader assigned to 21CTP matters, even though his responsibilities are listed as “Aftertreatment” in the Advanced Combustion Engines group. It would be most helpful if this person reported directly to the VTO director for 21CTP matters and, furthermore, had designated 21CTP counterparts at the other three agencies.
The other program management recommendation in the NRC Phase 2 report dealt with the lack of a clearly defined inventory of projects and budgets included under the 21CTP umbrella, and recommended a brief annual report documenting the projects funded and the progress made (Appendix C, Recommendation 2-2). The initial response from the Partnership concurred that no such report exists, and promised to consider the development of such a document as a nonduplicative complement to other 21CTP-related reports.
Subsequently, in responses to follow-up questions from the committee, the Partnership indicated its intention to strengthen the coverage of 21CTP activity on the DOE website (which currently contains only the 2006 and 2013 roadmaps and white papers) and provided the committee with a first draft of a proposed periodic report of recent Partnership accomplishments with digital links to further detailed information. The committee considers these actions to be fully responsive to the prior recommendations (21CTP, 2013).
In addition to the committee’s hearing a DOD presentation on May 14, 2014, a committee subgroup visited the U.S. Army Tank Automotive Research, Development, and Engineering Center (TARDEC) in Warren, Michigan, on December 5, 2014, to review 21CTP-related projects. Although the DOD places a high priority on reduced energy consumption, it is of necessity focused on a totally different operating environment, one that is exempted from emissions
FIGURE 2-4 DOE VTO organization. SOURCE: Submitted to the committee by the DOE VTO, October 7, 2014. NOTE: The “21” notations indicate some 21CTP activity within the indicated groups. APEEM, advanced power electronics and electric motors; VSST, vehicle systems simulation and testing.
regulations and emphasizes high power density and JP-8 fuel. Consequently, there is little synergy with the needs of the commercial heavy-truck industry or with the stated 21CTP vision of “making trucks and buses safer, cleaner, and more efficient,”2 given the unique nature of DOD’s goals and their dissimilarity to those of 21CTP. Consequently, as shown in Figure 1-1 in Chapter 1, the Partnership lists DOD funding related to 21CTP goals as approximately $9 million, or only 6 percent of total 21CTP federal spending.
The national laboratories conduct many DOE programs synergistically with the 21CTP. Examples of such projects include advanced combustion research, fuels and lubricants, exhaust aftertreatment, lightweight materials, simulation software for combustion and engine operation, hybrid systems, and vehicle parasitic losses, among many others. A committee subgroup visited Oak Ridge National Laboratory on October 22, 2014, to review the activities related to 21CTP. Participation in 21CTP fosters ongoing technical interchange with industry at the working level, facilitating collaboration between the national laboratories, the government agencies, universities and industry, thereby ensuring that the national laboratories know industry’s needs and priorities. It also expands the awareness across industry of activity at the national laboratories beyond that reported at the DOE Annual Merit Review. Data provided by DOE in response to committee questions shows that DOE funding related to 21CTP was split fairly equally between industry and the national laboratories: In FY 2014 it was 49 percent each, and over the last three fiscal years, it was 43 percent for industry and 55 percent for the national laboratories. Universities received the remaining 2 percent. While the university share appears to be small, it does not include the fundamental research sponsored under joint National Science Foundation/DOE programs or the many instances of universities partnering with national laboratories or industry on their projects. The importance of maintaining connections to universities suggests that DOE needs to determine whether the funding levels are sufficient. These connections are important for encouraging students to work on research projects related to the automotive industry so that there will be a next generation of engineers educated for government and industry. Unfortunately, no similar breakdown was available for the other three agencies.
Using the 21CTP to leverage the enormous capabilities in the national laboratories to address the specific research needs of industry adds considerable value and will facilitate the timely transition of promising technologies from the laboratories into the marketplace.
________________
2 K. Howden, DOE, “21CTP Overview,” presentation to the committee, May 14, 2014.
As noted in Chapter 1, another important information-sharing activity supported by DOE was the Directions in Energy and Emissions Research (DEER) conference. This conference brought together professionals in the engine community and addressed research and projects related to the 21CTP. The conference has traditionally been an important meeting for bringing these professionals together for in-depth discussions of engines and issues related to their emissions, including a wide variety of experts not necessarily involved directly with the 21CTP. In this way, the researchers involved in the 21CTP projects can learn from a wide variety of engine and emission control experts. Unfortunately, the last DEER conference was held in 2012.
The organizational structure of 21CTP precludes any systematic prioritization of research projects for the total program across all the participants. Each of the four agencies included in 21CTP has its own separate budgets and priorities, and the industrial partners also have their own needs, priorities, and resources.
The Partnership provided an inventory of 162 21CTP projects in October 2014 and then provided an updated inventory on December 29, 2014 (see Appendix D); the committee asked for clarification of which projects were considered “key” and how key projects are prioritized. On October 24, the Partnership provided the response shown in Box 2-1.
The NRC Phase 1 report recommended the creation of “a portfolio management process that sets priorities and aligns budgets among the agencies and industrial partners” (NRC, 2008; Recommendation 2-2). The Partnership responded that the recommendation “will be considered . . .[but] the ability to directly align budgetary decisions across the agencies, however desirable, may be outside the scope of this voluntary collaborative organization.”
Since that time, the DOE has alluded periodically to the adoption of an Advanced Research Projects Agency-Energy (ARPA-E)-like portfolio management approach to its projects, and the committee asked if that approach had been adopted and how it operates. The response shown in Box 2-2 was received on October 24, 2014.
In addition, upon request, DOE provided examples of two projects that had encountered difficulties in achieving their respective goals and were substantially renegotiated and redesigned.
While these responses are clearly focused on DOE activities, they represent a welcome move in the direction of more aggressive program management and a portfolio approach, and the committee would like to see a similar approach taken at the other participating agencies.
A major component of 21CTP in recent years has been the SuperTruck initiative, enabled by the American Recovery and Reinvestment Act (ARRA) of 2009. SuperTruck was discussed in the NRC Phase 2 report and is covered here in detail in Chapter 8. Four teams have been awarded cost-sharing contracts to develop prototype Class 8 trucks employing many of the technologies being pursued by 21CTP, with very specific performance goals and timetables. Two teams are supported by ARRA funds and two teams benefit from DOE internal funds redirected to this purpose. The committee fully endorses this reprioritization of DOE funds to enable the SuperTruck projects to proceed and applauds SuperTruck’s emphasis on applying a total systems approach to evaluating and demonstrating the candidate technologies in real-world vehicle applications against stringent test criteria. The SuperTruck teams have made significant vehicle and engine progress, as described in chapters 3 and 8, and in so doing have shown the value of carefully designed demonstration programs to complement component and system technology research. As the four SuperTruck projects approach their conclusion, it is not too soon to develop proposals to build on their momentum and success and to prioritize the next set of objectives in a future resource-constrained environment.
BOX 2-1
Partnership’s Response to Committee Questions on Project Prioritization
The federal agencies that are members of 21CTP do not explicitly “prioritize” their ongoing projects after the portfolio has been established to identify a subset of them as being “key.” The federal agencies do broadly prioritize their research efforts when they are building their research portfolio, based on agency mission, funding levels, and other factors, however. The federal agencies prioritize their technical focus areas through their strategic planning and goal setting work, and then use a variety of mechanisms to identify performers (industry, academia, laboratories) to complete the work. The mechanisms are frequently competitive in nature, effectively prioritizing the proposed projects for each technical focus area as they are selected for funding.
For DOE, the industry-led projects chosen in response to a funding opportunity announcement are competitively selected (or “prioritized”) by technical review committees and DOE technical staff, who are selecting projects that are likely to be best able to contribute successfully to the relevant DOE goals. The laboratory-led projects are selected through a different process that still prioritizes the projects most likely to be successful and achieve critical DOE goals. DOE technical staff works with the laboratories to select the appropriate mix of projects, based on available funding and technical priorities. In both cases (industry and lab), the portfolio selections are reviewed each year at the Annual Merit Review, and changes to portfolio components can be made as a result of that feedback.
Box 2-2
Partnership’s Response to Committee Questions on Management Approach
The DOE Office of Energy Efficiency and Renewable Energy’s (EERE’s) program management approach is broadly similar to the ARPA-E approach, in that it involves establishing and tracking of critical project milestones, and close coordination with the project performers to ensure that milestones are met and any issues are identified and resolved in a timely manner (see the ARPA-E Strategic Vision explanation of their management approach at http://arpa-e.energy.gov/sites/default/files/ARPA-E_Strategic_Vision_Report_101713.pdf).
EERE maintains a Project Management Center (PMC, viewable at https://www.eere-pmc.energy.gov/) to assist DOE HQ technology managers and project performers in managing their projects, and to provide a common framework and business practices across the diverse group of EERE offices. The PMC supports EERE through two field offices in Golden (Colorado) and NETL (Pittsburgh and Morgantown). DOE EERE technology managers and representatives from the PMC work together with the project performers to negotiate project awards, establish and track milestones, actively review project progress on a quarterly basis (either via webinar or in-person meetings), and gather project management data using centralized EERE software tools to track progress. Complete details on how EERE program management is conducted may be found at http://energy.gov/eere/about-us/eere-program-management-guide. As with any project management system, the EERE system is continually being reviewed and refined to keep pace with current best practices and lessons learned.
In summary, the 21CTP continues to operate as a virtual network of government agencies, industry, and national laboratories, led by DOE using a relatively flat and informal management structure to discuss research priorities, communicate research successes, and provide feedback on future trends.
The Partnership has responded positively to prior NRC recommendations to improve the functioning of this virtual management structure, particularly across the other three agencies involved; publish an inventory of Partnership projects and associated budgets; and consider dedicated communication, such as a brief annual report, of Partnership activities and accomplishments.
Finding 2-1. The 21CTP remains a virtual organization facilitating communication among four government agencies, the national laboratories, and industry, led by DOE but with no single-point authority over its activities, priorities, or budgets. While far from optimal, this structure is necessitated by the separate reporting and budgeting mechanisms for each agency. The Partnership has made good progress in adapting to this reality by improving communications, coordination and collaboration among the partners, and documenting most of the projects and budgets under the 21CTP umbrella.
Recommendation 2-1. The DOE is urged to continue to improve by maintaining and publishing the inventory of projects and budgets across all four agencies, tying those projects into the specific 21CTP goals and promoting a portfolio management approach or the DOE’s Office of Energy Efficiency and Renewable Energy’s Project Management Center (EERE PMC) equivalent within the other agencies. Furthermore, EPA, DOT, and DOD should appoint a dedicated counterpart to DOE’s designated 21CTP leader, who in turn should report directly to the director of the Vehicle Technologies Office on 21CTP matters.
Recommendation 2-2. The Partnership should develop and adopt criteria for including projects under the 21CTP umbrella, such as “Does the project clearly address one of the specific goals of 21CTP?” and “Does the project fall within the R&D interests of the member partners of the 21CTP?” The committee recognizes that there will be at least two levels of projects—those tightly connected to specific 21CTP goals and a supporting set of projects that have longer term impact. Better definition of the criteria for including a project and at what level would assist in evaluating and increasing the effectiveness of the Partnership.
Finding 2-2. While many projects deemed to fall under the 21CTP umbrella are reviewed in their own right at the annual DOE Merit Review and the Directions in Engine Efficiency and Emissions Research (DEER) conferences (until the latter ended in 2012), and the SuperTruck projects have an annual reporting requirement, there remains no dedicated report on 21CTP activities, in any medium. In response to prior NRC recommendations, DOE has proposed the development of such a report and given the committee a first draft proposal.
Finding 2-3. The annual DEER conferences sponsored by DOE for over 25 years were an excellent way to share research results among industry, national laboratory, government, and university personnel, but they were not held in 2013 and 2014.
Recommendation 2-3. The DOE should publish a brief annual report on 21CTP activities and accomplishments, such as the first draft provided to the committee on October 24, 2014, with references to published technical reports from all four agencies and the national laboratories. This would be a great help to Congress and to future review committees and the public in understanding the work and scope of the Partnership.
Recommendation 2-4. Because the DEER conference was an excellent approach to communicating research results of the 21CTP, the Partnership should consider holding it in 2015 and each year thereafter. If funding constraints prevent an annual meeting, it should be held at the very least every other year.
21CTP (21st Century Truck Partnership). 2006. 21st Century Truck Partnership Roadmap and Technical White Papers. 21CTP-003. December. Washington, D.C.: Office of FreedomCAR and Vehicle Technologies.
21CTP. 2013. Roadmap and Technical White Papers. Department of Energy, Office of Energy Efficiency and Renewable Energy. https://www1.eere.energy.gov/vehiclesandfuels/pdfs/program/21ctp_roadmap_white_papers_2013.pdf.
DOE (Department of Energy). n.d. Chapter 35.2, Cost Sharing in Research and Development Contracting. http://energy.gov/sites/prod/files/35.2_Cost_Sharing_in_Research_and_Development_Contracts_0.pdf. Accessed March 18, 2015.
National Research Council (NRC). 2008. Review of the 21st Century Truck Partnership. Washington, D.C.: The National Academies Press.
NRC. 2012. Review of the 21st Century Truck Partnership, Second Report. Washington, D.C.: The National Academies Press.
