National Academies Press: OpenBook

Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306 (2011)

Chapter: 3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals

« Previous: 2 Science and Technology Shaping Future Naval Fleets
Page 66
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 66
Page 67
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 67
Page 68
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 68
Page 69
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 69
Page 70
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 70
Page 71
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 71
Page 72
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 72
Page 73
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 73
Page 74
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 74
Page 75
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 75
Page 76
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 76
Page 77
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 77
Page 78
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 78
Page 79
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 79
Page 80
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 80
Page 81
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 81
Page 82
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 82
Page 83
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 83
Page 84
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 84
Page 85
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 85
Page 86
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 86
Page 87
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 87
Page 88
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 88
Page 89
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 89
Page 90
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 90
Page 91
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 91
Page 92
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 92
Page 93
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 93
Page 94
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 94
Page 95
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 95
Page 96
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 96
Page 97
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 97
Page 98
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 98
Page 99
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 99
Page 100
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 100
Page 101
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 101
Page 102
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 102
Page 103
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 103
Page 104
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 104
Page 105
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 105
Page 106
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 106
Page 107
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 107
Page 108
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 108
Page 109
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 109
Page 110
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 110
Page 111
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 111
Page 112
Suggested Citation:"3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals ." Transportation Research Board. 2011. Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306. Washington, DC: The National Academies Press. doi: 10.17226/13191.
×
Page 112

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

3 National Naval Responsibility for Naval Engineering Mission and Process for Achieving Goals A robust naval engineering science and technology (S&T) enterprise that supports the needs of the current and future Navy must perform its core functions effectively and efficiently, consistent with its mission and the expectations of a high-reliability research organization (Pelz 1956; Roberts 1990). The core functions include • Establishing the research agenda and allocating resources, • Identifying performers, • Measuring outcomes and evaluating results, • Maintaining connections among the wider naval engineering com- munity, and • Developing the requisite human capital to sustain the nation’s naval engineering capability. While effective performance of these functions is necessary, it is not sufficient for success in complex, dynamic research enterprises (NRC 1999; National Academies 2005). In addition, a high-performance orga- nization such as the Office of Naval Research (ONR) must clearly artic- ulate its mission and goals; measure and reward performance against those goals; incentivize and educate participants about desired organi- zational performance; and develop a robust continuous process improve- ment activity that assesses organizational performance; communicates best practices and lessons learned; provides for systematic dissemination of goals, activities, and achievements; and assesses organizational, group, and individual performance over time (Roberts 1990; Grabowski and Roberts 1999). These challenges are compounded for research organiza- tions whose missions involve interdisciplinary research, such as naval 66

National Naval Responsibility for Naval Engineering Mission 67 engineering. In such research organizations, measures and metrics of performance need to address the degree of integration and interdiscipli- nary activities required for mission success (National Academies 2005; Porter et al. 2006). This chapter presents a description of National Naval Responsibility for Naval Engineering (NNR-NE) core functions along with an examination of the NNR-NE’s interdisciplinary and integrative science and technology efforts. It also examines how well ONR performs its core functions and how effectively it achieves successful outcomes. ONR and its NNR-NE initiative have multiple processes and proce- dures in place that the committee believes are meant to support both the core and the integrative functions. For example, ONR has developed a Naval S&T Strategic Plan (ONR 2009b) that outlines the S&T vision and key objectives in 13 naval focus areas. ONR also tracks and reports on a variety of metrics, including the number of refereed papers that grow out of the projects it funds, the number of students it supports, and the num- ber of advanced degrees completed by individuals its funds support. However, the committee sensed that these individual processes and pro- cedures were not integrated into a cohesive whole that would support the alignment of NNR-NE’s research agenda, resources, activities, and incentive structure to its goals or to measurable objectives and outcomes. The following sections describe each of the NNR-NE core functions and how ONR’s processes support the NNR-NE mission. In addition, alternative methods to enhance organizational, individual, research, and educational performance are presented. ESTABLISHING THE RESEARCH AGENDA AND ALLOCATING RESOURCES As discussed in Chapter 1, naval engineering was designated a National Naval Responsibility in a 2001 ONR memorandum that specified the purpose of the designation and the activities that were to constitute the NNR-NE (ONR 2001). ONR was already engaged in all or nearly all of the specified activities before the memorandum was issued. Rather than initiating new programs, the memorandum served as a declaration of policy: assigning the NNR designation indicated that (a) the listed activ- ities are deserving of special priority in planning and budgeting at ONR because the identified S&T fields are critical to the Navy and no one else

68 Naval Engineering in the 21st Century will support them and (b) management of these activities must be coor- dinated with the declared policy objective in mind. The 2001 ONR memorandum set out the broad outlines of the orga- nization’s research agenda, envisioning an NNR-NE set of disciplines focused on the “development of educated and experienced people, expan- sion of the knowledge base, and cultivation of a climate supportive of innovation.” It also called on ONR to “formulate and maintain invest- ments” in these science and technology areas: ship design tools, ship structural materials, hydromechanics, advanced hull designs, ship propulsion, ship automation, and systems integration (ONR 2001). ONR has regrouped the NNR-NE S&T areas as follows: • Ship design tools; • Structural systems; • Hydromechanics and hull design; • Propulsors; • Automation, control, and system integration; and • Platform power and energy. Another category of activities that ONR includes within the NNR-NE definition is the University Laboratory Initiative, which concentrates on developing the future workforce and sustaining the education infrastruc- ture for naval engineering. In the current grouping, ONR has combined hydromechanics and hull design into a single area; renamed the ship propulsion area as propulsors; added the power and energy area; and grouped automation, control, and system integration into a single area. The committee’s analysis used the categories listed above. The overall scope of the NNR-NE research agenda is shaped to a large extent by the size of the budget devoted to NNR-NE research projects. In FY 2009, the Navy devoted $44.1 million to basic and applied research within the NNR-NE domain (Table 3-1), 3.4 percent of the Navy’s total $1.3 billion budget for basic and applied research (DON 2010, v, vii). The memorandum that established NNR-NE did not establish a pre- ferred level of funding or share of ONR budget for activities to be carried out under the initiative. The specifics of the research agenda are reflected in the projects that have been grouped under the NNR-NE technical areas. In presentations

National Naval Responsibility for Naval Engineering Mission 69 TABLE 3-1 ONR Outlays for NNR-NE Basic and Applied Research, by Technical Area, FY 2006–2009 Average Outlays ($ millions) Annual Outlay Total, per Project 2006 2007 2008 2009 4 years ($ thousands) Automation, control, and 2.2 2.8 2.0 3.2 10.2 232 system integration Basic 1.6 1.8 1.1 1.8 6.3 233 Applied 0.6 1.0 0.8 1.4 3.9 231 Ship design tools 2.4 3.4 3.0 3.0 11.9 165 Basic 2.4 3.4 3.0 3.0 11.9 165 Applied 0.0 0.0 0.0 0.0 0.0 Hydromechanics and 7.2 7.1 7.7 8.7 30.7 101 hull design Basic 4.8 5.5 5.5 5.4 21.2 94 Applied 2.4 1.6 2.2 3.3 9.5 121 Platform power and 20.2 13.7 20.6 18.7 73.3 852 energy Basic 1.4 1.3 1.4 1.9 6.0 136 Applied 18.8 12.4 19.2 16.8 67.3 1,601 Propulsors 2.0 2.1 2.0 2.4 8.5 105 Basic 0.8 0.8 0.9 1.0 3.5 82 Applied 1.2 1.4 1.0 1.4 5.0 131 Structural systems 6.5 6.9 4.7 8.1 26.2 133 Basic 4.1 3.7 3.7 3.5 15.0 106 Applied 2.4 3.2 1.0 4.6 11.2 203 Total 40.6 36.1 40.0 44.1 160.8 205 Basic 15.1 16.6 15.7 16.6 64.0 115 Applied 25.5 19.6 24.3 27.5 96.8 421 SOURCE: Tabulations of ONR 331 basic and applied research projects provided to the committee by ONR. to the committee, ONR delineated its research agenda within these categories for FY 2009 by using a combination of specific examples of funded projects and summary tables showing the number of projects and the level of funding in each of the technical areas. Data on funding trends for projects in each area are provided in Table 3-1. How much money ONR devotes to each of the NNR-NE S&T cate- gories each year is a crucial factor in setting the research agenda. The 2001 memorandum establishing the initiative called on ONR Code 33 to

70 Naval Engineering in the 21st Century “formulate and maintain investments in [all] seven key S&T areas in naval engineering.” The memorandum was silent on how any funds should be apportioned among the areas, however. ONR’s 2009 project list within NNR-NE categories shows an invest- ment profile with a large number of projects in hydromechanics and hull design ($8.7 million in FY 2009, or 19.7 percent of NNR-NE basic and applied research) and structures ($8.1 million, or 18.4 percent) and few in propulsors ($2.4 million, or 5.4 percent); ship design tools ($3.0 million, or 6.8 percent); and automation, control, and system integration ($3.2 million, or 7.3 percent). Much of the $73 million in 2006–2009 platform power and energy funding was the result of a short-term initiative. The Navy’s 2011 research and development (R&D) budget estimate reports a decline in all Navy applied research [Budget Area (BA) 2] spending for power and energy in 2010. Applied research funding for the budget category “surface ship and submarine hull mechanical and electrical (HM&E)” declined from $79 million in FY 2009 to $46 million in FY 2010 (DON 2010, 135). The budget estimate document states that “the funding decrease from FY 2009 to FY 2010 is due to the completion of the energy and power technology initiative that accelerated research in the following Energy and Power efforts: Distribution/Control and Alternative Energy efforts, Energy Storage and Power Generation efforts and the Medium Voltage Direct Current (MVDC) architecture efforts in support of the Next Generation Integrated Power System (NGIPS) Roadmap efforts,” as well as the tran- sition of some projects from applied research to the advanced technology development (BA 3) stage (DON 2010, 136). The Energy and Power Tech- nology Initiative was a 5-year program begun in 2002 throughout the Department of Defense (DOD) to coordinate R&D on energy-efficiency technology improvements (Taylor et al. 2010). ONR sees the development of a balanced portfolio as important: “Assessing the state of the health of Naval Engineering disciplines unique to the Navy is critical to ensure a balanced portfolio” (J. Pazik, briefing, Sept. 2009). That said, the annual share of NNR-NE designated projects and funding that go toward each of the technical areas depends on a vari- ety of factors. The question becomes how ONR decides on the amount of money to allocate to each of those categories. Determinants include

National Naval Responsibility for Naval Engineering Mission 71 the success of ONR program officers in negotiating for projects in the NNR-NE technical areas for which they are responsible. It is not clear that program officers and ONR managers have used NNR-NE designa- tion consistently as a determining factor in allocating funds to projects or in measuring the relative strength of the proposals submitted by offer- ors in response to ONR’s broad agency announcements (BAAs). The difficulty of planning and evaluating a basic research program should not be minimized. Outcomes often develop over years, and many important breakthroughs are unplanned. In developing its research port- folio, ONR appears to attempt to maximize outcomes by reliance on highly qualified managers with authority for program decisions, the tracking of short-term output indicators, feedback on the results of earlier efforts, advice from the technical community, and direction from Congress and the Navy. However, ONR does not appear to apply these informal pro- cesses explicitly to the NNR-NE as a coordinated program with specified objectives. (For example, program officers apparently do not consider whether an activity falls within the definition of the NNR-NE in making program decisions.) Furthermore, these informal processes do not match the requirements for monitoring and evaluation contained in the 2001 memorandum establishing the NNR-NE, which include monitoring of ONR’s traditional output metrics for the NNR-NE as a unified initiative, strategic planning of the NNR-NE, monitoring of the health of the S&T enterprise supporting naval engineering, and annual reporting and peri- odic external review of the NNR-NE. As a coordinating office that lacks direct authority over the funding and award decisions outside of Code 33, however, whether a project has NNR-NE designation generally does not determine in advance what share of the projects or funding will go toward each category. Moreover, as discussed in a later section, NNR-NE program officers strive to iden- tify projects within their portfolios that most merit funding, even though individual NNR-NE program officers may include S&T areas that fall outside the NNR-NE purview. However, the committee could not deter- mine whether anyone assumed responsibility for integrating research across NNR-NE functional areas or across naval weapons platforms. Achieving balance in a research portfolio is a desirable goal and has been achieved in a number of research settings by using techniques such

72 Naval Engineering in the 21st Century as the balanced scorecard method, which balances four perspectives to integrate quantitative and qualitative performance measures (Kaplan and Norton 1992). Studies evaluating the validity and strength of bal- anced scorecard methods have shown strong links between client or sponsor satisfaction and organizational performance, as well as between client satisfaction and economic variables such as client or sponsor retention, revenue, and revenue growth (Ittner and Larcker 1998a; Frigo and Krumwiede 2000). Conclusion: The committee could not identify a process by which NNR-NE mission area needs and research strategies were prioritized. In addition, the committee could not identify any systematic process by which ONR research funds were allocated by NNR-NE mission area needs or prioritized research strategies. Instead, it appears that NNR-NE program officers fund research projects and principal investigators as opportunities arise, without an enterprisewide eval- uation process that prioritizes and evaluates research project merit in a consistent manner across the NNR. Conclusion: The committee did not find evidence that NNR-NE is measuring or achieving balance in its research portfolio, despite its stated balance goal. The committee found no metrics to measure or establish balance in a research portfolio, leading to questions about how such a portfolio could be balanced or could demonstrate balance. Recommendation: ONR should establish an enterprisewide strate- gic planning and assessment process to develop a strategic plan for NNR-NE, link the plan to guiding goals and objectives, communicate those goals and objectives clearly throughout the naval research commu- nity, and evaluate and incentivize NNR-NE performance against the strategic plan and objectives. The NNR-NE strategic planning and assessment process should encompass all facets of the NNR-NE mission. The strategic planning and assessment process should include a process for NNR-NE research fund allocation that is aligned with mis- sion area needs and priorities so that resource allocation decisions are guided by a transparent, enterprisewide evaluation process that pri- oritizes and evaluates research project merit in a consistent manner across the NNR.

National Naval Responsibility for Naval Engineering Mission 73 Recommendation: ONR should identify, utilize, and periodically reassess metrics to measure NNR-NE portfolio balance, in line with ONR’s stated goals and articulated mission needs. Once established, these metrics should be incorporated into an enterprisewide assessment and continuous process improvement program, as described in subse- quent sections of this chapter. IDENTIFYING PERFORMERS ONR generally makes its research awards in response to BAAs.1 A con- solidated annual BAA pulls together instructions to potential research performers for submitting award requests for a large share of ONR’s proj- ects, including those related to the NNR-NE. Most such awards are solicited through that consolidated BAA. For example, ONR released ONR BAA 10-001, Long Range BAA for Navy and Marine Corps Science and Technology, on September 18, 2009, with the expectation that it would remain open for 1 year. Proposals can be submitted at any time during the year (ONR 2009a). Naval engineering research performers in the private sector include universities and industrial firms.2 Research within the University Labo- ratory Initiative is conducted by universities. For allocating projects among university and industry performers, ONR relies heavily on its program officers’ assessments of research merit, relevance to Navy mis- sions, the value of sustaining long-term relationships with productive principal investigators, and the need to develop new promising princi- pal investigators. ONR reported to the committee that program officers are mindful of the need to balance the long-term value of continued investment in ongoing research with research breakthrough opportuni- ties and shorter-term needs for research transitions in a constrained funding environment. 1 ONR occasionally uses requests for information and requests for proposals to solicit research offer- ings. For example, Solicitation No. N00014-10-0001 requests proposals for a contractor to operate the Navy Metalworking Center and conduct research on technical projects related to metalworking. ONR also makes use of other instruments for support contracts. 2 Basic research (Budget Activity 6.1) and applied research (Budget Activity 6.2) awards are usually provided as grants to universities and as contracts to industry. Advanced technology development (in Budget Activity 6.3) is usually performed under contracts. See ONR 2009a, 3.

74 Naval Engineering in the 21st Century Federally funded R&D centers, such as Rand, the MITRE Corporation, and the Department of Energy’s National Laboratories, are not eligible to receive awards under ONR’s consolidated BAA, although they may team with eligible partners. DOD laboratories, including the Navy’s own labo- ratories and warfare centers, are also precluded from bidding directly. ONR publishes on its website a list of technology areas in which it is interested, together with the names of and contact information for pro- gram officers who handle those areas. The BAA urges offerors to contact the program officer whose technology portfolio best matches their fields of interest before they develop their proposals. Program officers are responsible for evaluating the technical proposals that are submitted in their technical areas. As stipulated by the BAA, award decisions are “based on a competitive selection of proposals resulting from a scientific and cost review.” Box 3-1 lists the evaluation criteria to be con- sidered in evaluating the BAA for 2010. The BAA indicates that Factors 1 through 3—the technical factors— are of equal weight and that those technical factors are significantly more important than Factor 5, cost realism. BOX 3-1 Evaluation Criteria for ONR’s 2010 BAA 1. Overall scientific and technical merits of the proposal; 2. Potential Naval relevance and contributions of the effort to the agency’s specific mission; 3. The offerors’ capabilities, related experience, facilities, tech- niques or unique combinations of these which are integral factors for achieving the proposal objectives; 4. The qualifications, capabilities and experience of the proposed principal investigator, team leader and key personnel who are critical in achieving the proposal objects; and 5. The realism of the proposed costs and availability of funds. SOURCE: ONR 2009a, 21.

National Naval Responsibility for Naval Engineering Mission 75 One of the key inputs to the NNR-NE R&D process is knowledge of Navy needs and mission areas. ONR program officers often work as intermediaries between Navy laboratories, the academic and industrial research community, and other stakeholders. Such an integrative role is critical to the success of the NNR-NE initiative. The committee noted that links to the operational Navy community from designated NNR-NE projects were not as well articulated, nor could the committee identify a systematic mechanism that communicated Navy operational needs to program officers managing these projects. The committee concludes that no formal process exists within ONR for regular review of Navy mission needs relevant to its S&T planning for new projects with NNR-NE desig- nation or for determination of allocation plans for funding to performer organizations. ONR’s performer evaluation process, including that for its NNR-NE portfolio, differs from that of some other government research sponsors in not including an evaluation of its basic research proposals by external peer reviewers. External review of proposals can be a valuable tool for government agencies that fund basic research, whose impact on future capabilities systems may not become apparent for decades. Organiza- tions that use external scientific peer review for most or all of the basic research they fund include the National Science Foundation (NSF), the National Institutes of Health (NIH), and the Office of Research and Eval- uation of the National Institute of Justice (DHS 2009). Within DOD, the Air Force Office of Scientific Research employs a peer review process using review panels that typically include two reviewers from other DOD offices and one from outside of DOD (Sharp 2007). In contrast, the Defense Advanced Research Projects Agency (DARPA) generally does not bring external experts into its evaluation process. Instead, it uses its cadre of program officers, who typically rotate into the organi- zation from positions outside of government and serve in DARPA for only a few years, thus ensuring a fresh flow of expertise and perspective. The committee understands that in recent years, ONR’s program officers have stayed for substantially longer periods. Supporters of ONR’s proposal evaluation process argue that the com- munity of scientists with relevant expertise—particularly in the naval engineering fields—is small, making it hard to find outside technical

76 Naval Engineering in the 21st Century experts to serve as external peer reviewers. They might also point out that this committee’s assessment constitutes an external peer review of NNR- NE’s overall program and thus serves as an implicit review of the award choices made by NNR-NE program officers. However, the committee found that there are sound reasons to con- sider bringing external peer scientists in to help with the evaluation of proposals. Bringing outside experts into the proposal evaluation process can help an organization sustain competition and avoid parochialism. It can also help to build a cohort of outsiders familiar with and interested in the particular areas of research. In NNR-NE’s case, bringing experts from other government organizations into the proposal review process might help to forge and strengthen partnerships that the ONR organiza- tion aspires to develop. Observers have found that external assessments like the one con- ducted by this committee can be useful in helping government research organizations to improve the merit and relevance of the research they fund and to develop plans for the future (Lyons and Chait 2009). Because such reviews are aimed at the organizational level, however, they lack the immediate impact on funded projects of external scientific reviews of proposals. The Navy Warfare Centers use peer-review evaluation, with external reviewers encouraged, for proposal selection in certain pro- grams. Box 3-2 describes examples of the use of peer review of project proposals by research organizations within DOD and at other federal agencies. Box 3-3 summarizes conclusions of a 2002 National Research Council (NRC) study of approaches to organizing cooperative research on naval engineering, conducted at the request of ONR, concerning the value of peer review in the research programs it examined as models. ONR leadership has formed a similar opinion with regard to the mer- its of external review in the monitoring of projects that have already been selected for funding and is establishing a peer-review process. The process described to the committee involves assembling a panel of three to five external technical experts who review the project’s progress in the second (and potentially third) year of execution. The objective of these panels is to assess the efficacy of the ongoing project and to make recommenda- tions to the program officer for continuation or termination. Unfortu- nately, this process does not appear to achieve all of the benefits that can be accrued through early participation of peers in project selection.

National Naval Responsibility for Naval Engineering Mission 77 BOX 3-2 Examples of Peer Review of Project Proposals at Federal Research Institutions The practices of research funding agencies in DOD and elsewhere in the federal government suggest numerous alternative arrange- ments for conducting peer review of research project proposals and reviews at other stages of research program management. A survey of peer-review practices at Army establishments involved in R&D and at other federal agencies, conducted at the request of the Army Science and Technology Executive by the Center for Technology and National Security Policy, shows diversity in present practices and recommends best practices. The majority of the establishments reviewed were laboratories, but grant- making agencies (analogous to ONR) were included. The report describes review procedures applying to all stages of the produc- tion of scientific research, from project selection and program formation through work in progress to finished products, with the focus on individual projects or on the body of work of an organization. The survey identified two Army research grant-making agencies that conduct external reviews of project proposals. The Army Research Office sends proposals for new, single-investigator research to external technical expert reviewers to evaluate techni- cal merit. Separately, the proposals are also sent to Army and DOD scientists and engineers for evaluation of military relevance (Lyons and Chait 2009, 17). The Army Medical Research and Materiel Command sponsors and conducts research. Research proposals, including those from the command’s researchers, are reviewed by an independent organization, the American Institute of Biological Sciences (Lyons and Chait 2009, 10). (continued on next page)

78 Naval Engineering in the 21st Century BOX 3-2 (continued) Examples of Peer Review of Project Proposals at Federal Research Institutions The Air Force Office of Scientific Research, which manages basic research for the Air Force, appears to have less standardized pro- cedures but submits proposals in response to its BAAs to peer review in some circumstances. The office’s Proposer’s Guide states that “peer review and/or the scientific review process is used to conduct proposal evaluation” (AFOSR 2007, 5). Other grant-making agencies reviewed in the Army study do not have external review of proposals. They include the Amy Research Institute for the Behavioral and Social Sciences, which submits proposals to an internal review process (Lyons and Chait 2009, 10). The Army study describes peer-review procedures at NIH and NSF for comparison with those of Army and other DOD labora- tories (Lyons and Chait 2009, 17). Both NIH and NSF routinely submit grant applications to external experts for evaluation. NIH research proposal peer review is governed by federal law and regulations (42 CFR Part 52h, Scientific Peer Review of Research Grant Applications and Research and Development Contract Proposals). At NSF, all proposals are sent to three to 10 expert reviewers outside NSF. Proposers may suggest reviewers for their proposals. The external reviewer evaluations are advisory. The NSF program officer recommends whether to fund each proposal, and final decisions are made by senior management (NSF 2011b, III-1).

National Naval Responsibility for Naval Engineering Mission 79 BOX 3-3 External Review Conclusions of the Committee on Options for Naval Engineering Cooperative Research NRC’s Committee on Options for Naval Engineering Cooperative Research evaluated alternative organizational arrangements for a cooperative research program in naval engineering. The study was done at the request of ONR. The cooperative research model is appropriate for a research pro- gram that must serve a diverse community of users and sponsors. These interested parties are given defined roles in guiding the pro- gram, including responsibilities in program planning and in proj- ect selection. The model might be applicable to a part of ONR’s research related to naval engineering, if not necessarily all such ONR research. The NRC committee reviewed governance arrangements in suc- cessful cooperative research programs, including NSF’s Engineer- ing Research Centers Program, the National Ocean Partnership Program, and cooperative research programs of the oil and gas industry (TRB 2002, 31). The committee also was aware of the cooperative research programs of the Transportation Research Board. In addition, the committee received proposals for coop- erative research organizational structures from professional societies, university groups, the Naval Sea Systems Command (NAVSEA), and the National Shipbuilding Research Program (TRB 2002, 31). The committee reached general conclusions on essential orga- nizational features on the basis of the experience of the estab- lished research programs and the advice of the interested groups. With regard to external participation in the selection of research (continued on next page)

80 Naval Engineering in the 21st Century BOX 3-3 (continued) External Review Conclusions of the Committee on Options for Naval Engineering Cooperative Research projects, the committee concluded that “in a true cooperative program, all the major stakeholders have both a shared interest and shared ownership in the research agenda. For any of the orga- nizational models to be successful, it must provide a structure and mechanism to allow appropriately balanced representation and input to the research agenda from stakeholders” (TRB 2002, 7). With regard to evaluation in general, the committee concluded that “to be successful, merit review of the research . . . should take place at three stages in the process: when the proposal is approved, annually during the course of the research work, and when the project is completed. A merit review panel should be carefully balanced to ensure that innovative high-risk ideas are not lost and that the results address the Navy’s needs. . . . The small size of . . . [the naval engineering research] community will neces- sitate resourcefulness in assembling a qualified and conflict-free group of individuals with balanced biases as reviewers for research proposals, progress, and outcomes” (TRB 2002, 7). At the January 2010 committee workshop, a Warfare Centers participant identified the function of the mission capability manager as a possible model for ONR to emulate. Naval Undersea Warfare Center (NUWC) Newport Division (ND) uses mission capability managers who understand specific end-to-endNavy missions (e.g., antisubmarine warfare–antisurface warfare) and who are responsible not for specific projects but for ensuring that the center delivers mission capabilities that are required (P. Corriveau, briefing to the committee, Jan. 13, 2010). The mission capability man- ager facilitates communication and optimizes knowledge-sharing within the center to enhance the relevance of research efforts through cross- departmental collaboration and leverage.

National Naval Responsibility for Naval Engineering Mission 81 Conclusion: External peer review (i.e., review by technical experts from outside ONR) throughout the research project selection process offers the opportunity to strengthen research project selection and to obtain the advice and counsel of technical experts, Naval Sea Systems Command (NAVSEA) technical authorities, and industry practition- ers who are the ultimate recipients of the developed technology, while maintaining the ONR program officer’s independence in making decisions for his or her program. Recommendation: ONR should establish a process for NNR-NE (and potentially other programs) in which the program officer assembles a small group of Navy laboratory technical experts [e.g., from Naval Sur- face Warfare Center (NSWC) Carderock (CD)] and NAVSEA technical authorities (who also serve as industry surrogates) to review, assess, and rank relevant proposals received in response to ONR BAAs. The pro- gram officer then would be responsible for considering these recom- mendations and selecting projects. The midproject external review that ONR already conducts would be carried out by this panel with the addi- tion of external reviewers according to the requirements of the present midproject review procedure. The proposal review panel would not remove ultimate responsibility from the program officer. Instead, the panel would create a dialogue and open lines of communication among ONR and the key Navy constituencies. Recommendation: As input to the identification of performers, to enhance systematic dissemination of Navy mission and needs, and to improve communications between ONR and operational Navy units, NNR-NE should use mission capability managers who are responsible for understanding specific end-to-end Navy missions. Recommendation: To improve communication of operational requirements and the transitioning of technology to naval ships, ONR should implement the concept of a technology interpreter in the NNR-NE. The task of the technology interpreter would be to assist in the technology transition process. The recommended peer- review panels would implement the concept of a technology inter- preter in the program officer and technical authority communities. Frequent communication between these communities would inform

82 Naval Engineering in the 21st Century the program officer of technologies that the technical authorities need and want and inform the technical authorities of new tech- nologies as they emerge and mature. In combination with the review panels, personnel dedicated to improving communications and exe- cution could significantly improve NNR-NE integration with Navy missions, needs, and operational requirements. Case studies where such informal dialogue between program officer, technical expert, technical authority, and industry have been most con- structive and successful are documented in the committee’s commissioned papers (Hackett 2010; Doerry 2010). Doerry (2010) discusses the concept of relationships managers, “individuals that assist the technology transition process,” which has been identified by the Government Accountability Office as an industry best practice (GAO 2006). “Technology interpreter” may be a more descriptive term for this role. MEASURING OUTCOMES AND EVALUATING RESULTS In recent decades, the executive branch and Congress have emphasized the importance of setting goals and measuring progress toward them as a way for federal departments and agencies to improve their performance, develop relevant plans for the future, and build budgets. The Government Performance and Results Act of 1993, P.L. 103-62, requires federal agen- cies to prepare strategic plans, annual performance plans, and annual per- formance reports. The plans are meant to identify concrete, measurable goals and objectives and schedules for meeting them. The performance reports are meant to explain how well actual performance measures up to the plan and what the agency plans to do to narrow the gap between plans and performance. Although such documents are not generally required at the subagency level, high-performing research units recognize the importance of commit- ments to assessment, measurement, and continuous process improvement (Roberts 1990; Roberts and Rousseau 1989). High-performing lower-level organizations thus often derive their plans and objectives from the strate- gic plans formulated at the department or agency level. Components of an enterprisewide, systematic assessment process include metrics for measur- ing and incentivizing performance; a continuous process improvement

National Naval Responsibility for Naval Engineering Mission 83 activity that considers the outcomes of the metric evaluation processes; a benchmarking operation to determine organizational progress over time; and a systematic communication method to promulgate lessons learned, best practices, and organizational heuristics (Bond 1999; Brown 1996). As an illustration, Box 3-4 describes current performance assessment and strategic planning initiatives at NSF. Conclusion: ONR collects information on a variety of metrics that could be helpful in evaluating progress toward objectives, incentivizing performance, and improving the organization over time. However, it was not clear to the committee that these metrics are linked to a set of measurable objectives for the S&T enterprise in NNR-NE. The commit- tee also could not determine whether any NNR-NE guiding goals or objectives were tied to strategic plans at the department or agency level. The committee was unable to identify an NNR-NE strategic plan that establishes priorities and identifies measurable objectives, an annual performance plan, or annual performance reports. R&D portfolios are a composite of short- and long-term programs, collectively designed to foster discovery and innovation in support of an organization’s mission. Evaluations of R&D portfolios often measure the completeness, robustness, strength, and degree of innovation present in the portfolio (Reugg 2007). To measure performance, organizations often apply quantitative performance measures such as return on invest- ment or earned or economic value added, along with nonfinancial mea- sures such as stakeholder or sponsor satisfaction and measures of the quality of the research and innovation supported (Ittner and Larcker 1998b; Kaplan and Norton 1996a; Kaplan and Norton 1996b; Kaplan and Norton 1996c; U.S. Department of Energy 1995; U.S. Department of Energy 2001). Research portfolio outcome and process metrics are often integrated to develop a more holistic view of the portfolio’s perfor- mance and promise (Yeniyurt 2003; Eccles 1991; Eccles and Pyburn 1992; Kaplan and Norton 1992; Kaplan and Norton 2004; Ittner and Larcker 1998b; Tan and Platts 2003; Tan et al. 2004). A commonly used approach is the balanced scorecard method, which balances four perspectives to integrate quantitative and qualitative performance measures (Kaplan and Norton 1992).

84 Naval Engineering in the 21st Century BOX 3-4 NSF’s Performance Assessment Framework A number of federal agencies that sponsor or conduct research have implemented enterprisewide information and reporting systems that provide performance metrics and information for assessment and continuous process. NSF, in its FY 2012 Budget Request to Congress, describes its current efforts to strengthen per- formance assessment (NSF 2011a, Performance Information-3): NSF is reviewing its performance assessment framework, in keeping with the Administration’s commitment to establishing an evaluation infrastructure that complements and integrates efforts to strengthen performance measurement and manage- ment. This overall effort has been a specific focus of the recent update of the NSF Strategic Plan, which places special emphasis on testing and refining new approaches to assessment and eval- uation. The FY 2011 GPRA [Government Performance and Results Act, P.L. 103-62] Performance Plan . . . is the first such plan based upon the new Strategic Plan. A number of related efforts are also underway. These include: • Continued progress toward NSF’s STEM [science, technol- ogy, engineering, and mathematics] Workforce Priority Goal. • Sustained NSF support for the multi-agency data infrastruc- ture for monitoring and analyzing investments in science and engineering research and education [STAR METRICS (Science and Technology for America’s Reinvestment: Mea- suring the Effect of Research on Innovation, Competitive- ness and Science), a multiagency initiative to develop a data infrastructure to support evaluation of federal investment in research and development]. . . .

National Naval Responsibility for Naval Engineering Mission 85 • The establishment of an NSF-wide capability for assessment and evaluation planning for an expanded NSF-wide assessment and evaluation capacity. • Systematic efforts to improve evaluation and monitoring activities in STEM education and workforce programs. NSF’s 2011–2016 strategic plan identifies, as one of its three strategic goals, performance as a “model organization.” This goal “sets high standards for attaining excellence in operational activities, promotes a culture of integrity and accountability, and encourages new approaches to assessment and evaluation of NSF’s investment portfolio” (NSF 2011a, Overview-2). NSF states that the three goals, “Transform the Frontiers, Innovate for Society, and Perform as a Model Organization—lay out a path towards both longer-term outcomes and the more imme- diate impacts NSF’s investments can generate” (NSF 2011a, Per- formance Information-3). The importance of intellectual capital in an R&D portfolio suggests the need to incorporate measures of a portfolio’s intangible assets, in addition to balancing outcome and process measures (Yeniyurt 2003; Kaplan and Norton 2004). Skandia Insurance Company, for instance, uses assessments of human, intellectual, structural, and brand assets; intellectual property; and customer capital to evaluate its intangible assets (Edvinsson 1997; Edvinsson and Malone 1997; Joia 2000). Intellectual capital is often thought to be a function of human capital, structural capital, customer capital, and innovation capital, with the relationships among these factors varying by institution, available resources, and set- ting (Yeniyurt 2003; Chen et al. 2004). Other metrics have been proposed for intellectual capital criteria, including creativity and productivity, which vary according to individual attributes, task characteristics, and organizational contexts (Chang and

86 Naval Engineering in the 21st Century Birkett 2004). Comprehensive assessments of R&D portfolios therefore balance a number of criteria. First, they consider whether the portfolio’s goals are aligned with the mission of the parent organization or spon- sor. Second, they use quantitative and qualitative performance mea- sures along with metrics to assess the intellectual capital, creativity, and productivity of the intellectual enterprise. Finally, they assess the bal- ance, completeness, and expected longevity or sustainability of the port- folio, along with intangible factors such as management and investigator enthusiasm and commitment, and the importance of the expected impact of the portfolio on the organization and its wider setting (Bukowitz and Petrash 1997; Kaplan and Norton 2001; Kaplan and Norton 2004; Melnyk et al. 2004). Measuring outcomes in complex interdisciplinary research on an annual basis can be challenging because of its inherent unpredictability, but measures do exist: measures of quality, in terms of research advance- ment; relevance, in terms of application development; and leadership, in terms of the ability to take advantage of opportunities when they arise, as evaluated by experts and users of research (NRC 1999, 9). In addition, human resource development has been identified as a key outcome of an effective research program. A remaining challenge is to determine what additional measures, if any, are needed to evaluate interdisciplinary research and teaching beyond those shown to be effective for disciplinary activities. Successful outcomes of an interdisciplinary research program differ in several ways from those of a disciplinary program. First, a suc- cessful interdisciplinary research program will have an impact on mul- tiple fields or disciplines and produce results that feed back into and enhance disciplinary research. It will also create researchers and students with an expanded research vocabulary and abilities in more than one dis- cipline and with an enhanced understanding of the interconnectedness inherent in complex problems (NRC 2004, 150). The following section presents the committee’s assessment of NNR-NE performance with respect to aligning NNR-NE activities with Navy S&T strategic plans, ensuring consistency with NNR-NE objectives, and measuring and improving NNR-NE outcomes. The committee then presents its assessments of NNR- NE performance with respect to integrating NNR-NE activities and per- forming interdisciplinary research.

National Naval Responsibility for Naval Engineering Mission 87 Aligning NNR-NE Activities with Naval S&T Strategic Plans During the past decade, the Navy has developed strategic plans for its S&T efforts. The most recent of these is the 2009 Naval S&T Strategic Plan: Defining the Strategic Direction for Tomorrow (ONR 2009b), developed jointly by ONR and the Naval Research Laboratory and signed by the senior uniformed and civilian leaders of the Navy and Marine Corps form- ing the S&T Corporate Board. The strategic plan outlines the S&T vision and key objectives in 13 naval focus areas, which are listed in Box 3-5. Within ONR, the two-digit offices identify the focus areas that they sup- port. On its website, ONR’s Code 33 (Sea Warfare and Weapons, which houses the NNR-NE) lists its focus areas as fleet and force sustainment, maritime domain awareness, power projection, and power and energy. BOX 3-5 Naval S&T Focus Areas in 2009 Naval S&T Strategic Plan 1. Power and energy* 2. Operational environments 3. Maritime domain awareness 4. Asymmetric and irregular warfare 5. Information superiority and communication 6. Power projection 7. Assure access and hold at risk 8. Distributed operations 9. Naval warfighter performance 10. Survivability and self-defense* 11. Platform mobility* 12. Fleet and force sustainment 13. Total ownership cost* *Naval S&T focus areas most closely related to the scope of NNR-NE (ONR 2009b, 8–25).

88 Naval Engineering in the 21st Century Surprisingly, the website does not list survivability and self-defense, plat- form mobility, or total ownership cost even though much of these three focus areas relate directly to the NNR-NE technical areas and other Code 33 core programs. This apparent disconnect is emblematic of the issues related to linking the NNR-NE activities with the S&T Strategic Plan. Such connectivity is key to obtaining support within the ONR organiza- tion that would be reflected in appropriate investment levels for an NNR focus area. Table 3-2 highlights the objectives for each of those focus areas as out- lined in the S&T Strategic Plan. Many of the objectives listed in Table 3-2 are related to NNR-NE’s mission of developing educated and experienced people, expanding the knowledge base, and cultivating a climate support- ive of innovation in the S&T categories that fall within its purview. The objectives on this list, and more generally the objectives high- lighted among all 13 focus areas of the 2009 ONR strategic plan, might be a useful starting point for NNR-NE in identifying the broad categories of work in which the Navy has higher-priority interest. In that sense, they may provide useful guidance to the NNR-NE program officer who is faced with choosing among the projects offered by universities and industry. As discussed below, however, it is not clear how one would use them to measure progress toward goals. Ensuring Consistency with NNR-NE Objectives NNR-NE has developed a list of objectives for the work done within each of its six S&T categories as well as for its University Research Initiative. However, it was not clear to the committee how those NNR-NE objec- tives are aligned with those of ONR’s Naval S&T Strategic Plan (ONR 2009b). The committee was also unable to find evidence that NNR-NE set measurable objectives related to the S&T categories under its purview. In one example, objectives for the structural systems category include the following: • Develop technologies for life-cycle performance analysis and moni- toring of ship structural systems; • Develop an understanding of behavior of novel ship structures, such as composite or aluminum subsystems, during and after fire to enable modeling and prediction;

National Naval Responsibility for Naval Engineering Mission 89 TABLE 3-2 Objectives of the S&T Focus Areas Supported by ONR Code 33 Focus Area Objectives Fleet and force sustainment Sea-based sustainment • Flexible and responsive warehousing • At-sea assembly and reconstitution of forces Flexible and responsive delivery systems • Point-of-delivery systems • Heavy-lift vehicle launch and recovery • Ship-to-shore logistics Integrated logistics • Common operating picture—logistics • Autonomous resupply systems • Source-to-objective asset visibility Maritime domain awareness Pervasive and persistent sensor networks • All domain coverage • Mission-focused autonomy with near real-time self-tasking • Secure, survivable, self-healing, adaptable, and affordable Identification of hard targets through diverse sensing • Identification of entities and events via electromagnetic signatures • Development of SIGINT capability to understand human activity • Characterization of acoustic signatures • Use of tagging, tracking, and location to declutter battlespace picture Sensor and data integration and threat assessment • Automated image, video, and SIGINT processing • Rapid, accurate, multisource data integration including national and tactile sensors, intelligence, and open-source data • Automated decision tools • Automated ISR sensor retaskings to refine battlespace knowledge Automated assessment of events and entities to determine intent Power and energy Energy security • Alternative and renewable energy sources • Future logistics tools • Resilient power networks and systems Efficient power and energy systems • Materials, devices, and architectures to increase efficiency and power density for platforms and reduce weight for personal power • Efficient power conversion, switching, distribution, control, and thermal management • Engines, motors, generators, and actuators • Electromechanical, thermal, and kinetic energy storage High energy and pulse power • Energy storage power system architectures • Energy pulsed power switching and control systems (continued on next page)

90 Naval Engineering in the 21st Century TABLE 3-2 Objectives of the S&T Focus Areas Supported by ONR Code 33 (continued) Focus Area Objectives Power projection Future Navy fires • Increased fires volume and accuracy • GPS-denial compensation • Indirect fires to 250 miles from safe offshore locations • Long-range surface warfare capability Control collateral damage • Scalable effects weapons • Selectable and directional lethality Time-critical strike • Hardened target–moving target reach and destroy • Worldwide to meet warfighter requirements Small-unit combat power • Increased small-unit weapon lethality • Neutralize larger hostile forces • Application of Joint Fires • Advanced weapon sights, including multispectral Combat-insensitive munitions • Reduce system sensitivity to sympathetic detonation • Maintain payload range and lethality Survivability and self-defense Platform stealth • Reduce aircraft, above-water, and subsurface signatures • Multispectral LO technologies Force protection • Detect and determine threat intent to interrupt kill chain • Detect and deter small boat and unmanned threats • Antiswimmer technology • Nonlethal response Countermeasures and counterweapons • Threat weapon tracking and weapon–countermeasure–sensor selections • Automated decision making and battle management aids • Low-false-alarm-rate, 360-degree detection • Hard kill and soft kill against threat kinetic weapons • Extend standoff to beyond threat damage range • Directed energy weapons for speed of light engagement • Counter-LO capabilities Survivable platforms • Advanced platform construction materials • Damage-tolerant platform architectures • Automated damage-control focusing • Advanced materials for self-healing platforms

National Naval Responsibility for Naval Engineering Mission 91 TABLE 3-2 Objectives of the S&T Focus Areas Supported by ONR Code 33 (continued) Focus Area Objectives Platform mobility Efficient, high-endurance, high-speed platforms • New and novel advanced platform design supporting new directions in naval warfare (size, agility, modularity, etc.) • Higher-performance platforms at reduced fuel consumption • Efficient, all-terrain, lighter, more agile ground vehicles includ- ing suspensions and drivetrains • Manned vessel launch and recovery • Operator guidance tools • Lightweight, higher-strength advanced composites and struc- tural metals for optimized platform performance Vertical lift operations in challenging environments • High-performance vertical and short takeoff and landing • High sea states launch and recovery technology to enable manned or unmanned air operations Autonomous and unmanned vehicle mobility • New unmanned platform design technology • Advanced robotic systems for air, ground, and sea combat • Unmanned vessel launch and recovery Total ownership cost Platform affordability • Advanced modeling and simulation for design, test, and eval- uation • Advanced naval materials • Open architecture for hardware and software • Low-cost, reliable sensors and electronics • Innovative manufacturing technologies Maintenance and life-cycle cost • Condition-based maintenance systems • Anticorrosion and antifouling technologies • Wear-resistant lifetime materials • Energy-efficient systems • Software reliability Manning optimization • Human–systems integration • System automation • Autonomous systems NOTE: GPS = Global Positioning System; LO = low observable; ISR = intelligence, surveillance, and reconnaissance; SIGINT = signals intelligence.

92 Naval Engineering in the 21st Century • Provide protection system and armor that can defeat several threats and meet structural and stiffness requirements; and • Facilitate use of alternative hull forms that have a longer life than steel or aluminum hulls and are lighter, more survivable, stealthier, cheaper, and easier to maintain (J. Pazik, briefing, Sept. 2009). Such objectives clearly help guide the structural systems program offi- cers and potential offerors concerning which research areas to pursue, although the intent of some of the objectives is unclear. In the case of the second objective, “develop an understanding of behavior of novel ship structures,” for example, the committee could not ascertain how such an understanding could be reached or improved in a measurable way. The objectives on the ONR 2009 strategic priorities list, and more generally the objectives highlighted among all 13 focus areas of the 2009 ONR strategic plan, should be a starting point for NNR-NE in identify- ing the broad categories of work in which the Navy is interested. The research objectives for NNR-NE that ONR establishes in the rec- ommended enterprisewide strategic planning and assessment process should relate to its three interrelated missions: developing educated and experienced people, expanding the knowledge base, and cultivating a cli- mate supportive of innovation. For example, ONR might establish a goal for the number of undergraduates, graduates, and postdoctoral candi- dates to be offered fellowships each year. For graduate students and post- doctoral candidates, ONR might simply adopt the goal set by NNR-NE’s founding memorandum of October 22. 2001: “Develop about half of the ‘pipeline’ of future naval researchers required to sustain the expertise in naval engineering”—“about 5 graduate fellowships and 5 post-doctoral candidates per year” (ONR 2001, 4). Developing a measurable goal for the expansion of the knowledge base can be difficult, especially for basic research, where it can take decades for the knowledge developed to bear fruit. Some high-performing research organizations gauge their progress by using surrogate measures. For NNR-NE, traditional measures of knowledge base expansion might include the number of articles on work funded by NNR-NE that are published annually in peer-reviewed scientific journals, the number of citations of work funded by NNR-NE in such journals, or the num- ber of scientific or engineering awards received by those whose work was funded by the organization (DHS 2009). However, more recently,

National Naval Responsibility for Naval Engineering Mission 93 research organizations have also adopted measures representative of the integrative, interdisciplinary research required to address current and future grand challenges (Porter et al. 2006; National Academies 2005). For NNR-NE, integrative measures of research underscore the impor- tance of total ship solutions in naval engineering, recognizing the con- straints placed on hydrodynamics; structures; propulsors; power systems; and automation, control, and systems engineering by a restricted platform operating envelope and naval missions. Thus, integrative metrics encour- age program officers and principal investigators to consider research prior- ities and directions holistically and across platforms, rather than pursuing research success in a single functional area (e.g., hydrodynamics) or on single platforms. The committee identified several notable examples of excellent NNR- NE integrative research projects during its workshops and information- gathering activities (e.g., the advanced integrated mast, composites research) and in its commissioned papers (Hackett 2010; Hagan 2010), which provide an initial platform for integrative NNR-NE research. ONR actions to build incentives for multidisciplinary research initia- tives into the management of the NNR-NE are recommended at the end of this chapter. NNR-NE integrative metrics could include the number of interdisci- plinary projects, the number of interdisciplinary publications, impact measures of research conducted within and outside of primary disciplines, citations and funding received outside of primary disciplines, and the numbers of publications and citations within a single discipline and across multiple disciplines. Such metrics encourage program officers and princi- pal investigators to adopt interdisciplinary perspectives in the research projects, and they encourage program officers to look for opportunities for collaboration across naval engineering S&T and across ONR to address critical naval research priorities. For applied research or advanced technology development projects, NNR-NE might develop objectives related to technology transition into Navy R&D projects at the BA 3 level and above. Because the development of a climate of innovation is also one of the organization’s charter missions, it appears that NNR-NE should also develop objectives related to this area. The committee noted that several of the integrative research projects so critical to future advances in NNR-NE were the outgrowth of an

94 Naval Engineering in the 21st Century NNR-NE program officer’s or an industry representative’s individual leadership or foresight, rather than the natural result of an enterprisewide research strategy, planning, and prioritization process or of organiza- tional structures (e.g., technology interpreters) designed to produce cross-cutting, integrative advances across the NNR-NE. Measuring and Improving NNR-NE Outcomes This section suggests a template that ONR might use to integrate and delineate information and data that it already collects into a more coher- ent picture that could be used to measure progress toward desirable out- comes. The template is based on the three goals identified in the NNR-NE chartering memorandum: “(1) provide and sustain robust research exper- tise in the United States working on long-term problems of importance to the Department of the Navy [knowledge]; (2) ensure that an adequate pipeline of new researchers, engineers, and faculty continues [people]; and (3) ensure that ONR can continue to provide superior S&T in naval architecture and marine engineering [transition]” (see the committee’s statement of task in Box 1-1). ONR currently collects information related to these areas, as follows: • Knowledge – Publications (refereed papers) – Patents and licenses – Citations • Transitions – BA 1 to BA 2 transitions – Transition to Innovative Naval Prototype and Future Naval Capability – Transition to program offices • People – STEM program – Advanced degrees completed – Participants joining naval warfare labs Qualitative measures could be established by assessing performance in each of these areas as good, fair, or poor. The resulting template is shown in Table 3-3.

National Naval Responsibility for Naval Engineering Mission 95 TABLE 3-3 Metrics and Qualitative Measures of Effectiveness Measure of Effectiveness Metric Good Fair Poor Many publications, Some publications, Few publications, Knowledge patents, citations patents, citations patents, citations Many transitions Some transitions Few transitions 6.1 Transitions 6.1 to 6.2, INP, FNC 6.1 to 6.2, INP, FNC to 6.2, INP, FNC Many STEM students, Some STEM students, Few STEM students, People advanced degrees advanced degrees advanced degrees supported, NWC/ supported, NWC/ supported, NWC/ Laboratory hires Laboratory hires Laboratory hires NOTE: FNC = Future Naval Capabilities; INP = Innovative Naval Prototype; NWC = Naval Warfare Centers. Parsing such quantified measures of effectiveness at too low a level (e.g., individual Warfare Center, specific S&T area) can be misleading. However, when they are aggregated at a higher level, the results are meaningful as a health assessment summary, despite the fact that the metrics reported by the NNR-NE are primarily “lagging” as opposed to “leading” metrics, which in general require long dwell times before results can be measured and reported. As a means to improve on measuring outcomes and evaluating results for NNR-NE, the committee-commissioned paper on transitioning technology to naval ships provides seven recommendations for improv- ing S&T technology transition in general based on lessons learned from case studies (Doerry 2010). The recommendations are to • Use product lines and associated technology development road maps, • Use more robust metrics, • Improve technology transition agreements, • Fully implement technology interpreters (termed “relationships man- agers” in the paper), • Modify the DOD financial management regulation (DODFMR) to include technology transition activities in BA 3, • Modify the DODFMR to split BA 4 into product line development and advanced component development and prototypes, and

96 Naval Engineering in the 21st Century • Assign the Office of the Chief of Naval Operations N091 as the resource sponsor for product line development in addition to S&T. While many of these recommendations have direct application to higher levels of S&T funding than those considered in the committee’s task statement (e.g., BA 3 and BA 4), there is a clear opportunity to improve NNR-NE BA 1 and BA 2 outcomes. The committee assessed these recom- mendations in the context and scope of the NNR-NE and identified two recommendations that can produce metrics that are better leading indica- tors of NNR-NE program efficacy–the effectiveness of technology inter- preters and of product lines and associated technology development road maps. In the previous section, the value of the technology interpreter con- cept in improving communications between naval communities was pre- sented. Simply implementing (or measuring the frequency of) meetings with participants from multiple departments or divisions can serve as an indicator as to whether the communities are collaborating, which will help the program officer in making better-informed decisions. Application of the recommendation to promote the use of product lines and associated technology development road maps can also produce metrics that are leading indicators of NNR-NE program efficacy. As noted in the commissioned paper: The current model favoring transitioning technology directly from S&T to products directly supporting acquisition programs has lead to the R&D “Valley of Death.” The principal cause of the “Valley of Death” is that a ship acquisition program has a very short window following Milestone A to fund technology development that will mature in time to support integration into the overall ship design process. Technology that is not perceived to be ready during this short window will typically not be incorporated. Unfortunately, without a ship acquisition program supporting the technology development, the technology may not receive sufficient support and funding to be ready for the next ship design opportunity as well. (Doerry 2010, 56) The paper further observes: Transitioning to a Product Line approach is more likely to result in technol- ogy being ready for product development when specific ship acquisition pro- grams need them. In a product line approach, BA-4 programs partner with BA-3 S&T efforts to mitigate technical risks and build the industrial capabil- ity to produce a product meeting the ship acquisition needs quickly and

National Naval Responsibility for Naval Engineering Mission 97 affordably. While BA-3 efforts concentrate on achieving a TRL [Technology Readiness Level] level 5, BA-4 Product Line programs concentrate on achiev- ing an EMRL [Engineering and Manufacturing Readiness Level] 3. A signifi- cant advantage of using a Product Line Approach is that technologies are much more mature when incorporated into acquisition programs. . . . employing mature technologies has shown on average to significantly reduce RDT&E Cost Growth. Technology Development Roadmaps are excellent tools for keeping the Resource Sponsor, Science and Technology Community, Acquisition & Engineering Community, and Industry working towards a common vision. The development and promulgation of this shared vision is an important ele- ment of transitioning knowledge among the communities. (Doerry 2010, 56) MAINTAINING CONNECTIONS ACROSS THE WIDER NAVAL ENGINEERING COMMUNITY Maintaining connections across the wider naval engineering community is a key requirement for NNR-NE activities, given the small size of the community and its technical specialization. Two types of activities are important: those focused on bridging gaps between communities or disci- plines in the naval engineering community and those focused on enabling people within or connected with the naval engineering community to perform effectively. The committee considered both types of activities in its assessment. Bridging the Valleys Between Communities Maintaining connections across the wider naval ship engineering com- munity means bridging the valleys that naturally exist between the naval research, design, manufacturing, and operational communities and com- mercial and offshore communities. While these communities share a bond relating to the environments in which they operate, to the systems that they build, and to the manner in which they are deployed, there is an innate separation stemming from rules, regulations, cultures, val- ues, motivations, and behaviors. For example, the professional literature related to technology transfer is replete with discussions of the “valley of death” (NRC 2004). Sustaining an adequate naval engineering pipeline and achieving the twin goals of developing human capital and revitalizing naval ship systems engineering require a focus on ensuring effective connections

98 Naval Engineering in the 21st Century among the elements of the wider naval engineering community. That broad community consists of universities, industry (e.g., shipbuilders, mission system integrators, vendors), Department of the Navy research centers (e.g., NSWC-CD, NUWC-ND), DOD activities (e.g., DOD Ship High Performance Computing Modernization Office, DARPA), private research institutions (e.g., SAIC, APS), classification societies (e.g., the American Bureau of Shipping, Lloyds) and professional societies [e.g., the Society of Naval Architects and Marine Engineers (SNAME), the Amer- ican Society of Naval Engineers (ASNE)], naval activities (e.g., NAVSEA, ONR), and the fleet (e.g., the Navy Warfare Development Command). A 2002 NRC study of alternative approaches for organizing cooperative research addressed various options available to ONR for strengthening its naval engineering cooperative research programs (TRB 2002). To make the task more manageable and to focus on the core strategies to conduct cooperative research programs, the 2002 NRC committee described and evaluated a small number of underlying organizational models: • An individual principal investigator model, • A professional society–community of practitioners model, • A consortium or center model, and • A project-centered model. The 2002 committee used the features of each model to assess each rel- ative to goals for improved organization and management, research, edu- cation, and technology transfer. That committee found that each model had features making it unique and independent of the others, although there were common threads among the models in terms of project man- agement, research theme selection, use of peer-review processes, processes to engage stakeholders, and use of councils and committees to make rec- ommendations and decisions. The 2002 committee also found that there were advantages to hybrids or mixes of the above models under which practices typical of one model were embedded in the operation of another. The committee suggested that a sound strategy would be to include a major project in either the professional society–community of practice model or the consortium model. Table 3-4 summarizes the capacity of each of the models to meet the NNR-NE stated objectives as assessed by the 2002 NRC committee (TRB 2002).

National Naval Responsibility for Naval Engineering Mission 99 TABLE 3-4 Summary of Cooperative Research Organizational Models and How Well They Meet Objectives Model Professional Project- Objective Baseline Society Consortium Centered Human capital Attract students Medium High High Medium Retain and attract Medium Medium High Medium new faculty Provide continuing Low High High Medium education Foster total ship engineers Low High High Medium Naval engineering design Create new research Low Medium High Medium opportunities Promote innovation High Medium High High Ensure research useful Low Medium High High to ship design SOURCE: TRB 2002. The 2002 report found that all three models for cooperative research organizations that it evaluated were capable of meeting all of ONR’s pro- gram objectives. With regard to the ability to meet human capital and naval engineering and design objectives, the consortium model was found better than the professional society model, but both were significantly better than the project-centered model. The 2002 NRC committee, how- ever, suggested that the absolute ranking of these models should depend on the relative importance given by sponsors to each objective. Several cooperative research models have strengths that would be useful in meeting NNR-NE objectives. Specifically, the consortium and project-centered models can encourage innovative and integrative research through their inherent structures, since they involve a high degree of stakeholder participation and therefore have a high probabil- ity of meeting Navy needs. The 2002 NRC committee identified strengths of each of the three cooperative research models. The professional society–community of practice model was found to excel in meeting the need to develop human

100 Naval Engineering in the 21st Century capital. It is particularly strong in attracting and retaining students, supporting continuing education and training programs, and fostering the education and development of total ship engineers, which are principal missions of professional societies. The consortium model has characteris- tics that are well suited to meeting human capital development and naval engineering design objectives for cooperative research programs. How- ever, its success in meeting these objectives will be principally determined by the leadership of the consortium and its ability to represent and balance the needs of the various stakeholders. Finally, the project-centered model has the potential to excel in promoting innovation in naval engineering design and in promoting research that is useful to ship design and produc- tion. This strength is based on the strong, large-scale, interdisciplinary project focus inherent in the model, which includes participation and encourages collaboration of the key stakeholders (TRB 2002). Total ship engineers are developed through a combination of a formal total ship design curriculum and hands-on design experience in multi- disciplinary projects. Regardless of the model selected, the ability to fos- ter development of total ship engineers depends on the opportunities for attainment of the necessary formal education and design experience. Enabling People A critical aspect of developing human capital and revitalizing the naval ship systems engineering community is enabling the people who make up that community. Enabling naval engineers includes the following tasks, as defined by NAVSEA at the committee’s January 2010 workshop (see Appendix A) (H. Stefanyshyn-Piper, presentation to the committee, Jan. 13, 2010): • Providing naval engineering education; • Providing naval engineering training to keep the workforce up to date; • Providing naval engineering mentoring in and outside the workplace, including activities with and through professional technical societies; • Developing tools and collecting supporting data and supporting verification, validation, and accreditation activities; • Developing ship design processes, including those for continuous pro- cess improvement and technology transition; and

National Naval Responsibility for Naval Engineering Mission 101 • Developing documentation, including specifications, standards, hand- books, and rules. Specific needs with respect to developing human capital and revitalizing systems engineering are described in the following section. Conclusion: Connectivity, communication, and human resource and organizational development are important to the success of the naval engineering enterprise. However, the committee was unable to find evi- dence that NNR-NE strategic research planning makes use of measures of connectivity, communication effectiveness, or human capital or organizational development. Recommendation: ONR’s enterprisewide strategic planning and assessment process for NNR-NE should include the following: • A process to develop NNR-NE strategic priorities with respect to connectivity with the wider naval engineering community as well as with respect to communication with stakeholders, technical advisory groups, the user community, and the broader research community. The process should include adoption of one or more of the cooperative research models reviewed in the report of the 2002 NRC Committee on Options for Naval Engineering Cooper- ative Research; • A process to identify NNR-NE priorities associated with human capital and organizational development; and • Metrics associated with connectivity with the naval engineering community and human capital and organizational development. Recommendation: To maintain connectivity across the wider naval engineering community, NNR-NE should utilize the concept of tech- nology interpreter and should continue to support, participate in, and incentivize its ongoing connectivity and communication activities, including conferences, workshops, and seminars, and the activities of ONR Global. ONR should consider adopting additional connectivity and communication activities, including brown bag seminars, scholarly exchange events, and rotation and refreshment opportunities for NNR- NE program officers. The latter should include research sabbaticals at

102 Naval Engineering in the 21st Century Navy laboratories and academic research institutions and in opera- tional Navy settings. INTEGRATING NAVAL ENGINEERING S&T As discussed earlier, the committee suggests that ONR needs to take additional steps to enhance its organizational and management practices in setting performance goals and evaluating results. This is especially critical for research organizations such as ONR with significant multi- disciplinary programs and related challenges. The committee found several examples of interdisciplinary and inte- grative research in the NNR-NE portfolio. In its commissioned papers and workshops, the committee found additional evidence of integrative and interdisciplinary naval engineering projects such as the integrated composite mast (Hackett 2010), and it found a number of materials, hydrodynamics, and ship structures programs. However, the committee concluded that these projects resulted from the efforts of individual pro- gram officers or industry representatives who, for personal or profes- sional reasons, engaged in interdisciplinary research and played a key role in developing such programs, rather than being an outgrowth of system- atic ONR processes that fostered interdisciplinary or integrative research. Recommendation: As part of its enterprisewide strategic planning process, ONR should establish a culture of interdisciplinary and integra- tive research within and around the NNR-NE S&T enterprise and should establish processes that foster, encourage, and incentivize inter- disciplinary or integrative research. The NNR-NE interdisciplinary and integrative research objectives should be established as part of the strate- gic planning processes and should include assessment, benchmarking, and continuous process improvement components. DEVELOPING HUMAN CAPITAL AND REVITALIZING NAVAL SHIP SYSTEMS ENGINEERING The 1990s were a period of great change within DOD and the Depart- ment of the Navy precipitated by the fall of the former Soviet Union, the end of the cold war, and the desire to capitalize on the so-called “peace

National Naval Responsibility for Naval Engineering Mission 103 dividend.” One result was a substantial downsizing of the Navy organi- zations previously responsible for ship design and acquisition, accompa- nied by the outsourcing of these services to industry. According to the General Accounting Office, “DoD performed this downsizing [from 1989 to 2002] without proactively shaping the civilian workforce to ensure that it had the specific skills and competencies needed to accom- plish future DoD missions” (GAO 2004, 7). During that decade, the Department of the Navy in general and NAVSEA in particular saw a deep reduction in the human capital required to design, develop, acquire, deploy, and maintain the naval fleet. NAVSEA headquarters alone saw the cadre of highly experienced naval ship design engineers shrink from about 1,200 in 1992 to fewer than 300 in 2005 (Keane et al. 2009, 47). Concerns related to the naval acquisition work- force were articulated by then Secretary of the Navy Donald Winter in a 2007 speech before the Navy League: “There has been a steady erosion in domain knowledge within the Department of the Navy over the past several decades, resulting in an overreliance on contractors in the per- formance of core in-house functions” (Winter 2007). Secretary Winter went on to say that while “the Department’s level of technical expertise associated with naval architecture and design is relatively high, our knowledge of the shipbuilding process is short of what it has been in the past, and what it needs to be in the future. Our challenge is to under- stand how to integrate design and production technology into an acqui- sition process that industry can execute. This requires a deep knowledge of systems engineering and a profound understanding of the acquisi- tion process. Systems engineering is key to ensuring that each ship is configured to optimize the fleet” (Winter 2007). Secretary Winter discussed the steps necessary to correct the deficien- cies in naval ship acquisition, and the workforce in particular, saying that “the Navy needs to provide knowledgeable program oversight. Hiring top-quality people who have experience with large shipbuilding pro- grams is essential. The ability to assign an experienced and capable team must be a precondition to a program’s initiation. Finding and develop- ing the people we need is easier said than done, and it will take time to rectify this problem, but we cannot ignore the leverage that can be obtained by putting the right, experienced and prepared people, in the right posi- tions” (Winter 2007).

104 Naval Engineering in the 21st Century The need to develop the requisite human capital and revitalize naval ship systems engineering has been clearly recognized by the Navy leader- ship as a key goal. Today, efforts exist not only to protect and maintain the mission-critical competency areas but also to develop them for the present and future. The development and monitoring of the health of naval engi- neering human capital have been actively pursued within NAVSEA by using tools such as the Human Capital Digital Dashboard (Tropiano 2005), which provides an objective assessment of the following: • Alignment of engineers with the technical authority chain of command; • Availability and adequacy of technical documentation, including spec- ifications, standards, tools, and processes; • Workforce demographics, including age and levels of education; • Workforce skills, including experience, certifications, and other special abilities; • Workforce health metrics, including assessments of leadership skills, mission capability, and technical documentation; • Problem areas, such as critical vacancies, anticipated retirements, and substandard assessments; and • Long-term health actions in these areas. Developing the Navy’s next generation of naval engineering leaders is a challenging problem. During the 1990s, as a result of changes in acquisition policy, preliminary and contract design for Navy ships that NAVSEA had previously performed in-house began to be contracted out to shipbuilders. In addition, the rate of new ship acquisition declined in this period compared with that of the previous decade. The contraction of the NAVSEA headquarters ship design staff noted above was a conse- quence. This problem is being addressed on several fronts. One initiative was the creation of the Center for Innovation in Ship Design (CISD) in 2002 by NAVSEA, ONR, and NSWC. CISD was tasked in 2006 “to develop a Human Capital Strategy (HCS) for Ship Design Acquisition Workforce Improvement. The Ship Design Management HCS will ensure a highly experienced warship design workforce to sustain NAVSEA as the nation’s leader in naval ship design” (Keane et al. 2009, 46). The commit- tee noted that this focused program has in large part sustained the core

National Naval Responsibility for Naval Engineering Mission 105 competencies that are essential to rebuilding the naval ship systems engi- neering and acquisition workforce. The need to train, develop, and refresh the naval ship systems engineer- ing workforce and technology base continuously was articulated in previ- ous studies (NRC 2000; TRB 2002; U.S. Department of Commerce 2001). It was widely discussed in the naval engineering professional journals (ASNE 1992) and in academic settings (Chryssostomidis et al. 2000). These writings served to identify the “failure of government and industry research and development (R&D) organizations to stimulate the education, inno- vation, and competitiveness improvements needed to support the U.S. shipbuilding industry. These reports highlight the significant role the Department of Defense must play in leading the R&D investment stimu- lus for the cooperative development of innovative, cost and labor saving technologies by the U.S. shipbuilding industry and the supporting aca- demic institutions. Additionally, each subsequent report has continued to identify the areas of education, innovation and competitiveness as problematic in the U.S. shipbuilding industry” (ONR 2001, 1–2). The naval engineering human capital pipeline is illustrated in Fig- ure 3-1. The pipeline begins with the kindergarten through 12th grade (K-12) pool of STEM students. Those high school students who enter universities and colleges and graduate with a bachelor of science degree will enter the general engineering workforce in the tens of thousands annually, while thousands will continue on for advanced degrees. Each year, no more than a few thousand new graduates (and in some years probably less than a thousand) at all degree levels will enter the naval engineering enterprise workforce. Of those graduates who do enter naval engineering, a small number each year will leave the workforce to pur- sue a higher degree, motivated by their experience in naval engineering. A select few will stay on in academia to educate the next generation of naval engineers. The ever-present demand signal for graduates is driven by the natural progression of scientists and engineers in their careers and eventual attrition from the workforce either through a career change or retirement. Supporting this pipeline for the development of naval engi- neers is the infrastructure of primary and secondary schools, colleges and universities, government research activities, private-sector research

ONR, NAVSEA, $ Industry Research Funding Research Stay in Faculty to teach Faculty Undergraduate academia Projects Students Masters PhD Hundreds K-12 K2 Thousands BS STEM STEM Tens of thousands Enter the workforce Enter the workforce Thousands Hundreds Enter the workforce Enter the Naval Enterprise Enter the Naval Enterprise Tens Tens of thousands Hundreds Enter the Naval Enterprise Thousands Leave Naval Engineering Return to School FIGURE 3-1 Naval engineering value stream and pipeline. (SOURCE: National Shipbuilding Research Program 2009. Printed with permission from the National Shipbuilding Research Program, from the Shipbuilding Engineering Education Consortium Viability and Operational Concepts Final Report, June 16, 2009.)

National Naval Responsibility for Naval Engineering Mission 107 institutions, and university research centers. The National Shipbuilding Research Consortium’s 2009 study of the naval engineering workforce, conducted for NAVSEA, concluded that the demand for hiring of entry- level naval engineers by NAVSEA, U.S. shipbuilders, and the supporting industries is about 2,000 per year, while graduates of accredited pro- grams of naval engineering total only about 200 annually (National Ship- building Research Program 2009, 30). This demand estimate appears inconsistent with the report’s estimate of total employment of naval engineers in these sectors of 15,000 (National Shipbuilding Research Program 2009, 21). Any excess of demand over supply must be filled by hiring and training engineers from other specializations. Developing a robust naval engineering pipeline is critical to the devel- opment of a robust naval engineering enterprise. NNR-NE efforts in naval engineering S&T workforce development have been sporadic and inadequately supported to date. ONR has been designated the lead agency for STEM efforts for the Department of the Navy; however, such responsibilities are considered an ancillary rather than a core functional responsibility. Outreach programs have been successful in reaching students and cre- ating an interest in STEM education and potential naval and maritime careers. ONR supports SNAME efforts to deploy the SeaPerch program nationally and to develop ways to expand and enhance the promotion as part of ONR’s NNR-NE outreach. Professional technical societies such as SNAME and ASNE appear to be well positioned to provided leader- ship and support for these outreach initiatives. However, limitations do exist in the professional societies’ ability to perform this outreach given their modest number of volunteers and funding for professional staff in relation to the broad K-12 population. Recommendation: ONR should reinvigorate its efforts in developing the 21st century naval engineering workforce, including improvement of outreach activities to underrepresented groups. ONR’s lead role for STEM activities should be strengthened and incorporated into its enter- prisewide strategic planning processes, and performance metrics for workforce development and STEM achievements should be identified, measured, incentivized, and included in ONR’s assessment, benchmark- ing, and continuous process improvement activities.

108 Naval Engineering in the 21st Century ONR should consider additional approaches to increase the efficacy of the workforce development and STEM initiatives, including the following: • Targeting specific populations in a geographic region with professional connection to naval engineering activities (e.g., local naval architecture university, shipbuilder, naval facility); • Expanding funding and volunteer support for outreach programs though collaborative efforts between government activities, indus- try, and professional societies (e.g., the Junior Engineering Technical Society); and • Leveraging NAVSEA funding under the Naval Engineering Education Center Consortium to support SeaPerch and other initiatives. REFERENCES Abbreviations AFOSR Air Force Office of Scientific Research ASNE American Society of Naval Engineers DHS Department of Homeland Security DON Department of the Navy GAO General Accounting Office or Government Accountability Office NRC National Research Council NSF National Science Foundation ONR Office of Naval Research TRB Transportation Research Board AFOSR. 2007. Proposer’s Guide to the AFOSR Research Programs. Aug. ASNE. 1992. Preserving Our Naval Engineering Capability. Naval Engineers Journal, Vol. 104, No. 4, July, pp. 11–13. Revised May 1998. Bond, T. C. 1999. The Role of Performance Measurement in Continuous Improvement. International Journal of Operations and Production Management, Vol. 19, No. 2, pp. 1318–1334. Brown, M. 1996. Keeping Score: Using the Right Metrics for World Class Performance. American Management Association, Washington, D.C. Bukowitz, W., and G. Petrash. 1997. Visualizing, Measuring and Managing Knowledge. Research and Technology Management, Vol. 40, No. 4, pp. 24–31.

National Naval Responsibility for Naval Engineering Mission 109 Chang, L., and B. Birkett. 2004. Managing Intellectual Capital in a Professional Service Firm: Exploring the Creativity–Productivity Paradox. Management Accounting Research, Vol. 15, pp. 7–31. Chen, J., Z. Zhu, and H. Xie. 2004. Measuring Intellectual Capital: A New Model and Empirical Study. Journal of Intellectual Capital, Vol. 5, No. 1, pp. 195–212. Chryssostomidis, C., M. Bernitsas, and D. Burke, Jr. 2000. Naval Engineering: A National Naval Obligation. Massachusetts Institute of Technology Ocean Engineering Design Laboratory, May. DHS. 2009. Developing Technology to Protect America. Science and Technology Direc- torate. National Academy of Public Administration, Washington, D.C. Doerry, N. 2010. Transitioning Technology to Naval Ships. Paper commissioned by the committee, June. DON. 2010. Department of the Navy Fiscal Year (FY) 2011 Budget Estimates: Justifi- cation of Estimates: Research, Development, Test and Evaluation, Navy: Budget Activity 1–3. Feb. Eccles, R. 1991. The Performance Measurement Manifesto. Harvard Business Review, Jan.–Feb., pp. 131–137. Eccles, R., and P. Pyburn. 1992. Creating a Comprehensive System to Measure Perfor- mance. Management Accounting, Oct. Edvinsson, L. 1997. Developing Intellectual Capital at Skandia. Long Range Planning, Vol. 30, No. 3, pp. 366–373. Edvinsson, L., and M. Malone. 1997. Intellectual Capital: Realizing Your Company’s True Value by Finding Its Hidden Brainpower. Harper Business, New York. Frigo, M. L., and K. R. Krumwiede. 2000. The Balanced Scorecard: A Winning Perfor- mance Measurement System. Strategic Finance, Jan., pp. 50–54. GAO. 2004. DOD Civilian Personnel Comprehensive Strategic Workforce Plans Needed. Report GAO-04-753. June. GAO. 2006. Stronger Practices Needed to Improve DOD Technology Transition Processes. Report GAO-06-883. Sept. Grabowski, M., and K. H. Roberts. 1999. Risk Mitigation in Virtual Organizations. Orga- nization Science, Vol. 10, No. 6, Nov.–Dec., pp. 704–721. Hackett, J. P. 2010. Composites Road to the Fleet—A Collaborative Success Story. Paper commissioned by the committee, June 18. Hagan, J. 2010. Human Systems Integration/Crew Design Process Development in the Zumwalt Destroyer Program—A Case Study in the Importance of Wide Collabora- tion. Paper commissioned by the committee, June 8. Ittner, C. D., and D. F. Larcker. 1998a. Are Nonfinancial Measures Leading Indicators of Financial Performance? An Analysis of Customer Satisfaction. Journal of Accounting Research, Vol. 36, pp. 1–35.

110 Naval Engineering in the 21st Century Ittner, C. D., and D. F. Larcker. 1998b. Innovations in Performance Measurement: Trends and Research Implications. Journal of Management Accounting Research, Vol. 10, pp. 205–238. Joia, L. 2000. Measuring Intangible Corporate Assets: Linking Business Strategy with Intellectual Capital. Journal of Intellectual Capital, Vol. 1, No. 1, pp. 68–84. Kaplan, R., and D. Norton. 1992. The Balanced Scorecard—Measures That Drive Per- formance. Harvard Business Review, Vol. 70, No. 1, pp. 71–79. Kaplan, R., and D. Norton. 1996a. The Balanced Scorecard. Harvard Business School Press, Boston, Mass. Kaplan, R., and D. Norton. 1996b. Linking the Balanced Scorecard to Strategy. California Management Review, Vol. 39, No. 1. Kaplan, R., and D. Norton. 1996c. Using the Balanced Scorecard as a Strategic Manage- ment System. Harvard Business Review, Vol. 74, No. 1, pp. 75–85. Kaplan, R., and D. Norton. 2001. The Strategy-Focused Organization. Harvard Business School Press, Boston, Mass. Kaplan, R., and D. Norton. 2004. Strategy Maps: Converting Intangible Assets into Tangible Outcomes. Harvard Business School Press, Boston, Mass. Keane, R. G., Jr., H. Fireman, J. J. Hough, and K. Cooper. 2009. A Human Capital Strategy for Ship Design Acquisition Workforce Improvement: The U.S. Navy’s Center for Innovation in Ship Design. Naval Engineers Journal, Vol. 121, No. 4, pp. 45–67. Lyons, J. W., and R. Chait. 2009. Strengthening Technical Peer Review at the Army S&T Laboratories. Center for Technology and National Security Policy, National Defense University, Washington, D.C., March. Melnyk, S. A., D. M. Stewart, and M. Swink. 2004. Metric and Performance Measure- ment in Operations Management: Dealing with the Metrics Maze. Journal of Opera- tions Management, Vol. 22, pp. 16–29. National Academies. 2005. Facilitating Interdisciplinary Research. National Academies Press, Washington, D.C. http://www.nap.edu/catalog/11153.html. Accessed Dec. 8, 2010. National Shipbuilding Research Program. 2009. Shipbuilding Engineering Education Con- sortium (SEEC) Viability and Operational Concepts Final Report. June 16. NRC. 1999. Evaluating Federal Research Programs: Research and the Government Perfor- mance and Results Act. National Academy Press, Washington, D.C. NRC. 2000. An Assessment of Naval Hydromechanics Science and Technology. National Academy Press, Washington, D.C. NRC. 2004. Accelerating Technology Transition: Bridging the Valley of Death for Materi- als and Processes in Defense Systems. National Academies Press, Washington, D.C.

National Naval Responsibility for Naval Engineering Mission 111 NSF. 2011a. FY 2012 Budget Request to Congress. Feb. 14. http://www.nsf.gov/about/ budget/fy2012/pdf/fy2012_rollup.pdf. NSF. 2011b. Proposal and Award Policies and Procedures Guide. Jan. ONR. 2001. Memorandum: National Naval Program for Naval Engineering. Oct. 22. ONR. 2009a. Long Range BAA for Navy and Marine Corps Science and Technology. BAA 10-001. Sept. 18. ONR. 2009b. Naval S&T Strategic Plan: Defining the Strategic Direction for Tomorrow. Feb. http://www.dtic.mil/cgi- bin/GetTRDoc?AD=ADA499909&Location=U2&doc=GetTRDoc.pdf. Pelz, D. C. 1956. Some Social Factors Related to Performance in a Research Organiza- tion. Administrative Science Quarterly, Vol. 1, No. 3, Dec., pp. 310–325. Porter, A. L., J. D. Roessner, A. S. Cohen, and M. Perrault. 2006. Interdisciplinary Research: Meaning, Metrics and Nurture. Research Evaluation, Vol. 15, No. 3, Dec., pp. 187–195. Reugg, R. 2007. Quantitative Portfolio Evaluation of U.S. Federal Research and Devel- opment Programs. Science and Public Policy, Dec., pp. 723–730. Roberts, K. H. 1990. Some Characteristics of One Type of High Reliability Organization. Organization Science, Vol. 1, No. 2, pp. 160–176. Roberts, K. H., and D. M. Rousseau. 1989. Research in Nearly Failure-Free, High Relia- bility Organizations. IEEE Transactions on Engineering Management, Vol. 36, No. 2, May, pp. 132–139. Sharp, W. 2007. Research Managers Skillfully Navigate, Execute the Basic Research Funding Process. Wright–Patterson Air Force Base News, updated June 28. Tan, K., and K. Platts. 2003. Linking Objectives to Action Plans: A Decision Support Approach Based on Cause–Effect Linkages. Decision Sciences, Vol. 34, No. 3, pp. 569–593. Tan, K., K. Platts, and J. Nobel. 2004. Building Performance Through In-Process Mea- surement: Toward an “Indicative” Scorecard for Business Excellence. International Journal of Productivity and Performance Management, Vol. 53, No. 3, pp. 233–244. Taylor, J., J. L. Price, and D. K. Phelps. 2010. Energy and Power Community of Interest: Energy and Power S&T Overview. Presented at 46th AIAA–ASME–SAE–ASEE Joint Propulsion Conference and Exhibit and 8th International Energy Conversion Engi- neering Conference, Nashville, Tenn., July 28. TRB. 2002. Special Report 266: Naval Engineering: Alternative Approaches for Organizing Cooperative Research. National Academies, Washington, D.C. Tropiano, M., Jr. 2005. Human Capital Digital Dashboard: NAVSEA’s Future Method of Measuring Community Health. Defense AT&L Magazine, Nov.–Dec.

112 Naval Engineering in the 21st Century U.S. Department of Commerce. 2001. National Security Assessment of the U.S. Shipbuild- ing and Repair Industry. May. U.S. Department of Energy. 1995. How to Measure Performance: A Handbook of Tech- niques and Tools. Defense Programs, Special Projects Group (DP-31). U.S. Department of Energy. 2001. Establishing an Integrated Performance Measurement System. Performance-Based Management Special Interest Group. Winter, D. 2007. Remarks at Sea–Air–Space Exposition. Navy League, April 3. Yeniyurt, S. 2003. A Literature Review and Integrative Performance Measurement Framework for Multinational Companies. Marketing Intelligence and Planning, Vol. 21, No. 3, pp. 134–142.

Next: 4 Results and Future Prospects of the National Naval Responsibility for Naval Engineering »
Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets -- Special Report 306 Get This Book
×
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

TRB Special Report 306: Naval Engineering in the 21st Century: The Science and Technology Foundation for Future Naval Fleets examines the state of basic and applied research in the scientific fields that support naval engineering and explores whether Office of Naval Research (ONR) activities, under its National Naval Responsibility for Naval Engineering (NNR-NE) initiative, have been effective in sustaining these fields.

The committee developed a series of conclusions and recommendations in five areas--the value of the NNR-NE, the state of science and technology supporting naval engineering, the wholeness of the NNR-NE research portfolio, opportunities for enhancement of research and education, and the effectiveness of the NNR-NE initiative.

The report's recommendations are addressed to the administrators of the NNR-NE initiative and of ONR.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!