National Academies Press: OpenBook

Managing for High-Quality Science and Engineering at the NNSA National Security Laboratories (2013)

Chapter: F--The Investment/Value Returned Framework for Management of S&E

« Previous: E--What the Laboratories' Management and Operations (M&O) Contracts Say About the Conduct and Evaluation of Science and Engineering
Suggested Citation:"F--The Investment/Value Returned Framework for Management of S&E." National Research Council. 2013. Managing for High-Quality Science and Engineering at the NNSA National Security Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/13367.
×

F

The Investment/Value Returned Framework for Management of S&E

An effective management system for science and engineering (S&E) has four main responsibilities: deciding what to do and allocating the investment; supporting, monitoring, and facilitating the execution of the chosen portfolio; evaluation, and communication, of the results; and maintaining and growing capabilities, both human and technical facilities. Any comprehensive assessment of a particular management system must include both a structural portion—analyzing and assessing the systems, processes, and relationships that are in place—and an implementation portion, assessing the way in which the management system actually operates and is used.

A general approach to such an assessment is the Investment/Value Returned (I/V) framework, which has been used with good effect in industrial research settings. The core principle of the I/V framework is that S&E management should be driving toward an optimum value return on the R&D investment that is being supported. Creation of such an I/V framework provides a consistent set of intellectual underpinnings across a wide and diverse set of activities, and can serve as a key element of the core culture of an institution. The framework provides coherent and relatively explicit linkage between the enterprise’s mission to the specific results aimed for at the working level, and upward from the individual scientist and engineer back to that overarching mission.

An I/V approach to the effectiveness with which management is operating involves at least the following four management challenges:

•   Management must have a clear view of the nature of the returns—the value received—from investing in S&E. How do these values support the missions? Into what categories does the value fit? Inevitably there will be multiple “buckets” of value. What metrics and indicators are available and used to assess value returned, both retrospective and prospective? It is important to recognize that management must play the key role in working with the key investors to develop a shared understanding of the returns, their value, as well as appropriate metrics and indicators to assess value. Left to their own to decide, the investors will often fail to recognize how much value can be, or is being, delivered, and will thus suboptimize their investment, for example by putting too little value on areas with high value in the longer term.

•   On a prospective basis, how does management allocate investment, both to basic S&E versus the other elements, and within the S&E portfolio, aimed at maximizing the value created? For S&E it is clear that an optimum allocation methodology will involve both top-down and bottom-up approaches, vigorous debate, and ultimately a set of decisions. This discussion is expanded below in the section on “Best Practice Processes.”

•   How does management ensure that infrastructure and operational processes are in place relative to the S&E workforce, aimed at maximizing their ability and their motivation to create and deliver value? What metrics and indicators are used in this area? Is the set of tools and processes in place to track on an ongoing basis how much and how well value is being created and delivered by the S&E investment over the long term (for basic research) and in the nearer term (for applied research), and what inhibitors exist ?

•   Are closed-loop quality processes in place to continuously improve the output, to ensure that technical capabilities are sustained and grown, driving change in each step from portfolio selection to

Suggested Citation:"F--The Investment/Value Returned Framework for Management of S&E." National Research Council. 2013. Managing for High-Quality Science and Engineering at the NNSA National Security Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/13367.
×

operational processes to infrastructure investment? A key element of this aspect of management is the set of processes which ensure that the highest level of talent is recruited to the institution, nurtured, developed, and retained.

Evaluating the effectiveness of S&E management against these four management challenges provides a far more systematic picture than does the all-too-common collection of anecdotes and qualitative impressions.

For laboratories such as the NNSA National Security labs, the success of which is strongly dependent on S&E and which do not have a profit motive, the set of values returned from the S&E investment includes at least the following:

•   Essential support for achieving the labs’ core missions, through core work that exploits deep aspects of physics, materials science, engineering and computational science. Without a strong internal base in these areas of S&E the core programs of the laboratories would not be viable.

•   Highly talented and motivated staff, both in S&E itself and across the organization, to provide a basis for long-term success in addressing a changing portfolio of work.

•   Deep contact with, and the ability to draw on and contribute to (and to some degree shape), the agendas of the broad world of science, which can, in turn, be mined for the laboratories’ own uses.

•   Targeted basic research to fill gaps not pursued in the open community.

The process of investment allocation—both strategic and cyclical, and both human and fiscal—is clearly one of management’s major challenges, and has a huge impact on what value will be returned. This process is almost always seriously constrained by resource limitations, and demands that choices be made among a set of investment proposals each promising high value return. The investment allocation process, particularly for S&E, must effectively balance broad top-down objective setting with proposals driven by the instincts and intuitions of the S&E staff, both individuals and small groups. These processes must aim for value creation in a balanced way, and ultimately rely to a large degree on the judgment of talented and experienced management. There is no effective algorithmic approach.

A key role of management is to ensure that supportive infrastructure and processes enable optimum use of the S&E assets in executing the committed projects and programs. At a fundamental level this certainly must include minimizing the time S&E staff must spend on less productive, less valueproducing, or even interruptive activity. Metrics and indicators are key to achieving this goal, as are consequent ongoing attention to reduce non-productive expenditure of time. Clearly some portion of the S&E researcher time must be diverted from research to administrative activity, proposal preparation, reporting, training in security and safety, travel, and the like.

A BEST-PRACTICE MANAGEMENT CYCLE

In the experience of the committee, the most effective systems for managing laboratories are built around a closed-loop cycle involving planning and allocation, execution, and retrospective assessment, all driven by the I/V returned model. The cyclic processes are clearly linked to a strategic backdrop tied to the institutional mission. We describe one such cyclic set of processes in this section. This cycle and its processes do not deal with the core project management, which may drive the majority of institutional resource, but deal instead with the underlying and supporting S&E which is essential to success in many deeply technical enterprises. The period of this cyclic process would typically be annual.

A key backdrop and underpinning for this process is an annually updated Outlook, or world view. This Outlook accumulates knowledge and projections from numerous sources to provide a detailed view of the environment in which the institution must achieve its mission, and is sufficiently detailed that it can be used as an environment in which potential scenarios can be modeled. It includes information on, and potential impacts of, the relevant drivers of change—technical, economic, potentially political and

Suggested Citation:"F--The Investment/Value Returned Framework for Management of S&E." National Research Council. 2013. Managing for High-Quality Science and Engineering at the NNSA National Security Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/13367.
×

social—competitive factors, customer and marketplace drivers, etc. It captures the essence of “Here is the world we see in front of us in which we must succeed in our mission.” And it forms, in various versions, the base for much of the communication among the parties involved: the investors and beneficiaries of the S&E work, the S&E management and technical staff, partners, customers, and others involved in the enterprise. By its exposure to the relevant parties, and open discussion and debate about the assumptions built into the Outlook, inputs for change are continuously provided. A reasonable point to define as the beginning of the (annual) cycle is a fairy formal update of the Outlook, aimed at incorporating what has been learned in the past year, as well as identifying and dealing with newly emerging trends and developments in all the key driver areas.

With an updated Outlook having achieved some level of consensus acceptance, the Strategy and Planning work begins. This will update a durable long-term set of goals and approaches to their achievement, and lay out of a set of investment options for the S&E investment. This investment allocation process is clearly the most difficult part of the cycle, since the options inevitably cannot all be committed within the available resources. As is clearly the case for the NNSA labs, and also experienced in the industry sector, the resources for S&E must be allocated among three fairly distinct types of activities. First, the bulk of the S&E must support the mission and core project goals, and thus the prioritization of this portion is driven primarily top-down from the mission, but with significant input from and debate with the S&E management and staff, both on the “what” and the “how.” A second, and essential, portion of the resource—in NNSA’s case, mostly represented by the LDRD resources—is primarily allocated by S&E management, laboratory directors and their colleagues, with major bottom-up input. It is this portion of the resource which ensures the health of the labs; it aims to drive major breakthroughs and develop and exploit new areas of knowledge which are not on the direct and obvious path of the mission projects. The third source and use of resource, common in industry as well and in the NNSA labs, is external funding, in the best case driven by the outside party recognizing the potential of the S&T team to create major value for the outside investor. This is not simply “works for hire” but, with proper management, represents exploitation of unique resources and programs which are synergistic with the core S&E mission and creates value for both the external investor and the S&E organization. While one might aim for this portion of the portfolio to have stability of the same sort as the “base” programs, it will almost inevitably have more dynamics through the cycle than the others.

A key piece of this planning portion of the cycle is the allocation of resources for both equipment and facilities, driven by the strategic plan. The human resource aspects of the plan are also obviously key.

The output of this portion of the cycle is an updated strategy and operating plan for the period. Best-practice organizations have, as do the NNSA labs, multiple mechanisms of both internal and external review and advice on execution. These are best structured into a coordinated rhythm and calendar, with periodicities appropriate to the particular type of review. An obvious risk is excessive and intrusive reviewing and auditing, which distracts energy from the work at hand. Appropriate dashboards, metrics and indicators are useful during this process, although they do not substitute for deep dives into issues which are surfaced by the dashboards.

Closure in the cycle is typically represented by a series of higher level reviews focused on what has been accomplished relative to committed plans, other achievements, goals missed and causes, and some quantitative metrics constituting a balanced scorecard used for overall assessment of effectiveness. In addition to overall S&E assessment, a valuable output of this end-of-cycle summary is a set of lessonslearned, and inputs for the subsequent cycle.

There is evidence from industry that driving management and culture from this I/V framework and sustained use of such a process cycle can have major positive effects in many areas. These include sustaining support for the S&E enterprise, and building a coherent internal culture which effectively balances, for the scientist or engineer, the value system of his or her technical area with the value system uniquely appropriate to the institution in which his or her S&E in embedded.

Suggested Citation:"F--The Investment/Value Returned Framework for Management of S&E." National Research Council. 2013. Managing for High-Quality Science and Engineering at the NNSA National Security Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/13367.
×
Page 70
Suggested Citation:"F--The Investment/Value Returned Framework for Management of S&E." National Research Council. 2013. Managing for High-Quality Science and Engineering at the NNSA National Security Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/13367.
×
Page 71
Suggested Citation:"F--The Investment/Value Returned Framework for Management of S&E." National Research Council. 2013. Managing for High-Quality Science and Engineering at the NNSA National Security Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/13367.
×
Page 72
Next: G--Selected Supporting Information »
Managing for High-Quality Science and Engineering at the NNSA National Security Laboratories Get This Book
×
Buy Paperback | $39.00 Buy Ebook | $31.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The three National Security Laboratories--Los Alamos National Laboratory (LANL), Sandia National Laboratories (SNL), and Lawrence Livermore National Laboratory (LLNL)--are managed by private-sector entities under contract to the National Nuclear Security Administration (NNSA). The FY2010 Defense Authorization Act mandated that NNSA task the National Research Council (NRC) to study the quality and management of Science and Engineering (S&E) at these Laboratories.

This study (addressing a total of 5 tasks) is being conducted in two phases. This report covers the first phase, which addresses the relationship between the quality of the science and engineering at the Laboratory and the contract for managing and operating the Laboratory (task 4), and also addresses the management of work conducted by the Laboratory for entities other than the Department of Energy (task 5). The study's second phase will evaluate the actual quality of S&E in key subject areas.

Managing for High-Quality Science and Engineering at the NNSA National Security Laboratories presents assessments of the evolution of the mission of the NNSA Labs and the management and performance of research in support of the missions, and the relationship between the Laboratory Directed Research and Development (LDRD) program and the ability of the Labs to fulfill their mission. The report examines the framework for managing science and engineering research at the Labs and provides an analysis of the relationships among the several players in the management of the Labs--the NNSA, the site offices, the contractors, and the Lab managers--and the effect of that relationship on the Laboratories' ability to carry out science and engineering research.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!