F

The Investment/Value Returned Framework for Management of S&E

An effective management system for science and engineering (S&E) has four main responsibilities: deciding what to do and allocating the investment; supporting, monitoring, and facilitating the execution of the chosen portfolio; evaluation, and communication, of the results; and maintaining and growing capabilities, both human and technical facilities. Any comprehensive assessment of a particular management system must include both a structural portion—analyzing and assessing the systems, processes, and relationships that are in place—and an implementation portion, assessing the way in which the management system actually operates and is used.

A general approach to such an assessment is the Investment/Value Returned (I/V) framework, which has been used with good effect in industrial research settings. The core principle of the I/V framework is that S&E management should be driving toward an optimum value return on the R&D investment that is being supported. Creation of such an I/V framework provides a consistent set of intellectual underpinnings across a wide and diverse set of activities, and can serve as a key element of the core culture of an institution. The framework provides coherent and relatively explicit linkage between the enterprise’s mission to the specific results aimed for at the working level, and upward from the individual scientist and engineer back to that overarching mission.

An I/V approach to the effectiveness with which management is operating involves at least the following four management challenges:

•   Management must have a clear view of the nature of the returns—the value received—from investing in S&E. How do these values support the missions? Into what categories does the value fit? Inevitably there will be multiple “buckets” of value. What metrics and indicators are available and used to assess value returned, both retrospective and prospective? It is important to recognize that management must play the key role in working with the key investors to develop a shared understanding of the returns, their value, as well as appropriate metrics and indicators to assess value. Left to their own to decide, the investors will often fail to recognize how much value can be, or is being, delivered, and will thus suboptimize their investment, for example by putting too little value on areas with high value in the longer term.

•   On a prospective basis, how does management allocate investment, both to basic S&E versus the other elements, and within the S&E portfolio, aimed at maximizing the value created? For S&E it is clear that an optimum allocation methodology will involve both top-down and bottom-up approaches, vigorous debate, and ultimately a set of decisions. This discussion is expanded below in the section on “Best Practice Processes.”

•   How does management ensure that infrastructure and operational processes are in place relative to the S&E workforce, aimed at maximizing their ability and their motivation to create and deliver value? What metrics and indicators are used in this area? Is the set of tools and processes in place to track on an ongoing basis how much and how well value is being created and delivered by the S&E investment over the long term (for basic research) and in the nearer term (for applied research), and what inhibitors exist ?

•   Are closed-loop quality processes in place to continuously improve the output, to ensure that technical capabilities are sustained and grown, driving change in each step from portfolio selection to



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 70
F The Investment/Value Returned Framework for Management of S&E An effective management system for science and engineering (S&E) has four main responsibilities: deciding what to do and allocating the investment; supporting, monitoring, and facilitating the execution of the chosen portfolio; evaluation, and communication, of the results; and maintaining and growing capabilities, both human and technical facilities. Any comprehensive assessment of a particular management system must include both a structural portion—analyzing and assessing the systems, processes, and relationships that are in place—and an implementation portion, assessing the way in which the management system actually operates and is used. A general approach to such an assessment is the Investment/Value Returned (I/V) framework, which has been used with good effect in industrial research settings. The core principle of the I/V framework is that S&E management should be driving toward an optimum value return on the R&D investment that is being supported. Creation of such an I/V framework provides a consistent set of intellectual underpinnings across a wide and diverse set of activities, and can serve as a key element of the core culture of an institution. The framework provides coherent and relatively explicit linkage between the enterprise’s mission to the specific results aimed for at the working level, and upward from the individual scientist and engineer back to that overarching mission. An I/V approach to the effectiveness with which management is operating involves at least the following four management challenges: • Management must have a clear view of the nature of the returns—the value received—from investing in S&E. How do these values support the missions? Into what categories does the value fit? Inevitably there will be multiple “buckets” of value. What metrics and indicators are available and used to assess value returned, both retrospective and prospective? It is important to recognize that management must play the key role in working with the key investors to develop a shared understanding of the returns, their value, as well as appropriate metrics and indicators to assess value. Left to their own to decide, the investors will often fail to recognize how much value can be, or is being, delivered, and will thus sub- optimize their investment, for example by putting too little value on areas with high value in the longer term. • On a prospective basis, how does management allocate investment, both to basic S&E versus the other elements, and within the S&E portfolio, aimed at maximizing the value created? For S&E it is clear that an optimum allocation methodology will involve both top-down and bottom-up approaches, vigorous debate, and ultimately a set of decisions. This discussion is expanded below in the section on “Best Practice Processes.” • How does management ensure that infrastructure and operational processes are in place relative to the S&E workforce, aimed at maximizing their ability and their motivation to create and deliver value? What metrics and indicators are used in this area? Is the set of tools and processes in place to track on an ongoing basis how much and how well value is being created and delivered by the S&E investment over the long term (for basic research) and in the nearer term (for applied research), and what inhibitors exist ? • Are closed-loop quality processes in place to continuously improve the output, to ensure that technical capabilities are sustained and grown, driving change in each step from portfolio selection to 70

OCR for page 70
operational processes to infrastructure investment? A key element of this aspect of management is the set of processes which ensure that the highest level of talent is recruited to the institution, nurtured, developed, and retained. Evaluating the effectiveness of S&E management against these four management challenges provides a far more systematic picture than does the all-too-common collection of anecdotes and qualitative impressions. For laboratories such as the NNSA National Security labs, the success of which is strongly dependent on S&E and which do not have a profit motive, the set of values returned from the S&E investment includes at least the following: • Essential support for achieving the labs’ core missions, through core work that exploits deep aspects of physics, materials science, engineering and computational science. Without a strong internal base in these areas of S&E the core programs of the laboratories would not be viable. • Highly talented and motivated staff, both in S&E itself and across the organization, to provide a basis for long-term success in addressing a changing portfolio of work. • Deep contact with, and the ability to draw on and contribute to (and to some degree shape), the agendas of the broad world of science, which can, in turn, be mined for the laboratories’ own uses. • Targeted basic research to fill gaps not pursued in the open community. The process of investment allocation—both strategic and cyclical, and both human and fiscal—is clearly one of management’s major challenges, and has a huge impact on what value will be returned. This process is almost always seriously constrained by resource limitations, and demands that choices be made among a set of investment proposals each promising high value return. The investment allocation process, particularly for S&E, must effectively balance broad top-down objective setting with proposals driven by the instincts and intuitions of the S&E staff, both individuals and small groups. These processes must aim for value creation in a balanced way, and ultimately rely to a large degree on the judgment of talented and experienced management. There is no effective algorithmic approach. A key role of management is to ensure that supportive infrastructure and processes enable optimum use of the S&E assets in executing the committed projects and programs. At a fundamental level this certainly must include minimizing the time S&E staff must spend on less productive, less value- producing, or even interruptive activity. Metrics and indicators are key to achieving this goal, as are consequent ongoing attention to reduce non-productive expenditure of time. Clearly some portion of the S&E researcher time must be diverted from research to administrative activity, proposal preparation, reporting, training in security and safety, travel, and the like. A BEST-PRACTICE MANAGEMENT CYCLE In the experience of the committee, the most effective systems for managing laboratories are built around a closed-loop cycle involving planning and allocation, execution, and retrospective assessment, all driven by the I/V returned model. The cyclic processes are clearly linked to a strategic backdrop tied to the institutional mission. We describe one such cyclic set of processes in this section. This cycle and its processes do not deal with the core project management, which may drive the majority of institutional resource, but deal instead with the underlying and supporting S&E which is essential to success in many deeply technical enterprises. The period of this cyclic process would typically be annual. A key backdrop and underpinning for this process is an annually updated Outlook, or world view. This Outlook accumulates knowledge and projections from numerous sources to provide a detailed view of the environment in which the institution must achieve its mission, and is sufficiently detailed that it can be used as an environment in which potential scenarios can be modeled. It includes information on, and potential impacts of, the relevant drivers of change—technical, economic, potentially political and 71

OCR for page 70
social—competitive factors, customer and marketplace drivers, etc. It captures the essence of “Here is the world we see in front of us in which we must succeed in our mission.” And it forms, in various versions, the base for much of the communication among the parties involved: the investors and beneficiaries of the S&E work, the S&E management and technical staff, partners, customers, and others involved in the enterprise. By its exposure to the relevant parties, and open discussion and debate about the assumptions built into the Outlook, inputs for change are continuously provided. A reasonable point to define as the beginning of the (annual) cycle is a fairy formal update of the Outlook, aimed at incorporating what has been learned in the past year, as well as identifying and dealing with newly emerging trends and developments in all the key driver areas. With an updated Outlook having achieved some level of consensus acceptance, the Strategy and Planning work begins. This will update a durable long-term set of goals and approaches to their achievement, and lay out of a set of investment options for the S&E investment. This investment allocation process is clearly the most difficult part of the cycle, since the options inevitably cannot all be committed within the available resources. As is clearly the case for the NNSA labs, and also experienced in the industry sector, the resources for S&E must be allocated among three fairly distinct types of activities. First, the bulk of the S&E must support the mission and core project goals, and thus the prioritization of this portion is driven primarily top-down from the mission, but with significant input from and debate with the S&E management and staff, both on the “what” and the “how.” A second, and essential, portion of the resource—in NNSA’s case, mostly represented by the LDRD resources—is primarily allocated by S&E management, laboratory directors and their colleagues, with major bottom-up input. It is this portion of the resource which ensures the health of the labs; it aims to drive major breakthroughs and develop and exploit new areas of knowledge which are not on the direct and obvious path of the mission projects. The third source and use of resource, common in industry as well and in the NNSA labs, is external funding, in the best case driven by the outside party recognizing the potential of the S&T team to create major value for the outside investor. This is not simply “works for hire” but, with proper management, represents exploitation of unique resources and programs which are synergistic with the core S&E mission and creates value for both the external investor and the S&E organization. While one might aim for this portion of the portfolio to have stability of the same sort as the “base” programs, it will almost inevitably have more dynamics through the cycle than the others. A key piece of this planning portion of the cycle is the allocation of resources for both equipment and facilities, driven by the strategic plan. The human resource aspects of the plan are also obviously key. The output of this portion of the cycle is an updated strategy and operating plan for the period. Best-practice organizations have, as do the NNSA labs, multiple mechanisms of both internal and external review and advice on execution. These are best structured into a coordinated rhythm and calendar, with periodicities appropriate to the particular type of review. An obvious risk is excessive and intrusive reviewing and auditing, which distracts energy from the work at hand. Appropriate dashboards, metrics and indicators are useful during this process, although they do not substitute for deep dives into issues which are surfaced by the dashboards. Closure in the cycle is typically represented by a series of higher level reviews focused on what has been accomplished relative to committed plans, other achievements, goals missed and causes, and some quantitative metrics constituting a balanced scorecard used for overall assessment of effectiveness. In addition to overall S&E assessment, a valuable output of this end-of-cycle summary is a set of lessons- learned, and inputs for the subsequent cycle. There is evidence from industry that driving management and culture from this I/V framework and sustained use of such a process cycle can have major positive effects in many areas. These include sustaining support for the S&E enterprise, and building a coherent internal culture which effectively balances, for the scientist or engineer, the value system of his or her technical area with the value system uniquely appropriate to the institution in which his or her S&E in embedded. 72