Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 84
Appendix 6: The Investment/Value Returned Framework for
Management of S&E
An effective management system for science and engineering (S&E) has four main
responsibilities: deciding what to do and allocating the investment; supporting, monitoring,
and facilitating the execution of the chosen portfolio; evaluation, and communication, of the
results; and maintaining and growing capabilities, both human and technical facilities. Any
comprehensive assessment of a particular management system must include both a structural
portion—analyzing and assessing the systems, processes, and relationships that are in
place—and an implementation portion, assessing the way in which the management system
actually operates and is used.
A general approach to such an assessment is the Investment/Value Returned (I/V)
framework, which has been used with good effect in industrial research settings. The core
principle of the I/V framework is that S&E management should be driving toward an
optimum value return on the R&D investment that is being supported. Creation of such an
I/V framework provides a consistent set of intellectual underpinnings across a wide and
diverse set of activities, and can serve as a key element of the core culture of an institution.
The framework provides coherent and relatively explicit linkage between the enterprise’s
mission to the specific results aimed for at the working level, and upward from the individual
scientist and engineer back to that overarching mission.
An I/V approach to the effectiveness with which management is operating involves at
least the following four management challenges:
Management must have a clear view of the nature of the returns – the value
received – from investing in S&E. How do these values support the missions?
Into what categories does the value fit? Inevitably there will be multiple “buckets”
of value. What metrics and indicators are available and used to assess value
returned, both retrospective and prospective? It is important to recognize that
management must play the key role in working with the key investors to develop
a shared understanding of the returns, their value, as well as appropriate metrics
and indicators to assess value. Left to their own to decide, the investors will often
fail to recognize how much value can be, or is being, delivered, and will thus sub-
optimize their investment, for example by putting too little value on areas with
high value in the longer term.
On a prospective basis, how does management allocate investment, both to basic
S&E versus the other elements, and within the S&E portfolio, aimed at
maximizing the value created? For S&E it is clear that an optimum allocation
methodology will involve both top-down and bottom-up approaches, vigorous
debate, and ultimately a set of decisions. This discussion is expanded below in
the section on “Best Practice Processes”.
How does management ensure that infrastructure and operational processes are in
place relative to the S&E workforce, aimed at maximizing their ability and their
motivation to create and deliver value? What metrics and indicators are used in
this area? Is the set of tools and processes in place to track on an ongoing basis
how much and how well value is being created and delivered by the S&E
84
PREPUBLICATION COPY—SUBJECT TO FURTHER EDITORIAL CORRECTION
OCR for page 85
investment over the long term (for basic research) and in the nearer term (for
applied research), and what inhibitors exist ?
Are closed-loop quality processes in place to continuously improve the output, to
ensure that technical capabilities are sustained and grown, driving change in each
step from portfolio selection to operational processes to infrastructure investment?
A key element of this aspect of management is the set of processes which ensure
that the highest level of talent is recruited to the institution, nurtured, developed,
and retained.
Evaluating the effectiveness of S&E management against these four management
challenges provides a far more systematic picture than does the all-too-common collection of
anecdotes and qualitative impressions.
For laboratories such as the NNSA National Security labs, the success of which is
strongly dependent on S&E and which do not have a profit motive, the set of values returned
from the S&E investment includes at least the following:
Essential support for achieving the labs’ core missions, through core work that
exploits deep aspects of physics, materials science, engineering and
computational science. Without a strong internal base in these areas of S&E the
core programs of the labs would not be viable.
Highly talented and motivated staff, both in S&E itself and across the
organization, to provide a basis for long-term success in addressing a changing
portfolio of work.
Deep contact with, and the ability to draw on and contribute to (and to some
degree shape), the agendas of the broad world of science, which can, in turn, be
mined for the laboratories’ own uses.
Targeted basic research to fill gaps not pursued in the open community.
The process of investment allocation—both strategic and cyclical, and both human
and fiscal—is clearly one of management’s major challenges, and has a huge impact on what
value will be returned. This process is almost always seriously constrained by resource
limitations, and demands that choices be made among a set of investment proposals each
promising high value return. The investment allocation process, particularly for S&E, must
effectively balance broad top-down objective setting with proposals driven by the instincts
and intuitions of the S&E staff, both individuals and small groups. These processes must aim
for value creation in a balanced way, and ultimately rely to a large degree on the judgment of
talented and experienced management. There is no effective algorithmic approach.
A key role of management is to ensure that supportive infrastructure and processes
enable optimum use of the S&E assets in executing the committed projects and programs. At
a fundamental level this certainly must include minimizing the time S&E staff must spend on
less productive, less value-producing, or even interruptive activity. Metrics and indicators are
key to achieving this goal, as are consequent ongoing attention to reduce non-productive
expenditure of time. Clearly some portion of the S&E researcher time must be diverted from
research to administrative activity, proposal preparation, reporting, training in security and
safety, travel, and the like.
85
PREPUBLICATION COPY—SUBJECT TO FURTHER EDITORIAL CORRECTION
OCR for page 86
A Best-Practice Management Cycle
In the experience of the committee, the most effective systems for managing
laboratories are built around a closed-loop cycle involving planning and allocation,
execution, and retrospective assessment, all driven by the I/V returned model. The cyclic
processes are clearly linked to a strategic backdrop tied to the institutional mission. We
describe one such cyclic set of processes in this section. This cycle and its processes do not
deal with the core project management, which may drive the majority of institutional
resource, but deal instead with the underlying and supporting S&E which is essential to
success in many deeply technical enterprises. The period of this cyclic process would
typically be annual.
A key backdrop and underpinning for this process is an annually updated Outlook, or
world view. This Outlook accumulates knowledge and projections from numerous sources to
provide a detailed view of the environment in which the institution must achieve its mission,
and is sufficiently detailed that it can be used as an environment in which potential scenarios
can be modeled. It includes information on, and potential impacts of, the relevant drivers of
change—technical, economic, potentially political and social—competitive factors, customer
and marketplace drivers, etc. It captures the essence of “Here is the world we see in front of
us in which we must succeed in our mission”. And it forms, in various versions, the base for
much of the communication among the parties involved: the investors and beneficiaries of
the S&E work, the S&E management and technical staff, partners, customers, and others
involved in the enterprise. By its exposure to the relevant parties, and open discussion and
debate about the assumptions built into the Outlook, inputs for change are continuously
provided. A reasonable point to define as the beginning of the (annual) cycle is a fairy formal
update of the Outlook, aimed at incorporating what has been learned in the past year, as well
as identifying and dealing with newly emerging trends and developments in all the key driver
areas.
With an updated Outlook having achieved some level of consensus acceptance, the
Strategy and Planning work begins. This will update a durable long term set of goals and
approaches to their achievement, and lay out of a set of investment options for the S&E
investment. This investment allocation process is clearly the most difficult part of the cycle,
since the options inevitably cannot all be committed within the available resources. As is
clearly the case for the NNSA labs, and also experienced in the industry sector, the resources
for S&E must be allocated among three fairly distinct types of activities. First, the bulk of the
S&E must support the mission and core project goals, and thus the prioritization of this
portion is driven primarily top-down from the mission, but with significant input from and
debate with the S&E management and staff, both on the “what” and the “how”. A second,
and essential, portion of the resource—in NNSA’s case, mostly represented by the LDRD
resources—is primarily allocated by S&E management, lab directors and their colleagues,
with major bottom-up input. It is this portion of the resource which ensures the health of the
labs; it aims to drive major breakthroughs and develop and exploit new areas of knowledge
which are not on the direct and obvious path of the mission projects. The third source and use
of resource, common in industry as well and in the NNSA labs, is external funding, in the
best case driven by the outside party recognizing the potential of the S&T team to create
major value for the outside investor. This is not simply “works for hire” but, with proper
86
PREPUBLICATION COPY—SUBJECT TO FURTHER EDITORIAL CORRECTION
OCR for page 87
management, represents exploitation of unique resources and programs which are synergistic
with the core S&E mission and creates value for both the external investor and the S&E
organization. While one might aim for this portion of the portfolio to have stability of the
same sort as the “base” programs, it will almost inevitably have more dynamics through the
cycle than the others.
A key piece of this planning portion of the cycle is the allocation of resources for both
equipment and facilities, driven by the strategic plan. The human resource aspects of the plan
are also obviously key.
The output of this portion of the cycle is an updated strategy and operating plan for
the period. Best-practice organizations have, as do the NNSA labs, multiple mechanisms of
both internal and external review and advice on execution. These are best structured into a
coordinated rhythm and calendar, with periodicities appropriate to the particular type of
review. An obvious risk is excessive and intrusive reviewing and auditing, which distracts
energy from the work at hand. Appropriate dashboards, metrics and indicators are useful
during this process, although they do not substitute for deep dives into issues which are
surfaced by the dashboards.
Closure in the cycle is typically represented by a series of higher level reviews
focused on what has been accomplished relative to committed plans, other achievements,
goals missed and causes, and some quantitative metrics constituting a balanced scorecard
used for overall assessment of effectiveness. In addition to overall S&E assessment, a
valuable output of this end-of-cycle summary is a set of lessons-learned, and inputs for the
subsequent cycle.
There is evidence from industry that driving management and culture from this I/V
framework and sustained use of such a process cycle can have major positive effects in many
areas. These include sustaining support for the S&E enterprise, and building a coherent
internal culture which effectively balances, for the scientist or engineer, the value system of
his or her technical area with the value system uniquely appropriate to the institution in
which his or her S&E in embedded.
87
PREPUBLICATION COPY—SUBJECT TO FURTHER EDITORIAL CORRECTION
