National Academies Press: OpenBook

Determining Core Capabilities in Chemical and Biological Defense Science and Technology (2012)

Chapter: 5 Management of Science and Technology at CBDP

« Previous: 4 Proposed Approach
Suggested Citation:"5 Management of Science and Technology at CBDP." National Research Council. 2012. Determining Core Capabilities in Chemical and Biological Defense Science and Technology. Washington, DC: The National Academies Press. doi: 10.17226/13516.
×

5


Management of Science and Technology at CBDP

Chapter 4 describes a process for capabilities-based strategic planning, but such a shift ought not be implemented without considering programmatic changes that could support its adoption and successful implementation. This chapter highlights two possible areas, the relationship between the end user and the RDT&E performers, and technology transition within the CBDP, where changes could have a significant effect on improving programmatic efficiency, especially if a capabilities-based planning approach were developed.

In addition, since one of the objectives for this study is to provide input that will assist the DASD(CBD) in “identifying the current model for sustaining lab infrastructure within DoD and determining if this is the most effective method for maintaining DoD critical infrastructure,” this chapter also presents observations regarding laboratory management within the CBDP.

As emphasized in Chapter 3 and Recommendation 3.1, a comprehensive, effective RDT&E program must support a Culture of Collaboration. Such a culture encourages individuals with expertise across disciplines to share information and effort in order to achieve their goals and enable programs to effectively manage resources, minimize duplication of effort, and identify opportunities for sharing information and facilities. The sections in this chapter largely relate to aspects of the CBDP that can influence the culture of collaboration at the research and program management levels.

Suggested Citation:"5 Management of Science and Technology at CBDP." National Research Council. 2012. Determining Core Capabilities in Chemical and Biological Defense Science and Technology. Washington, DC: The National Academies Press. doi: 10.17226/13516.
×

MAINTAINING A CONNECTION TO THE END USER

The goal of the CBDP is to put effective tools in the hands of end users to minimize risk from chemical and biological threats. As described in Chapters 2 and 4, there are four identified missions for the CBDP, and the end users may differ between them. The warfighters and those who support them are critical, but, for example, civil defense authorities and personnel may need to be considered in some cases. In the current programmatic structure, CBD priorities are identified by those with operational knowledge, and products to address those priorities are developed by those with science and engineering expertise. Between these two levels, the S&T efforts required to develop the identified tools are recognized and broken into projects, which are then pursued by individuals or small teams, and the connection to the operational context is often lost until late-stage T&E. In the committee’s view, this distance between the R&D performers and the end users represents a lost opportunity to allow for multidisciplinary (and multiperspective) consideration of the challenges of CBD in a field or combat situation. In discussion with the committee, R&D performers who had contact with the warfighters directly, whether as a result of collaboration on specific projects (often noted in connection with USSOCOM) or in the pursuit of operationally relevant data in support of development of models, noted great value from interactions with operators. Other individuals in the basic research realm noted that understanding more fully the context in which their work would be used could assist in the development of operationally relevant projects and help identify overlooked variables or factors in their work. Strengthening the relationship between the warfighter and the R&D performers could support the development of specific capabilities by creating opportunities for innovation by allowing for informed “blue sky thinking” between the two groups.

The committee noted that laboratory personnel from some facilities have relationships with warfighters as a strong component of their research and development programs. We suggest that the CBDP survey the facilities to identify where positive relationships exist, between Special Forces or the Services broadly, and seek to replicate such interactions. At the program management level, it may be difficult to encourage the strengthening of such relationships unless and until a capabilities-based approach is adopted as it may be challenging to see how providing R&D performers with greater operational context can be relevant to meeting specific requirements.

Suggested Citation:"5 Management of Science and Technology at CBDP." National Research Council. 2012. Determining Core Capabilities in Chemical and Biological Defense Science and Technology. Washington, DC: The National Academies Press. doi: 10.17226/13516.
×

TECHNOLOGY TRANSITION WITHIN CBDP

When a project progresses from the initial research and development work performed in support of product development to the test and evaluation and approval and acquisition of the material, management shifts from JSTO-CBD, under DTRA, to JPEO-CBD. Though both offices are overseen by the CBDP, there is no one office or individual who oversees the entire process for any given product. The discussion below highlights some of the challenges that this structure can pose for efficient development and presents some possible alternative options for consideration by DASD(CBD). These changes could help to improve the current processes, should a requirements-driven process be maintained, or support a revised, capabilities-driven process, as discussed in Chapter 4.

The development of a final product is often iterative in nature, with failures in T&E identifying weaknesses that can only be resolved with additional R&D. At such times, the project management shifts from JPEO-CBD, which manages the T&E process, back to JSTO-CBD for additional R&D before returning to JPEO-CBD for the next round of testing and, potentially, approval. Successful transition between these offices requires a shared understanding of appropriate transition points. If an agreement is not in place, there is a risk that a gap between the perceived roles could result in the development of a project management “valley of death” and reduced program efficiency. While such issues can potentially be resolved through good working relationships between the offices, reliance on that relationship for success is not recommended as it can be strongly influenced by personality rather than policy. To address this concern, the committee recommends that DASD(CBD) consider alternative program management methods, including incorporation of an end-to-end project management authority to be maintained by an individual with scientific and technical expertise, as opposed to an acquisition specialist. In addition, liaisons such as those used between the management offices and the COCOMs, or secondments between offices and laboratories could support the development of both formal and informal relationships between program personnel and facilitate the development of trust and understanding between the various parties and offices.

One particularly stark example of the importance of an end-to-end project management authority, or at least consensus on the appropriate transition point, is in the development of medical countermeasures (see Box 5.1 highlighting some failures of medical products where poor transition between development stages was a contributing factor). The root of the difficulty is that development and acquisition of these items must contend with regulatory requirements—e.g., good laboratory and manufacturing processes, clinical trials, and acquisition of safety and efficacy data—and these requirements must be met in order to obtain FDA

Suggested Citation:"5 Management of Science and Technology at CBDP." National Research Council. 2012. Determining Core Capabilities in Chemical and Biological Defense Science and Technology. Washington, DC: The National Academies Press. doi: 10.17226/13516.
×

BOX 5.1
Examples of How Medical Product
Development and Fielding Has Failed

The transition from basic research to advanced development for medical CB countermeasures has been difficult and slow. The traditional approach in DoD’s medical laboratories has been to do the initial research, identify and characterize candidate products, perform pre-clinical testing, and prepare some sections of an Investigational New Drug (IND) filing. The subsequent steps that are necessary to move such products through the rest of the acquisition process and the FDA regulatory process have been stymied for a variety of reasons and can be typified by these examples:

  1. rPA vaccine: DoD conducted necessary S&T to identify and characterize Protective Antigen (PA) as the primary antigen for a new anthrax vaccine. Using recombinant DNA technology, GMP rPA was produced in pilot laboratories, safety and efficacy was shown in appropriate animal models (up to and including non-human primates) and the IND application was filed with the FDA. Unfortunantly, no practical expertise in obtaining scale-up development capabilities was resident in DoD; HHS took responsibility for production of large-scale lots of vaccine, clinical trials, and bringing the candidate though the FDA process. Technical issues related to the product stability, lot-to-lot variability, and other issues could not be resolved by DoD or HHS. rPA has yet to become a licensed product.
  2. RSDL (Reactive Skin Decontamination Lotion): The extraordinarily long process to achieve FDA approval of this product can be attributed to the fact that DoD laboratories did not have sufficient expertise to address some of the regulatory issues related to product safety, incompatibility, and/or efficacy. Collaboration with experts from the drug development industry could have addressed some of these issues. RSDL was cleared by FDA for military use in 2003.

approval of the final product. As these requirements necessarily influence product development pathways at an early stage, the current management structure within the CBDP is not well suited to the task because of the lack of a whole-process, integrated view of product development. In addition to challenges presented by the existing program management structure, the medical production process has not traditionally been a major focus of DoD, and as a result, the in-house expertise at the decision-making process is likely to be limited.

Suggested Citation:"5 Management of Science and Technology at CBDP." National Research Council. 2012. Determining Core Capabilities in Chemical and Biological Defense Science and Technology. Washington, DC: The National Academies Press. doi: 10.17226/13516.
×

Medical Product Development within the CBDP

Development of therapeutics under normal operating conditions within the pharmaceutical industry is a long and expensive process (commonly viewed as requiring “twelve years and one billion dollars”). Being successful necessitates extensive management and quality control features. The complexity is such that a team management approach, drawing upon internal expertise in the early development phase through to the later production phase, is a common approach for incorporating continuity through the entire process. Development of therapeutics in the context of CB agent exposure adds complexity to the process:

  • Documentation of pathophysiology of the disease in humans is minimal for most of the targets, and the efficacy of the medical products cannot be tested in humans against the relevant pathogens. Thus, extrapolation from efficacy studies in animals are required to predict efficacy in humans;
  • Animal models are difficult and not well defined for many of the pathogens;
  • The regulatory process for developing therapeutics, particularly vaccines, is not a well-delineated process in particular with relationship to the requirements for correlates of protection;
  • The acceptable risk for the therapeutic efficacy range appears to be set at a very high threshold, well above what is typically acceptable under other military operations.

The reason for listing some of these complexities is to demonstrate that production of vaccines and therapeutics in the DoD CBD context is extraordinarily challenging, and careful planning and project management is required to be successful. Over the course of the committee’s data-gathering, it was apparent that the current process for development of medical products has led to few, if any, new therapeutics. Discussions with CBDP staff indicated that the following factors are contributing to the low success rate:

  • Minimal communication between the JSTO-CBD and JPEO-CBD managers to facilitate product development;
  • An apparent expertise gap in the management offices, particularly regarding identification and handling of critical transition points in product development;
  • Turnover in personnel at several levels without a committed legacy of ongoing product development makes for multiple “starts and stops” during the process;
Suggested Citation:"5 Management of Science and Technology at CBDP." National Research Council. 2012. Determining Core Capabilities in Chemical and Biological Defense Science and Technology. Washington, DC: The National Academies Press. doi: 10.17226/13516.
×
  • Lack of a transparent process for determining long-range critical therapeutic or vaccine targets; and
  • No consistent methodology for selection of partner laboratories or companies with expertise in therapeutic development.

Note that these factors were not identified as part of a formal review of the program, but through formal and informal conversations. The committee believes that a formal, preferably external, review of the process would be valuable before taking action to modify the current process.

Establishing “Common Language” for Transition of Medical Countermeasures

One possible model for managing medical product development transitions is demonstrated by the integrated portfolio for CBRN countermeasure development through BARDA. BARDA’s role is to develop medical countermeasures for the Strategic National Stockpile, and as a result, it shares many of the challenges DoD faces in its therapeutic development process. Though housed within HHS, BARDA represents a collaborative effort between multiple agencies, including FDA, DoD, and DHS.

One of BARDA’s policies is to not accept potential products into the program until the work has progressed through an FDA-approved Phase I safety study. Using this structure to define the benchmark allows for clear communication of expectations across various agencies and companies. Using an FDA regulatory step as the transition point from JSTO-CBD to JPEO-CBD—or to or from another DoD/USG development partner—could provide a similar “common language” determining the transition point and establishing expectations within the CBDP. Such an objective transition point may improve project and personal efficiencies by allowing for

  • more accurate out-year staff and budgetary planning, which would hold value not only for the project scientists and managers, but also for higher-level commanders and directors for portfolio and/or budget planning and defense;
  • reduced burden of negotiation meetings to establish transition points for ongoing projects;
  • clearer determination of the responsibilities and expectations of JSTO-CBD and JPEO-CBD; and
  • easier to manage timelines for expected returns on R&D investments.
Suggested Citation:"5 Management of Science and Technology at CBDP." National Research Council. 2012. Determining Core Capabilities in Chemical and Biological Defense Science and Technology. Washington, DC: The National Academies Press. doi: 10.17226/13516.
×

BOX 5.2
Defining a Transition “Trigger”

DoD is not the only agency or organization that faces the challenges described herein. Others, such as pharmaceutical companies in the commercial sector and BARDA within the US government also must manage transitions effectively. There are three commonly used and referenced transition points early in a product’s development that are seen as logical “triggers” for the transition from basic research to development of a product. They are (1) submission of an investigational new drug (IND) and (2) completion of the Phase I trial of a material. a As one example of how these triggers could be incorporated into the CBDP, the Material Development Decision (MDD), which currently resides with JPEO-CBD, could become trigger for initiation and development of an IND application for submission to FDA. Practically, this would mean all discovery and preclinical activity would reside under JSTO-CBD management, and successful programs would be presented to JPEO-CBD for a MDD. MDD approval would trigger construction and submission of the necessary applications to enter an FDA-approved regulatory approval path. Alternatively, transition to JPEO-CBD management could occur after completion of a Phase I trial.

Under a third alternative (3), as part of a broader strategy within the CBDP for FDA-regulated products to more efficiently use available advanced development funds, successful programs could be “parked” after construction of the IND or after Phase I trial completion. This could be especially useful when the program has multiple potential products in any given area of need. Regardless of the chosen trigger, expertise, within or contracted by JSTO-CBD and JPEO-CBD, needs to be appropriately positioned. This approach would also be supportive of overlap in JSTO-CBD and JPEO-CBD personnel engagement on the project to ensure smooth and knowledgeable transitions.

_______________________

a Selection of either an IND submission or Phase I completion in the common JSTO-JPEO transition point would generally align with DoD 5000 service recognition milestones A (“first in human”) or B (“point of concept”), respectively. It would also generally align with transitions from 6.2 to 6.3 or 6.3 to 6.4 funding.

An additional potential benefit would be the inclusion of FDA scientists, such as those within the Medical Countermeasures Initiative (MCMI) program. Clearly, the earliest possible settings of expectations from the FDA—even from an advisory position—on the types of data sets that may be required to reasonably complete a Phase I trial would be helpful in managing the overall process. See Box 5.2 for a discussion of possible transition points for medical product development.

Suggested Citation:"5 Management of Science and Technology at CBDP." National Research Council. 2012. Determining Core Capabilities in Chemical and Biological Defense Science and Technology. Washington, DC: The National Academies Press. doi: 10.17226/13516.
×

LABORATORY MANAGEMENT

As was described in Chapter 1, the RDT&E elements within the CBDP draw upon and direct resources within a number of different laboratories and facilities. These include service laboratories, medical laboratories and facilities, and test and evaluation facilities. Each laboratory and facility has its own mission and management structure, and this has implications for the ability of the CBDP to successfully manage its projects and programs (see Box 5.3). This section presents a brief overview of the history of the management of laboratory research at medical and non-medical facilities, identifies elements of laboratory management the committee feels are critical for success, and presents possible alternative methods for

BOX 5.3
The Role of Medical Laboratories in Chemical
and Biological Defense Research

In the 1980s and 1990s, the funding at medical facilities for chemical and biological research was provided by DTRA directly to the commanders for their allocation and distribution. At that time, the laboratories performed basic science in support of understanding the medical response to exposure to chemical or biological agents. The research was not focused on the development of specific products, such as vaccines or countermeasures. Beginning in the early 1990s, DTRA began directing these R&D funds through MRMC (US Army Medical Research and Materiel Command), giving MRMC’s command structure both responsibility and authority for the science within their major laboratories. Under this new model, funding was allocated by program and, as a result, the laboratories became focused on the development of specific product targets, e.g., an anticonvulsant or a plague vaccine, though still on the basic science in support of that target. Advanced development leading to an FDA-licensed product was not within the laboratories’ purview.

When the JPO was formed, the role of the laboratories changed again. Now medical laboratories, specifically USAMRIID and USAMRICD, were asked to take candidate countermeasures beyond the basic science toward the development of a licensed product. This required the laboratories to produce pilot-scale lots of GMP (good manufacturing practices) candidate product and to conduct the GLP (good laboratory practices) pre-clinical studies in preparation for the initial submission to FDA for approval.

This continued until the formation of JSTO-CBD in 2003. Though the focus on specific product development continued after that point, investigators within the laboratories were asked to respond to specific requests for proposals in a competitive environment, rather than the laboratories pursuing a program (e.g., development of an anthrax vaccine) with guaranteed funding. With this change, the responsibility for performance remained with the laboratories and the authority to manage the programs resided at JSTO-CBD in Ft. Belvoir.

Suggested Citation:"5 Management of Science and Technology at CBDP." National Research Council. 2012. Determining Core Capabilities in Chemical and Biological Defense Science and Technology. Washington, DC: The National Academies Press. doi: 10.17226/13516.
×

laboratory management that could be considered. Overall facility management, which includes infrastructure, workforce, and research program management within the facility, plays a critical role in the stewardship necessary to support a successful S&T related endeavor. This stewardship also extends to funding and directing the scientific work of the program.

Successful Laboratory Management

The CBDP relies upon a laboratory network and test ranges to provide RDT&E and produce products critical to the chemical and biological defense of the nation. The network consists of DoD owned and operated laboratories (e.g., USAMRIID, ECBC, Air Force Research Laboratory (AFRL), NSRDEC, and Naval Surface Warfare Center (NSWC) Dahlgren), DOE national laboratories (e.g., LANL, SNL, LLNL, Pacific Northwestern National Laboratory (PNNL)), FFRDCs (Federally Funded Research and Development Centers), non-profit entities (e.g., Battelle), for-profit commercial laboratories, and universities. At many of the facilities the committee visited, the recent construction of new buildings will provide new capacity for RDT&E, adding both additional space and additional technical functionality.

Successful RDT&E leading to fielded systems1 is a long (perhaps a decade or more), arduous process. In the committee’s view, a successful RDT&E program requires at least the following six elements to ensure clarity of purpose, focus of investments, and coherence of management:

  1. Clear mission and objectives
  2. Continuity in leadership
  3. The ability to understand, accept, and manage risk throughout the process
  4. Predictable and stable funding
  5. Effective asset management at the laboratory level
  6. A sense of excitement and pride in the work among the staff

In the following sections, these elements will be defined and subsequently discussed in the context of the CBDP.

_______________________

1 Fielded systems are not exclusively hardware or medical countermeasures. For example, they may include an operational diagnostic and analysis systems including collection, transportation, analysis, and identification including all the required infrastructure, protocols, routinely exercised.

Suggested Citation:"5 Management of Science and Technology at CBDP." National Research Council. 2012. Determining Core Capabilities in Chemical and Biological Defense Science and Technology. Washington, DC: The National Academies Press. doi: 10.17226/13516.
×

Element 1: Clear Mission and Objectives

For an effective program, all participants in the endeavor need to understand its mission and objectives, how they contribute to achieving those goals, and why their work is important in that context. A laboratory cannot be successful if it proclaims it is undertaking all missions for the entire government. “Reinvention” may be required as the world changes, but this should only happen rarely. Without a clear, defined mission and objective, so-called “mission confusion” may lead to expansion into activities that do not directly support the objectives or that are duplicative of efforts in other laboratories. Lessons from successful laboratories (government and non-government) demonstrate that a shared mission understanding leads to long-term success.

Element 2: Continuity in Leadership

Strong leadership, and continuity of the stated mission and objectives during and after changes in leadership, supports the development of sense of mission within the workforce. In contrast, rapid changes in leadership and/or weak leadership can contribute to the mission confusion described above. Of course, sometimes rapid changes in facility directors or commanders is unavoidable, but in such cases, care should be taken to ensure that succession planning, pre-training, orientation of incoming personnel, and personnel continuity in senior positions is encouraged. These steps will help maintain a consistent vision and understanding of the facility’s mission and objectives, which is important for pursuit of programmatic and project-related goals. An additional benefit is that when individuals are in a position for a significant period of time, e.g., longer than two years, they have a greater ability to develop relationships and partnerships within the research and end-user communities than might otherwise be possible. This is beneficial for the identification of opportunities for collaboration.

Element 3: The Ability to Understand, Accept, and Manage Risk

RDT&E is a risky process, and a successful research and development program may have many false starts before achieving success. Thus, an essential element of managing such programs is accepting and balancing risks. There may be times when it is appropriate to undertake high-risk, high-payoff projects which may ultimately fail in addition to maintaining long-term focus on a specific area with low risk in order to provide fundamental understandings that support new developments. An effective program will create a balanced portfolio of these various research types. Similar to the process described in Chapter 4, in the section “From

Suggested Citation:"5 Management of Science and Technology at CBDP." National Research Council. 2012. Determining Core Capabilities in Chemical and Biological Defense Science and Technology. Washington, DC: The National Academies Press. doi: 10.17226/13516.
×

Specification to the Research-Development-Acquisition Process,” external review can help down-select projects to improve overall chances for programmatic innovation and success with reduced overall costs.

Care should also be taken to not introduce unnecessary risk of project failure through poor planning and management. Understanding the eventual goal of a project or program and identifying appropriate milestones that must be completed or addressed for success can allow for corrections of approach and provide confidence at critical junctures that all required elements are in place to minimize the chances of failure due to bureaucracy.

A necessary adjunct to a well-balanced risk approach is a method for continual assessment of program and/or project progress. Internal, and especially external, standing technical review committees (supplemented by those with operational knowledge) are required to ensure that unsuccessful programs are terminated, to provide technical review for high-risk/high-payoff projects, and to encourage consideration of programs that may not have originated within the facility. The committee cautions that each institution might appoint separate review boards with different membership; while a diversity of opinions is good, having numerous, separate groups could prevent identifying redundancy and duplications. It is essential that reviewers are able to consider the context and larger picture, and maintain continuity.2

Element 4: Predictable and Stable Funding

R&D takes time. Continual disruptions due to major funding shifts and delays lead at best to inefficiencies and, at worst, failure. Funding can roughly be considered at three levels (project, program, and laboratory):

  • Project. Projects are narrowly scoped activities with specific, well-defined goals. Projects may only require a few years for completion, but this timeframe places a high premium on efficiency. Gaps in funding can hinder that efficiency, and an uninterrupted funding stream for the duration of the project or at least between major project milestones/decision points is preferred.
  • Program. Programs have a broader scope than projects with desired outcomes that require multiple steps or inputs for completion. These can last up to a decade, and to be successful, funding needs to be assured (to the maximum extent possible) for that

_______________________

2 As one possibility, there are several existing government groups with non-governmental life scientists and others who hold clearances at sufficiently high levels and could be utilized, with augmentation as needed.

Suggested Citation:"5 Management of Science and Technology at CBDP." National Research Council. 2012. Determining Core Capabilities in Chemical and Biological Defense Science and Technology. Washington, DC: The National Academies Press. doi: 10.17226/13516.
×
  • duration. Some adjustments in priorities are inevitable, but major directions must be consistent and supported.

  • Laboratory. Laboratories exist for multiple decades and require predictable funding to ensure a high-quality workforce and up-to-date research facilities and infrastructure. Laboratory directors and commanders must be assured of some level of funding if they are to effectively plan for the future and manage a workforce and research infrastructure. This should not be confused with entitlements, where funding is continued independent of performance. In order to be effective, a balance should be found with significant amount of core funding “guaranteed” and the remainder provided through a competitive environment.

Element 5: Effective Asset Management at the Laboratory Level

Some local control of funding is considered by the committee to be an important component of successful laboratory management. A local ability to move people and resources between projects can assist in creating an efficient environment as decision making can be more responsive than if external approval is required. If a laboratory director is prevented from anticipating and acting on predicted future needs, the laboratory may find it challenging to capitalize on emerging technologies or provide for new operational needs. Workforce management should be left in the hands of laboratory management (with command oversight distinguished from management), with external laboratory reviews providing input to help identify emerging needs or areas of concern (see Element 2). Similarly, programs may benefit when laboratory directors have some amount of unallocated funds (internal research and development (IR&D) and capital funds) to invest as they deem necessary (e.g., providing funds to a researcher to pursue a new line of inquiry or addressing unmet maintenance or operational needs) to ensure that the laboratory will be able to continue to meet its mission obligations. This is part of instituting a balanced risk approach to RDT&E.

Element 6: A Sense of Excitement and Pride in the Work among the Staff

Ultimately the success of an RDT&E program depends on not only the creativity and skill of the staff, but also the environment in which they work. Modern research by its very nature is a collaborative effort. Engagement with the research community, both internal and external, gives research staff access to a diversity of ideas, which may open new areas of research or help make intellectual connections between different projects (see Box 5.4). Networking between facilities and researchers

Suggested Citation:"5 Management of Science and Technology at CBDP." National Research Council. 2012. Determining Core Capabilities in Chemical and Biological Defense Science and Technology. Washington, DC: The National Academies Press. doi: 10.17226/13516.
×

BOX 5.4
Description of the Human Component of S&T Capability

Professional development of performers, program, and middle and senior management is also important in creating a positive working environment and recruiting and maintaining quality within the program. While the CBDP should seek to recruit and retain the best possible personnel for each position across the enterprise, a coherent system of professional development is also needed. Without entry-level development, new performers or entry-level program managers should be provided indoctrination as to requirements and processes associated with their positions as well as exposure to their customers and stakeholders. From the outset they should have a clear understanding of why their program exists and who it serves. As experience is obtained and acceptable performance is maintained, broadening opportunities should be provided in relevant science and with organizations and experts who are working in closely related or complementary areas (both inside and outside of DoD and the USG). Regular professional development opportunities should be provided at specified intervals, based on individual qualifications, performance and career goals and the needs, requirements and strategic goals and objectives of the CBDP and subordinate components. Opportunities to refresh in relevant S&T should be part of professional development and will benefit the individual and program. “One size fits all” will not be an appropriate or value-added approach.

can also help identify potentially underused facilities or resources that can be repurposed or exploited by others in need of additional capacity (note that this can also result in identification of opportunities of “Work for Others” funding). Collaboration also provides an opportunity for researchers to be recognized by their peers as leaders or experts within a given area.

Engagement should not be limited to interactions between researchers. It is especially important to form collaborative efforts with the end users as this can sometimes lead to solutions to unacknowledged or unknown operational needs. These connections can also provide needed field feedback to developers of materials and systems. These collaborations can also provide a reinforcement of the overall mission of the agency and purpose of the program in support of Element 1. This particular point, the relationship between the end user and the developer, will be discussed in greater detail later in this chapter.

Suggested Citation:"5 Management of Science and Technology at CBDP." National Research Council. 2012. Determining Core Capabilities in Chemical and Biological Defense Science and Technology. Washington, DC: The National Academies Press. doi: 10.17226/13516.
×

LABORATORY MANAGEMENT WITHIN CBDP

The committee visited a number of facilities and spoke with many individuals, both formally and informally, over the course of the study. What follows is a broad summation of the observations and impressions of the management of RDT&E in the CBDP in the context of the six elements outlined in the previous section. Examples are provided where possible and appropriate. The committee recognizes that its interactions with the program have been necessarily limited, and these descriptions are not drawn from formal surveys or metrics. Also see Box 5.5.

Clear Mission and Objectives

The official CBDP mission is to “provide global chemical, biological, radiological, and nuclear defense capabilities in support of National Strategies.” This is a broad statement and can be applied to a variety of activities. In conversations with the committee members, facility personnel and program-level staff were in accord with this mission. However, the committee heard different responses with regards to the specific role of given offices, facilities, and laboratories in meeting this mission. Specifically, there seemed to be a lack of common vision between JSTO-CBD and the funding recipients. The recipients expressed concern that it was unclear how the requests for proposals and the competitive process fed into the strategic vision of the laboratory and the CBDP because the proposals did not seem to build on each other to build a comprehensive picture but rather reflected the identified, internal needs or concerns of the year. For the laboratories focused on the development of specific, complex products, an unclear mission or program objectives may hinder the ability of the laboratory to meet its goals.

Continuity in Leadership

Leadership plays an important role in communicating mission and objectives, and in S&T research, this role is often performed by a technical director with a long history within the organization and/or field of research. However, at a number of facilities within the CBDP, the facility military commander (or deputy commander) now changes every two years. Such rapid changes in leadership have the potential to disrupt the ability of an organization to meet its mission and objectives. In such cases, the senior civilian leadership is familiar with the program, which provides a level of continuity, but it is probable that longer-term appointments in the military leadership positions could be helpful in maintaining continuity of programmatic focus at facilities.

Suggested Citation:"5 Management of Science and Technology at CBDP." National Research Council. 2012. Determining Core Capabilities in Chemical and Biological Defense Science and Technology. Washington, DC: The National Academies Press. doi: 10.17226/13516.
×

The Ability to Understand, Accept, and Manage Risk

The committee recognizes that understanding, accepting, and managing risk in the CBDP is a difficult task. Within the RDT&E program, the research ranges from developing a fundamental understanding of the interactions of molecules with surfaces to developing reliable vaccines and medical countermeasures to be given to warfighters in theater. This being the case, careful management of programmatic risk is essential to ensure that effort and resources are directed appropriately, and there are tools that can be used to assist in this process, e.g., linking activities to a clear mission and set of objectives (see previous section) and robust external strategic and technical review processes.

With regards to external technical review, the committee saw little evidence that peer review of S&T is occurring or encouraged. If this is indeed the case, this is a lost opportunity to draw input from knowledgeable individuals who do similar research, to build connections to those researchers with relevant knowledge, and most importantly, to ensure that promising ideas are not lost and that fundamentally flawed projects do not continue any longer than necessary to identify the flaws. One concern that was mentioned to the committee was that innovative ideas have little room to be developed in the current funding environment, which may result in lost opportunities. With regards to strategic reviews, some facilities have engaged in individual efforts to develop a strategic plan, but those that have not done so within the last decade should be encouraged to create one. In addition to facility-level reviews, a strategic review of the CBDP could identify gaps between mission and activities and allow an opportunity for correction of any such issues.

The committee also notes that poor project management can introduce a higher risk of failure to meet an objective than the science might suggest. One area where this can be seen most clearly is development of licensed medical products. Coordination between multiple offices within DoD and with the FDA are required for successful transition of a prototypical medical product to clinical trials and ultimately to licensing. A poor understanding of the complexities of this process and a lack of end-to-end authority that manages the process can introduce risks of bureaucracy causing a failure to meet the objective. Within the CBDP, the committee notes that few medical products have successfully made the transition to licensing, and though many factors lead to such a situation, a review of the structures supporting this process by the DASD(CBD) may be merited.

Suggested Citation:"5 Management of Science and Technology at CBDP." National Research Council. 2012. Determining Core Capabilities in Chemical and Biological Defense Science and Technology. Washington, DC: The National Academies Press. doi: 10.17226/13516.
×

BOX 5.5
Waging Science: Re-casting Defense Science Management

To an outsider, DoD’s management of CBD science to aid the warfighter and the homeland appears to be run as an acquisition management problem, rather than a science management one. Currently, DoD CB science does not provide an efficient and productive path from needs to solutions. Making material improvements to this situation will not be accomplished easily; however, the committee believes that DoD is uniquely positioned to re-cast defense science management into a much more effective “program” by treating it in a fundamentally different way:

  • Science is a long-term campaign, not an acquisition of beans and bullets. Whether it is developing next-generation protective gear, rapid field diagnostic assays, new vaccines, or standoff CB detectors, there are significant R&D hurdles to be faced before the acquisition phase of “off the shelf” items can be reached. Frequently, multidisciplinary teams are required to tackle problems and eventual success in science is never guaranteed.
  • Meeting science challenges requires a strategy which requires long-term planning; treat scientific challenges like planning for campaign. Treating a scientific challenge more like a mission than an inconvenient prelude to an acquisition will provide the mindset needed to focus the right team of scientists towards a single goal.
  • The scientific program should be directed toward mission goals and to the building of an ability to react to emerging or unexpected threats. These goals should be understood by all involved. Examples of clear goals might be “Create effective diagnostics for all major would-be pathogens”; “Create an effective standoff biodetector that will permit sufficient warning time to don personal protective gear.” It is important that all scientists and engineers on the mission team understand that the team mission goal is more important than any organization or personal goals. Examples of such goals might include “Author a major paper on would-be pathogens”; “Keep Laboratory X’s share of biostandoff funding at #1.”
  • Both strategies and tactics will be important. The mission needs to be planned and managed end-to-end by someone with overall mission leadership and accountability. This is different from current DoD science management where “handoffs” are supposed to occur at various Technical Readiness Levels between organizations that may not be well coordinated end to end.
  • Plan for the unknown and the unexpected; anticipate change but don’t get hung up on predictions. Long-range planning to address major defense science challenges simply cannot be effectively planned in DoD standard 5-year cycles. Some challenges will prove much harder than expected, while in other cases new technology may make other challenges far easier than initially planned. About the only certainty is that any multiyear plan involving CBD science challenges will require major revisions by the second or third year, if not sooner. The team and its leadership need to have the capability to react to major changes in plans and still achieve victory.
Suggested Citation:"5 Management of Science and Technology at CBDP." National Research Council. 2012. Determining Core Capabilities in Chemical and Biological Defense Science and Technology. Washington, DC: The National Academies Press. doi: 10.17226/13516.
×
  • Assemble the right force to meet the challenge; teaming and coordination will be paramount to achieve victory. When planning for a major military battle, the commander in charge will attempt to assemble all the different kinds of forces needed to perform the expected tasks. This might involve land, sea, and air assets, plus satellites. Similarly, the commander of a science battle may need to assemble forces from multiple DoD laboratoriess, academia, industry, and other government science resources. Regardless of which service the commander belongs to, the assembled team must be the best available to achieve the mission goal. Other possible parochial goals (“Laboratory X doesn’t have the right expertise but they need some more funding this year”; “Laboratory Y has worked in this field for several decades, but they aren’t up to speed on the latest technologies”) cannot drive team selection. It is possible that for some scientific challenges most or all of the best expertise needed might lie outside DoD laboratories.
  • Each battle campaign needs a clear overall leadership chain. Waging science is not the same as a NIH or a DARPA basic research project. You do not win a battle with a number of autonomous commanders (principal investigators) who may be working toward different personal goals. Major DoD CB science challenges are inherently Big Science projects that need to be managed appropriately. The skill of effective team formation and motivation cannot be overestimated as necessary leadership criteria when it comes to managing Big Science challenges.
  • Embrace innovation and be flexible enough to realize when it is needed. Technology in the CB domains is evolving at warp speed these days. This is especially true in biology, where recent advances in genome sequencing have increased throughput by at least three orders of magnitude in the past three plus years. The management of DoD CBD science campaigns must be capable of understanding such rapid evolution and be able to manage to make major changes in strategy at the right times. This could be achieved with the establishment of an effective central “tech watch” component within the DoD science portfolio, but each campaign leader would still have to perform their own judgment of whether or when a technology jump is optimal for reaching their mission goal. Innovation can’t be mandated, and needs to be nurtured. Campaign leaders need to have the ability to have their teams test out promising innovative ideas.
  • Logistics and infrastructure win wars; have a flexible procurement strategy to keep your troops well-equipped as conditions change. The experiences in Iraq and Afghanistan painfully illustrated the high cost in terms of life and limb when a combination of factors delayed getting more effective protection for IEDs to troops. Inefficiencies in DoD procurement also affect every CB science campaign, particularly in cases where flexibility from a set plan is needed to deal with technology evolution.
  • Set meaningful milestones and adjust them as needed based on how the battle develops. The goal is to win the battle, not follow a static plan. Improvements that allow and enable innovative flexibility are sorely needed. Paperwork and inflexible regulations appear to cause major inefficiencies in how DoD wages science. Commanders need to have ways to innovate and react to improve their chances of winning.
Suggested Citation:"5 Management of Science and Technology at CBDP." National Research Council. 2012. Determining Core Capabilities in Chemical and Biological Defense Science and Technology. Washington, DC: The National Academies Press. doi: 10.17226/13516.
×

Predictable and Stable Funding

The committee was informed that, though the funding mechanism has changed over time, the funding levels for RDT&E have remained relatively stable in recent years. In general, discussions with researchers indicated that necessary resources are available for current activities, though multiple groups noted that any significant cut in funding would result in cuts to programs.

One concern expressed many times to the committee was that the competitive process has resulted in a sense that funding is unpredictable, making long-term endeavors challenging. Once a proposal had been approved, the committee heard no concerns regarding funding levels of a specific project. However, timing of receipt of funds was discussed, and delays in receipt of funds due to contracting difficulties were cited as a barrier to developing collaborations with external organizations. At the program level, the primary concern expressed to the committee was that the links between the project-level funding and the program-level priorities were not always clear. With regards to laboratory-level funding, care should be taken to ensure that the recent construction of new buildings, some of which will place new demands on operations and maintenance budgets, does not cause the resources of laboratories and facilities to become overstretched.

Effective Asset Management at the Laboratory Level

Funding for CBD research at the laboratories is awarded through a competitive process directly to the principal investigators. Barring the funds acquired through reimbursement of services provided to external agencies and entities (up to 2.5% of IR&D), the laboratory directors have little direct control over their budgets and the allocation of those budgets to the staff and facilities. During multiple site visits, the committee heard that though people felt they had the resources required to perform their work, there was a lack of flexibility of funding that could make it difficult to be responsive to emerging opportunities or concerns. Some of these concerns could be exacerbated in the near future with the opening of multiple new facilities with new capabilities and, likely, management challenges.

Some researchers who spoke with the committee expressed some concern regarding the degree of JSTO-CBD’s involvement in directing research. While it was acknowledged that a competitive process can assist in maintaining a strong research function, concerns that the focus was too great on projects rather than programs was expressed—i.e., no long-term vision or strategy supported the specific calls for proposals. This, of course, must be balanced against the views from the program managers

Suggested Citation:"5 Management of Science and Technology at CBDP." National Research Council. 2012. Determining Core Capabilities in Chemical and Biological Defense Science and Technology. Washington, DC: The National Academies Press. doi: 10.17226/13516.
×

who stated that the proposal requests are a product of the requirements-development process and are reflective of the needs of the CBDP and the agency as a whole. The role of this committee is not to decide who is “right” in this discussion, but to observe that the relationship between JSTO-CBD and the service laboratories appears strained. The lack of local autonomy (authority) within the current funding model is seen as implying a lack of trust, whether intended or not, and this seems to affect the relationship between the two groups. Whether these issues can be resolved through increased communication about the overall funding strategy or if a major revision of the program will be required to resolve these issues will have to be determined and addressed by the DASD(CBD).

A Sense of Excitement and Pride in the Work among the Staff

In light of the discussions above, as one might expect, significant frustration with the funding mechanisms and priorities were expressed many times to the committee. However, in general, staff at the facilities expressed a pride of purpose and a sense that the work being done at the laboratories is important and relevant.

One area that was highlighted by many PIs and performers was a desire to gain a greater understanding of end-user needs and the environment and circumstances in which the materials, products, and procedures will be used. For some, the connection between the research laboratory and the field is rather abstract. Reducing the gap between the bench and the end user could be beneficial in many ways, including increasing excitement and pride in the work among the staff.

Review and Assessment

All projects benefit from scientific peer review when done well, and these reviews keep the skills of scientists and engineers sharp. Reviews also provide an important function in ensuring that scientists within dedicated institutions stay in touch with the external community. Within DoD CBD laboratories, there was a noticeable lack of connectivity to research in other institutions and an independent review process, especially in the critical research phase.

FINDINGS AND RECOMMENDATIONS

This final chapter has brought forth a series of principle findings and recommendations that can assist the program in operating a successful laboratory environment.

Suggested Citation:"5 Management of Science and Technology at CBDP." National Research Council. 2012. Determining Core Capabilities in Chemical and Biological Defense Science and Technology. Washington, DC: The National Academies Press. doi: 10.17226/13516.
×

Program Management

Finding 5.1: Successful transition between the JSTO-CBD and the JPEO-CBD offices requires a mutual agreement on appropriate transition points, encoded in multiyear program plans and budgets. Regardless of the chosen trigger, expertise and resources within or contracted by JSTO-CBD and JPEO-CBD need to be appropriately positioned. This approach would also be supportive of overlap in JSTO-CBD and JPEO-CBD personnel engagement on the project to ensure smooth and knowledgeable transitions. However, the committee observed that the partnership between the JSTO-CBD and JPEO-CBD is weak and that neither office viewed transition plans as a responsibility.

Finding 5.2: There is no end-to-end authority for the CBDP, which is particularly problematic for medical products. Though both JSTO-CBD and JPEO-CBD are overseen by the CBDP, there is no one office or individual with the responsibility and authority for the entire process for any given product. The risk—and reality—is that a transition gap between R&D and acquisition could result in the development of a project management “valley of death.” The existing research-development-acquisition process may be adequate for acquiring the non-medical products in the CBDP. For the medical countermeasures program, however, FDA regulatory requirements must be considered early enough to influence product development decisions. The current management structure within the CBDP is not well suited to the task because of the lack of a whole-process, integrated view of product development.

Recommendation 5.1: The DASD(CBD) should evaluate alternative program management approaches, including incorporation of an end-to-end project management authority, especially for the medical countermeasures program.

Laboratory and Major Facility Management

Finding 5.3: The principal RDT&E military organizations associated with the CBDP are benefiting from major facility investments that are planned to provide both capabilities and capacities to meet the anticipated needs of the program. Operating and maintaining these facilities, however, will place a burden on both the owning Service (principally the Army) and the program. The initial operating plans appear to be resourced.

Finding 5.4: All or part of the elements required for healthy RDT&E activities were missing at the organizations visited by the commit-

Suggested Citation:"5 Management of Science and Technology at CBDP." National Research Council. 2012. Determining Core Capabilities in Chemical and Biological Defense Science and Technology. Washington, DC: The National Academies Press. doi: 10.17226/13516.
×

tee. A successful RDT&E enterprise should include the following elements to ensure clarity of purpose, focus of investments, and coherence of management:

  1. Clear mission and objectives
  2. Continuity in leadership
  3. The ability to understand, accept, and manage risk throughout the process
  4. Predictable and stable funding
  5. Effective asset management at the laboratory level
  6. A sense of excitement and pride in the work among the staff

Of special concern are strained relationships between JSTO-CBD and the laboratories, the new rotational policy for military commanders in the Army, and a trend toward increasing oversight of both technical work and operations at the facilities.

Recommendation 5.2: The DASD(CBD) should formally review alternative laboratory management models, taking advantage of the numerous prior studies, reviews, and evaluations of laboratory and large facility management of S&T organizations. A principal objective is to define the level of stewardship that the program should provide to the principal RDT&E in-house facilities and laboratories.

Scientific Peer Review

Finding 5.5: All programs benefit from scientific peer review when done well, and these reviews keep the skills of scientists and engineers sharp.

Recommendation 5.3: The DASD(CBD) should implement a nested review process for chemical and biological defense RDT&E bound by consistent standards of rigor, frequency, and reporting. The CBDP and its supporting laboratories would each benefit from independent, periodic review at the programmatic and scientific levels. The CBDP should also encourage and participate in institutional reviews. An annual roll-up of review outcomes could help identify thematic areas of promise and concern.

Suggested Citation:"5 Management of Science and Technology at CBDP." National Research Council. 2012. Determining Core Capabilities in Chemical and Biological Defense Science and Technology. Washington, DC: The National Academies Press. doi: 10.17226/13516.
×

This page intentionally left blank.

Suggested Citation:"5 Management of Science and Technology at CBDP." National Research Council. 2012. Determining Core Capabilities in Chemical and Biological Defense Science and Technology. Washington, DC: The National Academies Press. doi: 10.17226/13516.
×
Page 83
Suggested Citation:"5 Management of Science and Technology at CBDP." National Research Council. 2012. Determining Core Capabilities in Chemical and Biological Defense Science and Technology. Washington, DC: The National Academies Press. doi: 10.17226/13516.
×
Page 84
Suggested Citation:"5 Management of Science and Technology at CBDP." National Research Council. 2012. Determining Core Capabilities in Chemical and Biological Defense Science and Technology. Washington, DC: The National Academies Press. doi: 10.17226/13516.
×
Page 85
Suggested Citation:"5 Management of Science and Technology at CBDP." National Research Council. 2012. Determining Core Capabilities in Chemical and Biological Defense Science and Technology. Washington, DC: The National Academies Press. doi: 10.17226/13516.
×
Page 86
Suggested Citation:"5 Management of Science and Technology at CBDP." National Research Council. 2012. Determining Core Capabilities in Chemical and Biological Defense Science and Technology. Washington, DC: The National Academies Press. doi: 10.17226/13516.
×
Page 87
Suggested Citation:"5 Management of Science and Technology at CBDP." National Research Council. 2012. Determining Core Capabilities in Chemical and Biological Defense Science and Technology. Washington, DC: The National Academies Press. doi: 10.17226/13516.
×
Page 88
Suggested Citation:"5 Management of Science and Technology at CBDP." National Research Council. 2012. Determining Core Capabilities in Chemical and Biological Defense Science and Technology. Washington, DC: The National Academies Press. doi: 10.17226/13516.
×
Page 89
Suggested Citation:"5 Management of Science and Technology at CBDP." National Research Council. 2012. Determining Core Capabilities in Chemical and Biological Defense Science and Technology. Washington, DC: The National Academies Press. doi: 10.17226/13516.
×
Page 90
Suggested Citation:"5 Management of Science and Technology at CBDP." National Research Council. 2012. Determining Core Capabilities in Chemical and Biological Defense Science and Technology. Washington, DC: The National Academies Press. doi: 10.17226/13516.
×
Page 91
Suggested Citation:"5 Management of Science and Technology at CBDP." National Research Council. 2012. Determining Core Capabilities in Chemical and Biological Defense Science and Technology. Washington, DC: The National Academies Press. doi: 10.17226/13516.
×
Page 92
Suggested Citation:"5 Management of Science and Technology at CBDP." National Research Council. 2012. Determining Core Capabilities in Chemical and Biological Defense Science and Technology. Washington, DC: The National Academies Press. doi: 10.17226/13516.
×
Page 93
Suggested Citation:"5 Management of Science and Technology at CBDP." National Research Council. 2012. Determining Core Capabilities in Chemical and Biological Defense Science and Technology. Washington, DC: The National Academies Press. doi: 10.17226/13516.
×
Page 94
Suggested Citation:"5 Management of Science and Technology at CBDP." National Research Council. 2012. Determining Core Capabilities in Chemical and Biological Defense Science and Technology. Washington, DC: The National Academies Press. doi: 10.17226/13516.
×
Page 95
Suggested Citation:"5 Management of Science and Technology at CBDP." National Research Council. 2012. Determining Core Capabilities in Chemical and Biological Defense Science and Technology. Washington, DC: The National Academies Press. doi: 10.17226/13516.
×
Page 96
Suggested Citation:"5 Management of Science and Technology at CBDP." National Research Council. 2012. Determining Core Capabilities in Chemical and Biological Defense Science and Technology. Washington, DC: The National Academies Press. doi: 10.17226/13516.
×
Page 97
Suggested Citation:"5 Management of Science and Technology at CBDP." National Research Council. 2012. Determining Core Capabilities in Chemical and Biological Defense Science and Technology. Washington, DC: The National Academies Press. doi: 10.17226/13516.
×
Page 98
Suggested Citation:"5 Management of Science and Technology at CBDP." National Research Council. 2012. Determining Core Capabilities in Chemical and Biological Defense Science and Technology. Washington, DC: The National Academies Press. doi: 10.17226/13516.
×
Page 99
Suggested Citation:"5 Management of Science and Technology at CBDP." National Research Council. 2012. Determining Core Capabilities in Chemical and Biological Defense Science and Technology. Washington, DC: The National Academies Press. doi: 10.17226/13516.
×
Page 100
Suggested Citation:"5 Management of Science and Technology at CBDP." National Research Council. 2012. Determining Core Capabilities in Chemical and Biological Defense Science and Technology. Washington, DC: The National Academies Press. doi: 10.17226/13516.
×
Page 101
Suggested Citation:"5 Management of Science and Technology at CBDP." National Research Council. 2012. Determining Core Capabilities in Chemical and Biological Defense Science and Technology. Washington, DC: The National Academies Press. doi: 10.17226/13516.
×
Page 102
Suggested Citation:"5 Management of Science and Technology at CBDP." National Research Council. 2012. Determining Core Capabilities in Chemical and Biological Defense Science and Technology. Washington, DC: The National Academies Press. doi: 10.17226/13516.
×
Page 103
Suggested Citation:"5 Management of Science and Technology at CBDP." National Research Council. 2012. Determining Core Capabilities in Chemical and Biological Defense Science and Technology. Washington, DC: The National Academies Press. doi: 10.17226/13516.
×
Page 104
Next: Appendix A; Schedule of Data-Gathering »
Determining Core Capabilities in Chemical and Biological Defense Science and Technology Get This Book
×
Buy Paperback | $45.00 Buy Ebook | $36.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The goal of the U.S. Department of Defense's (DoD's) Chemical and Biological Defense Program (CBDP) is to provide support and world-class capabilities enabling he U.S. Armed Forces to fight and win decisively in chemical, biological, radiological, and nuclear (CBRN) environments. To accomplish this objective, the CBDP must maintain robust science and technology capabilities to support the research, development, testing, and evaluation required for the creation and validation of the products the program supplies.

The threat from chemical and biological attack evolves due to the changing nature of conflict and rapid advances in science and technology (S&T), so the core S&T capabilities that must be maintained by the CBDP must also continue to evolve. In order to address the challenges facing the DoD, the Deputy Assistant Secretary of Defense (DASD) for Chemical and Biological Defense (CBD) asked the National Research Council (NRC) to conduct a study to identify the core capabilities in S&T that must be supported by the program.

The NRC Committee on Determining Core Capabilities in Chemical and Biological Defense Research and Development examined the capabilities necessary for the chemical and biological defense S&T program in the context of the threat and of the program's stated mission and priorities. Determining Core Capabilities in Chemical and Biological Defense Science and Technology contains the committee's findings and recommendations. It is intended to assist the DASD CBD in determining the best strategy for acquiring, developing, and/or maintaining the needed capabilities.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!