5


Management of Science and Technology at CBDP

Chapter 4 describes a process for capabilities-based strategic planning, but such a shift ought not be implemented without considering programmatic changes that could support its adoption and successful implementation. This chapter highlights two possible areas, the relationship between the end user and the RDT&E performers, and technology transition within the CBDP, where changes could have a significant effect on improving programmatic efficiency, especially if a capabilities-based planning approach were developed.

In addition, since one of the objectives for this study is to provide input that will assist the DASD(CBD) in “identifying the current model for sustaining lab infrastructure within DoD and determining if this is the most effective method for maintaining DoD critical infrastructure,” this chapter also presents observations regarding laboratory management within the CBDP.

As emphasized in Chapter 3 and Recommendation 3.1, a comprehensive, effective RDT&E program must support a Culture of Collaboration. Such a culture encourages individuals with expertise across disciplines to share information and effort in order to achieve their goals and enable programs to effectively manage resources, minimize duplication of effort, and identify opportunities for sharing information and facilities. The sections in this chapter largely relate to aspects of the CBDP that can influence the culture of collaboration at the research and program management levels.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 83
5 Management of Science and Technology at CBDP Chapter 4 describes a process for capabilities-based strategic plan- ning, but such a shift ought not be implemented without considering programmatic changes that could support its adoption and successful implementation. This chapter highlights two possible areas, the relation- ship between the end user and the RDT&E performers, and technology transition within the CBDP, where changes could have a significant effect on improving programmatic efficiency, especially if a capabilities-based planning approach were developed. In addition, since one of the objectives for this study is to provide input that will assist the DASD(CBD) in "identifying the current model for sustaining lab infrastructure within DoD and determining if this is the most effective method for maintaining DoD critical infrastructure," this chapter also presents observations regarding laboratory management within the CBDP. As emphasized in Chapter 3 and Recommendation 3.1, a comprehen- sive, effective RDT&E program must support a Culture of Collaboration. Such a culture encourages individuals with expertise across disciplines to share information and effort in order to achieve their goals and enable programs to effectively manage resources, minimize duplication of effort, and identify opportunities for sharing information and facilities. The sections in this chapter largely relate to aspects of the CBDP that can influence the culture of collaboration at the research and program man- agement levels. 83

OCR for page 83
84 CORE CAPABILITIES IN CHEMICAL AND BIOLOGICAL DEFENSE MAINTAINING A CONNECTION TO THE END USER The goal of the CBDP is to put effective tools in the hands of end users to minimize risk from chemical and biological threats. As described in Chapters 2 and 4, there are four identified missions for the CBDP, and the end users may differ between them. The warfighters and those who support them are critical, but, for example, civil defense authorities and personnel may need to be considered in some cases. In the current pro- grammatic structure, CBD priorities are identified by those with opera- tional knowledge, and products to address those priorities are developed by those with science and engineering expertise. Between these two lev- els, the S&T efforts required to develop the identified tools are recognized and broken into projects, which are then pursued by individuals or small teams, and the connection to the operational context is often lost until late-stage T&E. In the committee's view, this distance between the R&D performers and the end users represents a lost opportunity to allow for multidisciplinary (and multiperspective) consideration of the challenges of CBD in a field or combat situation. In discussion with the committee, R&D performers who had contact with the warfighters directly, whether as a result of collaboration on specific projects (often noted in connec- tion with USSOCOM) or in the pursuit of operationally relevant data in support of development of models, noted great value from interactions with operators. Other individuals in the basic research realm noted that understanding more fully the context in which their work would be used could assist in the development of operationally relevant projects and help identify overlooked variables or factors in their work. Strengthening the relationship between the warfighter and the R&D performers could support the development of specific capabilities by creating opportunities for innovation by allowing for informed "blue sky thinking" between the two groups. The committee noted that laboratory personnel from some facili- ties have relationships with warfighters as a strong component of their research and development programs. We suggest that the CBDP survey the facilities to identify where positive relationships exist, between Special Forces or the Services broadly, and seek to replicate such interactions. At the program management level, it may be difficult to encourage the strengthening of such relationships unless and until a capabilities-based approach is adopted as it may be challenging to see how providing R&D performers with greater operational context can be relevant to meeting specific requirements.

OCR for page 83
MANAGEMENT OF SCIENCE AND TECHNOLOGY AT CBDP 85 TECHNOLOGY TRANSITION WITHIN CBDP When a project progresses from the initial research and development work performed in support of product development to the test and evalu- ation and approval and acquisition of the material, management shifts from JSTO-CBD, under DTRA, to JPEO-CBD. Though both offices are overseen by the CBDP, there is no one office or individual who oversees the entire process for any given product. The discussion below highlights some of the challenges that this structure can pose for efficient develop- ment and presents some possible alternative options for consideration by DASD(CBD). These changes could help to improve the current pro- cesses, should a requirements-driven process be maintained, or support a revised, capabilities-driven process, as discussed in Chapter 4. The development of a final product is often iterative in nature, with failures in T&E identifying weaknesses that can only be resolved with additional R&D. At such times, the project management shifts from JPEO- CBD, which manages the T&E process, back to JSTO-CBD for additional R&D before returning to JPEO-CBD for the next round of testing and, potentially, approval. Successful transition between these offices requires a shared understanding of appropriate transition points. If an agreement is not in place, there is a risk that a gap between the perceived roles could result in the development of a project management "valley of death" and reduced program efficiency. While such issues can potentially be resolved through good working relationships between the offices, reliance on that relationship for success is not recommended as it can be strongly influenced by personality rather than policy. To address this concern, the committee recommends that DASD(CBD) consider alternative program management methods, including incorporation of an end-to-end project management authority to be maintained by an individual with scientific and technical expertise, as opposed to an acquisition specialist. In addi- tion, liaisons such as those used between the management offices and the COCOMs, or secondments between offices and laboratories could support the development of both formal and informal relationships between pro- gram personnel and facilitate the development of trust and understand- ing between the various parties and offices. One particularly stark example of the importance of an end-to-end project management authority, or at least consensus on the appropri- ate transition point, is in the development of medical countermeasures (see Box 5.1 highlighting some failures of medical products where poor transition between development stages was a contributing factor). The root of the difficulty is that development and acquisition of these items must contend with regulatory requirements--e.g., good laboratory and manufacturing processes, clinical trials, and acquisition of safety and effi- cacy data--and these requirements must be met in order to obtain FDA

OCR for page 83
86 CORE CAPABILITIES IN CHEMICAL AND BIOLOGICAL DEFENSE BOX 5.1 Examples of How Medical Product Development and Fielding Has Failed The transition from basic research to advanced development for medical CB countermeasures has been difficult and slow. The traditional approach in DoD's medical laboratories has been to do the initial research, identify and characterize candidate products, perform pre-clinical testing, and prepare some sections of an Investigational New Drug (IND) filing. The subsequent steps that are necessary to move such products through the rest of the acquisition process and the FDA regulatory process have been stymied for a variety of reasons and can be typified by these examples: 1.rPA vaccine: DoD conducted necessary S&T to identify and characterize Protective Antigen (PA) as the primary antigen for a new anthrax vaccine. Using recombinant DNA technology, GMP rPA was produced in pilot labo- ratories, safety and efficacy was shown in appropriate animal models (up to and including non-human primates) and the IND application was filed with the FDA. Unfortunantly, no practical expertise in obtaining scale-up development capabilities was resident in DoD; HHS took responsibility for production of large-scale lots of vaccine, clinical trials, and bringing the candidate though the FDA process. Technical issues related to the product stability, lot-to-lot variability, and other issues could not be resolved by DoD or HHS. rPA has yet to become a licensed product. 2.RSDL (Reactive Skin Decontamination Lotion): The extraordinarily long process to achieve FDA approval of this product can be attributed to the fact that DoD laboratories did not have sufficient expertise to address some of the regulatory issues related to product safety, incompatibility, and/or efficacy. Collaboration with experts from the drug development industry could have addressed some of these issues. RSDL was cleared by FDA for military use in 2003. approval of the final product. As these requirements necessarily influence product development pathways at an early stage, the current manage- ment structure within the CBDP is not well suited to the task because of the lack of a whole-process, integrated view of product development. In addition to challenges presented by the existing program management structure, the medical production process has not traditionally been a major focus of DoD, and as a result, the in-house expertise at the decision- making process is likely to be limited.

OCR for page 83
MANAGEMENT OF SCIENCE AND TECHNOLOGY AT CBDP 87 Medical Product Development within the CBDP Development of therapeutics under normal operating conditions within the pharmaceutical industry is a long and expensive process (com- monly viewed as requiring "twelve years and one billion dollars"). Being successful necessitates extensive management and quality control fea- tures. The complexity is such that a team management approach, drawing upon internal expertise in the early development phase through to the later production phase, is a common approach for incorporating continu- ity through the entire process. Development of therapeutics in the context of CB agent exposure adds complexity to the process: Documentation of pathophysiology of the disease in humans is minimal for most of the targets, and the efficacy of the medical products cannot be tested in humans against the relevant patho- gens. Thus, extrapolation from efficacy studies in animals are required to predict efficacy in humans; Animal models are difficult and not well defined for many of the pathogens; The regulatory process for developing therapeutics, particularly vaccines, is not a well-delineated process in particular with rela- tionship to the requirements for correlates of protection; The acceptable risk for the therapeutic efficacy range appears to be set at a very high threshold, well above what is typically acceptable under other military operations. The reason for listing some of these complexities is to demonstrate that production of vaccines and therapeutics in the DoD CBD context is extraordinarily challenging, and careful planning and project manage- ment is required to be successful. Over the course of the committee's data-gathering, it was apparent that the current process for development of medical products has led to few, if any, new therapeutics. Discussions with CBDP staff indicated that the following factors are contributing to the low success rate: Minimal communication between the JSTO-CBD and JPEO-CBD managers to facilitate product development; An apparent expertise gap in the management offices, particu- larly regarding identification and handling of critical transition points in product development; Turnover in personnel at several levels without a committed leg- acy of ongoing product development makes for multiple "starts and stops" during the process;

OCR for page 83
88 CORE CAPABILITIES IN CHEMICAL AND BIOLOGICAL DEFENSE Lack of a transparent process for determining long-range critical therapeutic or vaccine targets; and No consistent methodology for selection of partner laboratories or companies with expertise in therapeutic development. Note that these factors were not identified as part of a formal review of the program, but through formal and informal conversations. The com- mittee believes that a formal, preferably external, review of the process would be valuable before taking action to modify the current process . Establishing "Common Language" for Transition of Medical Countermeasures One possible model for managing medical product development tran- sitions is demonstrated by the integrated portfolio for CBRN countermea- sure development through BARDA. BARDA's role is to develop medical countermeasures for the Strategic National Stockpile, and as a result, it shares many of the challenges DoD faces in its therapeutic development process. Though housed within HHS, BARDA represents a collaborative effort between multiple agencies, including FDA, DoD, and DHS. One of BARDA's policies is to not accept potential products into the program until the work has progressed through an FDA-approved Phase I safety study. Using this structure to define the benchmark allows for clear communication of expectations across various agencies and companies. Using an FDA regulatory step as the transition point from JSTO-CBD to JPEO-CBD--or to or from another DoD/USG development partner-- could provide a similar "common language" determining the transition point and establishing expectations within the CBDP. Such an objective transition point may improve project and personal efficiencies by allow- ing for more accurate out-year staff and budgetary planning, which would hold value not only for the project scientists and managers, but also for higher-level commanders and directors for portfolio and/or budget planning and defense; reduced burden of negotiation meetings to establish transition points for ongoing projects; clearer determination of the responsibilities and expectations of JSTO-CBD and JPEO-CBD; and easier to manage timelines for expected returns on R&D investments.

OCR for page 83
MANAGEMENT OF SCIENCE AND TECHNOLOGY AT CBDP 89 BOX 5.2 Defining a Transition "Trigger" DoD is not the only agency or organization that faces the challenges described herein. Others, such as pharmaceutical companies in the commercial sector and BARDA within the US government also must manage transitions effectively. There are three commonly used and referenced transition points early in a product's de- velopment that are seen as logical "triggers" for the transition from basic research to development of a product. They are (1) submission of an investigational new drug (IND) and (2) completion of the Phase I trial of a material. a As one example of how these triggers could be incorporated into the CBDP, the Material Develop- ment Decision (MDD), which currently resides with JPEO-CBD, could become trigger for initiation and development of an IND application for submission to FDA. Practically, this would mean all discovery and preclinical activity would reside under JSTO-CBD management, and successful programs would be presented to JPEO-CBD for a MDD. MDD approval would trigger construction and submission of the necessary applications to enter an FDA-approved regulatory approval path. Alternatively, transition to JPEO-CBD management could occur after completion of a Phase I trial. Under a third alternative (3), as part of a broader strategy within the CBDP for FDA-regulated products to more efficiently use available advanced development funds, successful programs could be "parked" after construction of the IND or after Phase I trial completion. This could be especially useful when the program has multiple potential products in any given area of need. Regardless of the chosen trigger, expertise, within or contracted by JSTO-CBD and JPEO-CBD, needs to be appropriately positioned. This approach would also be supportive of overlap in JSTO-CBD and JPEO-CBD personnel engagement on the project to ensure smooth and knowledgeable transitions. a Selection of either an IND submission or Phase I completion in the common JSTO-JPEO transition point would generally align with DoD 5000 service recognition milestones A ("first in human") or B ("point of concept"), respectively. It would also generally align with transitions from 6.2 to 6.3 or 6.3 to 6.4 funding. An additional potential benefit would be the inclusion of FDA sci- entists, such as those within the Medical Countermeasures Initiative (MCMI) program. Clearly, the earliest possible settings of expectations from the FDA--even from an advisory position--on the types of data sets that may be required to reasonably complete a Phase I trial would be helpful in managing the overall process. See Box 5.2 for a discussion of possible transition points for medical product development.

OCR for page 83
90 CORE CAPABILITIES IN CHEMICAL AND BIOLOGICAL DEFENSE LABORATORY MANAGEMENT As was described in Chapter 1, the RDT&E elements within the CBDP draw upon and direct resources within a number of different laboratories and facilities. These include service laboratories, medical laboratories and facilities, and test and evaluation facilities. Each laboratory and facility has its own mission and management structure, and this has implications for the ability of the CBDP to successfully manage its projects and pro- grams (see Box 5.3). This section presents a brief overview of the history of the management of laboratory research at medical and non-medical facilities, identifies elements of laboratory management the committee feels are critical for success, and presents possible alternative methods for BOX 5.3 The Role of Medical Laboratories in Chemical and Biological Defense Research In the 1980s and 1990s, the funding at medical facilities for chemical and biologi- cal research was provided by DTRA directly to the commanders for their allocation and distribution. At that time, the laboratories performed basic science in support of understanding the medical response to exposure to chemical or biological agents. The research was not focused on the development of specific products, such as vaccines or countermeasures. Beginning in the early 1990s, DTRA began directing these R&D funds through MRMC (US Army Medical Research and Materiel Com- mand), giving MRMC's command structure both responsibility and authority for the science within their major laboratories. Under this new model, funding was allocated by program and, as a result, the laboratories became focused on the development of specific product targets, e.g., an anticonvulsant or a plague vaccine, though still on the basic science in support of that target. Advanced development leading to an FDA-licensed product was not within the laboratories' purview. When the JPO was formed, the role of the laboratories changed again. Now medical laboratories, specifically USAMRIID and USAMRICD, were asked to take candidate countermeasures beyond the basic science toward the development of a licensed product. This required the laboratories to produce pilot-scale lots of GMP (good manufacturing practices) candidate product and to conduct the GLP (good laboratory practices) pre-clinical studies in preparation for the initial submission to FDA for approval. This continued until the formation of JSTO-CBD in 2003. Though the focus on specific product development continued after that point, investigators within the laboratories were asked to respond to specific requests for proposals in a competitive environment, rather than the laboratories pursuing a program (e.g., development of an anthrax vaccine) with guaranteed funding. With this change, the responsibility for performance remained with the laboratories and the authority to manage the programs resided at JSTO-CBD in Ft. Belvoir.

OCR for page 83
MANAGEMENT OF SCIENCE AND TECHNOLOGY AT CBDP 91 laboratory management that could be considered. Overall facility man- agement, which includes infrastructure, workforce, and research program management within the facility, plays a critical role in the stewardship necessary to support a successful S&T related endeavor. This stewardship also extends to funding and directing the scientific work of the program. Successful Laboratory Management The CBDP relies upon a laboratory network and test ranges to pro- vide RDT&E and produce products critical to the chemical and biological defense of the nation. The network consists of DoD owned and oper- ated laboratories (e.g., USAMRIID, ECBC, Air Force Research Laboratory (AFRL), NSRDEC, and Naval Surface Warfare Center (NSWC) Dahlgren), DOE national laboratories (e.g., LANL, SNL, LLNL, Pacific Northwest- ern National Laboratory (PNNL)), FFRDCs (Federally Funded Research and Development Centers), non-profit entities (e.g., Battelle), for-profit commercial laboratories, and universities. At many of the facilities the committee visited, the recent construction of new buildings will provide new capacity for RDT&E, adding both additional space and additional technical functionality. Successful RDT&E leading to fielded systems1 is a long (perhaps a decade or more), arduous process. In the committee's view, a successful RDT&E program requires at least the following six elements to ensure clarity of purpose, focus of investments, and coherence of management: 1. Clear mission and objectives 2. Continuity in leadership 3. The ability to understand, accept, and manage risk throughout the process 4. Predictable and stable funding 5. Effective asset management at the laboratory level 6. A sense of excitement and pride in the work among the staff In the following sections, these elements will be defined and subsequently discussed in the context of the CBDP. 1 Fielded systems are not exclusively hardware or medical countermeasures. For example, they may include an operational diagnostic and analysis systems including collection, trans- portation, analysis, and identification including all the required infrastructure, protocols, routinely exercised.

OCR for page 83
92 CORE CAPABILITIES IN CHEMICAL AND BIOLOGICAL DEFENSE Element 1: Clear Mission and Objectives For an effective program, all participants in the endeavor need to understand its mission and objectives, how they contribute to achieving those goals, and why their work is important in that context. A laboratory cannot be successful if it proclaims it is undertaking all missions for the entire government. "Reinvention" may be required as the world changes, but this should only happen rarely. Without a clear, defined mission and objective, so-called "mission confusion" may lead to expansion into activities that do not directly support the objectives or that are duplica- tive of efforts in other laboratories. Lessons from successful laboratories (government and non-government) demonstrate that a shared mission understanding leads to long-term success. Element 2: Continuity in Leadership Strong leadership, and continuity of the stated mission and objectives during and after changes in leadership, supports the development of sense of mission within the workforce. In contrast, rapid changes in lead- ership and/or weak leadership can contribute to the mission confusion described above. Of course, sometimes rapid changes in facility directors or commanders is unavoidable, but in such cases, care should be taken to ensure that succession planning, pre-training, orientation of incoming personnel, and personnel continuity in senior positions is encouraged. These steps will help maintain a consistent vision and understanding of the facility's mission and objectives, which is important for pursuit of pro- grammatic and project-related goals. An additional benefit is that when individuals are in a position for a significant period of time, e.g., longer than two years, they have a greater ability to develop relationships and partnerships within the research and end-user communities than might otherwise be possible. This is beneficial for the identification of opportuni- ties for collaboration. Element 3: The Ability to Understand, Accept, and Manage Risk RDT&E is a risky process, and a successful research and development program may have many false starts before achieving success. Thus, an essential element of managing such programs is accepting and balancing risks. There may be times when it is appropriate to undertake high-risk, high-payoff projects which may ultimately fail in addition to maintain- ing long-term focus on a specific area with low risk in order to provide fundamental understandings that support new developments. An effec- tive program will create a balanced portfolio of these various research types. Similar to the process described in Chapter 4, in the section "From

OCR for page 83
MANAGEMENT OF SCIENCE AND TECHNOLOGY AT CBDP 93 Specification to the Research-Development-Acquisition Process," external review can help down-select projects to improve overall chances for pro- grammatic innovation and success with reduced overall costs. Care should also be taken to not introduce unnecessary risk of proj- ect failure through poor planning and management. Understanding the eventual goal of a project or program and identifying appropriate mile- stones that must be completed or addressed for success can allow for corrections of approach and provide confidence at critical junctures that all required elements are in place to minimize the chances of failure due to bureaucracy. A necessary adjunct to a well-balanced risk approach is a method for continual assessment of program and/or project progress. Internal, and especially external, standing technical review committees (supplemented by those with operational knowledge) are required to ensure that unsuc- cessful programs are terminated, to provide technical review for high- risk/high-payoff projects, and to encourage consideration of programs that may not have originated within the facility. The committee cautions that each institution might appoint separate review boards with different membership; while a diversity of opinions is good, having numerous, separate groups could prevent identifying redundancy and duplications. It is essential that reviewers are able to consider the context and larger picture, and maintain continuity.2 Element 4: Predictable and Stable Funding R&D takes time. Continual disruptions due to major funding shifts and delays lead at best to inefficiencies and, at worst, failure. Funding can roughly be considered at three levels (project, program, and laboratory): Project. Projects are narrowly scoped activities with specific, well- defined goals. Projects may only require a few years for comple- tion, but this timeframe places a high premium on efficiency. Gaps in funding can hinder that efficiency, and an uninterrupted funding stream for the duration of the project or at least between major project milestones/decision points is preferred. Program. Programs have a broader scope than projects with desired outcomes that require multiple steps or inputs for com- pletion. These can last up to a decade, and to be successful, fund- ing needs to be assured (to the maximum extent possible) for that 2 As one possibility, there are several existing government groups with non-governmental life scientists and others who hold clearances at sufficiently high levels and could be utilized, with augmentation as needed.

OCR for page 83
94 CORE CAPABILITIES IN CHEMICAL AND BIOLOGICAL DEFENSE duration. Some adjustments in priorities are inevitable, but major directions must be consistent and supported. Laboratory. Laboratories exist for multiple decades and require predictable funding to ensure a high-quality workforce and up- to-date research facilities and infrastructure. Laboratory directors and commanders must be assured of some level of funding if they are to effectively plan for the future and manage a workforce and research infrastructure. This should not be confused with entitle- ments, where funding is continued independent of performance. In order to be effective, a balance should be found with significant amount of core funding "guaranteed" and the remainder pro- vided through a competitive environment. Element 5: Effective Asset Management at the Laboratory Level Some local control of funding is considered by the committee to be an important component of successful laboratory management. A local ability to move people and resources between projects can assist in creat- ing an efficient environment as decision making can be more responsive than if external approval is required. If a laboratory director is prevented from anticipating and acting on predicted future needs, the laboratory may find it challenging to capitalize on emerging technologies or provide for new operational needs. Workforce management should be left in the hands of laboratory management (with command oversight distinguished from management), with external laboratory reviews providing input to help identify emerging needs or areas of concern (see Element 2). Simi- larly, programs may benefit when laboratory directors have some amount of unallocated funds (internal research and development (IR&D) and capital funds) to invest as they deem necessary (e.g., providing funds to a researcher to pursue a new line of inquiry or addressing unmet main- tenance or operational needs) to ensure that the laboratory will be able to continue to meet its mission obligations. This is part of instituting a balanced risk approach to RDT&E. Element 6: A Sense of Excitement and Pride in the Work among the Staff Ultimately the success of an RDT&E program depends on not only the creativity and skill of the staff, but also the environment in which they work. Modern research by its very nature is a collaborative effort. Engagement with the research community, both internal and external, gives research staff access to a diversity of ideas, which may open new areas of research or help make intellectual connections between differ- ent projects (see Box 5.4). Networking between facilities and researchers

OCR for page 83
MANAGEMENT OF SCIENCE AND TECHNOLOGY AT CBDP 95 BOX 5.4 Description of the Human Component of S&T Capability Professional development of performers, program, and middle and senior man- agement is also important in creating a positive working environment and recruiting and maintaining quality within the program. While the CBDP should seek to recruit and retain the best possible personnel for each position across the enterprise, a coherent system of professional development is also needed. Without entry-level development, new performers or entry-level program managers should be provided indoctrination as to requirements and processes associated with their positions as well as exposure to their customers and stakeholders. From the outset they should have a clear understanding of why their program exists and who it serves. As experience is obtained and acceptable performance is maintained, broadening opportunities should be provided in relevant science and with organizations and experts who are working in closely related or complementary areas (both inside and outside of DoD and the USG). Regular professional development opportuni- ties should be provided at specified intervals, based on individual qualifications, performance and career goals and the needs, requirements and strategic goals and objectives of the CBDP and subordinate components. Opportunities to refresh in relevant S&T should be part of professional development and will benefit the individual and program. "One size fits all" will not be an appropriate or value-added approach. can also help identify potentially underused facilities or resources that can be repurposed or exploited by others in need of additional capacity (note that this can also result in identification of opportunities of "Work for Others" funding). Collaboration also provides an opportunity for researchers to be recognized by their peers as leaders or experts within a given area. Engagement should not be limited to interactions between research- ers. It is especially important to form collaborative efforts with the end users as this can sometimes lead to solutions to unacknowledged or unknown operational needs. These connections can also provide needed field feedback to developers of materials and systems. These collabora- tions can also provide a reinforcement of the overall mission of the agency and purpose of the program in support of Element 1. This particular point, the relationship between the end user and the developer, will be discussed in greater detail later in this chapter.

OCR for page 83
96 CORE CAPABILITIES IN CHEMICAL AND BIOLOGICAL DEFENSE LABORATORY MANAGEMENT WITHIN CBDP The committee visited a number of facilities and spoke with many individuals, both formally and informally, over the course of the study. What follows is a broad summation of the observations and impressions of the management of RDT&E in the CBDP in the context of the six ele- ments outlined in the previous section. Examples are provided where possible and appropriate. The committee recognizes that its interactions with the program have been necessarily limited, and these descriptions are not drawn from formal surveys or metrics. Also see Box 5.5. Clear Mission and Objectives The official CBDP mission is to "provide global chemical, biologi- cal, radiological, and nuclear defense capabilities in support of National Strategies." This is a broad statement and can be applied to a variety of activities. In conversations with the committee members, facility person- nel and program-level staff were in accord with this mission. However, the committee heard different responses with regards to the specific role of given offices, facilities, and laboratories in meeting this mission. Spe- cifically, there seemed to be a lack of common vision between JSTO-CBD and the funding recipients. The recipients expressed concern that it was unclear how the requests for proposals and the competitive process fed into the strategic vision of the laboratory and the CBDP because the proposals did not seem to build on each other to build a comprehensive picture but rather reflected the identified, internal needs or concerns of the year. For the laboratories focused on the development of specific, complex products, an unclear mission or program objectives may hinder the ability of the laboratory to meet its goals. Continuity in Leadership Leadership plays an important role in communicating mission and objectives, and in S&T research, this role is often performed by a techni- cal director with a long history within the organization and/or field of research. However, at a number of facilities within the CBDP, the facility military commander (or deputy commander) now changes every two years. Such rapid changes in leadership have the potential to disrupt the ability of an organization to meet its mission and objectives. In such cases, the senior civilian leadership is familiar with the program, which provides a level of continuity, but it is probable that longer-term appoint- ments in the military leadership positions could be helpful in maintaining continuity of programmatic focus at facilities.

OCR for page 83
MANAGEMENT OF SCIENCE AND TECHNOLOGY AT CBDP 97 The Ability to Understand, Accept, and Manage Risk The committee recognizes that understanding, accepting, and manag- ing risk in the CBDP is a difficult task. Within the RDT&E program, the research ranges from developing a fundamental understanding of the interactions of molecules with surfaces to developing reliable vaccines and medical countermeasures to be given to warfighters in theater. This being the case, careful management of programmatic risk is essential to ensure that effort and resources are directed appropriately, and there are tools that can be used to assist in this process, e.g., linking activities to a clear mission and set of objectives (see previous section) and robust external strategic and technical review processes. With regards to external technical review, the committee saw little evidence that peer review of S&T is occurring or encouraged. If this is indeed the case, this is a lost opportunity to draw input from knowledge- able individuals who do similar research, to build connections to those researchers with relevant knowledge, and most importantly, to ensure that promising ideas are not lost and that fundamentally flawed projects do not continue any longer than necessary to identify the flaws. One con- cern that was mentioned to the committee was that innovative ideas have little room to be developed in the current funding environment, which may result in lost opportunities. With regards to strategic reviews, some facilities have engaged in individual efforts to develop a strategic plan, but those that have not done so within the last decade should be encour- aged to create one. In addition to facility-level reviews, a strategic review of the CBDP could identify gaps between mission and activities and allow an opportunity for correction of any such issues. The committee also notes that poor project management can intro- duce a higher risk of failure to meet an objective than the science might suggest. One area where this can be seen most clearly is development of licensed medical products. Coordination between multiple offices within DoD and with the FDA are required for successful transition of a pro- totypical medical product to clinical trials and ultimately to licensing. A poor understanding of the complexities of this process and a lack of end-to-end authority that manages the process can introduce risks of bureaucracy causing a failure to meet the objective. Within the CBDP, the committee notes that few medical products have successfully made the transition to licensing, and though many factors lead to such a situation, a review of the structures supporting this process by the DASD(CBD) may be merited.

OCR for page 83
98 CORE CAPABILITIES IN CHEMICAL AND BIOLOGICAL DEFENSE BOX 5.5 Waging Science: Re-casting Defense Science Management To an outsider, DoD's management of CBD science to aid the warfighter and the homeland appears to be run as an acquisition management problem, rather than a science management one. Currently, DoD CB science does not provide an efficient and productive path from needs to solutions. Making material improvements to this situation will not be accomplished easily; however, the committee believes that DoD is uniquely positioned to re-cast defense science management into a much more effective "program" by treating it in a fundamentally different way: S cience is a long-term campaign, not an acquisition of beans and bullets. Whether it is developing next-generation protective gear, rapid field diagnostic assays, new vaccines, or standoff CB detectors, there are signifi- cant R&D hurdles to be faced before the acquisition phase of "off the shelf" items can be reached. Frequently, multidisciplinary teams are required to tackle problems and eventual success in science is never guaranteed. Meeting science challenges requires a strategy which requires long- term planning; treat scientific challenges like planning for campaign. Treating a scientific challenge more like a mission than an inconvenient prelude to an acquisition will provide the mindset needed to focus the right team of scientists towards a single goal. The scientific program should be directed toward mission goals and to the building of an ability to react to emerging or unexpected threats. These goals should be understood by all involved. Examples of clear goals might be "Create effective diagnostics for all major would-be patho- gens"; "Create an effective standoff biodetector that will permit sufficient warning time to don personal protective gear." It is important that all scien- tists and engineers on the mission team understand that the team mission goal is more important than any organization or personal goals. Examples of such goals might include "Author a major paper on would-be pathogens"; "Keep Laboratory X's share of biostandoff funding at #1." Both strategies and tactics will be important. The mission needs to be planned and managed end-to-end by someone with overall mission leadership and accountability. This is different from current DoD science management where "handoffs" are supposed to occur at various Technical Readiness Levels between organizations that may not be well coordinated end to end. Plan for the unknown and the unexpected; anticipate change but don't get hung up on predictions. Long-range planning to address major de- fense science challenges simply cannot be effectively planned in DoD stan- dard 5-year cycles. Some challenges will prove much harder than expected, while in other cases new technology may make other challenges far easier than initially planned. About the only certainty is that any multiyear plan involving CBD science challenges will require major revisions by the second or third year, if not sooner. The team and its leadership need to have the capability to react to major changes in plans and still achieve victory.

OCR for page 83
MANAGEMENT OF SCIENCE AND TECHNOLOGY AT CBDP 99 A ssemble the right force to meet the challenge; teaming and coordi- nation will be paramount to achieve victory. When planning for a major military battle, the commander in charge will attempt to assemble all the different kinds of forces needed to perform the expected tasks. This might involve land, sea, and air assets, plus satellites. Similarly, the commander of a science battle may need to assemble forces from multiple DoD labo- ratoriess, academia, industry, and other government science resources. Regardless of which service the commander belongs to, the assembled team must be the best available to achieve the mission goal. Other possible parochial goals ("Laboratory X doesn't have the right expertise but they need some more funding this year"; "Laboratory Y has worked in this field for several decades, but they aren't up to speed on the latest technologies") cannot drive team selection. It is possible that for some scientific challenges most or all of the best expertise needed might lie outside DoD laboratories. Each battle campaign needs a clear overall leadership chain. Waging science is not the same as a NIH or a DARPA basic research project. You do not win a battle with a number of autonomous commanders (principal investigators) who may be working toward different personal goals. Major DoD CB science challenges are inherently Big Science projects that need to be managed appropriately. The skill of effective team formation and mo- tivation cannot be overestimated as necessary leadership criteria when it comes to managing Big Science challenges. Embrace innovation and be flexible enough to realize when it is needed. Technology in the CB domains is evolving at warp speed these days. This is especially true in biology, where recent advances in genome sequencing have increased throughput by at least three orders of magni- tude in the past three plus years. The management of DoD CBD science campaigns must be capable of understanding such rapid evolution and be able to manage to make major changes in strategy at the right times. This could be achieved with the establishment of an effective central "tech watch" component within the DoD science portfolio, but each campaign leader would still have to perform their own judgment of whether or when a technology jump is optimal for reaching their mission goal. Innovation can't be mandated, and needs to be nurtured. Campaign leaders need to have the ability to have their teams test out promising innovative ideas. Logistics and infrastructure win wars; have a flexible procurement strategy to keep your troops well-equipped as conditions change. The experiences in Iraq and Afghanistan painfully illustrated the high cost in terms of life and limb when a combination of factors delayed getting more effective protection for IEDs to troops. Inefficiencies in DoD procurement also affect every CB science campaign, particularly in cases where flexibility from a set plan is needed to deal with technology evolution. Set meaningful milestones and adjust them as needed based on how the battle develops. The goal is to win the battle, not follow a static plan. Improvements that allow and enable innovative flexibility are sorely needed. Paperwork and inflexible regulations appear to cause major inef- ficiencies in how DoD wages science. Commanders need to have ways to innovate and react to improve their chances of winning.

OCR for page 83
100 CORE CAPABILITIES IN CHEMICAL AND BIOLOGICAL DEFENSE Predictable and Stable Funding The committee was informed that, though the funding mechanism has changed over time, the funding levels for RDT&E have remained relatively stable in recent years. In general, discussions with researchers indicated that necessary resources are available for current activities, though multiple groups noted that any significant cut in funding would result in cuts to programs. One concern expressed many times to the committee was that the competitive process has resulted in a sense that funding is unpredict- able, making long-term endeavors challenging. Once a proposal had been approved, the committee heard no concerns regarding funding levels of a specific project. However, timing of receipt of funds was discussed, and delays in receipt of funds due to contracting difficulties were cited as a barrier to developing collaborations with external organizations. At the program level, the primary concern expressed to the committee was that the links between the project-level funding and the program-level priori- ties were not always clear. With regards to laboratory-level funding, care should be taken to ensure that the recent construction of new buildings, some of which will place new demands on operations and maintenance budgets, does not cause the resources of laboratories and facilities to become overstretched. Effective Asset Management at the Laboratory Level Funding for CBD research at the laboratories is awarded through a competitive process directly to the principal investigators. Barring the funds acquired through reimbursement of services provided to external agencies and entities (up to 2.5% of IR&D), the laboratory directors have little direct control over their budgets and the allocation of those budgets to the staff and facilities. During multiple site visits, the committee heard that though people felt they had the resources required to perform their work, there was a lack of flexibility of funding that could make it dif- ficult to be responsive to emerging opportunities or concerns. Some of these concerns could be exacerbated in the near future with the opening of multiple new facilities with new capabilities and, likely, management challenges. Some researchers who spoke with the committee expressed some concern regarding the degree of JSTO-CBD's involvement in directing research. While it was acknowledged that a competitive process can assist in maintaining a strong research function, concerns that the focus was too great on projects rather than programs was expressed--i.e., no long- term vision or strategy supported the specific calls for proposals. This, of course, must be balanced against the views from the program managers

OCR for page 83
MANAGEMENT OF SCIENCE AND TECHNOLOGY AT CBDP 101 who stated that the proposal requests are a product of the requirements- development process and are reflective of the needs of the CBDP and the agency as a whole. The role of this committee is not to decide who is "right" in this discussion, but to observe that the relationship between JSTO-CBD and the service laboratories appears strained. The lack of local autonomy (authority) within the current funding model is seen as imply- ing a lack of trust, whether intended or not, and this seems to affect the relationship between the two groups. Whether these issues can be resolved through increased communication about the overall funding strategy or if a major revision of the program will be required to resolve these issues will have to be determined and addressed by the DASD(CBD). A Sense of Excitement and Pride in the Work among the Staff In light of the discussions above, as one might expect, significant frustration with the funding mechanisms and priorities were expressed many times to the committee. However, in general, staff at the facilities expressed a pride of purpose and a sense that the work being done at the laboratories is important and relevant. One area that was highlighted by many PIs and performers was a desire to gain a greater understanding of end-user needs and the environ- ment and circumstances in which the materials, products, and procedures will be used. For some, the connection between the research laboratory and the field is rather abstract. Reducing the gap between the bench and the end user could be beneficial in many ways, including increasing excitement and pride in the work among the staff. Review and Assessment All projects benefit from scientific peer review when done well, and these reviews keep the skills of scientists and engineers sharp. Reviews also provide an important function in ensuring that scientists within dedi- cated institutions stay in touch with the external community. Within DoD CBD laboratories, there was a noticeable lack of connectivity to research in other institutions and an independent review process, especially in the critical research phase. FINDINGS AND RECOMMENDATIONS This final chapter has brought forth a series of principle findings and recommendations that can assist the program in operating a successful laboratory environment.

OCR for page 83
102 CORE CAPABILITIES IN CHEMICAL AND BIOLOGICAL DEFENSE Program Management Finding 5.1: Successful transition between the JSTO-CBD and the JPEO-CBD offices requires a mutual agreement on appropriate transi- tion points, encoded in multiyear program plans and budgets. Regard- less of the chosen trigger, expertise and resources within or contracted by JSTO-CBD and JPEO-CBD need to be appropriately positioned. This approach would also be supportive of overlap in JSTO-CBD and JPEO- CBD personnel engagement on the project to ensure smooth and knowl- edgeable transitions. However, the committee observed that the partner- ship between the JSTO-CBD and JPEO-CBD is weak and that neither office viewed transition plans as a responsibility. Finding 5.2: There is no end-to-end authority for the CBDP, which is particularly problematic for medical products. Though both JSTO-CBD and JPEO-CBD are overseen by the CBDP, there is no one office or indi- vidual with the responsibility and authority for the entire process for any given product. The risk--and reality--is that a transition gap between R&D and acquisition could result in the development of a project manage- ment "valley of death." The existing research-development-acquisition process may be adequate for acquiring the non-medical products in the CBDP. For the medical countermeasures program, however, FDA regula- tory requirements must be considered early enough to influence product development decisions. The current management structure within the CBDP is not well suited to the task because of the lack of a whole-process, integrated view of product development. Recommendation 5.1: The DASD(CBD) should evaluate alternative program management approaches, including incorporation of an end- to-end project management authority, especially for the medical coun- termeasures program. Laboratory and Major Facility Management Finding 5.3: The principal RDT&E military organizations associated with the CBDP are benefiting from major facility investments that are planned to provide both capabilities and capacities to meet the antici- pated needs of the program. Operating and maintaining these facilities, however, will place a burden on both the owning Service (principally the Army) and the program. The initial operating plans appear to be resourced. Finding 5.4: All or part of the elements required for healthy RDT&E activities were missing at the organizations visited by the commit-

OCR for page 83
MANAGEMENT OF SCIENCE AND TECHNOLOGY AT CBDP 103 tee. A successful RDT&E enterprise should include the following ele- ments to ensure clarity of purpose, focus of investments, and coherence of management: 1. Clear mission and objectives 2. Continuity in leadership 3. The ability to understand, accept, and manage risk throughout the process 4. Predictable and stable funding 5. Effective asset management at the laboratory level 6. A sense of excitement and pride in the work among the staff Of special concern are strained relationships between JSTO-CBD and the laboratories, the new rotational policy for military commanders in the Army, and a trend toward increasing oversight of both technical work and operations at the facilities. Recommendation 5.2: The DASD(CBD) should formally review alterna- tive laboratory management models, taking advantage of the numerous prior studies, reviews, and evaluations of laboratory and large facility management of S&T organizations. A principal objective is to define the level of stewardship that the program should provide to the principal RDT&E in-house facilities and laboratories. Scientific Peer Review Finding 5.5: All programs benefit from scientific peer review when done well, and these reviews keep the skills of scientists and engineers sharp. Recommendation 5.3: The DASD(CBD) should implement a nested review process for chemical and biological defense RDT&E bound by consistent standards of rigor, frequency, and reporting. The CBDP and its supporting laboratories would each benefit from independent, periodic review at the programmatic and scientific levels. The CBDP should also encourage and participate in institutional reviews. An annual roll-up of review outcomes could help identify thematic areas of promise and concern.

OCR for page 83