Executive Summary

Background

In 1993, the U.S. Army established the Chemical and Biological Defense Command (CBDCOM) to conduct research, develop and procure support systems, and design equipment to protect U.S. military personnel from chemical and biological weapons used by foreign military forces and terrorist organizations. The CBDCOM is the latest in a long line of military organizations designated for chemical and biological defense research. Because of the critical nature of its mission, the CBDCOM requested that the National Research Council (NRC) establish an oversight committee of nationally recognized experts to provide ongoing, impartial, independent advice.

The NRC, responding through the Board on Army Science and Technology of the Commission on Engineering and Technical Systems, created the Standing Committee on Program and Technical Review of the U.S. Army Chemical and Biological Defense Command, better known as the CBDCOM Standing Committee (CSC). This committee was assembled to provide expertise in the areas of science and technology pertinent to the concerns of the CBDCOM commander and executive director and the technical director of the Edgewood Research, Development and Engineering Center (ERDEC), which historically has been an important organization for chemical and biological research in the Army and the U.S. Department of Defense.

The committee was asked to consider technological issues and systems to assist the CBDCOM in defining a vision for the future. After numerous visits and interviews with key personnel at the ERDEC and the CBDCOM and internal deliberations, the committee focused on two major areas: (1) a



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 1
--> Executive Summary Background In 1993, the U.S. Army established the Chemical and Biological Defense Command (CBDCOM) to conduct research, develop and procure support systems, and design equipment to protect U.S. military personnel from chemical and biological weapons used by foreign military forces and terrorist organizations. The CBDCOM is the latest in a long line of military organizations designated for chemical and biological defense research. Because of the critical nature of its mission, the CBDCOM requested that the National Research Council (NRC) establish an oversight committee of nationally recognized experts to provide ongoing, impartial, independent advice. The NRC, responding through the Board on Army Science and Technology of the Commission on Engineering and Technical Systems, created the Standing Committee on Program and Technical Review of the U.S. Army Chemical and Biological Defense Command, better known as the CBDCOM Standing Committee (CSC). This committee was assembled to provide expertise in the areas of science and technology pertinent to the concerns of the CBDCOM commander and executive director and the technical director of the Edgewood Research, Development and Engineering Center (ERDEC), which historically has been an important organization for chemical and biological research in the Army and the U.S. Department of Defense. The committee was asked to consider technological issues and systems to assist the CBDCOM in defining a vision for the future. After numerous visits and interviews with key personnel at the ERDEC and the CBDCOM and internal deliberations, the committee focused on two major areas: (1) a

OCR for page 1
--> technical assessment of the man-in-simulant test (MIST) program; and (2) a review of the processes and technical character of ERDEC's mass spectrometry and bioremediation programs. The results of the assessment of the MIST program are documented in a 1997 NRC report, Technical Assessment of the Man-in-Simulant Test (MIST) Program. The present report addresses the ERDEC's mass spectrometry and bioremediation programs. Current Study For this study, the committee was guided by the Statement of Task that directed the committee to "review the processes used by two representative, but very different ERDEC programs, mass spectrometry and bioremediation, to move basic research results and technology through development." The Statement of Task also directed the committee to comment on the "technical character of the two teams, since those processes must be assumed to affect overall technical quality," and, among other things, to review the procedures to "promote an overall quality focus." The Programs Under Review The mass spectrometry and bioremediation groups belong to two different ERDEC teams. The mass spectrometry group is part of the Chemical and Biological Point Detection Team, and the bioremediation group is part of the Environmental Technology Team. The mass spectrometry group is composed of the following three subgroups: the thermal degradation subgroup, whose objective is to determine whether analyses of the thermal degradation products of spores, bacteria, viruses, and proteins can be used as biomarkers the laser-based spectrometry subgroup, whose objective is to develop laser-based approaches to agent detection the direct mass spectral analysis subgroup, whose objective is to use matrix-assisted laser desorption/ionization (and electrospray ionization in the future) to obtain mass spectrometric fingerprints of spores, bacteria, viruses, and proteins by direct mass spectral analysis with the aim of identifying selected materials on the Department of Defense threat list.

OCR for page 1
--> The bioremediation group consists of 12 people who were working on 11 projects in fiscal year 1997 and 13 projects in 1998 including the following: The largest project, Enzymatic Decontamination, is the development of a new generation of chemical-biological warfare decontaminants based on catalytic enzymes. Unlike current decontaminants, these will be nontoxic, noncorrosive, nonflammable, and environmentally safe. The Alternative Technologies Program is focused on the demilitarization of chemical weapons stockpiles. Its objective is to develop biotechnical means for destroying the stockpiles of chemical agents as an alternative to incineration. A third project is the Biodegradation of Decontaminating Solution 2 (DS2). DS2, the current decontaminant in the Army inventory, is effective, but it is also toxic, flammable, and highly corrosive. Approach The data for this study were derived from two interview sessions and from responses to written questionnaires. The committee developed an assessment model of five categories subdivided into 31 characteristics (Table ES-1). Each of the 31 characteristics was described in terms of four stages of maturity. Interviews were conducted on two occasions, in October 1997 and February 1998. For the first interviews, the committee was divided into three teams based on the assessment categories and each member's background and experience. In the second set of interviews, the committee focused on the technical capabilities of the two groups and the effects of the Research and Technology Directorate (RTD) management on the research programs. Because the Statement of Task required the committee to focus on quality, the committee has commented extensively on the effectiveness of the quality management programs of the two groups at the time of the interviews. The topics listed below are some of the aspects of quality management touched on in this report: identifying customers and customers' needs focusing on the mission sharing information

OCR for page 1
--> TABLES ES-1 Categories and Characteristics of the Assessment Model ASSESSMENT MODELa Customer Focus Value Creation Customer Customer satisfaction Customer involvement Market diversification Portfolioc selection Cycle time and responsiveness Value of work in progress Resources and Capabilities Quality Focus Organizational culture Employee attitude People development Budget/funding RD&E capabilities, skills, and talents Intellectual property Technology sourcing Information technology Facilities and infrastructure Capacity for breakthroughs Continuous improvement Teams Evaluation and rewards Project management Regulatory compliance Commitment to quality Process management Metrics Safety Knowledge and Learning Strategic Vision   Mission and vision Strategic planning Stakeholderb buy-in Leadership   a See also NRC, 1996. World-Class Research and Development Characteristics for an Army Research, Development, and Engineering Organization. Washington, D.C.: National Academy Press. b Stakeholders refer to any group that has a stake in the success of the organization. In the case of the mass spectrometry and bioremediation groups, the stakeholders include employees, ERDEC and CBDCOM management, members of the "Executive Panel," scientific peers, etc. c The term "portfolio" refers to the collection of research projects of an organization. teamwork at all levels using performance metrics benchmarking (learning from others) using project management establishing a corporate culture that fosters quality management

OCR for page 1
--> Management is responsible for ensuring that the program focuses on quality at all levels of the organization. Although the primary purpose of this study is to review, the committee also suggests ways both groups could improve. The committee members pooled their experience and expertise to prioritize the areas of each program that need management's attention. Each program was described in terms of the 31 characteristics of the Assessment Matrix, and each characteristic was rated on a maturity scale of 1 to 4 and an "importance" scale of 1 to 4 (1 = low). The characteristics were then prioritized in terms of the need for improvement. Priorities were expressed on a scale of 1 to 12 (1 = low) based on a combination of the maturity and importance measures. Mass Spectrometry Findings Although the mass spectrometry group is progressing toward its goals of identifying biomarkers for bacteria, spores, and proteins, is working on protein biomarkers using electrospray ionization methods, and is developing a battlefield CW detector, improvements could be made in several areas. First, the committee was concerned about the lack of mission focus of the portfolio of projects. The selection of projects seems to have been driven by the technical interests of the scientists rather than by the mission of the RTD. Second, the mass spectrometry group could improve its internal communications and teamwork. The committee found that the three mass spectrometry subgroups had a competitive, rather than cooperative, attitude toward each other. Thus, they worked in relative isolation from each other and did not share ideas or results. Third, the program has highly sophisticated equipment but not enough qualified technicians to operate it. Fourth, metrics are not used to assess progress. Fifth, in 1997, an independent "Executive Panel" of 23 specialists in mass spectrometry drawn from government and academia made several recommendations for improving the ERDEC's mass spectrometry program. The committee noted that the mass spectrometry group has not followed these recommendations. Finally, the committee is concerned about the fundamental question of whether mass spectrometry is the best way to detect and identify biological agents. The committee notes that there are other methods that should be investigated, such as immunological methods and other antibody/antigen-based systems.

OCR for page 1
--> The following characteristics of the mass spectrometry group received the highest priority score of 12 indicating that they need the most urgent attention: Commitment to Quality, which encompasses training, focus, dedication, and the many activities espoused in the Baldrige National Quality Award1 and the ISO2 criteria; Metrics, which includes measures for evaluating the progress of an organization in key functions; Leadership, the ability of management to create an atmosphere of commitment throughout the organization; and Organizational Culture, an atmosphere in which people are valued and respected for their skills. Conclusions Conclusion 1. The subgroups of the mass spectrometry group were not able to identify their customers or their customers' needs. Conclusion 2. The work of the mass spectrometry group is not cohesively focused on its mission. Conclusion 3. The three mass spectrometry subgroups work in relative isolation from each other and do not regularly share ideas or results. Conclusion 4. The mass spectrometry group does not make full use of its sophisticated, expensive equipment. Conclusion 5. The mass spectrometry group does not use performance metrics to evaluate continuous improvement. Conclusion 6. The priorities and recommendations of the Executive Panel have not been followed. Conclusion 7. Mass spectrometry may not be the best system for detecting and identifying biological agents. 1   The Baldrige National Quality Award was established by Congress in 1987 to recognize U.S. companies for their achievements in quality and business performance and to raise awareness of the importance of quality and performance as competitive advantages. 2   The ISO 9000 series of International Standards for quality management and quality assurance has been adopted in more than 90 countries and is being implemented by thousands of manufacturing or service organizations, both public and private.

OCR for page 1
--> Recommendations Recommendation 1. The leadership of the mass spectrometry group should focus the work of the overall program by defining the mission and goals, setting scientific priorities, ending internal competition, and promoting cooperation, including data sharing. Recommendation 2. The management of the Research and Technology Directorate should bring all three subgroups of the mass spectrometry group into one laboratory complex. Recommendation 3. As soon as possible, the senior management of the Research and Technology Directorate should evaluate, with the assistance of outside experts, the effectiveness of mass spectrometry for detecting and identifying biological agents compared to other approaches, such as immunological methods, polymerase chain reactions, and other antibody/antigen-based systems. In addition, senior management should seek an independent evaluation of the recommendations made by the Executive Panel. Recommendation 4. The mass spectrometry subgroups should interact closely with other groups working with polymerase chain reactions, flow cytometry, and other immunological detection systems to compare the performance levels (e.g., analytical specificity and sensitivity) of these technologies. If mass spectrometry does not perform at an equal or higher level, the mass spectrometry group should develop a program that would meet the highest performance requirements. This program should then be evaluated by the management of the Research and Technology Directorate for its feasibility. Recommendation 5. The scientific capabilities and research projects of all three mass spectrometry subgroups should be evaluated in relation to the recommendations of the Executive Panel to help management make optimum assignments. Once priorities have been established for the mass spectrometry group, tasks should be assigned to the subgroups according to their scientific capabilities. Recommendation 6. The sophisticated, expensive equipment should be fully used. Recommendation 7. The mass spectrometry group and management of the Research and Technology Directorate should develop and implement performance metrics to measure progress.

OCR for page 1
--> Bioremediation Findings The esprit de corps in the bioremediation group was high among both employees and contractors. The quality of leadership was also high, and information between management and staff flowed in both directions. An air of excitement pervaded the laboratory, risk-taking was encouraged, and leads were quickly pursued. Nevertheless, several areas could be improved. First, portfolio selection appears to be technology driven rather than mission driven. Also, there has been little input from the intelligence community about potential new threats that could be useful for mission-oriented project selection. Second, the bioremediation group has no formal process for the selection of, and techniques and criteria for, outsourcing projects. Third, scientists have no opportunity to learn about related work being done in other parts of the RTD. Fourth, principal investigators receive no feedback during the proposal funding process. In other words, no explanations or critiques are offered explaining why proposals are not chosen for funding. Finally, annual Technology Base Reviews could be used to improve communication among technology teams. The highest priorities for management attention are Commitment to Quality, which encompasses training, focus, dedication, and the many activities espoused in the Baldrige National Quality Award and ISO criteria; and Metrics, which includes measures for evaluating progress in key areas. Conclusions Conclusion 1. The quality of leadership in the bioremediation group is high. Information flows in both directions between management and staff, and an air of excitement pervades the laboratory. Esprit de corps is shared by both employees and contractors. Conclusion 2. Although the portfolio seems to meet the strategic goals of the group, the overall program appears to be technology driven rather than mission driven. Conclusion 3. The bioremediation group does not have a formal process for selecting research to be outsourced (i.e., funded at a university, other government laboratory, or corporate laboratory) or established criteria for making outsourcing decisions.

OCR for page 1
--> Conclusion 4. Project management, including commitment to quality and metrics, could be greatly improved. For example, an adaptation of the stage-gate model might be used. Conclusion 5. The bioremediation group does not have opportunities to work with related groups in the Edgewood Research, Development and Engineering Center. Conclusion 6. The bioremediation group is given little or no feedback on the reasons technical proposals are funded or rejected. Conclusion 7. The bioremediation group is not kept up to date on intelligence estimates of biological and chemical agents being developed by potentially threatening groups and countries. This lack of information interferes with the selection of mission-oriented projects. Recommendations Recommendation 1. The portfolio of the bioremediation group should be linked to the larger mission of the organization. This can be facilitated by frequent intelligence briefings on biological and chemical agents that are being developed or are already in the arsenals of foreign countries. Recommendation 2. The bioremediation group should establish a more formal process for making decisions on outsourcing and for managing outsourced projects. Recommendation 3. The bioremediation group should establish a formal project management process to facilitate learning and improve efficiency. Recommendation 4. Management should internally publish all funded proposals and critiques and should provide principal investigators with critiques of unfunded proposals. Recommendation 5. The leadership of the Research and Technology Directorate should use annual Technology Base Reviews for sharing information among all technology teams.

OCR for page 1
--> Management Findings The committee found no evidence that the project portfolios of the mass spectrometry and bioremediation groups are coordinated by an overall strategy. Information is not exchanged freely among the staff. In the mass spectrometry group, contractors and government staff are both dissatisfied with their working relationships. Senior scientists are handicapped by having to perform tasks that could be performed by technicians. Finally, the committee believes that long-term basic research cannot be conducted effectively by either of the two groups because the time limitations imposed by annual funding and the corresponding procedure favor short-term projects. Conclusions The following conclusions, which pertain to both groups reviewed by the committee, may also apply to the RTD at large. Conclusion 1. The corporate cultures of the mass spectrometry and bioremediation groups are not conducive to a free exchange of information among the staff. Conclusion 2. Quality management principles have not been incorporated into the everyday operations of the mass spectrometry or bioremediation groups. Conclusion 3. The project portfolios of the mass spectrometry and bioremediation groups are not well balanced and are not coordinated by an overall strategy. Conclusion 4. Both contractors and government staff in the mass spectrometry and bioremediation groups are dissatisfied with their working relationships. Conclusion 5. Because the time limitations imposed by the annual funding mechanism and the corresponding procedure favor the selection of applied research projects, neither the mass spectrometry nor the bioremediation group can effectively conduct long-term basic research.

OCR for page 1
--> Recommendations Because these conclusions and recommendations are based on a small sample, the committee strongly suggests that the management of the Research and Technology Directorate review other groups to determine if the following recommendations apply to them. Recommendation 1. Senior management should ensure that cultural changes are made that support quality management. Recommendation 2. Senior management should establish a strategy, set goals, and define the mission for all of the laboratories. Coordinated portfolios of projects should be based on the needs of the customer, the overall strategy, and the mission of the Research and Technology Directorate, as well as on the resources and capabilities available both in-house and from outside contractors. Recommendation 3. Management should encourage a spirit of teamwork between contractors and government staff; investigate types of contracts that allow better day-to-day management of contractors on site; ensure that both government staff and contractors are completely familiar with the rules under which they both must work; and eliminate unnecessary obstacles, perceived or actual, to the contractors doing their jobs. Recommendation 4. Senior management should consider hiring more technicians to leverage scientists' and technicians' singular capabilities. Recommendation 5. The Research and Technology Directorate should de-emphasize in-house, long-term, basic research and should use these funds to sponsor research with universities or transfer projects to external Army agencies that are more clearly associated with long-term basic research. The organization could then focus on practical applications of new technologies. (This recommendation may not be applicable to all groups but should be considered in future evaluations of the organization.)