Review of Mass Spectrometry and Bioremediation Programs of the Edgewood Research, Development and Engineering Center (1998)

The mass spectrometry group, also called the biological point detection group, is part of the Chemical and Biological Point Detection Team, which is one of approximately 24 teams that report to the director of the Research and Technology Directorate of the ERDEC (Figure 2-1). The mass spectrometry group is not formally organized and does not have a separate leader but is composed of government principal investigators (PIs) and government and contractor staff. The group consists of nine, people, four government employees and five contractors who are divided into three subgroups, each consisting of a PI and one or more staff members. The PIs report to the leader of the Chemical and Biological Point Detection Team. These subgroups will be discussed in greater detail in the Technical Capabilities section.

In 1997, a group of 23 specialists in mass spectrometry from government, industry, and academia (including six from the ERDEC) were invited by members of the mass spectrometry group to recommend priorities for future work in mass spectrometry. This "Executive Panel," chaired by Professor Donald F. Hunt, University of Virginia, met on July 31 and August 1, 1997. The panel was asked how the mass spectrometry group, by the year 2000, could best meet the goal of developing a small, mobile, reliable, fully automatic, quick-response, unmanned, remotely controlled, hardened chemical biological-mass spectrometry unit for use on the battlefield. The panel established the following priorities based on the limited funds and short time available:

1. Use proteins as "biomarkers."
2. Use electrospray ionization (ESI) to ionize the analytes and MS/MS for confirmation.

6. Explore matrix-assisted laser desorption/ionization (MALDI) as a possible long-term solution in place of ESI.

The mass spectrometry group is investigating two technologies, mass spectrometry and, to a much lesser extent, ion mobility spectrometry, to determine their feasibility for detecting biological warfare agents in the field. The group is using several ionization techniques to develop a library of biomarkers for biological agents. Using the MALDI technique, the group has identified biomarkers for more than 90 of approximately 100 bacteria, spores, and proteins, both pathogenic and nonpathogenic. The group is now beginning to work on protein biomarkers using ESI-MS/MS methods.

The mass spectrometry group also participates in joint field trials to verify technologies and methodologies for sample extractions and sample preparations for standardizing the spectrum with ionization and pyrolysis techniques.

All of the subgroups in the mass spectrometry group use mass spectrometry technology, but each has a different approach to the overall mission, namely improving the methodology for chemical-biological point detection. The work of the subgroups is discussed below.

The objective of the first subgroup is to determine whether analyses of the thermal degradation products of spores, bacteria, viruses, and proteins can be used as biomarkers. BG spores, B. subtilis, Erwinia herbicola, MSII, and ovalbumin are being used as simulants for biological agents of a limited number of compounds or agents on DOD's threat list. The subgroup is attempting to improve the pyrolysis of these materials to limit their thermal degradation and produce larger fragments (mass 600 and higher) of the parent materials. When the larger fragments are then analyzed in a mass spectrometer, the resulting mass spectra should provide a more characteristic chemical fingerprint of the parent compound than conventional pyrolysis, which causes much more severe degradation and smaller fragments (mass 150 and lower).

The larger fragments are produced by quickly flushing the material out of the pyrolysis zone using air rather than helium as the flushing gas, which

makes the system more suitable for use in the field. This "gentle pyrolysis" method also makes it possible to detect dipicolinic acid (pyridine-2,6-dicarboxylic acid), a major component (~10 percent by weight) of the cell wall of spores, directly rather than after degrading it to the less characteristic pyridine. These improvements are expected to increase the sensitivity and specificity of the pyrolysis-mass spectrometry compared with the current chemical and biological agent mass spectrometry.

The thermal degradation subgroup is also investigating the use of phospholipids (biochemical constituents of bacteria) as biomarkers detectable by mass spectrometry. These experiments are being done with a commercial, laboratory-sized, triple quadrupole mass spectrometer equipped for both atmospheric pressure ionization (API), including atmospheric pressure chemical ionization (APCI), and ESI (electrospray ionization). In the future, these experiments will be carried out with a Finnigan LCQ mass spectrometer, which is more modern and more compact and uses an ion trap (the mass analyzer favored by the independent Executive Panel).

Two subcontractor personnel and one government employee are working with the subgroup leader, who also collaborates (via contract) with experts in two other laboratories, Pacific Northwest Laboratory and the Beckman Research Institute, City of Hope National Medical Center, to improve sample preparation prior to injection of the sample into the pyrolysis zone. One microbiologist (a contractor) works at the Fort Detrick facilities to prepare quantities of virulent agents that are then killed and sent to the ERDEC to be used as "real" samples rather than the nontoxic surrogates.

The second subgroup's objective is to develop laser-based approaches to agent detection. Presently, MALDI (matrix-assisted laser desorption ionization) is used to assess the reproducibility of the process for detecting proteins released during the lysis of cells. Mass spectra (representing signals corresponding to the molecular weights of the proteins present in the lysate) vary greatly, depending on the history of the sample (e.g., growth conditions, such as nutrient, time of growth, and pH). The spectra have been reproduced, under highly standardized conditions, only at the University of Alberta, with which this group collaborates. The experiments are carried out using various time-of-flight (TOF) mass spectrometers. The results of the MALDI approach are expected to be equivalent to those obtained with the ESI methodology using an ion-trap mass spectrometer, which was recommended by the Executive Panel.

The work of this subgroup has been severely limited by shortages in manpower because the leader spends about half of his time in another capacity and has only one co-worker (a contractor).

The objective of the third subgroup is to use MALDI (and ESI in the future) to obtain mass spectrometric fingerprints of bacteria by direct mass spectral analysis with the aim of identifying selected materials on the DOD threat list. A library of MALDI mass spectra has been generated from approximately 100 different pathogens, nonpathogens, and pure, known proteins. The spectrum of an unknown material is matched to the library using an algorithm developed under contract at the University of Washington, Seattle. In January 1998, this strategy was tested in blind studies at Dugway Proving Ground. For this purpose, the laboratory MALDI mass spectrometer was shipped from the ERDEC to Dugway to be used on site by the subgroup leader and the one (and only) co-worker, a contractor who began work about December 1, 1997. Because of the time consumed in the preparation for and participation in these tests, the subgroup leader had no time in 1997 to compile and evaluate the results of the work done in 1996.

The 1997–1998 Dugway tests involved the MALDI methodology with a TOF mass spectrometer rather than the ESI ion-trap methodology recommended by the Executive Panel. It appears to the committee that the older technology was used according to an established plan, without regard for the recommendations that the methodology be changed. The committee feels that both time and money would have been saved by canceling the test. Furthermore, if the 1997/98 tests are handled the way the 1996 tests were, it will be some time before the results are released.

This subgroup is very well equipped, in fact, it is overequipped. The group now works with ESI on a commercial spectrometer (Finnigan LCQ), which is being modified for very small samples and low volumes of liquid. But overall, there are more mass spectrometers than can be used by the two full-time employees.

The second and third subgroups have been working on the same basic approach, namely the detection of certain micro-organisms by MALDI

mass spectrometry to identify characteristic proteins. However, because all or most of the microbial samples used by the third subgroup were single runs taken from petri dish cultures, the cells that were analyzed had probably all shifted to stationary-phase physiology. The pattern of protein expression was, therefore, probably quite different from the pattern in cells that would be produced by an adversary for battlefield delivery. The committee believes this subgroup should refocus its attention on generating extracts from cells grown under conditions likely to be used in the production of biological warfare agents. During the course of this review, the committee learned that another subgroup in the mass spectrometry group was already aware of the importance of the variability of protein expression, but because of the lack of information exchange and discussion, this important information was not available to the other subgroups.

One senior member of the mass spectrometry group was concerned that the recommendations of the Executive Panel might be adopted before technologies other than mass spectrometry had been considered. The committee notes that there are other possible methodologies, including immunological methods, polymerase chain reaction (PCR), and other antibody/antigen-based systems.

The sensitivity goals for any detection system were set by the Operational Requirements Document (ORD) at 100 cells/ml of water in the accumulator of the air sampling device (1,000 L of air/min). None of the three subgroups has paid much attention to the detection limit in terms of the number of detectable cells. The samples used are 2 to 4 orders of magnitude more concentrated than the ORD specifies. Nobody in the mass spectrometry group could provide the committee with the scientific basis for using 100 cells/ml or knew whether the biomarkers present in 100 cells would be sufficient to produce a reliable signal in any system based on mass spectrometry.

Another possible approach to the detection-identification problem is the German mass spectrometer for detecting chemical and biological warfare agents that was used on a limited basis during the Gulf War. The committee learned that this system all but failed the 1996 Dugway field tests. It was only able to differentiate proteins from nonproteins and spores from nonspores. An improved version of the instrument has been developed by Bruker-Franzen (Germany), but because of its shortcomings, Oak Ridge National Laboratory has been told to ''take it apart and fix what is wrong.'' The committee feels that, even though the German mass spectrometer did not work well in the 1996 Dugway field tests, the mass spectrometry group should monitor the progress of Oak Ridge National Laboratory's efforts. The lessons learned by Oak Ridge might very well help the RTD mass

spectrometry group progress with the development of a battlefield detector by the year 2000.

The committee is concerned that the recommendations of the Executive Panel have not been considered by the mass spectrometry group in setting its priorities. By the committee's February meeting, seven months had elapsed, and this information had still not been passed on beyond the team leader. Thus, it will be a long time before the panel's recommendations will be known to the rest of the DOD community, which should have an opportunity to comment on the recommendations and work toward the same goals. The committee also believes that the recommendations of the Executive Panel should be circulated outside the RTD for comment. This can be accomplished by writing to colleagues in respected mass spectrometry laboratories, convening a discussion group of knowledgeable individuals, or disseminating the Executive Panel's recommendation by any other means that would open them to discussion and examination. In the meantime, the committee believes other technologies that could detect agents better than mass spectrometry should be investigated. If these other technologies are found to be superior to mass spectrometry, the RTD should consider reducing or eliminating the mass spectrometry program. If these technologies are not found to be superior and the Executive Panel's recommendations are validated, the mass spectrometry group should reevaluate its priorities.

The committee reviewed responses to questionnaires and conducted interviews with members of the mass spectrometry group. Based on the written responses to the questionnaires, the committee learned that RTD employees and contractors held very different views. For the interviews, the committee was divided into groups by review category, and the interviewees moved from group to group. The interviewees were frank and open, and their responses were consistent.

The mass spectrometry group has the potential to make important contributions to the ERDEC's mission. The group has good instrumentation, and some imaginative members. Nevertheless, a reorganization of the program and the nurturing of a spirit of teamwork could have a positive effect on its esprit de corps and productivity. The following sections contain

highlights of the committee's assessment. Detailed descriptions of all characteristics can be found in Appendix A.

Serious deficiencies were found in the Strategic Vision Category, especially the lack of management direction, or leadership. The management should coordinate the efforts of the subgroups toward the achievement of goals and the mission. At this point, it may be instructive to define the terms "mission" and "goals," as used by the committee. The mission is the long-term reason for the existence of the group. It seldom changes and is used to orient the work of all members of the group. Goals are short-term milestones toward fulfilling the mission. Based on the committee's interviews, the mission is to "improv[e] the methodology for chemical-biological point detection."

The mass spectrometry group has a number of goals, one of which is to develop the battlefield chemical biological mass spectrometry detector. Individual projects relate to this near-term goal of developing a battlefield detector, but are not coordinated from above. In general, the committee found that the PIs are pursuing their research in keeping with their individual concepts of the mission.

There were conflicting views as to the cause of this disorganization. Some interviewees told the committee that the group leadership was responsible because it did not have the expertise to understand the technical details of the scientific work. (The committee believes that, although the PIs must have expertise in mass spectrometry, the group leader does not need the same expertise. However, he should be knowledgeable in mass spectrometry, technology management, and strategic planning). Other interviewees were of the opinion that the PIs have "world-class" reputations and should not be micromanaged. In the judgment of the committee's technology experts, the PIs are competent and knowledgeable but not world-class experts, which the committee defines as individuals who are known and

respected throughout the world for major accomplishments in their fields. The committee concluded that neither opinion was correct.

The mass spectrometry group should be mission driven, and all of its work should be focused on achieving its goals by the shortest and most effective route. Some changes that might improve the situation are: coordination of the work of the subgroups to achieve the goals of the group; identification of customers and an understanding of their needs; implementation of better project management, perhaps stage-gate management, if applicable (Cohen et al., 1998). Stage-gate management prohibits a project from moving forward (passing through a gate) until a prespecified list of requirements has been satisfied. This management technique requires that a design be carefully planned up-front and then frozen, reducing the chances of costly changes part way through the project.

Although CBDCOM and ERDEC projects are focused mostly on the development of equipment and technologies, the mass spectrometry program also includes basic research. The committee's major concern in this category was the lack of focus of the overall portfolio of projects. The first priority of the PIs is that the work be interesting to them and that funds be available. The second priority is that the work further the broad mission of the group. This order of priorities does not usually contribute to the effective management of an overall program.

During the interviews, not all members of the mass spectrometry group were able to state their mission. The committee recognizes that the group is not a formal, separate team, but is part of the Chemical Biological Point Detection Team and, therefore, may not have a formal mission statement. Nevertheless, the members of the group must know their mission. A prominent example of the lack of focus is that the group is pursuing both long-term basic research and applied research, which require different training, different associations with both peers and customers, and sometimes different equipment and facilities. The committee believes that the

mixture of long-term basic research and applied research reflects a lack of focus, which could be improved by management using funding decisions to focus research projects on the mission.

Although the mass spectrometry program is making progress in focusing on its customer and the customer's needs, not all members of the group could even identify the customer. Nor has the group established a process for measuring customer satisfaction, which makes it difficult, if not impossible, to assess customer involvement and satisfaction. Generally, the scientists assured the committee that their customers were satisfied, and they cited the continued funding of their projects as proof. Nevertheless, the group seems to be reluctant, or very slow, to adopt ESI as the preferred ionization method and the quadrupole ion trap as the analyzer, even though the Executive Panel gave these two items a high priority. The committee observed that the PIs were equipped with a number of MALDI instruments but not ESI. Their work is thus involved in the former technology (except for one PI who uses pyrolysis GC-MS), and changing to ESI would require a major change in their experiments as well as new equipment. Only one PI has, in the meantime, acquired an ESI-ion trap (Finnigan LCQ) and is adapting it to work with very small sample volumes.

The Assessment Model (Appendix A) suggests that market diversification is beneficial to a laboratory. In general, such diversification promotes funding stability and long-term growth. The committee did not hear of any attempt by the group to diversify its customer base to include other military or national laboratories. The committee sees several potential customers for the mass spectrometry group, both internal and external. Even though the "end users," or soldiers, may not see the results of basic research for many years, their needs must set the goals. The proxies for soldiers are developers, testers, engineers, and applied scientists who incorporate the results of basic

research into their own work. The RTD management should prioritize these customers, as well as other stakeholders, such as scientific peers.

Based on information received during the interviews, the committee was disturbed by the lack of interaction among the mass spectrometry subgroups, which have adopted an attitude of protecting information from their colleagues. The committee found that many members of the mass spectrometry group have an independent, competitive attitude that stresses competition rather than a team-oriented, cooperative attitude. This has had a negative impact on the internal sharing of information and has been a barrier to productive research.

Some interviewees indicated that there are not enough qualified people to operate the state-of-the-art equipment. For example, the committee learned that a tandem mass spectrometer that cost at least $1.2 million has not been used for about 18 months. One solution would be to increase the authorization or funding to hire or train personnel. Another solution would be to increase the use of the equipment by outside groups, either government or nongovernment. In any case, the fact that the instruments are underused suggests that the procedures for acquiring appropriate skills need urgent attention.

During the second set of interviews, the committee detected a problem with high turnover but was not able to pursue this issue in sufficient detail to determine the cause or to suggest a possible remedy. However, the committee suggests that ERDEC and the RTD management assess the extent of the problem and develop ways to resolve it.

The committee could find no evidence that any performance metrics (see Table A-5) were being used, which suggests that the group is not totally committed to quality. A commitment to quality requires breaking down an operation into its processes (e.g., recruiting, training, planning, researching, observing, recording, etc.) and determining the steps in the process that add value, the steps that can be eliminated, and the issues that should be addressed. The committee found little evidence of this formal approach at the group or subgroup level.

The lack of attention to detail is reflected in the lack of attention to process management or project management. Although a group this small cannot, by itself, implement a Baldrige National Quality Award¹ or ISO 9000² program, high quality teams of this size are common. As some management experts have stated, in reference to project-oriented companies, "it is most effective to keep the groups small, four to fifteen members; about eight members is optimum" (Stephanou and Obradovitch, 1985, p. 329). A

stronger commitment to quality at the laboratory level might require the involvement of the RTD senior management, which has the responsibility for overall quality. (This subject is discussed further in Chapter 4).

The performance of the mass spectrometry group could be improved if a formalized project management process were implemented. The same principles of project management used to build a plant or manufacture a product can be applied to basic research projects. One example of project management at the basic research level (at Exxon) is described in an article by Cohen, et alia (1998). "A critical requirement for a productive centralized basic research unit is the achievement of seamless interfaces between it and decentralized functional laboratories" (p. 34). The authors state that, "The stage-gate system . . . provides direction for the success of science-based technology" (p. 37). Achieving a breakthrough requires teamwork, skill, adequate funding, and strong leadership. Personnel in the best laboratories are able to enumerate the breakthroughs they have made in the last five to ten years.

Breakthroughs need not be limited to basic research. The American Heritage Dictionary (3d ed.) defines a "breakthrough" as "1. An act of overcoming or penetrating an obstacle or restriction. 2. A major success that permits further progress, as in technology." The ERDEC has recognized the importance of breakthroughs in its criteria for the 1994 Department of the Army Research and Development Organization and Excellence Awards, in which breakthroughs are placed on the same level as "customer satisfaction," "meeting assigned objectives,'' ''technology product transitions," and "leveraging of industry, academia and services/agencies" (ERDEC, 1994). Clearly breakthroughs are desirable to the Army whether the group is doing basic or applied research. (For a description of a breakthrough, see "Capacity for Breakthroughs" under the Quality Focus Category, Table A-5.) The committee found very few examples of breakthroughs in the mass spectrometry program.

Evaluations and rewards was an emotional subject for the mass spectrometry group. Although numerous rewards were available, some employees stated that they did not understand the process by which rewards were given or the linkage between performance and rewards. Some interviewees also complained that financial rewards were meager. The financial rewards available for the mass spectrometry group, as reported by the interviewees, appear to be too small to have much effect in any case. Some of the interviewees felt that recognition by a respected person or organization would be more meaningful than a meager financial reward. On a more basic level, the committee is not convinced that financial rewards motivate quality work.

In many cases, attendance at a professional meeting has become a substitute for a financial reward. However, some interviewees suggested that attendance at meetings was not always granted for the best work (e.g., a person in a senior position may attend a meeting rather than a person whose work is more clearly related to the subject). Some interviewees felt that excessive attendance by seniors had interfered with work on some projects. This problem reflects on the RTD's procedures for retaining and developing research and engineering capabilities, skills, and talents.

Most members of the mass spectrometry group felt that safety was not an issue because they work with instruments and computers and are not exposed to hazardous environments. They believe they are exposed to the same dangers as secretaries. When the committee suggested the possibility of repetitive operations disorders, the interviewees responded that these hazards were not worth evaluating. Because of the absence of safety records, the committee could not evaluate the frequency of these disorders, which may occur but may not be recognized.

The three mass spectrometry subgroups do not willingly exchange scientific information because they consider each other competitors. Biweekly group meetings are largely concerned with management and funding issues rather than the exchange of results or scientific ideas. Also, annual or biannual program reviews with upper management are considered to be superficial and inadequate. This problem reflects on ERDEC's procedures to develop research and engineering capabilities, skills and talents. The prevalent culture, which discourages interaction among the subgroups, must be changed by subgroup, team, and, if necessary, RTD management. Informal meetings should be encouraged as forums for the exchange of information.

Based on the first interview session, the committee found that the mass spectrometry group could make improvements in all five categories. The committee then developed a priority index (see Table 2–1) to identify the characteristics that need the most improvement. The committee's most serious concern is the lack of metrics. Without metrics, progress cannot be measured, milestones cannot be defined, budgets and schedules cannot be controlled, and the risk of failure is increased. Although the use of metrics is only one part of quality management. it is important for quantifying customers' needs and ensuring that they are satisfied. For example, a customer describes a requirement for the development of a

system that can act in certain ways under certain circumstances. The PI then develops detailed metrics for measuring progress towards satisfying this requirement.

The lack of commitment to quality management filters down through the organization and has a direct impact on the performance environment of the groups and subgroups. According to the interviewees, the organization as a whole does not use performance metrics.

RTD management should establish and implement a strategy for focusing on quality management (see discussion in Chapter 4). RTD management should set an example of long-term cultural change, frequent follow-on help sessions to reinforce the serious intent of management to "stay the course," and other mechanisms that demonstrate a determination to change the institutional culture. (Daft, 1998 ). The ERDEC management could learn from the experience of other government agencies and the private institutions that have overcome destructive competitive atmospheres and established cooperative atmospheres. Successful institutions could also provide performance metrics, effective means for continuously improving processes. Finally, by observing other organizations, especially organizations similar to the mass spectrometry group, the RTD management might be able to find extremely good processes by which to benchmark their own progress.

The committee developed a priority index, based on maturity stage and importance, that identifies areas that need the most attention. The Priority Index column of Table 2-1 shows the characteristics that require immediate attention. The higher the priority index, the greater the need for attention. Expressed as an equation, Priority Index = (4-maturity) x importance. For example, in Table 2-1, the maturity stage of the second characteristic under Strategic Vision, strategic planning, is 2. The importance is 3. The priority index for strategic planning is therefore (4-2) x 3 = 6.

Conclusion 1. The subgroups of the mass spectrometry group were not able to identify their customers or their customers' needs.

Conclusion 2. The work of the mass spectrometry group is not cohesively focused on its mission.

Conclusion 3. The three mass spectrometry subgroups work in relative isolation from each other and do not regularly share ideas or results.

Conclusion 4. The mass spectrometry group does not make full use of its sophisticated, expensive equipment.

Conclusion 5. The mass spectrometry group does not use performance metrics to evaluate continuous improvement.

Conclusion 6. The priorities and recommendations of the Executive Panel have not been followed.

Conclusion 7. Mass spectrometry may not be the best system for detecting and identifying biological agents.

Recommendation 1. The leadership of the mass spectrometry group should focus the work of the overall program by defining the mission and goals, setting scientific priorities, ending internal competition, and promoting cooperation, including data sharing.

Recommendation 2. The management of the Research and Technology Directorate should bring all three subgroups of the mass spectrometry group together into one laboratory complex.

Recommendation 3. As soon as possible, the senior management of the Research and Technology Directorate should evaluate, with the assistance of outside experts, the effectiveness of mass spectrometry for detecting and identifying biological agents as other approaches, such as immunological methods, polymerase chain reactions, and other antibody/antigen-based systems. In addition, senior management should seek an independent evaluation of the recommendations made by the Executive Panel.

Recommendation 4. The mass spectrometry subgroups should interact closely with other groups working with polymerase chain reactions, flow cytometry, and other immunological detection systems to compare the

performance levels (e.g., analytical specificity and sensitivity) of these technologies. If mass spectrometry does not perform at an equal or higher level, the mass spectrometry group should develop a program that would meet the highest performance requirements. This program should then be evaluated by the management of the Research and Technology Directorate for its feasibility.

Recommendation 5. The scientific capabilities and research projects of all three mass spectrometry subgroups should be evaluated in relation to the recommendations of the Executive Panel to help management make optimum assignments. Once priorities have been established for the mass spectrometry group, tasks should be assigned to the subgroups according to their scientific capabilities.

Recommendation 6. The sophisticated, expensive equipment should be fully used.

Recommendation 7. The mass spectrometry group and management of the Research and Technology Directorate should develop and implement performance metrics to measure progress.