Assessment of the U.S. Army Natick Research, Development, and Engineering Center (1997)

In this chapter the committee focuses on the two support directorates of the Natick RDEC, STD (Science and Technology Directorate) and ASCD (Advanced Systems Concepts Directorate).¹ The committee again used the pillars, characteristics, and metrics established in phase-one for world-class performance. However, the characteristics and metrics were applied in a way that emphasized the supporting roles of these directorates. The assessment results reflect the committee's judgments of the contributions of the support directorates to the performance of the commodity directorates and the RDEC as a whole.

The committee decided early on not to subject STD and ASCD to the arduous assessment process undergone by the commodity directorates. The committee submitted fewer pre-visit questions and conducted fewer interviews of personnel in the support directorates. This approach recognized that STD and ASCD were created to complement the commodity directorates and the RDEC by (1) providing research to infuse new technology into the RDEC's products, and (2) by marketing products to external users. Therefore, in applying the characteristics and metrics of a world-class organization, the committee heavily weighted the supporting role of STD and ASCD in relation to the commodity directorates, their primary customers.

The committee gathered data for assessing the support directorates on the same visits to Natick. During the wrap-up session in February 1997, the committee scheduled additional interviews to ensure that sufficient information would be available for assessing the support directorates. The committee also obtained a great deal of information about the support directorates from personnel in the commodity directorates (i.e., the primary customers). However, members of the committee also interviewed people in the support directorates, who made many cogent observations regarding STD's and ASCD's abilities to provide needed support.

The committee considered each support directorate in relation to each characteristic of world-class performance. However, the committee believed it had sufficient information only to make summary judgments at the pillar level regarding the performance and major strengths and weaknesses of STD and ASCD. In making these judgments, the committee interpreted the various metrics in light of the relationship between the support directorates and the commodity directorates. Because of the fundamental differences between the support directorates, the committee decided to assess them separately.

The Natick RDEC must, at a minimum, maintain a research capability and continue to develop some technology to ensure that the RDEC personnel can make credible scientific and technical judgments. But the research and technology development must be relevant and must offer significant benefits to soldier-system performance. These considerations were important to the committee's assessment of STD's impact on the overall performance of the commodity directorates, the ASCD, and the Natick RDEC as a whole.

STD's customers' indicated that their expectations were seldom exceeded by STD. When research tasks were Well defined, their expectations were met; however, if tasks were less well defined, customers felt that the deliverables were either not of high enough quality or were delivered late. These problems may not be entirely attributable to personnel quality, but they indicate a disparity that should be addressed by acquiring skills or training in areas more compatible with customer expectations.

The committee was informed repeatedly that because of the hiring freeze STD had been forced to try to improve personnel quality by training and personnel transfers. The committee was informed about several training activities and programs but found that they were often pursued by employees for personal interests. The committee would have been pleased to see efforts being made to encourage and train current personnel or to obtain transfers of personnel with appropriate expertise to improve support for the commodity directorates. Unfortunately, efforts of this kind were not observed.

The committee was informed that STD is constantly struggling for money. STD's funding comes from two basic sources: the commodity directorates and other government organizations that need the type of research that STD can perform. Both sources require that STD justify those funds. Therefore, budgets are directly related to STD's ability to convince its customers that its research capabilities and services are worth funding.

STD budget shortfalls reflect doubts on the part of the DOD and the Army, as well as customers within the RDEC, about the benefits that will be realized from their investment in research. The committee believes that budget problems will persist. However, conflicts within the RDEC could be reduced by a much closer working relationship between STD and the other directorates and by STD accepting the need to demonstrate the benefits of its work to RDEC projects and soldier-system performance.

This characteristic reflects the broad capabilities of an organization to pursue a viable research program. Several interviewees contributed to the committee's assessment of this characteristic. For example, one person indicated that the hiring freeze is "crippling STD." Another interviewee, well known for his work, predicted that the best people will soon leave the Natick RDEC. The committee agrees that STD will probably incur losses, as most organizations do when budgets and resources are limited. The committee noted the departure of one key scientist from STD whose reasons for leaving included the inability of the organization to hire new people who could bring new capabilities and ideas, which are necessary for a thriving research organization; decreasing funds for equipment and facilities, which limits future work to present technological capabilities; and cutbacks in research by government organizations in favor of research by private industry.

STD leverages outside research programs in both academia and private industry. STD has organized its external multidisciplinary research into several technical areas: biodegradable microbiology and chemistry for bioremediation; biotechnology for materials application; biomimetic processing; and functionally tailored textile fabrics. In each of these areas, STD collaborates with one or more academic institutions and one or more research organizations in DOD (e.g., the Army Research Laboratory, the Edgewood RDEC, the Office of Naval Research, and the Air Force Office of Scientific Research). STD also has cooperative agreements with private organizations (e.g., agreements covering various biodegradable materials and synthetic polymers that could mimic the properties of spider silk).

The committee was informed that a lot of work is now being given to outside contractors because of the hiring freeze, thereby establishing connections with external resources. Although the range of external work is impressive, the committee is concerned that there is no master plan showing how external research complements the internal research by STD to fulfill the RDEC's needs.

Research at the Natick RDEC is divided between the commodity directorates, which conduct research that is uniquely applicable to their programs, and STD. STD maintains oversight of all of the basic research² at the RDEC, but the commodity directorates manage the day-to-day activities and progress of the research they conduct. The committee feels that the division of research elements amplifies the perception of a separation between researchers in STD and developers and engineers in the commodity directorates. In fact, information from the interviews implied that STD is preoccupied with its own research and has a tendency to pursue research for its own sake. The committee realizes that STD must focus its efforts on a few select technologies (e.g., the four thrust areas [identified in Chapter 1], as well as external initiatives) in order to achieve critical mass. However, because STD obtains funding from its primary customers—the commodity directorates—it must also ensure that technologies critical to the success of the commodity directorates take precedence. The RDEC's future depends on the maturation of technologies that can be applied to products.

The committee observed that STD and the other directorates were reluctant to engage in a frank dialogue about research programs. The commodity directorates need to clarify their research needs that have not been met by STD. Scientists in STD appeared to resent demands that they produce short-term results and argued that long-term results worthy of publication and patents require patience.

The committee accepts the fact that mechanisms like the planning integration team can assess the merits of research proposals and allocate research funding for the RDEC. Nevertheless, in the case of basic research, the committee believes there must be a closer relationship between STD and the commodity directorates. STD management and personnel indicated that important technologies that need to be added include work in materials science, human performance, and modeling and simulation (see discussion under the Value Creation pillar). The committee learned that only one-fourth of the proposals submitted by STD were funded. Better dialogue with the commodity directorates could improve the selection rate.

As the RDEC corporate laboratory, STD should be the center of excellence in understanding as well as in advancing critical technologies for the RDEC and the Soldier Systems Command. To this end, the committee considered ways STD could determine the technologies that are essential to the RDEC's future.

The committee suggests that STD and the commodity directorates agree to a closer working arrangement so important technological needs can be better communicated by the commodity directorates to STD and so STD can make fundamental changes in its approach to selecting technologies that deserve resources. This working arrangement was discussed during committee interviews. For example, one scientist's opinion was that research organizations must have an overall motivation, an overriding goal, and justification for what they do. There has to be accountability along the way (e.g., yearly technical reviews of research programs should be conducted by a team that collects, analyzes, and discusses data and how they fit the long-term goals of the organization). Some fringe ideas that could result in breakthroughs should also be funded. A certain percentage of funding (e.g., 80 percent) should be subject to review and approval by the commodity directorates, and a smaller percentage (e.g., 20 percent) should be provided for the research organization to direct on its own. The committee notes that it does not necessarily endorse this opinion but believes it offers reasonable options.

The committee believes that a formal, periodic review process would ensure that the commodity directorates could clarify technologies important for their needs and would give STD an opportunity to identify beneficial long-term technologies for investment within a small percentage of its budget. STD leadership must clarify which research projects are directly responsive to the

commodity directorates and which have been identified as beneficial by STD and ensure that the STD personnel maintain a balance between the two.

Some interviewees indicated that STD is not a part of the big picture; that the Soldier Systems Command does not understand research and supports administrative organizations that do marketing rather than scientific work; that government rules and regulations, which govern the commodity directorates, are different from the approach for performing basic research and force STD people to spend time on shorter-term acquisition rather than furthering a longer-term vision; that the RDEC does too much crisis management; that public relations are often substituted for substance; and that there are few opportunities for advancement. The committee suspects that the organizational climate in which STD is immersed will not improve until STD programs show more relevance to the commodity programs and soldier-system performance.

STD uses modem information technology to support research at the RDEC. Interviewees indicated that there is RDEC-wide cooperation in the use of information technology.

The committee observed that some renovations are being undertaken at Natick. Nevertheless, evidence suggests that the research facilities are barely adequate. For example, although personnel are pleased with the improved work space, they cited several facilities that were in disrepair and instances when necessary facilities or equipment for specific types of research were not available.

The general findings related to each of the characteristics and review of the associated metrics led the committee to an overall judgment of adequate for the Resources and Capabilities pillar. Indications are that there is a downward trend in STD's ability to maintain its RD&E capabilities, skills, and talents. This problem is similar to the problems the committee found for the commodity directorates (see Chapter 2).

STD's mission is to ''provide world-class research ... maximize combat effectiveness of [the] soldier system ... leverage outside research programs ... [and] maintain cutting edge research programs, facilities, and equipment'' (Salant, 1996). This statement reflects a desire to provide a wide range of scientific expertise in support of the commodity directorates and Soldier Systems Command and is in alignment with the mission and vision of the Natick RDEC. However, interviews conducted by the committee indicated that there is a dichotomy between the understanding and the implementation of this mission by STD personnel.

Although many STD personnel can relate to the vision and mission statements of the Soldier Systems Command and the RDEC, they do not appear to believe the STD's vision and mission statements. Some members of STD consider the commodity directorates to be their customers; others feel that basic research is a mission in and of itself and that the research activities in STD are worthy of support without application, short-term or long-term, to a current program. This difference in opinions about the fundamental role of STD was also reflected in interviews with commodity directorate personnels. Many interviewees in the commodity directorates found it difficult to name technologies developed by STD that have been, or are being, applied to products. They also indicated that key areas were not being researched by STD (e.g., research to assist MobD).

The committee concluded that the vision and mission statements are not helping STD to deliver research that consistently advances the commodity directorate programs or is demonstrably relevant to soldier-system performance. The committee believes that STD and the other directorates must define and focus on the scientific and engineering base critical to the RDEC. STD personnel should understand the areas for which they will be held responsible and identify the technologies for which the commodity directorates expect excellence from the STD.

The committee was briefed on recent formal strategic planning by the Scientific and Technical Advisory Council, which is directly under the office of

the RDEC director. This council and the planning integration team are intended to guide STD's planning and research. STD personnel are expected to develop research plans for STD and for the RDEC as a whole. Although some interviewees felt that the new strategic planning process was a significant improvement over the previous process, others felt it was not effective because it does not emphasize longer-term science and technology, for which the return on investment is 10 to 20 years away.

During interviews with members of STD, the committee perceived that the responses of internal stakeholders to the vision and mission of STD were either indifferent or negative. Stakeholders in the other directorates indicated to the committee that the results of STD research do not justify their investments. The committee concluded that either the return on investment in STD research is inadequate or it is not being clearly communicated to the RDEC community by STD.

The committee found that STD leaders thoroughly understand the RDEC mission and vision to support the soldier system. They stated clearly that the vision and research plan for the STD should be compatible with those of the RDEC and Soldier Systems Command, and they acknowledged that STD was obligated to justify the contribution of its research to the current commodity directorates. However, some of the researchers expressed their displeasure with having to justify their work on the basis of its relevance to development and engineering projects in the commodity directorates. Several interviewees appeared to be suspicious of, rather than in agreement with, the leadership of STD. The committee concluded that communication between the STD leadership and the staff was inadequate and that senior leadership has not been able to "sell" the vision and mission of STD to rank-and-file members of the directorate.

The committee arrived at an overall judgment of poor for the Strategic Vision pillar. Unfortunately, the committee found that STD has been unable to translate its vision and mission into a working philosophy that is accepted, let alone embraced, by the members of STD or the other directorates. The trend for the future was judged to be level.

Considering that existing research in STD is not well correlated with the commodity directorates' needs, the potential for breakthroughs from STD research is limited. Furthermore, as some interviewees within STD noted, the technologies associated with some commodities (e.g., food and clothing) are already mature, and only incremental advances can be expected in those areas. The most frequently cited example of a "leap-ahead" technology was in the area of biotechnology (the cited project involves developing materials with the properties of spider silk). STD personnel continue to publish papers and make steady, incremental improvements in their knowledge base.

STD personnel interviewed by the committee indicated that STD is not very flexible or capable of change; this inflexibility has contributed to STD's long-standing inward orientation. Interviewees noted, for example, that STD had not taken advantage of the RDEC's being designated as a personnel demonstration, which allows the organization to waive many civil service rules to find a better way of doing business. The quality of work at STD is assessed by STD managers and personnel using the same measures used by the academic community to assess research quality (e.g., number of publications in refereed journals, number of sessions chaired at national and international technical meetings, number of patents granted). But no measures are in place to determine how the output relates to the needs of the commodity directorates. The committee was informed that STD staff recommendations for improvements are considered only within STD, with no feedback from other directorates.

The deep division between STD and other RDEC organizations is of significant concern to the committee. Bringing STD closer to the commodity directorates should bring the research program more in line with the commodity directorates' needs and enhance the value of STD to the entire RDEC.

The committee was convinced in the interviews with STD personnel that they were committed to the highest quality in basic research. However, they judged the merits of their research on a traditional basis and did not consider the effects of their research on the commodity directorates. Several interviewees indicated that their ability to conduct high-quality research had been significantly affected by the hiring freeze, which has made it impossible to hire sufficient support staff and eliminated the influx of fresh ideas. Many interviewees were deeply troubled by the reduction in hands-on research, indicating that they were increasingly being forced to manage and administer external research projects rather than perform or supervise the research themselves. Attempts to offset the hiring freeze by using temporary technical personnel were viewed with skepticism; many felt that hiring temporary personnel compromised the organization's inherent capabilities and made it more difficult to maintain continuity. As a consequence, many interviewees said, the quality of research at the Natick RDEC was declining. Other interviewees, primarily senior scientists, stated that the number of publications and speaking invitations at international seminars indicated that the quality of research was still high.

STD leaders told the committee that they evaluate projects and personnel according to five performance objectives: well planned and executed research; demonstrations of scientific stature by editing and writing; safe research practices; collaboration; and leveraging technology and securing funding. Overall, research is monitored to control cost and to assess progress toward milestones.

Funds are available for STD personnel who desire to pursue advanced technical degrees or continuing education courses. However, the committee noted that STD does not have a uniform policy to encourage individuals to pursue opportunities germane to STD's mission, which could amplify and strengthen the current skill base. In addition, successful interactions with the commodity directorates that would motivate cooperative research programs with the remainder of the RDEC are not communicated to the rest of the RDEC. The committee feels that, as a minimum, STD should mount an organized campaign to encourage STD personnel to acquire knowledge in the technologies most important to the commodity directorates.

The committee has already noted the "disconnect" between STD's chosen areas of research and the commodity directorates' expectations. When STD has been asked to fulfill precise research needs by the commodity directorates, STD has performed well. In the research areas chosen solely by STD, STD's work has been recognized by the greater scientific community. But STD's research and technology programs are only occasionally correlated with the needs of its primary customers. The committee commends STD for maintaining the quality of its research vis-à-vis the community of its peers, but its activities should be relevant to the development and engineering needs of the commodity directorates.

The committee arrived at an overall judgment of poor to adequate for the Quality Focus pillar. Improvements will require that STD refocus its activities to support the commodity directorates. The committee is not advocating that STD be relegated to making short-term fixes, but STD should base its long-term programs on issues that are germane to the future needs of the commodity directorates.

The committee was informed by all of the commodity directorates that, with the exception of the support provided by STD's behavioral sciences group to SusD, they do not feel the amount or quality of STD research justifies the cost of supporting an independent STD. To compensate, all of the commodity directorates have mounted their own research projects to provide the information they need to facilitate their development and engineering projects. For example, the shelters area needs information on fabrics and composite materials and has asked STD for support. However, because STD did not start a relevant research program, SusD has initiated its own research in this area.

Based on the interviews, the committee concluded that STD does not initiate research relevant to the commodity directorates' needs. STD has helped only when it was asked for a solution to a specific, well defined problem (e.g., the polymeric tray developed for SusD). Many interviewees in the commodity directorates felt that STD was doing research for its own sake, a view that was confirmed by some STD interviewees. The committee was presented with very few examples of STD research fulfilling a need or making a significant improvement in the technical performance of an RDEC product.

The committee found very little evidence that STD proactively involves its primary customers (the commodity directorates) in its activities. The committee sensed that STD personnel had effectively communicated the message that their primary mission is basic research, regardless of their customers' needs. This attitude has created animosity between the commodity directorates and STD. In the committee's view, STD personnel should reorganize, redirect, and refocus their work to satisfy their customers' needs. Frequent participation by the commodity directorates in research planning could help bridge the gap between STD and its internal RDEC customers.

Although STD's mission and vision statements indicate the need for a broad spectrum of expertise within STD, the focus on four thrust areas significantly narrows the field. Nevertheless, STD personnel did indicate that they had a variety of outside customers (e.g., the Army Corps of Engineers, the Army Research Institute of Environmental Medicine, the Naval Seas and Supply System Commands, the Defense Logistics Agency, and the Canadian Ministry of Defense). It was not clear whether STD was providing feedback to the Natick RDEC directorates concerning relevant research by these customers. This kind of feedback could be very useful to the other directorates.

The committee arrived at an overall judgment of poor for the Customer Focus pillar. The committee found no evidence that STD is trying to improve communications with or identify the needs of the commodity directorates. Without this kind of communication, improvement in customer focus cannot be realized.

The current thrust areas within STD are biomolecular processes, materials functionality, ergonomics, and product acceptance. The research in these areas that addresses commodity directorates' needs is described in publications and presentations. STD was criticized during the interviews for not meeting customer needs for research and for publishing results that were not useful. STD personnel indicated, however, that current research programs were pursuing more relevant results.

Several interviewees inside and outside of STD indicated that STD needs more expertise in materials science. STD management indicated that they need technologies that could provide the following capabilities (see further discussion in Chapter 4):

• enhance the soldier's situational awareness and capability of acquiring, comprehending, and transmitting information
• materials that respond automatically, without soldier intervention
• better understanding of the stimuli and environmental factors that influence human performance and behavior
• accurate models and simulations of technical parameters to prepare for operation in the real world.

The committee agrees that STD needs additional technologies. However, in the committee's opinion, STD needs a master plan that integrates the research needs of the other directorates and the STD portfolio of programs. Otherwise, it is difficult to assess what should be added to (or subtracted from) STD's portfolio.

The patents granted to, and publications by, STD personnel are notable and commendable. However, several interviewees from the commodity directorates indicated that these achievements were not relevant to their needs.

Assessment of this characteristic with respect to the STD is complicated by the fact that basic research tends to pursue long-term benefits and improvements. The time frame for research results is unpredictable. However, STD does evaluate project times and publishes in a timely fashion.

The mainstays of STD support are basic-research and technology-development funding, much of it from the commodity directorates, who do not believe they are receiving sufficient benefits from STD in return. Progress by STD tends to be incremental and evolutionary rather than revolutionary (e.g., an assessment of the biodegradability of materials). Interviewees indicated that STD has not done a good job of transferring the technology it has developed. STD has attempted to address this problem by involving researchers in the product teams so that STD's research is fully understood. There were indications that, when people worked on teams during the integrated planning process, for example, they tended to cooperate more. Publications, research plans, and research proposals represent a historical database of research pursued by STD that could be assessed to justify and prioritize research projects.

The committee arrived at an overall judgment of poor to adequate for the Value Creation pillar. Improving the directorate's ability to create value for the commodity directorates requires targeting STD's research.

The results of the committee's assessment of STD are summarized in Table 3-1. The assessment is harsh and suggests that management must act expeditiously to resolve the problems associated with STD's relationship to the RDEC as a whole. The most serious problems seem to stem from the desire of STD personnel to perform research aimed at extending the general body of knowledge in their chosen fields rather than research that supports the needs of the commodity directorates. Striking a balance between the two kinds of research is not a new problem, but it needs to be resolved to reestablish synergistic interactions between STD and the other directorates of the Natick RDEC.

The committee considered two basic questions in assessing ASCD as a supporting directorate: (1) What models and simulations are necessary for evaluating enhancements to the combat effectiveness of the soldier-system? and (2) How should ASCD, as a whole, support the RDEC's efforts to improve soldier-system performance? The second question goes beyond modeling and simulation capabilities and focuses on the ASCD's marketing responsibilities.

A strong, positive image of the Natick RDEC and the Soldier Systems Command can send a persuasive message to external customers and organizations. This image can be developed and presented either by an organization's top management or by an ASCD working closely with top management. The image should reinforce the idea that the soldier system is an integrated, engineered system rather than an assemblage of components and subsystems. Developing the message requires identifying the modeling and simulations that accurately reflect relevant aspects of soldier-system performance and that demonstrate the combat value of Natick RDEC products. In the assessment of the impact of ASCD on the overall performance of its customers—the commodity directorates, STD, and top management of the Natick RDEC—the committee took into account how modeling and simulations were used.

In addition to interviews with personnel from the commodity directorates and STD, the committee interviewed personnel from all levels of ASCD. ASCD personnel were knowledgeable about modeling and decisions. Personnel who were involved with the customer simulation, as well as the analyses needed to support acquisition liaison function interacted with the Army battle laboratories and understood the benefits of operations research to the Army.

The committee learned that ASCD work usually meets, and occasionally exceeds, the expectations of the commodity directorates as well as RDEC management. The workforce has the necessary skills to accomplish assigned tasks and appears to be acquiring new skills as time and resources permit. Some ASCD personnel have participated in the Distributed Interactive Simulation/Synthetic Theater of War modeling and simulations. Now that the RDEC director's office has taken a new interest in strategic planning, ASCD's need for personnel with expertise in strategic planning should be reevaluated.

The budget for ASCD is included in the budgets of other programs in the RDEC. ASCD has adequate funding to fulfill its current responsibilities. The modeling and simulation function within ASCD appears to have expanded as far as current funding allows. Expanding ASCD's modeling and simulation capabilities will require support from RDEC leadership and justification to higher headquarters. ASCD expends its own resources to interface with the battle laboratories, but based on the interviews and the committee's observations, the RDEC does not receive sufficient requirements definition or user guidelines from the battle laboratories to justify ASCD's expenditures.

The committee believes ASCD's modeling and simulation and customer liaison capabilities can support acquisition decisions by other units of the RDEC. The personnel appeared to be well trained to carry out their tasks. Furthermore, the committee judged the modeling and simulations to be current and up to date. Nevertheless, the commodity directorates do not rely heavily on ASCD. Instead, other RDEC organizations have proceeded on their own in areas such as customer liaison, strategic planning, and obtaining requirements to justify RD&E programs. To bolster its capabilities to support the RDEC, ASCD could consider the following:

• improving tools to guide researchers and engineers through the phases of acquisition and provide feedback on their progress toward meeting the requirements for the next milestone
• improving contacts and working relationships between external customers and the other RDEC directorates (e.g., persuading users to establish requirements against which the RDEC directorates can work)
• increasing support for integrated planning teams to improve their selection of programs and projects for funding.

ASCD has established relationships with outside organizations and contractors that have made technological advances in modeling and simulation (e.g., organizations working on the Army helmet mounted display) or have supported modeling of the individual soldier. These outside contacts have enhanced the RDEC's understanding of the soldier-system concept. However, even better modeling and simulation will probably be needed as the DOD depends more and more heavily on information from models and simulations to make acquisition decisions. For example, the committee suggests that modeling the benefits and costs of a wide range of alternatives could influence DOD decisions to acquire the RDEC's products. ASCD can either use external resources to perform these analyses or perform them in-house.

ASCD is not responsible for finding new technologies but for evaluating the benefits of using various technologies in RDEC products. The better modeling and simulation gets, the better ASCD can evaluate the benefits of a technology. Although the commodity directorates also have the capability to model technologies, ASCD appears to be better equipped to determine in quantitative terms if using a certain technology could improve the combat effectiveness of the soldier system.

The committee observed that when commodity directorates requested ASCD assistance ASCD did incorporate their technologies into advanced models and simulations. However, ASCD management does not believe ASCD has sufficient staff and equipment to train other directorate personnel in using the ASCD facilities to model their own technologies. Therefore, ASCD is unable to take advantage of other directorates' personnel, who are the most qualified and knowledgeable persons to model their technologies. At the same time, the other directorates do not feel welcome to, or see the potential benefits of, their active participation in modeling their technologies, ideas, or products. The committee

believes ASCD should attempt to involve other directorates more in modeling their technologies while ASCD concentrates on integrating various technology models into simulations of a soldier-system archetype.

ASCD personnel seemed to be among the most pleased and actively engaged employees of the RDEC. This was evident by the interviewees' genuine expressions of pleasure with their work and their positive outlook for the future. The committee observed that ASCD is a cohesive unit with a productive climate in which innovative ideas are usually tolerated and sometimes encouraged.

ASCD personnel use information technology to support the progress of programs through the acquisition cycle. Generally, as new information technologies are implemented, ASCD's productivity improves. Personnel appear to be well trained, and appropriate hardware and software are available. Access to the internet and e-mail has connected ASCD personnel with internal and external coworkers, contractors, customers, and external information sources. ASCD's active participation in distributed simulations is a strong indication that these connections are up to date.

The tours and demonstrations of the ASCD's facilities revealed that they meet modem standards and can support the current operations, but they may not support significant growth, including implementation of some of the improvements suggested in this report. Given the rapid changes in computer technologies, the committee believes that ASCD's modeling and simulations may soon be out of date. In addition, the facilities cannot accommodate the participation of the other directorates in the modeling activities.

The committee arrived at an overall judgment of adequate to good for the Resources and Capabilities pillar. In contrast to other directorates where the committee noted indications of a decline in resources and capabilities, ASCD showed evidence of an upward trend.

Although many ASCD personnel could not describe the mission and vision statements of their directorate, they did have a good understanding of the RDEC mission and vision statements. ASCD personnel apparently realize that their role is to represent RDEC technologies for the soldier system that are worthy of funding for research, development, and engineering. ASCD helps the RDEC present an integrated, wholly engineered soldier system. ASCD also helps the RDEC demonstrate that its plans and programs contribute meaningfully to the soldier system. Modeling and simulation by ASCD are important to all these endeavors.

The commodity directorates currently take advantage of ASCD's modeling and simulation capabilities only in particular situations. The food program used the modeling and simulation capabilities to examine systems for food delivery. The clothing and individual equipment programs used them to model the combat value of ballistic protection items. MobD has used them to model airdrop delivery systems. In all of these cases, ASCD did the modeling and simulation. The commodity directorates did not send their personnel to ASCD, and ASCD did not have the time to train them to use the models and simulations. Some of the ASCD analyses showed that the systems being modeled had little or no combat value. The commodity directorates could not prove the worth of their systems or disprove ASCD's assessment. Unfortunately, this has discouraged the commodity directorates from using the modeling and simulation capabilities of ASCD.

The committee believes the commodity directorates can make better use of the RDEC's modeling and simulation capabilities. To begin with, there are only a limited number of instantiations of the soldier system to be modeled and simulated: the 21st Century Land Warrior, a multifaceted, high-technology soldier for the most challenging battlefield scenarios; the special-forces soldier, who is responsible for high performance in unique, specialized missions; the infantry soldier, who is expected to fight in more traditional situations; the peacetime soldier, who is deployed for operations other than war; the combat support soldier, who is responsible for battlefield support that requires improved technologies; and any other soldier who uses the products developed at the Natick RDEC.

Although assessing the performance for each version of the soldier system is complex, it is possible to determine a finite set of soldier-system archetypes that represent soldiers in various scenarios. The technologies being developed at the Natick RDEC can be modeled and integrated via simulation to support one or more archetypes. From scenarios and simulations, the benefits of specific RD&E programs can be estimated, and their impact on soldier-system performance can be assessed.

The committee believes that modeling and simulations by ASCD could contribute much more to an all-encompassing vision of the soldier system. This will require a paradigm shift in ASCD's views of modeling and simulation, however, as well as a change in the level of participation by the commodity directorates and STD. The committee believes the commodity directorates and STD could develop models that reflect the performance of their own technologies, and each group should be responsible for supporting its models within the ASCD modeling and simulation center. ASCD would cooperate with the developer of each model to incorporate the models into a soldier-system archetype. If the benefits of a technology could not be described via a model, the technology might offer little or no benefit for the soldier system.

ASCD now has soldier-system models that show the combat worthiness of proposed product-development programs. Some technical performance measures for these models are provided by the other directorates (e.g., information on heat stress and fatigue). However, the committee was told that the scientists and engineers in the other directorates have not provided the full range of technical parameters ASCD needs to model and simulate the performance of products accurately. The committee was astonished at the number of interviewees from the other directorates who could not identify a single technical parameter of the soldier-system their work would influence (although some could speculate). Perhaps the most beneficial aspect of involving other directorates in ASCD's modeling is that scientists and engineers would realize how their work would improve soldier-system performance. In order for an RD&E project to be funded, improving specific technical parameters should be an established goal.

The committee believes that ASCD has the potential to advance the work of the entire RDEC if it can involve the other directorates and help them to understand their technical contributions to the soldier-system. ASCD's modeling and simulation capabilities must reflect the full range of other directorates' visions, missions, plans, and programs. Currently, only select technologies, programs, and projects. are represented in the ASCD cadre of models and simulations. If ASCD can include models from each of the RDEC's programs and projects, it can help integrate the commodity directorates and STD with ASCD and ensure that models and simulations represent the value of RDEC products to soldier-system performance.

The question most often raised by interviewees was: What role in RDEC strategic planning should ASCD play? Interviewees explained that the recently established Scientific and Technical Advisory Council, which reports to the RDEC director, had taken on the RDEC's strategic planning role and that ASCD was supposed to furnish support, as requested. The information received by the committee, however, did not show that ASCD was making a valuable contribution to strategic planning. The committee suspects that ASCD was undergoing a change in roles and responsibilities related to strategic planning and that ASCD has much to offer in terms of support for strategic planning, but perhaps in a different way than before. ASCD should use the results of high-level models and simulations to confirm that the RDEC's plans and programs will benefit the soldier system.

ASCD clearly understands the vision and mission of the RDEC, and its personnel are continually developing models and improving simulations. ASCD's part in the overall RDEC mission has been well thought out, but the commodity directorates do not seem to be interested in providing adequate resources for ASCD to implement its mission. As one interviewee told the committee, customers don't always see the value of ASCD's involvement. In the case of the planning process used at the RDEC, however, customers must have recognized the value of ASCD's involvement because the profile of modeling improved greatly after the process went into effect, and funding was increased. Nevertheless, the committee did not see substantial buy-in by all RDEC stakeholders.

The committee observed that the leadership of ASCD had empowered its employees more than in other directorates. RDEC leadership must still correct the residual fragmentation of the old policy of emphasizing parts of the soldier system rather than an integrated whole. The new emphasis is on interdependent subsystems and cooperative programs and projects as the soldier-system concept evolves. Some interviewees were concerned that senior leadership had not effectively communicated the vision of ASCD as a separate entity. Given the understanding by ASCD employees of the big picture, the committee was not particularly troubled by this. However, if future changes in the way ASCD supports the other directorates are made (e.g., as a result of this report), ASCD leadership would do well to renew communications with ASCD employees.

The committee arrived at an overall judgment of adequate for the Strategic Vision pillar. The committee observed only a modest trend toward improvement in this pillar, primarily because of the empowerment of ASCD employees and their dedication to developing better models and simulations. The committee notes that substantial improvement will require the cooperation of other directorates in the RDEC.

During the interviews, the committee was told that the problems ASCD tries to solve, with limited resources, do not tend to allow for breakthroughs. The committee understands that breakthroughs in modeling and simulation depend on how robustly the models and simulations are designed, as well as on the potential for the modeling and simulation community to develop new and innovative methods. ASCD is making incremental improvements to models and simulations, as well as in its methods and procedures for supporting the acquisition process over the life cycle of RDEC products. Few if any of the innovations, however, could be called breakthroughs. The committee suggests that the true breakthroughs for ASCD will be (1) persuading the other directorates to participate fully in models and simulations of the soldier system, and (2) using the results to convince higher-level commands and users that the RDEC products add substantial value to the soldier system (assuming, of course, that they do add value).

The committee observed that ASCD has taken some measures to improve the quality of work and the quality of its support to the RDEC (e.g., better

modeling). Nevertheless, more attention could be paid to continuous improvement (e.g., better ways to obtain user requirements).

ASCD demonstrated a commitment to quality through its improved modeling and simulation capabilities that support acquisition decisions. However, the committee did not observe a significant commitment to improving the quality of program management (e.g., better ways to identify, monitor, and manage cost-risk and schedule-risk and better definitions and interpretations of requirements). The committee noted the lack of a requirements database as well as of capabilities for monitoring various acquisition activities. Management has invested some resources in training and implementing quality management, but this has not permeated the organization. Personnel are aware of the need to do a quality job, but measurable objectives for improvement have not been established.

ASCD models and simulations impose structure, and government procedures mandate structured programs (e.g., requirements must be explicit). The committee was told that ASCD holds regular team meetings to plan approaches to problems, check the status of programs, identify critical resources, develop contingency plans, and assign tasks. However, the committee did not find definitive processes tailored for supporting the modeling and simulation needs of STD and the commodity directorates. Instead of relying on ad hoc interactions between ASCD and other directorates, ASCD should consider establishing structured processes for assisting STD and the other directorates with modeling and simulation.

The committee observed that the RDEC in general supports continuing education for its employees. Training for ASCD personnel is necessary for them to keep up with changing computer technologies (e.g., distributed simulations). Currently, ASCD does not have the resources to train other RDEC personnel in modeling. Within ASCD, teamwork is apparent among personnel assigned to specific tasks; this encourages learning from mistakes because knowledge is retained and shared. However, RDEC-wide learning could be improved by participation of STD and the commodity directorates in the modeling phases of a program.

No patents have been awarded in ASCD, and only one program, the head mounted display, has been cited as ''very high level.'' The committee observed, however, that ASCD's activities are generally aligned with internal and external customer needs and that the results of ASCD analyses are often reviewed by the Army for the purpose of making acquisition decisions. ASCD personnel have participated in modeling and simulation meetings and have developed models and simulations to represent the soldier system in distributed interactive exercises. Results of the analyses that ASCD supports have been published. However, the quality of soldier-system models and simulations could be improved by more and better information from the other RDEC directorates. The committee believes that the quality of work elsewhere in the RDEC could profit by close interaction with ASCD.

The committee arrived at an overall judgment of adequate for the Quality Focus pillar. The trend was level, but the committee saw the potential for substantial improvement, especially if ASCD and the other directorates can work together more effectively.

ASCD meets customer expectations in many respects. The commodity directorates and RDEC management stated that, for the most part, they were pleased with ASCD support. Modeling and simulation results were provided in a reasonable time with demonstrated technical capabilities for performing operations research. Furthermore, the results were considered useful by the customers. ASCD also received complimentary reviews in the area of liaisons to customers,

although improvements could be made in requirements development and translation to the commodity directorates. However, the fact that the other directorates have had to do some customer surveys on their own and that strategic planning has been moved to another office suggests that satisfaction with ASCD support is not complete. Well satisfied customers would buy in fully to the value added by ASCD models and simulations.

Interviewees from the commodity directorates indicated that they do participate in analyses performed by ASCD to support RDEC positions or acquisition decisions by the Soldier Systems Command. Interaction between the commodity directorates and ASCD when an analysis is performed and its results are presented to higher-level commands and other DOD organizations means there must be customer involvement and feedback. However, some ASCD personnel mentioned that ASCD does not have enough time, equipment, or resources to increase the involvement of commodity directorates and STD. The committee also learned that customers are sometimes involved in setting program objectives and monitoring progress.

The committee judged that diversification in program management assistance, customer liaison, strategic planning, and modeling and simulation can best be measured by the breadth and robustness with which ASCD supports its customers. Many technologies and products at the Natick RDEC need to be modeled and simulated to estimate their combat worth to the soldier system. The committee acknowledges that ASCD's modeling and simulations have supported multifaceted and diverse analyses (e.g., work on head-mounted displays and ASCD's transfer of modeling and simulation improvements to allies, industry, and academia). However, a recurring theme at Natick is that the RDEC can meet customer needs in a crisis but that meeting customer needs under normal conditions seems to be difficult. The commodity directorates do not make full use of ASCD as a resource, especially to facilitate acquisition processes when the bureaucracy gets in the way.

The committee believes that ASCD should strive for its modeling and simulations to serve the entire range of technologies and products at the RDEC. ASCD should also support all RDEC research and technology programs by helping sponsors traverse the acquisition cycle without compromising technical quality.

The committee arrived at an overall judgment of adequate for the Customer Focus pillar. The trend appeared level, but the committee noted ample opportunities for improvement.

As a support organization for the entire RDEC, the ASCD portfolio must address the entire range of soldier-system needs and requirements. Although ASCD presented several examples of how it has provided analyses for each of the commodity directorates, in most cases these were special situations (e.g., a commodity directorate recognized a need for a particular analysis to justify a program). Product designs have also been influenced by these analyses. However, the modeling and simulations are not complete enough to treat the soldier-system comprehensively, and modeling and simulations have not been made an integral part of the strategic planning process.

The committee observed that problems throughout the RDEC indicate a need to add substance to the seminal concept of the soldier system. The Army procures individual items, each of which may meet a specific need but adds to the large number of items that could be integrated into the soldier system. One alternative to acquiring individual items is to integrate various components into a subsystem of the soldier system (e.g., the Soldier Integrated Protective Ensemble). The best alternative would be to engineer many technologies and their applications into a single soldier system. This alternative is most closely aligned with the 21st Century Land Warrior concept being developed in SurD.

In support of the ideal of engineering soldier systems (rather than merely acquiring individual items or integrating components into ensembles), the committee believes that ASCD modeling and simulation could be used to perform operations research analyses for entire soldier systems. This approach would be

analogous to the way DOD acquires major weapons systems. However, this approach would require a paradigm shift in the ASCD from individual analyses to multifaceted trade-off studies that combine models of candidate technologies into robust simulations of soldier-system archetypes. These simulations would allow various combinations of parts and components (representing technology applications) to be compared to determine the optimal combination of soldier-system components and subsystems for particular missions. Time lines shown to the committee by ASCD suggest that models and simulations of alternative soldier-system concepts could be available between 1999 and 2003.

ASCD currently satisfies customer needs in many respects. However, current models and simulations must be improved to meet the increasing expectations of DOD. Ultimately, these capabilities should be able to estimate modifications to the soldier system that would best enhance the soldier's combat effectiveness and lead to conclusions in the DOD virtual environment prior to fielding a new subsystem of the soldier system. Development of this analytical capability exceeds the current capabilities of ASCD.

Although some interviewees told the committee that ASCD had delivered analyses in a timely manner, several said that there had been problems. The committee was told that projects have been delayed for many reasons, sometimes because of the need to "fight fires" or to respond to a request for a quick fix or urgent support. The committee believes that, except in the case of urgent support, analysis results could be provided on time. ASCD should work on developing more robust modeling and simulation techniques to allow for responses to quick fixes without causing corresponding delays in the delivery of results to other customers.

Some interviewees told the committee that sponsors of programs request the help of ASCD because of its prior excellent work; requests like these demonstrate the value of ASCD support. ASCD employees explained that they had previously considered the soldier system as a collection of parts, but they now consider that the parts work together with the soldier. In other words, understanding of the soldier-system concept has improved. ASCD now models

squads with small arms and bursts of munitions, does a good deal of work on soldier equipment to support SurD, and has plans to include food in its analyses. All models must be validated, however, and the committee was told that ASCD has not been able to collect sufficient data to validate all of its models.

The committee arrived at an overall judgment of adequate for the Value Creation pillar. There were modest indications of an upward trend, particularly in the understanding of the soldier-system concept. ASCD has the potential to add substantial value to the RDEC through advanced modeling and simulation capabilities.

The results of the committee's assessment of ASCD are summarized in Table 3-2. ASCD was judged adequate for four pillars and adequate to good for one pillar. Several upward trends were noted. As ASCD continues to improve, the committee is hopeful that it will aspire to provide support for the entire cadre of soldier-system archetypes, subsystems, and components. The committee believes that the functions performed by ASCD are vital to the RDEC's realizing much more of its potential. The role of ASCD spans the entire RDEC, from research on new technologies in STD to the development and engineering of products in the commodity directorates, including support of the entire soldier-system concept.