Assessment of the U.S. Army Natick Research, Development, and Engineering Center (1997)

This report contains the results of the National Research Council (NRC) assessment of the U.S. Army Natick Research, Development and Engineering Center (RDEC). It was prepared by the Standing Committee on Program and Technical Review of the U.S. Army Natick Research, Development and Engineering Center, known as the Natick Standing Committee. The assessment period covered 1996 and the first half of 1997.

The Natick RDEC is a major component of the U.S. Army Soldier Systems Command, a subordinate command of the Army Materiel Command. The RDEC's vision is to be a ''world-class research, development, and engineering team.'' The RDEC's mission is to develop products that maximize the soldier's survivability, sustainability, mobility, combat effectiveness, and quality of life.

The committee's work was divided into two phases. In the first phase the committee established assessment criteria. In the second phase the committee applied the criteria. The first-phase report, World-Class Research and Development (NRC, 1996) defined the major components of world-class research. As shown in Figure ES-1, these components include five "pillars": Resources and Capabilities, Strategic Vision, Quality Focus, Customer Focus, and Value Creation. These pillars are the supporting elements of an organization's competitive advantages and require a foundation of demonstrated commitment on the part of the management and staff. Each of the pillars of a world-class R&D organization can be assessed by the characteristics listed in Figure ES-2. The metrics for assessing each characteristic are reproduced in Appendix C of this report.

In the second phase of its work, which is the subject of this report, the committee assessed the Natick RDEC against the characteristics and their metrics. The assessment was carried out in response to the statement of task for phase one, which directed that "the metrics derived from the committee's definition will be used as the benchmark by which the business areas and core technologies will be evaluated." The statement of task for phase two established two different stages of

Figure ES-1 The major components of world-class research and development organizations.

the assessment. The first stage consisted of assessing the RDEC in terms of the five pillars and the associated characteristics and metrics from phase one. The second stage consisted of assessing the RDEC in terms of ten key issues that are related to, but somewhat different from, the pillars and characteristics. The ten key issues and their connection to the "benchmark" for the assessment—the metrics, characteristics, and pillars from phase one—are discussed below.

The ten key issues, which were derived from the statement of work between the NRC and the Natick RDEC (see Appendix A), involve: (1) funding, personnel, and facilities; (2) organizational structure; (3) new organizational approaches; (4) the effect of the Soldier Systems Command; (5) the RDEC's products and system for measuring their quality; (6) customer satisfaction; (7) the. marketability of the RDEC's products; (8) world-class research and technology; (9) the continuation or elimination of research and technology programs; and (10) models and simulations. As can be seen by a review of the underlying characteristics and metrics, most of these issues cut across the five pillars and were therefore addressed as part of the overall framework developed by the committee in its phase-one report. However, some issues, the most notable being the effect of the Soldier Systems Command, required more direct inquiry and judgment on the part of the committee.

Figure ES-2 The five pillars and 25 characteristics.

The Natick RDEC was organized into five directorates: the Mobility Directorate (MobD), the Survivability Directorate (SurD), the Sustainability Directorate (SusD), the Science and Technology Directorate (STD), and the Advanced Systems Concepts Directorate (ASCD). The first three are called commodity directorates because they are responsible for developing RDEC products, such as rations, food-service equipment, clothing, shelters, and airdrop systems. The last two are called support directorates because they assist the commodity directorates with research (STD) and modeling and simulation (ASCD).

The committee's assessment reflected the RDEC organization. The committee formed three panels, one for each of the commodity directorates. The support directorates were considered primarily in the context of their assistance to the commodity directorates. During the visits of each panel and during the two general discussions (in February 1996 and February 1997), attention was devoted to the support directorates.

Before the panel visits, lengthy questionnaires were sent to and answered by the RDEC management. The questions were designed to illuminate the work of the five directorates in terms of the pillars, characteristics, and metrics of world-class performance and the ten key issues. Additional questions during on-site interviews of managers and staff were framed with these same points in mind. Collectively, the committee had discussions with a broad cross section of RDEC personnel during site visits in February, June, September, and December of 1996 and in February 1997 (see Appendix B).

The credibility of the committee's assessment is based on several factors. First, committee members include experts in the areas that are the subjects of research, development, and engineering at the Natick RDEC (e.g., parachutes, textiles, and food and food processing), as well as experts in the management of research and development (R&D), in the assessment of technical organizations, and in systems engineering and modeling and simulation. Second, the committee used criteria (i.e., pillars, characteristics, and metrics) that were based on the principles of excellence used for the Malcolm Baldrige National Quality Award and by leading organizations in the private sector. Third, the committee learned about the operations of the RDEC through visits, briefings by leaders of the RDEC, reviews of documents, and in-depth interviews of RDEC personnel.

The expertise of committee members, combined with the considerable knowledge gained during the criteria-development and data-gathering portions of the study, gave the committee a comprehensive understanding of the Natick RDEC and its performance. In the end, the soundness of the assessment is based on the application of pillars, characteristics, and metrics, as well as on informed judgments on the ten key issues. The major findings of the assessment are summarized below.

For the commodity directorates overall, three of the pillars were assessed as good (Resources and Capabilities, Customer Focus, and Value Creation), one was assessed as adequate to good (Quality Focus), and one was assessed as poor to adequate (Strategic Vision). A spider diagram that summarizes the assessment results for the commodity directorates is shown in Figure ES-3. Thus, although the assessment did not show the overall performance of the Natick RDEC to be at a world-class level, it did show that the commodity directorates were performing well. The assessment revealed a relatively uniform level of performance for the three commodity directorates.

The Strategic Vision pillar received the most poor or partially poor ratings. Based on a visit to Natick in early 1997, however, the committee was encouraged to see that the situation seemed to be improving because of recent emphasis on real strategic planning and because of the commitment of the RDEC leadership to strategic planning. The committee was concerned, however, that the trend for the Resources and Capabilities pillar appeared to be downward, largely because of the continuing loss of skilled personnel.

Although the committee found some instances in which the support directorates provided effective support for the commodity directorates, the overall support from both directorates was lacking and requires the attention of top management. The STD was of greatest concern because it was not adequately focused on the R&D needs of the commodity directorates, and the relevance of the STD portfolio of projects to those needs was questionable. ASCD, although it performs its internal role well, could play a larger external role for the entire

Figure ES-3 Spider diagram for the commodity directorates.

RDEC by demonstrating, through models and simulations, the utility of RDEC products. However, the committee found that ASCD's models and simulations did not fully address many aspects of soldier-system performance.

The committee assessed the effect of the establishment of the Soldier Systems Command on the Natick RDEC and found that the positive impact outweighed the increased burdens imposed by the command and the drain on RDEC resources. The new command embodies the soldier system, which is a unifying concept that focuses attention on how the products and services of the entire RDEC could influence the higher-level effectiveness of the soldier system.

The committee developed its conclusions and recommendations on the basis of the findings described above and the significant body of information assembled in the course of this study. The most important conclusions and recommendations are listed below, accompanied by discussions summarizing the committee's positions.

Conclusion 1. Realizing the strategic vision of the RDEC and facilitating strategic planning require continuous, high-priority attention from top-level management and the involvement of RDEC personnel at all levels.

Recommendation 1. The improvements under way at the Natick RDEC should be solidified and expanded. RDEC management should be especially receptive to adjusting plans and programs to meet new challenges. Senior managers of the RDEC should develop a comprehensive plan to solve the problems associated with the Strategic Vision pillar.

The Strategic Vision pillar is crucial to the RDEC's competitive advantage. Strategic vision, especially strategic planning, requires a strong foundation of demonstrated, steadfast commitment characterized by openness to the exchange of information at all levels of the organization, the involvement of all personnel in goal identification and analysis, and the efficient implementation of plans and programs. The demonstrated commitment of the leadership must be communicated to all levels of the organization in a way that all personnel both understand and accept. If this is done correctly, strategic vision will become a key factor in approaching world-class performance.

The committee found many problems related to strategic vision at the RDEC, including the following: (1) a lack of support for, and sometimes understanding of, the stated visions and missions of the organization; (2) broad strategic visions that exist only on paper because the directorate staffs have not embraced them; (3) some groups that were pursuing their own projects with little cross-fertilization of ideas or interaction with other groups; and (4) upper-level RDEC managers who were devoting too much time to solving near-term problems instead of working with groups and individuals to align their thinking and projects with the overall visions, missions, strategic plans, and programs of the RDEC.

The managers of the RDEC should take to heart the suggestions in the body of this report. In particular, they should examine current and planned work in the directorates to determine if it is in line with vision and mission statements. For example, although terrain traversal and personal augmentation are named as part of the mobility mission, it is not clear how these capabilities fit into the current MobD organization. Either these aspects of mobility should be made vital components of the MobD organization, or MobD should focus solely on airdrop technologies and transfer the other aspects of mobility to another organization.

Conclusion 2. Steps must be taken to stop the steady erosion of scientific and technical talent and experience.

Recommendation 2. Senior managers of the RDEC should develop a plan to stop the erosion of skilled personnel and experience. At a minimum, they should

develop strategies to retain in corporate memory the knowledge of people who retire or depart for other jobs.

Solving this problem is not completely within the control of the RDEC, and the RDEC's struggles to retain resources and capabilities are not unique. The entire U.S. defense establishment is trying to cope with cuts in forces, reduced modernization programs, tight budgets, and the loss of personnel. Furthermore, in the last decade, many corporations have also undergone radical reengineering, which has meant massive reductions in personnel. Recognizing that other organizations have similar problems may be of some help if the RDEC can learn from the experiences of others who have had to adjust to new conditions.

The RDEC must develop strategies for preserving its corporate memory. Given that replacing experienced people will continue to be difficult, the knowledge and expertise of departing staff members must be passed on to remaining personnel. The RDEC must also increase the efficiency of remaining resources and capabilities (e.g., drop programs or functions when a critical mass no longer exists).

Conclusion 3. The Quality Focus pillar was assessed to be at a lower level of performance than three other pillars, but the committee's principal concern was the uncertain trend for the future.

Recommendation 3. Management of the RDEC should take steps to improve RDEC-wide learning and to develop and implement an internal system for assessing quality.

The committee was most uneasy about problems with learning from team to team and the general lack of metrics for tracking quality. The RDEC should have a framework and methodology for measuring quality, measurable objectives for improvements in work processes, measurements for optimizing RD&E processes to deliver value, teams on one project teaching teams assigned to other projects, an organizational climate conducive to organizational learning, and methodologies to measure and evaluate organizational learning.

Conclusion 4. The Science and Technology Directorate was not adequately focused on its primary customers, the commodity directorates.

The committee determined that the arrangement between STD and the other directorates was not working. An STD that serves only some of the needs of the commodity directorates while pursuing its own agenda, thereby causing the other directorates to establish their own research functions, is not only inefficient, it should be unacceptable.

Recommendation 4a. The RDEC director should take immediate action to consolidate the RDEC's science and technology activities and focus them sharply on the needs of the commodity directorates. If a refocused, centralized Science and Technology Directorate cannot provide the necessary support, the director should consider distributing the entire research function among the other directorates.

Either of these options would eliminate the current arrangement between STD and the other directorates. The first option would consolidate the science and technology activities at the RDEC into a single unit that would serve all of the commodity directorates. The second would do away with a STD at the RDEC and replace it with science and technology traits distributed among the commodity directorates.

The committee favors the first option, primarily because it would concentrate the research talent of the RDEC in one organization, but only if the refocused organization could guarantee requisite support. A refocused STD could facilitate interactions between members of the research staff (e.g., to apply common technological solutions to the problems of various customers) and sharing of scarce research equipment. Nevertheless, concentrating the research talent in a single directorate is less important than contributing meaningfully to the science and technology needs of the RDEC.

If the first option is chosen, the new directorate would not just be a larger STD that operates the way the current STD does. Instead, the new STD would have tight links to each of the commodity directorates and oversight by RDEC senior management and the commodity directorates to ensure that their science and technology needs were being met. In turn, the commodity directorates would also have to specify their needs. Measures of performance for the new directorate that address the research needs of the entire RDEC would be established and monitored.

The new arrangement would be designed to take maximum advantage of outside research, although some in-house work on new technologies would still be done. The RDEC must concentrate its internal resources on maintaining the capability to judge the efficacy of external research and on fulfilling needs unique to the military that the private sector is unlikely to meet.

The committee's preference for a consolidated science and technology directorate should not be determinative on the RDEC management. If after due consideration the director of the RDEC decides that consolidation of the RDEC's science and technology activities cannot overcome the problems associated with the current STD, the STD should be broken up and its functions distributed to the commodity directorates.

Recommendation 4b. The new science and technology operation, consolidated or distributed, should ensure that its research is demonstrably relevant to the RDEC's overall mission.

The RDEC's new research arm, whatever its form, should thoroughly investigate the soldier-system concept and identify the full range of technologies within the RDEC's domain that could improve soldier performance. Then an evaluation should be made, in conjunction with ASCD and the commodity directorates, to determine which technologies offer the highest payoffs for the soldier system. Decisions could then be made on the technologies worthy of pursuit.

Conclusion 5. The Advanced Systems Concepts Directorate is not sufficiently supportive of the commodity directorates (its primary customers) or of the RDEC as a whole.

Recommendation 5a. ASCD must improve its capabilities to develop and use models and simulations in support of the other directorates and the entire RDEC. RDEC management should emphasize to all personnel the importance of (1) adequate modeling and simulations of the soldier system and (2) full cooperation of the other directorates with ASCD.

It is hard to overestimate the benefits to the Natick RDEC of a greatly improved capability to model relevant technologies and simulate the soldier system. The benefits include internal efficiencies and effectiveness, as well as external visibility and appreciation of the contributions of the RDEC to soldier-system performance. The committee firmly believes that RDEC participation in top-level analyses of military forces is necessary for the RDEC to compete successfully for scarce resources. Current constraints at the RDEC will make it difficult to hire new staff for the ASCD. Therefore, it is imperative that people in the other directorates with technical knowledge that is important for models and simulations contribute fully to ASCD.

Recommendation 5b. The RDEC managers should reexamine the other functions of ASCD and either expand them or eliminate them, as appropriate.

Conclusion 6. The RDEC director and managers face a major challenge in adjusting their priorities to cope with declining budgets.

Recommendation 6. As part of its strategic planning, the RDEC should develop strategies and tactics to cope with budget shortfalls.

Several research and technology programs at the Natick RDEC are unique, and some have been highly acclaimed. Two considerations should be taken into

account when assessing the mix and emphasis of various programs: (1) circumstances at the time of the assessment and (2) future needs.

Conclusion 7. The mix and emphasis of research and technology programs will have to be adjusted to improve the RDEC's support of the Army and other customers.

The committee considered the RDEC's core technologies and unique capabilities (e.g., in food processing, multicapability protective clothing, and airdrop technologies) in relation to its mission to support the soldier system. In most cases, the connection between what the RDEC was doing (or not doing) and its mission was clear, but in some cases it was not (e.g., some airdrop technology is not supported by the Army's requirements). Some of the areas on the RDEC's list of key technologies did not appear among STD's programs (e.g., airdrop technologies), and some were narrowly focused (e.g., biotechnology).

Recommendation 7. The RDEC should reevaluate its rationale for all in-house research and technology programs in light of the unique needs of the Army or other customers and the availability of sources outside the Army to advance the technology.

The RDEC should reevaluate various technologies to assess their payoffs in terms of improved soldier-system performance. If a program does not show a measurable payoff, the program should be considered suspect. If another agency or the private sector is advancing a technology much faster than the RDEC can with the limited funds available, the RDEC should reevaluate its decision to pursue that technology. Long-range needs for technologies like those associated with airdrop and ground mobility systems should also be reassessed. If the Army is not the customer, the RDEC should seek support from the likely customer (e.g., precision airdrop for the U.S. Air Force) or drop the program. If the Army is the real customer but does not provide adequate support, the effort should also be dropped (e.g., ground mobility).

As part of its assessment, the committee was asked to "identify opportunities for reengineering in areas judged deficient or worthy of improvement." In the committee's opinion, no single recommendation in this report amounts to a recommendation for reengineering, which is usually associated with large-scale corporate reinvention characterized by radical changes. Results of the committee's assessment did not indicate that radical changes were necessary for the Natick RDEC.

However, if the recommendations are taken together, they constitute a prescription for less-than-radical reengineering, a way of doing business that will

require persistent management attention; buy-in from all of the stakeholders, including personnel at the RDEC; and substantial time for implementation. An important component of this less-than-radical reengineering is the establishment of long-range strategies for the RDEC to maintain its competitive advantages. Another component is a demonstrated commitment by management to support those strategies and see that they are carried out.

Conclusion 8. The Natick RDEC is performing well. The RDEC's cycle time and responsiveness to urgent customer needs are particularly impressive. World-class performance, which requires widespread excellence, will be difficult for the RDEC to achieve but remains a worthwhile goal. Less-than-radical reengineering is an appropriate way for the RDEC to move toward that goal.

The RDEC can rightly be proud of its remarkable capabilities and noteworthy accomplishments. The fact that the RDEC had the courage to subject itself to this arduous assessment is indicative of its commitment to strive for world-class performance. The phrase world-class has been used as a description of excellence for all R&D in the U.S. Department of Defense. The Natick RDEC is the first agency, in the committee's knowledge, to bring concrete meaning to that phrase.

The methodology developed in the phase-one report and used in this assessment is sound. Although the characteristics and metrics might be modified to suit specific circumstances, the major components of world-class performance are useful parameters for assessing RD&E organizations. Examples of impressive past performance at the Natick RDEC should suggest ways to correct deficiencies revealed in this assessment. To achieve the goal of widespread excellence, the committee recommends that the Natick RDEC be reengineered in the way described below.

Recommendation 8a. The Natick RDEC should implement a five-step reengineering plan: (1) improve the Strategic Vision pillar to good, reverse the downward trend of the Resources and Capabilities pillar, and upgrade the Quality Focus pillar; (2) correct the problems associated with STD and the research and technology programs; (3) enhance ASCD's capability to support the RDEC; (4) raise selected pillars to the excellent level; and (5) conduct periodic self-assessments using the methods described in the phase-one report.

The committee offers the suggestions below for consideration by management of the Natick RDEC. First, the fact that the committee found significant problems with strategic vision suggests that there has been a shortcoming in management. The committee urges that greater emphasis be put on training to improve the leadership capabilities of the RDEC managers (see Recommendation 8b below). The goal of this training would be to bring about fundamental changes in the RDEC to make it a true learning organization that would develop its own improvement vectors and would produce a cultural change throughout the

workforce. Ideally, the RDEC would then have true leaders instead of just managers. Leaders motivate, empower, and coach continually; they have strategic vision, which they communicate to the workforce at every opportunity. Without this kind of leadership, the organization's vision statements will continue to fall on deaf ears. Appropriate elements of this training would be extended to RDEC team-leaders. The teams themselves should be taught to operate across organizational boundaries and to work toward common goals.

The committee believes that maintaining a highly qualified workforce is possible if management takes a long-term view and attacks the problem creatively. The committee urges that management pay special attention to recruiting high-quality personnel. Successful recruitment is an ongoing process that should begin long before spaces open up or the hiring freeze is lifted. Promising individuals must be targeted long before graduation and nurtured through cooperative projects, summer hiring programs, and temporary jobs sponsored by on-site contractors. Management must be ready to hire expeditiously as soon as a hiring window opens. Also, RDEC management should be aware that good people are attracted by superior facilities, a user-friendly research environment, and rewarding professional relationships.

Implementing this plan will require teamwork among all of the employees at the Natick RDEC, who will have to look beyond the narrow confines of their own programs and encompass the whole organization. The end result should be a cohesive organization that can approach the ideal described in this report.

Recommendation 8b. The Natick RDEC should begin an educational program in leadership development and modem principles of technology management.

In the committee's opinion, many of the problems associated with strategic vision, resources and capabilities, quality focus, and the two support directorates of the Natick RDEC stem from management limitations at the senior level. The committee does not intend to sound simplistic by throwing all of the problems at management and recommending that management fix them. But these problems are similar to the problems that have faced managers of other government and private-sector organizations that had not reengineered themselves to meet the challenges of a rapidly changing world. Problems cannot be excused because managers say they are overburdened or because resources are limited. The RDEC managers must learn and then transmit to all employees the techniques that have made some organizations pacesetters.

The training must be passed from the top down, with people at each level becoming mentors and trainers for people at the level below them. A new culture must be created and maintained through continuous coaching by leaders over a period of months and years. The training should include the many considerations related to the five pillars of a world-class R&D organization contained in the committee's phase-one report. Strong management commitment must be evident to the entire staff at the RDEC. With these improvements, the committee is hopeful that the organization will excel in the future.

In Chapter 1, the committee addresses the background, the statement of task, and the approach for the assessment of the Natick RDEC. The committee also provides an overview of the principal features of the organization, including descriptions of the commodity and support directorates.

In Chapter 2, the committee discusses its assessment of the three commodity directorates. The characteristics and metrics developed during phase one are applied to these directorates, followed by the committee's judgments. In Chapter 3, the committee discusses its assessment of the two support directorates. Chapter 4 contains the committee's judgments concerning the ten key issues discussed above, which apply to the entire RDEC. The judgments in this chapter are based on the assessments in Chapters 2 and 3 and the committee's broad expertise. In Chapter 5, the committee presents conclusions and recommendations based on the assessment.