3
Assessment of the Support Directorates

In this chapter the committee focuses on the two support directorates of the Natick RDEC, STD (Science and Technology Directorate) and ASCD (Advanced Systems Concepts Directorate).1 The committee again used the pillars, characteristics, and metrics established in phase-one for world-class performance. However, the characteristics and metrics were applied in a way that emphasized the supporting roles of these directorates. The assessment results reflect the committee's judgments of the contributions of the support directorates to the performance of the commodity directorates and the RDEC as a whole.

Assessment Process

The committee decided early on not to subject STD and ASCD to the arduous assessment process undergone by the commodity directorates. The committee submitted fewer pre-visit questions and conducted fewer interviews of personnel in the support directorates. This approach recognized that STD and ASCD were created to complement the commodity directorates and the RDEC by (1) providing research to infuse new technology into the RDEC's products, and (2) by marketing products to external users. Therefore, in applying the characteristics and metrics of a world-class organization, the committee heavily weighted the supporting role of STD and ASCD in relation to the commodity directorates, their primary customers.

The committee gathered data for assessing the support directorates on the same visits to Natick. During the wrap-up session in February 1997, the committee scheduled additional interviews to ensure that sufficient information would be available for assessing the support directorates. The committee also obtained a great deal of information about the support directorates from personnel in the commodity directorates (i.e., the primary customers). However, members of the committee also interviewed people in the support directorates, who made many cogent observations regarding STD's and ASCD's abilities to provide needed support.

1  

General references for this chapter are Malabarba (1996) and Salant (1996).



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 86
--> 3 Assessment of the Support Directorates In this chapter the committee focuses on the two support directorates of the Natick RDEC, STD (Science and Technology Directorate) and ASCD (Advanced Systems Concepts Directorate).1 The committee again used the pillars, characteristics, and metrics established in phase-one for world-class performance. However, the characteristics and metrics were applied in a way that emphasized the supporting roles of these directorates. The assessment results reflect the committee's judgments of the contributions of the support directorates to the performance of the commodity directorates and the RDEC as a whole. Assessment Process The committee decided early on not to subject STD and ASCD to the arduous assessment process undergone by the commodity directorates. The committee submitted fewer pre-visit questions and conducted fewer interviews of personnel in the support directorates. This approach recognized that STD and ASCD were created to complement the commodity directorates and the RDEC by (1) providing research to infuse new technology into the RDEC's products, and (2) by marketing products to external users. Therefore, in applying the characteristics and metrics of a world-class organization, the committee heavily weighted the supporting role of STD and ASCD in relation to the commodity directorates, their primary customers. The committee gathered data for assessing the support directorates on the same visits to Natick. During the wrap-up session in February 1997, the committee scheduled additional interviews to ensure that sufficient information would be available for assessing the support directorates. The committee also obtained a great deal of information about the support directorates from personnel in the commodity directorates (i.e., the primary customers). However, members of the committee also interviewed people in the support directorates, who made many cogent observations regarding STD's and ASCD's abilities to provide needed support. 1   General references for this chapter are Malabarba (1996) and Salant (1996).

OCR for page 86
--> The committee considered each support directorate in relation to each characteristic of world-class performance. However, the committee believed it had sufficient information only to make summary judgments at the pillar level regarding the performance and major strengths and weaknesses of STD and ASCD. In making these judgments, the committee interpreted the various metrics in light of the relationship between the support directorates and the commodity directorates. Because of the fundamental differences between the support directorates, the committee decided to assess them separately. Science and Technology Directorate The Natick RDEC must, at a minimum, maintain a research capability and continue to develop some technology to ensure that the RDEC personnel can make credible scientific and technical judgments. But the research and technology development must be relevant and must offer significant benefits to soldier-system performance. These considerations were important to the committee's assessment of STD's impact on the overall performance of the commodity directorates, the ASCD, and the Natick RDEC as a whole. Resources and Capabilities Pillar Characteristics of the Resources and Capabilities Pillar Personnel Quality Budget RD&E Capabilities, Skills, Talents Use of External Resources Important Technologies Organizational Climate Information Technology Facilities and Infrastructure Personnel Quality STD's customers' indicated that their expectations were seldom exceeded by STD. When research tasks were Well defined, their expectations were met; however, if tasks were less well defined, customers felt that the deliverables were either not of high enough quality or were delivered late. These problems may not be entirely attributable to personnel quality, but they indicate a disparity that should be addressed by acquiring skills or training in areas more compatible with customer expectations.

OCR for page 86
--> The committee was informed repeatedly that because of the hiring freeze STD had been forced to try to improve personnel quality by training and personnel transfers. The committee was informed about several training activities and programs but found that they were often pursued by employees for personal interests. The committee would have been pleased to see efforts being made to encourage and train current personnel or to obtain transfers of personnel with appropriate expertise to improve support for the commodity directorates. Unfortunately, efforts of this kind were not observed. Budget The committee was informed that STD is constantly struggling for money. STD's funding comes from two basic sources: the commodity directorates and other government organizations that need the type of research that STD can perform. Both sources require that STD justify those funds. Therefore, budgets are directly related to STD's ability to convince its customers that its research capabilities and services are worth funding. STD budget shortfalls reflect doubts on the part of the DOD and the Army, as well as customers within the RDEC, about the benefits that will be realized from their investment in research. The committee believes that budget problems will persist. However, conflicts within the RDEC could be reduced by a much closer working relationship between STD and the other directorates and by STD accepting the need to demonstrate the benefits of its work to RDEC projects and soldier-system performance. RD&E Capabilities, Skills, Talents This characteristic reflects the broad capabilities of an organization to pursue a viable research program. Several interviewees contributed to the committee's assessment of this characteristic. For example, one person indicated that the hiring freeze is "crippling STD." Another interviewee, well known for his work, predicted that the best people will soon leave the Natick RDEC. The committee agrees that STD will probably incur losses, as most organizations do when budgets and resources are limited. The committee noted the departure of one key scientist from STD whose reasons for leaving included the inability of the organization to hire new people who could bring new capabilities and ideas, which are necessary for a thriving research organization; decreasing funds for equipment and facilities, which limits future work to present technological capabilities; and cutbacks in research by government organizations in favor of research by private industry.

OCR for page 86
--> Use of External Resources STD leverages outside research programs in both academia and private industry. STD has organized its external multidisciplinary research into several technical areas: biodegradable microbiology and chemistry for bioremediation; biotechnology for materials application; biomimetic processing; and functionally tailored textile fabrics. In each of these areas, STD collaborates with one or more academic institutions and one or more research organizations in DOD (e.g., the Army Research Laboratory, the Edgewood RDEC, the Office of Naval Research, and the Air Force Office of Scientific Research). STD also has cooperative agreements with private organizations (e.g., agreements covering various biodegradable materials and synthetic polymers that could mimic the properties of spider silk). The committee was informed that a lot of work is now being given to outside contractors because of the hiring freeze, thereby establishing connections with external resources. Although the range of external work is impressive, the committee is concerned that there is no master plan showing how external research complements the internal research by STD to fulfill the RDEC's needs. Important Technologies Research at the Natick RDEC is divided between the commodity directorates, which conduct research that is uniquely applicable to their programs, and STD. STD maintains oversight of all of the basic research2 at the RDEC, but the commodity directorates manage the day-to-day activities and progress of the research they conduct. The committee feels that the division of research elements amplifies the perception of a separation between researchers in STD and developers and engineers in the commodity directorates. In fact, information from the interviews implied that STD is preoccupied with its own research and has a tendency to pursue research for its own sake. The committee realizes that STD must focus its efforts on a few select technologies (e.g., the four thrust areas [identified in Chapter 1], as well as external initiatives) in order to achieve critical mass. However, because STD obtains funding from its primary customers—the commodity directorates—it must also ensure that technologies critical to the success of the commodity directorates take precedence. The RDEC's future depends on the maturation of technologies that can be applied to products. 2   In this report the committee uses the term "basic research" to mean research to develop fundamental new knowledge. The committee believes that basic research usually (but not always) has a long-term orientation. The results of basic research can be used for specific applications in "applied research" programs. In other words, basic research develops new knowledge that can be used in applied research.

OCR for page 86
--> The committee observed that STD and the other directorates were reluctant to engage in a frank dialogue about research programs. The commodity directorates need to clarify their research needs that have not been met by STD. Scientists in STD appeared to resent demands that they produce short-term results and argued that long-term results worthy of publication and patents require patience. The committee accepts the fact that mechanisms like the planning integration team can assess the merits of research proposals and allocate research funding for the RDEC. Nevertheless, in the case of basic research, the committee believes there must be a closer relationship between STD and the commodity directorates. STD management and personnel indicated that important technologies that need to be added include work in materials science, human performance, and modeling and simulation (see discussion under the Value Creation pillar). The committee learned that only one-fourth of the proposals submitted by STD were funded. Better dialogue with the commodity directorates could improve the selection rate. As the RDEC corporate laboratory, STD should be the center of excellence in understanding as well as in advancing critical technologies for the RDEC and the Soldier Systems Command. To this end, the committee considered ways STD could determine the technologies that are essential to the RDEC's future. The committee suggests that STD and the commodity directorates agree to a closer working arrangement so important technological needs can be better communicated by the commodity directorates to STD and so STD can make fundamental changes in its approach to selecting technologies that deserve resources. This working arrangement was discussed during committee interviews. For example, one scientist's opinion was that research organizations must have an overall motivation, an overriding goal, and justification for what they do. There has to be accountability along the way (e.g., yearly technical reviews of research programs should be conducted by a team that collects, analyzes, and discusses data and how they fit the long-term goals of the organization). Some fringe ideas that could result in breakthroughs should also be funded. A certain percentage of funding (e.g., 80 percent) should be subject to review and approval by the commodity directorates, and a smaller percentage (e.g., 20 percent) should be provided for the research organization to direct on its own. The committee notes that it does not necessarily endorse this opinion but believes it offers reasonable options. The committee believes that a formal, periodic review process would ensure that the commodity directorates could clarify technologies important for their needs and would give STD an opportunity to identify beneficial long-term technologies for investment within a small percentage of its budget. STD leadership must clarify which research projects are directly responsive to the

OCR for page 86
--> commodity directorates and which have been identified as beneficial by STD and ensure that the STD personnel maintain a balance between the two. Organizational Climate Some interviewees indicated that STD is not a part of the big picture; that the Soldier Systems Command does not understand research and supports administrative organizations that do marketing rather than scientific work; that government rules and regulations, which govern the commodity directorates, are different from the approach for performing basic research and force STD people to spend time on shorter-term acquisition rather than furthering a longer-term vision; that the RDEC does too much crisis management; that public relations are often substituted for substance; and that there are few opportunities for advancement. The committee suspects that the organizational climate in which STD is immersed will not improve until STD programs show more relevance to the commodity programs and soldier-system performance. Information Technology STD uses modem information technology to support research at the RDEC. Interviewees indicated that there is RDEC-wide cooperation in the use of information technology. Facilities and Infrastructure The committee observed that some renovations are being undertaken at Natick. Nevertheless, evidence suggests that the research facilities are barely adequate. For example, although personnel are pleased with the improved work space, they cited several facilities that were in disrepair and instances when necessary facilities or equipment for specific types of research were not available. Overall Assessment The general findings related to each of the characteristics and review of the associated metrics led the committee to an overall judgment of adequate for the Resources and Capabilities pillar. Indications are that there is a downward trend in STD's ability to maintain its RD&E capabilities, skills, and talents. This problem is similar to the problems the committee found for the commodity directorates (see Chapter 2).

OCR for page 86
--> Strategic Vision Pillar Characteristics of the Strategic Vision Pillar Alignment of Vision and Mission Strategic Planning Stakeholder Buy-In Leadership Alignment of Vision and Mission STD's mission is to ''provide world-class research ... maximize combat effectiveness of [the] soldier system ... leverage outside research programs ... [and] maintain cutting edge research programs, facilities, and equipment'' (Salant, 1996). This statement reflects a desire to provide a wide range of scientific expertise in support of the commodity directorates and Soldier Systems Command and is in alignment with the mission and vision of the Natick RDEC. However, interviews conducted by the committee indicated that there is a dichotomy between the understanding and the implementation of this mission by STD personnel. Although many STD personnel can relate to the vision and mission statements of the Soldier Systems Command and the RDEC, they do not appear to believe the STD's vision and mission statements. Some members of STD consider the commodity directorates to be their customers; others feel that basic research is a mission in and of itself and that the research activities in STD are worthy of support without application, short-term or long-term, to a current program. This difference in opinions about the fundamental role of STD was also reflected in interviews with commodity directorate personnels. Many interviewees in the commodity directorates found it difficult to name technologies developed by STD that have been, or are being, applied to products. They also indicated that key areas were not being researched by STD (e.g., research to assist MobD). The committee concluded that the vision and mission statements are not helping STD to deliver research that consistently advances the commodity directorate programs or is demonstrably relevant to soldier-system performance. The committee believes that STD and the other directorates must define and focus on the scientific and engineering base critical to the RDEC. STD personnel should understand the areas for which they will be held responsible and identify the technologies for which the commodity directorates expect excellence from the STD. Strategic Planning The committee was briefed on recent formal strategic planning by the Scientific and Technical Advisory Council, which is directly under the office of

OCR for page 86
--> the RDEC director. This council and the planning integration team are intended to guide STD's planning and research. STD personnel are expected to develop research plans for STD and for the RDEC as a whole. Although some interviewees felt that the new strategic planning process was a significant improvement over the previous process, others felt it was not effective because it does not emphasize longer-term science and technology, for which the return on investment is 10 to 20 years away. Stakeholder Buy-In During interviews with members of STD, the committee perceived that the responses of internal stakeholders to the vision and mission of STD were either indifferent or negative. Stakeholders in the other directorates indicated to the committee that the results of STD research do not justify their investments. The committee concluded that either the return on investment in STD research is inadequate or it is not being clearly communicated to the RDEC community by STD. Leadership The committee found that STD leaders thoroughly understand the RDEC mission and vision to support the soldier system. They stated clearly that the vision and research plan for the STD should be compatible with those of the RDEC and Soldier Systems Command, and they acknowledged that STD was obligated to justify the contribution of its research to the current commodity directorates. However, some of the researchers expressed their displeasure with having to justify their work on the basis of its relevance to development and engineering projects in the commodity directorates. Several interviewees appeared to be suspicious of, rather than in agreement with, the leadership of STD. The committee concluded that communication between the STD leadership and the staff was inadequate and that senior leadership has not been able to "sell" the vision and mission of STD to rank-and-file members of the directorate. Overall Assessment The committee arrived at an overall judgment of poor for the Strategic Vision pillar. Unfortunately, the committee found that STD has been unable to translate its vision and mission into a working philosophy that is accepted, let alone embraced, by the members of STD or the other directorates. The trend for the future was judged to be level.

OCR for page 86
--> Quality Focus Pillar Characteristics of the Quality Focus Pillar Capacity for Breakthroughs Continuous Improvement Commitment to Quality Structured Processes Learning Environment Quality of Research Capacity for Breakthroughs Considering that existing research in STD is not well correlated with the commodity directorates' needs, the potential for breakthroughs from STD research is limited. Furthermore, as some interviewees within STD noted, the technologies associated with some commodities (e.g., food and clothing) are already mature, and only incremental advances can be expected in those areas. The most frequently cited example of a "leap-ahead" technology was in the area of biotechnology (the cited project involves developing materials with the properties of spider silk). STD personnel continue to publish papers and make steady, incremental improvements in their knowledge base. Continuous Improvement STD personnel interviewed by the committee indicated that STD is not very flexible or capable of change; this inflexibility has contributed to STD's long-standing inward orientation. Interviewees noted, for example, that STD had not taken advantage of the RDEC's being designated as a personnel demonstration, which allows the organization to waive many civil service rules to find a better way of doing business. The quality of work at STD is assessed by STD managers and personnel using the same measures used by the academic community to assess research quality (e.g., number of publications in refereed journals, number of sessions chaired at national and international technical meetings, number of patents granted). But no measures are in place to determine how the output relates to the needs of the commodity directorates. The committee was informed that STD staff recommendations for improvements are considered only within STD, with no feedback from other directorates. The deep division between STD and other RDEC organizations is of significant concern to the committee. Bringing STD closer to the commodity directorates should bring the research program more in line with the commodity directorates' needs and enhance the value of STD to the entire RDEC.

OCR for page 86
--> Commitment to Quality The committee was convinced in the interviews with STD personnel that they were committed to the highest quality in basic research. However, they judged the merits of their research on a traditional basis and did not consider the effects of their research on the commodity directorates. Several interviewees indicated that their ability to conduct high-quality research had been significantly affected by the hiring freeze, which has made it impossible to hire sufficient support staff and eliminated the influx of fresh ideas. Many interviewees were deeply troubled by the reduction in hands-on research, indicating that they were increasingly being forced to manage and administer external research projects rather than perform or supervise the research themselves. Attempts to offset the hiring freeze by using temporary technical personnel were viewed with skepticism; many felt that hiring temporary personnel compromised the organization's inherent capabilities and made it more difficult to maintain continuity. As a consequence, many interviewees said, the quality of research at the Natick RDEC was declining. Other interviewees, primarily senior scientists, stated that the number of publications and speaking invitations at international seminars indicated that the quality of research was still high. Structured Processes STD leaders told the committee that they evaluate projects and personnel according to five performance objectives: well planned and executed research; demonstrations of scientific stature by editing and writing; safe research practices; collaboration; and leveraging technology and securing funding. Overall, research is monitored to control cost and to assess progress toward milestones. Learning Environment Funds are available for STD personnel who desire to pursue advanced technical degrees or continuing education courses. However, the committee noted that STD does not have a uniform policy to encourage individuals to pursue opportunities germane to STD's mission, which could amplify and strengthen the current skill base. In addition, successful interactions with the commodity directorates that would motivate cooperative research programs with the remainder of the RDEC are not communicated to the rest of the RDEC. The committee feels that, as a minimum, STD should mount an organized campaign to encourage STD personnel to acquire knowledge in the technologies most important to the commodity directorates.

OCR for page 86
--> Quality of Research The committee has already noted the "disconnect" between STD's chosen areas of research and the commodity directorates' expectations. When STD has been asked to fulfill precise research needs by the commodity directorates, STD has performed well. In the research areas chosen solely by STD, STD's work has been recognized by the greater scientific community. But STD's research and technology programs are only occasionally correlated with the needs of its primary customers. The committee commends STD for maintaining the quality of its research vis-à-vis the community of its peers, but its activities should be relevant to the development and engineering needs of the commodity directorates. Overall Assessment The committee arrived at an overall judgment of poor to adequate for the Quality Focus pillar. Improvements will require that STD refocus its activities to support the commodity directorates. The committee is not advocating that STD be relegated to making short-term fixes, but STD should base its long-term programs on issues that are germane to the future needs of the commodity directorates. Customer Focus Pillar Characteristics of the Customer Focus Pillar Customer Satisfaction Customer Involvement Market Diversification Customer Satisfaction The committee was informed by all of the commodity directorates that, with the exception of the support provided by STD's behavioral sciences group to SusD, they do not feel the amount or quality of STD research justifies the cost of supporting an independent STD. To compensate, all of the commodity directorates have mounted their own research projects to provide the information they need to facilitate their development and engineering projects. For example, the shelters area needs information on fabrics and composite materials and has asked STD for support. However, because STD did not start a relevant research program, SusD has initiated its own research in this area.

OCR for page 86
--> believes ASCD should attempt to involve other directorates more in modeling their technologies while ASCD concentrates on integrating various technology models into simulations of a soldier-system archetype. Organizational Climate ASCD personnel seemed to be among the most pleased and actively engaged employees of the RDEC. This was evident by the interviewees' genuine expressions of pleasure with their work and their positive outlook for the future. The committee observed that ASCD is a cohesive unit with a productive climate in which innovative ideas are usually tolerated and sometimes encouraged. Information Technology ASCD personnel use information technology to support the progress of programs through the acquisition cycle. Generally, as new information technologies are implemented, ASCD's productivity improves. Personnel appear to be well trained, and appropriate hardware and software are available. Access to the internet and e-mail has connected ASCD personnel with internal and external coworkers, contractors, customers, and external information sources. ASCD's active participation in distributed simulations is a strong indication that these connections are up to date. Facilities and Infrastructure The tours and demonstrations of the ASCD's facilities revealed that they meet modem standards and can support the current operations, but they may not support significant growth, including implementation of some of the improvements suggested in this report. Given the rapid changes in computer technologies, the committee believes that ASCD's modeling and simulations may soon be out of date. In addition, the facilities cannot accommodate the participation of the other directorates in the modeling activities. Overall Assessment The committee arrived at an overall judgment of adequate to good for the Resources and Capabilities pillar. In contrast to other directorates where the committee noted indications of a decline in resources and capabilities, ASCD showed evidence of an upward trend.

OCR for page 86
--> Strategic Vision Pillar Characteristics of the Strategic Vision Pillar Alignment of Vision and Mission Strategic Planning Stakeholder Buy-In Leadership Alignment of Vision and Mission Although many ASCD personnel could not describe the mission and vision statements of their directorate, they did have a good understanding of the RDEC mission and vision statements. ASCD personnel apparently realize that their role is to represent RDEC technologies for the soldier system that are worthy of funding for research, development, and engineering. ASCD helps the RDEC present an integrated, wholly engineered soldier system. ASCD also helps the RDEC demonstrate that its plans and programs contribute meaningfully to the soldier system. Modeling and simulation by ASCD are important to all these endeavors. The commodity directorates currently take advantage of ASCD's modeling and simulation capabilities only in particular situations. The food program used the modeling and simulation capabilities to examine systems for food delivery. The clothing and individual equipment programs used them to model the combat value of ballistic protection items. MobD has used them to model airdrop delivery systems. In all of these cases, ASCD did the modeling and simulation. The commodity directorates did not send their personnel to ASCD, and ASCD did not have the time to train them to use the models and simulations. Some of the ASCD analyses showed that the systems being modeled had little or no combat value. The commodity directorates could not prove the worth of their systems or disprove ASCD's assessment. Unfortunately, this has discouraged the commodity directorates from using the modeling and simulation capabilities of ASCD. The committee believes the commodity directorates can make better use of the RDEC's modeling and simulation capabilities. To begin with, there are only a limited number of instantiations of the soldier system to be modeled and simulated: the 21st Century Land Warrior, a multifaceted, high-technology soldier for the most challenging battlefield scenarios; the special-forces soldier, who is responsible for high performance in unique, specialized missions; the infantry soldier, who is expected to fight in more traditional situations; the peacetime soldier, who is deployed for operations other than war; the combat support soldier, who is responsible for battlefield support that requires improved technologies; and any other soldier who uses the products developed at the Natick RDEC.

OCR for page 86
--> Although assessing the performance for each version of the soldier system is complex, it is possible to determine a finite set of soldier-system archetypes that represent soldiers in various scenarios. The technologies being developed at the Natick RDEC can be modeled and integrated via simulation to support one or more archetypes. From scenarios and simulations, the benefits of specific RD&E programs can be estimated, and their impact on soldier-system performance can be assessed. The committee believes that modeling and simulations by ASCD could contribute much more to an all-encompassing vision of the soldier system. This will require a paradigm shift in ASCD's views of modeling and simulation, however, as well as a change in the level of participation by the commodity directorates and STD. The committee believes the commodity directorates and STD could develop models that reflect the performance of their own technologies, and each group should be responsible for supporting its models within the ASCD modeling and simulation center. ASCD would cooperate with the developer of each model to incorporate the models into a soldier-system archetype. If the benefits of a technology could not be described via a model, the technology might offer little or no benefit for the soldier system. ASCD now has soldier-system models that show the combat worthiness of proposed product-development programs. Some technical performance measures for these models are provided by the other directorates (e.g., information on heat stress and fatigue). However, the committee was told that the scientists and engineers in the other directorates have not provided the full range of technical parameters ASCD needs to model and simulate the performance of products accurately. The committee was astonished at the number of interviewees from the other directorates who could not identify a single technical parameter of the soldier-system their work would influence (although some could speculate). Perhaps the most beneficial aspect of involving other directorates in ASCD's modeling is that scientists and engineers would realize how their work would improve soldier-system performance. In order for an RD&E project to be funded, improving specific technical parameters should be an established goal. The committee believes that ASCD has the potential to advance the work of the entire RDEC if it can involve the other directorates and help them to understand their technical contributions to the soldier-system. ASCD's modeling and simulation capabilities must reflect the full range of other directorates' visions, missions, plans, and programs. Currently, only select technologies, programs, and projects. are represented in the ASCD cadre of models and simulations. If ASCD can include models from each of the RDEC's programs and projects, it can help integrate the commodity directorates and STD with ASCD and ensure that models and simulations represent the value of RDEC products to soldier-system performance.

OCR for page 86
--> Strategic Planning The question most often raised by interviewees was: What role in RDEC strategic planning should ASCD play? Interviewees explained that the recently established Scientific and Technical Advisory Council, which reports to the RDEC director, had taken on the RDEC's strategic planning role and that ASCD was supposed to furnish support, as requested. The information received by the committee, however, did not show that ASCD was making a valuable contribution to strategic planning. The committee suspects that ASCD was undergoing a change in roles and responsibilities related to strategic planning and that ASCD has much to offer in terms of support for strategic planning, but perhaps in a different way than before. ASCD should use the results of high-level models and simulations to confirm that the RDEC's plans and programs will benefit the soldier system. Stakeholder Buy-In ASCD clearly understands the vision and mission of the RDEC, and its personnel are continually developing models and improving simulations. ASCD's part in the overall RDEC mission has been well thought out, but the commodity directorates do not seem to be interested in providing adequate resources for ASCD to implement its mission. As one interviewee told the committee, customers don't always see the value of ASCD's involvement. In the case of the planning process used at the RDEC, however, customers must have recognized the value of ASCD's involvement because the profile of modeling improved greatly after the process went into effect, and funding was increased. Nevertheless, the committee did not see substantial buy-in by all RDEC stakeholders. Leadership The committee observed that the leadership of ASCD had empowered its employees more than in other directorates. RDEC leadership must still correct the residual fragmentation of the old policy of emphasizing parts of the soldier system rather than an integrated whole. The new emphasis is on interdependent subsystems and cooperative programs and projects as the soldier-system concept evolves. Some interviewees were concerned that senior leadership had not effectively communicated the vision of ASCD as a separate entity. Given the understanding by ASCD employees of the big picture, the committee was not particularly troubled by this. However, if future changes in the way ASCD supports the other directorates are made (e.g., as a result of this report), ASCD leadership would do well to renew communications with ASCD employees.

OCR for page 86
--> Overall Assessment The committee arrived at an overall judgment of adequate for the Strategic Vision pillar. The committee observed only a modest trend toward improvement in this pillar, primarily because of the empowerment of ASCD employees and their dedication to developing better models and simulations. The committee notes that substantial improvement will require the cooperation of other directorates in the RDEC. Quality Focus Pillar Characteristics of the Quality Focus Pillar Capacity for Breakthroughs Continuous Improvement Commitment to Quality Structured Processes Learning Environment Quality of Research Capacity for Breakthroughs During the interviews, the committee was told that the problems ASCD tries to solve, with limited resources, do not tend to allow for breakthroughs. The committee understands that breakthroughs in modeling and simulation depend on how robustly the models and simulations are designed, as well as on the potential for the modeling and simulation community to develop new and innovative methods. ASCD is making incremental improvements to models and simulations, as well as in its methods and procedures for supporting the acquisition process over the life cycle of RDEC products. Few if any of the innovations, however, could be called breakthroughs. The committee suggests that the true breakthroughs for ASCD will be (1) persuading the other directorates to participate fully in models and simulations of the soldier system, and (2) using the results to convince higher-level commands and users that the RDEC products add substantial value to the soldier system (assuming, of course, that they do add value). Continuous Improvement The committee observed that ASCD has taken some measures to improve the quality of work and the quality of its support to the RDEC (e.g., better

OCR for page 86
--> modeling). Nevertheless, more attention could be paid to continuous improvement (e.g., better ways to obtain user requirements). Commitment to Quality ASCD demonstrated a commitment to quality through its improved modeling and simulation capabilities that support acquisition decisions. However, the committee did not observe a significant commitment to improving the quality of program management (e.g., better ways to identify, monitor, and manage cost-risk and schedule-risk and better definitions and interpretations of requirements). The committee noted the lack of a requirements database as well as of capabilities for monitoring various acquisition activities. Management has invested some resources in training and implementing quality management, but this has not permeated the organization. Personnel are aware of the need to do a quality job, but measurable objectives for improvement have not been established. Structured Processes ASCD models and simulations impose structure, and government procedures mandate structured programs (e.g., requirements must be explicit). The committee was told that ASCD holds regular team meetings to plan approaches to problems, check the status of programs, identify critical resources, develop contingency plans, and assign tasks. However, the committee did not find definitive processes tailored for supporting the modeling and simulation needs of STD and the commodity directorates. Instead of relying on ad hoc interactions between ASCD and other directorates, ASCD should consider establishing structured processes for assisting STD and the other directorates with modeling and simulation. Learning Environment The committee observed that the RDEC in general supports continuing education for its employees. Training for ASCD personnel is necessary for them to keep up with changing computer technologies (e.g., distributed simulations). Currently, ASCD does not have the resources to train other RDEC personnel in modeling. Within ASCD, teamwork is apparent among personnel assigned to specific tasks; this encourages learning from mistakes because knowledge is retained and shared. However, RDEC-wide learning could be improved by participation of STD and the commodity directorates in the modeling phases of a program.

OCR for page 86
--> Quality of Research No patents have been awarded in ASCD, and only one program, the head mounted display, has been cited as ''very high level.'' The committee observed, however, that ASCD's activities are generally aligned with internal and external customer needs and that the results of ASCD analyses are often reviewed by the Army for the purpose of making acquisition decisions. ASCD personnel have participated in modeling and simulation meetings and have developed models and simulations to represent the soldier system in distributed interactive exercises. Results of the analyses that ASCD supports have been published. However, the quality of soldier-system models and simulations could be improved by more and better information from the other RDEC directorates. The committee believes that the quality of work elsewhere in the RDEC could profit by close interaction with ASCD. Overall Assessment The committee arrived at an overall judgment of adequate for the Quality Focus pillar. The trend was level, but the committee saw the potential for substantial improvement, especially if ASCD and the other directorates can work together more effectively. Customer Focus Pillar Characteristics of the Customer Focus Pillar Customer Satisfaction Customer Involvement Market Diversification Customer Satisfaction ASCD meets customer expectations in many respects. The commodity directorates and RDEC management stated that, for the most part, they were pleased with ASCD support. Modeling and simulation results were provided in a reasonable time with demonstrated technical capabilities for performing operations research. Furthermore, the results were considered useful by the customers. ASCD also received complimentary reviews in the area of liaisons to customers,

OCR for page 86
--> although improvements could be made in requirements development and translation to the commodity directorates. However, the fact that the other directorates have had to do some customer surveys on their own and that strategic planning has been moved to another office suggests that satisfaction with ASCD support is not complete. Well satisfied customers would buy in fully to the value added by ASCD models and simulations. Customer Involvement Interviewees from the commodity directorates indicated that they do participate in analyses performed by ASCD to support RDEC positions or acquisition decisions by the Soldier Systems Command. Interaction between the commodity directorates and ASCD when an analysis is performed and its results are presented to higher-level commands and other DOD organizations means there must be customer involvement and feedback. However, some ASCD personnel mentioned that ASCD does not have enough time, equipment, or resources to increase the involvement of commodity directorates and STD. The committee also learned that customers are sometimes involved in setting program objectives and monitoring progress. Market Diversification The committee judged that diversification in program management assistance, customer liaison, strategic planning, and modeling and simulation can best be measured by the breadth and robustness with which ASCD supports its customers. Many technologies and products at the Natick RDEC need to be modeled and simulated to estimate their combat worth to the soldier system. The committee acknowledges that ASCD's modeling and simulations have supported multifaceted and diverse analyses (e.g., work on head-mounted displays and ASCD's transfer of modeling and simulation improvements to allies, industry, and academia). However, a recurring theme at Natick is that the RDEC can meet customer needs in a crisis but that meeting customer needs under normal conditions seems to be difficult. The commodity directorates do not make full use of ASCD as a resource, especially to facilitate acquisition processes when the bureaucracy gets in the way. The committee believes that ASCD should strive for its modeling and simulations to serve the entire range of technologies and products at the RDEC. ASCD should also support all RDEC research and technology programs by helping sponsors traverse the acquisition cycle without compromising technical quality.

OCR for page 86
--> Overall Assessment The committee arrived at an overall judgment of adequate for the Customer Focus pillar. The trend appeared level, but the committee noted ample opportunities for improvement. Value Creation Pillar Characteristics of the Value Creation Pillar Proper Portfolio Product Performance Cycle Time and Responsiveness Value of Work in Progress Proper Portfolio As a support organization for the entire RDEC, the ASCD portfolio must address the entire range of soldier-system needs and requirements. Although ASCD presented several examples of how it has provided analyses for each of the commodity directorates, in most cases these were special situations (e.g., a commodity directorate recognized a need for a particular analysis to justify a program). Product designs have also been influenced by these analyses. However, the modeling and simulations are not complete enough to treat the soldier-system comprehensively, and modeling and simulations have not been made an integral part of the strategic planning process. The committee observed that problems throughout the RDEC indicate a need to add substance to the seminal concept of the soldier system. The Army procures individual items, each of which may meet a specific need but adds to the large number of items that could be integrated into the soldier system. One alternative to acquiring individual items is to integrate various components into a subsystem of the soldier system (e.g., the Soldier Integrated Protective Ensemble). The best alternative would be to engineer many technologies and their applications into a single soldier system. This alternative is most closely aligned with the 21st Century Land Warrior concept being developed in SurD. In support of the ideal of engineering soldier systems (rather than merely acquiring individual items or integrating components into ensembles), the committee believes that ASCD modeling and simulation could be used to perform operations research analyses for entire soldier systems. This approach would be

OCR for page 86
--> analogous to the way DOD acquires major weapons systems. However, this approach would require a paradigm shift in the ASCD from individual analyses to multifaceted trade-off studies that combine models of candidate technologies into robust simulations of soldier-system archetypes. These simulations would allow various combinations of parts and components (representing technology applications) to be compared to determine the optimal combination of soldier-system components and subsystems for particular missions. Time lines shown to the committee by ASCD suggest that models and simulations of alternative soldier-system concepts could be available between 1999 and 2003. Product Performance ASCD currently satisfies customer needs in many respects. However, current models and simulations must be improved to meet the increasing expectations of DOD. Ultimately, these capabilities should be able to estimate modifications to the soldier system that would best enhance the soldier's combat effectiveness and lead to conclusions in the DOD virtual environment prior to fielding a new subsystem of the soldier system. Development of this analytical capability exceeds the current capabilities of ASCD. Cycle Time and Responsiveness Although some interviewees told the committee that ASCD had delivered analyses in a timely manner, several said that there had been problems. The committee was told that projects have been delayed for many reasons, sometimes because of the need to "fight fires" or to respond to a request for a quick fix or urgent support. The committee believes that, except in the case of urgent support, analysis results could be provided on time. ASCD should work on developing more robust modeling and simulation techniques to allow for responses to quick fixes without causing corresponding delays in the delivery of results to other customers. Value of Work in Progress Some interviewees told the committee that sponsors of programs request the help of ASCD because of its prior excellent work; requests like these demonstrate the value of ASCD support. ASCD employees explained that they had previously considered the soldier system as a collection of parts, but they now consider that the parts work together with the soldier. In other words, understanding of the soldier-system concept has improved. ASCD now models

OCR for page 86
--> TABLE 3-2 Summary of ASCD Assessment Pillar Assessment Trend Resources and Capabilities Adequate to Good Improving Strategic Vision Adequate Improving Quality Focus Adequate Level Customer Focus Adequate Level Value Creation Adequate Improving squads with small arms and bursts of munitions, does a good deal of work on soldier equipment to support SurD, and has plans to include food in its analyses. All models must be validated, however, and the committee was told that ASCD has not been able to collect sufficient data to validate all of its models. Overall Assessment The committee arrived at an overall judgment of adequate for the Value Creation pillar. There were modest indications of an upward trend, particularly in the understanding of the soldier-system concept. ASCD has the potential to add substantial value to the RDEC through advanced modeling and simulation capabilities. Summary The results of the committee's assessment of ASCD are summarized in Table 3-2. ASCD was judged adequate for four pillars and adequate to good for one pillar. Several upward trends were noted. As ASCD continues to improve, the committee is hopeful that it will aspire to provide support for the entire cadre of soldier-system archetypes, subsystems, and components. The committee believes that the functions performed by ASCD are vital to the RDEC's realizing much more of its potential. The role of ASCD spans the entire RDEC, from research on new technologies in STD to the development and engineering of products in the commodity directorates, including support of the entire soldier-system concept.