4

Characteristics and Metrics for a World-Class Army Research, Development, and Engineering Organization

Working from the discussion of pillars in Chapter 3, the committee developed characteristics and descriptions of the metrics associated with them.

CHARACTERISTICS

For the purposes of this study, characteristics are the distinguishing qualities, properties, or features of the pillars. The previous discussion of the five pillars ended with Figure 3-1, which designated 25 important features. These features characterize the pillars, and the committee believes they represent the distinguishing qualities of each pillar. Accordingly, the committee judged these 25 features to be the characteristics most relevant to an Army RD&E organization. A discussion of these characteristics and how they can be assessed appears below. The discussion follows a pillar-by-pillar format, with particular emphasis on determining metrics for evaluating the characteristics of each pillar.

Customer Focus Pillar

Customer focus is directed toward internal customers (e.g., product development teams) and external customers (e.g., soldiers). Both groups of customers can be surveyed to ascertain their satisfaction with the technological solutions and products delivered, the timeliness of delivery, and the quality of technical capabilities and support provided. Customer involvement in setting program objectives and following program progress can also be evaluated.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 31
WORLD-CLASS RESEARCH AND DEVELOPMENT: Characteristics for an Army Research, Development, and Engineering Organization 4 Characteristics and Metrics for a World-Class Army Research, Development, and Engineering Organization Working from the discussion of pillars in Chapter 3, the committee developed characteristics and descriptions of the metrics associated with them. CHARACTERISTICS For the purposes of this study, characteristics are the distinguishing qualities, properties, or features of the pillars. The previous discussion of the five pillars ended with Figure 3-1, which designated 25 important features. These features characterize the pillars, and the committee believes they represent the distinguishing qualities of each pillar. Accordingly, the committee judged these 25 features to be the characteristics most relevant to an Army RD&E organization. A discussion of these characteristics and how they can be assessed appears below. The discussion follows a pillar-by-pillar format, with particular emphasis on determining metrics for evaluating the characteristics of each pillar. Customer Focus Pillar Customer focus is directed toward internal customers (e.g., product development teams) and external customers (e.g., soldiers). Both groups of customers can be surveyed to ascertain their satisfaction with the technological solutions and products delivered, the timeliness of delivery, and the quality of technical capabilities and support provided. Customer involvement in setting program objectives and following program progress can also be evaluated.

OCR for page 31
WORLD-CLASS RESEARCH AND DEVELOPMENT: Characteristics for an Army Research, Development, and Engineering Organization Although an RDEC must focus on the primary markets it serves, the committee believes that some market diversification (i.e., looking to related markets for technology or products) is proper for any RD&E organization. Indeed, in the private sector world-class RD&E organizations seek to exploit fully the results of their research and product development. Market diversification has merit especially for an Army RD&E organization that has DoD-wide or even broader responsibilities. The extent of market diversification can be determined by examining such things as (1) the penetration and extension of markets for products and technologies and (2) whether these markets lead to expanded or entirely new lines of business. However, the committee notes that market diversification must be considered carefully in the case of Army RDECs because they exist primarily to support their Army missions. Also, RDECs rely on government funding, which is usually authorized to satisfy specific government needs rather than to diversify markets served by individual organizations. Measuring customer satisfaction and customer involvement in research, development, and engineering, and the nature and appropriateness of market diversification, indicates how well the RDEC is connected with and focused on the long-term and short-term needs of the various customers the RDEC serves.1 Resources and Capabilities Pillar Resources and capabilities can be evaluated in terms of personnel (i.e., human resources); facilities and infrastructure (i.e., physical resources); budget (i.e., financial resources); RD&E capabilities, skills, and talents (i.e., intellectual resources); the use of external resources; important technologies for each directorate; applied information technology; and organizational climate. Internal (e.g., staff and management) and external (e.g., peer, higher headquarters, and customer) reviews can be conducted periodically to assess resources and capabilities. These reviews may include analyses of the core technical programs, evaluations of employee morale and the research climate, 1   Other indicators can be used to assess customer focus (e.g., an organization 's success at anticipating unstated customer needs or how well an organization “hears” the voice of the customer in determining overall direction). However, the committee decided that, for an Army RDEC, it is preferable to concentrate on satisfaction, involvement, and diversification.

OCR for page 31
WORLD-CLASS RESEARCH AND DEVELOPMENT: Characteristics for an Army Research, Development, and Engineering Organization and assessments of the ability and effectiveness of program managers to acquire technology from outside vendors (i.e., reach “make versus buy” decisions). Measuring the quality and quantity of the human, physical, and financial resources and the core capabilities of the RDEC gives an indication of the ability and power of the RDEC to achieve world-class results. A positive organizational climate is most often correlated with high productivity (Miller et al., 1996). Strategic Vision Pillar The strategic vision must be shared with (i.e., communicated to and discussed with) and understood by staff and stakeholders alike. The vision must then be translated into action. How well this is done depends on the quality of the leadership of the RDEC. Reviews to assess strategic vision should include assessing RDEC leadership. Internal and external (e.g., peer) reviews can determine if the strategic vision and the mission are aligned. They can also determine if anticipatory strategic planning is sufficient to develop future Army and joint service products rapidly. To assess stakeholder buy-in, important classes of stakeholders must first be identified; then the extent to which buy-in by particular stakeholders is critical and the degree to which buy-in is obtained can be assessed. Measuring the quality of the strategic vision of the RDEC (i.e., the extent to which the vision and mission are aligned, to which planning is anticipatory, and to which stakeholders “live” the vision) will give a reading of the enduring capability of the RDEC to plan and achieve world-class results. Value Creation Pillar Value creation is often a perception. Perceived values are based on comparisons of the benefits (or lack thereof) of previous products with current products or of the properties of products and costs. Another perception is whether the right products are being delivered to the right place at the right time. Reviews of the breadth of RD &E (i.e., the portfolio of programs), the performance of products and the benefits of services, cycle time and responsiveness, and the value of work in progress are all important to assessing value creation (Miller et al., 1996). Reviews can be both internal and external evaluations of programs in progress, products developed, or services delivered. Measuring the

OCR for page 31
WORLD-CLASS RESEARCH AND DEVELOPMENT: Characteristics for an Army Research, Development, and Engineering Organization extent to which the RD&E organization produces outstanding, meaningful results (i.e., creates value for customers) yields an understanding of the present and potential impact of the organization. Quality Focus Pillar Several important characteristics are associated with quality focus. First is a capacity for scientific, technological, and engineering breakthroughs. This characteristic can be assessed, in part, by reviewing past performance (e.g., how many breakthroughs have already occurred). Next is the ability to improve continuously, which can be assessed by reviewing specific efforts to improve processes and products. Commitment to quality products and services must be assessed at all levels, from topmost management to the lowest working level. To improve the quality of work, work processes must first be understood, defined, and (to a degree) structured. Structured processes (i.e., working in a disciplined and organized fashion) can be assessed by examining processes and results. Reviews can determine the ability of staff members and the organization as a whole to learn, acquire knowledge (and capabilities), and use this knowledge to achieve outstanding results. Finally, the quality of research can be assessed by expert reviews on several levels (e.g., to determine whether high standards of technical excellence are being maintained). Measurements of all these characteristics can give an overall assessment of the focus on quality in an Army RD&E organization. METRICS For the purposes of this study, metrics are defined as standards for measuring the characteristics of each pillar. Before applying metrics in a particular situation, it is necessary to understand some aspects of metrics in general. This understanding is revealed by the answers to several questions. Who, What, Why, When, and How Metrics can be used in many ways for evaluating an RD&E operation (IRI, 1996). Long lists of metrics have been developed, but selecting

OCR for page 31
WORLD-CLASS RESEARCH AND DEVELOPMENT: Characteristics for an Army Research, Development, and Engineering Organization the most pertinent metrics is important. Critical questions can be asked about using metrics, the answers to which are applicable to Army RDECs. Who develops the metrics? Metrics can be established by the involved group (e.g., an RDEC or its directorates) or by others interested in the performance of the group. The metrics must be understood by those making assessments and by those being assessed. What specific metrics should be developed? Different sets of metrics are meaningful to different groups. Metrics must be useful for the organization; specific metrics will drive the behavior and actions of people within the organization. For an RD&E organization, metrics should foster improvement and be related to the vision and mission of the organization. When selecting metrics one must keep in mind that RD&E efforts are part of a system (Brown and Svensen, 1988). Figure 4-1 shows input into the system, which comes from various sources (e.g., personnel, dollars, equipment, scientific knowledge) and moves through RD&E processes to produce output (e.g., reports, patents, concepts). Product output is converted by receivers into outcome (e.g., new products, services, cost savings, benefits for soldiers). 2 Metrics can be developed to measure (1) the quality of input, (2) the operation of RD&E processes, (3) the quality of output, and (4) the value of outcome. Also, metrics can be developed that relate these four items to the past, present, and future. For example, variations on one measure of an RD&E organization's output might be the number of patents used in the last five years (past), number of patents filed (present), and the number of patent disclosures anticipated during the next five years (future). 2   The portrayal of RD&E as a system is not meant to exclude the myriad of interactions with external stakeholders, especially customers. The diagram is useful for isolating several key RD&E processes. However, from a larger perspective (e.g., an entire business), the RD&E system is only a subsystem.

OCR for page 31
WORLD-CLASS RESEARCH AND DEVELOPMENT: Characteristics for an Army Research, Development, and Engineering Organization FIGURE 4-1 Research, development, and engineering as a system. Why use metrics? Metrics can help an organization assess and determine business or technical objectives, encourage changes, and serve as mechanisms for planning, screening (e.g., setting priorities for improvement), and managing RD&E programs. Metrics can also measure contributions from management, research scientists and engineers, and support staff toward developing, producing, and delivering products. When should metrics not be used? The wrong metrics can be harmful. Some metrics may actually limit performance or lead to inappropriate behavior, actions, or results (e.g., metrics based on false cause-effect relationships or wrong work-process models). When should metrics be developed? Ideally, metrics should be developed as part of the process of setting objectives or part of the evaluation of how well objectives have been met.

OCR for page 31
WORLD-CLASS RESEARCH AND DEVELOPMENT: Characteristics for an Army Research, Development, and Engineering Organization How are metrics developed? Metrics can be developed collectively or individually, and they can be tailored to match the unique environment of the group or organization being assessed. Referring to metrics described by the Industrial Research Institute (IRI, 1996) and others (Roussel et al., 1991; Davidson and Prudent, 1996), the committee developed a set of metrics to assess Army RD &E organizations like the Natick RDEC. Four metrics were developed for each of the 25 characteristics identified at the beginning of this chapter. Of paramount concern during the development of the metrics was the committee's desire to emphasize the importance of demonstrated commitment by the organization's senior leadership (and, where applicable, the next level of command) to attaining and maintaining world-class performance throughout the RD&E organization. Without this commitment, attaining world-class performance will be impossible. Applying Metrics The 100 metrics described below can be used to monitor improvements and assess the Natick RDEC in terms of world-class performance. The metrics can also be used by Natick RDEC or other RDEC personnel for self-evaluation or by higher-level Army commands for evaluating other RD&E organizations. In other situations metrics are used to provide numerical measurements, such as the number of patents written per year or the number of Ph.D.s on staff. Although these measures clearly are relevant to the performance of an RDEC, the committee does not believe that numbers, by themselves, accurately describe the many facets of RDEC performance.3 Accordingly, the metrics chosen for this assessment are based on qualitative descriptors for four levels of performance (Adler 3   “Numerical indexes are easily understood and readily compared.... However, there are also some serious drawbacks associated with quantitative measures that counterbalance their virtues....They don't work well in professional groups, such as R&D organizations, where much of the work is characterized by uncertainty and variability…” (Brown and Gobeli, 1992).

OCR for page 31
WORLD-CLASS RESEARCH AND DEVELOPMENT: Characteristics for an Army Research, Development, and Engineering Organization et al., 1992).4 These levels are poor, adequate, good, and excellent.5 The committee believes for an organization to be deemed world-class, performance must be predominantly excellent. The committee also considered the category of best-in-class, which is a level of performance beyond excellent. Best-in-class is not included in the following metrics because descriptors for this category apply to unique situations (e.g., an organization or process that is, indeed, the very best) and might not be applicable in a situation where there are no comparisons. Table 4-1, Table 4-5 summarize, pillar by pillar, the committee's metrics and the four levels of performance for each characteristic. These tables are not necessarily all inclusive; additional tailored characteristics and metrics may be necessary for some evaluations. The descriptors should be looked upon as conceptual guides that can be modified for specific situations. The results of assessments can be recorded as illustrated in Table 4-6. Overall assessments for each pillar would be arrived at on the basis of judgments of the assessments for each characteristic in that pillar. The committee believes that good or excellent performance in each characteristic, and excellent overall performance in all five pillars, are necessary for an organization to be judged world-class. The results might also be presented in graph form in typical “spider diagrams” (see Figure 4-2). Spider diagrams may have several uses. For example, Figure 4-2 could be a convenient summary showing if an RDEC approaches or achieves world-class performance on the basis of the pillars alone. A more complicated diagram could include radials for each of the 25 characteristics. Spider diagrams for each directorate could provide visual comparisons. In the case of multiple assessments, a spider diagram for the Natick RDEC could be compared with diagrams for other RDECs. 4   Many recent metrics (e.g., the Industrial Research Institute's Technology Value Program) that began with the Adler article now use the four-stage model; hence the committee adopted this model to be more consistent with the current literature and the practice of several companies. 5   The committee also considered using the terms Stage 1, etc. to designate levels of performance. However, the committee was concerned that using numbered stages might lead to a “numbers game” and an inflexible scoring system; hence the committee decided to use words. The words do have some negative and positive connotations, which are intended.

OCR for page 31
WORLD-CLASS RESEARCH AND DEVELOPMENT: Characteristics for an Army Research, Development, and Engineering Organization FIGURE 4-2 Spider diagram. It should be noted that the committee has implicitly given equal weight to all five pillars. Under some circumstances, it may be appropriate to assign greater weight to one pillar or another. Finally, the committee observes that application of the metrics in this report need not be limited to organizations striving to attain world-class performance. For example, if the metrics are used for self-assessment or self-understanding, they should be used in the context of an organization's role, responsibilities, and goals. If the organization's goals do not include world-class performance, then the organization could choose to use the metrics and approach the assessment in a different way. In some cases an organization could decide to use the metrics as a guide for self-assessment and develop its own definitions of poor, adequate, good, or excellent. Also, an organization might wish to focus on some characteristics or pillars more than others. This adaptation process in itself can be important for self-improvement.

OCR for page 31
WORLD-CLASS RESEARCH AND DEVELOPMENT: Characteristics for an Army Research, Development, and Engineering Organization REFERENCES Adler, Paul, William McDonald, and Fred MacDonald. 1992. Strategic Management of Technical Functions. Sloan Management Review. vol. 33 (2): 19–37. Brown, Warren and David Gobeli. 1992. Observations on the Measurements of R&D Productivity: A Case Study. IEEE Transactions on Engineering Management. 39 (4): 325–331. Brown, Mark and Raynold Svenson. 1988. Measuring R&D Productivity. Research-Technology Management. 31 (4): 67–71. Davidson, Jeffrey and Ann Lorette Prudent. 1996. Quality Deployment in R&D Organizations. Research-Technology Management. 39 (1): 49–55. IRI. (Industrial Research Institute). 1996. Measuring the Effectiveness of R&D: A set of 50+ metrics available on a searchable disk. Available from IRI, Suite 1100, 1550 M Street, N.W., Washington, D.C., 20005-1708. (Phone: 202-296-8811). Miller, Joseph, Parry Norling, and John Collette. 1996. Research/Technology Management: Leading, Managing, and Getting Results . Kirk-Othmer Encyclopedia ofChemical Technology. Volume R.In press. Roussel, Philip, Kamal Saad, and Tamara Erickson. 1991. Third Generation R&D. Cambridge, Mass.: Harvard Business School Press.

OCR for page 31
WORLD-CLASS RESEARCH AND DEVELOPMENT: Characteristics for an Army Research, Development, and Engineering Organization Tables 4-1 through 4-6

OCR for page 31
WORLD-CLASS RESEARCH AND DEVELOPMENT: Characteristics for an Army Research, Development, and Engineering Organization TABLE 4-4 Metrics of the Value Creation Pillar Characteristics Performance Level Metrics Proper Portfolio Poor Products are developed that do not meet customer needs; products have poor customer acceptance; customers perceive that commercial alternatives are cheaper, perform better, and are more durable.   Adequate An analytical process to examine the product portfolio is used to design and field products that have greater value and soldier acceptance; results of the analytical process lead to modifications in product design; major changes may be made after fielding the initial product.   Good Portfolio analyses of a program are an integral part of the strategic planning process; there is broad and active customer involvement in the portfolio analysis; programs yield products that have significant customer acceptance, meet or exceed customer requirements, and demonstrate increased value compared to current products or commercial alternatives; minor changes in product design occur after initial fielding.   Excellent Portfolio analyses result in RD&E processes that yield products and services with excellent value, performance, and customer acceptance.

OCR for page 31
WORLD-CLASS RESEARCH AND DEVELOPMENT: Characteristics for an Army Research, Development, and Engineering Organization Product Performance Poor Products do not meet customer requirements (e.g., in terms of weight, volume, function, durability, or maintainability); customers complain that product performance does not meet the developer's claims; products are not suitable for use in certain locations or environmental extremes.   Adequate Products meet customer requirements, needs, and expectations.   Good Products fully meet or exceed customer requirements; products are perceived as better than the ones they replace.   Excellent Products not only exceed customer expectations, but product performance includes some pleasant, unexpected surprises (e.g., reduced maintenance requirements, longer shelf life, longer mean time to failure, resource savings).

OCR for page 31
WORLD-CLASS RESEARCH AND DEVELOPMENT: Characteristics for an Army Research, Development, and Engineering Organization Cycle Time and Responsiveness Poor Cycle time for project completion is longer than anticipated; milestones are routinely missed; program delays result in increased end-item cost; research programs do not anticipate customer needs; management and staff are not flexible to modifications of product requirements.   Adequate Elapsed time from project initiation to project completion is measured and can be reliably forecasted; research programs are described as being on-time and on-budget.   Good RD&E programs are initiated and completed significantly faster than similar government or commercial programs; research staff is responsive to “quick fixes” for troops, and numerous examples are readily available for major products; senior management ensures that adequate resources are reprogrammed to fulfill quick-fix requests.   Excellent RD&E programs are initiated and completed substantially (e.g., one third) quicker than similar government or commercial programs; innovative processes and technical solutions reduce typical quick-fix response times by nearly half; the staff monitors foreign and domestic industrial and academic research for solutions to new and unanticipated technical problems; commanders directly and indirectly express gratitude for responsive quick fixes.

OCR for page 31
WORLD-CLASS RESEARCH AND DEVELOPMENT: Characteristics for an Army Research, Development, and Engineering Organization Value of Work in Progress Poor No evaluations of historical RD&E programs are available for comparison to current programs; no methodology is in place to assess current RD&E programs; customer perception of prior RD&E programs is predominantly critical and negative, and little or no value is placed upon the current programs by the customers.   Adequate A database on select historical RD&E programs and all current programs is available; current RD&E programs are vividly described, and these descriptions are used during peer-review discussions to justify programs and prioritize personnel and budget requests; customer perception of prior RD& E programs is generally positive; customer perception of current RD&E programs is positive (i.e., the products and services will generally meet user requirements and be delivered on time and on budget).   Good A database is maintained on all past major projects (e.g., for the last decade) and their primary and secondary impacts; the database is used for comparison with current RD&E programs; leadership creates a scale to rate continuously the potential value of current programs compared with previous programs and show improvements; customers rate RD&E programs as very good (i.e., products are expected to fully meet or exceed customer requirements; products are perceived as likely to be better than the ones they replace).   Excellent A complete historical database and evaluation methodology are used to demonstrate the value of the organization's products and services; data are used to justify and defend program expenditures; customers rate products and services as excellent (e.g., product performance exceeds customer expectations); product performance exceeds anything projected to be available from domestic and foreign sources for at least several years.

OCR for page 31
WORLD-CLASS RESEARCH AND DEVELOPMENT: Characteristics for an Army Research, Development, and Engineering Organization TABLE 4-5 Metrics of the Quality Focus Pillar Characteristics Performance Level Metrics Capacity for Breakthroughs Poor RD&E programs are routine and unimaginative; there is no evidence of imaginative or innovative solutions being applied to RD&E tasks; resources are directed to meeting specific customer requirements only.   Adequate RD&E programs are characterized by steady but incremental improvement; several innovative solutions can be pointed out; minimal funding is available for programs that anticipate future military requirements.   Good Although most programs are characterized by incremental improvements in technology, the organization has demonstrated several leap-ahead improvements; the organization encourages and funds opportunities to seek truly innovative, moderate-risk solutions.   Excellent Unexpected innovation based on breakthroughs in technology occur fairly regularly among internal and external (cooperative) RD&E programs; moderate- and high-risk research that offers high return receives stable funding; numerous examples of breakthrough research are cited from the previous five to ten years.

OCR for page 31
WORLD-CLASS RESEARCH AND DEVELOPMENT: Characteristics for an Army Research, Development, and Engineering Organization Continuous Improvement Poor There is no tangible evidence of senior management commitment to continuous improvement; the need and ability to focus on continuous improvement are recognized, but not funded; products and services show incremental changes; innovations are not rewarded; solutions from industry and academia are discounted as “not invented here.”   Adequate Quality of the work is discussed and several measures of quality are used routinely; innovative solutions are encouraged; staff members frequently make suggestions for improvement; several changes are made (and documented) each month for improving the work and the output of the organization.   Good The organization takes steps to improve work processes and RD&E results significantly; quality audits are performed periodically by internal and external review groups; numerous improvements can be pointed out; productivity is an important topic of discussion; report cards are issued annually by senior leadership; senior managers have the resources to enact recommendations.   Excellent Greater productivity, enhanced research and product quality, improved customer involvement and satisfaction, and continuing education of the work force are areas of primary interest to senior management; the concepts of continuous improvement and excellent product value are embedded in the goals of each RD&E and support function; there is a systematic analysis of research and support processes to eliminate non-value-added activities; research personnel are renowned for finding innovative solutions to technically difficult problems.

OCR for page 31
WORLD-CLASS RESEARCH AND DEVELOPMENT: Characteristics for an Army Research, Development, and Engineering Organization Commitment to Quality Poor Management espouses a commitment to quality, but no formal process to review and evaluate quality is in place; some quality-related results are managed by exception; the quality of products and services varies between RD&E units in the organization.   Adequate Management is investing resources for total quality training and implementation; the variability of products and services is being measured and tracked; personnel are aware of the importance of quality.   Good Total quality implementation is a major goal in the organization's strategic plans; a framework and methodology for measuring and assessing total quality is in place; measurable objectives for work-process improvement are established; there are methods (e.g., statistical process controls) to improve effectiveness and product quality with existing resources.   Excellent The commitment to total quality is inherent and pervasive throughout the organization; the focus of all measurements is on optimizing the RD&E processes to deliver value; frameworks, such as ISO 9000/2 (international quality standards), Baldrige criteria, or locally developed systems, are used for assessment; recommendations to improve quality are immediately funded and implemented.

OCR for page 31
WORLD-CLASS RESEARCH AND DEVELOPMENT: Characteristics for an Army Research, Development, and Engineering Organization Structured Processes Poor Work processes and procedures are understood and milestones are established, but there is no system of internal or external review; project management results in products or services that are delivered late and over budget; delays result in termination of projects; disciplined approaches to defining problems and the scientific method are rarely used.   Adequate Work processes and procedures are monitored; project costs and milestones are closely tracked; processes are established to improve quality incrementally, contain or reduce RD&E cost, and reduce product cycle time; disciplined approaches and the scientific method are used most of the time.   Good Program managers are flexible and adaptive; senior leadership and staff are receptive to innovative ideas for improving work processes and procedures; product quality and customer focus mean continuous improvement; disciplined approaches and the scientific method are used consistently.   Excellent The senior leadership strives to identify and incorporate best business practices into the organization; processes are considered flexible and not overly restrictive, prescriptive, or bureaucratic; management is focused on achieving superior performance and product quality; emphasis on cross-project management ensures timeliness and the proper allocation of resources; disciplined approaches to problem solving include an extensive network linked to Army technological resources worldwide; the scientific method is strictly followed.

OCR for page 31
WORLD-CLASS RESEARCH AND DEVELOPMENT: Characteristics for an Army Research, Development, and Engineering Organization Learning Environment Poor Senior leadership is characterized as reactive; little if any learning takes place on an organizational basis; some managers and staff learn from mistakes.   Adequate Senior leadership recognizes and communicates the importance of organizational learning; management and staff learn from mistakes and from others; personnel are well networked both inside and outside the organization; teams on one project teach teams assigned to other projects; new skills and techniques are acquired through new hires and continuing professional education.   Good Organizational learning is characterized as adaptive; the organizational climate is conducive to learning; personnel are rewarded and encouraged for taking risks and entrepreneurial initiatives despite occasional mistakes; personnel learn from others and by doing; management experiments with new organizational concepts to discover new ways of doing things.   Excellent Organizational learning is adaptive and anticipatory; research and technical capabilities continually expand, and management anticipates change; traditional and innovative methodologies are used to measure and evaluate organizational learning.

OCR for page 31
WORLD-CLASS RESEARCH AND DEVELOPMENT: Characteristics for an Army Research, Development, and Engineering Organization Quality of Research Poor Research and technology programs are not generally aligned with customer requirements and needs; records of research methodology and results are poor; although recorded in technical reports, data are not published in peer-reviewed journals or cited by other scientists in academia or industry; research results cannot be replicated by scientists and engineers outside the organization.   Adequate Research and technology programs are aligned with customer requirements and needs; research methodology and results are peer-reviewed and published as both technical reports and journal articles; the research staff is invited to participate in scientific meetings and workshops; research results are easily replicated by other laboratories.   Good The research and technology programs are recognized by peers as being of very high caliber; several programs are among the best in the federal government and are described as innovative and original; some patents are awarded.   Excellent The quality of the research and technology programs is considered to be among the best in the world; basic research not only fulfills customer needs, but also anticipates future requirements, thus reducing cycle time for new products; research and technology programs are innovative and state-of-the-art; new procedures, processes, and materials are developed by personnel; numerous patents are issued for RD& E innovations.

OCR for page 31
WORLD-CLASS RESEARCH AND DEVELOPMENT: Characteristics for an Army Research, Development, and Engineering Organization TABLE 4-6 Organizational Assessment   Assessment   Component Poor Adequate Good Excellent Customer Focus Pillar         Customer Satisfaction ____ ____ ____ ____ Customer Involvement ____ ____ ____ ____ Market Diversification ____ ____ ____ ____ Pillar Assessment: ____ ____ ____ ____ Resources and Capabilities Pillar         Personnel Quality ____ ____ ____ ____ Budget ____ ____ ____ ____ RD&E Capabilities, Skills, Talents ____ ____ ____ ____ Use of External Resources ____ ____ ____ ____ Important Technologies ____ ____ ____ ____ Organizational Climate ____ ____ ____ ____ Information Technology ____ ____ ____ ____ Facilities and Infrastructure ____ ____ ____ ____ Pillar Assessment: ____ ____ ____ ____ Strategic Vision Pillar         Alignment of Vision and Mission ____ ____ ____ ____ Anticipatory Strategic Planning ____ ____ ____ ____ Stakeholder Buy-In ____ ____ ____ ____ Leadership ____ ____ ____ ____ Pillar Assessment: ____ ____ ____ ____ Value Creation Pillar         Proper Portfolio ____ ____ ____ ____ Product Performance ____ ____ ____ ____ Cycle Time and Responsiveness ____ ____ ____ ____ Value of Work in Progress ____ ____ ____ ____ Pillar Assessment: ____ ____ ____ ____ Quality Focus Pillar         Capacity for Breakthroughs ____ ____ ____ ____ Continuous Improvement ____ ____ ____ ____ Commitment to Quality ____ ____ ____ ____ Structured Processes ____ ____ ____ ____ Learning Environment ____ ____ ____ ____ Quality of Research ____ ____ ____ ____ Pillar Assessment: ____ ____ ____ ____