Appendices



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 57
--> Appendices

OCR for page 57
This page in the original is blank.

OCR for page 57
--> Appendix A Assessment Model

OCR for page 57
This page in the original is blank.

OCR for page 57
--> TABLE A-1 Customer Focus Category Characteristics Performance Level Descriptions Customer Stage 1 The concept is not understood.   Stage 2 The focus is on the internal customer; attempts are made to align R&D with the needs of the internal customer.   Stage 3 The focus is on the external customer; the external customers' needs are determined by direct contact with scientists; the objective is to ''give them what they want''; Quality Functional Deployment is used to identify customer needs.   Stage 4 The focus extends to the soldier; R&D people spend time in the field to learn about the soldier's business; the objective is to "give them what they need." Customer Satisfaction Stage 1 Customer is dissatisfied with the strategy used to develop the product or service; appropriateness of the technological solutions; fulfillment of the operational capability requirements; technical capability, quality, and performance of the service or product; product cycle time and delivery time of the first equipped unit; technical support for fielded products developed at the RD&E organization; technical capabilities of the product or the service of the organization.   Stage 2 Customer is satisfied with the items listed in Stage 1.   Stage 3 Customer is very satisfied with the items listed in Stage 1.   Stage 4 Customer is delighted with the items listed in Stage 1.

OCR for page 57
--> Characteristics Performance Level Descriptions Customer Involvement Stage 1 Neither internal nor external customers are involved in program planning, evaluation, or early "results" (prototype) testing.   Stage 2 Internal or external customers are sometimes consulted on various aspects of the research program or are involved primarily in program reviews.   Stage 3 Internal customers are involved from time to time in setting program objectives and following progress; there are opportunities for customer feedback.   Stage 4 Customers feel completely involved, almost like partners; customers feel they can and do have a major impact in the life-cycle development of the product or service. Market Diversification Stage 1 Although diversification is an aspect of strategic and business plans, senior management has not effectively broadened the customer-base for products that are developed only for the Army; few joint service RD&E programs are in place.   Stage 2 RD&E programs provide products for the Army and the other uniformed services; the organization provides products to other federal agencies; some of the budget is devoted to developing partnerships with industry and academia.   Stage 3 The organization is assigned DoD lead on joint programs; a significant amount of the budget is devoted to developing partnerships with industry and academia; research partnerships yield products that fulfill military needs and fill a void in the needs of other federal agencies.   Stage 4 The organization's products serve a wide range of customers, including DoD, other U.S. government organizations, and global allies of the United States; much technology is transferred between the organization and the private sector; industry and academic partnerships result in the rapid transfer of technology between the organization and its partners; high quality products are developed, manufactured, and distributed to global customers.

OCR for page 57
--> TABLE A-2 Resources and Capabilities Category Characteristics Performance Level Descriptions Organizational Culture Stage 1 People are considered as statistics; there is a clear hierarchy depending on grade level; there is a need-to-know attitude.   Stage 2 People are valued but are perceived to be interchangeable; management makes some effort to be seen by the workers; high level information is shared with employees.   Stage 3 People are valued for their unique skills; grade level is no barrier to communication; all information except the most confidential information is available to the whole staff.   Stage 4 People are viewed as valued assets and are managed strategically; managers are trained in behavior modification to optimize their dealings with employees; management performance rewards includes rewards for developing people and communicating with employees. Employee Attitude Stage 1 The prevailing attitude is "all for one, me."   Stage 2 Employees are, willing to work hard for 40 hours/week; focus is on entitlements; some attempts are made to assess employee satisfaction.   Stage 3 Employees are anxious for the organization to succeed; employees are willing to make extra efforts to get work done; employees are willing to work on committees, represent the organization at professional societies, and travel on their own time; a formalized employee satisfaction process is in place but actions for improvement do not appear to be linked to the findings.   Stage 4 The concept of "technical vitality" is operative; individuals take responsibility for their own personal and professional growth; focus is on job satisfaction; a formalized employee satisfaction process is in place and is acted upon; a 360-degree feedback is used; developing subordinates is an accepted responsibility.

OCR for page 57
--> Characteristics Performance Level Descriptions People Development Stage 1 No training or development programs are available; there is little encouragement for employees to improve their skills.   Stage 2 Training programs are left to the individual to pursue; training is secondary to service and is not fully budgeted.   Stage 3 Career development programs are in place; training is available at all levels and is given priority; cross-functional development opportunities are encouraged.   Stage 4 Interfunctional and international career opportunities are available; learning by teaching others is encouraged; training time is fully budgeted to avoid service conflicts; the faculty and students at internal training sessions are drawn from all levels of the organization; time off without pay is an option to get further training; off-site collaborations are used to expand knowledge. Budget/Funding Stage 1 Budgets are last year's plus inflation, at best; budgets are severely constrained; mid-year cuts in budgets are made common.   Stage 2 Major projects are constantly in jeopardy because of uncertainty in year-to-year funding; the number of new programs is limited; no new construction programs are funded.   Stage 3 Budgets are linked to strategy; budget levels are determined by customers in negotiation with R&D personnel; one-year time horizons with some following year expectations are typical; resources are leveraged through creative approaches, such as collaborations with industry and academia; management is knowledgeable about funding sources.   Stage 4 Budgets are strongly tied to the organization's strategy; budget levels are determined during strategy setting exercises; time horizons are three years; adjustments are made at the end of each year; a backlog of high-quality unfunded projects is maintained to take advantage of any funding that becomes available.

OCR for page 57
--> Characteristics Performance Level Descriptions RD&E Capabilities Skills, Talents Stage 1 Technical skills, capabilities, and talents are inadequate to support current and future customer requirements; few new techniques and skills are acquired via new hire, continuing education or retraining of personnel; personnel cannot fully operate or maintain available equipment; continuing education is not promoted, encouraged, or funded.   Stage 2 Plans are developed and funding is provided for maintaining the present core capabilities; personnel are trained to operate and maintain equipment and use equipment as specified by the manufacturer; personnel skills are current and competent for their technical specialties.   Stage 3 The organization possesses the skills and talents to fulfill customer requirements for the foreseeable future; new and innovative techniques, skills, and processes are incorporated into RD&E processes; newly acquired skills result in improved product engineering, manufacturing, or performance; new personnel are recruited to bring state-of-the-art techniques into the organization; personnel are encouraged to participate in formal continuing education programs; members of the research staff are encouraged to participate in professional societies, serve on external committees, etc.; program managers recognize new skills that will benefit their programs and plan for the acquisition of these skills and talents.   Stage 4 The research and support staffs are recognized as possessing superb technical and administrative skills and talents; many members of the support staff are recognized as artisans of their trade; research personnel incorporate state-of-the-art techniques into their work and develop pioneering methods of their own; a clearly articulated plan describes how needs and voids in core capabilities are identified and filled; new capabilities that must be developed are also addressed and acted upon; a growing inventory of skills is maintained.

OCR for page 57
--> Characteristics Performance Level Descriptions Intellectual Property Stage 1 Intellectual property is considered the responsibility of the legal department.   Stage 2 Intellectual property is considered the responsibility of the technology people; modest recognition is given for patents regardless of value.   Stage 3 Issues become a shared concern for the organization and technology unit; selective patenting is facilitated by patent liaisons or people who have a good understanding of selection criteria.   Stage 4 Issues are addressed as part of the strategic planning process and are reviewed periodically; licensing is used to speed progress and learning.

OCR for page 57
--> Characteristics Performance Level Descriptions Technology Sourcing Stage 1 The organization relies on internally-developed technology; work is contracted out on an ad hoc basis with little or no planning; contract managers do not ensure that statements of work are fulfilled on time or on budget; no formalized processes are used to determine what work should be outsourced.   Stage 2 Working relationships with universities and other groups have been established; outside participation in professional associations is encouraged; there is no external technology sourcing process but there is a fit with the organization's plans; products and services are obtained from external sources to fulfill the statement of work.   Stage 3 Ongoing relationships with universities, other government labs, and private companies are viewed strategically; the organization is recognized as a "smart buyer" of services and work of other parties; clear make-vs.-buy decisions and criteria are established; outside work is managed in detail; the contribution of external work is clearly demonstrated and adds value and reduces cycle time to the organization.   Stage 4 Partnerships and contracts with organizations that are recognized as the best in their field complement RD&E programs and result in leap-ahead (and occasional breakthrough) technological advances; the organization takes a leadership role in professional societies; outsourcing is considered strategically; establishing alliances and collaborations is considered to be a critical competence; buy-manage-do decisions are based on an established process with clearly identified criteria.

OCR for page 57
--> TABLE A-4 Value Creation Category Characteristics Performance Level Descriptions Portfolio Selection Stage 1 There is no project planning or monitoring and little interfunctional participation in project teams; products are developed that do not meet customer needs because of little customer input.   Stage 2 An analytical process to examine the product portfolio is used to design and field products that have greater value and customer acceptance; a major focus is the expansion of successful business areas; the organization is risk averse so new research approaches are not encouraged and major changes are made to products after they have been introduced.   Stage 3 Portfolio analyses of programs are an integral part of the strategic planning process; there is broad and active customer involvement in the portfolio analysis; risk analysis is incorporated at key phases; projects are schedule driven; criteria are in place for go/no go decisions; there are regular milestone reviews of projects; probability of success is built into the portfolio process, identifying clear values and trade-offs, as is a prioritization process.   Stage 4 Project analysis involves framing the project, identifying alternatives, and making a commitment to action; decision and risk methodologies are used frequently; linkage between criteria and business strategy and line strategy is clear; portfolio analyses result in RD&E processes that yield products and services with excellent value, performance, and customer acceptance.

OCR for page 57
--> Characteristics Performance Level Descriptions Cycle Time and Responsiveness Stage 1 The cycle time for project completion is longer than anticipated, milestones are routinely missed, and program delays result in increased end-item cost; research programs do not anticipate customer needs; management and staff are not flexible to modifications of product requirements.   Stage 2 The elapsed time from project initiation to project completion is measured and can be reliably forecasted; research programs are on time and on budget.   Stage 3 RD&E programs are initiated and completed significantly faster than similar government or commercial programs; research staff is responsive to "quick fixes," and numerous examples of quick fixes to major products are readily available; senior management ensures that adequate resources are reprogrammed to fulfill requests for quick fixes.   Stage 4 RD&E programs are initiated and completed quicker than similar government or commercial programs; innovative processes and technical solutions reduce typical quick-fix response times by nearly half; staff monitors foreign and domestic industrial and academic research for solutions to new and unanticipated technical problems; customers directly and indirectly express gratitude for responsive quick fixes.

OCR for page 57
--> Characteristics Performance Level Descriptions Value of Work in Progress Stage 1 No evaluations of historical RD&E programs are available for comparison to current programs; no methodology is in place to assess current RD&E programs; customer perception of prior RD&E programs is predominantly critical and negative; little or no value is placed upon the current programs by the customers.   Stage 2 A database on select historical RD&E programs and all current programs is available; current RD&E programs are vividly described, and these descriptions are used during peer-review discussions to justify programs and prioritize personnel and budget requests; customer perception of prior and current RD&E programs is generally positive (i.e., the products and services generally meet user requirements and are delivered on time and on budget).   Stage 3 A database is maintained on all past major projects (e.g., for the last decade) and their primary and secondary impacts; the database is used for comparison with current RD&E programs; leadership creates a scale to compare the potential value of current programs to previous programs and show improvements; customers rate RD&E programs as very good (i.e., products are expected to fully meet or exceed customer requirements; products are perceived as likely to be better than the ones they replace).   Stage 4 A complete historical database and evaluation methodology are used to demonstrate the value of the organization's products and services; data are used to justify and defend program expenditures; customers rate products and services as excellent (e.g., product performance exceeds customer expectations); product performance exceeds anything projected to be available from domestic and foreign sources for at least several years.

OCR for page 57
--> TABLE A-5 Quality Focus Category Characteristics Performance Level Descriptions Capacity for Breakthroughs Stage 1 RD&E programs are routine and unimaginative; there is no evidence of imaginative or innovative solutions being applied to RD&E tasks; resources are directed to meeting specific customer requirements only.   Stage 2 RD&E programs are characterized by steady but incremental improvement; several innovative solutions can be pointed out; minimal funding is available for programs that anticipate future military requirements.   Stage 3 Although most programs are characterized by incremental improvements in technology, the organization has demonstrated several leap-ahead improvements; the organization encourages and funds opportunities to seek truly innovative, moderate-risk solutions.   Stage 4 Unexpected innovations based on breakthroughs in technology occur fairly regularly among internal and external (cooperative) RD&E programs; moderate-and high-risk research that offers high return receives stable funding; numerous examples of breakthrough research are cited from the previous five to ten years.

OCR for page 57
--> Characteristics Performance Level Descriptions Continuous Improvement Stage 1 No apparent effort is made by management to improve processes; solutions from industry and academia are discounted as "not invented here."   Stage 2 A quality model has been adopted but is not universal; innovative solutions are encouraged, and staff members frequently make suggestions for improvement; some training is offered; ideas for improvement are solicited and acted upon.   Stage 3 Learning from others is encouraged; quality audits are performed periodically by internal and external review groups; report cards are issued annually by senior leadership; external surveys are conducted; ISO 9000 certification is widely sought; internal quality assessments are used.   Stage 4 Greater productivity, enhanced research and product quality, improved customer involvement and satisfaction, and continuing education of the workforce are areas of primary interest to senior management; there is a systematic analysis of research and support processes to eliminate non-value-added activities; benchmarking is used proactively; best practices are shared; focus is on the external customer; the Baldrige National Quality Award criteria are used to drive improvements in performance. Teams Stage 1 There is frequent turnover of team staffing; project leaders are not trained and their roles are not defined; charters for teams are not clear.   Stage 2 There is some interfunctional participation; teams are mostly stable but run into conflicts with functional priorities; project leaders are given some guidance; teams have formal charters.   Stage 3 Interfunctional teams are used when needed; responsibility are delegated to teams; training for project leaders is provided; charters are published widely; a process is in place to facilitate interteam communications.   Stage 4 Teams are critical to success; there are rewards for team performance; teams are often self-managed.

OCR for page 57
--> Characteristics Performance Level Descriptions Evaluation and Rewards Stage 1 The process is a mystery.   Stage 2 There are no meaningful rewards except for salary increases; there is no correlation between rewards and performance level.   Stage 3 Written evaluations and periodic reviews are utilized; focus is on improvements in performance; rewards and recognition relate to performance; recognition is given for suggestions and ideas.   Stage 4 Multilevel feedback is used to focus on improvement; team and individual awards are given; a single committee makes decisions on salaries, rewards, and promotions for all business functions. Project Management Stage 1 Each project is treated as a new experience; the management process is ad hoc.   Stage 2 Several stage-gate models1 are used on most projects; the process itself is not reviewed.   Stage 3 A uniform approach to project management is used; a strong focus is on up-front planning; reducing cycle time is an important factor; some projects are reviewed, but there is no ongoing use of reviews to improve the process.   Stage 4 A standard methodology is used for all projects and is widely accepted; the methodology is applied to widely different projects; the inclusion of new technology is encouraged; each project is considered to be a learning opportunity, and there are formal ways to modify the process. 1 A stage-gate model divides a process (like a product development model) into several subprocesses. At the end of each subprocess is a "gate." Before passing through the gate, certain requirements must be met. Once those have been met, the organization can move to the next subprocess, which has its own gate. An efficient organization has a single stage-gate model, the benefit of which is that each team can learn from the prior team's experience.

OCR for page 57
--> Characteristics Performance Level Descriptions Regulatory Compliance Stage 1 Management shows no interest in complying with regulations; the prevailing attitude is "do what you can get away with."   Stage 2 Formal policies are in place; internal audits are used to enforce compliance.   Stage 3 A strong effort is made to comply; an organizational policy is in place that is supported by management's actions.   Stage 4 Well publicized organizational ethics provide overall guidance; proactive efforts are made to protect the environment; management takes the lead in regulatory compliance. Commitment to Quality Stage 1 Management espouses a commitment to quality, but no formal process to review and evaluate quality is in place; some quality-related results are managed by exception; the quality of products and services varies between RD&E units in the organization.   Stage 2 Management invests resources for total quality training and implementation; the variability of products and services is measured and tracked; personnel are aware of the importance of quality.   Stage 3 Total quality implementation is a major goal in the organization's strategic plan; a framework and methodology for measuring and assessing quality is in place; measurable objectives for work-process improvement are established; there are methods (e.g., statistical process controls) to improve effectiveness and product quality with existing resources.   Stage 4 The commitment to total quality is inherent and pervasive throughout the organization; the focus of all measurements is on optimizing the RD&E processes to deliver value; frameworks, such as ISO 9000/2 (international quality standards), the Baldrige National Quality Award criteria, or locally developed systems, are used for assessment; recommendations to improve quality are immediately funded and implemented.

OCR for page 57
--> Characteristics Performance Level Descriptions Process Management Stage 1 The focus is not on core processes; activities are thought to be unrelated processes with no clear priorities.   Stage 2 Some important processes emerge from the profusion of activities; some processes are managed systematically; processes are monitored against budgets and milestones.   Stage 3 There is a high-level overview of the key processes; processes are prioritized; process performance is judged against external standards and internal milestones; senior leadership and staff are receptive to innovative ideas for improving work processes and procedures.   Stage 4 Process management is well established and accepted; processes are recognized at the organizational level, the process level, and the job level; links between levels unify the strategy throughout the organization; the senior leadership strives to identify and incorporate best business practices into the organization. Metrics Stage 1 Focus is on the short term.   Stage 2 Measures include customer satisfaction; the goal is to ensure that customer requirements are met; the search is for "the few key measures" of progress toward that goal.   Stage 3 A variety of measures linked to the corporate goals are used; the organization recognizes that different measures are needed for different purposes; measures related to cost, time, and quality are used.   Stage 4 A balanced list of measures is used to ensure that all key aspects of the organization are considered including financial, external and internal customers, innovation and learning and societal perspectives; the emphasis is on measuring customer value.

OCR for page 57
--> Characteristics Performance Level Descriptions Safety Stage 1 Safety is not fully addressed; employees are chastised for accidents.   Stage 2 Safety is said to be important, but management does not ''walk the talk,'' and little thought is given to safety training.   Stage 3 Managers "walk the talk"; safety awards are given for group successes; organizational standards are posted and compared to industry standards.   Stage 4 Safety issues are addressed as part of strategic planning; employees receive behavioral modification training; safety is addressed at every management meeting; employees take pride in their safety records.

OCR for page 57
--> Characteristics Performance Level Descriptions Knowledge and Learning Stage 1 Senior leadership is characterized as reactive; knowledge as an asset is not recognized; knowledge tends to be associated with degree level.   Stage 2 Senior leadership recognizes and communicates the importance of organizational learning; personnel are well networked both inside and outside the organization; best practices are shared within functions but not beyond; knowledge tends to be associated with level in the company; new skills and techniques are acquired through new hires and continuing professional education.   Stage 3 Organizational learning is characterized as adaptive; the need for sharing best practices is recognized; personnel are rewarded and encouraged for taking risks and entrepreneurial initiatives despite occasional mistakes; systems are in place to promote information-sharing; there are rewards for proactive information-sharing; employees are encouraged to take risks and are not chastised for failures.   Stage 4 The concept of a "Learning Organization" is understood and valued by management; organizational learning is adaptive and anticipatory; the organization works to differentiate itself on the basis of the tacit and specific knowledge of its people; attempts are made to quantify the value of intangible assets.

OCR for page 57
This page in the original is blank.