Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 59
7
Organizational Structures
and Related Issues
In this chapter, we focus on two topics: the lack of enforcement of
existing U.S. Department of Defense (DOD) guidelines and procedures
and the role of the program manager in the acquisition process.
ENFORCEMENT OF DOD DIRECTIVES AND PROCEDURES
Conclusion 10: Many of the critical problems in the U.S. Department
of Defense acquisition can be attributed to the lack of enforcement of
existing directives and procedures rather than to deficiencies in them
or the need for new ones.
Christie (2011) discussed this issue and pointed to several aspects
of it:
1. a lack of discipline in decision making concerning advancement of
programs through the defense acquisitions milestone system;
2. unfortunate incentives that result in overly optimistic initial state -
ments of systems requirements as well as optimism regarding the
expeditiousness of development and the costs of production and
fielding;
3. failure to rigorously demonstrate, through empirical testing, the
required technological maturity of a component or subsystem
before each major milestone decision point;
59
OCR for page 60
60 INDUSTRIAL METHODS FOR EFFECTIVE DEVELOPMENT AND TESTING
4. failure to first establish and then to carry along event-based
strategies—instead employing schedule-based strategies—and fail-
ure to use strict pass/fail criteria for each phase of development;
5. failure to carry out continuous, independent assessments of the
effectiveness and suitability of defense systems in development
from initial development through the various stages of testing and
production, extending to early introduction to the field; and
6. failure to use feedback loops to inform the broad acquisition com-
munity as to when acquisition methods have worked and when
they have failed so that all can learn from others’ experiences.
We discuss several of these factors throughout this report.
The following actions, some of which are discussed in the report, can
help ameliorate these problems:
• Competitive prototype development and testing should be a strict
prerequisite for any new development program prior to entry into
engineering and manufacturing development.
• Emphasis should be on an event-based strategy, rather than a
schedule-based strategy, with meaningful and realistic pass/fail
criteria for each stage of development. In particular, systems should
not be allowed to proceed to operational testing unless that step is
supported by developmental test performance that strongly antici-
pates that the system will pass; such a determination can be greatly
aided through the conduct of a rigorous operational test readiness
review.
• Use of continuous and independent evaluation tracking of each
program through the stages of development, testing, and produc-
tion should be required. These assessments should rely heavily on
empirical tests and should focus on those capabilities that were the
basis for program approval.
Problems with suitability performance of defense systems are just as
widespread, and the Defense Science Board (2008) made the following
recommendations for remedying them:
1. Identify reliability, availability, and maintainability (RAM) require-
ments during the joint capabilities integration development system
process, and “incorporate them in the request for proposal as a
mandatory contractual requirement” (p. 6).
2. When evaluating proposals, evaluate the bidder’s approaches to
satisfying RAM requirements.
OCR for page 61
61
ORGANIZATIONAL STRUCTURES AND RELATED ISSUES
3. Include a robust reliability growth program as a mandatory con-
tractual requirement and document progress during each major
program review.
4. Include a credible reliability assessment as part of the various
stages of technical review.
5. Raise the importance of achieving RAM-related goals in the respon-
sibilities of program managers.
6. Develop a standard for RAM development and testing.
7. Increase the available expertise in reliability engineering.
THE ROLE OF A PROGRAM MANAGER
The concept of having a strong project manager, sometimes called
a chief engineer, was pioneered by Honda. It was pervasive in Japan as
early as the 1980s (Box et al., 1988) and has become a standard practice
in the automotive industry in the United States. The program manager is
appointed early in an acquisition process, as soon as product feasibility
is demonstrated through a successful market study. The program man -
ager’s responsibility covers the entire spectrum, from planning, design,
development, and manufacturing to even initial phases of sales and field
support.
The organizational structure of the teams and implementation
details vary across companies, but there is usually continuity with a
few team members going from one phase to be part of the team for the
next phase. This practice ensures a smooth transition as well as transfer
of knowledge. But the key person is the program manager, who is fully
responsible and accountable for all phases of the product realization
process. If the system has difficulties in development, such as delays
or cost increases, or if the system underperforms when fielded, final
responsibility lies with the program manager. A strong program man -
ager has the authority to assemble the right team members from inside
the corporation; to hire or contract other needed skills; to approve final
designs, requirements, vendors and suppliers; and to set the final sched -
ule. Input from all divisions—including sales, marketing, dealers, and
manufacturing plants is actively solicited—but the final decisions are
made by the program manager. Other industries, besides automotive,
also use a similar concept of having a single person in charge of the
entire product realization process.
The same activities occur in DOD programs in the broader context
of the acquisition cycle. Every program is managed sequentially through
phases, all followed by major milestones in which decision makers
OCR for page 62
62 INDUSTRIAL METHODS FOR EFFECTIVE DEVELOPMENT AND TESTING
approve or disapprove of the acquisition strategy before the program
moves to the next phase of development.1
For DOD programs, however, there are two people with the title
of program manager. One is appointed by the defense contractor and
generally remains in charge for an extended period of time. The other
is designated by DOD: that person is typically a military officer whose
chief responsibility is to manage the system development to the next mile-
stone, but his or her tenure is often shorter than the time span between
milestones. Tenures have been lengthening of late, but they are still much
shorter than development times. The DOD norm is that after a program
manager’s tour is concluded, the person is generally promoted and
replaced, and the status of the acquisition program during that person’s
tenure is not carefully assessed (as it often is in industry). The short tenure
and lack of accountability lead to disincentives. For example, there is no
motivation for a program manager to be comprehensive in discovering
system defects or design flaws in the early stages of system development.
Furthermore, given the turnover, any deficiencies are unlikely to be attrib-
uted to the efforts of a specific program manager. This approach is in stark
contrast with industry, which has more stability and the right incentive
structure, which includes being aggressive about finding system defects
as early in system development as possible.2
The panel recognizes the challenges associated with program man-
agement and does not expect any significant changes to the present sys-
tem of short-term rotations of military officers as program managers.
Nevertheless, we believe that DOD should explore ways to provide more
stability, and thereby accountability, to the project management process.
Two possibilities include (1) developing a new civilian position in which
a person can serve as deputy to each of the program managers and whose
tenure spans a substantial portion of the system development cycle, and
(2) appointing a deputy program manager at each milestone with the
expectation that the deputy will be promoted to program manager. 3 Of
course, the problem with the incentive structure for program managers
will remain, and it is unclear how they would respond to a civilian or to
a deputy.
Regardless of these possibilities, the panel believes that there has to
be an independent third-party assessment of ACAT (acquisition category)
1See Appendix B for an overview of the defense acquisition process; see U.S. General
Accounting Office (1998) for the role of a program manager.
2For an analogous discussion of space systems, see Defense Science Board (2003).
3Bell (2008:277) argues: “On the other hand, PMs and their PMOs have to start taking
the long-term or enterprise view. That is, it is not OK to delay the discovery of technical,
schedule, or budget problems until a future PM has no choice but to acknowledge them.
PMs need to be rewarded for solving problems, not for postponing them.”
OCR for page 63
63
ORGANIZATIONAL STRUCTURES AND RELATED ISSUES
I systems whenever a program manager leaves. This assessment needs to
be carried out by personnel who are outside the influence of the services
and, in particular, external to the acquisition contract for the program.
This assessment would allow for the progress of the system under that
program manager to be determined objectively. Moreover, the success of
each new program manager should be assessed only on the basis of the
status and progress from the point of transition. Such an assessment may
also change the incentive structure: each program manager will have an
incentive to discover design flaws because the improvement of the system
under his or her tenure would now be directly evaluated.
We do not offer any suggestions on how the performance of program
managers should be assessed if they failed to discover design flaws and
system defects. Also, guidelines would have to be developed on how
problems from earlier stages of development—for example, that a sys -
tem’s performance was not comprehensively tested or discovered flaws
were left unaddressed—would affect the assessment of subsequent pro-
gram managers.
Recommendation 5: The Under Secretary of Defense for Acquisi-
tion, Technology, and Logistics should provide for an independent
evaluation of the progress of acquisition category I (ACAT I) sys-
tems in development when there is a change in program manager.
This evaluation should include a review by the Office of the Sec-
retary of Defense (OSD), complemented by independent scientific
expertise as needed, to address outstanding technical manufactur-
ing and capability issues, to assess the progress of a defense system
under the previous program manager, and to ensure that the new
program manager is fully informed of and calibrated to present and
likely future OSD concerns.
Clearly, there are many details and challenges associated with devel-
oping and implementing this recommendation. These are beyond the
panel’s scope and expertise, but we conclude there are systemic problems
with the current system of program management that are obstacles to the
implementation of efficient practices.
OCR for page 64
