National Academies Press: OpenBook
Suggested Citation:"Front Matter." National Research Council. 2015. Reliability Growth: Enhancing Defense System Reliability. Washington, DC: The National Academies Press. doi: 10.17226/18987.
×

Reliability Growth

ENHANCING DEFENSE SYSTEM RELIABILITY

Panel on Reliability Growth Methods for Defense Systems

Committee on National Statistics

Division of Behavioral and Social Sciences and Education

NATIONAL RESEARCH COUNCIL
                              OF THE NATIONAL ACADEMIES

THE NATIONAL ACADEMIES PRESS

Washington, D.C.

www.nap.edu

Suggested Citation:"Front Matter." National Research Council. 2015. Reliability Growth: Enhancing Defense System Reliability. Washington, DC: The National Academies Press. doi: 10.17226/18987.
×

THE NATIONAL ACADEMIES PRESS    500 Fifth Street, NW    Washington, DC 20001

NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance.

This study was supported by an award between the National Academy of Sciences and the U.S. Department of Defense through the National Science Foundation. Support for the work of the Committee on National Statistics is provided by a consortium of federal agencies through a grant from the National Science Foundation (award number SES-0453930). Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the view of the organizations or agencies that provided support for this project.

International Standard Book Number-13: 978-0-309-31474-9
International Standard Book Number-10: 0-309-31474-7

Additional copies of this workshop summary are available for sale from the National Academies Press, 500 Fifth Street, NW, Keck 360, Washington, DC 20001; (800) 624-6242 or (202) 334-3313; http://www.nap.edu.

Copyright 2015 by the National Academy of Sciences. All rights reserved.

Printed in the United States of America

Suggested citation: National Research Council. (2015). Reliability Growth: Enhancing Defense System Reliability. Panel on Reliability Growth Methods for Defense Systems, Committee on National Statistics, Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.

Suggested Citation:"Front Matter." National Research Council. 2015. Reliability Growth: Enhancing Defense System Reliability. Washington, DC: The National Academies Press. doi: 10.17226/18987.
×

THE NATIONAL ACADEMIES

Advisers to the Nation on Science, Engineering, and Medicine

The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Ralph J. Cicerone is president of the National Academy of Sciences.

The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. C. D. Mote, Jr., is president of the National Academy of Engineering.

The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Victor J. Dzau is president of the Institute of Medicine.

The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Ralph J. Cicerone and Dr. C. D. Mote, Jr., are chair and vice chair, respectively, of the National Research Council.

www.national-academies.org

Suggested Citation:"Front Matter." National Research Council. 2015. Reliability Growth: Enhancing Defense System Reliability. Washington, DC: The National Academies Press. doi: 10.17226/18987.
×

This page intentionally left blank.

Suggested Citation:"Front Matter." National Research Council. 2015. Reliability Growth: Enhancing Defense System Reliability. Washington, DC: The National Academies Press. doi: 10.17226/18987.
×

PANEL ON RELIABILITY GROWTH
METHODS FOR DEFENSE SYSTEMS

ARTHUR FRIES (Chair), Institute for Defense Analyses, Alexandria, VA

W. PETER CHERRY, Science Applications International Corporation (retired), Ann Arbor, MI

ROBERT G. EASTERLING, Statistical Consultant, Cedar Crest, NM

ELSAYED A. ELSAYED, Department of Industrial and Systems Engineering, Rutgers University

APARNA V. HUZURBAZAR, Statistical Sciences Group, Los Alamos National Laboratory, Los Alamos, NM

PATRICIA A. JACOBS, Operations Research Department, Naval Postgraduate School, Monterey, CA

WILLIAM Q. MEEKER, JR., Department of Statistics, Iowa State University

NACHI NAGAPPAN, Empirical Software Engineering Group, Microsoft Research, Redmond, WA

MICHAEL PECHT, Center for Advanced Life Cycle Engineering, University of Maryland

ANANDA SEN, Department of Family Medicine, University of Michigan Health System

SCOTT VANDER WIEL, Statistical Sciences Group, Los Alamos National Laboratory, Los Alamos, NM

MICHAEL L. COHEN, Study Director

ERNEST SEGLIE, Consultant

MICHAEL J. SIRI, Program Associate

Suggested Citation:"Front Matter." National Research Council. 2015. Reliability Growth: Enhancing Defense System Reliability. Washington, DC: The National Academies Press. doi: 10.17226/18987.
×

COMMITTEE ON NATIONAL STATISTICS
2013-2014

LAWRENCE D. BROWN (Chair), Department of Statistics, The Wharton School, University of Pennsylvania

JOHN M. ABOWD, School of Industrial and Labor Relations, Cornell University

MARY ELLEN BOCK, Department of Statistics, Purdue University

DAVID CARD, Department of Economics, University of California, Berkeley

ALICIA CARRIQUIRY, Department of Statistics, Iowa State University

MICHAEL E. CHERNEW, Department of Health Care Policy, Harvard Medical School

CONSTANTINE GATSONIS, Center for Statistical Sciences, Brown University

JAMES S. HOUSE, Survey Research Center, Institute for Social Research, University of Michigan

MICHAEL HOUT, Department of Sociology, New York University

SALLIE KELLER, Virginia Bioinformatics Institute at Virginia Tech, Arlington, VA

LISA LYNCH, The Heller School for Social Policy and Management, Brandeis University

COLM O’MUIRCHEARTAIGH, Harris School of Public Policy Studies, University of Chicago

RUTH PETERSON, Criminal Justice Research Center, Ohio State University

EDWARD H. SHORTLIFFE, Department of Biomedical Informatics, Columbia University, and Department of Biomedical Informatics, Mayo Clinic Campus of Arizona State University

HAL STERN, Donald Bren School of Information and Computer Sciences, University of California, Irvine

CONSTANCE F. CITRO, Director

JACQUELINE R. SOVDE, Program Coordinator

Suggested Citation:"Front Matter." National Research Council. 2015. Reliability Growth: Enhancing Defense System Reliability. Washington, DC: The National Academies Press. doi: 10.17226/18987.
×

Acknowledgments

We first thank Frank Kendall, the Under Secretary of Defense for Acquisition, Technology, and Logistics (AT&L) and Michael Gilmore, the Director of Operational Test and Evaluation (DOT&E) for their interest in and support for this study. Over the past 20 years, these two individuals and their predecessors have provided support for a series of related projects that have produced useful studies and, equally important, helped to establish a greater degree of collaboration between the defense testing community and leading members of the statistical, system engineering, and software engineering disciplines.

We also thank our primary contacts at the U.S. Department of Defense (DoD), Nancy Spruill, director of AT&L acquisition resources and analysis, and Catherine Warner, DOT&E science advisor, who were always ready to assist the panel’s work. They helped us clarify the issues for which we could have the greatest impact, they identified appropriate DoD staff to provide presentations at panel meetings to help us better understand the department’s current environment and operations, and they provided us with DoD documents (e.g., handbooks, guidances, memos, etc.) relevant to our study. We are greatly indebted to these four people for their help to the panel.

The panel members are also very indebted to the many experts who provided presentations at our first three panel meetings: Darryl Ahner (AFIT/ENS), Karen T. Bain (NAVAIR), Gary Bliss (USD AT&L), Albert (Bud) Boulter (USAF SAF/AQRE), Steve Brown (Lennox), David Burdick (Boeing), Michael J. Cushing (AEC, retired), Paul Ellner (AMSAA), Michael Gilmore (DOT&E), Martha Gardner (General Electric), Don Gaver (NPGS), Jerry Gibson (ASC/ENDR), Lou Gullo (Raytheon), Brian Hall (ATEC), Frank

Page viii Cite
Suggested Citation:"Front Matter." National Research Council. 2015. Reliability Growth: Enhancing Defense System Reliability. Washington, DC: The National Academies Press. doi: 10.17226/18987.
×

Kendall (USD AT&L), Shirish Kher (Alcatel-Lucent), Eric Loeb (DOT&E), Andy Long (LMI), William McCarthy (OPTEVFOR), Stephan Meschter (BAE Systems), Andy Monje (DASD SE), Ken Neubeck (Exelisinc), David Nicholls (RIAC), Paul Shedlock (Raytheon), Tom Simms (USD AT&L), Nozer Singpurwalla (GWU), Jim Streilein (DOT&E), Patrick Sul (DOT&E), Daniel Telford (AFOTEC), Nicholas Torellis (OSD), Tom Wissink (Lockheed Martin), James Woodford (ASN(RD&A)), and Guangbin Yang (Ford).

We are grateful for the help from Michael Siri on administrative arrangements, and we thank Eugenia Grohman for extremely comprehensive technical editing.

The report has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise, in accordance with procedures approved by the Report Review Committee of the National Research Council. The purpose of this independent review is to provide candid and critical comments that assist the institution in making its reports as sound as possible and to ensure that the reports meet institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process.

The panel thanks the following individuals for their review of the interim report: Karen T. Bain, Reliability and Maintainability, NAVAIR; Michael J. Cushing, U.S. Army Evaluation Center (Retired); Kathleen V. Diegert, Statistics and Human Factors, Sandia; Richard T. Durrett, Math Department, Duke University; Millard S. Firebaugh, Department of Mechanical Engineering, University of Maryland, College Park; Donald P. Gaver, Jr., Operations Research, Emeritus, U.S. Naval Postgraduate School; Pradeep Lall, Department of Mechanical Engineering, University of Auburn; Paul E. Shedlock, Reliability and System Safety Department, Engineering Product Support Directorate, Raytheon Company; Neil G. Siegel, Office of the Chief Technology Officer, Northrop Grumman Information Systems; and Marlin U. Thomas, Department of Industrial and Operations Engineering, University of Michigan.

Although the reviewers listed above have provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations, nor did they see the final draft of the report before its release. The review of the report was overseen by Thom J. Hodgson, Fitts Industrial and Systems Engineering Department, North Carolina State University and Roderick J. Little, Department of Biostatistics, School of Public Health, University of Michigan. Appointed by the National Research Council, they were responsible for making certain that the independent examination of this report was carried out in accordance with institutional procedures and that all review comments were carefully considered. Responsibility for the final content of the report rests entirely with the

Suggested Citation:"Front Matter." National Research Council. 2015. Reliability Growth: Enhancing Defense System Reliability. Washington, DC: The National Academies Press. doi: 10.17226/18987.
×

authoring committee and the National Research Council. We also thank Ali Mosleh, Henry Samueli School of Engineering and Applied Science, University of California, Los Angeles, for his review of Appendix D. Finally, we thank the panel members who drafted large sections of this report and who devoted a great deal of time, energy, and expertise to this effort and always found time to contribute to the work.

Arthur Fries, Chair
Michael L. Cohen, Study Director
Panel on Reliability Growth Methods
for Defense Systems

Suggested Citation:"Front Matter." National Research Council. 2015. Reliability Growth: Enhancing Defense System Reliability. Washington, DC: The National Academies Press. doi: 10.17226/18987.
×

This page intentionally left blank.

Suggested Citation:"Front Matter." National Research Council. 2015. Reliability Growth: Enhancing Defense System Reliability. Washington, DC: The National Academies Press. doi: 10.17226/18987.
×

This page intentionally left blank.

Suggested Citation:"Front Matter." National Research Council. 2015. Reliability Growth: Enhancing Defense System Reliability. Washington, DC: The National Academies Press. doi: 10.17226/18987.
×
Page R1
Suggested Citation:"Front Matter." National Research Council. 2015. Reliability Growth: Enhancing Defense System Reliability. Washington, DC: The National Academies Press. doi: 10.17226/18987.
×
Page R2
Suggested Citation:"Front Matter." National Research Council. 2015. Reliability Growth: Enhancing Defense System Reliability. Washington, DC: The National Academies Press. doi: 10.17226/18987.
×
Page R3
Suggested Citation:"Front Matter." National Research Council. 2015. Reliability Growth: Enhancing Defense System Reliability. Washington, DC: The National Academies Press. doi: 10.17226/18987.
×
Page R4
Suggested Citation:"Front Matter." National Research Council. 2015. Reliability Growth: Enhancing Defense System Reliability. Washington, DC: The National Academies Press. doi: 10.17226/18987.
×
Page R5
Suggested Citation:"Front Matter." National Research Council. 2015. Reliability Growth: Enhancing Defense System Reliability. Washington, DC: The National Academies Press. doi: 10.17226/18987.
×
Page R6
Suggested Citation:"Front Matter." National Research Council. 2015. Reliability Growth: Enhancing Defense System Reliability. Washington, DC: The National Academies Press. doi: 10.17226/18987.
×
Page R7
Page viii Cite
Suggested Citation:"Front Matter." National Research Council. 2015. Reliability Growth: Enhancing Defense System Reliability. Washington, DC: The National Academies Press. doi: 10.17226/18987.
×
Page R8
Suggested Citation:"Front Matter." National Research Council. 2015. Reliability Growth: Enhancing Defense System Reliability. Washington, DC: The National Academies Press. doi: 10.17226/18987.
×
Page R9
Suggested Citation:"Front Matter." National Research Council. 2015. Reliability Growth: Enhancing Defense System Reliability. Washington, DC: The National Academies Press. doi: 10.17226/18987.
×
Page R10
Suggested Citation:"Front Matter." National Research Council. 2015. Reliability Growth: Enhancing Defense System Reliability. Washington, DC: The National Academies Press. doi: 10.17226/18987.
×
Page R11
Suggested Citation:"Front Matter." National Research Council. 2015. Reliability Growth: Enhancing Defense System Reliability. Washington, DC: The National Academies Press. doi: 10.17226/18987.
×
Page R12
Page xiii Cite
Suggested Citation:"Front Matter." National Research Council. 2015. Reliability Growth: Enhancing Defense System Reliability. Washington, DC: The National Academies Press. doi: 10.17226/18987.
×
Page R13
Suggested Citation:"Front Matter." National Research Council. 2015. Reliability Growth: Enhancing Defense System Reliability. Washington, DC: The National Academies Press. doi: 10.17226/18987.
×
Page R14
Next: Summary »
Reliability Growth: Enhancing Defense System Reliability Get This Book
×
Buy Paperback | $60.00 Buy Ebook | $48.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

A high percentage of defense systems fail to meet their reliability requirements. This is a serious problem for the U.S. Department of Defense (DOD), as well as the nation. Those systems are not only less likely to successfully carry out their intended missions, but they also could endanger the lives of the operators. Furthermore, reliability failures discovered after deployment can result in costly and strategic delays and the need for expensive redesign, which often limits the tactical situations in which the system can be used. Finally, systems that fail to meet their reliability requirements are much more likely to need additional scheduled and unscheduled maintenance and to need more spare parts and possibly replacement systems, all of which can substantially increase the life-cycle costs of a system.

Beginning in 2008, DOD undertook a concerted effort to raise the priority of reliability through greater use of design for reliability techniques, reliability growth testing, and formal reliability growth modeling, by both the contractors and DOD units. To this end, handbooks, guidances, and formal memoranda were revised or newly issued to reduce the frequency of reliability deficiencies for defense systems in operational testing and the effects of those deficiencies. Reliability Growth evaluates these recent changes and, more generally, assesses how current DOD principles and practices could be modified to increase the likelihood that defense systems will satisfy their reliability requirements. This report examines changes to the reliability requirements for proposed systems; defines modern design and testing for reliability; discusses the contractor's role in reliability testing; and summarizes the current state of formal reliability growth modeling. The recommendations of Reliability Growth will improve the reliability of defense systems and protect the health of the valuable personnel who operate them.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!