National Academies Press: OpenBook
Suggested Citation:"Front Matter." National Research Council. 2012. Industrial Methods for the Effective Development and Testing of Defense Systems. Washington, DC: The National Academies Press. doi: 10.17226/13291.
×

Industrial Methods for the

EFFECTIVE DEVELOPMENT AND TESTING
OF DEFENSE SYSTEMS

Panel on Industrial Methods for the Effective Test
and Development of Defense Systems

Committee on National Statistics
Division of Behavioral and Social Sciences and Education

Board on Army Science and Technology
Division on Engineering and Physical Sciences

NATIONAL RESEARCH COUNCIL
OF THE NATIONAL ACADEMIES

THE NATIONAL ACADEMIES PRESS
Washington, D.C.
www.nap.edu

Suggested Citation:"Front Matter." National Research Council. 2012. Industrial Methods for the Effective Development and Testing of Defense Systems. Washington, DC: The National Academies Press. doi: 10.17226/13291.
×

THE NATIONAL ACADEMIES PRESS   500 Fifth Street, N.W.   Washington, DC 20001

NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance.

Support for the work of the Committee on National Statistics is provided by a consortium of federal agencies through a grant from the National Science Foundation (grant number SES-0453930). The project that is the subject of this report was supported by an allocation of the National Science Foundation by the U.S. Department of Defense under this grant. Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the view of the organizations or agencies that provided support for this project.

International Standard Book Number-13: 978-0-309-22270-9
International Standard Book Number-10: 0-309-22270-2

Additional copies of this report are available from the National Academies Press, 500 Fifth Street, N.W., Lockbox 285, Washington, DC 20055; (800) 624-6242 or (202) 334-3313 (in the Washington metropolitan area); Internet, http://www.nap.edu.

Copyright 2012 by the National Academy of Sciences. All rights reserved.

Printed in the United States of America

Suggested citation: National Research Council. (2012). Industrial Methods for the Effective Development and Testing of Defense Systems. Panel on Industrial Methods for the Effective Test and Development of Defense Systems. Committee on National Statistics, Division of Behavioral and Social Sciences and Education and Board on Army Science and Technology, Division on Engineering and Physical Sciences. Washington, DC: The National Academies Press.

Suggested Citation:"Front Matter." National Research Council. 2012. Industrial Methods for the Effective Development and Testing of Defense Systems. Washington, DC: The National Academies Press. doi: 10.17226/13291.
×

THE NATIONAL ACADEMIES
Advisers to the Nation on Science; Engineering; and Medicine

The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Ralph J. Cicerone is president of the National Academy of Sciences.

The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Charles M. Vest is president of the National Academy of Engineering.

The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Harvey V. Fineberg is president of the Institute of Medicine.

The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Ralph J. Cicerone and Dr. Charles M. Vest are chair and vice chair, respectively, of the National Research Council.

www.national-academies.org

Suggested Citation:"Front Matter." National Research Council. 2012. Industrial Methods for the Effective Development and Testing of Defense Systems. Washington, DC: The National Academies Press. doi: 10.17226/13291.
×

This page intentionally left blank.

Suggested Citation:"Front Matter." National Research Council. 2012. Industrial Methods for the Effective Development and Testing of Defense Systems. Washington, DC: The National Academies Press. doi: 10.17226/13291.
×

PANEL ON INDUSTRIAL METHODS FOR THE EFFECTIVE
TEST AND DEVELOPMENT OF DEFENSE SYSTEMS

VIJAY NAIR (Chair), Department of Statistics and Department of Industrial and Operations Engineering, University of Michigan

CHARLES E. (PETE) ADOLPH, Independent Consultant, Albuquerque, NM

W. PETER CHERRY, Science Applications International Corporation, Ann Arbor, MI (Retired)

JOHN D. CHRISTIE, Logistics Management Institute, Alexandria, VA

THOMAS P. CHRISTIE, Independent consultant, Arlington, VA

A. BLANTON GODFREY, College of Textiles, North Carolina State University

RAJ KAWLRA, Manufacturing Quality, Chrysler LLC, Auburn Hills, MI

JOHN E. ROLPH, Department of Industrial Operations and Management, Marshall School of Business, University of Southern California

ELAINE WEYUKER, AT&T Laboratories, Florham Park, NJ

MARION L. WILLIAMS, Institute for Defense Analyses, Alexandria, VA

ALYSON G. WILSON, Science and Technology Policy Institute, Institute for Defense Analyses, Washington, DC

MICHAEL L. COHEN, Study Director

MICHAEL J. SIRI, Program Associate

Suggested Citation:"Front Matter." National Research Council. 2012. Industrial Methods for the Effective Development and Testing of Defense Systems. Washington, DC: The National Academies Press. doi: 10.17226/13291.
×

COMMITTEE ON NATIONAL STATISTICS 2011-2012

LAWRENCE D. BROWN (Chair), Department of Statistics, The Wharton School, University of Pennsylvania

JOHN M. ABOWD, School of Industrial and Labor Relations, Cornell University

ALICIA CARRIQUIRY, Department of Statistics, Iowa State University

WILLIAM DuMOUCHEL, Oracle Health Sciences, Waltham, MA

V. JOSEPH HOTZ, Department of Economics, Duke University

MICHAEL HOUT, Survey Research Center, University of California, Berkeley

KAREN KAFADAR, Department of Statistics, Indiana University

SALLIE KELLER, Science and Technology Policy Institute, Institute for Defense Analyses, Washington, DC

LISA LYNCH, Heller School for Social Policy and Management, Brandeis University

SALLY C. MORTON, Department of Biostatistics, University of Pittsburgh

JOSEPH NEWHOUSE, Division of Health Policy Research and Education, Harvard University

RUTH D. PETERSON, Department of Sociology (emeritus), Ohio State University

HAL S. STERN, Donald Bren School of Computer and Information Sciences, University of California, Irvine

JOHN H. THOMPSON, National Opinion Research Center at the University of Chicago

ROGER TOURANGEAU, Joint Program in Survey Methodology, University of Maryland, and Survey Research Center, University of Michigan

ALAN ZASLAVSKY, Department of Health Care Policy, Harvard University Medical School

CONSTANCE F. CITRO, Director

Suggested Citation:"Front Matter." National Research Council. 2012. Industrial Methods for the Effective Development and Testing of Defense Systems. Washington, DC: The National Academies Press. doi: 10.17226/13291.
×

BOARD ON ARMY SCIENCE AND TECHNOLOGY

ALAN H. EPSTEIN (Chair), Pratt & Whitney, East Hartford, CT

DAVID M. MADDOX (Vice Chair), Independent consultant, Arlington, VA

DUANE ADAMS, Carnegie Mellon University (Retired)

ILESANMI ADESIDA, College of Engineering, University of Illinois at Urbana-Champaign

RAJ AGGARWAL, College of Engineering, University of Iowa

EDWARD C. BRADY, Strategic Perspectives, Inc., Fort Lauderdale, FL

L. REGINALD BROTHERS, BAE Systems, Arlington, VA

JAMES CARAFANO, The Heritage Foundation, Washington, DC

W. PETER CHERRY, Science Applications International Corporation, Ann Arbor, MI

EARL H. DOWELL, School of Engineering, Duke University

RONALD P. FUCHS, Independent consultant, Bellevue, Washington

W. HARVEY GRAY, Oak Ridge National Laboratory, Oak Ridge, TN

CARL GUERRERI, Electronic Warfare Associates, Inc., Herndon, VA

JOHN H. HAMMOND, Lockheed Martin Corporation, Fairfax, VA (Retired)

RANDALL W. HILL, JR., Institute for Creative Technologies, University of Southern California

MARY JANE IRWIN, Department of Computer Science and Engineering, Pennsylvania State University

ROBIN L. KEESEE, Independent consultant, Fairfax, VA

ELLIOTT D. KIEFF, Departments of Medicine and Microbiology, Harvard University

LARRY LEHOWICZ, Quantum Research International, Arlington, VA

WILLIAM L. MELVIN, Sensors, and Electromagnetic Applications Laboratory, Georgia Institute of Technology Research Institute

ROBIN MURPHY, Department of Computer Science and Engineering and Cognitive NeuroSciences and Psychology, University of South Florida

SCOTT PARAZYNSKI, The Methodist Hospital Research Institute, Houston, TX

RICHARD R. PAUL, Independent consultant, Bellevue, WA

JEAN D. REED, Independent consultant, Arlington, VA

LEON E. SALOMON, Independent consultant, Gulfport, FL

JONATHAN M. SMITH, School of Engineering and Applied Sciences, University of Pennsylvania

MARK J.T. SMITH, School of Electrical and Computer Engineering, Purdue University

MICHAEL A. STROSCIO, Nanoengineering Research Laboratory, University of Illinois, Chicago

JOSEPH YAKOVAC, JVM, LLC, Hampton, VA

BRUCE A. BRAUN, Director

Page viii Cite
Suggested Citation:"Front Matter." National Research Council. 2012. Industrial Methods for the Effective Development and Testing of Defense Systems. Washington, DC: The National Academies Press. doi: 10.17226/13291.
×

This page intentionally left blank.

Suggested Citation:"Front Matter." National Research Council. 2012. Industrial Methods for the Effective Development and Testing of Defense Systems. Washington, DC: The National Academies Press. doi: 10.17226/13291.
×

Acknowledgments

The Panel on Industrial Methods for the Effective Test and Development of Defense Systems expresses its appreciation to the many individuals who provided valuable assistance in producing this report. We appreciate the support of Michael Gilmore, Director of Operational Test and Evaluation (DOT&E), and Frank Kendall, Principal Deputy Under Secretary of Defense (Acquisition, Technology, and Logistics) at the U.S. Department of Defense (DOD). We are also greatly indebted to Nancy Spruill, director, Acquisition Resources and Analysis, and Ernest Seglie, recently retired science advisor to the director of operational test and evaluation at DOD.

The success of the study depended greatly on the presentations at the workshop, which was the panel’s main fact-finding activity. The panel is extremely grateful to the major speakers who represented industry perspectives: Donald Bollinger, Hewlett-Packard; Salim Momin, SRS Enterprises; Sham Vaidya, IBM; and Jeffrey Zyburt, DCYI Engineering Consulting & Development Process. The other presentations by Michael Cushing, Army Evaluation Center, and Robin Pope, SAIC (Science Applications International Corporation), were also very helpful in preparing this report. We also thank Steve Hutchinson, Defense Information System Agency, DOD, and Dmitry Tananko, General Dynamics, who served as discussants. The workshop also included a very productive panel session with reactions of the defense test community to the presentations by the experts from industry, and we thank them: William McCarthy, DOT&E; Steve Welby, director, Systems Engineering; and Chris DiPetto, acting director, Development Test.

Suggested Citation:"Front Matter." National Research Council. 2012. Industrial Methods for the Effective Development and Testing of Defense Systems. Washington, DC: The National Academies Press. doi: 10.17226/13291.
×

We also thank the staff of the Committee on National Statistics, especially Michael Siri and Anthony Mann, for their smooth organization of our meetings, and Julie Schuck for her work on the report draft and for helping plan and support the panel’s meetings. We would also like to express our gratitude to Eugenia Grohman for the technical editing of the report.

Most importantly, we were fortunate to have an outstanding group of colleagues on the panel. They provided critical insights and expertise on industrial processes and systems engineering as well as defense acquisition and testing. They volunteered their time and service generously before, during, and after the panel meetings and were involved extensively in the writing of the report.

This report has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise, in accordance with procedures approved by the Report Review Committee of the National Research Council (NRC). The purpose of this independent review is to provide candid and critical comments that will assist the institution in making its published report as sound as possible and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process. We thank the following individuals for their review of this report: Donald Bollinger, distinguished technologist, Hewlett-Packard; Gilbert F. Decker, consultant, Los Gatos, CA; Arthur Fries, staff member and project leader, Institute for Defense Analyses; Charles E. McQueary, consultant, former under secretary for science and technology, U.S. Department of Homeland Security and former director of operational test and evaluation; Department of Defense, Greensboro, NC; William Meeker, Department of Statistics, Iowa State University; Duane Steffey, director, Statistical and Data Sciences, Exponent®, Menlo Park, CA; Dmitry Tananko, manager, Reliability, General Dynamics Land Systems; and Jeff Zyburt, president, DCYI Consulting, New Hudson, MI.

Although the reviewers listed above have provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations nor did they see the final draft of the report before its release. The review of this report was overseen by Thom J. Hodgson, distinguished university professor, Fitts Industrial and Systems Engineering Department, North Carolina State University Appointed by the NRC’s Report Review Committee, he was responsible for making certain that an independent examination of this report was carried out in accordance with institutional procedures and that all review comments were carefully considered. Responsibility for the final content of this report rests entirely with the authoring panel and the institution.

Suggested Citation:"Front Matter." National Research Council. 2012. Industrial Methods for the Effective Development and Testing of Defense Systems. Washington, DC: The National Academies Press. doi: 10.17226/13291.
×

The panel recognizes the many federal agencies that support the Committee on National Statistics directly and through a grant from the National Science Foundation. Without their support and their commitment to improving the national statistical system, the work that is the basis of this report would not have been possible.

Vijay Nair, Chair
Michael L. Cohen, Study Director
Panel on Industrial Methods for the Effective
Test and Development of Defense Systems

Suggested Citation:"Front Matter." National Research Council. 2012. Industrial Methods for the Effective Development and Testing of Defense Systems. Washington, DC: The National Academies Press. doi: 10.17226/13291.
×

This page intentionally left blank.

Suggested Citation:"Front Matter." National Research Council. 2012. Industrial Methods for the Effective Development and Testing of Defense Systems. Washington, DC: The National Academies Press. doi: 10.17226/13291.
×

Glossary and Acronyms

ACAT: Acquisition category, a designation for each defense program based on program costs that determines both the level of review that is required by law and the level at which Milestone decision authority rests in DOD.

ACAT I: Of four acquisition categories (ACAT I to ACAT IV), the most expensive systems, which are estimated to require either more than $365 million (fiscal 2000) for research and development or more than $2.19 billion (fiscal 2000) for purchase of the specified number of delivered systems.

Defense Acquisition Board (DAB): A senior advisory board for defense acquisitions in DOD that includes the vice chairman of the Joint Chiefs of Staff and the service secretaries, among others, and that plays a key role since it is responsible for approving major defense acquisition programs.

Developmental test (and evaluation): Typical testing of a defense system early in development, analogous to laboratory or bench testing, sometimes involving only components or subsystems, that often does not represent full operational realism, in contrast with Operational test (and evaluation).

Director, Cost Assessment and Program Evaluation (CAPE): The principal staff assistant to the secretary of defense for cost assessment and program evaluation, whose responsibilities include analysis and evalua-

Suggested Citation:"Front Matter." National Research Council. 2012. Industrial Methods for the Effective Development and Testing of Defense Systems. Washington, DC: The National Academies Press. doi: 10.17226/13291.
×

tion of plans, programs, and budgets in relation to U.S. defense objectives, projected threats, allied contributions, estimated costs, and resource constraints and ensuring that the costs of DOD programs, including classified programs, are presented accurately and completely.

Director, Defense Research and Engineering (DDR&E): The principal staff adviser to USD-AT&L for matters of research and engineering.

Director, Operational Test and Evaluation: The office or the person who heads DOT&E.

DOT&E: Office of the Director of Operational Test and Evaluation (or, sometimes, the person who holds the office), a unit in the Office of the Secretary of Defense, which also reports directly to Congress, responsible for DOD policies and procedures for analyzing and interpreting the results of operational testing and evaluation for each major DOD acquisition program, approving test plans, and providing independent evaluations of ACATI systems.

Effectiveness and suitability1: A measure of the overall ability of a system to accomplish a mission when used by representative personnel in the environment planned or expected for operational employment of the system considering organization, doctrine, tactics, supportability, vulnerability, and threat. Effectiveness is the degree to which a system can carry out its mission when fully operational. (Operational) suitability is the degree to which a system can be placed and sustained satisfactorily in field use.

Evolutionary acquisition: The development of a defense system in stages, with the systems that result from each stage of development potentially released to the field.

5000.01: DOD directive that provides management principles and mandatory policies and procedures for managing all acquisition programs.

5000.02: DOD instruction that establishes a simplified and flexible management framework for translating capability needs and technology opportunities.

image

1Definition adapted from Joint Capabilities Integration and Development System, CJCSI 3170.01G. See http://jitc.fhu.disa.mil/jitc_dri/pdfs/3170_01g.pdf [December 2011].

Page xvii Cite
Suggested Citation:"Front Matter." National Research Council. 2012. Industrial Methods for the Effective Development and Testing of Defense Systems. Washington, DC: The National Academies Press. doi: 10.17226/13291.
×

Full-rate production: The final step of procurement, in contrast to release to the field of a small number of units as part of low-rate initial production, which requires either the judgment that it is effective and suitable by DOT&E or by a full-rate production decision review.

HP-UX: Hewlett-Packard’s implementation of the UNIX operating system.

Initial Operational Test and Evaluation (IOT&E): The first large operational test of a system or system element [see Operational test (and evaluation)].

Joint Capabilities Integrated Development System (JCIDS): A formal DOD procedure that defines requirements and evaluation criteria for defense systems in development.

Materiel developer: The organization or command responsible for providing materiel to DOD or specific service forces, with responsibilities that include research and development of weapon systems.

Milestone A: The step in the Milestone system that promotes a system to the technology development phase of development.

Milestone B: The step in the Milestone system that promotes a system to the engineering and manufacturing development phase of development.

Milestone decision authority: The person or office responsible for the decision to promote a system to the next step of development in the Milestone system.

Milestone system: A set of three milestones that bridge the four steps of defense acquisition: (1) materiel solution analysis, (2) technology development, (3) engineering and manufacturing development, and (4) production and deployment.

Model-based engineering: Systems engineering, starting from development of requirements, through development of components and subsystems, then integration into full systems, that is guided throughout by the use of models that simulate overall system performance of systems comprised of various kinds of subsystems and components, which enforces collaboration across multiple engineering departments.

Page xviii Cite
Suggested Citation:"Front Matter." National Research Council. 2012. Industrial Methods for the Effective Development and Testing of Defense Systems. Washington, DC: The National Academies Press. doi: 10.17226/13291.
×

Modeling and simulation: Various methods for simulating, sometimes with system components in the loop and sometimes entirely computer based, the functioning of a (proposed) defense system.

Operational test (and evaluation): Testing of a defense system relatively late in development, involving the full system in whatever numbers will be used cooperatively in the field, in scenarios that attempt to represent full operational realism, including representation of enemy systems, countermeasures, and operated by users with training typical of fielded systems.

Program Management Office (PMO): The office tasked with development, production, and sustainment of a defense system on a timely basis that satisfies a set of requirements at a given price.

Program manager2: The person with responsibility for and authority to accomplish program objectives for development, production, and sustainment to meet the user’s operational needs and accountable for credible cost, schedule, and performance reporting to the Milestone decision authority.

Reliability, Availability, and Maintainability (RAM)3: The probability of an item to perform a required function under stated conditions for a specified period of time (reliability), degree to which it is in an operable state and can be committed at the start of a mission when the mission is called for at an unknown (random) point in time (availability), and its ability to be retained in, or restored to, a specified condition when maintenance is performed by personnel having specified skill levels, using prescribed procedures and resources, at each prescribed level of maintenance and repair (maintainability).

Technology readiness level: The degree to which the behavior of a newly developed technology is understood well enough for incorporation into a system in Full-rate production.

image

2Definition adapted from the U.S. Department of Defense Directive 5000.01. See http://www.dtic.mil/whs/directives/corres/pdf/500001p.pdf [December 2011].

3Definition adapted from the U.S. Department of Defense Guide for Achieving Reliability, Availability, and Maintainability. See http://www.acq.osd.mil/dte/docs/RAM_Guide_080305.pdf [December 2011].

Suggested Citation:"Front Matter." National Research Council. 2012. Industrial Methods for the Effective Development and Testing of Defense Systems. Washington, DC: The National Academies Press. doi: 10.17226/13291.
×

Test and Evaluation Master Plan (TEMP): A formal document that provides a scheme to be used to create detailed test and evaluation plans, especially schedule and resource commitments.

U.S. Army Training and Doctrine Command (TRADOC): An Army element that provides training to soldiers and, as part of that training, helps design, develop, and integrate new capabilities and doctrine.

USD-AT&L: The Under Secretary of Defense for Acquisition, Technology, and Logistics, the primary office in the Office of the Secretary of Defense responsible for the development and acquisition of defense systems.

Suggested Citation:"Front Matter." National Research Council. 2012. Industrial Methods for the Effective Development and Testing of Defense Systems. Washington, DC: The National Academies Press. doi: 10.17226/13291.
×

This page intentionally left blank.

Suggested Citation:"Front Matter." National Research Council. 2012. Industrial Methods for the Effective Development and Testing of Defense Systems. Washington, DC: The National Academies Press. doi: 10.17226/13291.
×
Page R1
Suggested Citation:"Front Matter." National Research Council. 2012. Industrial Methods for the Effective Development and Testing of Defense Systems. Washington, DC: The National Academies Press. doi: 10.17226/13291.
×
Page R2
Suggested Citation:"Front Matter." National Research Council. 2012. Industrial Methods for the Effective Development and Testing of Defense Systems. Washington, DC: The National Academies Press. doi: 10.17226/13291.
×
Page R3
Suggested Citation:"Front Matter." National Research Council. 2012. Industrial Methods for the Effective Development and Testing of Defense Systems. Washington, DC: The National Academies Press. doi: 10.17226/13291.
×
Page R4
Suggested Citation:"Front Matter." National Research Council. 2012. Industrial Methods for the Effective Development and Testing of Defense Systems. Washington, DC: The National Academies Press. doi: 10.17226/13291.
×
Page R5
Suggested Citation:"Front Matter." National Research Council. 2012. Industrial Methods for the Effective Development and Testing of Defense Systems. Washington, DC: The National Academies Press. doi: 10.17226/13291.
×
Page R6
Suggested Citation:"Front Matter." National Research Council. 2012. Industrial Methods for the Effective Development and Testing of Defense Systems. Washington, DC: The National Academies Press. doi: 10.17226/13291.
×
Page R7
Page viii Cite
Suggested Citation:"Front Matter." National Research Council. 2012. Industrial Methods for the Effective Development and Testing of Defense Systems. Washington, DC: The National Academies Press. doi: 10.17226/13291.
×
Page R8
Suggested Citation:"Front Matter." National Research Council. 2012. Industrial Methods for the Effective Development and Testing of Defense Systems. Washington, DC: The National Academies Press. doi: 10.17226/13291.
×
Page R9
Suggested Citation:"Front Matter." National Research Council. 2012. Industrial Methods for the Effective Development and Testing of Defense Systems. Washington, DC: The National Academies Press. doi: 10.17226/13291.
×
Page R10
Suggested Citation:"Front Matter." National Research Council. 2012. Industrial Methods for the Effective Development and Testing of Defense Systems. Washington, DC: The National Academies Press. doi: 10.17226/13291.
×
Page R11
Suggested Citation:"Front Matter." National Research Council. 2012. Industrial Methods for the Effective Development and Testing of Defense Systems. Washington, DC: The National Academies Press. doi: 10.17226/13291.
×
Page R12
Page xiii Cite
Suggested Citation:"Front Matter." National Research Council. 2012. Industrial Methods for the Effective Development and Testing of Defense Systems. Washington, DC: The National Academies Press. doi: 10.17226/13291.
×
Page R13
Suggested Citation:"Front Matter." National Research Council. 2012. Industrial Methods for the Effective Development and Testing of Defense Systems. Washington, DC: The National Academies Press. doi: 10.17226/13291.
×
Page R14
Suggested Citation:"Front Matter." National Research Council. 2012. Industrial Methods for the Effective Development and Testing of Defense Systems. Washington, DC: The National Academies Press. doi: 10.17226/13291.
×
Page R15
Suggested Citation:"Front Matter." National Research Council. 2012. Industrial Methods for the Effective Development and Testing of Defense Systems. Washington, DC: The National Academies Press. doi: 10.17226/13291.
×
Page R16
Page xvii Cite
Suggested Citation:"Front Matter." National Research Council. 2012. Industrial Methods for the Effective Development and Testing of Defense Systems. Washington, DC: The National Academies Press. doi: 10.17226/13291.
×
Page R17
Page xviii Cite
Suggested Citation:"Front Matter." National Research Council. 2012. Industrial Methods for the Effective Development and Testing of Defense Systems. Washington, DC: The National Academies Press. doi: 10.17226/13291.
×
Page R18
Suggested Citation:"Front Matter." National Research Council. 2012. Industrial Methods for the Effective Development and Testing of Defense Systems. Washington, DC: The National Academies Press. doi: 10.17226/13291.
×
Page R19
Suggested Citation:"Front Matter." National Research Council. 2012. Industrial Methods for the Effective Development and Testing of Defense Systems. Washington, DC: The National Academies Press. doi: 10.17226/13291.
×
Page R20
Next: Summary »
Industrial Methods for the Effective Development and Testing of Defense Systems Get This Book
×
Buy Paperback | $35.00 Buy Ebook | $27.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

During the past decade and a half, the National Research Council, through its Committee on National Statistics, has carried out a number of studies on the application of statistical methods to improve the testing and development of defense systems. These studies were intended to provide advice to the Department of Defense (DOD), which sponsored these studies. The previous studies have been concerned with the role of statistical methods in testing and evaluation, reliability practices, software methods, combining information, and evolutionary acquisition.

Industrial Methods for the Effective Testing and Development of Defense Systems is the latest in a series of studies, and unlike earlier studies, this report identifies current engineering practices that have proved successful in industrial applications for system development and testing. This report explores how developmental and operational testing, modeling and simulation, and related techniques can improve the development and performance of defense systems, particularly techniques that have been shown to be effective in industrial applications and are likely to be useful in defense system development. In addition to the broad issues, the report identifies three specific topics for its focus: finding failure modes earlier, technology maturity, and use of all relevant information for operational assessments.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!