National Academies Press: OpenBook
« Previous: Appendix C A Potential Taxonomy for Operational Tests
Suggested Citation:"Appendix D Elements of ISO 9000." National Research Council. 1998. Statistics, Testing, and Defense Acquisition: New Approaches and Methodological Improvements. Washington, DC: The National Academies Press. doi: 10.17226/6037.
×

APPENDIX D
Elements ISO 9000

The International Organization for Standardization (ISO) is a federation of organizations that represent 92 member countries established in 1946 to "promote the development of international standards and related activities to facilitate the exchange of goods and services" (Breitenberg, 1993). The United States representative to the ISO is the American National Standards Institute (ANSI).

In 1987 ISO issued the ISO 9000 Standard Series, a set of five individual but related standards (9000 through 9004) that provide guidance on requirements for quality systems; see Box D-1. The most comprehensive set of standards are ISO 9001; its various elements or specific requirements are listed in Box D-2. Significant changes were made to the ISO 9000 standards in 1994. ISO 9004-1 is now the overall guidance standard in the basic set of quality management standards; however, it also retains its original role of a separate standard for organizations to use internally to improve their quality management practices. Further revisions are anticipated in 1998 to move the ISO 9000 family toward "total quality management." The ISO's Technical Committee 176 has already started developing the revisions, based on quality management principles and extending beyond the product realization process.

Initially designed to be used in contractual arrangements between two parties or for internal auditing, use of ISO 9000 standards has greatly expanded over the last 10 years. More than 96 countries have adopted the ISO quality standards, and most industries throughout the world, including many in the United States. recognize them. Today, many government agencies and private industries worldwide require the organizations with which they do business to comply with one of the standards in the series. The standards have been applied to manufacturing,

Suggested Citation:"Appendix D Elements of ISO 9000." National Research Council. 1998. Statistics, Testing, and Defense Acquisition: New Approaches and Methodological Improvements. Washington, DC: The National Academies Press. doi: 10.17226/6037.
×

service industries, software development organizations, and the process industries. Currently, the standards are most often applied to business processes that directly affect the products and service provided by an organization. In the future, the scope of application will be expanded to all areas of enterprises, including finance, accounting, human resources, market research, and marketing.

BASIC PRINCIPLES

The ISO 9000 standards are based on three basic principles: processes affecting the quality of products and services must be documented; records of important decisions and related data must be archived; and when these first two steps are complied with, the product or service will enjoy continuing improvement. We now discuss each of these principles and discuss how adherences results in product and service improvement.

Documenting Processes Documentation often helps to discover important ad hoc processes that are inconsistently applied. This discovery then leads to changes in the process that results in their more consistent application, which, in turn, reduces the variability in the output of these processes.

Documenting processes also provides a better understanding of how an orga

Suggested Citation:"Appendix D Elements of ISO 9000." National Research Council. 1998. Statistics, Testing, and Defense Acquisition: New Approaches and Methodological Improvements. Washington, DC: The National Academies Press. doi: 10.17226/6037.
×
Suggested Citation:"Appendix D Elements of ISO 9000." National Research Council. 1998. Statistics, Testing, and Defense Acquisition: New Approaches and Methodological Improvements. Washington, DC: The National Academies Press. doi: 10.17226/6037.
×
Suggested Citation:"Appendix D Elements of ISO 9000." National Research Council. 1998. Statistics, Testing, and Defense Acquisition: New Approaches and Methodological Improvements. Washington, DC: The National Academies Press. doi: 10.17226/6037.
×

nization does business. First, customer requirements are better understood. In addition, since the process of designing and producing products and services is well understood, the customer needs can be more directly addressed. Finally, there is a clear process for exploring the sources of and resolving customer complaints.

Retaining Records and Data Supporting Decisions Decisions, only made by those with the authority to make them, that affect the quality of a product or service, need to be documented, along with all supporting data and records. These decisions need to be periodically reviewed for their basis and authorization. Data obtained during the life of the product or service must be archived for a specified period of time if the data provide information about the product or service quality. These data are then used for product or service improvement, or for development of new products or services.

Continuing Improvement The three elements of continuing product improvement are: audit, corrective and preventive action, and management review.

Internal auditing provides important information as to the potential sources of ineffectiveness in a business process. In conjunction with surveillance by ISO 9000 registrars, internal auditing and external surveillance produce corrective action requests that lead to an understanding of the cause of the ineffectiveness and its resolution. The final step of management review assures that continuing product or service improvement is in place and the integration of these improvements into the overall goals of the business.

BENEFITS OF APPLICATION TO DoD

DoD might draw constructively on industrial practices, particularly in such areas as documentation, uniform standards, and the pooling of information on operational suitability, following the direction of the ISO 9000 series.1 Documentation of processes and retention of RAM-related records (for important decisions and valuable data) are practices now greatly emphasized in industry. The same should be true for DoD, especially for the purposes of assessing operational suitability in support of major production decisions.

DoD should document, consistently across the services, all relevant sources

1  

Interestingly, many aspects of the ISO 9000 standards were originally developed from DoD's Quality Management Program. MIL-Q-9858. It was revised in 1963. and in 1968 NATO adopted its provisions as the Allied Quality Assurance Publication (AQAP-1). In 1970 the Ministry of Defence of the United Kingdom adopted AQAP-1. and in 1979 the British Standards Institution developed BS 5750. the first commercial quality management system. The ISO 9000 standard series was created from these predecessors.

Suggested Citation:"Appendix D Elements of ISO 9000." National Research Council. 1998. Statistics, Testing, and Defense Acquisition: New Approaches and Methodological Improvements. Washington, DC: The National Academies Press. doi: 10.17226/6037.
×

of test data and information. Development teams would not have to ''reinvent the wheel" when beginning a project. Instead, they would identify and learn from previous studies and findings. Setting multiservice operational test and evaluation standards similar to those in ISO 9000 is critical to the creation of an environment of more systematic data collection, analysis, and documentation.

The panel recommends (see Chapter 3) that DoD and the services develop a centralized test and evaluation data archive and standardize test data archival practices based on ISO 9000. This archive should include both developmental and operational test data; use of uniform terminology in data collection across services; and careful documentation of development and test plans, development and test budgets, test evaluation processes, and the justification for all test-related decisions, including decisions concerning resource allocation.

Standardization, documentation, and data archiving facilitate use of all available information for efficient decision making. Routinely taking these steps will:

  • provide the information needed to validate models and simulations, which in turn can be used to plan for (or reduce the amount of) experimentation needed to reach specified operational test and evaluation goals;

  • allow the "borrowing" of information from past studies (if they are clearly documented and there is consistent usage of terminology and data) to inform the assessment of a system's performance based upon limited testing, by means of formal and informal statistical methods and other approaches;

  • make possible the use of data from developmental testing for efficient operational test design;

  • allow learning from best current practices across the services; and

  • lead to an organized accumulation of knowledge within the Department of Defense.

A key benefit of documentation and archival of test planning, test evaluation, and in-use system performance is the creation of feedback loops to identify system flaws for system redesign, and to identify where tests or models have missed important deficiencies in system performance.

Retention of records may involve additional costs, but it is clearly necessary for accountability in the decision-making process. The trend in industry is to empower employees by giving them more responsibility in the decision-making process; along with this responsibility comes the need to make people accountable for their decisions. This consideration is likely to be an important organizational aspect of the operational testing of defense systems.

Suggested Citation:"Appendix D Elements of ISO 9000." National Research Council. 1998. Statistics, Testing, and Defense Acquisition: New Approaches and Methodological Improvements. Washington, DC: The National Academies Press. doi: 10.17226/6037.
×
Page 204
Suggested Citation:"Appendix D Elements of ISO 9000." National Research Council. 1998. Statistics, Testing, and Defense Acquisition: New Approaches and Methodological Improvements. Washington, DC: The National Academies Press. doi: 10.17226/6037.
×
Page 205
Suggested Citation:"Appendix D Elements of ISO 9000." National Research Council. 1998. Statistics, Testing, and Defense Acquisition: New Approaches and Methodological Improvements. Washington, DC: The National Academies Press. doi: 10.17226/6037.
×
Page 206
Suggested Citation:"Appendix D Elements of ISO 9000." National Research Council. 1998. Statistics, Testing, and Defense Acquisition: New Approaches and Methodological Improvements. Washington, DC: The National Academies Press. doi: 10.17226/6037.
×
Page 207
Suggested Citation:"Appendix D Elements of ISO 9000." National Research Council. 1998. Statistics, Testing, and Defense Acquisition: New Approaches and Methodological Improvements. Washington, DC: The National Academies Press. doi: 10.17226/6037.
×
Page 208
Suggested Citation:"Appendix D Elements of ISO 9000." National Research Council. 1998. Statistics, Testing, and Defense Acquisition: New Approaches and Methodological Improvements. Washington, DC: The National Academies Press. doi: 10.17226/6037.
×
Page 209
Next: Appendix E Glossary and Acronyms »
Statistics, Testing, and Defense Acquisition: New Approaches and Methodological Improvements Get This Book
×
Buy Paperback | $57.00 Buy Ebook | $45.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

For every weapons system being developed, the U.S. Department of Defense (DOD) must make a critical decision: Should the system go forward to full-scale production? The answer to that question may involve not only tens of billions of dollars but also the nation's security and military capabilities. In the milestone process used by DOD to answer the basic acquisition question, one component near the end of the process is operational testing, to determine if a system meets the requirements for effectiveness and suitability in realistic battlefield settings. Problems discovered at this stage can cause significant production delays and can necessitate costly system redesign.

This book examines the milestone process, as well as the DOD's entire approach to testing and evaluating defense systems. It brings to the topic of defense acquisition the application of scientific statistical principles and practices.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!