NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the panel responsible for the report were chosen for their special competencies and with regard for appropriate balance.
This report has been reviewed by a group other than the authors according to procedures approved by a Report Review Committee consisting of members of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine.
The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Frank Press is president of the National Academy of Sciences.
The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Robert M. White is president of the National Academy of Engineering.
The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Kenneth I. Shine is president of the Institute of Medicine.
The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy's purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Frank Press and Dr. Robert M. White are chairman and vice-chairman, respectively, of the National Research Council.
This study was supported by Contract NASW-4003 between the National Academy of Sciences and the National Aeronautics and Space Administration.
Library of Congress Catalog Card Number 93-84549
International Standard Book Number 0-309-04880-X
Available in limited supply from:
The Aeronautics and Space Engineering Board
2101 Constitution Avenue, N.W.
Washington, D.C. 20418
Additional copies available for sale from:
National Academy Press
2101 Constitution Avenue, N.W., Box 285 Washington, D.C. 20055 1-800-624-6242 or (202) 334-3313
Copyright 1993 by the National Academy of Sciences. All rights reserved.
Printed in the United States of America
First Printing, June 1993
Second Printing, November 1993
COMMITTEE FOR REVIEW OF OVERSIGHT MECHANISMS FOR SPACE SHUTTLE FLIGHT SOFTWARE PROCESSES
Nancy G. Leveson, Chair,
Boeing Professor of Computer Science and Engineering, University of Washington
Robert N. Charette, Chairman,
ITABHI Corporation, Fairfax, Virginia
B. A. Claussen, Executive Vice President,
CTA INCORPORATED, Denver, Colorado
Carl S. Droste, Manager,
Flight Control Systems, Lockheed Fort Worth Company, Fort Worth, Texas
Roger U. Fujii, Operations Manager,
Systems Technology Operation, Logicon, San Pedro, California
John D. Gannon, Professor of Computer Science,
The University of Maryland, College Park, Maryland
Richard A. Kemmerer, Professor of Computer Science,
The University of California, Santa Barbara, California
Robert O. Polvado, Senior Scientist,
Office of Research and Development, Central Intelligence Agency, Arlington, Virginia
Willis H. Ware, Senior Member,
Corporate Research Staff, The RAND Corporation, Santa Monica, California
Wallace H. Whittier, Program Engineering Manager,
Lockheed Missiles and Space Company, Sunnyvale, California
Martin J. Kaszubowski, Study Director
JoAnn C. Clayton, Director,
Aeronautics and Space Engineering Board
Christina A. Weinland, Senior Project Assistant
Maria M. Kneas, Project Assistant
AERONAUTICS AND SPACE ENGINEERING BOARD
Duane T. McRuer, Chairman, President and Technical Director,
Systems Technology, Inc., Hawthorne, California
Steven Aftergood, Senior Research Analyst,
Federation of American Scientists, Washington, D.C.
James M. Beggs, Senior Partner,
J.M. Beggs Associates, Arlington, Virginia
John K. Buckner, Vice President,
Special Programs, Lockheed Fort Worth Company, Fort Worth, Texas
Ruth M. Davis, President and Chief Executive Officer,
Pymatuning Group, Inc., Alexandria, Virginia
Wolfgang H. Demisch, Managing Director,
UBS Securities, New York, New York
Owen K. Garriott, Vice President,
Space Programs, Teledyne Brown Engineering, Huntsville, Alabama
John M. Hedgepeth, President,
Digisim Corporation, Santa Barbara, California
Takeo Kanade, Professor of Computer Science,
Robotics and Electrical Engineering, Carnegie Mellon University, Pittsburgh, Pennsylvania
Jack L. Kerrebrock, R.C. Maclaurin Professor of Aeronautics and Astronautics,
Massachusetts Institute of Technology, Cambridge, Massachusetts
Bernard L. Koff, Executive Vice President,
Engineering and Technology, Pratt & Whitney, West Palm Beach, Florida
Robert G. Loewy, Institute Professor,
Aeronautical Engineering and Mechanics, Rensselaer Polytechnic Institute, Troy, New York
John M. Logsdon, Director,
Center for International Science and Technology Policy, Space Policy Institute, George Washington University, Washington, D.C.
Robert R. Lynn,
Bell Helicopter Textron, Euless, Texas
Frank E. Marble,
Richard L. Hayman and
Dorothy M. Hayman
Professor of Mechanical Engineering and
Professor of Jet Propulsion, Emeritus,
California Institute of Technology, Pasadena, California
Garner W. Miller, Retired Senior Vice President for Technology,
USAir, Naples, Florida
Harvey O. Nay, Retired Vice President of Engineering,
Piper Aircraft Corporation, Marysville, Washington
Frank E. Pickering, Vice President and Chief Engineer,
Aircraft Engines, General Electric Company, Lynn, Massachusetts
Anatol Roshko, Theodore von Karman Professor of Aeronautics,
California Institute of Technology, Pasadena, California
Alfred Schock, Director,
Energy System Department, Fairchild Industries, Germantown, Maryland
Thomas P. Stafford, Vice Chairman,
Stafford, Burke, and Hecker, Inc., Alexandria, Virginia
Martin N. Titland, Chief Operating Officer,
CTA INCORPORATED, Rockville, Maryland
John D. Warner, Vice President, Computing,
The Boeing Company, Seattle, Washington
JoAnn C. Clayton, Director
Martin J. Kaszubowski, Senior Program Officer
Allison C. Sandlin, Senior Program Officer
Noel E. Eldridge, Program Officer
Paul J. Shawcross, Program Officer
Anna L. Farrar, Administrative Associate
Christina A. Weinland, Administrative Assistant
Susan K. Coppinger, Senior Secretary
Maria M. Kneas, Senior Secretary
Maryann Shanesy, Senior Secretary
The National Aeronautics and Space Administration (NASA) not only leads the world in space exploration and space science, but, dating back to the early space flights in the 1960s, it has led the world in the use of computers to control complex systems. While others were struggling to automate relatively simple business applications, NASA was stretching the technological envelope to build real-time computer systems to control complicated spacecraft and their support systems in programs such as Gemini, Apollo, and the Space Shuttle.
Just as the Shuttle stretched the limits of the technology of its time, current projects such as Space Station Freedom and the Earth Observing System stretch the limits of technology today. In order to successfully build these future space systems, NASA needs not only to be at the technological forefront but to go beyond the state of the art and lead the world in software engineering.
After the Challenger accident, the Rogers Commission Report made many recommendations for change at NASA and suggested that, after a reasonable time, a National Research Council (NRC) Committee be formed to evaluate the progress that had been made toward implementation of those recommendations. This latter committee was formed in 1988 and recommended that NASA adopt Independent Verification and Validation (IV&V) of the Shuttle software. The NRC's recommendation was later echoed by other reports and NASA ultimately instituted a fairly robust IV&V effort. Over time, that effort was reduced due to resource constraints and because of the belief that the maturity of the software reduced the need for such a robust oversight activity. Our committee was formed at the beginning of 1992, at the request of NASA, to reevaluate the need for IV&V and to investigate other aspects of NASA's software development and oversight processes.
It is, of course, easy to be critical; we want to stress that we found the software and software development procedures for the Space Shuttle to be, in the main, excellent. However, the requirements of space science, applications, and exploration demand that the software be as good as possible. This report describes some ways in which we feel NASA can improve its software oversight activities to continue the successful operation of the Space Shuttle for as long it continues to be a part of the nation's space launch infrastructure.
Our committee met over a period of 12 months, conducting interviews, listening to presentations, submitting questions for NASA and its contractors to answer, and reading copious amounts of material. I would personally like to thank the members of the Committee for their hard work.
I would also like to thank the NASA and contractor personnel who did their best to provide us with the information we needed for the investigation (see Appendix A). Finally, we could never have completed this project without the hard work and dedication of the staff of the Aeronautics and Space Engineering Board (ASEB). I would especially like to thank the Director of the ASEB, JoAnn Clayton; the senior project assistant, Christina Weinland; the project assistant, Maria Kneas; and the study director, Marty Kaszubowski, whose technical expertise, hard work, organizational skills, and sense of humor are responsible for the success of this study.
Dr. Nancy G. Leveson
Chair, Committee for Review of Oversight Mechanisms for Space Shuttle Flight Software Processes
Acronyms and Abbreviations
Backup Flight Software — The software, developed by Rockwell/Downey, that monitors the progress of the primary software and intervenes in the case of a severe error that disables the primary system.
Code Q — Another name for the headquarters Safety and Mission Quality (S&MQ) Office. Each NASA headquarters office is given a code designation along with its formal name (e.g., the Development Office is Code D, the Space Station Office is Code S). In this case Code Q is the designator that corresponds to the S&MQ Office.
Change Request — An official request by a member of the Shuttle flight software community to change the software to add to, or simplify, its functionality.
Discrepancy Report — An official request by a member of the Shuttle flight software community to change the software because an error has been identified.
General Purpose Computers — The set of five independent computers used to run the primary and backup software.
Independent Verification and Validation
Johnson Space Center — The NASA center at which the bulk of the software development and assurance activity takes place.
Marshall Space Flight Center — The Marshall Space Flight Center is responsible for developing and assuring the software that controls the Space Shuttle Main Engines.
National Aeronautics and Space Administration
Operational Increment — A planned update to the flight software. Updates occur approximately every year and each OI requires approximately 28 months to completely develop and test.
Primary Avionics Software System — The primary on-board software developed by IBM.
Shuttle Avionics Software Control Board — The NASA body that is ultimately responsible for the safety and effectiveness of the flight software.
Safety and Mission Quality — The headquarters office that is responsible for NASA wide safety and quality activities.
Safety, Reliability, and Quality Assurance — The safety offices at the Johnson Space Center and the Marshall Space Flight Center.
Space Shuttle Main Engine Controller — The software system used to control the actions of the Space Shuttle main engines. The SSMEC is developed by Rocketdyne for the Marshal Space Flight Center.
Verification and Validation