National Academies Press: OpenBook

An Assessment of Space Shuttle Flight Software Development Processes (1993)

Chapter: Appendix E: Flight Software Verification and Validation Requirements

« Previous: Appendix D: Overview of ASET IV & V Methodology
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×

APPENDIX E

Flight Software
Verification and Validation Requirements

NSTS-08271

November 21, 1991

Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
This page in the original is blank.
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×

National Aeronautics and Space Administration

NSTS 08271

Lyndon B. Johnson Space Center

Houston, Texas 77058

SPACE SHUTTLE

FLIGHT SOFTWARE

VERIFICATION AND VALIDATION REQUIREMENTS

NOVEMBER 21, 1991

Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×

REVISION LOG

REV LTR

CHANGE NO

DESCRIPTION

DATE

   

BASELINE ISSUE (Reference PRCBD S052486, dated 10/04/91)

11/21/91

Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×

SPACE SHUTTLE

FLIGHT SOFTWARE

VERIFICATION AND VALIDATION REQUIREMENTS

Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
THIS PAGE INTENTIONALLY LEFT BLANK
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×

FOREWORD

Efficient management of the Space Shuttle program dictates that effective control of program activities be established. To provide a basis for management of the program requirements, directives, procedures, interface agreements, and information regarding system capabilities are to be documented, baselined, and subsequently controlled by the proper management level.

Program requirements to be controlled by the Director, Space Shuttle (Level I), have been identified and documented in Level I program requirements documentation. Program requirements controlled by the Deputy Director, Space Shuttle Program (Level II), are documented in, attached to, or referenced from Volume I through XVIII of NSTS 07700.

This document, which is to be used by members of the Flight Software community, defines the Space Shuttle Program baseline requirements for the Flight Software Verification and Validation process. All Flight Software Verification and Validation activity should be consistent with this plan and the unique items contained herein. The top level policies and requirements for Flight Software Verification and Validation are contained in NSTS 07700, Volume XVIII, Computer Systems and Software Requirements, Book 3, Software Management and Control.

All changes to NSTS 08271, Space Shuttle Program Flight Software Verification and Validation Requirements Document, in the form of change requests will be presented to the Shuttle Avionics Software Control Board (SASCB) for disposition. Change authority and management of the implementation strategy for the Verification and Validation requirements and processes in NSTS 08271 are hereby delegated to WA/Space Shuttle Engineering Integration Office via the SASCB. Revisions to this plan will be made as required to incorporate baseline changes to NSTS 07700, Volume XVIII, Book 3.

Leonard S. Nicholson

Deputy Director, Space Shuttle Program

Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
THIS PAGE INTENTIONALLY LEFT BLANK
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×

TABLE OF CONTENTS

 1.0

 

PURPOSE

 

1-1

 2.0

 

APPLICABLE DOCUMENTS

 

2-1

 3.0

 

INTRODUCTION

 

3-1

 4.0

 

SPACE SHUTTLE FLIGHT SOFTWARE (FSW) COMMUNITY

 

4-1

 4.1

 

SPACE SHUTTLE PROGRAM OFFICE (SSPO)

 

4-1

 4.2

 

MISSION OPERATIONS DIRECTORATE (MOD)

 

4-1

 4.3

 

ENGINEERING DIRECTORATE (ED)

 

4-2

 4.4

 

SAFETY, RELIABILITY, & QUALITY ASSURANCE (SR&QA)

 

4-3

 4.5

 

FLIGHT CREW OPERATIONS DIRECTORATE (FCOD)

 

4-3

 4.6

 

FLIGHT SOFTWARE DEVELOPMENT CONTRACTORS (IBM, ROCKWELL INTERNATIONAL)

 

4-3

 4.7

 

OPERATIONS CONTRACTORS (SHUTTLE TRANSPORTATION SYSTEMS OPERATIONS CONTRACTOR (STSOC), IBM ROCKWELL, ETC.)

 

4-4

 4.8

 

SYSTEMS DESIGN CONTRACTORS (ROCKWELL, LOCKHEED, CHARLES STARK DRAPER LABS)

 

4-4

 5.0

 

DEVELOPMENT APPROACH

 

5-1

 5.1

 

FLIGHT SOFTWARE DEFINITION ROADMAP

 

5-1

 5.1.1

 

Flight Software Needs

 

5-1

 5.1.2

 

Needs Analysis

 

5-1

 5.1.3

 

Discrepancy Report Analysis

 

5-2

 5.1.4

 

Space Shuttle Program Control

 

5-2

 5.1.5

 

Requirements Inspection

 

5-2

 5.1.6

 

Requirements Analysis

 

5-2

Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×

 5.1.7

 

Space Shuttle Program Authorization

 

5-3

 5.2

 

FLIGHT SOFTWARE DEVELOPMENT ROADMAP

 

5-3

 5.2.1

 

Design, Code, Unit/Module Test

 

5-3

 5.2.2

 

Load Build and System Test

 

5-4

 5.2.3

 

First Article Configuration Inspection

 

5-5

 5.2.4

 

Verification Test Procedure Reviews

 

5-5

 5.2.5

 

Functional Verification Testing

 

5-6

 5.2.6

 

Performance Verification Testing

 

5-6

 5.2.7

 

Configuration Inspection

 

5-7

 5.3

 

FLIGHT SOFTWARE MISSION PREPARATION ROADMAP

 

5-7

 5.3.1

 

Reconfiguration Data

 

5-7

 5.3.2

 

Vehicle Cargo System (VCS) Reconfiguration Data

 

5-8

 5.3.3

 

Reconfiguration Activities

 

5-8

 5.3.4

 

Integrated Mass Memory Unit Load

 

5-8

 5.3.5

 

Operational Validation and Certification Testing

 

5-8

 5.3.6

 

Performance and Certification Test Reviews

 

5-9

 5.3.7

 

Flight and Software Readiness Reviews

 

5-9

 5.3.8

 

Mass Memory Dump and Compare

 

5-10

Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×

LIST OF FIGURES

 5-1

 

Flight Software Definition Roadmap

 

5-11

 5-2

 

Flight Software Development Roadmap

 

5-12

 5-3

 

Flight Software Mission Preparation Roadmap

 

5-13

 A-1

 

SSMEC Software Requirements Definition Roadmap

 

A-9

 A-2

 

SSMEC Software Development Roadmap/p>

 

A-10

A-3

 

SSMEC Software Verification/Validation/Certification Roadmap

 

A-11

 A-4

 

SSMEC Software Mission Readiness Roadmap

 

A-12

Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×

1.0 PURPOSE. The purpose of this document is to define and establish the Space Shuttle Program baseline requirements for the Flight Software (FSW) Verification and Validation (V&V) process and to establish the activities and the responsible program elements in this process for both the Space Shuttle General Purpose Computer (GPC) and the Space Shuttle Main Engine (SSME). This baselines the V&V process roadmap utilized for FSW requirements definition, FSW development, and FSW mission preparation.

Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
THIS PAGE INTENTIONALLY LEFT BLANK
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×

2.0 APPLICABLE DOCUMENTS. The following documents are applicable to the requirements contained in this document. “(Current Issue)” is shown in place of the specific date and issue when the document is under Level II PRCB control. The current status of documents shown with “(Current Issue)” may be determined from NSTS 08102, Level II Document Description and Status Report.

NSTS 07700, Computer Systems and Software Requirements

Volume XVIII Software Management and Control

Book 3

(Current Issue)

Ref. Foreword

Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
THIS PAGE INTENTIONALLY LEFT BLANK
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×

3.0 INTRODUCTION. SSP FSW is defined, developed and used by the FSW community. Prime members of the FSW technical community are: the NASA Space Shuttle Program Office (SSPO), Flight Crew Operations Directorate (FCOD), Mission Operations Directorate (MOD), Engineering Directorate (ED), Safety, Reliability, and Quality Assurance (SR &QA), and their supporting contractors (IBM, Rockwell International, Loral, McDonnell Douglas, Lockheed, STSOC, etc.). In general, the primary responsibilities of these organizations in the FSW development, test and use are as follows: the NASA SSPO approves all FSW requirements changes, post development performance tests specifications (Level 7 & 8), and SAIL test requirements; the Flight Data Systems Division (FDSD) of the Engineering Directorate (ED) is responsible for technical management of the Operational Increment (OI) FSW development, verification, and maintenance and the FSW parallel certification activity; the flight crews of the FCOD are the ultimate end users of FSW during a Shuttle Transportation System (STS) mission; Mission Operations Directorate (MOD) develops the mission FSW requirements for each STS mission and is responsible for technical management of FSW reconfiguration, Level 8 verification testing, reconfigured FSW maintenance, crew training, and Shuttle mission simulation operation; SR&QA monitors FSW requirements, documentation, and tests to ensure that they are in accord with approved NASA standards and procedures; and the NASA supporting contractors perform the actual translation of FSW requirements into FSW computer programs and integrated mass memory loads for use in the Space Shuttle general purpose computers (GPCs) and independently verify and validate the operational FSW for each STS flight. MSFC is responsible for the Space Shuttle main engine controller software (reference Appendix A).

Two contractors, IBM and Rockwell International, respectively, are responsible for the development and verification of the Primary Avionics Software System (PASS) and Backup Flight System (BFS) basic software.

STSOC is responsible for the mission specific reconfiguration of the FSW and the flight Integrated Mass Memory Unit (IMMU) load build. Additionally, STSOC is responsible for the verification and validation of the reconfigured product per the program approved Performance Test Plan (PTP).

The FSW development contractors (IBM and Rockwell International) perform a parallel certification activity, which consists of a parallel build of the PASS/BFS images and a compare to that produced by STSOC. The development contractors also perform an independent set of verification testing of the reconfigured flight software.

Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
THIS PAGE INTENTIONALLY LEFT BLANK
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×

4.0 SPACE SHUTTLE FLIGHT SOFTWARE (FSW) COMMUNITY. Each member of the FSW technical community identified above has different objectives, goals, or perspectives with respect to the actual development and operational utilization of Shuttle FSW. Members of the FSW community support FSW development, test, and operations in multiple facilities. The various viewpoints and operational use, by members of the FSW community, provide an effective V&V function throughout the FSW life cycle. Examples of the different viewpoints with an indication of its role in the V&V process are provided in the remainder of this section for the GPC FSW and in Appendix A for the SSME FSW.

4.1 SPACE SHUTTLE PROGRAM OFFICE (SSPO). The SSPO has the final authority for all the FSW requirements. As such, each change in the existing FSW must be fully justified as a desirable and/or needed change to accomplish the planned mission with the minimum risk to flight safety, crew work load, mission objectives, and budgeted resources. A functional package concept was developed as a means of developing a long term approach to establishing the contents of the flight software. This approach was accepted by the program in order to allow more efficient use of the developers and program elements resources. The functional package concept is the implementation of a group of software changes that accomplish a specific goal (i.e., contingency abort envelope expansion, etc). The process begins with the development of functional packages for potential implementation. The package may contain Change Requests (CRs) being proposed for the current baseline as well as candidates for subsequent releases. The packages are presented to the Shuttle Avionics Software Control Board (SASCB) for prioritization and concurrence. The prioritized packages are then taken forward to the Program Requirements Control Board (PRCB) with the SASCB recommendation. After PRCB concurrence/direction is obtained on the content and priority of the packages, the SASCB community manages the implementation strategy.

A ceiling list of CRs is established from the approved functional packages. Each CR is then reviewed in detail by the developer to determine readiness for baselining, impact to software memory, and impact to resources (manpower). The requirements inspection review should identify issues with requirements or risks associated with implementation of each CR. When the CRs are determined to be mature, they are scheduled to the SASCB for dispositioning. The SASCB addresses benefits verses risk of implementation and identifies any outstanding technical issues with the CR and establishes an OI baseline which is then returned to the PRCB. Additions and/or deletions of functional packages, significant changes to package content or priority, and schedule changes are identified. The OI content and schedule are then baselined by the PRCB.

Appendix A describes the preparation and development cycle for the Space Shuttle main engine controller software. In this appendix the SASCB is identified as the final approving authority for software to be flown on a Space Shuttle mission.

Proposed FSW changes are presented to representatives of the technical community during SSPO control board meetings. Any issues that arise during these meetings are resolved prior to implementation approval.

4.2 MISSION OPERATIONS DIRECTORATE (MOD). The MOD develops the operational requirements for all components of a Shuttle mission. Included are the plans

Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×

and procedures for all communications, mechanical systems, remote manipulator system, electrical and environmental systems, flight design, flight dynamics, navigation (ascent/entry/orbital), ground support, reconfiguration, and mission training. MOD is composed of independent divisions of two or more branches supported by multiple contractors. The flight planning process involves a top down - bottom up structured approach to mission planning. Top level objectives are broken down into individual objectives for MOD divisions and/or branches who develop plans within their area of responsibility to attain their assigned mission objectives. Each lower level plan is integrated into the final mission plan and subjected to objective testing and management review prior to approval. MOD uses the Shuttle Mission Simulator (SMS) complex located in JSC Building 5 for validation of mission plans and procedures. When the FSW mass memory loads are released for mission operations, MOD uses the SMS with this load to train the flight and ground crews. The SMS is not used as a formal GPC FSW V&V tool. However, discrepancy reports (DRs) written against the SMS GPC software are reviewed for applicability to the mission GPC FSW load.

V&V Role: Once the mission plan has been approved, MOD organizations and/or their support contractors review and update mission requirement documents as required to accomplish the mission objectives stated in the areas of communications, mechanical systems, remote manipulator system, electrical and environmental systems, flight design, flight dynamics, navigation (ascent/entry/orbital), ground support, reconfiguration, and mission training. Changes are validated in MOD flight simulations using the SMS and flight planning software tools. The evaluation and approval process within MOD performs an effective V&V role for developing and verifying the FSW requirements.

4.3 ENGINEERING DIRECTORATE (ED). The responsibility of the ED is to ensure that the Shuttle Vehicle and its supporting equipment can functionally perform the mission objectives without exceeding safety limits and to ensure that the Shuttle FSW is developed and verified to meet all approved requirements. The ED supports the design and development of SSP hardware and software systems. Included are: regenerative life support systems; guidance, navigation, and control hardware and software systems; data systems hardware and software; electrical power, and propulsion systems; and remote manipulator systems. ED is composed of independent divisions with two or more branches supported by systems engineering contractors. Each Shuttle hardware or software system is subjected to detailed analysis by ED personnel to ensure design limitations of Shuttle hardware and software systems are not exceeded. ED personnel use the Software Development Facility (SDF) to perform all Level 6 and 7 verification tests prior to the OI FSW release. After the FSW OI is released, the Software Production Facility (SPF) is used to generate and verify all post OI delivery changes. ED personnel with Rockwell-Downey contractor support utilizes the SAIL to analyze Shuttle avionics hardware and software interfaces and operations. If ED determines that new hardware or software systems are required, appropriate systems requirements specifications are prepared and then submitted to the SSPO for approval. After the requirements have been approved, FSW implementation, if required, is then performed by the ED/FDSD organization with contractor support from IBM for the Primary FSW and Rockwell-Downey for the Backup FSW.

V&V Role: The ED has systems engineering responsibility for the total Shuttle hardware and software systems and evaluates the capability of each system to accomplish planned mission objectives. The ED/FDSD reviews each change in the

Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×

FSW (including post OI delivery patches) by means of Level 6 and 7 SDF testing to provide an independent NASA assessment and signoff on the completeness and correctness of all FSW changes. The mission plan is evaluated by ED personnel for each phase of flight operations and FSW logic or constraints to ensure that mission objectives can be achieved. When the FSW mass memory loads are released for mission operations, ED uses the SAIL with this load to verify hardware and software compatibility. The independent evaluation of mission performance by ED ensures that the modified software is compatible with the requirements as approved by the Space Shuttle Program Office.

4.4 SAFETY, RELIABILITY, & QUALITY ASSURANCE (SR&QA). The SR&QA is concerned with Shuttle vehicle, ground support systems and personnel safety; reliability of SSP hardware and software systems; maintainability of SSP equipment and documentation; and SSP quality assurance of hardware, software, and documentation. To this end, SR&QA is an active voting member of the SASCB, ensuring appropriate dispositions for FSW issues/changes. The SR&QA tracks Operation (OPS) Notes, User Notes, waivers associated with flight software discrepancies.

4.5 FLIGHT CREW OPERATIONAL DIRECTORATE (FCOD). FCOD is concerned with the satisfactory operation of the total integrated Shuttle system, including both hardware and software in the full range of nominal and off-nominal mission tasks. FCOD initiates changes and evaluates proposed changes and identified discrepancies for acceptability in the following functional areas: flight safety, crew interface suitability, closed-loop performance, and operational effectiveness. The SMS, SAIL, and SES are the primary tools for flight crew evaluations.

V&V ROLE: The flight crew assesses each change or discrepancy for flight safety and operational impacts. Depending on the situation, desktop review, SMS or SES simulation, or some combination of the three is used in the evaluation. The SAIL is used to validate flight software performance in a variety of nominal and stressed scenarios.

4.6 FLIGHT SOFTWARE DEVELOPMENT CONTRACTORS (IBM, ROCKWELL INTERNATIONAL). The development contractors are contracted to the ED FDSD (PASS) and the Orbiter and GFE Projects Office (BFS). Development contractors are primarily concerned with the implementation of FSW modules and their operation in Shuttle computers. Each contractor uses functionally independent organizations to analyze change requirements, design and code FSW changes, manage FSW configuration, build FSW OI loads, and verify that changes are correctly implemented. The development contractors perform rigorous reviews throughout the software definition, implementation, and verification cycles. These review processes cover requirements, design, code, test procedures, and test results and are designed to eliminate errors early in the software life cycle.

V&V Role: The development contractors maintain functionally independent organizations that review and examine the FSW at each stage of development. The requirements group ensures that the specified requirements are understood and that the FSW module designs incorporate the intent of these requirements. The programming group ensures that the FSW module designs are coded properly according to approved development standards. The test group verifies that the code executes properly and accomplishes the functions stated in the

Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×

requirements. The build group ensures that only approved FSW modules are used in loads released for verification and final delivery.

4.7 OPERATIONS CONTRACTORS (SHUTTLE TRANSPORTATION SYSTEM OPERATIONS CONTRACTOR (STSOC), IBM, ROCKWELL, ETC.). Operations contractors are defined as those contractors who reconfigure the FSW OI loads delivered by the development contractors for use on specific missions. The STSOC is responsible for preparing all reconfigured mission loads from the OI base delivered from the development contractor. The STSOC personnel integrate development loads with Government Furnished Equipment (GFE) FSW data, initialization data, telemetry format data, and FSW patches (late CR/DR correction) to prepare an integrated mass memory load for the Shuttle flight computers. STSOC personnel then perform a mission specific series of tests (Level 8) to verify the final integrated mass memory system performance. IBM and Rockwell International personnel perform parallel reconfiguration load builds and compare their resulting loads with STSOC products in a parallel certification process (IBM also performs parallel Level 8 testing for PASS software). STSOC prepared mission specific releases are used by various operations contractors in JSC simulation facilities (SMS and SAIL) to train and prepare for each specific mission and/or validate the ability of the integrated mass memory loads to perform the specified mission. For example, STSOC personnel are concerned with the telemetry and command compatibility with the MCC software. STSOC and flight crew personnel are concerned with operational flight training for the planned mission; and Lockheed examines the avionics hardware compatibility with the STSOC prepared integrated mass memory load, as well as its interface with the launch processing system software.

V&V Role: IBM and STSOC personnel independently perform validation testing on the STSOC integrated mass memory loads. IBM and Rockwell International perform a parallel build of the PASS/BFS images and conduct a bit for bit compare with the STSOC produced images. Other operations contractors evaluate FSW performance in detail for each of their areas of concern. This provides many views of the FSW by different contractors which result in an effective V&V look at the delivered FSW product. Problems found during operations by any user, are documented via Discrepancy Reports (DRs) and tracked by the SSP, FSW development contractor technical manager (ED/FDSD), FSW reconfiguration contractor technical manager (MOD/Reconfiguration Management Division (RMD), and the SR&QA until corrected or satisfactorily resolved.

4.8 SYSTEMS DESIGN CONTRACTORS (ROCKWELL, LOCKEED, CHARLES STARK DRAPER LABS). Systems design contractors are defined as those contractors who utilize the SAIL to verify: (a) The FSW loads are compatible with hardware interfaces; (b) the FSW performs as designed; and (c) the FSW is compatible with the mission requirements. These contractors include Rockwell-Downey and the ED subcontractors, Lockheed and Charles Stark Draper Labs.

The system design contractors form the SAIL verification test team sponsors who are responsible for recommending a series of tests for the purpose of integrated verification of the FSW and vehicle hardware. They establish those test requirements in team meetings and propose them to the SAIL Management Working Group which submits the package to the Shuttle Avionics Systems Review (SASR) board for approval. Once approved, the tests are scheduled and conducted. This process is followed for both the engineering and flight cycles of the FSW.

Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×

In preparation for SAIL testing, test procedures are generated and reviewed. A series of meetings ensure proper test design. For each test, signature plots are generated to anticipate test results. If the test results do not favorably agree with signatures generated from independent offline simulations, anomalies are documented and may result in FSW DRs being generated. In addition, the digital test results are subjected to post test pass/fail criteria which may uncover other anomalies.

Both the PASS and BFS are tested by this process. In as much that both systems are derived from the same set of software requirements, when appropriate, they should provide similar results given similar conditions. Flight critical mission phases (ascent and descent) are tested utilizing identical conditions and run scenarios for each system and the results compared at key mission points to determine if both systems provide performance agreement. Again, if the test results do not favorably agree, anomalies are documented which may result in FSW DRs being generated.

V&V Role: The system design contractors independently perform verification of the FSW loads in an integrated hardware/software manner in the SAIL. Test requirements are independently generated and approved. Test results are compared to independently generated signature data. In addition to the explicit testing mentioned above, the BFS is a validation of the PASS. Since the software for the PASS and the BFS are developed and coded by different organizations, under different constraints and requirements, comparable critical outputs provide for a validation and goodness of the design of both software systems. Also, performance agreement between the two systems given similar conditions, is a strong case of V&V. Miscomparisons result in FSW DRs. Dispositioning of the DRs are worked through the SASCB.

Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
THIS PAGE INTENTIONALLY LEFT BLANK
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×

5.0 DEVELOPMENT APPROACH. There are three distinct “roadmaps” for the current SSP FSW development process - Definition, Development, and Mission Preparation. The FSW Definition Roadmap identifies the activities and SSP approval processes (SASCB/PRCB) used to define the FSW requirements and ensure program resources are allocated to facilitate implementation schedules. The FSW Development Roadmap identifies the activities and FDSD/development contractor controls used to implement approved SSP requirements and verifies that the delivered FSW correctly implements these requirements. The FSW Mission Preparation Roadmap identifies the activities and programmatic controls used to transform the delivered FSW into a Flight Computer Mass Memory Unit (MMU) Load and to validate that the MMU is capable of properly and safely supporting the Shuttle design mission.

The SSP FSW process is an ongoing, iterative, and dymamic process. Provisions have been made to accommodate FSW changes throughout this FSW process.

5.1 FLIGHT SOFTWARE DEFINITION ROADMAP. The FSW Definition phase begins with a SSP requirement defined by the technical community and ends with an approved FSW Implementation Plan. The implementation plan includes approved requirements, resource allocations, and development schedules. The SSP FSW provides evolving capability to accomplish a wide range of Shuttle missions. FSW requirements changes are defined in SCRs. Problems found during operations of a FSW load are documented in Discrepancy Reports (DRs) that may require changes to the operational code or FSW requirements to correct. Each major capability change set is identified as an OI. Shuttle missions use a specified OI modified by mission or vehicle specific requirements. Mission and vehicle specific requirements are uniquely described in Data Change Requests (DCRs) approved in the SASCB weekly meetings. The FSW Definition Process is allocated approximately three months on the FSW development template, and ends with an approved baseline CR identifying the FSW CR/DRs to be implemented in an OI (see Figure 5.1).

5.1.1 Flight Software Needs. New OIs, FSW modifications, mission data, new designs and FSW corrections begin with an expressed need defined by the SSP FSW community. These needs are identified through flight or mission plans, vehicle or equipment modificatioms, flight or ground crew requests, program directives or objectives, etc.

5.1.2 Needs Analysis. Once a need is defined, the FSW community must perform analysis to determine if these needs should become approved requirements for the SSP FSW. These analyses are performed by knowledgeable Shuttle avionics engineering personnel through Mode (multi-organizational design engineering) Teams by mission planning personnel, vehicle or flight equipment designers, FSW development personnel, payload users, or flight and ground crew personnel.

The end result of this analysis will define the actual FSW requirements for further consideration either into am OI or adding to a specific STS flight or mission.

Embedded V&V Activity: V&V activity is accomplished through the system engineering analyses performed by FSW community members. The FSW needs formulated by the community at large are subjected to systems engineering analysis by other members of the FSW community to validate requirements. Once

Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×

the knowledgeable FSW community personnel determine a valid FSW requirement exists, a sponsor prepares the necessary change documentation.

5.1.3 Discrepancy Report Analysis. DRs are problems or anomalies discovered in the operational FSW or potential hazards identified in the requirements design. DRs are generated throughout the software life cycle by the various members of the FSW community involved in development, verification, testing and/or operations (e.g., FSW developers, flight crew, mission controllers, Level 8 testing, certification testing, SAIL integrated hardware/software testing, etc.).

DRs are analyzed to determine the appropriate disposition (i.e. waive, fix, Program notes, no DR). This analysis includes a determination of a need for a FSW requirement change. If analysis indicates that a requirements change is needed, the DR disposition will recommend that a CR or DCR be submitted by the FSW community for consideration. Otherwise, if a code fix is required, the appropriate FSW development group will provide the necessary implementation plan for correction.

Embedded V&V Activity: Discrepancy reporting is a V&V activity performed by the continuous utilization, evaluation, and review of the operational FSW by the technical community. The FSW evaluation DRs found are subjected to detailed systems engineering analysis to determine their criticality and validity. The FSW community software engineers evaluate the range of options available to correct the discrepancy and prepare the necessary disposition recommendations for action by the SASCB.

5.1.4 Space Shuttle Program Control. The sponsor for a FSW change will prepare the necessary CR/DCR and present it to the SASCB. If additional resources or SSPO approval are required, the sponsor must also defend the proposed change to the PRCB.

5.1.5 Requirements Inspection. Requirement Inspections are formal requirement reviews with FDSD analyst, FSW contractor requirement analysts, FSW community peers, software programmers, and Level 6/7 verification representatives with a moderator for the reviews. These reviews are open to all members of the technical community and will often include the author of the requirements documents. The purpose of this function is to ensure that the intent of the requirements is understood and to clarify the interaction of multiple principle functions affected by the new or modified requirements. The requirements inspection should identify issues with the requirements or risks associated with the implementation of each CR and resolve any requirements issues identified.

Embedded V&V Activity: The V&V activity is through the involvement of all organizations in the FSW community. They effectively validate the interface compatibility and appropriate interactions between all the affected functions. As a team, they verify that the requirements are correct and complete assuring that the intent is uniformly understood throughout the FSW community.

5.1.6 Requirements Analysis. The FSW development contractors evaluate the requirements and determine an approach to implement them. Once this approach is determined, the development contractor must evaluate the resources required for implementation and develop an implementation schedule. This schedule becomes a

Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×

recommendation to FDSD from the development contractor. FDSD reviews the recommended implementation plan and approves their presentation to the SASCB. If there are issues with the development contractor 's understanding of the requirements or their intent, these issues are resolved with the sponsor and reviewed by the community in a formal requirements inspection. A correction CR is submitted if required.

Embedded V&V Activity: V&V activity is accomplished through the development contractor's system requirements analyses organization. Communications with other FSW community members adds required insight to evaluate and identify requirements issues. Corrective actions are recommended as necessary.

5.1.7 Space Shuttle Program Authorization. The NASA FSW management and their development contractor present an implementation plan for either a new OI or a mission specific CR/DR for a current OI to the SASCB. If additional budgeted FSW resources are required, the proposed change must be presented to the PRCB for approval.

The output of the SASCB is an approved OI baseline content and schedule identifying the CR/DRs to be implemented for a specific FSW operational capability. The SASCB takes the recommended OI baseline content and schedule forward to the PRCB for formal program approval. The SASCB meets weekly, and approves mission specific CR/DCR/DRs for implementation or acceptance for flight with waivers or user notes up to flight time.

5.2 FLIGHT SOFTWARE DEVELOPMENT ROADMAP. The FSW Development Phase begins with the approved baseline identifying the CR/DRs approved for implementation in a new OI and ends approximately 16 months later with the delivery of new Primary Avionics Software System (PASS) and Backup Flight Software (BFS) software OI loads to the FSW Mission Preparation Phase. This OI software is released to the NASA users by FDSD at the formal OI Configuration Inspection (CI) milestone (see Figure 5.2).

The FSW development is the responsibility of the Primary Avionics Software System contractor - IBM, and the backup flight software contractor - Rockwell International under the technical management of FDSD. Both contractors utilize the NASA JSC SDF to develop and test FSW until a new OI is delivered to NASA at CI. The SDF activities are referred to as “Backroom” activities. The PASS and BFS FSW is designed, coded, tested, and verified in this phase. The FSW is subjected to two levels of independent verification - Level 6 (Functional) testing and Level 7 (Performance) testing.

5.2.1 Design, Code, Unit/Module Test. The development contractors use separate groups to develop FSW in the SDF. Separate groups are responsible for all requirements analysis and programming: one for managing configuration and building FSW releases; and another group is responsible for verification testing of the FSW for the new OI delivery. Members of these groups attend inspections as presenter or peers, as required by the type or complexity of the changes, to review the developed products. Each inspection follows an inspection checklist to ensure that all procedures, and standards have been followed. Approval is received from the moderator reflecting the direction of the inspection team.

DESIGN: Approved CRs contain the requirement specifications that the new OI delivery is expected to provide. These requirements are the basis for FSW

Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×

designs. IBM and Rockwell convert the requirements stated in approved CR/DRs to detailed software designs which are documented in Detailed Design Specification (DDS) documents. Design inspections are then held where the designers present their designs to knowledgeable PASS or BFS FSW engineers for review.

CODE: Upon completion of the detailed design, the PASS or BFS software developer then writes FSW code implementing the design. A listing of the code is prepared and presented to knowledgeable PASS or BFS programmers for review at a Code Inspection. The design inspections and code inspections are sometimes combined for less complex implementations.

UNIT/MODULE: Once the code is completed, Unit (PASS Level 1, BFS Level 2) tests are performed to verify equations, logic paths, and/or range of values. Module (PASS Level 2, BFS Level 3) tests are executed, if required, to verify the module interface (Input/Output) performance. These tests are sometimes combined for less complex changes. The results of these Unit Tests are presented to knowledgeable PASS or BFS programmers for review at a Test Inspection.

Embedded V&V Activity: Each activity has detailed written procedures which the developer's software quality assurance personnel monitor for compliance. Preparation for each inspection includes a review of the procedures and standards utilized to accomplish a design, code a module, or perform a test. Detailed checklists are completed and then reviewed by the attendees prior to inspections required for code design and test reviews.

V&V is the responsibility of the development contractors. They have accomplished this by forming independent organizations responsible for tracking and verifying the approved requirement changes to the FSW. All reviews and inspections are controlled by peer moderators, without management involvement other than oversight review and approval of FSW development standards and procedures.

The design is inspected to ensure that the design reflects both the stated requirements as well as the intended requirement. The code is inspected to ensure conformity to FSW standards, prevent unintended functions, and control inefficient Central Processing Unit (CPU)/memory consumption. Design and code inspections are sometimes combined for less complex changes. Tests are inspected to ensure that tests are performed at applicable levels of development (i.e., Unit, and Module) prior to beginning FSW integration via load build process.

5.2.2 Load Build and System Test. The OI development cycle has approximately a 16-month template. During this period, multiple load releases will be built. Each FSW load release contains the preceding load release plus updates that have completed the development process (design, code, unit/module test). As each load is built, it will receive system level (PASS Level 2, BFS Level 3) testing in the SDF. Both PASS and BFS loads receive Level 3/4 testing before release for verification testing. The object of these tests is to test functional interfaces, multiple functions, timing, system interface, and mission profile. Each new load is released to the Level 6 test group for detailed verification tests upon successful completion of the system level tests. Level 7 test group begins performance verification tests when all of the approved CR/DRs have been included in a load release at the First Article Configuration Inspection (FACI).

Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×

The final development OI load release is known as the Configuration Inspection (CI) load.

Embedded V&V Activities: The PASS or BFS development contractor maintains responsibility for all V&V activities until the CI load is released.

The development contractor FSW configuration management ensures that FSW modules are never added or changed unless proper authorization and procedures have been followed. The system level (Level 3/4) tests conducted on each new load build consist of standardized system tests of the basic load characteristics and capabilities. Tests are performed using SDF ground unit (nonflight) GPCs with a functionally complete FSW MMU load.

5.2.3 First Article Configuration Inspection. This is a formal review milestone in the OI development template. This milestone officially begins the verification phase of an OI. At this point all CR/DRs have been incorporated into the FACI Verification Load, which normally becomes the base load for the next OI entering development. This milestone occurs approximately 8 months after the initial OI baseline has been approved by the SASCB. The development contractor reports on OI development progress, Level 6/7 verification testing planned, and any planned post-FACI work.

Embedded V&V Activity: This is the first review in the OI development cycle where all elements of the FSW community participate. This review allows appropriate members of the FSW community to evaluate the OI status and determine if required development for all functions has been achieved prior to proceeding to independent verification testing.

5.2.4 Verification Test Procedure Reviews. Two levels of testing are performed on operational hardware by independent development contractor organizations. Detailed functional (Level 6) testing consists of module functional tests against requirements. System level (Level 7) performance testing is conducted under operational flight conditions.

Inputs to this activity are the CR/DR baseline documents approved by the SASCB. Level 6 test analysts develop Verification Test Procedures (VTPs) to be used during testing. Level 6 VTPs are standard functional tests for FSW Principle Functions documented in SDF data sets. Specific tests are selected or modified from these standards. New tests are prepared, as appropriate, by Level 6 test analysts to test new or modified functional capabilities. Generic Level 7 tests consist of Guidance, Navigation, and Control (GN&C) System Integrity Tests, System Services Tests, and Vehicle Cargo Systems Tests. Level 7 OI specific tests are New Capability Performance Tests designed to verify the new performance capability provided by one or more CRs implemented in the new OI. Level 7 Verification Tests are developed through a community review process and are documented in a Verification Test Specification CR approved by the SASCB.

Embedded V&V Activities: During the Level 6 Verification Test Procedure Inspections conducted by the development contractors, interested parties from the FSW technical community provide inputs, identify issues or review tests for use and approved by FDSD. The Level 7 test specifications are reviewed in Test Coordination Team (TCT) meetings attended by interested parties from the community. The resulting Level 7 Verification Test Specification is documented

Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×

in a CR and formally approved by the SASCB. The object is to ensure that planned tests verify requirements as well as overall system performance.

5.2.5 Functional Verification Testing. This activity is the execution of the Level 6 Functional Tests approved in the preceding activity. Level 6 testing is very flexible in that each test focuses on FSW module changes. The FSW is functionally tested by exercising, on flight equivalent operational computer hardware in the SDF, FSW Principle Functions affected during CR/DR implementation. Tests can include partial trajectories and engagement transitioning (BFS only) if a function was affected by changes. Tests may include overriding math model inputs with out-of-limit stress conditions.

Functional Test Reviews: Level 6 functional tests are reviewed independently. Tests are conducted on all software changes throughout the development template. Each Level 6 test case has a review scheduled by the development contractor to review the test results. These reviews are attended by development contractor personnel, NASA FDSD analysts, and other FSW community personnel as required. The test results are reviewed, and issues are recorded for resolution. Level 6 issues are reported by the developer at the CI. Level 6 Epilogues (Test Reports) are published approximately 6 weeks after the CI, and delivered to members of the FSW community upon request.

Embedded V&V Activities: Development contractors are responsible for performing the tests according to the procedures and conditions approved in the verification test procedure. Functional tests are designed to examine the total functional range of specific principle functions provided by the CR/DRs implemented in the new OI. Participation of affected parties from the FSW community in the VTP Inspections and use of independent organizations by the development contractor for Level 6 testing accomplish the V&V task during the design, conduct, and review of tests. Detailed results from each Level 6 test case are evaluated with FDSD and other interested technical community members.

5.2.6 Performance Verification Testing. This activity performs the Level 7 Performance tests contained in the Verification Test Specification CR reviewed in TCT meetings and approved by the SASCB. Level 7 testing normally begins with the delivery of the FACI Verification Load, and may also utilize later verification load deliveries to complete Level 7 testing. The tests are performed in the SDF using operational flight equivalent computer hardware, and simulated mission conditions emulating an OIs operational mission environment.

Level 7 tests place emphasis on evaluating PASS or BFS system performance instead of Principle FSW Functions. The Level 7 tests more closely resemble the flight profile then the Level 6 tests. The tests do include engage transition testing (BFS only).

Performance Test Reviews (PTR): Level 7 performance tests are reviewed as a group at a formal PTR. Each New Capability and Generic Test report is mailed to members of the FSW community on the Level 7 Test Report Distribution List 4 to 6 weeks prior to the PTR for review and evaluation. The PTR is held 1 week prior to the CI and any unresolved Level 7 issues are reported by the developer at the CI. The developer will resolve all Level 7 issues remaining open at the CI, and prepare a supplemental report for the CI attendees.

Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×

Embedded V&V Activities: By use of standardized generic Level 7 tests, each OI delivery is tested to the same specifications under the same conditions. New Capability Performance tests are designed to exercise the full envelope of capabilities provided by the specific CR/DRs implemented in the new OI. Participation of the FSW community in the TCTs and PTRs in addition to use of independent organizations by the development contractor for Level 7 testing accomplish the V&V tasks during the design and conduct of tests.

5.2.7 Configuration Inspection. This is a formal review milestone in the OI development template. This milestone officially completes the development phase of an OI. At this point all CR/DRs have been incorporated into the CI Load. This milestone occurs approximately 8 months after FACI. The development contractor reports on OI development issues, Level 6/7 verification test issues, delivers updated FSW documentation, and releases the CI load to NASA.

Embedded V&V Activity: The CI is preceded by Level 6 test results review meetings, and a formal Level 7 Performance Test Review. Each review performs an V&V function by including members of the technical community in the review and verification of test results. The purpose of a review is to ensure that the requirements contained in the CR/DRs approved by the SASCB for implementation in an OI have been implemented correctly and verified according to approved SSP standards for FSW development.

5.3 FLIGHT SOFTWARE MISSION PREPARATION ROADMAP. The FSW Mission Preparation Phase begins with release of the PASS/BFS OI loads from the development contractors. Mission specific requirements documents are developed by NASA MOD/RMD and approved by the SASCB. These inputs are integrated in the SPF into an integrated mass memory unit software load and submitted to various operational users for mission preparation and testing including final flight operations. SPF activities are referred to as “Frontroom” activities. The mission preparation phase requires approximately 9 months from the delivery of the OI loads until the first STS mission is flown using the newly developed OI capability. Mission preparation activities have two major cycles; one for the initial FSW mission reconfiguration (engineering cycle) at approximately 6 months prior to flight (L-161 days), and the flight cycle at approximately 3 months prior to flight (L-77 days). Partial updates and corrections may be applied as part of the reconfiguration process. Parallel mission preparations are performed for multiple STS missions utilizing the same FSW OI load (see Figure 5.3).

5.3.1 Reconfiguration Data. The STSOC personnel support NASA MOD/RMD who define the mission requirements and vehicle specific data (I-Loads), which are used to reconfigure the PASS and BFS OI baseline loads for specific missions and vehicles. STSOC prepares input data for the Shuttle Transportation Automated Reconfiguration (STAR) and Measurement and Stimulus (MAST) FSW reconfiguration tools. MSFC personnel develop Space Shuttle Main Engine Controller (SSMEC) software to be used with each Shuttle engine and deliver the SSME software to the mission preparation process as GFE software (Reference Appendix A). SSMEC software configuration is managed by the MSFC NASA SSP Project Office, similar to the SASCB at JSC. Reconfiguration also includes providing SPF simulator initial conditions and simulation model preparation data.

Embedded V&V Activities: I-Loads are audited by I-Load owners prior to approval and after flight cycle load build. Identical simulator test conditions are

Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×

provided to STSOC and IBM for their Validation (Level 8) Test groups. Performance tests are independently executed by STSOC and IBM to perform parallel certification of the reconfigured FSW.

5.3.2 Vehicle Cargo System (VCS) Reconfiguration Data. STAR/MAST data are independently processed by two different contractors, IBM and STSOC, using configuration controlled processing tools to generate the VCS software inputs required for a mission specific FSW load. STSOC is the STS operations contractor responsible for producing the Integrated Mass Memory Unit (IMMU) loads used during an STS mission. An IBM organization separate from the development organization is responsible for independently duplicating and comparing STSOC software products during the mission preparation phase.

Embedded V&V Activities: Each contractor verifies the data source inputs, checks the resulting syntax, and verifies consistency of individual products. The independently produced products are compared and any unexpected results are reported to the FSW Integrated Baseline Control Board (IBCB) community and resolved.

5.3.3 Reconfiguration Activities. The FSW development contractors are responsible for developing and maintaining all software tools which can affect the reconfigured FSW memory loads. At CI, STSOC receives FSW build tools that are under SASCB control.

The OI validated loads are reconfigured by the implementation of Mission/Vehicle unique data, and the VCS Recon products. IBM and STSOC independently reconfigure the baseline PASS OI FSW while Rockwell-Downey and STSOC independently reconfigure the baseline BFS OI FSW. The STSOC BFS FSW load is then delivered to IBM for application to the IMMU.

Embedded V&V Activities: IBM and Rockwell-Downey parallel certification groups compare the STSOC developed FSW loads to theirs and report any unexpected results to the IBCB community.

5.3.4 Integrated Mass Memory Unit Load. The Integrated Mass Memory Unit (IMMU) load contains the actual flight programs cycled in the Space Shuttle GPCs and/or flight equivalent hardware used in SSP ground facilities. The PASS, BFS, SSME, etc. software are integrated by STSOC into a mast er IMMU load for operational use by all FSW users. IBM parallel certification builds an independent IMMU load for comparison with the IMMU load built by STSOC.

Embedded V&V Activities: A copy of the STSOC IMMU load is compared to the IBM IMMU load and any unexpected results are reported to the IBCB community by IBM parallel certification via the certification audit report. IBM then uses this copy of the STSOC IMMU load for their parallel certification process certification tests. The STSOC produced IMMU load is provided to the SAIL, Shuttle Mission Simulator (SMS), KSC Cargo Integration Test Equipment, KSC Avionics Test Set (KATS), Orbiter, and other FSW users for operations and/or mission testing. The Shuttle Engineering Simulator (SES) does not receive a copy of the IMMU load but does use a fortran equivalent build of the IMMU load. This fortran equivalent build is independently supplied by ED.

5.3.5 Operational Validation and Certification Testing. Level 8 (Mission) testing is performed in the SPF using flight equivalent GPCs interfaced with a

Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×

mainframe computer containing Shuttle math models simulating the mission conditions necessary to test the FSW. The SPF simulator conditions and math model data are built into a simulation load prior to beginning FSW testing. Level 8 testing, whose requirements are controlled by the SASCB in the Performance Test Plan, is conducted using the final (L-77) reconfiguration load which contain mission unique I-loads. The SPF simulation does not provide a realtime simulation of mission operations which requires scripting of test scenarios. Validation testing is performed by STSOC using STSOC prepared SPF simulation and test scenarios. Parallel certification testing is performed by IBM using IBM prepared SPF simulation aud test scenarios. The IBM SPF simulator build is compared to the STSOC equivalent.

Operational testing is defined as the operational use of the FSW during mission preparation (i.e., flight and ground operations training, mission procedures development, etc.) and SAIL testing. Operational testing is a realtime operation using flight equivalent and simulated flight hardware, as well as a full complement of flight computers. The SAIL, SMS and SES provide a flight crew interface. The entire mission is flown in the SMS during flight crew training. Problems found during operational testing are recorded in DRs, and submitted to the appropriate organization for analysis or resolution.

Embedded V&V Activity: The STSOC SPF simulator datasets are compared to those developed by IBM to ensure functional compatibility. IBM performs parallel certification test cases which are similar, but not identical, to the STSOC Validation Level 8 test cases to ensure software mission performance.

Crew and mission operations training in the SES and SMS exercise the man-in-the-loop FSW interface to validate mission capability. SAIL is used to verify the integrated hardware/software interfaces as well as mission capability and the man-in-the-loop FSW interface testing.

5.3.6 Performance and Certification Test Reviews. These reviews are milestones leading to the release of the FSW for use in a STS mission. STSOC conducts the Performance Test Review (PTR) and presents the results of their analysis of the Level 8 tests conducted during Performance testing to the FSW community for concurrence. IBM prepares a Certification Test Report (CTR) for each STS mission that presents the results of their analysis of the test cases executed during certification testing.

5.3.7 Flight and Software Readiness Reviews. The Software Readiness Review (SRR) is held approximately 3 weeks prior to flight. The SRR is conducted by NASA to allow all members of the FSW community to review FSW open issues relating to the software 's ability to perform the planned mission. The results of the Level 8 and certification testing are reviewed, as well as any software issues encountered during operations.

The Flight Readiness Review (FRR) is held approximately 2 weeks prior to flight, with a follow up FRR held approximately 2 days prior to flight to resolve any remaining issues that may affect the planned mission. The FRR is held by the SSPO to allow all members of the STS community to review and disposition open STS hardware and software issues related to the planned mission. All aspects of flight vehicle preparation are reviewed and flight or mission related concerns recorded and dispositioned.

Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×

Embedded V&V Activities: Each FSW contractor and NASA FSW organization having a role in preparation of FSW for the flight/mission is required to certify that preparations are completed and that to the best of their knowledge there are no known problems that affect the safety of the flight or completion of the STS mission.

5.3.8 Mass Memory Dump and Compare. Five days prior to launch, the Orbiter MMUs are dumped and compared: (1) To the STSOC mission baseline load (by STSOC); and (2) to the IBM parallel certification mission baseline load (by IBM). All differences are analyzed and evaluated to ensure that only approved changes have been implemented in the final flight MMU. The MMUs are mass storage devices (magnetic tapes) in the Orbiter on which the IMMU load is loaded and from which the flight computers receive the FSW load for mission support.

Embedded V&V Activities: The MMU loads are compared bit by bit by the reconfiguration and parallel certification contractors, and any difference must be explained prior to flight authorization by the SSPO.

Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×

Figure 5-1 Flight Software Definition Roadmap

Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×

Figure 5-2 Flight Software Development Roadmap

Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×

Figure 5-3 Flight Software Mission Preparation Roadmap

Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
This page in the original is blank.
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×

APPENDIX A

SPACE SHUTTLE MAIN ENGINE CONTROLLER SSME FLIGHT SOFTWARE DEVELOPMENT AND VERIFICATION AND VALIDATION

Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
This page in the original is blank.
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×

APPENDIX A

SPACE SHUTTLE MAIN ENGINE CONTROLLER SSME FLIGHT SOFTWARE DEVELOPMENT AND VERIFICATION AND VALIDATION

1.0 PURPOSE. The purpose of this report is to describe the Block I SSMEC flight software development process and identify the role of embedded V & V in this process.

2.0 INTRODUCTION. The Marshall Space Flight Center (MSFC) in Huntsville, Alabama is the NASA center and Rocketdyne of Canoga Park, California is prime contractor responsible for the Space Shuttle Main Engines including the SSMEC flight software. The SSMEC flight software, currently used in the Space Shuttle Program (SSP), consists of a baseline assembly, and changes which implement approved requirements and/or correct minor problems. When sufficient changes have accumulated, a new assembly is developed and baselined. SSMEC software provided to JSC/KSC is customized for use on a specific SSME through the incorporation of Logic Change Notices (LCN) and Operational/Adaptation Data (OAD) values provided for individual STS flights.

SSMEC software changes, released as LCNs, are packages containing the definition of the change as described in a Controller Logic Change Request (CLCR); the generating Unsatisfactory Condition Report (UCR), if applicable; and the appropriate requirements and/or design document changes or redlined mark-ups. Also, the Verification Test Outline and the appropriate Test Requirement Document Changes are generated. The LCN number is assigned when a CLCR is approved by Rocketdyne Software Change Control Board (SCCB).

The software is developed at Canoga Park, California, is verified/validated in the Hardware Simulation Laboratory (HSL) at MSFC, and is certified at Stennis Space Center (SSC). Software verification/validation/certification is performed prior to release to JSC/KSC. Software Quality Assurance (SQA) monitors the software development process through the life cycle. Honeywell is the subcontractor responsible for the design and development of the SSMEC hardware and provides independent engineering assessment of SSMEC software changes.

The Software Review Group (SRG) is an informal review group consisting of MSFC, JSC, KSC, Rocketdyne including SSC, Honeywell, and other personnel which addresses the concerns of the appropriate SSMEC software community in the software development process. The SRG meets weekly via telecon to review the SSMEC software status/schedule, discuss software changes or potential software changes and any UCRs. Through the SRG, the SSMEC software community is able to provide technical assessments of possible system impacts and/or areas of concern early in the development process.

3.0 DEVELOPMENT APPROACH. There are four distinct “Roadmaps” for the current SSMEC software development process: Requirement Definition, Software Development, Verification/Validation/Certification and Mission Readiness. The Requirement Definition Roadmap identifies the activities and related control mechanism used to control changes to the SSMEC software. The Software

Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×

Development Roadmap identifies the development contractor activities and controls used to develop an LCN package at the Rocketdyne Canoga Park facility. The Verification/Validation/Certification Roadmap describes the activities and controls used to verify that the software delivered to MSFC meets approved requirements. The Mission Readiness Roadmap describes the activities associated with insuring that the software products delivered to JSC are ready for use with the target STS flight.

3.1 SSMEC REQUIREMENTS DEFINITION. Prospective changes are generated by the entire SSMEC software community. These prospective changes consist of requirement enhancements or corrections to SSMEC software discrepancies. SSMEC software discrepancies are documented in Software Problem Reports (SPRs) and UCRs. SPRs are a Rocketdyne mechanism for tracking discrepancies which are discovered during the verification process. UCRs are written directly against problems found in released software or converted from open SPRs when the software is released. Rocketdyne Engine Systems group documents proposed changes in a Controller Logic Changes Request (CLCR). The CLCRs are presented to Rocketdyne 's SCCB for review and disposition, and the CLCRs and any UCRs are presented to the SRG for its review. The SCCB dispositions the CLCRs as: revise, approved, or canceled. They are signed by the System Software Manager, the SSME Flight Operation Manager, and the SCCB Chairman. CLCRs which are dispositioned to be revised are iterated with the SCCB until they are approved or canceled. Approved changes are provided to the software development group for implementation and will become part of an LCN package. The SCCB consists of system analysts, flight performance specialists, avionics integration personnel, Canoga software personnel, and Software Quality Assurance (SQA) personnel.

Embedded V & V: Rocketdyne evaluates SPRs/UCRs or changes proposed by the SSMEC software community to ensure that the SPRs/UCRs are valid and that the proposed changes reflect valid SSME system changes and/or SSP changes. Honeywell ensures compatibility of software changes with the SSMEC hardware and verifies that no single point failures are introduced. The SRG members assess the impact in their area of responsibility and will address any concerns at the SRG. Approval of requirements are defined in the Embedded V & V Paragraph 3.3.

3.2 SSMEC SOFTWARE DEVELOPMENT. The Canoga Software Group prepares LCN packages which contain the CLCR and marked-up pages from all affected requirements and design documents. Also, the test requirements document, which is not a part of the LCN package is updated. If the CLCR is the result of a UCR, the UCR is also included. The LCN package goes through three implementation phases: requirements, development, and test. The CLCR is analyzed and the requirement documents are marked-up to reflect the requirement changes. The LCN marked-up requirements are analyzed and the design documents are marked-up to reflect the updated design. The marked-up requirement/design documents are reviewed in an informal development contractor Critical Design Review (CDR) and the design is coded and assembled. The code receives an internal desk audit of the source code listings. Once the code is approved, the software patch is tested in the Canoga Software Laboratory (CSL) with the appropriate baseline. The Verification Test Outline (VTO) is prepared to identify suggested verification activities for each change. The LCN package is then reviewed, and delivered alone with the software patches, the VTO, and the marked-up test requirements document to the HSL for verification test with the specified SSMEC baseline assembly.

Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×

Embedded V & V: Personnel from software, Engine Systems, and SQA review and sign the LCN package to ensure that the intent of the requirements is understood and can be implemented correctly. Rocketdyne ensures that all modifications to the SSMEC software are compatible with the current SSME and SSMEC hardware. Rocketdyne verifies that the design correctly implements the requirements. The code is inspected and analyzed to ensure the design is implemented properly and efficiently. SSMEC software integration test results are reviewed and problems encountered during tests are corrected and the software is retested. Rocketdyne verifies that all development activities have been completed. The LCN package is signed off by personnel responsible for requirement/design and code. Honeywell SSMEC systems engineers review the LCN to ensure that the modified design is compatible with SSMEC hardware operations and that no single point failures are introduced.

3.3 SSMEC VERIFICATION/VALIDATION/CERTIFICATION. SSMEC software verification is conducted in the HSL at MSFC and software certification is conducted on the engine Hotfire test stand at SSC. LCN packages are delivered to Rocketdyne HSL personnel at MSFC, who review the software requirement changes and the VTO from the Canoga Park Software Group. From the analysis of the requirement changes, the test procedures are updated, as required, and reviewed for approval. Each LCN is then verified in the HSL. All discrepancies encountered during verification are reported by SPR and, if necessary, corrections are made to the LCN and the verification is then repeated. Upon successful completion of the verification, which includes all data analysis, the test procedures and test results are transmitted to Rocketdyne at Canoga Park for review. Upon completion of this review, the LCN Verification Complete Block is signed by the Software, Engine Systems, and SQA personnel. Complete LCN packages are provided to the SSMEC software community. Rocketdyne at Canoga Park prepares a Hotfire Simulation Request Package that specifies the software configuration, test profile, and special tests, as required. These tests are performed in the HSL. In addition, a Data Base Compare is performed on the software that is to be used for engine hot fire test. Upon completion of these tests and approval by MSFC, the software is authorized for use at SSC for engine hotfire test. Engine hotfire tests certify the SSMEC software.

Upon completion of the software certification and approval of the ECP and the associated Verification Complete Package (VCP), the software is then acceptable for use for STS flight. A SSMEC software delivery, with the appropriate OAD and LCNs incorporated, is prepared for the STS flight. The software configuration is verified by Rocketdyne and MSFC to be the configuration required by the Field Engineering Change (FEC). This SSMEC software delivery, including the FEC and Software Authorization Notice (SAN) is authorized by MSFC for specified functions: check-out, Flight Readiness Firing (FRF), or flight. Updates to a software delivery are made when changes to the OAD are required or when new LCNs are approved. An updated SAN, and FEC if required, is provided with the update to the software delivery.

Embedded V & V: Rocketdyne Engine Systems and software personnel review the test procedures and results prior to final approval of the LCN. The test procedures are reviewed by Honeywell and SQA. Rocketdyne reviews the results of the Hotfire Simulation and Data Base Compares prior to approval for engine hotfire. SQA ensures that any issues are resolved. MSFC verifies that all verification is complete prior to use of the software for engine hotfire.

Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×

MSFC approves all logic changes for flight via an Engineering Change Proposal (ECP) and all changes for a specific STS flight by FEC. MSFC approval is documented by a Configuration Change Board Directive (CCBD). The review by Rocketdyne and MSFC assures that the software delivered for an STS flight is correct and complete and that the software meets any engine unique requirements.

3.4 SSMEC SOFTWARE MISSION READINESS ROADMAP. JSC SASCB receives the LCN package from MSFC. A baseline Change Request (CR) is prepared by JSC personnel citing the LCN package for incorporation into the FSW. The SASCB then reviews and provides technical concurrence of the LCN in the SASCB meeting minutes. A load for each mission is received approximately 6 months prior to a scheduled mission. The SASCB does not approve LCN packages, however, as the SSPO, they concur that the LCN package is technically required, and acceptable for use in the FSW. SSMEC software is delivered to the STSOC IMMU load build process. As OAD and LCN updates are received, these changes are loaded into each FSW MMU build, as required, to maintain a load for all SSME operations. The SSMEC software received from MSFC is supplied to the STSOC load build activity for inclusion into the integrated mass memory load build. Periodic updates may be received in the form of OAD changes or LCN changes for specific SSME or mission upgrade. When MSFC delivers SSMEC changes, the appropriate SSME configuration is also provided. The SSME configuration is used both to configure the FSW, and establish test conditions in the SAIL, if appropriate. Once the SSMEC software is integrated into a MMU load, it is included in SAIL avionics integration testing and is considered at all STS mission FRR/SRRs. If a SSME capability has been modified, or expected operational environment has changed, the test environment (JSC tools such as SPF, SAIL, SMS SSME hardware and/or performance simulation models) may have to be modified.

Operational testing is defined as the operational use of the SSMEC FSW during mission preparation testing in the SAIL. Operational testing is a real-time operation using flight equivalent and simulated flight hardware, as well as a full complement of flight computers. The SAIL provides a flight crew interface. Operational avionic system hardware/software integration test scenarios and mission scenarios are performed at SAIL. Problems found during testing are recorded in DRs and submitted to the appropriate organizations for analysis or resolution.

KSC builds a SSMEC software load compare tape, using the memory configuration that is specified by MSFC SAN, that consists of a file for each memory configuration for each SSMEC. The SSMEC compare tape is transmitted from KSC to MSFC HSL where a bit-by-bit comparison is made to the originating database produced the SSMEC software. The compare tape is used by KSC to verify that the SSMEC software was correctly loaded into the SSMEC.

The Software Readiness Review (SRR) is held approximately 3 weeks prior to flight. The SRR is conducted by NASA to allow all members of the FSW community to review FSW open issues relating to the software 's ability to perform the planned mission. The results of SAIL, Level 8 and certification testing are reviewed, as well as any software issues encountered during operations.

The Flight Readiness Review (FRR) is held approximately 2 weeks prior to flight, with a follow up FRR held approximately 2 days prior to flight to resolve any remaining issues that may effect the planned mission. The FRR is held by the SSPO to allow all members of the STS community to review and disposition open

Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×

STS hardware and software issues related to the planned mission. All aspects of flight vehicle preparation are reviewed and flight or mission related concerns recorded and dispositioned.

Embedded V & V: The software builds are validated through bit by bit tape comparisons. SSMEC software is included in the IMMU loads, and exercised in the SAIL. When changes are made to test tools, the simulated hardware and/or performance operational data is verified against the real world. Two groups provide independent sets of software mission performance tests - STSOC performs an approved set of validation tests while IBM, in parallel, performs a separate set of certification tests on the flight load. Tests in the SAIL are avionics integration tests performed under sponsors from the FSW community. Specific tests are not performed in the SMS, however, Flight Crew training usually exercises the full range of missions operations, and a subset of off-nominal operations which have the potential of occurring during the mission. Any discrepancies encountered during SMS training or SAIL testing, are documented in DRs.

The comparison of the compare tape bit-by-bit to the SSMEC software at MSFC verifies that the compare tape reflects the SSMEC software configuration authorized by the SSMEC SAN. The use of the compare tape to verify the SSMEC software load verifies that the SSMEC was loaded correctly.

Each contractor or NASA organization having a role in preparation for the flight and mission is required to certify that preparations are completed and that to the best of their knowledge there are no known problems that affect the safety of the flight or completion of the STS mission.

Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
THIS PAGE INTENTIONALLY LEFT BLANK
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×

Figure A-1 SSMEC Software Requirements Definition Roadmap

Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×

Figure A-2 SSMEC Software Development Roadmap

Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×

Figure A-3 SSMEC Software Verification/Validation/Certification Roadmap

Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×

Figure A-4 SSMEC Software Mission Readiness Roadmap

Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×

APPENDIX B

ACRONYMS AND ABBREVIATIONS

Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
THIS PAGE INTENTIONALLY LEFT BLANK
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×

APPENDIX B

ACRONYMS AND ABBREVIATIONS

BFS

Backup Flight System

CCB

Change Control Board

CCBD

Configuration Control Board Directive

CDR

Critical Design Review

CI

Configuration Inspection

CPU

Central Processing Unit

CR

Change Request

CTR

Certification Test Report

DCR

Data Change Requests

DDS

Detailed Design Specification

DPS

Data Processing System

DR

Discrepancy Reports

ED

Engineering Directive

EPDC

Electrical Power Distribution and Control

ET

External Tank

FACI

First Article Configuration Inspection

FDSD

Flight Data Systems Division

FRR

Flight Readiness Review

FSW

Flight Software

GFE

Government Furnished Equipment

GLS

Ground Launch Sequencer

GN&C

Guidance, Navigation and Control

GPC

General Purpose Computer

HSL

Hardware Simulation Laboratory

HSLII

Hardware Simulation Lab II

IBCB

Integrated Baseline Control Board

IMMU

Integrated Mass Memory Unit

KCR

KSC Change Request

LCC

Launch Control Center

LPS

Launch Processing System

Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×

MAST

Measurement and Stimulus

MCC

Mission Control Center

MMU

Mass Memory Unit

MOD

Mission Operations Directorate

OI

Operational Increment

OMRS

Orbiter Maintenance Requirements Specification

OPS

Operations

PASS

Primary Avionics Software System

PRCB

Program Requirements Control Board

PRCBD

Program Requirements Control Board Directive

PTR

Performance Test Reviews

RCN

Requirements Change Notice

RMD

Reconfiguration Management Division

RSS

Range Safety System

SAIL

Shuttle Avionics Integration Laboratory

SASCB

Shuttle Avionics Software Control Board

SASR

Shuttle Avionics Systems Review

SCCB

Software Change Control Board

SCR

Software Change Request

SDF

Software Development Facility

SES

Shuttle Engineering Simulation

SMS

Shuttle Mission Simulator

SPF

Software Production Facility

SQA

Software Quality Assurance

SRB

Solid Rocket Booster

SRG

Software Review Group

SRM&QA

Safety, Reliability, Maintainability & Quality Assurance

SRR

Software Readiness Review

SSC

Stennis Space Center

SSMEC

Space Shuttle Main Engine Controller

SSP

Space Shuttle Program

SSPO

Space Shuttle Program Office

STAR

Shuttle Transportation Automated Reconfiguration

STS

Shuttle Transportation System

STSOC

Shuttle Transportation System Operations Contractor

SVP

Software Verification Procedure

TCT

Test Coordination Team

TCTI

Time Compliance Technical Instruction

TDCC

Technical Directive Change Control

TRP

Technical Review Panel

Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×

V&V

Verification and Validation

VCS

Vehicle Cargo System

VTP

Verification Test Program

Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
THIS PAGE INTENTIONALLY LEFT BLANK
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 139
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 140
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 141
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 142
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 143
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 144
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 145
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 146
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 147
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 148
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 149
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 150
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 151
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 152
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 153
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 154
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 155
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 156
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 157
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 158
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 159
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 160
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 161
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 162
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 163
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 164
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 165
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 166
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 167
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 168
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 169
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 170
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 171
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 172
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 173
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 174
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 175
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 176
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 177
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 178
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 179
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 180
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 181
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 182
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 183
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 184
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 185
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 186
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 187
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 188
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 189
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 190
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 191
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 192
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 193
Suggested Citation:"Appendix E: Flight Software Verification and Validation Requirements." National Research Council. 1993. An Assessment of Space Shuttle Flight Software Development Processes. Washington, DC: The National Academies Press. doi: 10.17226/2222.
×
Page 194
An Assessment of Space Shuttle Flight Software Development Processes Get This Book
×
Buy Paperback | $45.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Effective software is essential to the success and safety of the Space Shuttle, including its crew and its payloads. The on-board software continually monitors and controls critical systems throughout a Space Shuttle flight. At NASA's request, the committee convened to review the agency's flight software development processes and to recommend a number of ways those processes could be improved.

This book, the result of the committee's study, evaluates the safety, oversight, and management functions that are implemented currently in the Space Shuttle program to ensure that the software is of the highest quality possible. Numerous recommendations are made regarding safety and management procedures, and a rationale is offered for continuing the Independent Verification and Validation effort that was instituted after the Challenger Accident.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!