As part of the national effort to improve aviation safety, the Federal Aviation Administration (FAA) chartered the National Research Council to examine and recommend improvements in the aircraft certification process currently used by the FAA, manufacturers, and operators. The Committee on Aircraft Certification Safety Management was formed to execute this task, which included the following key elements:
define an improved approach for managing risk and promoting the safety of U.S. civil aircraft, with a focus on the continued airworthiness of large transport airplanes
identify barriers to implementing the recommended approach and how they might be overcome
discuss the special needs of general aviation and rotor-craft
The safety of major U.S. airlines is unmatched by comparable modes of public transportation. The effectiveness of the aircraft certification process is an important factor contributing to this successful record. Major organizations involved in the aircraft certification process are the FAA, manufacturers, and operators. The aircraft certification process encompasses three primary elements:
rulemaking and policy development: defining and implementing new and modified regulations and associated policy guidelines for use by the FAA and industry
certification: issuing new and amended type certificates, production certificates, and airworthiness certificates for new and modified aircraft, engines, and other equipment
continued airworthiness and other activities related to continued operational safety: verifying the ongoing safety of products manufactured in accordance with approved designs by monitoring existing aircraft
The Aircraft Certification Service (AIR)1 is the department within the FAA that has lead responsibility for carrying out these actions. Specific functions include issuing initial and amended type certificates for designs of new and derivative aircraft, supplemental type certificates (STCs) for designs of modifications to existing aircraft, production certifications to certify a manufacturer's ability to build an aircraft in accordance with an approved design, airworthiness certificates for individual aircraft verifying that they have been manufactured in accordance with approved designs, and air-worthiness directives to correct unsafe conditions in existing aircraft.
Federal Aviation Regulations generally do not stipulate how certification standards should be met because design processes typically do not lead to a single ''best'' solution to meet a given set of certification standards. Writing effective regulations that focus on the characteristics of systems and aircraft instead of on specific design procedures can be difficult unless current technology, and how it is likely to evolve in the future, is well understood.
The most important function of aviation safety management is to prevent accidents. As shown in Figure ES-l, the "primary" causes of most accidents are associated with human error. For example, controlled-flight-into-terrain and loss-of-control-in-flight accidents, which almost by definition involve human factors, account for more than half of all fatal accidents. Aircraft system malfunctions, on the other hand, are involved in a relatively small fraction of aircraft incidents and accidents. However, it is also true that most accidents are caused by a chain of events, any one of which could have prevented the accident. This provides multiple opportunities to improve safety. By addressing individual factors in the chain of events, the accident rate can be reduced.
The Major Finding. The recommended safety management process should improve the ability of the FAA/AIR, manufacturers, and operators to take corrective action based on incident data—before an accident takes place—and to set priorities based on assessments of current and future risk. However, the current process is already highly effective— as indicated by the small contribution of aircraft system malfunctions to the overall accident rate—and changes to the current system must be carefully structured to avoid unintended consequences that might reduce safety in some situations.
Major Recommendation 1. It is critically important that the FAA and AIR conduct business in a new fashion with regard to aircraft certification and continued airworthiness. As an essential first step, AIR should revise its budget and manpower allocations to better reflect its mission priorities, which are as follows:
continued airworthiness and other activities related to continued operational safety
rulemaking and policy development
The vast majority of aircraft that will operate during the next 10 years have either already been manufactured or will be manufactured to already certificated design specifications. Monitoring the safety of operating aircraft is essential to obtain a true picture of safety, to detect and resolve problems as soon as possible, and to validate airworthiness standards. Improvements in standards for initial type certification are typically based on lessons learned from the continued airworthiness process. Therefore, making the continued airworthiness process more effective is essential to improving safety in the near term and providing the foundation for long-term improvements. The primacy of this task is acknowledged in the FAA's stated priorities. Currently, however, AIR's type certification activities receive more resources than the other two areas combined.
SAFETY MANAGEMENT PROCESS
Major Recommendation 2. It is essential that the FAA improve its safety management process. The FAA should work with the operators and manufacturers of large transport airplanes and engines to define and implement a proactive process that includes the following elements and tasks:
Manufacturers, with the advice and consent of operators and the FAA, should define data requirements and processes for sharing data. Comprehensive flight operations quality assurance systems similar to the British Airways Safety Information System (BASIS) should be used as a starting point.
Operators should provide required data, as agreed upon.
Manufacturers should solicit data from additional sources, such as the National Transportation Safety Board, International Civil Aviation Organization, and National Aeronautics and Space Administration, to augment the operational database.
Manufacturers, with oversight from the FAA and the assistance of operators, as required, should collect, organize, and analyze data to identify potential safety problems.
Manufacturers should recommend corrective action for potential safety problems and seek consensus by operators. The FAA should make sure that actions proposed by manufacturers and operators will be effective, making regulatory changes and mandating compliance, as appropriate.
Manufacturers and operators, with oversight from the FAA, should monitor the effectiveness and timeliness of corrective action and the safety management process (see Figure ES-2).
The thrust of this recommendation is that industry should collect, organize, and analyze safety data and take appropriate corrective action to protect the safety of the fleet. The FAA should not independently collect, organize, or analyze safety data for large transport aircraft. Instead, the FAA should oversee the entire process, providing direction, assessing the accuracy and objectivity of industry's risk analyses, and mandating corrective action, as appropriate. The overall objective is to produce a more effective safety management process that routinely monitors operations and maintenance, uses data on incidents and other abnormalities to identify potential hazards proactively, and takes corrective action before hazards cause an accident.
Many systems are currently used by industry and the FAA for generating, collecting, and storing data. Many of these data systems are not coordinated or used effectively, however, consuming scarce resources that could and should be applied to other safety-related activities. Some of the databases cannot be fully utilized because of poor data quality and difficulties in interpretation. Most existing data collection and monitoring systems have not been formulated to identify hazards that may arise from unusual combinations of factors that may not individually present a significant hazard. To establish a more proactive safety management system, a database management process is needed that focuses on accurately identifying precursors to potential accidents. Such a process would rely heavily on incident data. Currently, the FAA does not have access to detailed information about many incidents throughout the world.
The current safety management process lacks a widely accepted risk analysis system or methodology. Such a methodology is necessary to establish credible priorities for effective and timely resource allocation and action. With regard to risk management/action, the basic elements are already in place but must be enhanced to attain desired improvements in aviation safety. Currently, there is a tremendous backlog of pending regulatory actions, including hundreds of airworthiness directives issued by foreign air-worthiness authorities. There is no legal requirement for U.S. operators of the affected aircraft to implement any of these directives unless the FAA concurs that the action should be mandatory and issues an equivalent airworthiness directive. The size of the backlog could be reduced if more FAA personnel were dedicated to reviewing foreign directives. In this regard, the FAA is not following its own stated safety priorities.
A method for accurately assessing the effectiveness of the safety management process is important because remedial action can disrupt airline operations and reduce the competitive standing of operators and manufacturers. Accurate information on the effectiveness of remedial action would put the FAA in a better position to justify its own priorities and allocate resources to areas with the highest potential for improving aviation safety. A proactive safety management process would frequently recommend action in response to incidents, before an accident occurs. Demonstrating the effectiveness of safety actions will be important for building confidence in future recommendations.
Significant improvements in the current safety management process would be greatly facilitated by better cooperation among federal agencies, operators, and manufacturers. The committee believes that the overriding role of the FAA should be to provide encouragement and leadership in the United States and internationally to maximize industry
participation, to implement a more standardized global system, and to overcome the barriers that will hinder implementation of the recommended process. In parallel with efforts to make appropriate regulatory changes, the FAA should expeditiously negotiate binding letters of agreement with manufacturers and operators to implement as much of the recommended safety management process as possible.
In developing the recommended safety management process, the committee considered several possible approaches for improving aviation safety. For example, the current safety management process, which has achieved an excellent safety record, could be continued with only minor changes. Another possibility would be for a single organization to collect and analyze safety data for all types of aircraft instead of sharing this responsibility among many different organizations. The committee also considered how much of the process should be voluntary and how much should be mandated. The committee believes the recommended safety management process draws appropriately from these options and provides practical guidance for enhancing the safety of U.S. civil aviation.
APPROVED DESIGN ORGANIZATIONS
Major Recommendation 3. AIR should promote aircraft safety by certifying the competency of applicants' design organizations rather than relying on the FAA's ability to detect design deficiencies through spot checks. The FAA should work with industry and Congress to obtain legislative and regulatory authority in a timely fashion to do the following:
Certificate and rate approved design organizations (ADOs) and invest them with the responsibility for ensuring that applications for type certificates, type certificate amendments, STCs, technical standard order authorizations (TSOAs), and parts manufacturer approves (PMAs) comply with applicable airworthiness standards. ADOs would be required to have the technical capabilities necessary for competently approving designs only within the limitations of their rating.
Require ADOs and holders of production certificates to collect and analyze relevant safety data received from operators and to define corrective action in the event unsafe conditions are detected.
Require applicants for design approvals to either hold an ADO certificate or employ the services of an ADO.
As an interim step, give higher priority to the ongoing rulemaking action that would increase organizational delegation to manufacturers of large aircraft and engines under the FAA's current legislative authority. The FAA already uses this authority to grant organizational delegation to manufacturers of small aircraft and engines.
Existing legislation and regulations do not require applicants for type certificates, STCs, and other design approvals (TSOAs and PMAs) to show that they have the technical qualifications to develop a safe design or to conduct the engineering evaluations and certification tests necessary to show compliance with applicable airworthiness standards. There are no requirements for type certificate or STC applicants to establish or maintain a technical organization to monitor, evaluate, and propose corrective action in response to operator reports of safety problems for which they are responsible. The current process unrealistically assumes that spot checks by the FAA during reviews of new designs and design changes will reveal all items of noncompliance with airworthiness standards. The current process also limits the ability of the FAA to take advantage of the capabilities of certificated design organizations. The present system requires the FAA to spend considerable resources on "false starts" by applicants, particularly STC applicants, that do not have the technical qualifications to complete the engineering process required for design approval.
The committee believes that safety would be enhanced if the FAA focused its design approval process on determining that applicants' design organizations are technically qualified and have internal review processes that ensure compliance with the applicable airworthiness standards, rather than continuing to rely on its own ability to determine compliance through spot checks of the applicant's analyses and tests. The FAA should examine the technical qualifications and integrity of design organizations, including their understanding of regulations and policies and their ability to properly implement them. Qualified organizations should then be certificated as ADOs, allowing them to make detailed findings of compliance in accordance with published policies. FAA audits would verify continued compliance, in part by ensuring that ADOs' level of involvement in specific projects is appropriate in light of the technical issues involved.
Establishing a system of ADOs would reduce FAA resources required to conduct its certification functions, making additional resources available for continued airworthiness activities as recommended by Major Recommendation 1.
The FAA is working with industry to develop regulatory changes that would delegate additional certification functions to industry. The committee urges the FAA to continue working in this direction as an interim step toward the certification of applicants' design organizations. However, a more comprehensive restructuring of the process is needed to implement the ADO concept envisioned by the committee. This restructuring would require legislative authorization in the form of changes to Title 49 of the U.S. Code.
Major Recommendation 4. The FAA should support and accelerate efforts (1) to define the minimum data required by the flight crew to maintain adequate situational awareness during all phases of flight and reasonable emergency scenarios and (2) to determine how this data can be presented most effectively.
Human factors issues, specifically human errors, are significant contributors to most incidents and accidents. Improving the situational awareness of flight crews and air traffic controllers and improving the effectiveness of maintenance personnel are essential for preventing most serious incidents and accidents associated with human error. Thus, it is important that the FAA harness the increasing body of human factors knowledge being developed by other organizations. That is one of the tasks of the FAA's Human Factors Study Group. The FAA should ensure that this group includes strong representation in the fields of cognitive science and basic neuroscience so that it can form a cohesive framework for understanding the very large number of human factors studies that are now being conducted, especially with regard to cockpit design. Training is another important tool for reducing many types of human error. However, training was outside the scope of this study and, thus, it is not addressed in the report.
Major Recommendation 5. In order for AIR to contribute as much as possible to improvements in aviation safety, the FAA—in partnership with industry, Congress, the Department of Transportation, and other involved parties—must take aggressive action to overcome barriers associated with the following:
external pressures and influences faced by the FAA
coordination and communications within the FAA
the rulemaking process
the economic impact of proposed changes to the safety management process
There are a number of barriers, both internal and external, that will make it difficult to implement the recommended safety management process. The FAA must overcome these barriers to achieve current national goals for improving aviation safety over the next 10 years.
External pressures and influences faced by the FAA. Highly publicized accidents are often caused by factors not associated with the greatest aviation hazards. Political and public pressure to "solve" these highly publicized accidents can divert attention, personnel, and funds from efforts to address more significant risk factors. Crisis management and the impetus to take quick action can also result in action that is less effective in the long run than taking the time to develop a more effective initial response. As a first step toward reducing the negative impact of external pressure on the safety management process, the FAA should work with other responsible agencies to educate the public better about ongoing efforts to improve aviation safety. However, the committee believes that fully addressing this issue is likely to require major organizational changes, such as establishing a senior interagency communications or safety management board, that were beyond the scope of this study.
Coordination and communications within the FAA. Many of the organizational elements in the FAA enjoy considerable autonomy over their assigned areas of responsibility and lack an effective means of communicating and resolving differences. More effective communications—within AIR and between AIR and other FAA offices, such as the Flight Standards Service—would considerably improve the aircraft certification safety management process by facilitating the exchange of information within the FAA and the dissemination of complete and consistent information to industry.
Legal issues. Legal issues are associated with the potential for public disclosure of sensitive information under the Freedom of Information Act, the possibility of regulatory enforcement against individuals or companies who voluntarily disclose information about safety problems, and increased exposure to legal liability arising from the litigation discovery process in an environment where more data is collected, stored, and shared. The FAA has significant leeway in addressing the first two problems by modifying internal policies and the Federal Aviation Regulations. Legal discovery issues are likely to remain a significant problem unless Congress enacts legislation to protect voluntarily shared data from the threat of discovery action directed toward parties with whom such data are shared.
Rulemaking process. The FAA's rulemaking process is defined by internal policies and, more importantly, by legislation and executive branch regulations the FAA cannot waive unilaterally. As currently implemented, the rule-making process quite often takes 5 to 10 years. Although timely action is possible, especially in the case of highly publicized or emergency safety actions, many worthwhile, safety-related activities linger without action for unreasonably long periods of time. These delays are a significant safety issue. During 1998, the FAA plans to implement recommendations from an internal study on how to improve the efficiency of the FAA rulemaking process. This is a positive first step, but much more needs to be done. The Department of Transportation, other executive branch agencies, and Congress should also work with the FAA to modify legislation, directives, and regulations, as necessary, to substantially improve the responsiveness of the rulemaking process.
Economic impact of proposed changes to the safety management process . The air transport industry is highly competitive, and this natural competitiveness is a potential barrier to the voluntary sharing of data required to implement the recommended safety management process. Manufacturers and operators bear the cost of making safety improvements, and their support will be forthcoming only to the extent that the identified risks are credible and the corrective action seems reasonable in terms of effectiveness and cost. The FAA should work with industry to develop confidence in the cost/benefit analyses used to justify changes in the safety management process. The FAA should also subsidize
pilot projects by operators and manufacturers to validate the cost effectiveness of new systems for data collection, database management, and analysis.
SMALL AIRPLANES AND ROTORCRAFT
Major Recommendation 6. Plans to implement the recommended safety management process within the small airplane and rotorcraft communities should be developed in cooperation with small airplane and rotorcraft operators, manufacturers, and associations of operators and manufacturers. The FAA should establish cooperative agreements that define the roles of individual operators, individual manufacturers, their associations, and AIR. These agreements should define the following:
responsibilities of operators for submitting data
responsibilities of operators, manufacturers, associations of operators and manufacturers, and AIR for data collection, database management, risk analysis, risk management/action, and monitoring effectiveness
processes for the routine exchange of data and risk analysis results between operators, manufacturers, associations, and AIR to facilitate effective risk management/action
a publicity program to inform the small airplane and rotorcraft communities of the new safety management process
There are many differences between large transport airplanes and small airplanes. The number of small aircraft registered in the United States and the number of pilots licensed to fly them far exceeds the numbers of large transport airplanes and pilots. On average, large transport airplanes and pilots are in the air for many more hours per year than small airplanes and pilots. The pilots of small airplanes have a much wider range of experience and skills and operate in a much broader spectrum of functional modes than the pilots of transport airplanes. Small airplanes and rotorcraft are used for recreation, sightseeing, pipeline patrols, scientific experimentation, crop dusting, and firefighting.
Thousands of civil rotorcraft are registered in the United States, most of them operating as general aviation aircraft. Rotorcraft operating characteristics and some of the missions they undertake, such as logging, law enforcement, and emergency rescue, create a risk environment that is quite different from the risk environment of most fixed wing airplanes. Rotorcraft and small airplanes also operate out of many more airports and landing areas than large airplanes, and many of these airports do not have control towers or other landing and takeoff aids.
The safety management process for small airplanes and rotorcraft must be flexible enough to accommodate the diverse nature of these communities, and this is likely to be a difficult challenge. Final accident investigation reports for small airplanes and rotorcraft show that the majority of accidents are attributable to human error, and the small role played by aircraft system malfunctions indicates that the current aircraft certification and continued airworthiness process is working well.
Boeing. 1997. Primary Cause Factors for Hull Loss Accidents Involving Large U.S. Registered Commercial Jet Airplanes. Seattle, Wash.: Boeing Commercial Airplane Group.