On January 18, 2018, in New South Wales, Australia, a drone—otherwise referred to as an unmanned aircraft system (UAS)1—was used to save two swimmers who had been caught in rough ocean surf. Australian lifeguards operating the drone were not even using it in an operational capacity that day. A lifeguard supervisor was practicing with the drone, which was designed for spotting sharks, when he spotted the swimmers in trouble and used it to drop an inflatable device to them. Normally, lifeguards have to swim out to people to make a rescue, endangering their own life and increasing the time to bring aid to people in a dangerous situation.
This recent incident, which occurred while this report was being written and which received extensive coverage in the news media, highlights the potential value of this still-emerging technology to reduce risk and save lives. There are numerous other examples of UAS that can be used in various applications to reduce risk to civilian populations. From long-range inspection of rail lines to prevent derailments, to inspection of power lines and cell phone towers, to delivery of medicine and automated external defibrillators to people in cardiac distress, to assessment of wildfires to assist firefighters, the full value of UAS has yet to be realized. What these various examples illustrate is that when discussing the risk of introducing drones into the National Airspace System, it is necessary to consider the increase in risk to people in manned aircraft and on the ground, as well as the various ways in which this new technology may reduce risk and save lives, sometimes in ways that cannot readily be accounted for with current safety assessment processes.
The Committee on Assessing the Risks of Unmanned Aircraft Systems (UAS) Integration examined the various ways that risk can be defined and applied to integrating UAS into the National Airspace System managed by the Federal Aviation Administration (FAA). The committee looked at recent developments in this field and consulted numerous experts in academia, industry, and government.
The committee has drawn the following key conclusions, listed alphabetically:
- Consider the de minimis risk. With regard to the risk that an aircraft accident poses to people on the ground, the public already accepts a background level of risk that is extraordinarily low. The public also accepts the higher level of risk that the crew and passengers of general aviation aircraft currently face, likely because the vast majority of the public does not fly in general aviation aircraft and has no intention of doing so. The
1 “Unmanned aircraft system” (UAS) is a more encompassing term that refers to the aircraft, the control system, and the system for communicating between them. Technically speaking, a drone or an unmanned aircraft is only one part of a UAS. Throughout this report, the committee primarily uses the term UAS.
public also accepts that medical evacuation helicopters face a risk that is higher still. The level of acceptable de minimis risks varies widely for other societal activities such as traveling by car or motorcycle, swimming in the ocean, or walking across the street. Understanding the level of de minimis risk that the public is likely to accept for small UAS operations, in the context of levels of de minimis risk for other levels of societal activities, would be useful in establishing safety standards for small UAS operations.
- Consider the safety benefits. Some UAS operations will increase safety both inside and outside the aviation system. These safety benefits could be considered as UAS operations are considered for approval.
- Delegate responsibility. Where it can be demonstrated that the risk is low enough and can be mitigated in this manner, the FAA could delegate to the UAS industry responsibility for quantitative risk assessment activities for UAS operations or it could require the UAS industry to obtain insurance for UAS operations in lieu of having a separate risk analysis.
- One size does not fit all. The level of FAA scrutiny for approval of a UAS operation needs to match the level of potential risk.
- Philosophy is not reflected in the practice. FAA executives speak about the importance of taking a performance- and risk-based approach for approval of UAS operations, with streamlining where appropriate. However, the committee heard both from within the FAA and from the UAS industry that such an approach is not being reflected in actual approvals of UAS operations.
- Promote the systematic collection and analysis of empirical data. Such collection and analysis is needed to inform the evolution of quantitative risk assessment for UAS operations.
- The FAA Safety Management System (SMS) process as applied to approval of UAS operations is highly subjective. Because of its qualitative nature as applied to UAS operations, the SMS process is not repeatable and not predictable. Quantitative risk assessment techniques are needed.
Consistent with these key conclusions, the committee developed 11 recommendations that are presented in this report.
The committee concluded that “fear of making a mistake” drives a risk culture at the FAA that is too often overly conservative, particularly with regard to UAS technologies, which do not pose a direct threat to human life in the same way as technologies used in manned aircraft. An overly conservative attitude can take many forms. For example, FAA risk avoidance behavior is often rewarded, even when it is excessively risk averse, and rewarded behavior is repeated behavior. Balanced risk decisions can be discounted, and FAA staff may conclude that allowing new risk could endanger their careers even when that risk is so minimal that it does not exceed established safety standards.
The committee concluded that a better measure for the FAA to apply is to ask the question, “Can we make UAS as safe as other background risks that people experience daily?” As the committee notes, we do not ground airplanes because birds fly in the airspace, although we know birds can and do bring down aircraft.
The safety of the National Airspace System has been achieved in large part as a result of the FAA’s risk decision process, which has been characterized by a culture with a near-zero tolerance for risk. Applying this same culture to safety risk management (SRM) processes for UAS, however, has too often resulted in overly conservative risk assessments that have prevented safety-beneficial operations from entering the airspace. In many cases, the focus has been on “What might go wrong?” instead of a holistic risk picture: “What is the net risk/benefit?” Closely related to this is what the committee considers to be paralysis wherein ever more data are often requested to address every element of uncertainty in a new technology. Flight experience cannot be gained to generate these data due to overconservatism that limits approval of these flights. Ultimately, the status quo is seen as safe. There is too little recognition that new technologies brought into the airspace by UAS could improve the safety of manned aircraft operations or may mitigate, if not eliminate, some nonaviation risks.
Recommendation: The FAA should meet requests for certifications or operations approvals with an initial response of “How can we approve this?” Where the FAA employs internal boards of executives throughout the agency to provide input on decisions, final responsibility and authority and accountability for the decision should rest with the executive overseeing such boards. A time limit should be placed on responses from
each member of the board, and any “No” vote should be accompanied with a clearly articulated rationale and suggestion for how that “No” vote could be made a “Yes.” (Chapter 3)
Due to the lack of empirical data in this nascent industry, the current FAA approaches to risk management are based on fundamentally qualitative and subjective risk analysis. These subjective approaches require a depth and breadth of subject matter expertise for the approval process that the FAA generally does not possess for UAS operations. The qualitative nature of the current approach leads to results that fail to be repeatable, predictable, and transparent. Evolution to an approach more reliant on applicant expertise and investment in risk analysis, modeling, and engineering assessment, as is practiced in many other areas of federal regulation, might better achieve a quantitative probabilistic risk analysis basis for decisions.
Traditionally in manned aviation, assessments of risk have focused on probability of passenger fatality. This measure clearly does not correspond well to UAS operations. Further, given the substantial variety of types of UAS operations, no single measure of risk can likely be found that can adequately characterize the benefit and risk of all UAS operations. Concerns by the drone industry of overly stringent certification requirements for relatively low-risk operations place unnecessary burden on the business case and can stifle innovation.
Recommendation: The FAA should expand its perspective on a quantitative risk assessment to look more holistically at the total safety risk. Safety benefits, including those outside of aviation (e.g., the benefit of cell tower inspections without a human climbing a cell tower), should be part of the equation. UAS operations should be allowed if they decrease safety risks in society—even if they introduce new aviation safety risks—as long as they result in a net reduction in total safety risk. (Chapter 4)
Recommendation: Within the next 12 months, the FAA should establish and publish specific guidelines for implementing a predictable, repeatable, quantitative, risk-based process for certifying UAS systems and aircraft and granting operations approval. These guidelines should interpret the Safety Risk Management Policy process described in Order 8040.4B (and in accordance with International Civil Aviation Organization Doc. 9859) in the unique context of UAS. This should include the following: (1) Provide, within 18-24 months, risk-based quantitative performance standards that can serve to establish compliance with FAA rules and regulations. (2) In the interim, encourage applicants to provide quantitative probabilistic risk analyses (PRAs) to demonstrate that their operation achieves the requisite level of safety. (3) Within 18-36 months, update FAA rules to reference new performance standards with the goal of minimizing the need to grant waivers or Certificates of Authorization (COAs). (Chapter 4)
Recommendation: Where operational data are insufficient to credibly estimate likelihood and severity components of risk, the FAA should use a comparative risk analysis approach to compare proposed UAS operations to comparable existing or de minimis levels of risk. The FAA should research and publish applicable quantitative levels of acceptable risk in comparison to other societal activities that pose de minimis risk to people. Risk level and risk mitigation strategies should consider not only aircraft collisions but also third-party risks (e.g., to people on the ground). (Chapter 4)
Recommendation: Over the next 5 years, the FAA should evolve away from subjectivities present in portions of the Order 8040.4B process for UAS to a probabilistic risk analysis (PRA) process based on acceptable safety risk. In the interim, the FAA should improve the 8040.4B process to conform better with quantitative PRA practice. For the new acceptable risk process, the FAA should consider relying on the applicant to provide a PRA demonstrating the achieved level of safety, as is common in other regulatory sectors such as nuclear, dam, or drug safety.
- The FAA should screen applicant PRAs by comparison to existing or de minimis levels of risk. The FAA needs to research applicable quantitative levels of acceptable risk in comparison to other societal activities in establishing a level of de minimis risk for aviation.
- These acceptable levels of risk need to include risk to people on the ground and risk of collisions with a manned aircraft, particularly with regard to collision with a large commercial transport.
- In evaluating applicant-generated PRA, the FAA should value the importance of risk mitigation opportunities and their potential for simplifying the analysis of risk.
- In situations where the risk is low enough, the FAA should encourage applicants to obtain insurance for UAS operations in lieu of having a separate risk analysis. (Chapter 4)
Recommendation: The FAA should create the following two mechanisms that empower and reward safety risk management decisions that consider the broad charter of the Department of Transportation to “serve the United States by ensuring a fast, safe, efficient, accessible and convenient transportation system that meets our vital national interests and enhances the quality of life of the American people, today and into the future” (DOT, 2018):
- The FAA administrator should establish an incentive system that measures, promotes, and rewards individuals who support balanced comparative risk assessments.
- Within the next 6 months, the FAA administrator should publicly commit to ensuring time-bound reviews of risk assessments so that proponents receive timely feedback. (Chapter 4)
Recommendation: Within 6 months, the FAA should undertake a top-to-bottom change management process aimed at moving smartly to a risk-based decision-making organization with clearly defined lines of authority, responsibility, and accountability. To that end, the FAA should establish and maintain technical training programs to ensure that agency risk decision professionals can fully comprehend the assumptions and limitations of the probabilistic risk analysis techniques appropriate to current and future UAS operations. (Chapter 4)
Recommendation: The FAA should identify classes of operations where the level of additional risk is expected to be so low that it is appropriate to base approval of those operations on requiring insurance in lieu of having a separate risk analysis. (Chapter 4)
More empirical data are needed to support probabilistic risk analyses for UAS collision modeling. Rapid advances in autonomous vehicle technology are providing effective integration of sensors and analytics. These developments present an opportunity for the FAA to learn and test new models for better data collection and analysis with the aim of improving overall safety. Even so, it may be difficult to collect enough data to assess some risks that have a very low probability of occurrence. In those cases, it could be useful to draw upon research being conducted for other applications that is exploring how to use limited data in combination with simulations to draw conclusions about safety.
Accepting risk is far easier when the risk is well quantified by relevant data. Uncertain risk does not equate to high risk, however. By accepting the uncertain risk associated with a new technology, with reasonable mitigations, one can obtain the data needed to better quantify that risk. As the uncertainty diminishes, one can remove or augment the mitigations as appropriate. In the current environment, uncertain risk has made operational approvals for routine civil UAS operations difficult to obtain and, when issued, unnecessarily restrictive. As a result, the ability to collect data that might reduce uncertainty in the risk has been severely limited.
Recommendation: The FAA should, within 6 months, collaborate with industry to define a minimum operational safety data set and develop a plan for the voluntary collection and retention of data by the operators in a central repository, following the model of the Commercial Aviation Safety Team (CAST) and the General Aviation Joint Steering Committee (GAJSC), with a goal of full implementation within 1 year. The FAA should also consult with the Drone Advisory Committee to help define the minimum operational safety data set and plan for collecting, archiving, and disseminating the data. (Chapter 4)
Recommendation: For operations approvals for which there are no standards, as operational data are collected and analyzed, the FAA should, as part of Improved Safety Risk Management,
- Publish requirements for operational approvals with associated restrictions that can be adjusted and scaled based on industry past experience and the accumulation of related data;
- Expand single operation approvals as experiential data accumulate and risks are assessed;
- Permit repeated or routine operations based on the accumulation and analysis of additional data; and
- Continuously update operational approval practices to incorporate emerging safety enhancements based on industry lessons learned until standards have been established. (Chapter 4)
Increased levels of autonomy have the potential to improve the operational safety of UAS. However, it cannot currently be guaranteed that such a nondeterministic learning system would respond safely in every conceivable situation. For this reason, true autonomy, as opposed to well-defined automatic operation in well-defined circumstances, is not currently allowed for commercial UAS flying within the National Airspace System. Opportunities to increase the safety of UAS operations, and of aviation in general, through increased autonomy are being missed, however, due to a lack of accepted risk assessment methods.
Recommendation: In coordination with other domestic and international agencies, the FAA should pursue a planned research program in probabilistic risk analysis (PRA), including the aspect of comparative risk, so that FAA personnel can interpret or apply PRA for proposed technology innovations. (Chapter 5)
During the course of its deliberations, the committee heard from a variety of experts from academia, other government agencies (e.g., the U.S. Navy), and even other international civil aviation authorities such as the German Aerospace Center (DLR). The committee consulted with industry groups such as Google, Boeing, Airbus, PrecisionHawk, and others, as well as a representative of the aviation insurance industry. Their input helped to inform the committee’s report and shape its findings and recommendations.
Overall, this report endorses a more holistic approach to assessing UAS integration into the airspace based directly on risk (using other factors such as size, weight, and location only as inputs to the assessment of risk, rather than as broad-brush constraints). Such a holistic approach should also account for mitigations to potential risks within the entire UAS system (including its interactions with a human operator and ground control stations) and operational factors constructed to mitigate potential risks.
The committee has concluded that the introduction of a robust set of UAS operations into the National Airspace System both is achievable and has the potential to provide significant net safety benefits to society in addition to whatever economic benefits those operations might provide. Following the recommendations in this report would accelerate and facilitate the safe integration of UAS operations into the nation’s airspace.
DOT (Department of Transportation). 2018. “Mission.” https://www.transportation.gov/tags/about-us. Accessed August 31, 2018.