Section 212 of the Federal Aviation Administration Modernization and Reform Act of 2012, Public Law 112-95, calls for an examination by the National Research Council (NRC) of the Next Generation Air Transportation System’s (NextGen’s) enterprise software development approach and safety and human factor design. In response to this request, the NRC formed the Committee to Review the Enterprise Architecture, Software Development Approach, and Safety and Human Factor Design of the Next Generation Air Transportation System to conduct this study.
The committee’s overarching conclusions are as follows: The original vision for NextGen is not what is being implemented today. Instead, NextGen today primarily emphasizes replacing and modernizing aging equipment and systems. This shift in focus has not been clear to all stakeholders. Nevertheless, modernization is critical, and large-scale, software-intensive systems such as NextGen require ongoing support for operations and maintenance.
To be successful, even as a modernization project, NextGen needs a system architecture that defines how the pieces of the system fit together and allows for modeling and reasoning about possible futures. The existing National Airspace System (NAS) enterprise architecture is not that; it primarily documents existing systems and plans. Among other things, a system architecture is an essential tool for managing risk. The Federal Aviation Administration (FAA) should create an architecture community that can produce and evolve a system architecture and should also strengthen its workforce in systems engineering and integration, digital communications, and cybersecurity to increase the likelihood it will suc-
ceed in developing the architecture and managing the implementation of the systems it describes.
NextGen and its system architecture should be developed to cope with change. Two newly important areas, cybersecurity and unmanned vehicles, make this need particularly resonant. Human factors will also play an important role in NextGen and the NAS as each evolves. Finally, regarding anticipated costs and benefits, airlines are not motivated to spend money on equipment and training for NextGen because they do not receive most of the benefits directly and because of implementation schedule uncertainties. The rest of this summary elaborates these and related observations in more detail and highlights several of the committee’s findings and recommendations in bold.
ALIGNING EXPECTATIONS FOR NEXTGEN
Throughout the committee’s work, it became clear that “NextGen” means different things to different people, ranging from a wide-ranging transformational vision to a much more concrete set of phased incremental changes to various parts of the NAS. With so many stakeholders and so many moving parts, different understandings of “what is NextGen” arose. As the committee has come to understand it, NextGen today is a set of programs to implement a suite of incremental changes to the NAS. Although some technologies and/or systems will be new, in most cases, current plans call for them to be used to closely replicate existing capabilities (such as satellite navigation used to replace radar functionality rather than the reinvention of flight).
The executive order establishing the Joint Planning and Development Office (JPDO) was entitled “Transformation of the National Air Transportation System,”1 and early vision documents referred to ambitious goals such as integrated data streams for situational awareness in seamless multi-agency global operations, scalability, the use of emerging space-based communications and surveillance technologies.2 Although progress has been made, not all parts of the original JPDO vision will be achieved in the foreseeable future. This was true even at the outset of NextGen and is understandable, given changes over time.3 However, even
1 White House, “Transformation of the National Air Transportation System,” Executive Order, released November 18, 2008, http://georgewbush-whitehouse.archives.gov/news/releases/2008/11/20081118.html.
2 See Appendix B of the 2005 National Research Council report Technology Pathways: Assessing the Integrated Plan for a Next Generation Air Transportation System (The National Academies Press, Washington, D.C.) for an overview of JPDO objectives.
3 For instance, the substantial future demand growth anticipated in early planning did not materialize.
the limited vision embraced at the start of NextGen has been reduced over time, while increasingly important challenges have not received adequate attention.4 Partly as a result of these issues, stakeholder expectations for NextGen have become misaligned with current planning as NextGen and its constituent programs have changed over time. This misalignment causes challenges both for the FAA and for stakeholders.
An important part of NextGen is addressing the need to replace aging equipment. Such modernization is essential and important. Replacing or upgrading systems while continuously and safely operating the whole system is an intricate undertaking, a process that the FAA seems to have mastered. The successful operation of such systems requires ongoing alterations and improvements, not just the occasional repair of faulty equipment and software. While not the transformation originally envisioned for NextGen, continuing to refresh the technology-driven systems is necessary for safety critical systems like the NAS.
As described to the committee, NextGen also includes efforts to further deploy performance-based navigation,5 redesign certain aspects of the airspace, and equip aircraft with technology (such as Automatic Dependent Surveillance-Broadcast (ADS-B) that can form the basis for future capabilities, along with a broad range of activities. These plans are expressed in various implementation plans, the NAS enterprise architecture, roadmaps, and calls for research, experimentation, and further incremental programs. NextGen, as currently executed, is not, however, broadly transformational. That is, it does not set out a series of planned steps toward a fundamentally transformed end-state (such as free flight, decommissioning surveillance radar stations, automating air traffic control processes with a completely digital information infrastructure, or shifting authority from ground to air). Moreover, the system has not been significantly changed to take advantage of available information and communications technologies or to enable major improvements in how the airspace can be organized and managed. Unfortunately, over the course of the committee’s work, it was clear that some stakeholders were still anticipating these capabilities from NextGen. Such goals await the now-distant full deployment of technical capabilities, the integration of these capabilities to support higher levels of automation and more distributed and autonomous operation, full equipage of virtually all aircraft with new components, and widespread revisions to procedures. “NextGen” has become a misnomer.
4 For instance, cybersecurity was not a significant concern in early JPDO planning.
5 Performance-based navigation refers to a range of approaches that emphasize the performance and capabilities of aircraft over more conventional ground-based navigation systems.
Recommendation: The Federal Aviation Administration (FAA), Congress, and all National Airspace System stakeholders should reset expectations for the Next generation Air Transportation System. The FAA should explicitly qualify the early transformational vision in a way that clearly articulates the new realities.
The committee’s conclusion that NextGen today is primarily an incremental modernization effort should not suggest that NextGen therefore has an obvious completed state or that future significant change should not happen. Given the continuing rapid pace of technological evolution and ongoing changes in what is demanded of the NAS, the NextGen effort is properly seen as an ongoing process, punctuated by particular efforts focused on particular capabilities. Resetting expectations with a clear baseline will provide a useful foundation on which to build.
ASSERTING ARCHITECTURAL LEADERSHIP
The statement of task for this study (Box 1.2) uses the term “enterprise architecture.” Like other terms associated with software-intensive systems, this term is used in different ways by different organizations and in different contexts, but typically, an enterprise architecture serves as documentation and support of existing systems and business processes. An enterprise architecture alone cannot usually manage large, distributed, real-time systems where safety-critical concerns predominate, nor is it clear that even the best instantiation of an enterprise architecture is intended for such uses. An enterprise architecture is typically interpreted as a set of documents instead of a set of decisions. This is consistent with what the committee learned in its briefings about the FAA’s approach to enterprise architecture. However, a focus on documentation over decision making is a significant problem.
For a system such as NextGen, a more comprehensive notion of architecture is needed. A system architecture, by contrast with an enterprise architecture, models and defines the structure and behavior of a system in a way that supports reasoning about the system and its characteristics. Accordingly, and consistent with other elements of its task, such as software development, the committee explored the question of architecture more broadly, focusing also on the system architecture for NextGen.
Discerning precisely what FAA’s architectural approach and strategy is was challenging, and some of it had to be inferred. The current enterprise architecture as presented to the committee appears to be a set of functional enclaves that are providing individual services, described in a set of documents at the NAS enterprise architecture level. Additional improvements and modifications seem to be either changes to what is already deployed or overlays onto what is already there.
Ultimately, the committee’s conclusion with regard to the NAS enterprise architecture is that the existing design and deployment of the NAS embodies a tacit architecture—the de facto system architecture is the system as it is today. This induced architecture is therefore bottom-up and program-driven and imposes implicit limits on what (and how) system capabilities can be realized. This has ramifications for how effective it can be, especially in laying groundwork for future flexibility and enhancements.
A tacit architecture is not appropriate for a system of NextGen’s scale and importance and is at odds with recommendations in standards such as ISO/IEC/IEEE 42010.6 FAA’s approach to architecture (which focuses on the enterprise architecture) is not an adequate technical foundation for steering NextGen’s technical governance and managing the inevitable changes in technology and operations.
The changes to the NAS envisioned under the NextGen umbrella should provoke changes and adjustments in the NAS system architecture. Change can be thought of as the ongoing management of trade-offs, which are not clearly identified in the existing tacit architecture. That tacit architecture is diffused through many different programs, not all of which are under NextGen control. A system architecture for the NAS should help ensure proper operation of the system and allow proper analyses for prediction of system behavior and performance and ensure future evolvability. Absent such an architecture, whether NextGen can meet its stated objectives and requirements is unknown and, probably, unknowable. That the system architecture is not well developed is hard to discern because of the nearly exclusive focus by the FAA on the enterprise architecture.
Unfortunately, having de facto established the existing (baseline) architecture as the architecture without a clear architectural expression of future expectations regarding change, trade-offs, and evolvability, many opportunities to use the architecture in forward-looking ways have been ruled out, and some important advances are going to be extremely challenging to accomplish (such as the ability to create persuasive and credible forecasts of change costs, technical risks, capability upgrades, and performance improvements). The committee’s recommendations take this into account and offer suggestions as to how to move forward most productively to develop better architectural approaches.
The most important thing on which the FAA should focus with respect to architecture is building a community of architecture leaders within and
6 International Organization for Standardization (ISO)/International Electrotechnical Commission (IEC)/Institute of Electrical and Electronics Engineers (IEEE), Standard ISO/IEC/IEEE 42010:2011, “Systems and Software Engineering—Architecture Description,” December 2011, http://www.iso.org/iso/catalogue_detail.htm?csnumber=50508.
outside the agency. The FAA will need to add more system architecture skill and establish a more capable architecture community. Architectural leadership should encompass multiple perspectives (including, but not limited to, the enterprise architecture) and provide diversity of thought and approach, emphasizing flexibility and evolvability, consistency and alignment, and right-sizing of architectural documentation. To be clear, the committee does not urge the premature creation of more detailed specifications and artifacts absent deeper insights and stronger analyses of risks and trade-offs. In many ways, such efforts would be counterproductive, translating into more overhead (process and documentation) and less attention, resources, and expertise focused on better design, decisions, tests, and earlier integration.
Like other federal agencies, the FAA faces challenges in implementing information technology systems and in recruiting and retaining the workforce needed for designing, maintaining, and enhancing systems such as NextGen. In particular, the FAA is ill-equipped to perform as a systems integrator. If the FAA is to succeed in both the medium and long term, it will require enhancements to its technical expertise. Architecture and systems engineering, which are needed for successful integration of capabilities and platforms into a coherent NAS system, have been undervalued. Program management and systems engineering process are important, but are not a substitute for talent that can effectively guide the design and evolution of NextGen. Even if the FAA were not acting as systems integrator, it would still need to be a “smart customer”—it needs expertise that will enable it to effectively structure and manage its supplier relationships.
Today, the FAA relies greatly on its vendors and other external talent. For architectural insight and expertise internally, it depends on a very small number of individuals and lacks the critical mass that characterizes a vibrant and effective technical community. Digital communications will take on increasing importance as the NAS is modernized, so the FAA will need additional technical expertise in designing modern digital networks and protocols. Cybersecurity is a challenge facing all who use modern computing and communications technology, and the potential threats and risks are magnified for critical infrastructure like the NAS. The FAA needs strong cybersecurity expertise in designing, implementing, integrating, and operating NextGen systems. Cybersecurity requires a system-wide approach and cannot be addressed piecemeal by each contractor (or program) separately.
Recommendation: The Federal Aviation Administration (FAA) should nurture workforce talent in the areas of systems engineering, architecture, systems integration, digital communications, and
cybersecurity. Significant effort will be required to attract, develop, and retain this talent, given the high demand outside the FAA.
Recommendation: The Federal Aviation Administration (FAA) should initiate, grow, and engage a capable architecture community—leaders and peers within and outside the FAA—who will expand the breadth and depth of expertise that is steering architectural changes.
Recommendation: The Federal Aviation Administration should conduct a small number of experiments among its system integration partners to prototype candidate solutions for establishing and managing a vibrant architectural community.
Recommendation: Should the Federal Aviation Administration continue to act as the systems integrator of Next generation Air Transportation System programs, it should maintain architectural leadership and not delegate architecture definition and control to contractors.
Recommendation: The Federal Aviation Administration should use an architecture leadership community and an effective governance approach to assure a proper balance between documents and artifacts and to provide high-level guidance and a capability that (1) enables effective management and communication of dependencies, (2) provides flexibility and evolvability to ensure accommodation of future needs, and (3) communicates changing circumstances in order to align expectations.
OPERATIONS AND MAINTENANCE
A common fallacy with software-intensive systems is that they can be built, deployed, and then operated with relatively little “maintenance” and modernization effort. The surprise, for those unfamiliar with such systems, is that operations and maintenance will very often include substantial modernization effort. This effort is needed both in response to new requirements and also in response to rapid growth and change in technological infrastructure. This is true for NextGen and the NAS, and this fact has implications for how the FAA should explain its needs to Congress and its overseers. Although Congress has been supportive of FAA efforts, in the committee’s view, there is a specific need for support of ongoing maintenance and modernization (upgrades), including refreshing and modernizing both the hardware and software so as to provide reliable, cost-effective operation. A typical pattern in government
is that funds are allocated for specific (new) programs or projects without sufficient allocation for out-year maintenance or for maintenance and refresh of existing (and still important) programs. Modernizing software is just as important to safety and operational efficiency as modernizing hardware.
Finding: As a large-scale, software-intensive system, Nextgen and the NAS will benefit if ongoing maintenance of the NAS and its hardware and software systems are supported—in addition to programmatic investments; such an approach will make the most of past and ongoing investments.
The risks and uncertainties in NextGen are inherently difficult to quantify. However, quantifying risks and value offers means for better planning and management. The challenge for complex systems such as NextGen is not how to eliminate risks but rather how to manage them successfully. In all engineering projects, and particularly software engineering projects, this usually means understanding the consequences of risky decisions as early in the life cycle as possible, lest the costs of unwinding previous bad decisions become prohibitive, and the architecture becomes a source of change friction that burdens efficiency of execution. By contrast, an effective architecture can be a basis for risk assessment and mitigation and can be used as a tool to support decision making and the recording of decisions.
NextGen today embodies a set of (often implicit) decisions to not dramatically change a wide range of current operations. Those decisions, along with an analysis of their implications, are not explicit in the tacit architecture. But a decision to not change carries heavy implications for the realization of any gains that would require such changes. To cope well with uncertainty and risk, it is important to explicitly state value attributes (with scales), to develop multiple alternative architectures, and to have evaluation models that compare those alternatives to the value attributes. The committee was struck by the lack of alternatives analysis in NextGen. Nor is there a well-specified overview of what is and is not known about the value of various proposed levels of change (e.g., partial deployment of certain technologies or features).
Schedule risks in NextGen have multiple sources, including budget, approval, certification, and procedure design. With the exception of resourcing and budgets, architecture can help mitigate these. However, a poorly developed system architecture makes it a challenge to determine how well the overall system will address system requirements (e.g., for
security and robustness), causing risks of many kinds, including schedule risks. The risks to NextGen are not clearly articulated or quantified in order of importance, making it difficult to make sound decisions about how to prioritize efforts and allocate resources.
Recommendation: The Federal Aviation Administration should use an architecture leadership community and a system architecture, with input from specialists in probability and statistics, as key tools in managing and mitigating risks and in assessing new value opportunities.
COPING WITH CHANGE
The national airspace is a critical infrastructure for the United States. In concert with revising the architectural approach for NextGen, plans to cope with unanticipated change are needed. Indeed, any architectures developed must themselves reflect planning for resilience. Cybersecurity, safety, and unmanned aircraft systems (UAS) illustrate why planning for resilience in NextGen and in the NextGen system architecture is so important. UAS were not explicitly anticipated in NextGen. And cybersecurity by its very nature demands constant adaptation to a dynamic threat environment.
NextGen is no exception to modern cybersecurity risks and threats; indeed, the safety of life implications and the vital economic importance of air travel make the security of NextGen and the NAS critically important. As various programs and components of the national airspace are modernized, the security implications of the changes will need to be taken into account. The criticality of cybersecurity for NextGen increases as more services rely on digital technologies, networked communications, and commercial-off-the-shelf software. Cybersecurity challenges extend from major software platforms into the specification and design of embedded (avionics) equipment that connects directly to the NAS. Although there will always be risk, the lack of appropriate architectural approaches to security and safety that allow for reasoning about risks, and uncertainty only increase the likelihood that risks of unknown magnitude can remain embedded in the NAS.
The committee’s impression from briefings and discussion with the FAA is that cybersecurity, although acknowledged as an issue with some efforts under way to address it, has not been fully integrated into the agency’s thinking, planning, and efforts with respect to NextGen and the NAS generally. In particular, as new technologies and procedures are rolled out, there will inevitably be new vulnerabilities. Moreover, changes in the way existing, long-stable technologies are used may intro-
duce new security issues. So threat analyses both on existing systems, with any expected changes, and on new components are needed. Cybersecurity will need to be an important and integral part of safety activities and is an ongoing operational matter (not only a question of design and architecture).
Recommendation: The Federal Aviation Administration (FAA) should incorporate cybersecurity as a systems characteristic at all levels of the architecture and design. The FAA should begin by developing a threat model followed by an appropriate set of architectural and design concepts that will mitigate the associated risks, support resilience in the face of attack or compromise, and allow for dynamic evolution to meet a changing threat environment. The FAA should inculcate a cybersecurity mindset complementary to its well-established safety mindset throughout the organization, its contractors, and leadership.
UAS are already in use as hobbyist craft, and the FAA estimates that thousands of small UAS could be active over the next 5 years. Several NextGen technologies are essential to the safe integration of UAS: the NAS voice system, which will allow UAS pilots to communicate with air traffic control over ground-to-ground communication networks; Data Communications (Data Comm), which will support the transmission of digital messages to the flight crew; and System Wide Information Management (SWIM), which will support more timely and improved information access to all users of the NAS.
However, NextGen planning and architecture did not explicitly anticipate the introduction of UAS and thus does not readily lend itself to incorporating these new types of aircraft that will place new demands on the system. The challenge of integrating UAS into the national airspace illustrates the challenges of accommodating changing requirements within the current approach to managing architectural and system evolution. The expected integration of UAS into the NAS will present new safety issues stemming from increased reliance on data links, limited operator sensory and environmental cues, and so on. An insufficiently developed system architecture is one of several obstacles to introducing UAS into the NAS. The integration of UAS is an example of a rapidly emerging requirement that could provoke disruptive changes to both technology and to roles and responsibilities.
Per its charge, the committee’s focus has been on architecture, especially system architecture. The committee urges the FAA to use UAS as a use case for developing a better approach to system architecture (and associated technical and procedural designs). One measure of the quality
of the NAS system architecture is (and will be) its flexibility in addressing UAS operations as they unfold, recognizing that UAS requirements and capabilities are likely to change a great deal as these technologies mature.
Recommendation: The Federal Aviation Administration (FAA) and its architecture leadership community should look for and apply lessons from the challenge of integrating unmanned aircraft systems (uAS) into the National Airspace System (NAS) as it develops an effective system architecture. The FAA and its architecture leadership community should incorporate measures in the NAS system architecture to address uAS integration.
INCORPORATING HUMAN FACTORS
The medium-term plans for the NAS will not fundamentally change the roles and activities of pilots and controllers. However, even with modest changes, misunderstandings and errors can result. Numerous constraints challenge the ability of the FAA to smoothly and effectively make changes to its systems and procedures. Furthermore, human factors for crew and controllers alike are an important ingredient in successful changes. Procedures and airspace redesign go hand-in-hand with technical changes and adjustments and are often just as complicated—and thus a bottleneck to realizing expected benefits of new technologies and approaches.
Human factors need to be incorporated in design, technical, engineering, and architectural discussions as early as possible, not after the design is complete (e.g., as a check on the design). This is both an organizational challenge for the FAA—as it may not have sufficient human factors personnel to integrate contractors’ work with system design—as well as a technical and engineering challenge—determining how requirements and constraints flow to early stage technical requirements, so that human factors perspectives can contribute to early design work. When human factors are not included at the outset, products and services need to be modified subsequently to meet the human factors requirements, which delays the release of products and services and significantly increases cost.
Recommendation: The Federal Aviation Administration (FAA) should recognize and incorporate in early design phases the human factors and procedural and airspace implications of stated goals and associated technical changes. In addition, the FAA should ensure that a human factors specialist, separate from the research and certification groups, has sign-off authority within the Next generation Air Transportation System approval process.
ASSESSING COSTS AND BENEFITS
As part of its charge, the committee also explored anticipated costs and benefits to NextGen and their implications. The FAA has put forward a business case for NextGen,7 and the committee held several discussions with FAA staff as well to understand the analysis used to develop the costs and benefits of implementing NextGen.
NextGen plans require a substantial investment, both by the taxpayer via the FAA for infrastructure, and by carriers and aircraft owners for equipage and training. At best, benefits—however quantified—to carriers will lag deployment costs; and benefits that accrue to the carriers will be less than the projected social benefits (quantified in the form of reduced delays to passengers, as is standard for Department of Transportation analyses of this sort) to the system as a whole. Many of the benefits of NextGen, such as those enabled by increased automatic communication between aircraft, cannot be meaningfully realized unless air carriers each equip their fleets with the requisite technology and, indeed, depend on all or nearly all aircraft being so equipped. The carriers will also incur training costs, both for new equipage and for new procedures that use old equipage. For airlines to gain significant benefit, NextGen capabilities will need to be deployed at sufficient scale. Given the delay in implementing new procedures and technologies at major airports, airlines may not see benefits for some time. The costs and benefits analysis presented to the committee was sensible; however, that mismatch presents an ongoing challenge for the FAA and Congress. Current short- and medium-term goals for NextGen emphasize new technologies to improve and enhance existing capabilities. Although modernization efforts are important and can bring significant benefits, it remains a challenge to incentivize uptake (e.g., equipage, training, or changes in procedures) absent clear benefits.
Recommendation: Preceding any further equipage mandate, the Federal Aviation Administration (FAA) should provide an estimated statement of costs and benefits that is mutually reviewed and agreed upon with the relevant stakeholders. It should be based on a mature and stable technical specification and a committed timeline for FAA deliverables and investment (for procedure and airspace design, infrastructure deployment, training, and so on). On this basis, industry could responsibly invest as required, given a reasonable expected return.
7 Federal Aviation Administration, The Business Case for the Next Generation Air Transportation System: FY 2013, Washington, D.C., 2013.