3
Emerging Topics of Focus and Concern
In this Chapter, the committee outlines several topics and areas of focus and concern that have emerged from its data gathering efforts so far. The committee will provide its findings and recommendations in its final report. Some of the questions that have been raised and discussed at briefing sessions are noted here, clustered roughly according to topic. Note that these preliminary observations, questions, and topics are raised as part of the committee’s information-gathering efforts and may not reflect the emphases that will be given to topics in the committee’s final report, and the committee does not anticipate addressing every question and issue outlined here in the final report. This chapter concludes with a brief comment about the importance of modernization.
UNDERSTANDING AND MANAGING BENEFIT AND COST EXPECTATIONS
The committee is concerned about the alignment among (1) the overarching vision for the Next Generation Air Transportation System (NextGen), (2) the expected benefits and the risks to achieving those benefits, and (3) the estimated costs (and who bears those costs). Because all three are subject to change as the context and underlying assumptions change, it is to be expected that all three would have changed since the launch of NextGen. But the committee is concerned that these changes were not fully reflected in the briefings it has received or the documents it has reviewed and that there are not clear mechanisms to track these changes over time or to make them known to stakeholders. The vision, the benefits, and the costs of NextGen are all models (that is, predictions) of a future state with many sources of uncertainty. The risks to achieving the anticipated benefits on the expected schedule are not clear to the committee, because of uncertainty in the value framework delivered to the users and stakeholders. In addition, the development risks confronted by the developers are not well prioritized or well quantified, and the evolutionary commitments for the short, medium, and long term are not well articulated.
In an effort to understand these and related issues better, the committee will explore further the vision for NextGen and how it has changed over time and the risks and benefits analysis for NextGen and how those analyses have changed over time. Examining the state changes of the vision
from its original inception to the current baseline provides a window into how the predictions of the benefits have performed since NextGen was first envisioned.
More generally, the committee is concerned about how uncertainty in this large, complex undertaking is understood, managed, and synthesized into more predictable plans and expectations. The FAA is correctly taking an incremental approach to system development. The committee seeks to ascertain how the FAA channels the validated learning in previous increments into better predictions of benefits, costs, and outcomes. Focusing on FAA’s approach to programmatic, engineering, and operational risk for NextGen, the committee requested information on the following: the FAA’s assessment of each of these types of risk, how risk assessment is done, ongoing or anticipated mitigation strategies and how those are built into the development process, and expectations for how these risk profiles will change in the medium and long term. The committee has not been able to elicit a clear articulation of the significant risks perceived within the NextGen systems and software or how the FAA would quantify or monetize these risks to substantiate the priority ordering and anticipated schedule. Specific questions the committee will explore further include the following:
• What is the largest source of NextGen benefits today? What is the largest source of benefits expected to be in the future?
• What are the most uncertain benefits planned in the FAA’s NextGen system and software capability?
• What are the most significant uncertainties in NextGen requirements that drive development risk?
• What are the most significant uncertainties in NextGen design and system architecture?
• What are the most significant uncertainties in the planned sequence of milestone expectations?
• What measures and metrics will be used to quantify and steer risk management priorities?
The committee wants to better understand the FAA’s perspective on the uncertainty in the upsides (benefits) and the downsides (programmatic risks). The FAA could quantify these uncertainties by modeling the benefits and risks as probability distributions of possible outcomes. Even if these distributions were represented as simple triangular distributions (best case, worst case, expectation), it would provide the committee with a set of critical priorities and a quantified model of the perceived uncertainties.
The committee’s statement of task (see Box P-2) asks for an examination of the NextGen enterprise architecture. The committee interprets this use of the term “enterprise architecture” to mean the Office of Management and Budget requirement that every government agency have an enterprise architecture designed to “promote mission success by serving as an authoritative reference, and by promoting functional integration and resource optimization with both internal and external service partners.”1 The enterprise architecture is thus focused on business structures and processes,
_________________
1 See Office of Management and Budget, The Common Approach to Federal Enterprise Architecture, Washington, D.C., May 2012, available at http://www.whitehouse.gov/sites/default/files/omb/assets/egov_docs/common_approach_to_federal_ea.pdf. The term “enterprise architecture” does not have a single definition and is used by others outside the federal context with slightly different meaning. Peter Weill at MIT defines an enterprise architecture as “the organizing logic for key business processes and IT capabilities reflecting the integration and standardization requirements of the firm’s operating model” (Page 2 of P. Weill, “Innovating with Information Systems: What Do the Most Agile Firms in the World Do,” presented at the Sixth e-Business Conference, Barcelona, Spain, March 2007, available at http://www.iese.edu/en/files/6_29338.pdf.) Maier and Rechtin observe that
and the NextGen enterprise architecture2 addresses these matters. However, enterprise architecture is only one piece of the architectural definition that is needed for NextGen.
Any large-scale, software-intensive systems endeavor requires a technical and systems architecture that specifies how all of its parts fit together and interact, which can be used in a dynamic way to help inform and drive decision making.3 The systems architecture of the NAS would encompass the technical and operational aspects of the system and its components—individual avionics systems, communications facilities, ground equipment, airport facilities, routes, approach procedures, personnel roles and training, and so on. In addition to addressing the technical enablement of the full set of functional capabilities, a system architecture also addresses “nonfunctional” attributes such as critical quality attributes, management of variabilities (that is, anticipated changes over a system lifetime), roles of services vendors, and alignment with organizational and supply-chain structure.
The committee’s discussions have focused on both the particulars of the enterprise architecture as well as the technical and systems architecture. Questions and topics related to these architectures that have been discussed thus far include the following:
• Who are the specific intended users of the various architectures, and what types of decisions are they expected to influence?
• What are the processes for managing architecturally significant decision making within the NextGen effort? Are these processes explicitly managed, or is architecture more of an emergent outcome from a multiplicity of separate processes—and one that may, or may not, have the necessary technical and structural attributes?
Given the enormous range of activities within NextGen, considerations related to the congruence of system architecture and organizational (and supply-chain) architecture are likely to be significant, prompting the following questions:
• How does the architecture address the goal of risk mitigation, and how is this determined?
_________________
“If we take an enterprise to be an organization with a defined mission, . . . the practice of enterprise architecture would concern itself largely with business strategy and business processes” (M. Maier and E. Rechtin, The Art of Systems Architecting, Third Edition, CRC Press, Boca Raton, Fla., 2009, p. 353).
2 Mike Hritz, Role of Enterprise Architecture NextGen, Briefing to the Committee Review the Enterprise Architecture, Software Development Approach, and Safety and Human Factor Design of the Next Generation Air Transportation System, March 2013. See also The National Airspace System Enterprise Architecture at https://nasea.faa.gov.
3 Chapter 3 of National Research Council, Critical Code: Software Producibility for Defense (The National Academies Press, Washington, D.C., 2010, pp. 68-69) offers a useful description of architecture and its importance:
Just as in physical systems, architectural commitments comprise more than structural connections among components of a system. The commitments also encompass decisions regarding the principal domain abstractions to be represented in the software and how they will be represented and acted upon. The commitments also include expectations regarding performance, security, and other behavioral characteristics of the constituent components of a system, such that an overall architectural model can facilitate prediction of significant quality-related characteristics of a system that is consistent with the architectural model. Architecture represents the earliest and often most important design decisions—those that are the hardest to change and the most critical to get right. Architecture makes it possible to structure requirements based on an understanding of what is actually possible from an engineering standpoint—and what is infeasible in the present state of technology. It provides a mechanism for communications among the stakeholders, including the infrastructure providers, and managers of other systems with requirements for interoperation. It is also the first design artifact that addresses the so-called non-functional attributes, such as performance, modifiability, reliability, and security that in turn drive the ultimate quality and capability of the system. Architecture is an important enabler of reuse and the key to system evolution, enabling management of future uncertainty. In this regard, architecture is the primary determiner of modularity and thus the nature and degree to which multiple design decisions can be decoupled from each other. Thus, when there are areas of likely or potential change, whether it be in system functionality, performance, infrastructure, or other areas, architecture decisions can be made to encapsulate them and so increase the extent to which the overall engineering activity is insulated from the uncertainties associated with these localized changes.
• How, for example, is the system architecture designed to accommodate novel uses of the airspace such as unmanned aircraft systems?
• How are the interplays between security and safety and between efficiency and safety in the system architecture being dealt with and assured?
• To what extent are technological requirements and specifications for near-term systems developed to accommodate future needs, and how is this managed in the overall systems architecture?
• How are changing requirements and changing stakeholder needs expressed, modeled, and accommodated over time?
• How are process or technology innovations suggested or developed by local area experts (e.g., at individual towers), contractors, or other stakeholders within the NAS vetted for potential impacts and potential wider adoption/implementation, and can the system architecture flexibly accommodate such innovations? The committee is particularly interested in examples of where these sorts of accommodations have been accomplished.
SYSTEM INTEGRATION AND SOFTWARE DEVELOPMENT APPROACHES
The committee is focused on the existing and anticipated processes for integrating new capabilities into the system over time. Regarding system integration, the committee has been seeking input about the incremental build plan for NextGen and how new capabilities will be integrated, including the NextGen target architecture and the primary desired behaviors and attributes that drove efforts toward this target architecture. As discussed above, understanding the primary constraints (economic, technical, political, physical, human, and so on) that bound decisions will be important to aligning the evolving NextGen vision with near- and long-term benefits.
The development of a large-scale, software-intensive system includes requirements elicitation and analysis, specification, architecture definition, design, coding, testing and analysis, and evolution. Historically, the trade-offs captured in requirements elicitation, specification, and architecture have proven to be strong indicators of success in reducing risks and uncertainties, especially in larger, more complex software systems such as those being developed for NextGen.
The committee is particularly interested in quantified measures and expert engineering judgments of software change costs (e.g., requirement change, design change, code change, regression test cycle, bug fix, build time, or others) and trends (how are these change costs increasing or decreasing over time). More abstractly, the committee will examine further how the FAA characterizes its software posture. The following considerations have been discussed: How to characterize and quantify the “mass” of software to be developed, and how to characterize and quantify the cost of the software to be developed. With respect to contractors in particular, the committee has been discussing and learning about how technical and system requirements and expectations (as well as changes) are communicated to and from the FAA, what the underlying technical and architectural assumptions are and how they are made explicit, and how testing and integration is managed.
A particular program or enabling technology can be used as an exemplar to consider some of these issues. With regard to the major NextGen programs, the committee is interested in their detailed technical specifications and operational dates. For the Data Communications program, for example, these areas would include what modes of digital data will be supported (i.e., protocols, bandwidth, security, availability, reliability, and so on) and what the expected aircraft equipage rates will be. In general, the requirements for a given program—what the new capability will be used for in the medium and long term, and more generally the current status of the program and
projected timeline for its implementation, are of interest. Equipment cost expectations—both for commercial carriers and general aviation users—will also play a role in uptake in some situations. For a given program, the general questions about uncertainties and risk apply as well: What have been the biggest challenges (technical, process, operational, or organizational)? and What are current uncertainties and risks and anticipated mitigation plans?
In considering the system safety aspects of NextGen, two factors are key: (1) the development of NextGen provides an opportunity to introduce new air traffic control (ATC) safety capabilities, and (2) the development of NextGen requires that historic safety performance of ATC systems either be maintained or improved. The committee is seeking to understand the FAA’s safety management system process and implications for the development and deployment of anticipated NextGen capabilities so as to understand how these two factors are being addressed.
The committee is focused on the safety objectives that the FAA has for NextGen, the processes and techniques by which the FAA expects these objectives to be met, any safety indicators and metrics used in an ongoing way in the development process, and any preliminary NextGen safety data and associated analyses that are currently available. In order to understand the basis for the safety objectives for NextGen, the committee is interested in learning more about the following: the safety objective defined for the current ATC system, the current mechanism for safety assessment for the NAS, the status of assessed safety metrics in the NAS, and how current safety assessment mechanisms and procedures might change as NextGen capabilities continue to be put in place.
Safety is not a system property that can be added after basic functionality has been addressed. Safety is an emergent property that derives from careful design at all levels. Thus, the committee is interested to learn what techniques are included in the NextGen architecture, design, and implementation to mitigate residual risk. The committee also notes that comprehensive hazard identification is a critical input to the safety-engineering process. The committee is interested to learn what specific techniques are being used to undertake hazard identification and estimate residual risk at each phase of deployment. In briefings to the committee, quantification of various safety items has been summarized. The committee expects to further explore what quantification approaches are being used in NextGen to estimate probabilities of hazardous states and the role of quantification in deployment decisions for NextGen features and capabilities.
Other air traffic control systems throughout the world have been upgraded and have added novel capabilities over time. The committee also plans to explore the following: to what extent the safe-design concepts and safety-assessment techniques planned for NextGen are consistent with and compatible with those used by other agencies around the world (such as the United Kingdom’s Civil Aviation Authority, Eurocontrol, and the International Civil Aviation Organization) and to what extent the safety technology and associated experience elsewhere have been reviewed by and considered for adoption by the FAA.
HUMAN FACTORS, AUTOMATION, AND DECISION SUPPORT TOOLS
Experience has shown that human factors challenges arise when different systems and different people interact. The scale, heterogeneity, and complexity of NextGen mean that there will be many upgraded or new systems and operational procedures, all being developed under different programs, at different stages of implementation, tested at different airports, and coming online at
different times. This is understandable given the complexity and scale of the NextGen effort. The committee is interested in understanding the current and anticipated automation and decision-support capabilities and how users of the airspace will be trained, including plans for managing the transitions from existing tools and processes to NextGen, recognizing that it is difficult to specify a priori design, and guidelines for human-systems interaction.
With regard to roles and boundaries in human-systems interactions, there is generally a shift, from human to system, taking place in the interpretation of sensor data and, increasingly, decisions and actions based on that data.4 This has profound significance for the design of the NextGen system. It applies both to piloting and to controller responsibilities—raising issues that connect policy, user-experience design, and usability evaluation. It also creates challenges for capability assurance, because some of rules of engagement previously belonging in the human domain, and typically relatively informally modeled, may need to be formalized to the point where there can be assurance that system behavior will respect these rules. There are trade-offs to be studied between enacting desired mission goals through changing the human (selection, training, staffing) and changing the technology.
Thus far, the committee has been exploring some of the topics and questions in this area, asking, At what level of management is the responsibility to ensure that critical human factors considered? and At what stage in the system development process does the human factors team become involved, and what is their method of involvement? The committee will explore examples of past human-factors recommendations that have been integrated into the system (or not) and why they were (or were not) integrated. The committee will also explore the differences between current ATC technologies and processes and anticipated NextGen ATC technologies and processes with regard to needed skill sets and knowledge requirements for controllers and pilots. A large cadre of controllers will be retiring soon, so the committee is interested in understanding what measures are being taken to analyze the job skills required for ensuring that new controllers understand and make use of the computer-based systems being introduced in NextGen.
Because human performance capabilities and risks are difficult to quantify in engineering terms sufficient to specify requirements, human factor aspects of new system designs must be evaluated by means of human-in-the-loop simulations (HITLSs). One issue is how and when to use HITLSs, and with what simulator fidelity. Experience has shown that at initial stages of system design, much understanding of the issues can be gleaned from relatively simple part-task HITLSs.
New computer-based decision aids are bound to make controllers more dependent on the computer advice given. Questions include, How is that expected to affect policies for assigning authority and responsibility of controllers? What types of automation of capabilities are anticipated in the near and long term, what capabilities exist for human override of automation, and what changes are anticipated in the future? NextGen assumes a level of teamwork between sector controllers, flow controllers, tower controllers, and pilots—much greater than in current ATC. The committee is interested in what means are being used to ensure that all parties to the cooperation are seeing the picture in the same way in an adequate time window.
_________________
4 National Research Council, Human-System Integration in the System Development Process: A New Look, The National Academies Press, Washington, D.C., 2007.
The designers and developers of any software- and communications-intensive system deployed today must grapple with questions of system security.5 Understanding the security risks and threats and developing appropriate threat models and mitigations are challenges endemic across government and industry. NextGen is no exception; indeed, the safety of life implications and the vital economic importance of air travel make the security of NextGen and the NAS critically important. As various programs and components of the national airspace are modernized, upgraded, and transformed, the security implications of the changes will need to be taken into account.
The committee is concerned about the plans, processes, and mechanisms for managing system security in the national airspace. Some of the topics and questions in this area the committee has been exploring thus far include the following: the threat model, how it is being validated, and how threats will be monitored over time as context and adversaries change; the scope and focus of security concerns for NextGen and how they are accommodated in the system architecture; and the FAA’s assessment of its most significant security risks and challenges and what plans are in place to address them. For example, is there a well-articulated a process for addressing security attributes as early as possible in the process, as opposed to relying on process compliance and intensive after-the-fact acceptance evaluation?6 Programmatically, questions to consider further include these: Where does overall responsibility for security reside? and How are cybersecurity considerations managed and addressed in the various programs of NextGen (such as ADS-B and DataComm)? Finally, the committee believes it will be important to understand how NextGen and the NAS cope with the insider threat—that is, authorized users of the systems with malicious intent.
UNMANNED AIRCRAFT SYSTEM INTEGRATION
Unmanned aircraft systems pose numerous procedural and technical challenges and introduce new requirements; they also will involve both safety and security challenges. NextGen architectures will need to be designed to manage, accommodate, and integrate this new class of aircraft. Depending on what rules are promulgated with respect to unmanned aircraft systems (UAS) in the airspace, integration of UAS will likely require that the NAS accommodate a wider spectrum of aircraft weights and sizes; aircraft operating over a larger range of flight profiles using non-traditional routes; and a larger variety of aircraft with a broader range of capabilities.7 Additional challenges include managing differences between see-and-avoid capability (the capability traditionally provided by human pilots) and sense-and-avoid operation; autonomous operation of UAS, either as part of a mission profile or as a result of the loss of a command link; and ensuring efficient communication with the UAS crew. Operational expectations within the NAS would almost certainly have to change. For example, procedures to ensure adequate separation of UAS from other aircraft and recovery of safe flight for manned aircraft in the event of a loss of separation will be important.
Perhaps most critically, failure modes of UAS will differ substantially from the failures modes of manned aircraft, affecting safety, reliability, and security analyses. For instance, vulnerabilities may exist in the command and control link—How should a UAS behave if it loses communications capability along that link or is spoofed or jammed? The committee is concerned about current and
___________________
5 Here the committee refers to what some call cybersecurity—system, data, and communications security, which is distinct from the physical security required for airport and aircraft operation, provided in part by the Transportation Security Administration.
6 See, for example, M. Howard and S. Lipner, The Security Development Lifecycle, 2006.
7 Unmanned aircraft systems could operate from altitudes of a few hundred feet to perhaps 50,000 feet with dwell times potentially beyond 24 hours. The ability of a typical UAS to maneuver is also severely limited, complicating collision avoidance.
anticipated plans for safe integration of UAS into the NAS, recognizing that planning for UAS implementation has just begun. Questions include: What are the key factors that will guide FAA work in this space, and what is the projected time line for policy decisions and any associated implementation? What design and architectural decisions (if any) have been or will need to be taken in NextGen to accommodate UAS of varying flight profiles, capabilities, and weights and types?
One of the FAA’s most valuable capital assets is its allocated spectrum. The management and use of this asset in the future will need to be a critical element of the systems architecture. It takes significant time and effort to achieve international agreement and standardization before any changes can be made to the allocation of the spectrum. Aircraft equipage considerations require standardization and implementation to be coordinated worldwide; otherwise, international flights would be required to carry multiple sets of equipment for different parts of the world. Spectrum efficiency is also important. Like other entities that manage government-held spectrum, the FAA is under pressure to share spectrum and to use it more efficiently.8 The 108-137 MHz very-high-frequency (VHF) and 960-1215 MHz ultrahigh-frequency airbands, which are used for navigation aids, precision approach systems, and voice communications, are used inefficiently by modern standards, having been allocated in a different technology era. Most notable is the continued use of amplitude modulation (AM) channels for voice communication. But the transition to efficient digital voice, low-cost, high-performance data radios and the shutdown of some radars and old VHF navigation equipment to free up spectrum will be a challenging process. The committee is focused on current and anticipated plans with regard to spectrum management for the NAS and NextGen.
THE IMPORTANCE AND NECESSITY OF MODERNIZATION
FAA’s NextGen efforts are broadly aimed at transforming U.S. airspace, and toward that end, there are significant modernization opportunities along the way. Opportunities to replace and upgrade aging equipment are certainly within the scope for NextGen and offer the potential to reduce the risk of failure, to reduce maintenance costs, and to enhance capabilities. Modernization efforts also afford opportunities to increase the flexibility and extensibility of existing software-intensive systems to allow for incremental advancement over time. The trade-off between the near term and long term is, in part, about delivering economic benefits in the near term while taking compatible near-term steps that support longer-term transformation of the system. NextGen aims to deliver near-term benefits for its user community (which has a near-term planning horizon driven by business objectives) while establishing and retaining a focus on longer-term transformative changes in the system. The challenge is balancing evolutionary changes with revolutionary changes and aligning these changes with the most significant problems that most critically require solutions.
The long-term vision for NextGen is ambitious. Some aspects of NextGen are anticipated to be transformational. Others, no less critical, are important and necessary modernization efforts and can pave the way for potential eventual transformations. An ambitious long-term vision for NextGen includes short- and medium-term initiatives that will (1) provide a foundation for the
__________________
8 Two Presidential Memoranda in recent years have emphasized the importance of wise spectrum management: June 28, 2010, “Unleashing the Broadband Revolution” (http://www.whitehouse.gov/the-press-office/presidential-memorandum-unleashing-wireless-broadband-revolution), and June 14, 2013, “Expanding America’s Leadership in Wireless Innovation” (http://www.whitehouse.gov/the-press-office/2013/06/14/presidential-memorandum-expanding-americas-leadership-wireless-innovatio).
longer-term vision and (2) enable critically needed modernization of aging elements of the NAS. In the committee’s view, both of these elements are critically important.
The study committee will receive additional briefings about these and other aspects of NextGen in order to fulfill its charter. A final report, with the committee’s findings and recommendations, is anticipated in 2014. In that report, although it will not necessarily address each of the questions raised here in this preliminary snapshot, the committee expects to say more about each of the above topics.
This page intentionally left blank.
