The objective of Chapter 4 is to propose recommendations designed to guide the Air Force toward a new intelligence, surveillance, and reconnaissance (ISR) Capability Planning and Analysis (CP&A) process that enhances rather than replaces the current process. Chapter 4 begins by presenting three overall recommendations associated with the proposed process. The chapter then presents a set of desired attributes for this process that are based on the strengths and shortcomings of the current process, as well as on best practices provided by government and industry. Lastly, the chapter describes in detail a proposed ISR CP&A process that employs the best practices identified in Chapter 3 for overcoming the shortfalls with the current process as described in Chapter 2.
Chapter 2 describes the current process used by the Deputy Chief of Staff of the Air Force for ISR (AF/A2) to plan for and assess ISR capabilities. It concludes by summarizing strengths and shortfalls of the current process garnered from interactions with various ISR and best-practice stakeholders. Chapter 3 identifies several best practices associated with government and industry CP&A processes
that suggest solutions for the shortfalls identified in Chapter 2. After considering best practices in the context of strengths and shortfalls, the committee arrived at three major recommendations designed to guide the Air Force toward a new and more comprehensive ISR CP&A process. This section presents and provides a rationale for each recommendation.
Recommendation 4-1. The Air Force should adopt an ISR CP&A process that incorporates the following attributes:
• Encompasses all ISR missions;
• Addresses all ISR domains and sources, including non-traditional ISR;
• Includes all ISR assets in a sensor-to-user chain (e.g., PCPAD and communications);
• Collaborates with ISR-related entities;
• Provides traceability from process inputs to outputs;
• Is mission/scenario-based;
• Is repeatable and enduring;
• Supports trade-off analyses;
• Is scalable in size, time, and resolution; and
• Reduces labor and cost over time.1
Rationale: The Air Force currently has a reasonable ISR CP&A process but has indicated that this process requires improvements. The committee identified gaps in capability in the existing Air Force process, explored best practices for CP&A in both government and some industry organizations, and developed a set of desired attributes from its analysis of gaps and best practices that represent a robust ISR CP&A process. With the addition of three capabilities, the Air Force can attain the desired attributes by enhancing rather than replacing the current process. The three capabilities are as follows: (1) a front-end Problem Definition and Approach (PDA) capability, (2) a robust Multi-resolution Gap Analysis (MGA) capability, and (3) a suite of automated tools that underpin that analysis of the cost, risk, and utility associated with investment alternatives. The full process and an explanation of how it satisfies the desired attributes are presented later in this chapter. The committee acknowledges that the proposed process should accommodate the use of all levels of classified material in the analysis. Although industry presentations included examples of tools being used to process classified information, both security and
1The committee acknowledges that any process needs to accommodate the use of all levels of classified material in the analysis. However, security and time constraints precluded the committee from making recommendations for multi-level security analysis. Chapters 2 and 4 provide supporting discussions.
time constraints precluded the committee from making detailed recommendations regarding analyses involving multi-level security.
Recommendation 4-2. The Air Force should evolve its ISR CP&A process to an integrated, overarching ISR investment process with clear organizational responsibility identified for each subprocess.
Rationale: One of the most important actions that the Air Force can take is to implement an integrated ISR CP&A process. As described in Chapter 2, the Air Force has evolved into a current situation that has multiple, overlapping investment processes that appear to duplicate effort. The integrated process should have clearly defined roles and responsibilities for participants and clear identification of the lead for each portion of the process. Although an integrated process is recommended, this really means an overarching process with multiple subprocesses. A single organization should be responsible for each subprocess. Different subprocesses may have different organizational leads. A candidate overarching process is described in detail in the section below entitled “Proposed Air Force Intelligence, Surveillance, and Reconnaissance Capability Planning and Analysis Process.” An example of a subprocess is the materiel Solution Analysis process led by the Air Force Materiel Command (AFMC). Lastly, in many but not all cases the ISR CP&A process will feed the Air Force Corporate Process, in which components of the ISR process will be assigned to panels (e.g., ISR communications to the Communications Panel). During the Air Force Corporate Process, the impact of board and panel decisions on the overall required ISR capability should be continuously monitored to preclude, for example, a panel’s failing to fund a key ISR capability component that may have a low priority as far as that panel is concerned but is vital to the fulfillment of a high-priority ISR need (e.g., a communications link). This can best be self-monitored by establishing a set of interface or giver/receiver relationships across the Air Force. These interfaces become part of a set of agreements on what another board or element is expected and committed to perform. Updating these agreements on a regular basis, with signature concurrence from both sides of the interface, allows timely responses and should result in establishing areas of higher risk when the risk is not the usual technical or schedule risk but rather can be expressed in terms of the risk of an activity’s being funded. Mitigation plans for these risks may be developed in much the same manner as for technical or schedule risk.
Recommendation 4-3. The Air Force should adopt the proposed ISR CP&A process by incrementally building on its existing process using pilot projects. The scope of each pilot project should be compatible with available resources, be relevant to both current and future mission scenarios, and include metrics
to measure achievement of the desired improvements (e.g., manpower reductions and increased timeliness).2
Rationale: The committee was sensitive to the investment that the Air Force has made in its current ISR CP&A process and to the resource-constrained environment in which the Air Force currently finds itself. Hence, the committee designed the proposed process in a manner that would allow the Air Force to expand the capabilities of the current process incrementally over time in response to real-world demands and resource availability. The Air Force is encouraged to develop incrementally, as needed, a reusable information repository and associated suite of analytical tools, models, and simulations that can be used to automate the exploration of trade space of various ISR architectures against mission requirements and cost profiles. This capability would support the sharing of ISR capability information and metrics across multiple analyses and should be retained and evolved from one planning cycle to the next. The Air Force would thus develop an institutional knowledge base of ISR capability and analysis that would allow more rapid and effective decision making. The ISR Capabilities Analysis Requirements Tool (ISR-CART) is a substantive start for this reusable information repository; the Air Force should build on it by populating various tools, models, and simulations that would execute using the shared information. Funding for the pilot project(s) would most likely come from the Air Force organization responsible for the ISR CP&A process. Lastly, a pilot project would provide insight into how challenging the recommended process improvements would be to implement throughout the entire ISR CP&A process.
As described in Chapter 2, the current Air Force ISR CP&A process has several strengths. It is based on a concerted effort by the Air Force to organize a comprehensive planning approach for the ISR portfolio—a process that began with the ISR Flight Plan, evolved into the ISR CP&A process, and broadened to include the Global Integrated ISR (GIISR) Core Function Lead Integrator (CFLI) process. The current processes endeavor to be inclusive and collaborative by coordinating
2The proposed process is described in this chapter in the section entitled “Proposed Air Force Intelligence, Surveillance, and Reconnaissance Capability Planning and Analysis Process.” Also, notional scenarios are discussed in Chapter 1; they range from regional conflicts (Persian Gulf and Pacific Rim) to global, non-traditional conflicts, to homeland security scenarios.
inputs across relevant CFLIs and associated Major Commands (MAJCOMs). The process also maintains a fairly comprehensive repository of ISR needs, capabilities, and gaps in the ISR-CART tool.
The current processes tend to focus somewhat narrowly on needs that require “big iron” air platform solutions rather than on capability-based needs that might be better addressed by non-airborne solutions.3 Not surprisingly, the current process also focuses on solutions to near-term problems associated with the ongoing counterinsurgency (COIN) conflict in Southwest Asia. In addition, the process appears to lack a systematic methodology for ensuring that the needs, gaps, and solutions of non-Air Force organizations—particularly those of the intelligence community (IC)—are factored into the mix. In short, the current process does not address all domains, all relevant operational scenarios, or all ISR customers and providers. These and other shortfalls, summarized at the conclusion of Chapter 2, combined with the many strengths of the current process and the best practices of government and industry, lead the committee to articulate the following set of attributes that it believes should serve as guiding design criteria for an enhanced ISR CP&A process.
In order to meet the range of ISR demands described in the Chapter 1, the ISR CP&A process should take advantage of all potential sources, including the following: the IC (e.g., the National Reconnaissance Office [NRO]), non-traditional sources (e.g., non-ISR imaging satellites, F-22, F-35), the Joint community, and coalition partners. It should also address the air, space, and cyberspace domains to fulfill the requirements of the diverse community of users who both depend on and contribute to Air Force ISR. It should include the operational, tactical, and strategic levels of war to provide ISR support for the wide range of Combatant Command (COCOM) Operation Plans and current operations. And it should consider all assets in the sensor-to-user chain, including communications links and the downstream planning and direction, collection, processing and exploitation, analysis and production, and dissemination (PCPAD) portions of the chain.
The ISR CP&A process should be responsive and credible. A responsive process should support a span of requests, from quick-look analyses to deliberative, longer-term analyses that support the Future Years Defense Program (FYDP). To be credible, the process should be transparent, the outcomes should include divestitures as well as additions, and the results should be traceable and repeatable. Table 4-1 summarizes the above attributes. Following is a detailed description of a proposed process that the committee believes will embody these attributes.
3Col Brian Johnson, Chief, ISR Plans and Integration Division (AF/A2DP), Headquarters, U.S. Air Force. “Air Force ISR CP&A Overview.” Presentation to the committee, October 6, 2011.
TABLE 4-1 Desired Attributes of a Comprehensive Intelligence, Surveillance, and Reconnaissance (ISR) Capability Planning and Analysis (CP&A) Process
|Desired ISR CP&A Process Attribute||Description|
|Encompasses all ISR missions||While the process may be primarily driven by near-term mission needs such as the current fight in Southwest Asia and by the counterinsurgency battle doctrine, it should also be able to address future conflicts informed by new doctrines such as Air Sea Battle. The process should look beyond theatre-specific Air Force missions to incorporate those of Joint Forces and the IC around the globe as well.|
|Addresses all ISR domains and sources||The Air Force should broaden the process aperture to address all domains (including air, space, cyberspace). The process should also address the complete range of multi-INT data sources provided by both traditional and non-traditional platforms.|
|Includes all ISR assets in the sensor-to-user chain||All end-to-end capabilities required to produce quality intelligence rapidly for the warfighter, including sensors, platforms, data links, and planning and direction, collection, processing and exploitation, analysis and production, and dissemination (PCPAD), are addressed holistically.|
|Collaborates with ISR-related entities||Key stakeholders and/or representatives in ISR mission and/or scenarios need to be involved in the analysis process in order to take advantage of synergies and to ensure that Joint ISR mission needs are met.|
|Provides traceability from process inputs to outputs||The underlying data, assumptions, and models used to generate outputs at each stage of the analysis process are revealed.|
|Is mission/scenario-based||Mission needs and operational constructs are explicitly factored in to the gap analysis and prioritization.|
|Is repeatable and enduring||The process is sufficiently simple and transparent to ensure that the Air Force can repeat it on both short and long timescales. The process endures through inevitable changes in Air Force leadership, organization, strategy, and budgets.|
|Supports trade-off analyses||The decision maker is allowed to rigorously trade off the costs, risks, and utility of alternative ISR force mixes.|
|Is scalable in size, time, and resolution||A “multi-resolution” ability is provided in order to quickly answer investment questions at a coarse level of resolution, or more deliberately analyze answers at finer resolution. “What-if” analyses are supported for a quick response to focused, specific investment questions as well as large-scale scenario-based investment questions for budget deliberations.|
|Reduces labor and cost over time||Automated tools are leveraged to produce faster, more accurate results with fewer resources.|
This section presents and describes major elements of a recommended, enterprise-wide ISR CP&A process. Suggested methodologies and time lines for reviewing and assessing information are briefly discussed for each function. Tools that may help automate and accelerate the process are offered. As a reminder, the proposed process will provide actionable ways to improve the ISR CP&A process to yield an effective end-to-end investment approach for the integrated mix of ISR capabilities across all domains.4
A graphical representation of these vectors in a proposed process diagram is provided in Figure 4-1. Additions to the current process are shaded in red. The two “big vector” changes recommended to the current process are these: (1) the inclusion of a Problem Definition and Approach function that sets up the planning and analysis process, and a (2) Multi-resolution Gap Analysis function that provides the ability to rigorously trade off the costs, risks, and utility of alternative ISR force mixes in an end-to-end system context. The major functions depicted in Figure 4-1 are described in more detail in the following sections.
As described in Chapter 2, the current ISR CP&A process is designed to consider all needs and gaps on a periodic basis. It is not designed to rapidly answer “what if ” questions posed by decision makers faced with urgent issues. A good example of the need for “quick-turnaround” analysis capabilities recently occurred when the chairman of the House Armed Services Committee, in a letter to the Secretary of Defense, expressed the committee’s concern that the proposal of the Department of Defense (DoD) to stop purchasing Global Block 30 aircraft was “entirely budget driven with no underlying ISR analysis to support the U-2’s ability to fill the gap.”5 The current process, which is designed to consider all needs and gaps on a periodic basis, does not have the “machinery” to address individual questions rapidly.
In order to provide the ability to address specific issues, the committee recommends initiating the process with a PDA step—shown in Figure 4-2. The primary
4It would be counterproductive to attempt to recommend a highly detailed ISR CP&A process here. The committee does not possess the in-depth knowledge that would be needed regarding all the required process attributes for it to be able to offer effective, actionable suggestions for detailed elements of a comprehensive process. Instead, the committee offers a high-level view that describes “big vectors” that any effective, enterprise-wide ISR CP&A process should possess.
5Letter from the Chairman of the House Armed Services Committee to the Secretary of Defense, May 11, 2012.
product of the PDA step is an analytical framework designed to guide and focus subsequent steps of the proposed process, shown in Figure 4-1. This framework is carefully crafted through knowledge-elicitation sessions with key stakeholders to drive the ISR CP&A process toward answers that enable and justify investment and divestment decisions. The analytical framework captures, among other things, decision makers’ questions, relevant documents, metrics, scenarios, models, and analysis tools needed to configure and support the downstream Needs Analysis and MGA steps in the ISR CP&A process.
In the proposed PDA step, investment questions would be carefully developed and documented through literature research and knowledge-elicitation sessions with decision makers and relevant stakeholders. Literature research includes an analysis of relevant strategic guidance, such as National Intelligence Estimates, the FYDP, Air Force Chief of Staff Initiatives, and Global Threat Analyses. The resulting set of focused questions would guide collaboration among analysts and various domain experts, such as financial analysts, engineers, operators, and intel-
ligence analysts during Multi-resolution Gap Analysis. If the questions are properly designed up-front, the downstream process should yield quantitative answers that enable decisions in short or long time intervals, with less or more confidence depending on the selected resolution of the analysis.
For example, it is likely that Air Force ISR decision makers have received strategic guidance causing them to consider investment decisions about the ISR capabilities needed to support jungle operations in areas that prohibit overflight. In preparation for assessing the ISR capabilities needed in such contexts, the Problem Definition and Approach team might review various mission needs documents indicating that high-priority jungle operations monitor the movements and actions of guerilla warfare factions under triple-canopy foliage in order to understand their methods and procedures.6 The team would then work with various stakeholders to elicit specific questions that guide the analysis process toward answers needed to support investment decisions. In the simple example above, one might imagine framing questions such as this: Do we have a standoff sensor in the inventory that can penetrate jungle foliage and detect vehicles? Assuming that there is such a standoff sensor in the inventory, these questions might follow: Do we have communication and data links of sufficient bandwidth in place to support command-
6National Research Council. 2009. Sensing and Supporting Communications Capabilities for Special Operations Forces. Washington, D.C.: The National Academies Press.
and-control and data exfiltration needs? Do we have a PCPAD capability for the foliage-penetrating sensor data? How well do these solutions work? What are the costs and risks associated with acquiring and/or re-deploying known platform, sensor, communications, and PCPAD solutions?
Once a set of specific questions is developed, the PDA team would construct an analytical framework to focus subsequent steps in the process on answers to these questions. The framework would embody the mission needs in an operational scenario built from studying documents like existing operations plans and foliage maps for the area of interest. The analytical framework would describe the desired performance factors, such as payload standoff distances, sensor resolutions, and communications and data link bandwidth and latencies. The framework would also likely include a set of engineering models configured for the desired operational scenario. The model set might include physical models for propagation at various frequencies; communications models for communications alternatives, such as mobile ad hoc network, remotely piloted aircraft (RPA), and satellite communications; and foliage-penetrating (or poking) Moving Target Indicator (MTI) capabilities for tracking vehicles and dismounts and vehicles under dense foliage. In as many cases as possible, models would be chosen for their ability to scale, to support both coarse-resolution, quick-look analyses and finer-resolution, deliberate-look analyses.
Performance measures and automated tools are also included in the analytical framework. Performance measures (e.g., “average track length” for an MTI sensor) are chosen to help assess the cost, risk, and utility of payload, sensing, communications, and PCPAD alternatives. Automated or semi-automated tools are chosen for their ability to use the models, operational scenario, and performance measures. One example of such a tool is the physics-based Monte Carlo simulation tool used by RadiantBlue to assess the efficacy of alternative platforms, sensing modalities, and communications solutions.7
The initial set of models available to support “multi-resolution” may be somewhat small compared to the broad range of investment questions that decision makers might ask. Initially, models might need to be supplanted with subject-matter experts. Over time, it is envisioned that there would be built a library of reusable models that can be stored in an ISR Analytic Information Repository from which they can be easily retrieved and reconfigured to support a wide range of analyses.
With the above example in mind, the Problem Definition and Approach phase can be completed in few days for well-understood issues or for quick-look analyses in which coarse answers are needed quickly to support, for example, an urgent planning, programming, or budgeting question. Alternatively, a deliberate deci-
7Larry Shand, President, RadiantBlue, Inc. “RadiantBlue Modeling and Simulation Capability.” Presentation to the committee, January 5, 2012.
sion analysis phase can take weeks filled with workshops designed to interactively elicit information and decision needs from government stakeholders or staffs. For either quick-look or deliberate decision analysis approaches, this phase of the process should allow time for the performance of a limited trade-off analysis based on existing information and input from available subject-matter experts in order to produce coarse, initial answers to questions that might be sufficient for decision makers’ needs, and to develop a more informative analytical framework for subsequent phases of the process.8
The PDA step should be led by experts with experience eliciting actionable questions and related information from stakeholders in a form suitable for downstream analysis. The size and composition of the expert team will likely vary over time in response to changing investment decision needs. The team will likely consist of a small number of permanent members who have access to government, industry, and academic experts available on call to provide assistance on an as-needed basis. The organization responsibility for maintaining the team, funding requirements, and other team characteristics should be considered during the pilot project phase of the process implementation.
The primary purpose of the Needs Analysis step, shown in Figure 4-3, is to transform the set of investment questions contained in the analytic framework into focused needs that can be rigorously analyzed during Multi-resolution Gap Analysis. Unlike the Needs Analysis step in the current ISR CP&A process, which produces an unconstrained list of needs, the committee’s proposed Needs Analysis step would produce a constrained list of capability needs designed to focus the process on answers to specific questions.
Needs Analysis for a deliberative, long-term capability-planning exercise might employ subject-matter experts to assess likely scenarios provided by the analytical framework and make recommendations about needs for these scenarios. In the case of the earlier “triple canopy” example, needs might focus on the reconnaissance of a group of guerilla fighters, or on the exfiltration of unattended ground-sensor (UGS) data. More specifically, the Needs Analysis step might express the previous question—Do we have a standoff sensor in the inventory that can penetrate jungle foliage and detect vehicles?—in the form of the specific, analyzable need: We need the ability to detect 5-meter-square objects under triple-canopy jungle foliage at a range of 20 km. Although specificity is helpful, the Needs Analysis team should
8Doug Owens, Manager, Enterprise Analysis, Defense Business Unit, TASC. “An Enterprise Approach to Capability-Based Analysis: Best Practices, Tools, and Results.” Presentation to the committee, January 5, 2012.
avoid the temptation to express the need as a solution. For example, it would not be appropriate to express the same need exemplified above as: The Air Force needs a gimbal-mounted Geiger-mode LIDAR mounted on a Predator remotely piloted aircraft that can detect 5-meter-square objects under double-canopy jungle foliage at a range of 20 km. Disguising solutions as needs would minimize the value of the trade-space machinery used in the next step of the ISR CP&A process.
A focused needs list reduces process time and cost and provides the ability to answer specific questions rapidly. That said, the process should also periodically assess a wider set of needs so as to prevent the Air Force from myopically focusing on the mission of the day. The Air Force should consider reviewing all needs on a 3-year cycle, with no attempt to prioritize or filter needs during these broader looks.9 By avoiding the temptation to eliminate seemingly extraneous or unimportant needs during the process, the Air Force would retain an inventory of stated needs that might become relevant under a different set of strategic assumptions or operational scenario conditions. This would also help maintain a holistic view, which ensures that the perspectives of non-Air Force organizations do not wind up “on the cutting-room floor.”
The primary participants in the Needs Analysis function are the COCOMs, which typically express their needs by means of Integrated Priority Lists; the MAJCOMs, which represent the interests of their affiliated COCOMs; and the national IC. Because the IC, rather than the Air Force Space and Missile Systems Center, provides the majority of space-based ISR capabilities, it is imperative that the Air Force ISR Agency (AFISRA) reach out to the IC so that the ISR needs of their respective stakeholders can be shared in a disciplined, systematic way. Accordingly, the Air Force should actively work with the IC to formally link the ISR CP&A
9A similar approach is used by the National Geospatial-Intelligence Agency (NGA).
process with the processes of IC partners, particularly the National Geospatial Intelligence Agency (NGA), the NRO, and the National Security Agency.
Needs should be documented in writing using structured fields that can be stored and easily searched using a database-management system. At a minimum, needs records should include a unique identifier, description, and measures of effectiveness and/or measures of performance that allow the Air Force to quantitatively assess progress toward meeting the desired end state. Finally, as is the case today, the Air Force should continue to capture all needs in the ISR-CART database. Although the process of accurately and completely documenting, gathering, reviewing, and updating needs records is potentially tedious and time-consuming, once the initial set of needs is entered into the ISR-CART database, future Needs Analysis activities might be shortened considerably through the updating of the initial set on a change-only basis.
The primary objective of the Multi-resolution Gap Analysis step, shown in Figure 4-4, is to rigorously compare focused needs with existing capabilities and to
produce a list of prioritized gaps in a trade space that assesses mission utility, cost, and risk from multiple perspectives and at multiple levels of fidelity. The MGA step is a highly collaborative process among analysts, domain experts, and related stakeholders whose shared perspectives span a broad ISR capabilities trade space. MGA is a modeling and simulation-based approach to problem analysis that allows the Air Force to examine complex problems systematically by quantitatively analyzing the problem from different perspectives and at different resolutions. The approach uses a wide variety of different models and tools tailored for each perspective. It integrates different capability perspectives in a manner that provides traceability between causes and effects across the ISR enterprise.10
Exploring the problem from multiple domain perspectives is valuable because different domains often describe the problem with different representations and semantics, making it difficult to “understand the elephant” when described by just one of its many domain parts. Exploring the problem at multiple resolutions provides the ability to rapidly develop answers to questions using coarse, low-resolution models, or to deliberately produce accurate answers over longer time intervals using fine, high-resolution models. For example, it is possible to obtain a coarse estimate of the volume of water in a lake by multiplying the lake’s maximum depth, width, and length. Or, a much more accurate estimate can be produced using a high-resolution model that sums small volumes estimated at each point on a grid laid across the lake floor.
MGA approaches are particularly useful when the high- or low-resolution models or their input data are infused with uncertainty. These circumstances are common in planning for capabilities associated with military applications for which models of complex combat operations do not exist—and even if they did, would likely suffer from highly uncertain inputs associated with the fog of war.11 Theoretically sound, well-constructed models are consistent across resolution levels. And a well-designed Multi-resolution Gap Analysis approach scales in order to address large, complex problems, albeit often with less detailed and accurate solutions.
MGA is not a model or methodology within a model, but a methodology for the application of multiple models. The capabilities within the MRA methodology would not include models in which resolution could be dialed for applications. Mixing levels of resolution within models is not a desired practice. MGA’s driving concept is that high-resolution models can generate metrics that are rolled into
10Doug Owens, Manager, Enterprise Analysis, Defense Business Unit, TASC. “An Enterprise Approach to Capability-Based Analysis: Best Practices, Tools, and Results.” Presentation to the committee, January 5, 2012.
11James H. Bigelow and Paul K. Davis. 2003. “Implications for Model Validation of Multiresolution, Multiperspective Modeling (MRMPM) and Exploratory Analysis.” Santa Monica, Calif.: RAND. Available at http://www.rand.org/pubs/monograph_reports/2005/MR1750.pdf. Accessed March 22, 2012.
lower-resolution models. The various levels of resolution come into play in terms of the layering of models that employ more and more aggregation to meet decision needs sufficiently and responsively. Analysis can be conducted at lower levels of resolution for aggregate levels of trade-offs if such information is sufficient to aid decisions at those levels, as long as those aggregate-level effects maintain traceability to higher-resolution performance factors.
The MGA construct may consist of numerous models and tools to flesh out trade-offs sufficiently to support decisions. For example, a series of tools may be appropriate to assessing cyberthreat impacts to missions and determining options for mitigation of high-order effects. Technical, high-resolution nodal analysis of a physical network could determine the pervasiveness of a specified cyber attack. That analysis could produce metrics on estimated network downtime and degree of degradation to specific segments of the network architecture. Rather than carrying the high-resolution effects of the cyber virus to the next level of analysis, only the effects-based metrics would be carried forward. These could then serve as planning factors in a process flow model to determine data-throughput impacts of critical information delayed by the cyber effects on segments of the architecture. Those throughput-metric results could then serve as uncertainty bands in a simulation of the affected missions, such as communications support to joint operations in a specified scenario.
The MGA box in Figure 4-4 shows four domain perspectives and one integrated perspective, all interconnected to emphasize that the MGA process requires joint and several interactions among all perspectives. The four independent perspectives initially suggested when analyzing the utility, cost, and risk trade-off space are the Mission Perspective, which focuses on operational plans, force structure, and the command-and-control of operational assets; the Collection Perspective, which primarily includes ISR sensors and platforms; the Infotecture Perspective, which focuses on PCPAD activities in support of operations; and the Comms Perspective, which includes cyber, network, and communications capabilities that enable command and control and sensor data exfiltration. Additional perspectives undoubtedly exist and bear consideration when circumstances require.
More importantly, MGA uses an Integrated Perspective that allows trade-offs in utility, cost, and risk among the other perspectives. Tailored and scaled to the needs of the decision maker, the iterative, integrated process provides quick-look assessments through streamlined analysis processes early in the analysis cycle. And it increasingly adds layers of fidelity that allow broader and deeper analyses of the ISR capability trade space. Integrating different domain perspectives at different resolution levels then provides an enterprise view of needs associated with existing or missing capabilities. Examples of methods used to integrate operations and cost perspectives include multi-attribute utility analysis, which integrates multiple metrics into a set of value metrics, and inference analysis, which maps capability
performance metrics into a spider web mosaic for diagnostic assessments of potential impacts of variations in each metric.12
Many of these methods employ hierarchical structures, in which each layer has its own elements, properties, relations, and metrics that characterize the important behavior or operations within that layer. Elements of the layer may depend on other layers in the hierarchy, such as the interdependencies between communications and ISR. The attributes or metrics of these elements can be mapped through transformations that translate those attribute dependencies onto the next layer’s elements or behaviors. The result is a layering of capabilities and metrics that begin at the top in a more aggregated, higher level of abstraction, flowing down to lower (higher-resolution) layers, enabling trade-offs that are still rooted in the detailed physics levels but understood and assessed at higher decision levels. By this transformation layering, attributes from one layer to another can be decoupled and re-characterized to enable analysis at higher levels that, though rooted in the high-resolution physics of technologies and systems, are not strictly linked to specific parameters of individual systems or concepts. As long as the transformations that enable mapping from one layer to another can be constructed, the relations can be preserved and tracked through the entire enterprise.
MGA brings all perspectives to bear in order to find a viable solution, with viability conditioned by utility, cost, and risk assessments. These assessments provide separate, quantitative insights using interactive, model-based analyses driven by capability metrics assigned to PCPAD information and data flows; command, control, communications, and computer network trade-offs; sensors and platforms; operational concepts of operation; and cyber/information operation impacts. They also map capability metrics to cost estimating and risk-analysis trade-offs, and project costs over planning horizons. Cost-benefit analyses can be developed for individual domain perspectives as well as for combined families of systems, architectures, or the entire enterprise. Initially coarse, cost-benefit assessments are refined during later stages of the Multi-resolution Analysis, and subsequent Solution Analysis. A good example of a multi-resolution financial analysis capability is provided by TASC’s Financial and Business Analytics tool, which maps capability metrics to cost estimating and risk-analysis trade-offs, plus cost projections over planning horizons (Analysis of Alternatives, Program Objective Memorandum [POM] inputs).
Continuing with the previous jungle operations example, an RPA solution might be preferred on a cost and accuracy basis, but it might present a risk by indicating Air Force presence and interest unless flown at high altitudes to avoid detection. Risks might lie in the trade space between standoff distance, detectabil-
12TASC has developed an example of a methodology and associated tool—called Integrated Decision Analysis—that enables decision trade-offs among risks, sensitivities, and programmatic considerations across multiple perspectives.
ity by the adversary, and the wait for the development of a less-observable RPA capability. The mission perspective would influence the trade space in multiple ways. For example, high-resolution video (part of the collection perspective) would require “boots-on-the-ground” (meaning that the collection perspective affects the integrated perspective, since a Joint capability, e.g., a UGS, may be needed). Exfiltration of the video would require taking the communication perspective into account—for example, a data network consisting of ground elements connected to airborne and space relays to get the data into the hands of analysts and decision makers. This information flow from the theater to the users reflects the analysis and dissemination represented by the infotecture perspective and also illustrates the end-to-end analysis designed into the process. This process can be (and likely will be) iterated—for example, the use of robotic UGSs may reduce risks to human operators but may demand the development of new Air Force air deployment capabilities and communications architectures.13
MGA is executed as an interactive collaborative process among analysts, subject-matter experts, and various process stakeholders. The key to achieving a flexible, robust analytic capability is founded on the application of quantitative, model-based methods that allow an examination of the entire enterprise from different capability perspectives, integrated for a complete view of total capability. The use of model-based analytics within an integrated, interactive process allows one to determine at various times and complexity scales whether the capabilities exist to satisfy the stated needs, or, if gaps exist, what their order of priority is for subsequent consideration in the solutions analysis phase of the process.
An MGA process is envisioned that is jointly funded by and distributed among AF/A2, AFISRA, and Air Combat Command (ACC), and is at various times led by either the AF/A2 or the GIISR CFLI. For example, AF/A2 may choose to lead the process when responding to a “short-fuse request,” from the Office of the Secretary of Defense, the Office of the Director of National Intelligence, or the Congress, to develop and justify a budget position. And the CFLI may be better suited to lead periodic, deliberate analyses of a broader set of needs and gaps on a POM planning cycle. The distributed components of the MGA process include the ISR-CART database, a repository of reusable models that can be accessed by means of metadata stored in the ISR-CART, and the addition of a variety of modeling and simulation and other tools needed for assessing the utility, cost, and risk of ISR capabilities within and across domain perspectives that could become the substrate of an ISR CP&A Analytics Repository. Because the ISR-CART is currently maintained
13J.M. Smith, M. Olivieri, A. Lackpour, and N. Hinnerschitz. 2009. “RF-mobility Gain: Concept, Measurement Campaign, and Exploitation.” IEEE Wireless Communications 16(1):38-44. Available at http://repository.upenn.edu/cgi/viewcontent.cgi?article=1435&context=cis_papers. Accessed March 22, 2012.
by AFISRA, it might be easiest in the near term to build and maintain an initial repository and tool set at AFISRA. Over time, however, a distributed, networked capability is envisioned that can be readily accessed through the Internet by all MGA participants and their designated contractors. For example, one might envision a distributed, physics-based modeling-and-simulation capability that might use communications models maintained by a group in Los Angeles, infotecture models from a group located in San Antonio, Texas, and platform and sensor models maintained by a group in Dayton, Ohio.
The core MGA team would consist of “on-call” subject-matter experts in each of the perspective domains, and an “integrator” with the breadth of skills and experience in Multi-resolution Analysis required to lead and manage the process. Because of the potentially large degree of iterative interactions between the PDA and Needs Analysis steps, it may be beneficial to have the integrator also serve as the PDA lead. Doing so might shorten the lines of communication so as to speed the process, reduce confusion, and manage costs efficiently. The domain experts would most likely not be co-located. And networked elements of the integration team might also collaborate from multiple locations, such as Washington, D.C.; Langley, Virginia; and San Antonio, Texas.
With the many variables associated with implementing an MGA process, it is unclear how much to suggest that the Air Force annually budget in order to fund the MGA team or the purchase and maintenance of models, tools, and so on. It might be best to begin with a small “calibration” project, co-funded by AF/A2 and the GIISR CFLI, that would help them gain an understanding of the major cost drivers and use lessons learned to refine future MGA requirements. Working together, the two organizations would develop a plan and budget for expanding and improving the process over time.
The primary purpose of the Solution Analysis step of the ISR CP&A process, shown in Figure 4-5, is to analyze and recommend materiel and non-materiel solutions that fill prioritized gaps provided by the MGA phase. As shown in Figure 4-5, the Air Force Materiel Command is generally responsible for developing materiel solutions, whereas AFISRA develops non-materiel solutions. Recommended solutions are forwarded to decision makers who work within the Air Force Corporate process to prioritize and seek funds to implement them.
It is important to note that both the AFMC and the AFISRA rely on established processes for assessing and developing solutions. For example, AFMC’s Development Planning community has an established capability planning and assessment process that analyzes solutions, with capability management teams consisting of stakeholders across the science and technology, acquisition, and operational com-
munities. Rather than calling for a reinvention of the process wheel, the committee recommends that the Air Force continue to rely on established AFMC and AFISRA processes to assess and develop solutions for capability gaps. That said, there is a need for a decision-making organization to determine initially whether a particular gap requires a materiel or a non-materiel solution. In addition, such an organization should assign MAJCOM owners to gaps and analysis solutions, serve as an interface and advocate with Air Staff and other stakeholders when required, and generally oversee the analysis of the various materiel and non-materiel solution processes. Because AF/A2 would primarily invoke the ISR CP&A process to seek rapid answers to “what if ” questions that do not require deep solutions analyses, the GIISR CFLI might be the best organization to oversee what can sometimes be protracted materiel and non-materiel solution analyses in support of the development of new and significant capabilities.
TABLE 4-2 The Proposed Process Satisfies the Desired Attributes of an Enterprise-wide ISR Capability, Planning and Analysis (CP&A) Process
|Desired Process Attribute||Proposed Process Capability|
|Encompasses All ISR Domains and Entities|
|Encompasses all ISR missions||
• The Problem Definition and Approach (PDA) step brings the right players to the table to ensure that the process is configured to address all relevant investment questions.
|Addresses all ISR domains||
• The Multi-resolution Gap Analysis (MGA) framework explicitly incorporates and integrates the perspectives of all relevant domains, sources of ISR needs and capabilities.
|Includes all ISR assets in sensor-to-user chain
Collaborates with ISR-related entities
• Sensor-to-user assets, including platforms, communications and command-and-control links, and PCPAD capabilities are modeled and analyzed by means of simulations to answer investment questions in a performance versus cost versus risk trade space.
|Produces Credible Outcomes|
|Provides traceability from process inputs to outputs||
• The multi-resolution framework ties the analysis process to authoritative data sources and the analysis guidance established during the PDA step. It is executed as an interactive, collaborative process among various analysis elements and players to provide a more complete cost, risk, and utility trade-space analysis.
Is repeatable and enduring
• Complex domains of capability are analyzed from different perspectives with tailored models and tools appropriate for each perspective, but with the various segments of the analysis integrated to provide traceability of cause and effect for combined total impact.
|Supports trade-off analyses||
• High-fidelity technical performance measures on sensor/system effects are mapped against specific mission scenarios to assess operational impacts and prioritize gaps.
|Efficiently Uses Limited Resources|
|Is scalable in size, time, and resolution||
• The multi-resolution framework provides quick-look assessments of capability and cost and/or affordability through streamlined applications of cost, risk, and utility trade-space analyses. The framework adds more and more layers of fidelity where needed to refine quick-look assessments over time.
|Reduces labor and cost over time||
• Automated analysis tools, such as modeling and simulation, reduce the number of people and amount of time required to conduct trade-space analyses. Models are refined, stored, and reused to the greatest extent possible to minimize duplication of modeling effort.
NOTE: Acronyms are defined in the list in the front matter.
The recommended process described in this chapter is intended to enhance, not to replace, the current ISR CP&A process in a manner that achieves the desired attributes listed in
Despite the attempt to design a process that enhances rather than replaces the current process, the addition of the PDA and MGA functions would require careful planning prior to their implementation. Further, the Air Force may wish to implement the process changes in a staged fashion designed to minimize disruptions to the ongoing process. The Air Force is urged to roll out the recommended process enhancements by way of a pilot project, or a series of pilot projects, to lay the foundation of a future process that the Air Force can thoughtfully build on over time to achieve the desired end state.
In summary, the value inherent in achieving this end state derives from its ability to effect the following: (1) enhance the quality, transparency, repeatability, and credibility of proposed investments; (2) provide greater insight into cost, risk, and mission utility assessments; (3) scale from quick-look through long-term analyses; (4) expand consideration and analysis of Joint and interagency capabilities; (5) more fully address all ISR domains (air, space, land, maritime, cyberspace); (6) encompass complete “sensor-to-user” chain including PCPAD; and (7) reduce the time and labor required to answer investment questions.