2

An Envisioned Future of Operational Test and Evaluation

In its statement of task the committee was asked to assess the physical and technical suitability of the U.S. military’s system of operational test and evaluation (OT&E) and to provide recommendations on addressing any deficiencies in that system relative both to existing technologies and to any technologies expected to arrive over the next decade and a half. To guide itself in that task, the committee developed a vision of what an ideal OT&E system would look like in 2035. Working with this envisioned future allowed the committee to be more methodical and consistent in identifying the ways in which today’s military ranges could be improved and helped in developing recommendations for how best to achieve that desired state. While the vision is by necessity incomplete and will certainly have failed to anticipate some future developments, the committee believes that moving in the direction of this envisioned future will produce a future OT&E that is characterized by agility, flexibility, and speed.

This chapter paints a vision for the future that provides foundation and context for the remainder of the report. In the following chapters the committee analyzes in detail the challenges facing today’s military ranges, offers its conclusions about the current state of OT&E, and provides recommendations for improving the ranges and preparing them for the challenges they will face in the coming years. Before that, however, it is important to pull back and see the “big picture” of what the nation’s military test ranges could ideally become.

The description of OT&E’s envisioned future takes place in three steps. The first is a vision of what warfare in the future is likely to look

like, particularly warfare with a near-peer or peer adversary. This is an exercise that has been carried out at multiple times by multiple groups, and the committee has relied on such outside work for its vision of the future of warfare. From there, the next step is to envision what a system of OT&E would need to look like to help produce weapon systems that would operate successfully in such conflict. A crucial principle here is “test as you fight”—that is, the testing of weapon systems should take place in an environment that is as close as possible to the environment in which they will actually be used. This principle, combined with a vision of how future warfare will be conducted, leads to a vision of what military ranges should look like in this future. Finally, working from the joint visions of future warfare and future OT&E, this chapter describes an envisioned future for the organizational and funding structures necessary to enable the future ranges and testing. Thus this chapter will lay out an envisioned future in three steps: future warfighting, future military ranges and OT&E, and the future enablers of such testing.

THE FUTURE OF WARFIGHTING

How will wars be fought in the coming decades? Much has been written on this topic from defense analysts and think tank scholars working to anticipate the military’s needs. This section focuses on the parts of that future that will pose the greatest challenges to the nation’s military ranges.

Novel Weapons and Domains

Weapons and military technologies have steadily become more sophisticated and more effective, but few, if any, eras in history have seen the pace of changes in military systems being experienced today. Not only new weapons, but entirely new classes of weapons are in use or under development, leading to fundamental changes in the nature of warfighting.

Consider hypersonic weapons, which are capable of traveling vast distances through the atmosphere at high Mach numbers and able to maneuver in order to avoid standard anti-missile defenses. Hypersonic weapons have capabilities that are fundamentally different from previously existing weapons and thus require new systems and strategies to counter them (Stone, 2020; Vergun, 2020).

An even more revolutionary transformation is promised by artificial intelligence (AI) and autonomous systems (Hoadley and Sayler, 2020; NSCAI, 2021). Powered by the dramatically increasing speed and power of digital technologies, AI is already applied in a wide variety of areas,

both military and civilian, and the number of applications is expected to grow in coming years. In particular, many weapons and military systems in the future will be “smart”—able to carry out many functions without human input. Military applications will range from the rapid processing of intelligence data to the control of autonomous vehicles and systems, which will in some cases be given the independence to make battlefield decisions without direct human control.

Both AI and hypersonics reflect the increasing speed of warfare, and their expected integration may prove very powerful. As stated in a 2019 RAND report, “The pace of war can exceed the speed at which humans can observe what is happening, conceptualize a strategy, and deliver commands” (Winkler et al., 2019, p.13), which will drive the need to develop and test approaches for automated decision making. Although it is impossible at this point to predict exactly which AI technologies will play a significant military role, AI and autonomous systems are already spearheading fundamental shifts in military conflict.

Furthermore, warfighting in the future will be marked not only by new types of weapons, but by new domains of warfare. In addition to the traditional military domains of air, sea, and land, the emergence of the cyber domain as a contested space has been recognized for more than a decade (CSIS, 2021). There have been opening gambits made, such as the covert introduction of programs into the U.S. electric grid, which may have been made in anticipation of launching serious, large-scale cyberattacks in the event of a major conflict (Gorman, 2009). Recent ransomware attacks on the Colonial Pipeline Co. and JBS shut down fuel pipelines and meat packing plants, respectively, highlighting how cyber vulnerabilities create national security threats that extend to the nation’s energy and food supplies (Lane, 2021; Williams, 2021). Conflicts in the future are likely to include cyber strikes not only on military forces but on their civilian infrastructure, including communications, power, and transportation. It is even possible that an adversary might engage in such a widespread cyberattack on infrastructure without attacking with more traditional military weapons in the hope of destabilizing an opponent while reducing the risk of triggering a full-scale war (CLAWS, 2020).

Another increasingly contested domain is space. Communication and observational satellites provide precision navigation and timing, which play major roles in modern conflicts, and thus they are targets for adversaries seeking to disrupt an opponent’s communications or limit the opponent’s ability to monitor multiple locations on or above the earth’s surface. This in turn has led to a growing focus on both offensive and defensive weapons that can be deployed in space, with the United States officially creating the Space Force, its first new military service since the creation of the Air Force in 1947, and other countries also placing a new

emphasis on space as a domain of military operations (Spirtas et al., 2020). However, as noted at the January 2021 workshop by the acting director of OT&E, Raymond O’Toole, a priority gap for the U.S. Department of Defense (DoD) testing community is the lack of a dedicated range for testing space weapons.

In short, warfighting of the future will involve not just newer and more sophisticated weapons, but new classes of weapons and new domains, signaling the emergence of entirely new approaches to conflict. Furthermore, it is clear that the changes in weapons and military systems in the future will take place at an increasingly rapid pace, bringing new technologies into play at a rate that will be unlike anything that has been seen before. This will be particularly true for digital capabilities, such as AI-enabled systems and the software used in data analysis, command and control systems, other software-intensive aspects of the military enterprise, and even the design and development of new warfighting technologies themselves.

Multi-Domain Operations and Kill Chains

While new weapons and new domains will shape the future of warfighting, an even more transformative factor is the ongoing combination of systems across multiple domains to create an integrated fighting force that is greater than the sum of its parts. The vision for such multi-domain operations is that they will take advantage of rapidly improving communication capabilities and increasing computing power to tie together many different systems, from detection to strike, in such a way that the different pieces act in a coordinated manner, react quickly to changing conditions, and overpower adversaries through the combination of their forces.

The next chapter will dive into deeper detail on multi-domain operations (MDOs), but broadly, the operational view of MDOs is that platforms in different domains share information to accomplish an objective or set of objectives in a given combat scenario. In particular, the kill chain will be apportioned across different elements in different domains, which is in sharp contrast to the traditional kill chain that existed prior to the information revolution. A traditional kill chain is generally contained within a single platform, such as a fighter pilot detecting where an enemy aircraft is, deciding what should be done about it, and then carrying out that action (Clark et al., 2019). In recent years, this traditional kill chain has expanded to include different components; it has become somewhat common, for instance, for data from an observation drone to be sent back to a command-and-control center, which then orders a strike by a stealth fighter. But in the coming years, with the addition of new

sensing capabilities, new weapon systems, and new domains of warfighting, the kill chain is only going to become increasingly more complex and sophisticated.

Ultimately, the effectiveness of kill chains and the success of MDOs will depend on how well integrated the various components of an operation are. As stated in the 2018 National Defense Strategy, “Success no longer goes to the country that develops a new technology first, but rather to the one that better integrates it and adapts its way of fighting” (DoD, 2018, p. 10).

In short, warfare in the next 15 years or so can be expected to have the following attributes that will make it different from warfare today: Weapons will be more sophisticated, more complex, and more effective. A variety of new technologies will play a role, from hypersonic weapons to autonomous systems. The domains of cyber and space warfare will be part of the picture. The pace of weapons development will continue rapidly increasing, with new technologies brought online more quickly than in the past. And different technologies and domains will be more tightly integrated than in the past.

In the committee’s envisioned future, the testing-and-evaluation community will engage in strategic planning efforts along these mission threads, assessing the ability to test both evolutionary and revolutionary advances in technology at a speed that assures continued military advantage. To understand how this works, consider the test and evaluation assessment framework shown in Figure 2.1.

The horizontal axis in this framework indicates testing scale. At the far left is component-level testing that addresses specific subsystems within a given system, or foundational military technology such as radar signal processing. Further along the axis are platforms, such as avionics systems, weapons platforms, and autonomous vehicles. On the far right end of the scale are systems of increasing complexity and number of elements, including the sorts of systems expected in the future, such as human–machine teams operating in consort to achieve mission objectives (Winkler et al., 2019). Current test range capability is well designed to address the component level, and platform level test requirements as new sensors, weapons and vehicle upgrades are worked through the acquisition pipeline (Dahmann et al., 2010). Future test ranges must include this current capability as well as the ability to test larger-scale system of systems.

The vertical axis corresponds to the nature of the new technology under consideration, from incremental improvements to existing capability to novel game-changing technologies that could disrupt military operations. Many of the technologies predicted to come online in coming decades fall into the revolutionary technology category and have the

potential to upend military conventions. These capabilities include multi-domain distributed sensors, complex emitters, and hypersonic weapons as well as future cyber- and space-based defensive and offensive systems.

Ensuring that the U.S. military will be successful in future conflicts requires the ability to operate—that is, to effectively carry out testing and evaluation—in the upper right hand corner in this landscape. In the committee’s envisioned future, DoD testing and evaluation is a driver in this strategic discussion, working with research and development organizations to explore how a new technology will be tested and carrying that perspective through the system life cycle to streamline the development, evaluation, and fielding of new capabilities.

THE ENVISIONED FUTURE OF MILITARY TEST RANGES

Given this vision of future warfighting, what changes will be required for the nation’s military test ranges in order to prepare for it? From its discussions with Test Resource Management Center (TRMC) leadership, DoD stakeholders, leading scientists, and military personnel, the committee contends that a paradigm shift in testing approach will be necessary in

order to reach the appropriate future. In particular, ensuring the nation’s future warfighting abilities will require integrated capabilities, MDOs, and the seamless adoption of new technologies at the speed of innovation.

The nation’s test ranges have never been static, and in coming years their capabilities and infrastructure will need to be refreshed periodically—just as always has been the case. However, the ranges are also facing new challenges unlike any they have faced before, and these will require responses that are also novel. One such challenge is to modernize test range capabilities to match the rapidly increasing pace of technological innovation, particularly in digital technologies such as AI, autonomous systems, and cyber warfare. There are essentially no areas of technology that have been untouched by this accelerating innovation. If the speed of testing and evaluation does not match the speed of innovation, then the testing can serve as a chokepoint, significantly slowing the pace at which new systems can be brought into service and potentially putting the nation at a disadvantage to others who are able to innovate and field faster. Alternatively, testing needs may get neglected, thus passing risk to the operational users and leaving decision makers uninformed about the capabilities and limitations of their systems.

A second—and perhaps more fundamental—challenge will be to test new weapons and systems as part of larger and more complex operations in a way that mimics how the weapon systems will be used in real-world situations. This is not something that has been an emphasis for military ranges in the past. Historically, most of OT&E has been focused on individual weapons and systems and making sure that they work as they are supposed to under conditions similar to those that would be encountered in combat. Although operational plans may involve the use of multiple platforms working together, the capability and understanding of that system of systems integration has often not been a major design or test requirement. With the information revolution, however, the effectiveness of the connections between systems has become more critical than the capability of any one system acting alone.

Given these trends and forces, the committee offers a vision of the military test range of the future (Figure 2.2), and the following discussion delves into some of the details of that vision.

Testing New Technologies

Ideally, by 2035 a close operational testing (OT) and developmental testing (DT) partnership will drive a holistic approach that allows TRMC to address the increasing pace of technological development by developing a set of approaches to testing at a pace that matches that development. For example, TRMC will have collaborated with various research and

development organizations within the Department of Defense (DoD)—the Defense Advanced Research Projects Agency (DARPA), the Air Force Research Laboratory (AFRL), the Office of Naval Research (ONR), the Army Research Laboratory (ARL), etc.—to identify effective test approaches and standard metrics that can be carried through the life cycle for developing and evaluating critical emerging and new technologies. This approach may require giving TRMC the authority to hire staff directly, akin to DARPA and other DoD laboratories in order to support the full life cycle view of test planning efforts.

Test ranges will have used a “TestDevOps” approach that connects developmental model-based testing with OT&E real-world scenarios in order to rapidly field and iterate on advances in new technology. A TestDevOps model is one that merges the testing and operational communities in evaluating, refining, and deploying agile solutions to the field. More specifically, TestDevOps takes advantage of the traditional aspects of the DoD test enterprise (i.e., traditional component-level testing to determine if a unit under test has met specific performance requirements plus system-level testing to assess overall use as a DoD capability) into operational scenarios based on advanced modeling and simulation (M&S) as well as real-world multi-domain exercises to determine how a capability will be used as part of a human–machine teamed solution. The user input on the performance of the system in real-world scenarios is then used to inform any system updates and to facilitate release to the field.

In the time between the release of this report and 2035, TRMC and the services will have defined the range requirements for various new capabilities, such as the space systems test range and ranges for testing hypersonic missile technologies, and they will have used that framework as a model to retrofit the test ranges of 2021 to bring them in line with future needs; the requirements in that new framework will have addressed test approaches, computational infrastructure, TestDevOps approaches, the balanced use of M&S versus real-world testing, and networking and data interoperability.

Testing Kill Chains and Multi-Domain Operations

In 2035, as envisioned by the committee, the nation’s military ranges will be fully capable of testing kill chains and multi-domain operations, as OT&E is dedicated to the principle of “test as we fight.” By 2021 there had already been initial steps toward this end, such as the combined Orange Flag–Black Flag large force test event that took place on March 2–4, which allowed “for improved integration and the combining of resources and participants to provide better test data and a more robust operationally relevant environment” (Saunders, 2021). An Emerald Flag event was

introduced in December 2020 which provided a realistic operating environment linking ground, air, and space systems together to demonstrate joint and multi-domain operational capabilities while identifying tangible shortcomings to these systems (Rodriguez, 2020). A look at a few of the details of that operation offers an indication of what is involved in that operation.

Orange Flag test events are focused mainly on technical innovation and integration, and that particular event in March 2021 examined integrated kill chains—or “kill webs,” as they were referred to—which involved sensors and tactical networks from the Air Force, Army, Navy, Marine Corps, and Space Force connected via current command and control capabilities that will evolve into future Joint All-Domain Command and Control (Saunders, 2021). Black Flag events, by contrast, focus on ways to improve tactics—“tactic improvement protocols”—for existing weapons and systems. In the joint test event, Orange Flag and Black Flag “combined their mission planning processes and streamlined test objective synthesis,” although the execution of the Orange Flag and Black Flag tests was actually separate (Saunders, 2021). The Emerald Flag events are multi-domain test exercises that incorporate technology and prioritize efficiency for joint warfare through rapid data-driven analysis.

Over the next decade and a half, according to the committee’s vision, this sort of testing of end-to-end kill chains over multiple domains will have become much more commonplace and much more highly integrated. “Testing as you fight” will require the testing of multiple integrated technologies acting in concert in a way that closely mimics what would happen in real combat.

It should also be noted that, in this envisioned future, the data from these tests of kill chains and multi-domain operations will be collected and analyzed in such a way that makes it possible to pinpoint weaknesses and failures—which can actually be quite difficult in integrated tests involving multiple components interacting in complex ways. This approach will prove valuable for high level operational test objectives, and will require further iteration when complex technical calculations and required to configure test instrumentation. Furthermore, because of the value of M&S to these sorts of tests (see next section), OT&E will have implemented a policy that a key objective of every end-to-end kill chain test is to produce data for use in the validation of simulation models and dynamics.

Modeling and Simulation

In the envisioned future, as today, it will not always be practical to carry out physical tests of a technology. In some cases—such as a nuclear bomb or a weapon designed to disable computers or communications—it

is simply too dangerous or disruptive to realistically test military technologies. In other cases it may be critical to control what adversaries observe about U.S. military systems in action. In still other cases it may simply be too expensive to carry out the number of tests required for a full evaluation of a technology. In all of these cases and others, simulated tests using a model of the technology is often the best option. And, with the increased computing power available in the future and the more complex and sophisticated technologies that must be tested, M&S will play a much more integral role in OT&E.

By 2035, according to the committee’s vision, M&S will have been integrated into nearly all operational test and evaluation activities. While the promise of M&S capabilities has fallen short in some attempts to date, the science continues to evolve and improve and it is necessary that M&S remains a DoD focus area committed to making this approach work. There will be a tight interplay between physical testing and M&S, with the physical tests providing data to guide the development of or validate the models, and the M&S indicating which particular aspects of a system should be tested and what data should be collected. Planning for M&S will begin early in the concept-development phase in order to support design decisions during development and subsystem integration, with later integration into developmental and operational test programs, campaign-level exercises, and, eventually, operational sustainment. The models will have evolved over the course of the program and will be adaptable for use at different levels within DoD, providing the required capability at each. Early digital engineering development in the form of model-based systems engineering (MBSE) will address a system’s overall requirements, structure, behavior, and data input and output interactions that will aid in viewing composability of a system—a system design principle that deals with the interrelationships of components (SBIR, 2014). A highly composable system provides components that are selected and assembled in various combinations to satisfy specific user requirements. Models within simulations will be developed that are consistent with the MBSE development, resulting in M&S that is relevant throughout the T&E life cycle.

Furthermore, M&S will not be used just in the testing of individual technologies, but also for integrated systems. Models will be used to emulate other technologies that interact with the system under development or elements of the system under development that will not be fully functional in time to exercise other subsystems on the critical path. Certain models that are used over and over again, such as models that represent DoD infrastructure or adversary equipment or threats against which multiple development systems will be tested, will be maintained as part of a common test infrastructure. More generally, there will be a widely shared

and accessible M&S ecosystem that includes common scenarios, models, and data that can be used by concept developers, requirements developers, research and development (R&D) programs, and acquisition programs. This M&S ecosystem will have spread across not just DoD but also industry, so that industry models can be used by the military and DoD models can be used to support digital engineering by industry partners.

Also, in this envisioned future, M&S will be carried out according to a set of widely accepted standards that address data interoperability and other issues that result from the growing complexity of the systems being modeled. The M&S architecture can adapt to the evolution of testing programs, including uncertainty quantification. The future must include the ability to design, develop, and test data storage at unprecedented speeds in trusted cross-domain architectures at TTS/SCI/SAP (Top Secret/Sensitive Compartmented Information/Special Access Program) down to unclassified levels. Chapter 4 provides a deeper rationale behind this vision.

Data Sharing, Repositories, and Accessibility

In the envisioned future, military ranges and other development and testing facilities will have the bandwidth and connectivity to share data and models and enable rapid data analysis across multiple classification levels. To effectively handle data from testing, program managers work early in the design phase with the ranges to develop a data strategy to inform operational testing. Some of the improvement will be due to the presence of a properly curated and protected data and model repository that will be widely accessed by those in the testing and evaluation community. A second factor is the development of data protocols for the real-time transfer of data at appropriate classification levels and the increased interconnectivity of ranges.

More generally, the T&E system will have the instrumentation, telemetry, data collection, data handling, and data analysis capabilities necessary to collect, transfer, store, and manipulate the huge amounts of data that are generated by the many different types of sensors observing tests. The capacity will exist to handle new and emerging types of data-collection systems, including those that are not stationary, such as instruments about wave runners, aircraft, and satellites. Systems will have been developed to collect and transfer data from tests carried out in integrated environments across multiple ranges.

ENABLING THE ENVISIONED FUTURE OF MILITARY RANGES

Given this vision of the future of OT&E—which is significantly different from the present version of OT&E—what sorts of changes will have

been made by 2035 in policies, requirements, acquisitions, and funding to enable this new approach to testing? The committee envisions many such changes.

Critical Joint Mission Threads

In the envisioned future of 2035, it will be understood there are a few mission threads so important to DoD that their execution should be tested in a coordinated fashion. These are referred to as “critical joint mission threads,” which are end-to-end sets of activities and systems that accomplish the execution of a joint mission.¹ Most future advanced technologies are integral to these critical joint mission threads and therefore require a new perspective on testing and evaluation, including how tests are planned, funded, and used to drive new technology into the field.

The test planning for these critical joint mission threads will not be owned by a program, but by a larger organization empowered by the J-8 Directorate of the Chairman of the Joint Chiefs of Staff with the responsibility for integrated test and evaluation for that particular mission thread. This “joint program office” will have a larger set of responsibilities beyond integrated testing: It will serve as the certifying authority for DoD’s capability to perform critical joint mission threads, it will perform mission engineering for its mission thread, it will develop an authoritative operational and top-level systems architecture for the mission, it will develop and analyze integrated requirements for the participating systems, and it will inform acquisition decision milestones for participating programs.

Essentially, such a joint program office will use testing and evaluation as a mechanism to foster agile acquisition and development in its mission area. To support agile “TestDevOps,” the joint program office will drive an integrated set of development and test activities, integrating live test and modeling and simulation to take advantage of the strengths of each. It will steward and fund persistent range capabilities and M&S repositories, along with a test data repository, and it will enable access to these repositories across the services, programs, and industry partners.

Funding and Acquisition

The envisioned future for T&E will have funding and acquisition mechanisms that represent improvements in various ways over the current system, such as a resolution to complex and disconnected funding

___________________

¹ “Critical joint mission threads,” as defined in the Defense Acquisition University glossary, https://www.dau.edu/glossary/Pages/Glossary.aspx#!both|J|27776, accessed June 16, 2021.

streams; flexibility to apply investments from different sources to priority needs; predictable multi-year funding with adequate authorities to obligate/expend at the right timing for T&E needs; adequate funding for ongoing sustainment and modernization of existing capabilities; and funding approaches that are well suited to software-intensive test systems. Chapter 5 recommends a pilot program to build closer cooperation between the OT and DT communities that includes a new process for funding ranges and infrastructure to make it simpler, more responsive, and more effective. Still, there are a few things that can be confidently stated about the envisioned future of funding and acquisition.

For example, as part of the coordination between development and testing, DOT&E will be included early in the acquisition process in order to coordinate testing requirements and to collaboratively identify shortcomings in testing capabilities. Furthermore, T&E funding streams will be established early in the development process to ensure the ranges will be ready to do appropriate testing when the system being developed is ready to be tested, and operational and developmental testing requirements will be synchronized early in the acquisition process. Additionally, TRMC will have set out a mutual capability requirement process that is more responsive to emerging technology testing than it was in 2021.

The process for funding testing and evaluation ranges and infrastructure will be simpler in this idealized future, and barriers to range modernization will have been identified and, where possible, removed. Better mechanisms will have been developed for funding range maintenance and the development of new capabilities to meet emerging technology requirements. There will be a working capital fund for the ranges. And better funding mechanisms will have been identified for software-enabled capabilities and the maintenance of software over time.

In the specific area of M&S, in the envisioned future, requirements for the full hierarchy of M&S to support a system through its entire life cycle will be accounted for and funded during early concept development. M&S will be persistent so that it supports nearly all life cycle activities, from concept and requirements development through operational testing and sustainment. To enable this, the necessary resources will have been provided to support M&S with a stable funding profile, and requirements will be established for specific M&S capabilities to support development decisions and integration with the test program.

Finally, given the importance of testing kill chains and multi-domain operations and the lack of any natural home for these activities, the envisioned future includes a joint program office to support connected concurrent kill chain operations as an OT&E activity. This activity provides support for enterprise-level simulation environments; identifies dedicated funding for the development, sustainment, and management

of T&E data and a model repository; and includes test infrastructure for assessing the integration of a new capability as part of the development of that new capability, not as a separate effort. Such a system ensures that range capabilities can support the operational assessments of concurrent kill chain operations.

Mitigating Range Encroachment

In the envisioned military range system of the future, range encroachment will continue to be a concern and constant issue, but there will be improvement in the use of mitigations that will preserve DoD test capability. DoD will preserve dedicated frequencies for weapon and threat testing, and will have more capability to efficiently manage and utilize dedicated spectrum and have capability to continue some test operations in areas of shared spectrum. The potential for physical encroachment will still exist but issues of internal encroachment will be addressed and managed to prevent impact to test operations. And better range management will have led to fewer test failures caused by issues with emitters from other tests.

This envisioned future offers a preview of the remainder of the report. In the following chapters the committee offers its analysis of the current state of the systems of military ranges, including both its strengths and weaknesses, describes specific changes that could be made to improve those ranges and prepare them for 2035, and then provides specific recommendations on how those changes could be accomplished. But the end goal of those recommendations is right here in this chapter—with the vision of what OT&E could be and should be a decade and a half from now.

REFERENCES

Gorman, S. 2009. “Electricity Grid in U.S. Penetrated by Spies.” Wall Street Journal, April 8. https://www.wsj.com/articles/SB123914805204099085.

Hoadley, D.S., and K.M. Sayler. 2020. Artificial Intelligence and National Security. Congressional Research Service Report R45178. https://crsreports.congress.gov/product/pdf/R/R45178/10.

Kushner, D. 2013. “The Real Story of Stuxnet.” IEEE Spectrum. February 23. https://spectrum.ieee.org/telecom/security/the-real-story-of-stuxnet.

Lane, S. 2021. “JBS Attack Unlikely to Cause Major Meat Disruption: USDA.” The Hill, June 3. https://thehill.com/policy/finance/556696-jbs-attack-unlikely-to-cause-major-meat-disruption-usda.

NASEM (National Academies of Sciences, Engineering, and Medicine). 2018. Multi-Domain Command and Control: Proceedings of a Workshop—In Brief. Washington, DC: The National Academies Press.

NSCAI (National Security Commission on Artificial Intelligence). 2021. Final Report. https://www.nscai.gov/wp-content/uploads/2021/03/Full-Report-Digital-1.pdf.

Rodriguez, K. 2020. “Emerald Flag Exercise Begins.” Wright-Paterson AFB News, December 1. https://www.wpafb.af.mil/News/Article-Display/Article/2432103/emerald-flag-exercise-begins/.

Saunders, C. 2021. “Test like We Fight: ‘Orange Flag,’ ‘Black Flag’ Collaborate to Accelerate Change.” Edwards News, March 8. https://www.edwards.af.mil/News/Article/2537319/test-like-we-fight-orange-flag-black-flag-collaborate-to-accelerate-change/.

SBIR (Small Business Innovation Research). 2014. “Predictive Modeling Tools for Metal-Based Additive Manufacturing.” Department of Commerce. https://www.sbir.gov/content/predictive-modeling-tools-metal-based-additive-manufacturing-0.

Sick, A. 2018. “Looking Beyond Your Service for Multi-Domain Success.” Over the Horizon. December 24. https://othjournal.com/2018/12/24/oth-anniversary-looking-beyond-your-service-for-multi-domain-success/.

Spirtas, M., Y. Kim, F. Camm, S.M. Ross, D. Knopman, F.E. Morgan, S.J. Bae, M.S. Bond, J.S. Crown, and E. Simmons. 2020. A Separate Space: Creating a Military Service for Space. RAND Report RR-4263-AF. Santa Monica, CA: RAND Corporation. https://www.rand.org/content/dam/rand/pubs/research_reports/RR4200/RR4263/RAND_RR4263.pdf.

Stone, R. 2020. “‘National Pride is at Stake’: Russia, China, United States Race to Build Hypersonic Weapons.” Science, January 8. https://www.sciencemag.org/news/2020/01/national-pride-stake-russia-china-united-states-race-build-hypersonic-weapons.

Vergun, D. 2020. “Shortfalls of Defensive Hypersonic Weapons Must be Addressed, NORAD General Says.” DoD News, October 29. https://www.defense.gov/Explore/News/Article/Article/2399093/shortfalls-of-defensive-hypersonic-weapons-must-be-addressed-norad-general-says/.

Williams, B.D. 2021. “Colonial Pipeline Cyberattack Follows Years of Warnings.” Breaking Defense. May 10. https://breakingdefense.com/2021/05/pipeline-cyberattack-follows-years-of-warnings/.

Winkler, J.D., T. Marler, M.N. Posard, R.S. Cohen, and M.L. Smith. 2019. Reflections on the Future of Warfare and Implications for Personnel Policies of the U.S. Department of Defense. RAND, Washington, DC. https://www.rand.org/pubs/perspectives/PE324.html.