B
Site Visit Summaries
Serving as a significant component of the committee’s information-gathering efforts, from March through May 2021, a subset of committee members conducted in-person and virtual informational site visits at six DoD test ranges and received response documents from two additional ranges. The purpose of the site visits was to gather test range perspectives on the statement of task (Box 1.1). The committee selected test ranges that represented different services. They also sought to gather information from both major range and test facility bases (MRTFBs) and non-MRTFBs.
At each in-person or virtual site visit, candid discussions were held with range personnel. As a result of the open and candid discussion held during these site visits, the committee was able to collect unique data, both qualitative and quantitative in nature, on current and projected operational testing and evaluation (OT&E) challenges. The site visit discussions inform several of the findings, conclusions, and recommendations within this report.
Sample site visit questions include the following:
- What are the top three current OT&E challenges facing your range?
- What are the top three future or projected OT&E challenges facing your range?
- What are the encroachment concerns at your range?
- Are there any issues related to OT&E funding sources?
- What is the whole spectrum of your funding stream, including commercial activities, for FY2020?
Below is a summary of the committee’s site visit discussions and response documents:
ABERDEEN PROVING GROUND
Background.
Aberdeen Proving Ground (APG) is an Army Major Range and Test Facility Base (MRTFB). Under the command of the Army Test and Evaluation Command (ATEC), the Aberdeen Test Center (ATC) at APG serves as the lead test center for unmanned ground vehicles, vulnerability/lethality, automotive/tracked and wheeled, direct-fire systems, small arms systems, direct-fire weapons performance, and littoral warfare.1 APG was selected to examine Army Futures activities, including autonomous vehicles, range for vehicle testing, and the roadway simulator. ATEC, in coordination with ATC, provided a response document to a set of questions prepared by the committee to gain insight into the OT&E issues and challenges at APG. Their responses are summarized below.
Current and Future OT&E Challenges.
The top three current OT&E challenges outlined by APG were:
- Securing adequate funding necessary for sustainment, operational, and modernization costs.
- The ability to expedite data transport, reduction, analysis, and visualization, which requires fiber optic infrastructure and wide area network bandwidth.
- Personnel skillsets to support modeling and simulation, artificial intelligence, machine learning, and cyber infrastructure.
The top three future OT&E challenges for APG include:
- The ability to safely operate robotics and autonomous controls.
- A lack of adaptable test instrumentation, methods, and infrastructure that can be rapidly applied to changing requirements for novel and increasingly complex next-generation weapon systems and accessories.
- The integration of modeling and simulation with traditional live testing.
Connectivity and Security Challenges.
Analysis of secure test data requires the manual transport and processing of classified data from unconnected test events. Connectivity at APG is currently through standard network routers, but there are current investments for fiberoptic
___________________
1 CBRNE Central, “Aberdeen Test Center (ATC),” profile, https://cbrnecentral.com/profiles/name/aberdeen-test-center-atc.
modernization. However, that modernization is years away. APG is currently investigating applicability of cellular technologies (4G/5G).
Funding.
APG noted that ATC is funded as an Army MRTFB activity for developmental test and evaluation (DT&E). Operational test and evaluation (OT&E) is customer-funded activity and is therefore restricted based on the availability of resources. Current resource restraints on developmental testing (DT) directly affect operational testing (OT) support operations, which are increasing (18 OT support operations in the last year). This restraint makes it difficult to support projects like Cross Functional Teams (CFTs) Acquisition Category (ACAT).
Encroachment.
APG noted the following encroachment concerns on operational test activity: climate change, noise, air quality, and spectrum availability.
EGLIN AIR FORCE BASE/EDWARDS AIR FORCE BASE/AIR FORCE RESEARCH LABORATORY
Background.
The Air Force Test Center (AFTC) is overseen by the Air Force Materiel Command (AFMC) and oversees a broad array of test facilities including the 96th test wing at Eglin AFB, the 412th test wing at Edwards AFB, and the Air Force Research Laboratory (AFRL) at Wright-Patterson AFB.2 The 96th and 412th test wings are Air Force MRTFBs. These sites were selected to better understand the interaction between DT and OT and how connectivity is achieved between ranges that are not in geographical proximity to one another. This visit was conducted virtually on April 6, 2021.
Current and Future OT&E Challenges.
The top three current OT&E challenges outlined during the site visit included lack of space to conduct expanding kill chains from launch to target, technical challenges with test infrastructure, and the range capacity with associated scheduling issues. Looking ahead, the representatives viewed the following as future OT&E challenges: (1) Lack of facilities to conduct kill chain testing, including both “on range” and “off range” test infrastructure. The “off range” infrastructure includes facilities for modeling and simulation (M&S), software-in-the-loop (SIL), human-in-the-loop (HITL), and live, virtual, and constructive (LVC) approaches. (2) Cybersecurity testing. The current process is forcing testers to reinvent processes every 16 months.
___________________
2 See https://www.edwards.af.mil/News/Article/394391/afftc-re-designated-as-air-force-test-center.
Problem stems from testers not knowing how code is written. (3) Integrating ranges so that larger tests can be conducted. Several representatives also mentioned the need for a national road map for emerging technology areas that includes funding, maintenance, modernization.
Funding.
Representatives noted funding shortfalls in a variety of areas including, hypersonic investments, maintenance for legacy systems, and the pot of money available for modernization. Long lead times also results in a slow process in which funding must be advocated for in advance, cannot move fast. Finally, there is not sufficient funding to close identified capability gaps, leading ranges to create patchwork-funding arrangements to close capability gaps.
Encroachment.
The range representatives noted that spectrum encroachment and internal encroachment posed the biggest threats to Eglin AFB and Edwards AFB going forward. The representatives provided examples of internal encroachment including the insertion of the 7th Special Operations Forces group at Eglin AFB, the government restricting/taking away telemetry for test, Navy training operations at Fallon and Ramore training in the R2508 airspace, training missions increasingly conducted on test space, and F-16s taking up test space at White Sands Missile Range (WSMR).
MISSILE DEFENSE AGENCY
Background.
The Missile Defense Agency (MDA) is a research, development, and acquisition agency within DoD whose mission is to develop and deploy a layered Missile Defense System to defend the United States, its deployed forces, allies, and friends from missile attacks in all phases of flight. MDA was selected to better understand end-to-end testing across multiple ranges. This visit was conducted virtually on March 19, 2021.
Current and Future OT&E Challenges.
The top current OT&E challenges noted by MDA included high demand for services, a lack of trained personnel, the inability to share information across ranges, and aging infrastructure and telemetry assets. Looking ahead, the representatives viewed achieving automated flight safety system (AFSS) compliance by 2030 as a future OT&E challenges because AFSS requirements are not set by all programs utilizing MDA’s services.
Funding.
The current funding model for MDA was recognized by personnel as one that works very well. MDA operates with Integrated Master Test Plans (IMTPs) that are updated twice per year. “.0” informs the program
objective memorandum (POM), and “.1” informs the President’s budget. The funding requirement includes ranges and flight test rates. Funding includes both research, development, test, and evaluation (RDT&E) for development, operations and maintenance (O&M) for fielded systems, and military construction (MILCON) for both mission and support. The resources provided include both fixed and variable, with the fixed resources generally remaining the same (predictable) in requests and budgets.
Encroachment.
MDA noted spectrum encroachment, particularly in S-band, as a primary encroachment concern. Representatives also noted that Alaska Aerospace, formerly Kodiak test site, is facing encroachment because of supplemental environmental assessment restrictions.
NATIONAL CYBER RANGE COMPLEX
Background.
The National Cyber Range Complex (NCRC) was created as a Defense Advanced Research Projects Agency (DARPA) function and is now overseen by the Test Resource Management Center (TRMC). The committee was able to engage with NCRC Director AJ Pathmanathan in a virtual setting to discuss the connection between the cyber range and operational test. This visit was conducted virtually on March 24, 2021.
Current and Future OT&E Challenges.
NCRC identified four key current OT&E issues: (1) a lack of a trained cyber workforce; (2) a lack of requirements and funding for programs to use a cyber range; (3) compatibility issues are arising from the fact that next-generation systems need to be integrated with older programs, such as Microsoft Windows 98; and (4) an increased need to work with agencies in the intelligence community to close the loop on integrating current threat intelligence into the virtual test environment.
Funding.
NCRC noted that cyber testing will become an unfunded mandate if a funding stream for cyber OT is not established. Currently, NCRC tests are paid for by programs that have extra funding available to come to the cyber range for testing.
Encroachment.
Encroachment was not identified as a concern during the NCRC site visit.
NEVADA TEST AND TRAINING RANGE
Background.
The Nevada Test and Training Range (NTTR) is an Air Force MRTFB. NTTR provides training, test, and developmental testing area and
hosts test operations of the 98th Test Wing and other customers for OT&E test. The committee was able to engage in a full-day virtual discussion with individuals from various NTTR elements including plans and programs, operations, financial management, physical security, range support, and program management. This visit was conducted virtually on April 13, 2021.
Current and Future OT&E Challenges.
Managing increasing capacity was noted as a major current challenge at NTTR. Additionally, personnel indicated the need to better anticipate future program requirements. A possible solution mentioned was connecting earlier with the DT community to learn about and coordinate test requirements. Another challenge is data transmission speed, with personnel noting that customers may have to wait 50-60 days to receive data results from their test event. Future OT&E challenges included a lack of understanding as to how to incorporate artificial intelligence (AI) into test events, the lack of definitions for MDOs, and the need to purchase radar arrays that are not run on proprietary software. A final future OT&E issue mentioned was that the stove piping of programs, meaning the lack of coordination and integration across programs and missions, makes MDO testing difficult to perform.
Funding.
Personnel characterized the funding methods for NTTR as generally good. Funds for future investment in infrastructure and instrumentation are provided by Air Combat Command. Personnel did note that RDT&E funding provides more flexibility and there is a need for more RDT&E funding going forward.
Encroachment.
NTTR personnel pointed out multiple areas where encroachment is impacting operations on the range. First, a recent land withdrawal strategy that was rejected resulted in the loss of test space for NTTR.3 Additionally, NTTR’s 2.9 million acres are protected by fencing and require upkeep and audits. Spectrum encroachment also affects the NTTR operations. NTTR no longer receives request for GPS jamming tests because of Federal Aviation Administration (FAA) restrictions. Finally, internal encroachment creates issues for test and training events at NTTR. Foreign partners frequently utilize range space to test aircraft like the F-35 Joint Strike Fighter. The presence of countries such as the United Kingdom, Singapore, Italy, and Australia creates security issues and results in the rescheduling of test events.
___________________
3 Proposal to Withdrawal and Reservations of Public Lands in Nevada to Support Military Readiness and Security, https://fas.org/man/eprint/ndaa-2021-prop/04172020-nevada.pdf, accessed June 4, 2021.
ATLANTIC TEST RANGE AND AIR COMBAT ENVIRONMENT TEST & EVALUATION FACILITY AT PATUXENT RIVER
Background.
A small subset of the committee was able to travel to the Atlantic Test Range (ATR), a Navy MRTFB, and the Air Combat Environment Test & Evaluation Facility at Patuxent River. The committee sought to better understand integrated physical/virtual testing, as well as operations, financial management, range support, and program management. Of particular interest to the committee was the Joint Simulation Environment (JSE), which provides a high-fidelity modeling and simulation environment to conduct testing on fifth-plus generation aircraft and systems. This visit was conducted in person on March 10, 2021.
Current and Future OT&E Challenges.
The current OT&E challenges noted by ATR included (1) the need to have greater test coordination to ensure the most effective use of range time; (2) the lack of a centralized database and repository for M&S and threat references for OT&E; (3) multi-level security and classification raises issues for cross platform testing; and (4) the integration of M&S with OT&E. Projected challenges include the capabilities to test AI systems.
Funding.
ATR noted that there were several issues that arose from securing resources for modifying range infrastructure. Representatives from ATR pointed out that the POM process is not versatile enough and that MILCON is not approved fast enough. ATR personnel indicated that they had available resources for supporting range modernization needs, but were constrained because of strict limitations on mixing investment streams.
Encroachment.
Spectrum encroachment was noted as a primary concern.
POINT MUGU SEA RANGE
Background.
The Point Mugu Sea Range is a Navy MRTFB selected by the committee because of its electronic warfare testing capabilities. This site visit was conducted virtually on April 7, 2021.
Current and Future OT&E Challenges.
Personnel from Point Mugu noted that current OT&E issues stem from costly test failures that result from losing or not having enough targets/kill removal systems on the range. These test failures can cost up to $5 million per target lost. The second challenge noted was the struggle to maintain cybersecurity in an ever-changing cyber environment. Future concerns for OT&E include advancements in hypersonic test, multi-level security, infrastructure modernization, keeping pace with advancements in directed energy, lack of
understanding for authority on cross-domain testing, and range demand exceeding capacity. A final note by personnel was that many infrastructure upgrades are paid for by program customers. This means that the new infrastructure is locked to the customer that paid for them. Point Mugu therefore can have modern infrastructure but cannot utilize it for any other tests outside the program that paid for the infrastructure.
Funding.
Point Mugu personnel stated that there were large unfunded requirements that lead to cascading funding issues. While Point Mugu does receive MRTFB funding, 85 percent of those funds go to personnel costs. The Investment and Modernization (I&M) budget covers high-cost upgrade items, but about a third of these funds go directly toward maintenance on existing infrastructure. Point Mugu personnel stated that the result of these funding issues was that customers bear the brunt of aging infrastructure costs.
Encroachment.
Not identified as a major concern.
VANDENBERG SPACE FORCE BASE
Background.
Previously the Vandenberg Air Force Base, Vandenberg now supports the United States Space Force (USSF). Vandenberg is a space launch base for USSF, but it also provides space launches for commercial entities and non-defense agencies. The committee selected Vandenberg as a site to understand the complications to future testing for USSF. Vandenberg provided a read-ahead document in response to questions provided by the committee.
Current and Future OT&E Challenges.
The current challenges outlined in the response document highlighted how USSF priorities do not always align with DoD T&E. This leads to conflicting priorities for test events. An additional challenge noted was the deconflicting of activities with the Navy sea ranges on the West Coast.
Funding.
The response document stated that the current range costs are high and unsustainable. Representatives stated that this is the result of the 2003 National Defense Authorization Act (NDAA) change to the direct cost only model.
Encroachment.
Vandenberg personnel shared operational security concerns arising from commercial test, limitations imposed on the electromagnetic spectrum, and the loss of air and sea space for test operations in recent years.
