Necessary DoD Range Capabilities to Ensure
Operational Superiority of U.S. Defense Systems
Testing for the Future Fight
Committee on Assessing the
Physical and Technical Suitability of
DoD Test and Evaluation Ranges and Infrastructure
Board on Army Research and Development
Division on Engineering and Physical Sciences
A Consensus Study Report of
THE NATIONAL ACADEMIES PRESS
Washington, DC
www.nap.edu
THE NATIONAL ACADEMIES PRESS 500 Fifth Street, NW Washington, DC 20001
This activity was supported by Contract W911NF-18-D-0002 with the Office of the Secretary of Defense. Any opinions, findings, conclusions, or recommendations expressed in this publication do not necessarily reflect the views of any organization or agency that provided support for the project.
International Standard Book Number-13: 978-0-309-49857-9
International Standard Book Number-10: 309-49857-0
Digital Object Identifier: https://doi.org/10.17226/26181
Limited copies of this report may be available through the Board on Army Research and Development, 500 Fifth Street, NW, Washington, DC 20001; (202) 334-3111.
Additional copies of this publication are available from the National Academies Press, 500 Fifth Street, NW, Keck 360, Washington, DC 20001; (800) 624-6242 or (202) 334-3313; http://www.nap.edu.
Copyright 2021 by the National Academy of Sciences. All rights reserved.
Printed in the United States of America
Suggested citation: National Academies of Sciences, Engineering, and Medicine. 2021. Necessary DoD Range Capabilities to Ensure Operational Superiority of U.S. Defense Systems: Testing for the Future Fight. Washington, DC: The National Academies Press. https://doi.org/10.17226/26181.
The National Academy of Sciences was established in 1863 by an Act of Congress, signed by President Lincoln, as a private, nongovernmental institution to advise the nation on issues related to science and technology. Members are elected by their peers for outstanding contributions to research. Dr. Marcia McNutt is president.
The National Academy of Engineering was established in 1964 under the charter of the National Academy of Sciences to bring the practices of engineering to advising the nation. Members are elected by their peers for extraordinary contributions to engineering. Dr. John L. Anderson is president.
The National Academy of Medicine (formerly the Institute of Medicine) was established in 1970 under the charter of the National Academy of Sciences to advise the nation on medical and health issues. Members are elected by their peers for distinguished contributions to medicine and health. Dr. Victor J. Dzau is president.
The three Academies work together as the National Academies of Sciences, Engineering, and Medicine to provide independent, objective analysis and advice to the nation and conduct other activities to solve complex problems and inform public policy decisions. The National Academies also encourage education and research, recognize outstanding contributions to knowledge, and increase public understanding in matters of science, engineering, and medicine.
Learn more about the National Academies of Sciences, Engineering, and Medicine at www.nationalacademies.org.
Consensus Study Reports published by the National Academies of Sciences, Engineering, and Medicine document the evidence-based consensus on the study’s statement of task by an authoring committee of experts. Reports typically include findings, conclusions, and recommendations based on information gathered by the committee and the committee’s deliberations. Each report has been subjected to a rigorous and independent peer-review process and it represents the position of the National Academies on the statement of task.
Proceedings published by the National Academies of Sciences, Engineering, and Medicine chronicle the presentations and discussions at a workshop, symposium, or other event convened by the National Academies. The statements and opinions contained in proceedings are those of the participants and are not endorsed by other participants, the planning committee, or the National Academies.
For information about other products and activities of the National Academies, please visit www.nationalacademies.org/about/whatwedo.
COMMITTEE ON ASSESSING THE PHYSICAL AND TECHNICAL SUITABILITY OF DoD TEST AND EVALUATION RANGES AND INFRASTRUCTURE
DANA “KEOKI” JACKSON, NAE,1 MITRE Corporation, Chair
DARRYL AHNER, Air Force Institute of Technology
KAREN BUTLER-PURRY, Texas A&M University
GRAHAM V. CANDLER, University of Minnesota
GORDON FORNELL, United States Air Force, Retired
DERRICK HINTON, Scientific Research Corporation
ROB KEWLEY, stimlytics.cloud, LLC
LAURA J. McGILL, Sandia National Laboratories
HANS MILLER, MITRE Corporation
HEIDI C. PERRY, Massachusetts Institute of Technology Lincoln Laboratory
GARY POLANSKY, Sandia National Laboratories
KARL F. SCHNEIDER, Department of the Army, Retired
WILLIAM WILSON, Carnegie Mellon University
Staff
LIDA BENINSON, Senior Program Officer, Board on Higher Education and Workforce, Study Director
WILLIAM “BRUNO” MILLONIG, Director, Board on Army Research and Development (BOARD)
STEVEN DARBES, Program Officer, BOARD
CHRIS JONES, Senior Finance Business Partner, BOARD
CAMERON MALCOM, Research Assistant, BOARD
CLEMENT MULOCK, Program Assistant, BOARD
RYAN MURPHY, Program Officer, Air Force Studies Board
LINDA WALKER, Program Coordinator, Board on Physics and Astronomy
SAMUEL ZINKGRAF, Research Assistant, BOARD (through May 2021)
Consultant
ROBERT POOL, Writer
___________________
NOTE: See Appendix D, Disclosure of Unavoidable Conflicts of Interest.
1 Member, National Academy of Engineering.
BOARD ON ARMY RESEARCH AND DEVELOPMENT
KATHARINA McFARLAND, U.S. Army (retired), Chair
MICHAEL BEAR, BAE Systems, Vice Chair
ANDREW ALLEYNE, University of Illinois, Urbana-Champaign
DAVID AUCSMITH, University of Washington
JAMES BAGIAN, NAE1/NAM,2 University of Michigan
JOAN BIENVENUE, University of Tennessee
LYNN DUGLE, Independent Consultant
JOHN FARR, United States Military Academy at West Point
GEORGE “RUSTY” GRAY III, NAE, Los Alamos National Laboratory
WILLIAM HIX, U.S. Army (retired)
GREGORY JOHNSON, Lockheed Martin
DUNCAN McGILL, Mercyhurst University
CHRISTINA MURATA, Deloitte
ADITYA PADHA, Deloitte
ALBERT SCIARRETTA, CNS Technologies, Inc.
GEOFFREY THOME, SAIC
JAMES THOMSEN, Seaborne Defense, LLC
JOSEP TORRELLAS, University of Illinois, Urbana-Champaign
Staff
WILLIAM “BRUNO” MILLONIG, Director
STEVEN DARBES, Program Officer
SARAH JUCKETT, Program Officer
TINA LATIMER, Program Coordinator
CAMERON MALCOM, Research Assistant
CLEMENT MULOCK, Program Assistant
CHRIS JONES, Senior Finance Business Partner
___________________
1 Member, National Academy of Engineering.
2 Member, National Academy of Medicine.
Preface
Our nation’s warfighters go into combat to fight and win equipped with weapon systems that must operate under the harshest conditions, against determined and capable adversaries. They rightfully expect that these weapons have been tested and proven effective under operationally relevant conditions, against realistic threats that represent the battlefield they will confront. The Department of Defense’s (DoD’s) test and training range enterprise makes possible this essential developmental and operational testing, and these key resources for national security rest on the dedicated contributions of thousands of military personnel, civil servants, defense contractors, and representatives of national laboratories and federally funded research and development centers. They are at the heart of the range enterprise, and labor under extremely challenging conditions, generally unseen and unknown to the public due to the criticality of their work. The future viability of DoD’s range enterprise depends on addressing dramatic changes in technology, rapid advances in adversary military capabilities, and the evolving approach the United States will take to closing kill chains in a Joint All Domain Operations environment. This recognition led DoD’s Director of Operational Test and Evaluation (OT&E), the Honorable Robert Behler, to request that the National Academies of Sciences, Engineering, and Medicine examine the physical and technical suitability of DoD’s ranges and infrastructure through 2035.
The study committee brought a diverse set of perspectives and expertise to the questions posed in the statement of task, with members from industry, academic, and government backgrounds, versed in the application of emerging technology, the operational use and test of advanced
weapon systems, the rapidly changing landscape of digital technologies, and the organizational and budgetary complexity faced by the OT&E community and the range enterprise. The committee readily acknowledges that the extraordinary diversity of DoD missions and test environments, and the large quantity of range locations and installations, precluded an exhaustive evaluation of all range capabilities and gaps in relation to the future OT&E landscape. Nevertheless, the committee is confident that the findings and conclusions described in this report represent common themes fully supported by a survey of several of the most significant ranges, and an extensive review of prior studies and reports on OT&E needs and the implications for the range enterprise. The committee also notes that this unclassified study addresses certain key challenges and solutions at a general level due to the sensitive nature of many U.S. military capabilities and the intelligence gathered on current and future threats posed by U.S. adversaries. The combined background in national security matters of the committee underpins its belief that this report’s recommendations address DoD’s overarching range enterprise needs, while recognizing that the second, classified phase of this study will provide important additional detail and context regarding the test and evaluation requirements for the ranges posed by new weapons capabilities and threat characteristics.
The committee is grateful for the contributions of a wide range of noted experts and thought leaders in military weapon systems development, test, and evaluation; innovation and emerging technologies; software-intensive systems and digital capabilities; and the operational challenges both current and future faced by the U.S. military. Likewise, we received outstanding support from representatives of many test and training ranges spanning warfighting domains across land, sea, air, space, and cyberspace, who contributed their time and insights Many of the experts who participated in the study’s workshops and committee meetings have a distinguished record of public service, including in the military, and we thank them for that service to our nation. We also are pleased to acknowledge the gracious support from Mr. Robert Arnold, Senior Advisor of Sustainable Ranges, and Dr. Raymond O’Toole, acting Director, Operational Test and Evaluation, in providing connections and access to key officials, DoD resources, and reference materials that were indispensable to the study committee. It has been a privilege to work with these dedicated public servants and subject matter experts on this important priority for the nation’s defense.
Keoki Jackson, Chair
Committee on Assessing the Physical
and Technical Suitability of DoD Test and
Evaluation Ranges and Infrastructure
Acknowledgments
The committee would like to thank the following individuals for providing input to this study:
ANDRE’ “DRE” ABADIE, U.S. Army Futures Command
JAMES AMATO, Army Test and Evaluation Command
ZACH BARBER, Nevada Test and Training Range
LISA BARNEBY, Point Mugu Sea Range
STEPHEN BEARD, Missile Defense Agency
ROBERT BEHLER, Former Director, Operational Test and Evaluation
MARC BERNSTEIN, Assistant Secretary of the Air Force (Acquisition, Technology, and Logistics)
ASHTON BURKE, Test Resource Management Center
DEVIN CATE, U.S. Air Force
ERIC CLINTON, Test Resource Management Center
VICTORIA COLEMAN, U.S. Air Force
CHRIS COLLINS, Under Secretary of Defense for Research and Engineering
RYAN “RHINO” CONNER, Electromagnetic Spectrum Superiority, U.S. Air Force
MICHAEL CONTRATTO, 96th Test Wing
JAMES COOKE, U.S. Army
DENNIS CRALL, Joint Staff J6
FREDERICK CRAWFORD, Institute for Defense Analyses
MISSY CUMMINGS, Duke University
BILL DARDEN, Atlantic Test Range
EVAN DERTIEN, Air Force Materiel Command
JESSIE DIETZ, Pacific Multi-Domain Training Experimentation Capability
FRED DRUMMOND, Office of the Secretary of Defense
JASON ECKBERG, U.S. Air Force
VIV EDWARDS, Nevada Test and Training Range
JOHN ELLIS, Missile Defense Agency
FRED ENGLE, Office of the Assistant Secretary of Defense for Readiness
ERIC FELT, Air Force Research Laboratory
JOHN FIORE, Naval Surface Warfare Center
MATT FUNK, NAVAIR Acquisition and Tech Support Division
JOHN GARSTKA, Office of the Secretary of Defense
JEFFREY GERAGHTY, Wright-Patterson Air Force Base
CONRAD GRANT, Johns Hopkins University Applied Physics Laboratory
WILLIAM GREENWALT, American Enterprise Institute
DEREK GREER, NAVAIR Integrated Battlespace Simulation and Test
ED GREER, Formerly with the Office of Developmental Test & Evaluation
ROBERT GRIMES, Nevada Test and Training Range
SCOTT HOSCHAR, Atlantic Test Range
ARTHUR HUBER, Air Force Materiel Command
CHRIS JARBOE, Atlantic Test Range
PAUL KAWSHNAK, Aberdeen Proving Ground
PAUL KETRICK, National Cyber Range Complex
MICHAEL LABER, Point Mugu Sea Range
EDGAR LACY, Naval Surface Warfare Center Dahlgren Division
BRIAN LEONG, Pacific Multi-Domain Training Experimentation Capability
PETER LEVINE, Institute for Defense Analysis
RYAN “CHEECH” LUCERO, Nevada Test and Training Range
MIKE MACKINAW, Pacific Multi-Domain Training Experimentation Capability
JOSHUA MARCUSE, Google
DONALD MARTIN, Nevada Test and Training Range
BARRY MOHLE, Naval Surface Warfare Center Dahlgren Division
CARL MURPHY, Test Resource Management Center
BRIAN NOWOTNY, Test Resource Management Center
JOHN OKUMA, Institute for Defense Analyses
DANIEL OSBURN, 412th Test Wing
RAYMOND O’TOOLE, Director, Operational Test and Evaluation
BRENT PARKER, Pacific Multi-Domain Training Experimentation Capability
DAN PATT, Thomas H Lee Partners
JOHN PEARSON, Office of the Secretary of Defense Air Warfare
JANE PINELIS, Joint Artificial Intelligence Center
CARROLL “RICK” QUADE, Deputy Assistant Secretary of the Navy Research, Development, Test & Evaluation
JACK RILEY, Pacific Multi-Domain Training Experimentation Capability
STEVE ROGERS, Air Force Space Command
LEE ROSEN, SpaceX
DANIEL ROSS, Naval Surface Warfare Center Dahlgren Division
ROBIE SAMANTA ROY, Lockheed Martin
GEORGE RUMFORD, Test Resource Management Center
DAVID SAYRE, Missile Defense Agency
SCOTT SBUKOFF, Pacific Multi-Domain Training Experimentation Capability
HERMAN “HEMET” SCHIRG, Nevada Test and Training Range
CAPT WILLIAM SELK, Commanding Officer, VX-1
KENNETH SENECHAL, NAVAIR
ARUN SERAPHIN, Senate Armed Services Committee
JASON STEWART, Atlantic Test Range
JACOB SUGGS, Missile Defense Agency
ROBERT TAMBURELLO, Test Resources Management Center
MICHAEL TAYLOR, SpaceX
NEIL THURGOOD, Office of the Assistant Secretary of the Army
BRYAN TITUS, Air Force Space Command
GIL TORRES, Naval Air Warfare Center Weapons Division
RODNEY TRAYLOR, Nevada Test and Training Range
ANDREW TREE, Point Mugu Sea Range
DAVID TREMPER, Office of the Secretary of Defense
EDWARD TUCKER, Arnold Engineering Development Complex
ROBERT VARGO, Atlantic Test Range
JEFFREY WHITE, Secretary of the Army
MICHAEL WHITE, Under Secretary of Defense for Research and Engineering
KEVIN WILLIAMS, Missile Defense Agency
LEMUEL WILLIAMS, Missile Defense Agency
GEOFFREY WILSON, Test Resource Management Center
ERIC “GLOCK” WRIGHT, Nevada Test and Training Range
GREG ZACHARIAS, Chief Scientist, Director of Operational Test and Evaluation
PETER “ZUPP” ZUPPAS, Nevada Test and Training Range
The committee would also like to express its gratitude to Maya Thomas and Christopher Lao-Scott, Research Librarians at the National Academies Research Center, for their assistance with fact checking.
Acknowledgment of Reviewers
This Consensus Study Report was reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise. The purpose of this independent review is to provide candid and critical comments that will assist the National Academies of Sciences, Engineering, and Medicine in making each published report as sound as possible and to ensure that it meets the institutional standards for quality, objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process.
We thank the following individuals for their review of this report:
Sharon Beerman-Curtin, Strategic Consulting, LLC,
Russel Caflisch, NAS, New York University,
Stephen Di Domenico, Coldsquared Consulting,
Kathleen Dussault, Lemon Grove Associates,
James Michael Gilmore, Institute for Defense Analysis,
Lester Lyles, NAE, Independent Consultant,
Chris Maston, Georgia Tech Research Institute, and
Julie Ryan, Wyndrose Technical Group.
Although the reviewers listed above provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations of this report nor did they see the final draft before its release. The review of this report was overseen by John Tracy,
NAE, Boeing (retired). He was responsible for making certain that an independent examination of this report was carried out in accordance with the standards of the National Academies and that all review comments were carefully considered. Responsibility for the final content rests entirely with the authoring committee and the National Academies.
Contents
Military Ranges Past, Present, and Future
2 AN ENVISIONED FUTURE OF OPERATIONAL TEST AND EVALUATION
The Envisioned Future of Military Test Ranges
Enabling the Envisioned Future of Military Ranges
Testing for the Multi-Domain Battlespace
A Joint Program Office to Support DoD Multi-Domain Testing Needs
4 DIGITAL INFRASTRUCTURE NEEDS FOR OPERATIONAL TESTING
Increasing the Usability and Value of Data
5 SPEED-TO-FIELD: RESTRUCTURING THE REQUIREMENTS AND RESOURCES PROCESSES FOR DoD TEST RANGES
Program Requirements Drive Range Funding Investments
Colors of Money for Range Modernization and Maintenance
Strategies to Improve Test Range Modernization
6 CONCLUSION AND SUMMARY OF RECOMMENDATIONS BY ACTOR
The Recommendations—By Stakeholder
A Statement of Task and Completion Matrix
C Committee Member Biographies