Proceedings of a Workshop
Energy Challenges and Opportunities for Future Data-Driven Operations in the United States Air Force
Proceedings of a Workshop—in Brief
On April 27-29, 2020, the National Academies of Sciences, Engineering, and Medicine’s Air Force Studies Board (AFSB) convened a virtual workshop on U.S. Air Force (USAF) data-driven operations’ energy challenges as part of a broader study focused on this topic. The goals of the study are as follows:
- Investigate the current state of Air Force planning, research and development, and expectations related to energy usage for military operations in the 2030 time frame.
- Investigate potential threats to energy assurance and access based on recent events and assumptions of future energy dependencies that should inform Air Force/government planning for energy generation, storage, and use.
- Investigate and describe current research and state of the art in energy efficient computation including hardware, software, and big data.
- Investigate and describe the energy needs for advanced weapons platforms, including static infrastructure, that provide support to machine learning, artificial intelligence, and integrated operations.
- Recommend manpower, research, and expertise requirements needed for the future energy environment.
The data gathered at the workshop, focusing on items 1 to 4 above, will assist the committee in forming the recommendations for a final report that will address all five items. During the workshop, committee members heard from a wide range of subject experts from government, industry, and academia to inform them about what steps and plans the USAF is taking and/or should be considering now to successfully develop, deploy, and sustain the weapons systems needed to compete in an emerging information-rich environment. The information summarized here reflects the knowledge and opinions of individual workshop participants and should not be viewed as a consensus of the workshop participants, the AFSB, or the National Academies. The purpose of the workshop was to begin investigating energy demands for weapons systems, from the airbase to the battle space, including information requirements and needs associated with capture, curation, storage, exploitation, and transmission of energy to enable the deployment and operation of data reliant systems.
CURRENT AIR FORCE AND DEPARTMENT OF DEFENSE ACTIVITIES
The Joint All Domain Command and Control (JADC2) Operational Environment
Dr. Julie Ryan (Committee Chair) convened the workshop and introduced the first speaker, Maj. Mary Kuconis (Intelligence, Surveillance, Reconnaissance and Cyber Effects Operations (A2/6)1). Kuconis discussed her office’s current efforts in infrastructure modernization, especially in relation to Joint All Domain Command and Control (JADC2) and the Advanced Battle Management System (ABMS).2
Kuconis explained that A2/A6 contributes to ABMS by updating the information technology (IT) infrastructure in order to develop the sensing grid. Kuconis’s team is focusing on research, development, and experiments to develop reference architectures, guidance, and policies in order to realize the vision of this sensing grid. Most of their time is currently spent on cataloging data sets, curating data, getting access to the right data, and merging data sets.
Her team is conducting cost-benefit analyses in terms of resources and operational benefits in determining which old infrastructures should be migrated into a digital cloud environment. Instead of hosting data servers on the premises, Kuconis said that moving to the Cloud might be cheaper and easier to scale if need be. When that is not feasible, they are assessing the ability to move data centers onto forward-operating bases and data analytics tools onto aircraft for more immediate data analysis capabilities during field operations. They are also testing the feasibility of hybrid storage of data for larger data analytics loads, such as when assessing imagery data. Kuconis said that their data reference architecture will help manage large data sets in different data centers or on premise at the edge to get necessary data to operators who will use that data, because it is too complicated to keep all of the data in one big data center.
Air Force Warfighting Integration Capability (AFWIC)
Lt. Col. Charles Bris-Bois (Air Force Warfare Integration Capability (AFWIC)) discussed how AFWIC is starting to organize cross functionally across the USAF and developing the future USAF rapidly.3 Bris-Bois mentioned how their innovative solutions division is researching and writing opportunity concepts on cutting-edge technologies from academia, industry, laboratories, and federally funded research and development centers to try and predict what is going to have an impact in 2030 and beyond. Those concepts are then given to groups at the Air Force Research Laboratory (AFRL) and rapid-incubation arms forces to develop those technologies for USAF applications.
When discussing a technology perspective to addressing energy needs of forward-operating bases, Bris-Bois said that one of the design choices could be to move more operations to long-range air superiority, so that fewer forward-operating bases will be needed. He noted that one challenge of operating dynamic bases on the edge is to ensure that it is more expensive for potential adversaries to fire missiles at that base than it is for the Department of Defense (DoD) to defend it. He pointed out that cost-effective base defense will require a lot of energy, and the USAF has not figured out exactly how to manage that electric power requirement yet. AFWIC is actively searching for alternative technologies that are going to allow the USAF to agilely push energy to the edge, store it, transport it, and manage that power in a way that is not easily attacked or disrupted by adversaries.
1 U.S. Air Force, Sensitive Compartmented Information Security and Intelligence, Surveillance, and Reconnaissance Systems Cyber-security and Governance, Air Force Manual 14-403, September 3, 2019, https://fas.org/irp/doddir/usaf/afman14-403.pdf.
2 M. Dwyer, “Making the Most of the Air Force’s Investment in Joint All Domain Command and Control,” Commentary, Center for Strategic and International Studies, March 6, 2020, https://www.csis.org/analysis/making-most-air-forces-investment-joint-all-domain-command-and-control.
Dr. Dorothy Robyn (planning committee member) asked about the support for micro-nuclear technology within DoD, and Captain Brandon Hua (AFWIC) gave a vignette of how AFWIC is looking at bringing that power to the edge. Hua explained that AFWIC is giving more attention to zero-gamma isotopes, where the half-life is only 3 months and no gamma rays are emitted. In answer to a question from Robyn, Hua clarified that AFWIC is doing preliminary research on scientific possibilities on this technology, and it is not at the official research and development (R&D) phase or prototype phase yet. Robyn and Bris-Bois noted that other countries will need to be consulted when deploying nuclear technologies in forward-operating bases in those areas.
Air Force Installation Energy
Robyn introduced Mr. Mark Correll (Air Force Deputy Assistant Secretary for Environment, Safety, and Infrastructure (SAF/IEE)). Correll’s office is in the process of completing an installation energy plan that will discuss how robust, resilient, redundant, responsive, and resourceful each installation is in managing its energy infrastructure.
When Robyn asked if data centers have more demanding needs for energy than other fixed installations, Correll said, “Yes, and no.” The data center itself is susceptible because it needs very reliable power, such as properly working heating, ventilation, and air conditioning (HVAC) systems, in order to function. However, from a mission-assurance perspective, redundancy does not just mean how many generators are backing up one data center but also how many data centers are in the system. If one data center goes down, the mission can be transferred to another data center. Correll said that the system can take more risk if there is greater power resiliency.
In answer to a question from Dr. Subhash Singhal (planning committee member), Correll said that the USAF is only in the power generation business if it has to be; when there is an energy requirement on a base, a contractor or some other group is brought on board to supply the power that the USAF needs. In answer to a question from Robyn, Correll said that for the USAF to adopt more microgrids,4 the operators have to be invested and really want it, because microgrids are difficult to install and manage.
Environmental Security Technology Certification Program (ESTCP) Installation Energy and Water
Mr. Tim Tetreault (Program Manager for Energy and Water, Strategic Environmental Research and Development Program (SERDP) & Environmental Security Technology Certification Program (ESTCP)5) began by saying how energy demands are increasing requirements both for reliability and quality across the board, specifically as it relates to data centers and other sensitive electronic devices. Tetrault said that there is pressure for the USAF to provide more energy infrastructure in the event that the commercial grid cannot supply USAF needs for extended periods of time.
Tetrault explained that ESTCP provides data and information for installation, management guidance, regulatory bodies, and technology developers. Every mission on every installation relies on electric power, either directly or indirectly, and is dependent on the reliable delivery of electric power. The ESTCP seeks to identify high-impact technologies and solutions, and then through the execution of demonstration projects, provide DoD stakeholders with a better understanding of critical issues and the benefits of specific solutions.
Microgrids are an area of active research, and the Tinker Air Force Base (AFB) experiment Correll mentioned was actually cofounded with ESTCP. Microgrids offer a networked design, so there is no one point of failure. They can also operate both in parallel with the grid and islanded to provide emergency
4 U.S. Department of Energy, “How Microgrids Work,” June 17, 2014, https://www.energy.gov/articles/how-microgrids-work.
backup power. Tetreault said that ESTCP is looking into integrated design approaches to reduce the complicated engineering and up-front costs that go into developing and installing microgrids. He completed his talk by emphasizing the importance of pull-the-plug exercises, where energy grids are tested in real life by shutting off different sections to verify how the systems will react in an emergency.
ENERGY SOURCES: NOW AND IN THE FUTURE
The Impact of Changing Energy Resource Mix on the Power Grid
Dr. Deniz Ozkan (planning committee member) introduced Prof. Willett Kempton (Department of Electrical and Computer Engineering, University of Delaware) as the Cofounder and Associate Director of the Center for Research in Wind (CReW) and who is affiliated with the College of Earth, Ocean, and Environment.
Kempton discussed the requirements for integrating variable generation6 into the grid, such as the role of transmission, energy storage, and the need for large-scale resources. The scale is important because clean, low-cost resources that are too small will not have a significant impact on the overall system. Systems where all the power in a grid comes from one energy source are intermittent and unreliable, because even a very well run thermal plant has 5 percent unscheduled outages, Kempton said. And so, while the problem of power source reliability is not new, the problem of variable generation is new.
Large power sources that could contribute to solid grid systems, Kempton continued, include land-based and offshore near-surface winds, mid-altitude and high-altitude winds, ocean thermal gradients and ocean saline gradients, ocean currents, and rooftop solar. Some of these technologies are still being developed but will be applicable within the 2030 time frame, and some of these are only viable regionally, but where they do occur they have high potential (the steady winds of the Great Plains, and the steady sunlight of the Southwestern States, for instance). The implementation of these technologies in a grid system requires an understanding of how these sources fluctuate by season as well as accurate weather forecasting. Maj. Gen. John Ferrari (planning committee member) asked which technology would be the most mobile, and thus most available to forward-operating bases and field operations, and Kempton said that solar is the most mobile.
Electrochemical Energy Conversion and Storage Devices
Singhal introduced Prof. Eric Wachsman (University of Maryland) and the topic of electrochemical energy conversion and storage devices, such as fuel cells, batteries, and hydrogen. Wachsman studies solid ion–conducting materials and electrocatalysts and the development of solid-state batteries, fuel cells, ion transport membrane reactors, and gas sensors. Wachsman said that the high-temperature requirements of solid-oxide fuel cells puts constraints on the operation of the fuel cell—specifically, lifetime, reliability, and cost. In addition, most of the cost of the solid-oxide fuel cell is not the fuel itself, but its other components, so using simple stainless steel materials could lower the cost.
Wachsman said that with less insulation, there is lower mass, volume, and cost. The higher the operating temperature, the lower the thermal mismatch between when the fuel cell is started up and when its operating. However, efficiency decreases as temperature increases. And so, if the fuel cell operates at a lower temperature, then the theoretical efficiency of the fuel cell increases.
When speaking to the importance of cooling in the goals of the study, Wachsman said that heat can be used in a variety of different ways, such as for an absorption chiller to provide cooling. Ferrari asked where the USAF should invest in this area, and Wachsman recommended the high-temperature capability of solid-state batteries or solid-oxide fuel cells that combine cooling and power, especially for
the potential usage in data centers.
Exploring Power Security with Microgrids
Singhal introduced the next speaker, Mr. Asim Hussain (Vice President of Commercial Strategy and Customer Experience, Bloom Energy),7 who discussed solid-oxide fuel-cell–based microgrids and the power security implications of microgrid technology. Bloom Energy was founded with a mission to make clean, reliable, and affordable energy. They work toward this using a solid-oxide fuel cell and by deploying Bloom Energy servers. They are currently deployed across four countries with about 450 megawatts in 10 U.S. states.
Hussain mentioned how Bloom Energy had a system in Delaware in 2012 during Hurricane Sandy that was powering a substation where they kept 60,000 electricity customers functioning in partnership with local utilities. He said they have tried to use solid-oxide fuel-cell technology as a building block and build a modular and scalable system that incorporates many capabilities into its model. Bloom Energy has different tiered solutions based on how much disruption the customers’ missions can stand. At each site, they take the load curve projections to figure out how much total power they need to provide and what the peak power looks like, so they determine if they need to supplement with batteries—to essentially charge the batteries during the off-peak hours and dispatch them during the peak hours, such that the system follows the load curve as closely as possible. In a worst case scenario, they would provide a diesel or other type of backup generator. Hussain explained that there is an economy of scale during the installation phase based on the configuration that the customer wants verses what is the facility’s reliability requirement.
ENERGY AND ARTIFICIAL INTELLIGENCE
Dr. Jill Crisman (Technical Director for Artificial Intelligence in the Office of the Undersecretary of Defense for Research and Engineering) spoke about artificial intelligence (AI), machine learning, and ubiquitous computing sensing. Crisman said that there is a growing realization in media and across the country that AI consumes a large amount of resources. She noted that determining how to stop data centers from using all of the world’s electricity is a serious AI concern that should not be just a DoD problem.
Crisman explained an AI algorithm competition in which the winners release their data and code at the end, showing that the models they made could be trained on different data sets by removing just the top layer of their models. Crisman said that training just the top layers of an AI algorithm is more energy efficient because it requires less data and uses less power during the training process. This allows users to deploy the same network to solve different problems, which is called transfer learning.
Crisman mentioned how helpful open AI is for training different data sets and finding existing available resources. She used AlphaGo as an example of a popular AI algorithm that consumes a lot of computing power in order to win at games of Go. Since 2012, the computing power needed to run new algorithms has been doubling about every 3.4 months.
Crisman also explained that one of the things that enabled AI was the exponential growth in actual data, because AI learns from the data. Data gathering is growing very rapidly around the world, and she projected that if DoD starts saving more and more of its data for training AI algorithms, a similar type of increase in data growth will be observed. Crisman said that commercial industry is handling this data inflow by keeping available the data that people access all the time, and archiving the data that is rarely accessed, taking it off of some of their cloud platforms, and potentially destroying data that is never really accessed.
7 Bloom Energy, “Resiliency: AlwaysON Microgrids,” https://www.bloomenergy.com/solutions/advanced-applications/microgrid.
Defense Science Board Study on Energy Systems for Forward-Operating Bases
Gen. Paul Kern (U.S. Army, retired) was asked to speak because of the 2016 Defense Science Board (DSB) report8 on which he was chair and which recommends that DoD consider micro-nuclear reactor technology to power forward-operating bases. Bris-Bois and Hua had previously mentioned they were researching micro-nuclear reactor capabilities for field operations in part due to this 2016 DSB report’s recommendations.
Kern explained that the focus of that study was pointing out the energy demand for operations on those bases and discussing how to get the energy needed to meet those demands. He pointed out that 90 percent of the demand at the time of their study was for two things: fuel and water. He said that the demand for bandwidth was not addressed in the study, but it is clearly one of the demand structures that is growing. He projected that possible future engagements in near-peer-type battles will result in demands for energy in smaller locations. Those installments will consolidate for tactical operations, move on, and then redistribute again so they don’t become targets themselves.
Experience Evaluating USAF Mission Assurance and Resilience
Mr. James Murphy (Idaho National Laboratory) is a senior system engineer and chemist who focuses on model-based systems engineering technology development and decision analysis techniques for the Department of Energy (DOE) and the Department of Homeland Security (DHS). He is currently involved in the DEEPR (Decomposition for Energy Assurance and Electrical Power Resilience) project, which is a mission thread analysis technique with associated metrics to dynamically evaluate USAF’s mission assurance effectiveness when subject to various installation threats that impact enabling assets and energy systems.
He was asked to demonstrate a useful approach to mission threat analysis with a focus on helping the USAF value resiliency and help it consider options for improving resiliency beyond the typical “adding a generator” approach that it had experienced so much. They found that the civil engineering, the communications squadron, and the installation support elements could all help Murphy’s team to understand how energy got to the facilities and how those utilities then fed the facilities at the transformers. Through this information gathering, Murphy’s team allowed these different groups to see how their work was tied to other teams’ successes through seeing how the different energy-dependent elements of the facility were all connected.
This DEEPR model focused on task enablers in order to strengthen mission resilience. Task enablers are what enables the mission operators to perform the important things necessary to achieve outcomes, and the mission resilience was measured by trying to understand how well the facility can stand up to different threat-informed scenarios.
RAND Project Air Force Research for the USAF on Energy Resiliency
Dr. Anu Narayanan (Engineer, RAND) focuses her research on the intersection of critical infrastructure systems, particularly energy systems and national security. Narayanan has been the lead on RAND’s extensive work for the USAF on installation energy issues ranging from valuing electric power resilience to strategies for deterring a tax on the power grid. She is an expert in probabilistic risk assessment scenarios and capabilities-based planning and systems analysis, and she recently finished a report on adaptive basing.
8 Defense Science Board, Task Force on Energy Systems for Forward/Remote Operating Bases: Final Report, prepared for the Department of Defense, Office of the Undersecretary of Defense for Acquisition, Technology, and Logistics, 2016, https://dsb.cto.mil/reports/2010s/Energy_Systems_for_Forward_Remote_Operating_Bases.pdf.
Narayanan described how RAND’s Project Air Force (PAF)9 group fits within the item 2 of the goals of the study, with its focus on potential threats to energy assurance and the implications of USAF’s dependence on energy today and in the future. The USAF tasked RAND with coming up with a systematic and defensible approach for deciding where and how much to spend on electric power resilience, and part of that is figuring out where there are problems and then deciding what to do about them. There is a need to be strategic about where these investments are made, because there is no such thing as risk elimination. The point is to have something defensible, she said.
PERVASIVE ADVANCED DATA USES
Dr. Mark Linderman (Senior Scientist for Command and Control Decision Support and the Information Directorate, AFRL) serves as the principal scientific authority and independent researcher in the field of command and control, and supporting technologies, including information sharing, data fusion, machine learning, performance optimization techniques, and systems resiliency.
Linderman noted that at the tactical edge, “we do not seem to measure things in gigawatts as often as we do in watts.” Sometimes at the tactical edge, some small advantages can have out-sized effects. Because the USAF burns about 2.5 billion gallons of gas a year, efforts to make missions more effective to reduce the number of times aircraft need to reengage a target or reassess some intelligence, surveillance, and reconnaissance can save a lot of money and resources. Deploying aircraft and other weapon and surveillance technologies around the globe is a logistical nightmare, which is why the USAF is the only service that can project power globally, although it is expensive and very energy intensive in order to accomplish those missions.
He then discussed what his team has been doing over the years to try and get this high-performance computing up and into the field and how that generates revolutionary capability. Linderman explained that the constraints of these systems are their size, power, and weight loaded onto airborne systems and even more so into satellite-based systems. For his team, the generation of power is not as big a problem as getting rid of the heat that comes off that system, which prioritizes computational efficiency in order to get the best decision making in the shortest amount of time to prevent overheating.
ENERGY-EFFICIENT COMPUTATIONAL APPROACHES
Artificial Intelligence for Autonomous Systems: Algorithm and Hardware Perspectives
Dr. Mark Costello (planning committee member) said that the next group of speakers is focused on energy-efficient computational approaches, and for the next several speakers he wanted to explore basic computational algorithms that are designed for energy efficiency, try to understand what the state of the art is in this area, and then discuss the potential of future research. Costello introduced Dr. Saibal Mukhopadhyay (Professor, School of Electrical and Computer Engineering, Georgia Institute of Technology), whose research is on low-power, variation-tolerant, and reliable very-large-scale integration (VLSI) systems, memory design for VLSI applications, and ultra-low-power and fault-tolerant nanoelectronics technology circuits and computing platforms.
Mukhopadhyay said that energy-efficient AI is a challenge that needs to be solved by looking at algorithms and hardware together. Resiliency is an important factor when you start porting this algorithm to the edge, which has a direct consequence in terms of energy efficiency. Making things resilient more often than not comes at the cost of energy and performance. He discussed the Defense Advanced Research Projects Agency’s (DARPA’s) ReImagine Program10 and how his team is trying to make AI algo-
rithms more computationally efficient, which includes making algorithms less complicated and reliant on less data inputs.
A Discussion with Carole-Jean Wu
Dr. Carole-Jean Wu (Associate Professor, School of Electrical Computer and Energy Engineering, Arizona State University; Research Scientist, Facebook AI Infrastructure Research Team) focuses on high-performance and energy-efficient computer architectures through hardware heterogeneity, energy harvesting techniques for emerging computing devices, and understanding inference at the edge. She mentioned working on the system infrastructure development challenges specifically for machine learning in the Cloud. She described energy-efficient computation, including hardware, software, and big data.
She first discussed the energy challenges and opportunities from the angle of high performance and system design, and then mobile and edge deployment challenges and how this focuses on machine learning inference. Wu said that we need to pay more attention to advanced energy and cooling management solutions because of ultra-high-performance requirements coming from highly parallel heterogeneous computing systems. In addition to improving the systems, performance, and energy efficiency, we start to see large-scale long running programs such as machine learning training or applications in the high-performance computing domain to have significant environmental impact, Wu said.
Embedded Systems for Data-Rich Processing
Dr. Sherief Reda (Professor, School of Engineering, Brown University) researches hardware systems with a focus on energy-efficient computing, embedded systems, design automation of integrated circuits, and computer architecture. Reda pointed out that we’ve seen a class of applications that rely on a lot of data in their processing, which includes signal and image processing, computer vision, deep neural network, and machine learning in general, and they’re all characterized by massive amounts of data that need to be processed in order to get results.
Reda used the example of a missile traveling at supersonic speeds, whether you want to track it or you’re tracking something from it. He noted that a difference in microseconds in processing could mean that you’re off-target, which is why hardware time constraints is important in this context. He then mentioned that custom accelerators could be used, and they’ve been used in embedded Field Programmable Gate Arrays (FPGAs) or Application Specific Integrated Circuits (ASICs), and ASICs provide an additional energy improvement advantage over FPGAs at the expense of cost. Reda said that “when you couple approximations and embedded accelerators, you get high energy, more benefit than you would if you applied approximations in the central processing unit (CPU) or graphics processing unit (GPU).” He advised focusing on acceleration, approximation, and then software and hardware codesign to maximize the efficiency of the system.
DATA INFRASTRUCTURE ENERGY NEEDS
Energy and Remote AI Architecture
Maj. Gen. John Ferrari (planning committee member) introduced the next three speakers. Col. Matt Benigni (Joint Special Operations Command (JSOC)) uses big data at the edge, and JSOC is usually the group that implements future technologies on the field first, Ferrari said. Benigni talked about the tipping point happening on the battlefield, which is important because time horizons and projecting forward what the USAF might need is changing. Over the past 3 years, Benigni said that he traveled to
10 W. Mason, “Reconfigurable Imaging (ReImagine),” Defense Advanced Research Projects Agency, https://www.darpa.mil/program/reconfigurable-imaging.
a number of U.S.-deployed locations in order to work directly with operators, sometimes for extended amounts of time.
Benigni mentioned that he has observed the “flipping of the paradigm” this past year, in which the Internet of Things (IoT) is collecting more data than the systems can analyze. They collected more data within the tactical bubble than what’s collected outside of it; the tactical bubble is the geographical space between the forward-operating base and the objective. This paradigm shift stresses USAF’s architecture in that it will no longer have the ability to backhaul that volume of information in a timely manner.
A Discussion with Nikhil Krishnan and Udit Garg
C3.ai has positioned itself to use software and AI to manage the IoT at the edge, and the energy management team discussed how it’s using its software and data to do energy management around the world to make better decisions. Ferrari said that Tom Siebel’s (C3.ai founder’s) Digital Transformation book11 outlines the coming evolution in IoT technology, and introduced the speakers Dr. Nikhil Krishnan (Group Vice President, C3.ai) and Mr. Udit Garg (Senior Director, C3.ai).
Krishnan said that C3.ai provides a platform that abstracts away the complexity of the underlying cloud services, whether that is run on Amazon, Microsoft, or another cloud provider. They accomplish this through a model-driven architecture that uses conceptual domain models (C3 models) that allows developers and data scientists to operate on the business objects that they understand. Garg noted that IT infrastructure, data centers, and IoT devices are extremely important for DoD operations, and they are seeing this via the idea of digital transformation, digital tooling, and how digital applications are becoming increasingly mission critical. C3.ai uses AI to analyze USAF bases as though they were mini-cities to optimize energy management and operations. For instance, if the system does maximum conversion efficiency on the plant feeding an Air Force base, then the team can identify losses in the facility’s heating and cooling network.
A Discussion with Anoop Mavath and Rob Nolan
Amazon Web Services (AWS) is a practitioner that deploys, manages, and decommissions data centers, and the effect of minimizing the electromagnetic, thermal, and visual footprint is important because force protection and mission protection is important to military operations. Mr. Anoop Mavath (Director of Infrastructure Products, AWS) and Mr. Rob Nolan (AWS) spoke about designing, deploying, operating, and decommissioning data center infrastructures. Ferrari noted that what’s important on the battlefield is minimizing detection and maximizing protective measures.
Mavath began by saying he generally looks at data center design considerations under the premises of needing to operate remotely, such as on forward-operating bases mentioned in the study’s goals. He discussed the requirements that need to be established before setting up a remote data center, such as mobility requirements, how fast it needs to be deployed, and how fast it needs to be decommissioned, especially since all data centers carry a lot of critical information. AWS has a number of “plug and play” data center configurations that can be used at the edge, wherein front-end set-up costs, time, and personnel training can be minimized.
ENERGY SUPPLY CHAIN, INFRASTUCTURE, AND ECOSYSTEM ISSUES
A Discussion with Marty Edwards
Mr. Marc Sachs (planning committee member) introduced Mr. Marty Edwards (Vice President of Op-
11 T.M. Siebel, Digital Transformation: Survive and Thrive in an Era of Mass Extinction, Rosetta Books, New York, N.Y., 2019.
erational Technology Security, Tenable12) and a globally recognized operational technology and an industrial control system (ICS) expert. Edwards said that when he worked in homeland security, his team would conduct facility assessments, and the number one finding was improper, inappropriate, or inadequate segmentation of networks.
Through Tenable’s asset identification and vulnerability management tools, his teams notice interconnections that customers did not even know existed. Tracking and managing everything that goes into a network is challenging, especially on a complex type of installation such as a military base. Edwards also explained Tenable’s Vulnerability Priority Rating (VPR) mechanism that ingests threat feeds from a variety of providers in order to use machine learning around the threat environment.
A Discussion with Sam Chanoski
Mr. Sam Chanoski (Director of Intelligence, Information Sharing and Analysis Center) said he works with private-sector organizations to exchange and analyze all source intelligence in the context of the electric sector. He said that energy is injected into a system from resources that create or store energy, and noted he that the limiting factor in scaling energy is due to hydroelectric pump storage capacity, since batteries are useful but not as available at a utility scale. Furthermore, depending on the location of where energy is injected into the system, it flows into a complicated and dense network topology with different injection points from various resources with different operating capabilities.
Chanoski discussed how government and military facilities, because they do not want to ever experience outages, require a different design approach to combining outside-the-fence and inside-the-fence operations on an installation. Outside the fence refers to engineering with local utilities and construction plans, while inside the fence refers to energy storage, backup power, batteries, monitoring, awareness, and understanding of how these systems interact from a system of systems perspective on the installation’s property. More stringent pre-contingency operating criteria at the utilities will operate the bulk of the power system and its dense network as well.
Cyber Resilient Hardware Controller
Mr. Perry Pederson (Pacific Northwest National Laboratory) is a technical advisor and cybersecurity researcher in the development and execution of a national strategy to improve cybersecurity for the US critical infrastructure. Pederson spent his talk describing a project he is working on, in collaboration with colleagues at Idaho National Laboratory and Oak Ridge National Laboratory, which rethinks the concept of digital by trying to accomplish cyber-secure safety measures in microchips through hardware metrics rather than software malware adjustments. His prototype FPGA development board uses tools like Verilog, a hardware description language, to physically burn in deterministic behavior to chips that have advantages against bad-acting or corrupt software. “If you’re printing your own logic you’re almost baking the security in, and it’s visible security”, Pederson said.
Substation Physical Security Initiatives
Mr. Ernie Hayden (443 Consulting) was introduced as a highly experienced and seasoned technical consultant, author, speaker, strategist, and thought leader with extensive experience in the power utility industry with a background in informational security and industrial controls. Hayden spoke to the physical security threat assessment of facilities, such as direct attack, power-line damage, vandalism, and severe weather. He said that a utility has to do an initial risk assessment to identify whether a substation is a critical facility or not; how critical the facility is determines the standards for managing its physical security measures.
Electric Sector Threats
Mr. Chris Sistrunk (Technical Director, ICS/IoT Security Consulting Team, Mandiant, FireEye) was awarded the Energy Sector Security Professional of the Year distinction in 2014 and is a senior member of the Institute of Electrical and Electronics Engineers (IEEE). Sistrunk mentioned ICS security topics, electric sector threats and risks, and incident response and forensic assessments. He said that many things can threaten a system, such as old programs and platforms, data theft, and hacktivism.13 He also discussed the case study of the blackout that occurred during the 2013 Super Bowl, which was caused by relay misoperations.14 Sistrunk said that there were some other contributing factors that caused that blackout, but the main reason was a defect in the design of that relay.
Sistrunk said that when his team is assessing facilities, they look at fences, tree cover, and storm drainage tendencies. He suggested that the USAF should consider where its substations are, what the threat environment is like, and what visibility might be needed on local power companies and the substation itself. This includes training personnel in physical security of facilities and looking for how a bad actor might exploit vulnerabilities. “You may have a gaping hole in your physical security, even though your cybersecurity may be top notch,” Sistrunk stressed.
A Discussion with Art Conklin
Dr. Art Conklin’s (Professor and Director, Center for Information Security Research and Education, College of Technology, University of Houston) research interests include the use of systems theory to explore information security, specifically in cyber-physical systems and critical infrastructures. Conklin said that science, engineering, and business are all reliant on technology, and each sector tries to manage technology in different ways. He explained that he looks at the management of technology from a systems perspective, such as assessing needs for electricity, communications, etc., on a base or for a utility and recognizing that these elements are all connected.
Conklin used an example from the response effort after Hurricane Ike hit Houston, when the city filed and successfully received over a billion dollars’ worth of bond funding. Two weeks later, the financial collapse of 2008 happened, and had the hurricane been 2 weeks later, Conklin said, Houston would not have recovered from the hurricane without a federal intervention, because the bond market would have been unavailable. “It wasn’t a matter of was the utility available, it wasn’t a matter of was the hardware available, we had workers, but literally the financial markets,” Conklin noted.
DISCUSSION AND CONCLUSION
When reflecting on the workshop’s discussions with the other workshop participants, Ryan said that there is an uncertainty about the power needs for operations and how to logistically deliver that power to the edge. The USAF is conducting experiments and testing different technologies, and the logistics-under-attack issue impacts the agility of energy at the edge. The challenges discussed during the workshop included the following: the trade-offs between on premise systems and the Cloud, back-end computing, computing on dedicated storage, data as a service, software as a service, and how to manage these elements. Ferrari stressed that if military operations are going to be data driven, then it is important to manage the movement of that data and the energy requirement for the chain of fiber, cellular, Wi-Fi, and satellites. Cell towers, fiber cables, etc., all need energy, and the movement and transmission of that data is important to the mission at the edge.
13 IT Pro Team, “What Is Hacktivism?,” ITPro.co.uk, August 9, 2018, https://www.itpro.co.uk/hacking/30203/what-is-hacktivism.
14 North American Electric Reliability Corporation, Analysis of System Protection Misoperations, December 2015, https://www.nerc.com/pa/RAPA/PA/Performance%20Analysis%20DL/2015_Analysis_of_System_Protection_Misoperations_Final.pdf.
DISCLAIMER: The Proceedings of a Workshop—in Brief has been prepared by Catherine Puma as a factual summary of what occurred at the meeting. She was assisted by Ryan Murphy. The committee’s role was limited to planning the event. The statements made are those of the individual workshop participants and do not necessarily represent the views of all participants, the planning committee, or the National Academies. This Proceedings of a Workshop—in Brief was reviewed in draft form by Mark Costello, Georgia Institute of Technology, and David Van Wie, Johns Hopkins University Applied Physics Laboratory, to ensure that it meets institutional standards for quality and objectivity. The review comments and draft manuscript remain confidential to protect the integrity of the process.
PLANNING COMMITTEE: Dr. Julie H. Ryan, Chair, Wyndrose Technical Group; Dr. Mark Costello, Georgia Institute of Technology; Major General (ret) John Ferrari, U.S. Army, QOMPLX; Dr. Deniz Ozkan, Shell New Energies; Dr. Dorothy Robyn, Boston University Institute for Sustainable Energy; Mr. Marc Sachs, Pattern Computing; Dr. Michael Schneider, Lawrence Livermore National Laboratory; Dr. Subhash Singhal (NAE), Pacific Northwest National Laboratory.
RAPPORTEUR: Catherine Puma, Research Associate, Air Force Studies Board
STAFF: Ellen Chou, Director, Air Force Studies Board; George Coyle, Senior Program Officer; Ryan Murphy, Program Officer; Adrianna Hargrove, Finance Business Partner; Marguerite Schneider, Administrative Coordinator; Catherine Puma, Research Associate
SPONSORS: This workshop was supported by the U.S. Air Force.
Suggested citation: National Academies of Sciences, Engineering, and Medicine. 2020. Energy Challenges and Opportunities for Future Data-Driven Operations in the United States Air Force: Proceedings of a Workshop—in Brief. Washington, D.C.: The National Academies Press. https://doi.org/10.17226/25872.
Division on Engineering and Physical Sciences
Copyright 2020 by the National Academy of Sciences. All rights reserved.