Complex Systems: Task Group Summaries (2009)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Task Group Summary 3 How can we enhance the robustness via interconnectivity? Challenge Summary In contrast with most human designs, which are prone to failures once their components fail, natural and some human-made but self-organized systems display a high degree of robustness to component failures. Indeed, living systems can carry on their activity despite the many molecular errors at the cellular level and the Internet does not collapse despite the fact that at any moment hundreds of routers are not functional. Many living systems, like bacteria, have been shown to be able to withstand the removal of several key enzymes. It is increasingly believed that the robustness of these systems is rooted in their networked nature. Early attempts to address a network’s response to attack and failures indicated that real networks are highly ro- bust to random failures but fragile against attacks. Subsequent work has shown that the interplay between the resources and the demand can lead to cascading failures, uncovering a high degree of fragility of some systems. A good example is offered by the US electrical power grid, whose cascading East Coast breakdown was initiated by local failures. In general, a series of recent studies suggest that networked systems are not only robust but also suffer from vulnerability due to interconnectivity, as local failures can spread and turn global. Despite the recent fundamental advances, a deep understanding of the origins and mechanism or robustness across many complex systems is lacking. Little is known, for example, of the role of the dynamics (commu- nication protocols, flow processes) on the network, and how the dynamics and the topology influence each other to promote or undermine robustness. 21 

22 COMPLEX SYSTEMS Thus the role of the present working group is to explore what factors con- tribute to a system’s robustness. To achieve this goal, the group is asked to choose a system that is of major importance for the research community and explore the origins of robustness in this system. The system of choice could range from man-made systems, like the Internet or other communication networks, to natural systems, like the cell or an organism. Key Questions • What are the proper metrics of robustness? • How does one quantify the relative contributions of network struc- ture and dynamical effects to robustness? • Are there universal design principles to robust systems? • Is robustness more than redundancy? • Designing networks that are robust to both failures and attacks. • Cascading failures—can they ever be remedied? • What measures are appropriate to enhance robustness on a given system? Required Reading Albert R, Jeong H, Barabási A-L. Error and attack tolerance of complex networks. Nature 2000;406:378–482. Motter AE. Cascade control and defense in complex networks. Phys Rev Lett 2004;93(9):098701. [http://lanl.arxiv.org/PS_cache/condmat/pdf/0401/0401074v2. pdf.} Barabási A-L, Bonabeau E. Scale-free networks. Scientific American 2003:May:50-59. Suggested Reading Levin SA, Lubchenco J. Resilience, robustness, and marine ecosystem-based management Bioscience 2008;58(1):27-32. Paul G, Sreenivasan S, Havlin S, Stanley HE. Optimization of network robustness to random breakdowns. Physica A 2006;370:854-862. Barkai N, Leibler S. Robustness in simple biochemical networks. Nature 1997;387(6636):913- 917. Due to the popularity of this topic, two groups explored this subject. Please be sure to review the second write-up, which immediately follows this one. 

TASK GROUP SUMMARY 3A 23 TASK GROUP MEMBERS – GROUP A • Ramanand Dixit, Washington University in St. Louis • Rebecca Goolsby, Office of Naval Research • Natali Gulbahce, Northeastern University • John Hartman IV, University of Alabama at Birmingham • Pradeep Kumar, Rockefeller University • Arthur Lander, University of California, Irvine • Christopher Myers, Cornell University • Aristides Requicha, University of Southern California • Qian Wang, University of South Carolina • Casey Rentz, University of Southern California At the 2008 National Academies Keck Futures Initiative Conference on Complex Systems, one of two Task Groups (3A) charged with thinking about how to enhance robustness via interconnectivity, considered several areas on which to focus. Group members from universities and govern- ment research centers saw complex systems from a variety of different perspectives: nanoscale bond interactions; microtubules and systems of self-organization in cell growth; sensor networks in the coordination of robots; human disaster relief social networks; water and turbulence; virus life cycles in the human body; and gene interaction networks derived by quantitative phenotyping. What Is Robustness? The group initially grappled with what we mean by robustness. The general consensus was that we needed to define what the system is before we talk about its robustness. For example, is the system a cell, an organism or species? Because there are many facets to complex systems across differ- ent scale-levels, different perturbations and performance measures need to be considered. One must be careful of what scale is selected when initially defining the complex system (and related optimization goals). In certain sys- tems, one might find performance fluctuations on a small scale that would not appear at a larger scale. Or, a loss of robustness at a certain scale might be accompanied by a gain in robustness at another scale. These robustness trade-offs, where performance or robustness of a system is sacrificed at one level to be enhanced at another level, exist and must be taken into account when robustness of a complex system is measured. A complex system is also 

24 COMPLEX SYSTEMS typically fluid. Connections within an engineered or biological complex system are always breaking, reforming, and changing. Perturbances often seem inseparable from the networks and complex systems themselves. Everyone in Task Group (3A) seemed to agree that robustness is a continuum, not a case of have or have not. Exploring the nature of robustness and fragility in complex systems, Task Group (3A) attempted to abstract from real-life examples of com- plex systems how connectivity within the system might influence its robustness. “Networks” provide a useful way to depict a complex system through component nodes and functional connections. For convenience of discus- sion, the group identified four systems that were deemed relatively easy to deconstruct into their component parts: power grid structures, health care, the Internet, and yeast genetic interactions. For example, power grids have as nodes houses/businesses, substations, and central power stations; highly connected nodes (central stations) are considered as hubs. Similarly, nodes of a health care system could consist of patients, doctors and other health providers, connected by their respective encounters. The Internet can be broken down into personal computers as nodes and servers as hubs of in- formation distribution. A genetic interaction network has genes as nodes, and connectivity between the nodes represent “interactions,” defined as dependencies that genes share with respect to expression of a phenotype, like cell growth. An abstract examination of network dynamics and degree of intercon- nectivity within the structure of our selected complex systems allowed us to make recommendations for increasing robustness. A network’s behavior can be thought of as based on its performance in a particular context due to the effect of a particular perturbation. We delineated hypothetical performance objectives and relevant perturbations for selected complex systems, with an eye toward abstracting general robustness strategies from one system that could be applied in an analogous way to increase robustness in other systems. In the case of power grids, the objective in enhancing robustness was to maximize the number of people with electrical power and to minimize the risk of cascading power failure. In the case of health care, the objective was to prevent epidemics caused by a novel pathogen. In the case of the Inter- net, the objective was to maximize available online time for each individual while preventing service failures. In the case of yeast genetic interactions, the objective was to characterize robustness by “reverse-bioengineering;” 

TASK GROUP SUMMARY 3A 25 cell proliferation is a robust property of cells based on the observation that individual deletion of most yeast genes has little effect on growth. However, high throughput phenotypic analysis of cell proliferation of all 5000 gene deletion mutants in many different types of media provides a systematic, quantitative means to ascertain how genes contribute to the cellular robust- ness and ability to adapt to changing environments. Interconnectivity and Dynamics Imagining the effect of a particular perturbance, as it would spread throughout each model complex system, it is clear that interconnectivity can lead to both robustness and vulnerability. For example, “essential genes” (deletion results in lethality) have a greater number of physical (protein- protein) interactions than non-essential genes, and thus can be considered as cellular “hubs” of function. Likewise, the more interconnected a power substation, the more people it supplies, but it is also an easy target for a blackout. A perturbance, such as a novel disease, would spread through and weaken a complex system, such as the health care system, through nodes that are highly connected. But spread of a perturbance is also greatly affected by system dynamics. For example, a sick individual, one node of the health care system, may have a particularly virulent strain of a specific disease (or be somehow better at transmitting the disease). Though they are not well connected to the rest of the system, the disease would pass through the sys- tem more quickly due to the strength of this nodal connection. Additional factors that can affect network dynamics are directionality and strength of links; both of which can be used to determine where a perturbance will flow and where vulnerabilities in the system will arise. Knowledge of directionality and strength of interconnectivity can po- tentially be exploited to increase robustness. For example, such knowledge would allow us to “park” fragilities where they are least vulnerable and most easily managed. A simple example of this is power stations, which as sources of electrical power in the power grid system should be the most protected hubs in the system in order to lessen the consequences of a malfunction or attack that would otherwise result in a catastrophic breakdown of the sys- tem. In the case of genetic interactions, understanding fragilities that result from cancer-causing mutations would reveal targets for selectively killing cancer cells. Control in a complex system does not necessarily have to coin- cide with hubs of that system. In most dynamic complex systems, blending of centralized and distributed control would enhance robustness. 

26 COMPLEX SYSTEMS In addition to active control, adaptability of system topography in- creases robustness in a complex system. The group discussed random and scale-free networks as two kinds of network topologies that respond differ- ently to perturbations/attacks and would thus affect robustness. A random network is one where the nodes are equally interconnected and/or evenly distributed throughout the network while a scale-free network has a few hubs that are highly interconnected, with the majority of nodes having fewer connections. One way to enhance robustness in a complex system is to create a topol- ogy that is an adaptive mix of a random and scale-free network. With some knowledge of the kind of an attack or perturbation to a complex system, the system would be able to switch states depending on the nature of the perturbation. For example, if substations of a power grid were attacked, the system could switch topologies and begin to distribute power evenly through all its nodes, switching from a discrete to diversified network. This adaptability increases robustness of the system, but would require a tradeoff in expense to implement and maintain necessary resources. The group also discussed vertical connections and redundancies as at- tributes of a complex system that could contribute to robustness of the sys- tem. Some biological systems are among the most robust complex systems in existence. Experimental data from yeast genetic interaction experiments, indicate, for example, that simple redundancy of function accounts for a small amount of the observed robustness. It seems “alternative pathways” and dynamic rerouting of system fluxes in response to perturbation are often the adaptive mechanisms that contribute to robustness in biologi- cal systems. “Vertical connections” are also very important in robustness considerations. These are connections in a network of a complex system that span different hierarchical levels, scales, or employ different definitions within a sub-system. Additional mechanisms are required of a system to maintain and enhance robustness. These include feedback mechanisms in networks, self- organization and self-repair mechanisms. Feedback is also important for establishing buffering mechanisms that increase robustness by stabilizing a dynamic system against perturbations. A simple example of this would be Internet servers. Often, if one server is down, information gets passed to another server in a cluster so that Internet clients can still access infor- mation though their home computers. This buffering mechanism ensures that clients have maximal access to the Internet at all times. Gene-gene and gene-environment interactions reveal buffering relationships through 

TASK GROUP SUMMARY 3A 27 analysis of combinations of perturbations that are synergistic or antagonistic with respect to cellular robustness. Self-organization and self-repair are also present in biological systems and are beneficial for enhancing robustness in any complex system. Moving Forward As the group moved from thought experiments, and attempted to extrapolate findings into the real world, it became possible to generally but solidly define the task of seeking robustness in a system as: Keeping the magnitude of the change in a set of performances (with respect to a set of perturbations) within some limits, and doing so subject to given restraints. Definitions of complex systems in real life can be difficult. There is a hierarchical structure of biological systems such that different measurement scales apply to different levels of the system; different types of perturbations are relevant, and different performance measures must be considered. One must be careful of what scale is selected when initially defining the complex system (and related optimization goals). In certain systems, one might find performance fluctuations on a small scale that would not appear at a larger scale. Or, a loss of robustness at a certain scale might be accompanied by a gain in robustness at another scale. These robustness trade-offs, where per- formance or robustness of a system is sacrificed at one level to be enhanced at another level, exist and must be taken into account when robustness of a complex system is measured. Defining a complex system is also difficult due to inherent dynamics. Connections within an engineered or biological complex system are always breaking, reforming, and changing. Perturbances (e.g., signal transduction) often seem intrinsic to the networks and complex systems themselves. It is challenging to study a robust system that by definition consists of dynamic interactions rendering it resistant to observable change. Despite this challenge, we can still make recommendations for enhancing its ro- bustness. Robustness is incremental and non-linear, so we need to establish quantitative models and tools to measure sources of buffering capacity and better model phenomena such as stability thresholds. Adaptability of net- work topology within system structure is also key to enhancing robustness, as are built-in feedback mechanisms and active control. 

28 COMPLEX SYSTEMS When enhancing robustness or minimizing vulnerability, it is highly advantageous to know as much as possible about the interconnectivity and dynamics of a system. For example, if biological attack were to be mounted within our health care system, it would be advantageous for the attackers to know which individuals are the most connected, and would spread the disease most quickly. This knowledge could be used to attack fragilities as well as hide them, to decrease or enhance robustness of this system. System dynamics, along with interconnectedness, allow prediction of how the system will function and respond. Analysis of diverse types of complex systems, such as biological and man-made systems, is an important aspect of our aspiration to derive principles for how robustness is achieved through network structure and connectivity. TASK GROUP MEMBERS – GROUP B • Kirstie Bellman, The Aerospace Corporation • Jennifer Couch, National Institutes of Health • Raissa D’Souza, University of California, Davis • Tony H. Grubesic, Indiana University • Stephen J. Kron, The University of Chicago • Luis Rocha, Indiana University • Caterina Scoglio, Kansas State University • Alejandra C. Ventura, University of Michigan • Stuart Fox, New York University TASK GROUP SUMMARY – GROUP B By Stuart Fox, Graduate Science Writing Student, New York University Fragility is an inherent component of all systems. But unlike a simple system, in which fragility is equally distributed, complex systems present an uneven landscape of strength and weakness. As a result, robustness, the ability of a system to limit, within some specified range, the magnitude of change in performance with respect to perturbations, can only be under- stood, enhanced, or engineered with the proper intellectual framework. Researchers at the 2008 National Academies Keck Futures Initiative Confer- ence on Complex Systems were unable to develop a complete version of that intellectual system; nonetheless, the Task Group (3B) successfully identified 

TASK GROUP SUMMARY 3B 29 the key questions that the intellectual framework would need to answer, and began the process of constructing that framework. The first step in understanding robustness is acknowledging that ro- bustness is context dependent. This quickly became apparent as the group attempted to answer its original question, which focused on interconnec- tivity and robustness. In some cases, like a gene regulatory network, more connections mean more robustness. But it is not hard to imagine a reverse scenario, like increased plane travel helping spread a deadly pandemic, where interconnectivity decreases the robustness of a system. It turns out robustness does not result from finite set of qualities, the application of which could steel any system against failure, but instead de- pends on the system in question, the goals of the system and the perturba- tions that system faces. Different strategies will work for some systems and not others. Robustness may mean preservation or adaptation depending on the system, and there is always a cost. The cost may come in the form of money, metabolic energy, loss of robustness against a different set of perturbations, and many more forms. In fact, the costs of robustness are as numerous as the strategies for creating and enhancing it. The costs of robustness can never be eliminated, only shunted from one area of the system to another. The very dependence of robustness on context naturally suggests a vague outline of the intellectual framework needed for the understanding of robustness in specific examples, and for the engineering of robustness in man-made complex systems. That framework, at least insofar as the group was able to divine, requires asking four key questions: What is the goal of the system? What are the perturbations? What strategies preserve the goal of the system in the face of those specific perturbations? And what is the cost of each strategy? To test the elegance of those questions, the group used that preliminary framework to analyze four model systems (see Table 1). Looking at each system, whether natural or man-made, the group found that the robustness strategies lend themselves to grouping far more naturally than the goals of the systems, the perturbations, or the costs. This early result hints that while perturbations and costs vary as widely as complex systems themselves, there may be a finite set of robustness strategies applicable in different situations (see Table 2). The first, and simplest, systems the group looked at were the proteins RNAse A and green fluorescent protein (GFP). RNAse A is an enzyme that cuts up RNA molecules in liver cells and serves as the standard biochemi- 

30 TABLE 1  Example Systems System Perturbation Strategy Cost What Is Preserved RNAse A, GFP Temp, pH, cutting into Self-healing, Failure to adapt, design Enzymatic function, glowing piece redundancy time Central and peripheral Temp, chemicals Hardening, diversity, Metabolic Life, cognition, ability to nervous system imbalance, tumors, avoidance… energy (vastly adapt stroke, blow to the everything in Table disproportionate head, etc. 2 consumption), design time Federal highway network Congestion, blockages, Diversity of paths, Construction, Transport, origin-destination link failure spatial separation, maintenance paths, flow provisioning Gene regulatory network Mutations, Canalization, Constructing additional Phenotype, multiple viable environmental connectivity, molecules, design modes (adaptability) conditions, redundancy, time development stages multitasking 

TASK GROUP SUMMARY 3B 31 TABLE 2  General Robustness Strategies • Redundancy (repetition, substitution, overlap) • Feedback • Diversity • Smart nodes/smart edges • Modularity • Self-healing • Spatial separation • Regeneration • Fortification • Balancing control • Buffering between centralized • Cutting losses and decentralized • Canalization components • Avoidance • Trusted/collective • Connectivity intelligence for • Multiple viable modes centralized aspects • Computational reflection (system’s awareness and planning of resource allocation) cal model for protein studies. It gained that model status when Armour & Co., the company that makes Armour hotdogs, purified 10 kilograms of the material and distributed the enzyme for free to research institutions. GFP is a luminescent protein found in jellyfish that is widely used as a marker for biological processes in experiments. Both proteins are far more robust than other similar proteins in the face of denaturing as a result of heat or pH, being cut up, and to the re- placement of component amino acids with other amino acids. When the proteins are denatured and the perturbations of heat or pH are removed, or the cut up parts of the proteins are put together and heated, the proteins spontaneously reform. If they are improperly assembled, the proteins still retain some functionality. That robustness comes at the cost of adaptation. The robustness focuses on maintaining a consistent form, preventing the proteins from undergoing any changes that might enhance the robustness of the protein in the face of other perturbation. Another biological system the group examined was the nervous system, both central and peripheral. The central nervous system consists of the brain and spinal cord, and controls cognition. The peripheral nervous system con- nects the brain to the rest of the body, and controls all other functions. The nervous system faces perturbations such as chemical imbalance, change in pH, loss of oxygen, prion disease, stroke, and blunt force trauma, among others. 

32 COMPLEX SYSTEMS To deal with those problems, the nervous system uses nearly every robustness strategy known to science. For example, a hard skull protects against trauma, an immune system protects against disease, redundancy protects against stroke and single channel failure, flight or fight response avoids or defeats threats to the system. The cost of that wide spectrum of robustness comes in the form of metabolic energy and design time. The brain takes up most of the energy in the body, and a lot of food is needed to fuel a brain that can think its way around perturbation. As far as design time goes, it took evolution hundreds of millions of years to progress from the simplest nerves to the human brain. Of the man-made systems that can be analyzed for robustness, the group spent the most time examining the federal highways. The govern- ment created the highway system in the 1950s to ensure cross continental military supply lines would not be interrupted by Soviet nuclear attack. As the threat of nuclear war receded, the highway system shifted to primarily providing travel arteries for citizens and companies. In both cases, traffic congestion, blockages and link failures were identified as problems. Making sure people, products and tanks get where they are going in the face of those problems requires multiple paths to the same destination, increased linkage between the roads of the federal highway system and state and local highways, and geographic distance between hubs to protect against catastrophic, regional problems like hurricanes and nuclear explosions. Robustness of the highway system has both straightforward and more nuanced costs. The obvious cost is money. Construction and maintenance of the highway system need to be funded. The more subtle cost arises through a concept called Braess’ paradox. It turns out that the robustness strategy of adding more roads to decrease congestion actually leads to more traffic in some cases. While these examples set the group on the right track, many questions remain unanswered. How does one generalize strategies from examples? How best to implement a general robustness strategy with the proper specific detailed information to make it work on the target system? What benefits and costs do different general robustness strategies provide for specific complex systems? Moving forward, the group decided to continue working on this problem, and over the course of time, to finish developing the framework for understanding, leveraging and enhancing the robustness of complex systems.