Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
6 What Are the Engineering Principles of Life? In order for a space shuttle to launch into orbit, dock with the space station, and return safely to Earth, thousands of highly trained individuals and countless sophisticated machines, computer programs, and commu- nications devices need to be engineered, tested, and coordinated. When an orchestra plays a symphony or a basketball team executes a perfect last-second play, each of the participants has dedicated years of training, practice, and discipline. These achievements are examples of human skill, ingenuity, and application of knowledge. Some characteristics of these quintessentially human enterprises are mirrored in basic biology, and nature is full of examples of complex outcomes that result from the coordinated behavior of many simple parts. Across many fields of biologyâfrom the organization of the cell, to the development of multicellular organisms, to the function of the brain, to the group behavior of insects and birds, to the response of ecosystems to environmental changeâcomplex coordinated phenomena are seen to arise out of interaction of a myriad of components. The engineering principles that make possible a space shuttle can be en- capsulated in an engineering textbook. Is it possible that there are similarly fundamental principles governing the organization of dynamic interacting systems that hold across all scales of biology? The key to understanding such organizational principles will involve developing a theoretical basis for how biological entities generate aggregates of higher complexity: that is, the constructive principles of biological organizations. Advances in understand- ing of these biological systems is an especially promising area of research in biology that could have immediate consequences for the understanding of organisms and further applications to complex, human-engineered systems. 90
WHAT ARE THE ENGINEERING PRINCIPLES OF LIFE? 91 An alternative view of engineering is that the field deals with solving con- straints or understanding constraints imposed by the characteristics of the parts or organizational structures. Therefore, understanding or developing a theory of constructive engineering principles of life will also yield insights into limits and constraints of biological systems. The previous chapter discussed how the interior of the cell is highly or- ganized. In fact, much of nature is highly organized, and the organization, or regularity, often seems to emerge without any external direction. A single fertilized egg develops into a mature multicellular organism with all of its many organs, limbs, and blood vessels in the right places. An ecosystem damaged by fire gradually returns to its original mix of species, reorganiz- ing the interdependent community. This chapter will explore the common organizational characteristics and constructive principles of biological sys- tems that lead to complex behaviors, products, and processes. CORE CONCEPTS Some core concepts that link different kinds of complex systems are modules, nodes, networks, emergent behavior, topology (or architecture), and robustness. Table 6-1 provides definitions of these terms and gives examples from several different kinds of systems. A brief caveat is in order. In this chapter, the terms âmodularity,â âemergence,â and ârobustnessâ will be used to describe characteristics of biological systems that arise at different scales and are in need of fur- ther conceptual development. However, the terms have been used in other ways in different domains. However described and however generalizable they may be, the phenomena of modular organization, complex ensemble behavior that might be called emergent behavior, and robustness in bio- logical processes exist and can be described and measured. Whether the best approach will be classical, using existing tools, or whether an entirely new set of formalisms will be required, the problem remains that effective conceptual and theoretical treatment of those topics is not yet available. A satisfactory description or computation of those phenomena is a critical challenge for the future of biology. Characteristics of Modules In every biological organization certain divisible parts are recognizable whose repetition and elaboration seem to generate the whole. These parts are often recognized as physically distinct unitsâthe canonical example being the individual organism. In some cases, such units had a conceptual existence before their physical manifestation was known. An example is âthe geneâ as described before the development of the chromosomal theory
92 THE ROLE OF THEORY IN ADVANCING 21ST-CENTURY BIOLOGY TABLE 6-1â Core Concepts Describing Complex Systems Term Definition Examples Modules/nodes Integrated units that Musicians in an orchestra can be combined in Genes in a developmental pathway many ways Neurons in the brain Locusts in a swarm Networks Systems of Orchestra connected nodes Regulatory feedback loop Brain Swarm Food chain Topology The way the nodes Hierarchical (all individuals connected to (or architecture) are connected in a one leader) network Scale-free (some individuals connected to lots of others, most connected only to a few others) Distributed (individuals connected to neighbors) Emergent behavior The output of a Music (or properties) network Development of a limb or an eye Memory/thought/perception Migration Community Robustness Ability of the Many different orchestras can play same network to provide music; orchestras can play many different the same output pieces of music despite internal Many genes can experience mutation but (e.g., the loss of limb or eye still develops normally some modules) or Some neurons die and most brain activity external changes continues normally Swarm travels despite death of individual locusts or geographical barriers Communities continue despite extinction of some species of inheritance. Genetic units were defined by certain abstract properties such as segregation of phenotypes upon genetic segregation without knowl- edge of their physical embodiment. In other cases, loose collections can sometimes be considered a unit with respect to some process or function, as in, for example, a population of individuals that is spatially dispersed can act as a module in an ecosystem. âModuleâ is a term that seems to capture the sense of these biologically relevant units. While the term is not completely precise, it captures the notion of components or parts organized
WHAT ARE THE ENGINEERING PRINCIPLES OF LIFE? 93 into larger units that are integrated within and independent (or dissociated) of other units. Furthermore, these modules are generally finite in variety, have properties of superposition such that multiple modules can be com- bined, and have a certain uniformity to external interface (like bumps on a Lego block) such that a module A that is part of a larger complex can be swapped with module B (Bolker, 2000; Winther, 2001; Schlosser and Wagner, 2004). Such composition may be physical, but the concept of interchangeability within populations (or âdemographic replaceabilityâ) is an important aspect of modularity that connects modules to evolution- ary dynamics (Wagner, 1996). Finally, modularity, like other architectural principles, also constrains or determines the limits of design and function of evolved biological systems. Examples of modular organization are found at all levels, from sub- structures of proteins or RNA (Ponting and Russell, 1995; Corbi et al., 2004; Pasquali et al., 2005; Del Sol et al., 2007), to assemblies of proteins that seem to be comprised of surprisingly small numbers of components in a wide variety of combinations (e.g., Devos et al., 2006), to cellular organiza- tions in brains (Redies and Puelles, 2001), to anatomical structures (Raff, 1996; Yang, 2001) and regulatory or metabolic function Â(Magwene, 2001; SegrÃ¨ et al., 2005), to classic ideas of ecological communities Â (Clements, 1936) and even abstract processes such as cognition (Barrett and ÂKurzban, 2006). Across these scales and substrates, modular organization has been described in terms of physical structure (e.g., anatomical parts or macroÂ molecular geometry), biological function (e.g., cognitive processes), com- ponent interactions (e.g., protein complexes), temporal processes (e.g., metabolic flux or development), and genetic architecture. Modularity at these different levels is sometimes coincident, for example, a modular pro- tein domain may carry out a modular function, while at other times no such correspondence can be found. It is an open question the extent to which the concordance of modularity at these different physical and functional levels is promoted by evolutionary dynamics (e.g., Cheverud et al., 2004; Snel and Huynen, 2004). Characteristics of Interfaces Between Modules Biological modules typically are made up of other modules at a smaller scale. For example, a cell (itself a module) contains various structural, metabolic, and gene regulatory networks as modules, which in turn contain proteins and RNA molecules as modules. Therefore, the critical aspect of a module is that it might contain many internal parts whose interactions and dynamics are extremely complex, but it has a defined and finite âexternalâ interface that can be connected to other modules. A cellâs internal physiol- ogy and structure might be dynamic and complex, but to other cells what
94 THE ROLE OF THEORY IN ADVANCING 21ST-CENTURY BIOLOGY matters are the cell membrane and interface components such as receptors, transporters, and junctions (Bonnefont et al., 2005). The internal dynam- ics of a module and the interaction of its parts could be both cooperative and antagonistic and both optimized and random; the parts could also be ephemeral, experiencing constant turnover. In fact the degradation or death of individual modules, while the overall function is maintained, is another universal theme of biological organization. The critical point is that mod- ules should show external coherence, independent of internal complexities. In addition to an invariant external interface, the interfaces should be finite in kindâsimilar to the finite number of interfaces on a computer such as the USB interface. Sharing a uniform or finite interface (e.g., the phospho-diester bond in nucleotides), especially across functionally distinct modules such as the mitochondrion and the Golgi, allows exchangeability of the modules (Del Sol et al., 2007; Pereira-Leal et al., 2007). The ability to exchange modules creates the possibility of generating combinatorial complexity. For example, during development, gene regulatory feedback loops that have the property of driving cells into a new developmental stage can, through evolution, be linked to other developmental modules to implement major phenotypic changes. In Box 6-1 an example is given of a regulatory loop preserved in star fish and sea urchins but which in sea urchins has evolved to link to another module that drives the development of a skeletal system. A module as described here is made up of interacting parts, which together interface with the external environment. Variations of the ques- tions âHow are such interacting ensembles constructed?â and âHow are they maintained?â are found in all subfields of biology. Enumerating the composition and interaction of parts in a cell, in an organ, in a popula- tion, and in a community are classic research programs. What varieties of RNA are in a cell and how do they interact with the DNA genome? What are the different types of neurons constituting a hippocampus? How many different species of bacteria make up a gut community? Such inqui- ries might be considered an essential part of the classic reductive research paradigm, the goal of which is to use the enumeration to build a construc- tive understanding of emergent properties from the bottom up. Attempts at a constructive understanding of the combined action of the parts lead to the next level functional or interrelational questions: Are all the entities essential? Do the entities segregate into functional groups? What types of interactions are present and, at an abstract level, what is the network topol- ogy of their interactions? What are the forces that maintain the ensemble through dynamic changes? Although the research program of characteriza- tion and assembly of parts is classically reductive, from a modular perspec- tive, these questions or approaches clearly apply throughout the scales of modular hierarchyâfrom molecular parts to ecosystems. To put it broadly,
WHAT ARE THE ENGINEERING PRINCIPLES OF LIFE? 95 a key conceptual challenge is to ask whether there is a common theoretical framework to how modules are created, maintained, and disposed of at all different scales of organization. Because modules are ensembles of interacting components, it seems intuitive that cooperation is critical to generating emergent properties from the interacting entities. For example, a single individual must be able to perform many tasks to succeed in a changing environment. Sometimes, however, different individuals, and even different species living together, perform different functions in seeming cooperation. Cooperation can in- clude complex social interactions such as division of labor among or- ganisms with complementary metabolic abilities, the provision of shelter, resource gathering, reproduction, and dispersal (Box 6-2). Even within interacting parts that form a coherent whole (such as a module), competition or antagonistic interaction may also be an essential force. For example, many gene regulation processes within a cell involve antagonistic interaction of two regulatory proteins competing for the same space on the DNA. Some models have suggested that learning and cogni- tion involve competition among neurons and pruning of connections during early development (Rakic et al., 1994). Food webs are an essential part of a community structure and involve antagonistic relationships. Some theo- retical models (Livnat and Pippenger, 2006) suggest that internal conflicts might be an essential component of system optimization and that there might be optimally selected levels of modular integration (Hansen, 2003). In some sense, categorization of component interactions into cooperation or antagonism may reflect an anthropomorphic point of view; scientists can choose to describe the interaction of two proteins competing for the same DNA location as antagonistic. From a control system point of view, however, this is simply one way to implement a bi-stable switch. Is a stone arch held up by the cooperation of appropriately molded stones or by the antagonistic opposing forces acting on the keystone? Similarly, the participation of two species in a mutualistic relationship can be characterized as cooperative or as a tough and ongoing negotiation. For example, in the mutualistic interaction between rhizobia (a group of nitrogen-fixing bacteria) and legumes (including such vegetables as peas), the extent to which the bacteria supply nitrogen to the plant and the plant supplies carbon to the bacteria is just beginning to be approached using cooperative game theory (Akcay and Roughgarden, 2007). In another example, the quintessentially âcooperativeâ act whereby maternal and paternal genomes are combined through reproduction also includes a com- petitive element (Haig, 1993). Thus, one key conceptual question is whether a unified framework for understanding the dynamics of components can be constructed not so much in terms of proximal quality of interactions (such as cooperation
96 THE ROLE OF THEORY IN ADVANCING 21ST-CENTURY BIOLOGY Box 6-1 Comparative Network Architecture The genetic networks regulating the development of the embryonic endo- mesoderm in two echinoderms, sea urchin and starfish, provide an example of comparative network architecture. The regulatory network for endomesoderm development in the sea urchin has been worked out in significant detail (see H Â inman below) allowing comparison of the same set of genes in a related but long diverged lineage. The sea urchin S. purpuratus and the starfish Asterina minita diverged from their common ancestor more than 500 million years ago. The endomesoderms in these animals develop similarly except that the sea urchin has a cell lineage that develops into a prominent skeleton that is entirely lacking in star fish. When a set of key regulatory genes for endomesoderm development was examined in starfish, a three-gene feedback loop that is a key component of the sea urchin system, was found to be almost unchanged in starfish. The conserved circuit has been preserved since the Cambrian era in both lineages. The structure of the basic developmental circuit is remarkably conserved. The five genes in the regulatory circuit are wired together in essentially the same way. But there are a few key changes in the circuitry. For example, the sea urchin has an autoregulatory loop (of the Krox gene) not present in the starfish, while the GataE gene is auto-activated in the starfish and not in the sea urchin. Also, the FoxA gene represses the GataE gene in starfish but not in sea urchins. These three changes, indicated as red lines in the figure, represent divergences of the two circuits. The major difference between the two systems is a major rewiring of the external connections of this circuit. In the sea urchin the Tbr gene is not con- nected at all to this circuit. Tbr is used entirely in the skeletogenic network in sea urchin, a function not present at all in starfish. In the starfish the Tbr gene is still there but is regulated by this circuit through connections (shown in red) to three genesâOtx, GataE, and FoxAâthat are not present in the sea urchin. or antagonism) but in terms of how such interaction contributes to the control architecture of maintaining the whole module. Understanding how cooperation and competition may be viewed as two sides of the same coin poses a conceptual issue whose resolution offers the prospect of greater understanding of the ecology and evolution of mutualism among species. In light of modularity, can concepts of population genetics and evolu- tionary change be modified? What are the levels of modularity at which nat- ural selection can and cannot act? Evolutionary progress depends on some aggregate of modules. This situation requires thinking beyond individual se- lectionâor what has been called the problem of âlevels of selectionâ (Buss, 1987). Evolution in the context of teams or coalitionsâthat is, ensembles of modulesâwould apply to the many organisms that forage, evade predators, and reproduce in the context of teams within a social system (Roughgarden
WHAT ARE THE ENGINEERING PRINCIPLES OF LIFE? 97 This example illustrates that developmental circuitry can be conserved over very long periods of time, but that it can be modified by evolutionary processes in several waysâconnections can be gained and lost. This will alter the computation that is made by this circuit but only in small ways. The circuit can also be rewired to drive an entirely new function by adding connections to the cis-regulatory region of genes in other developmental networks (Hinman et al., 2003). PMC-specific Eve Transcription factors Krox Otx Krox Otx Endo16 Tbr Tbr Lim Bra FoxA GataE Bra FoxA GataE Endomesoderm Endomesoderm Skeletogenic Specification Specification Structural Genes Sea Urchin evolution of network over ~ 500M yrs Starfish Figure and photos courtesy of David Galas. Box 6-1.1 color et al., 2006). Understanding the relation of team selection to individual se- lection, together with the adaptive formation and dissolution of such teams, poses a major conceptual challenge for the future. Because networks of modules are often embedded within networks at higher scales (networks of networks), mathematical tools such as nonlinear dynamics and numerical simulations are critical to understanding how these biological systems depend on the properties of their components. This can be seen directly in neuroscience, where the intrinsic electrical activity of individual neurons depends on the number and kind of voltage-dependent currents, and simulations are crucial for understanding how the properties of the currents alter the excitability of the neurons. Likewise, simulations and mathematical analyses are crucial for understanding how the behavior of networks of neurons is influenced by changes in synaptic strength. At still
98 THE ROLE OF THEORY IN ADVANCING 21ST-CENTURY BIOLOGY Box 6-2 Cooperative Behavior of the Slime Mold Dictyostelium discoideum The slime mold Dictyostelium discoideum is one of the best-studied exam- ples of cooperative behavior. When their food supply is exhausted, large numbers of single-celled amoebas of this organism coalesce into a wandering multicellular slug-like creature that then differentiates into an immobile spore-producing âfruit- ing body.â The fruiting body has well-differentiated structuresâa base, a stalk, and a reproductive head. In a sense, the cooperation of individual slime mold cells produces a coherent higher-scale individual as a slug and a fruiting body. The slug-like assemblies have a definite anterior and posterior, respond to envi- ronmental gradients, and have coordinated motility: that is, the complex internal interactions of the individual cells are hidden to produce a higher organization that has distinct interface with the external environment. higher levels of organization, simulations and mathematics are important for understanding how neuronal circuits operate in behavior. Building and interpreting the results of mathematical models are important ways to gain insight into how the interactions of nonlinear processes give rise to system behavior. Because biological systems are typically composed of a hierarchy of modular units, it is a challenge to gain an understanding of Âevolutionary dynamics at various levels of organization. But what accounts for the emergence of the modular units themselves: that is, what accounts for the evolution of the modular architecture? An example of evolving modular ar- chitecture can be seen in the genomes of organisms. Genomes are organized into modules at various different scales. At the largest scale, ensembles of genetic material are organized into chromosomes. Within a chromosome, material is organized into contiguous blocks of information that code for proteins, and sometimes groups of functionally similar proteins are orga- nized into neighboring blocks called operons. However, there are variations of all kinds: introns that break up the protein-coding regions, alternatively spliced proteins, heterochromatin, euchromatin, dynamically remodeled chromatin modifications, insulators, DNA modification blocks, and in- trons within introns. Thus, there is a tremendous and dynamic variation in modularity of the genome within the same individual and across different species. What evolutionary forces govern the level of modularity in these genomes? Or more broadly, given the preponderance of modular organiza- tion in biological systems at all levels, what are the evolutionary dynamics that lead to such modularity? A typical modern computer central process-
WHAT ARE THE ENGINEERING PRINCIPLES OF LIFE? 99 ing unit has 100 million transistorsâit is impossible for a single person to design such a construct from the ground up with a global understanding of all the individual transistors. Thus, modular architecture allowing incre- mental buildup of complexity is an engineering imperative. Is that modular architecture also an imperative for biological systems? Does modularity have direct impact on fitness under some suitably posed dynamics? Is it a byproduct of selection for robustness or ability to evolve (such as the ability to generate variations)? Network Topologies/Architecture The interactions of modules form architectural organizations at a higher level. A common theme is that modules become hierarchically or- ganized to produce modules at a larger scale. This characteristic is seen at multiple levels from metabolic pathways (Ravasz et al., 2002; Yu and Gerstein, 2006) to food webs (Pimm et al., 1991) and of course to the organization of interacting circuits in the brain. The recent explosion of functional genomics data has led to unprecedented large-scale assays of biological component interaction such as gene regulatory interaction and protein-protein interaction. A natural representation of such interactions is as a graph where each node represents an interacting unit (e.g., a protein) and each edge represents the functional interaction (e.g., physical collision). These graphs are commonly called networks, and the availability of large- scale networks has led investigators to notice certain statistical regularities in the structure of the node-edge connectivities (or the so-called topology of the graph). One statistical quality that has been suggested is that in many biological systems there are a few highly connected nodes while most other nodes are sparsely connectedâthis type of network has been called scale- free (BarabÃ¡si and Albert, 1999; Jeong et al., 2000). It has been suggested that this statistical characteristic contributes to stable function in the face of network perturbation. The representation of biological interactions as networks is a theo- retical abstraction that has led to a wealth of descriptions of complex bio- logical ensembles as architectural motifs. For example, network Âtopologies seem frequently to have certain subnetworks that may allow certain kinds of dynamics or information processing (Yeger-Lotem et al., 2005; Jiang et al, 2006), and the network topologies may be correlated in functional ( Â Magwene, 2001) and co-evolutionary groups (Qin et al., 2003; Tan et al., 2007). While a network is a static representation of component interÂ actions, a dynamical view of biological processes may be obtained by considering how network topologies change over time. This approach has led to statistical characterization of dynamical structure of modularity in networks (Han et al., 2004) and, furthermore, suggestions that the logic of
100 THE ROLE OF THEORY IN ADVANCING 21ST-CENTURY BIOLOGY process regulation may be embedded in a dynamical network representa- tion (Tu et al., 2005). The interest in network abstraction has been such that some have suggested that ânetwork scienceâ may comprise a new subÂ discipline linking physics, biology, and chemistry and spanning scales from the molecular to the ecological (BarabÃ¡si, 2002). EMERGENT BEHAVIORS The compass termites Amitermes meridionalis and A. laurensis build complex colony mounds reaching up to 20 feet in height with such distinct global properties as specific compass orientation and air columns that help regulate temperature (Korb and Linsenmair, 1999). These properties are thought to be ecological adaptations to local environments. The physical scale of the mounds is several orders of magnitude larger than the indi- vidual termites. The behavior of each individual gives no clue as to how the group manages to construct ventilation shafts that would seem to require a blueprint at the scale of the mound itself. Closer to home, despite more than 50 years of intensive study, no explanatory model connects the activity of individual synapses to how humans store the memory of, for example, the face of an individual and then retrieve the same image among thousands of similar stored images (Kandel, 2001). As complex as are memory encoding and retrieval, they are simpler processes than higher cognitive processes like writing a sonnet. Thus, although biological systems are comprised of modules that hide the internal complexities, the composition of the modules generates unexpected ensemble behavior at a higher scale that is difficult to predict from knowl- edge of the parts themselves, even when the composition is of multiples of the same modules (cf., termites within termite colonies). Because the collec- tive behavior of these parts can be so surprising, the term âemergenceâ has been used to describe phenomena that seem to defy reductive understand- ing. This term has become somewhat burdened because of its use by some scientists to argue that certain natural phenomena cannot be understood by current scientific methodsâa contention that is widely disputed. Neverthe- less, the term carries an important metaphor of ensemble properties that are difficult to predict from our current models and therefore the term is used here in a strictly descriptive sense. A reasonable way of thinking about emergent behavior might be to focus on the level or scale at which the rules reside. If the rules are specified at a low level, for example, the individual termites, and the patterns and structures, like termite mounds, emerge at a scale where there are no rules specified, we may call this emergent behavior. Ideas of how some component interactions might give rise to emergent behaviors in biological systems can be deduced by analogy with engineered
WHAT ARE THE ENGINEERING PRINCIPLES OF LIFE? 101 systems such as electrical circuits. For example, positive feedback loops, where an enzyme converted from an inactive to an active state in turn ac- tivates more copies of the same enzyme, are able to amplify small signals and give rise to large-scale switch-like responses to important but small changes in a cellâs environment (Hlavacek et al., 2006). Conversely, negative feedback loops can dampen the effects of fluctuations in system inputs and allow cells to ignore uninformative noise in their environments. Combining feedback loops with time delays can give rise to oscillatory behaviors, such as the daily changes in plant metabolism that accompany the rising and setting of the sun. In many biological systems, relatively simple feedback loops and oscillators seem to act as modules within very large-scale net- works. Some of these networks produce extremely stable overall behavior, such as the physiological regulatory mechanisms that maintain our body temperature and blood pH within very narrow ranges. Other networks are able to generate irreversible switch-like behaviors, as when different cell types within a developing multicellular organism become committed to particular cell fates (cf. Alon, 2006)(see Figures 7-1 and 7-3 in the next chapter). Many of the most interesting biological networks combine aspects of reversible and irreversible behaviors that in ensemble produce complex behavior, such as the ability of an animalâs nervous system to learn and remember. Earlier discussions in this chapter suggested that modules are made up of parts that collectively show coherent invariant properties in their inter- face with the exterior. These invariant external properties are derived from the complex and dynamic interactions of the parts and encapsulate the parts in a simpler and uniform interface with the environment. (Here the term âenvironmentâ refers to all that is external to an individual module, includ- ing other similar modules.) The âinvariant properties for external interfaceâ can also (again loosely) be called emergent properties of the ensemble that comprise the module. In the example of a slime mold slug, the ensemble displays an emergent property of coherent directional motion (Box 6-2). A protein is composed of a string of amino acids that, when placed in a solvent medium, folds into geometrical shapes. The folded structure then displays an emergent property of catalyzing chemical reactions with exqui- site single-molecule specificity. Can emergent properties be predicted from the knowledge of parts? If so, would the same theory apply to all different scales at which modules can be identified? Consider the problem of protein folding and prediction of its func- tion. At first glance it appears that the only barrier to understanding is computational. A sufficiently fast and large computer could allow models of molecular motion in a force field to yield a prediction of equilibrium form. Given the form, the model of geometric lock-and-key for protein- based catalysis can be applied and brute force computations can be applied
102 THE ROLE OF THEORY IN ADVANCING 21ST-CENTURY BIOLOGY again to ask what reactions the folded protein would carry out. Then, are emergent properties simply those consequences of group interactions that are currently too difficult to compute? In the past 20 years, considerable development has taken place in the somewhat diffuse area of multiscale computing (Theodoropoulos et al., 2000; Brandt, 2001; Kobayashi et al., 2001). Those efforts cover areas from multiscale functional analysis such as wavelet analysis in signal processing and image analysis, multiscale clustering for databases, coarse-fine time-stepper dynamical systems (with recursion), multiscale optimizations, and piece-wise linear hybrid systems, as well as techniques such as Monte Carlo, Markov Chain, and grid com- puting. Those developments have enabled progress in many areas, including large-scale solid-state physics, fluid mechanics, molecular dynamics, image handling, genomics, and others. One idea common to those techniques is computation on small patches at a lower scale (microscopic scale) that can be used to interpolate at a coarser scale (macroscopic scale) but in a controlled, bounded manner. Bounded approximation at multiscalesâas canonically expressed by the wavelet analysisâis an integrative approach that can be used to connect phenomena at different scales. A useful con- ceptual development in any area must be eventually connected to data and theory posed in a computable form. Development of multiscale computing and multiscale integration methods is critical to many of the cross-cutting questions discussed in this report. Many of what are called emergent properties involve physical Âgeometric form and direct interaction mediated by spatial and structural contexts. But many aspects of emergent properties, such as the construction of air shafts in termite mounds, require information processing among the participating components so that each component reacts in accordance with the informa- tion acquired from other components. Earlier in this chapter, cooperative interaction was suggested as a key ingredient in module formation. Bacte- ria, for example, form complex biofilms on human teeth. These biofilms contain hundreds of species in relatively stable interacting communities. M Â icrobiologists call these interacting communities in which functions are divided âconsortia.â Many bacteria within these consortia make âautoÂ inducers,â which are chemicals thought to permit communication, not only within, but also among, species. The existence of message-Âsending and m Â essage-receiving capacity in diverse species of bacteria has been Âfacilitated by the ability of these organisms to exchange genetic information through the transfer of plasmids and phages. Therefore, a key ingredient for in- teracting parts, such as different species of bacteria, to display emergent behavior seems to be the existence of some process to pass information, sense information, and react to information. Information could exist as a minute quantum to a single part, but the collective computational action of the ensemble could lead to changes in synaptic strength that is the substrate
WHAT ARE THE ENGINEERING PRINCIPLES OF LIFE? 103 Box 6-3 Reconfigurable Robots as an Analogy of Emergent Properties of Biological Systems The potential utility of understanding how communication and interaction of modular parts lead to emergent properties can be found in the area of engineer- ing reconfigurable robots. Rather than engineering specialized robots for each specific purpose, it would be desirable to develop modular components so that robots could be reconfigured for specific tasks. This goal raises a number of inter- esting information problems. Assuming that a human programmer could provide the proper programming for each possible configuration of modules and that the modules are assembled in the proper form, how does the robot recognize its configuration and find the right program among the suite of available programs? Having defined the program (e.g., the robot is able to determine that the current configuration of modules is appropriate for detecting land mines, not evacuating wounded soldiers or entering a building), how does each module recognize what part it is to play in this program? That is, if there is a particular control process for the left side of the robot and another for the right side of the robot, how does each module recognize that it is in fact a module on a particular side? In a both abstract and very real sense, individual termites appear to carry out just such a computa- tional paradigm and calculation to produce a mound. What is this computational architecture? Is there a common information-processing framework for emergent properties from individual synapses in a human brain to cells in a flowering plant to individuals in a communityâan architecture that could be applied in systems engineered by humans? of memory formation or the construction of a ventilation shaft in a termite mound. How components interact locally to produce global patterns is at the heart of the matter of emergent properties. A fundamental problem is what type of information transfer is carried out and how individuals are equipped to sense the input and act on it appropriately to produce the global patterns (Box 6-3). In engineered systems, a basic step in creating complex behavior in- volves the construction of a broad implementation plan or architecture for the desired system. For example, the idea of computers communicating over shared lines with electromagnetic signals is a simple concept. The physi- cal implementation of this idea, however, requires decisions on whether to encode bits in voltage or frequencies of the electrical signals, how different machines should share the same physical line, and a scheme to parse up the individual messages, just to name a few. In biological systems, evolution has preserved a number of architectures that underlie complex processes. These architectures represent the broad
104 THE ROLE OF THEORY IN ADVANCING 21ST-CENTURY BIOLOGY implementation plans of an ensemble of components to produce emergent properties: that is, the Bauplan of the organisms at all scales (cf., Raff, 1996). Details of the architecture determine the efficiency of the implemen- tation, constraints arising from the chosen architecture, and all the possible different forms that can be derived from that architecture. Architectural considerations explain certain component actions that might be difficult to understand without the broader overview. For example, the presence of certain cells in the limb buds of mammals is difficult to understand unless it is known that the architecture of the developmental process calls for digit formation by programmed cell death of interdigit tissue rather than apical growth of digit tissue. Therefore, to achieve a conceptual understanding of emergent properties requires the development of a theory on the architec- ture of biological systems, a theory of the Bauplan applicable to scales from protein structure to ecosystems. ROBUSTNESS OF BIOLOGICAL PHENOMENA By technological standards, all organisms are highly complex, consist- ing of hundreds of thousands of interacting chemical species and thousands of regulated genetic elements. Intuitively, complexity seems to imply insta- bility: The more things that can go wrong, the more likely the system will fail. Yet biological systems are stable. The ability of biological systems to maintain similar states or robust processes even when perturbed is mani- fested at all levels of organization from the regenerative dynamics of forests after a fire (so-called gap dynamics), to the development of whole organisms from fractions of the initial embryo (twins), to the stable folding of large proteins at boiling temperatures (e.g., within thermophilic microorganisms). Such robustness or stability is difficult to achieve in engineering settings; for example, despite many safeguards and redundancies, a single power station failure brought down the entire northeastern U.S. power grid in 2003. It is hard to find any biological processes that do not have specific features that promote robust function under varying conditions. Develop- mental biologist C. H. Waddington coined the term âcanalizationâ to de- scribe organismsâ ability to carry out the same function in various different environments. âDifferent environmentsâ or âvarying conditionsâ can also include genomic variability. Biological robustness can be classified into at least two types: robustness vis-Ã -vis environmental perturbations and ro- bustness vis-Ã -vis genetic perturbations. Stable development of an embryo despite temperature fluctuations is an example of environmental robustness, whereas the redundancy of the genetic code is an example of robustness against mutational changes of DNA. A tension arises, though, when one considers that if an organism were completely robust to genetic change (in
WHAT ARE THE ENGINEERING PRINCIPLES OF LIFE? 105 other words, if mutation never led to change in form or function), there would be no phenotypic variation for natural selection to act upon. The ability to function despite external change, by contrast, seems to be an immediate fitness-enhancing factor, so a model to predict the evolution of robustness to environmental fluctuation might not be as problematic as robustness to genetic change (Wagner et al., 1997). Indeed as noted, h Â omeostasis, that is, constancy in relation to perturbations, is a fundamen- tal property of living systems. Are there general principles by which an organism maintains such robustness under a wide range of perturbations? What are the limits to that robustness? Why are organisms not omnipotent in their biological function? Organisms clearly display robustness to environmental and genetic per- turbations, but they also display a certain kind of âprocess robustnessâ that is related but not necessarily connected to robustness against perturbations. Recently, studies of molecular processes at the microscopic level (such as the cellular level) suggest that these molecular activities are extremely vari- ableâor noisy (Samoilov et al., 2006). For example, with respect to gene transcription, a gene was considered as âonâ or âoffâ or perhaps âhighly expressed,â but detailed measurements suggest that the transcriptional a Â ctivity of an individual gene is variable and carried out in stochastic bursts (Elowitz et al., 2002). At the other end of the scale, simple ecological popu- lation dynamics such as prey-predator dynamics can be described reason- ably well as a limit cycle, but the individual dynamics of prey capture, birth, death, and other parameters are extremely variable (Ellner and Turchin, 1995; Grenfell et al., 1998). Thus, dynamic biological processes, such as those in cell cycles, organismal development, or ecosystem nutrient cycling, may have very high component-wise variation. How, then, do these systems achieve precise system-level function despite such noise? An important aspect of robustness is that biological systems display ro- bust ensemble behavior at one scale despite the dynamic turnover of modules at a lower scale. An individual displays robust function while its component cells are constantly undergoing birth and death. Individual neurons maintain relatively constant activity patterns for much of the lifetime of the animal, despite the fact that the ion channels and receptors that control excitability are constantly being replaced at time scales of hours or days. Communities show consistent properties while individuals undergo birth and death, and even when an entire species becomes extinct. As discussed above, biological systems show robust external properties independent of internal complexi- ties like turnover and noise. Thus, biological robustness is not just a static property obtained from materials or construction, but often a dynamic prop- erty where system function is maintained by dynamic organization such as various feedback and feed forward circuits or stable attractors.
106 THE ROLE OF THEORY IN ADVANCING 21ST-CENTURY BIOLOGY Despite their robustness, biological systems also have profound vulner- abilities; for example, a single genetic change can cause a fly to develop without any eyes or with legs where its antennae ought to be (Raff, 1996). The fact that biological networks such as gene regulation networks have such critical nodes allows relatively small changes generated by mutation to give rise to new emergent properties. Within a given network, scientists currently lack general methods to predict which nodes are most likely to be critical and are, therefore, loci of both vulnerability and evolutionary opportunity and which nodes are relatively unimportant. A critical set of questions related to robustness is when or how a biological systems (e.g., networks of interaction) are robust and when or how they are sensitive (Samoilov et al., 2006). Engineering robustness in manufactured products is an extremely dif- ficult task. Using redundant parts is one standard engineering solution to increase robustness, but this strategy only works for catastrophic failures that can be recovered by backup partsânot for constant noise. Standard feedback control strategies also are applicable but only up to a certain degree of noise. By contrast, living organisms seem to have built-in mecha- nisms of robustness; remarkably, these robustness properties are distrib- uted throughout different scales of organization. For example, because of the way that multiple codons can indicate the same amino acid, a protein is robust to some (but not all) possible mutational changes. When the protein folds, it is guarded against misfolding by chaperone proteins. If it misfolds, it is discarded by a quality control monitoring system and a fresh new copy of the protein is generated in its place. Even when a protein is entirely removedâfor example, by deletion of a geneâthe cell often has checkpoints for detecting such events and system-level regulation to compensate for the perturbation. Loss of cells in an individual can trigger stem cell proliferation. Likewise, aberrant cell proliferation is checked by induced cell death. At a different scale, individual loss from a population leads to increased birth rates and the loss of a particular group of species from communitiesâfor example, loss of certain trees from wind damage in a forestâleads to adaptive recovery by other opportunistic species. Conversely, organisms are also exquisitely vulnerable to particular per- turbations. Loss of a key predator could lead to a qualitative reorganiza- tion of a community, changing the level of a key molecule could lead to a switch in metabolism from dormancy to active proliferation, and chang- ing the right set of amino acids could change the structure of a protein from an alpha-helix to a fundamentally different beta-sheet. Therefore, a key conceptual challenge is to understand and develop a theory of how robustÂness is promoted in biological systems and how it interplays with the control of sensitivity of the same systems. Formal mathematical models and simulations can quantitatively explore the boundaries of parameters
WHAT ARE THE ENGINEERING PRINCIPLES OF LIFE? 107 for robust action, and this kind of question is one for which formal models and simulations are useful. Although there is encouraging progress in demonstrating robustness and its evolution, there are still great challenges ahead and unforeseen im- plications. One possibility is that interplay between robustness and break- down of robustness can facilitate the evolution of novel phenotypes through the accumulation of hidden genetic variationâthus, robustness may be precisely the characteristic that produces unexpected new forms. Another possible consequence of robustness came from a simulation study on the evolution of RNA secondary structure (Ancel and Fontana, 2000). That study showed that evolution of robustness in RNA secondary structure led to a selection for modular decomposition of different âmorphologicalâ elements (stem loop regions) in the melting profile of the molecule. Those results suggest that modularity could be a coupled feature of robust sys- tems. There are still challenges to understanding the interaction of modular architectures and robustness properties, and using model systems and high- performance computational simulations will be an important approach. Studies of molecular processes at the microscopic level (such as the cellular level) suggest that molecular activities are extremely variableâor noisy (e.g., Pedraza and van Oudenaarden, 2005; Rosenfeld et al., 2005). At the other end of the scale, individual turnover in a food web is subject to great variability and stochasticity. How do these processes with high in- herent variability achieve precise system-level function despite such noise? Synthetic gene regulatory circuits have been constructed in bacteria that display precise dynamic behavior such as an inducible bi-stable switch (Isaacs et al., 2003). However, the dynamic behavior is usually at the mean population level and individual cells vary widely in their dynamics. If this were generally true, how would a multicellular organism ever function? In fact, how would even a single cell function when all of its processes could end up being uncoordinated? Similarly, evidence shows that a yeast cell may contain only a few copies of many of its RNA transcripts. Any regulatory processes involving these transcriptsâunless mechanisms are in place for precise single molecular reactionsâare likely to be stochastic; mass kinetic models as used in standard chemistry cannot apply to these molecules. Could the dynamic principles of control processes in organisms be completely different from standard systems such that they allow inher- ently robust dynamics from noisy components (Samoilov et al., 2006)? In fact, could noisy dynamics be an adaptive characteristic as suggested for individual cell behavior in bacterial chemotaxis (Korobkova et al., 2004)? Coupled chaos as a model for quasi-stable ecological communities has been suggested; could such control processes operate at all scales of biological systems? Many models of biological processes in cells, organisms, or even ecosystems are derived from static and coarsely quantitative measurements.
108 THE ROLE OF THEORY IN ADVANCING 21ST-CENTURY BIOLOGY Once real-time in vivo dynamic measurements of large-scale multiple com- ponents are achieved, biologists will likely develop a different view of bio- logical control processes and the resulting robust functions. Such data will likely demand very different theoretical models of biological function. CONCLUSION The question âWhat are the engineering principles of life?â begs the development of a conceptual framework for understanding how biological systems take on particular forms and robustly carry out their functions. Even a small part of an answer to this question, especially when derived in a computable form applicable to data, will have great impact in all subfields of the biological sciences. For example, in protein and metabolic engineer- ing, even an approximate understanding of how to manipulate modules to produce desired forms or biochemical processes would be highly desirable. The field of biomimetics attempts to use biological engineering principles to generate devices that have desirable biological properties such as robustÂness and reconfiguration. Restoration ecology attempts to manipulate certain community ecological functions to remedy human perturbations to ecosys- temsâat the largest scale, even up to possible remediation of the effects of global warming. All of these applications require computable predictions of emergent properties and understanding of how biological systems achieve robustness. At a grand level, understanding human cognitive function r Â equires an understanding of how modular processes from individual syn- aptic vesicles, to synaptic boutons, to neural networks, to neuroanatomical r Â egions all integrate across scales to enable speech, memory, and thought. Is modular architecture a necessary requirement for generating complex biological objects? Modular construction is a human engineering concept and need not be a characteristic of evolutionarily derived biological ob- jects or ecological assemblies. Does the process of evolution promote the appearance of modular units? If so, are certain architectural elaborations likely or inevitable? Understanding the emergence of modular architecture across all scales and its possible contribution to properties unique to living systems, such as variation and robustness, is a key conceptual challenge of the future. Twentieth century theoretical biology provided the framework for mathematical and probabilistic dynamics of the turnover of individual components (such as alleles) in a closed system (such as population). In a way the theoretical foundations are similar to that of classical mechan- ics in physics. Given a closed system, the theory makes predictions about the motions of indivisible units. Similar to the development from classical mechanics to solid-state physics, pattern formation, and engineering, in the 21st century, development of a new theoretical framework is necessary for
WHAT ARE THE ENGINEERING PRINCIPLES OF LIFE? 109 understanding how the ensemble of units produces new emergent modules and emergent properties and for understanding the architectural principles of how biological systems are assembled from such modules across scales from individual molecules to entire ecosystems. As the new theoretical framework develops, constant evaluation and reevaluation are necessary to evaluate whether any given theory can make âpredictions.â In physical systems, some processes are intrinsically un- predictable because they involve features that can only be described in probabilistic terms. Other systems are unpredictable even though they are completely deterministic, because of their chaotic dynamics and extreme sensitivity to initial conditions. For some physical systems, attempts to predict fail simply because the important controlling details of the compo- nents are not well understood. Most current biological modeling implicitly assumes that accurate predictions can be made once sufficient informa- tion about the biological system is available. However, it is possible that some biological processes will be intrinsically unpredictable because of principles analogous to chaos or quantum indeterminism. A fundamental goal of chemistry or engineering is to understand and predict the behavior of compositions of parts. A major area of biological theory will be de- veloping a similar understanding of constructive principles of biological organizations.