Cover Image

Not for Sale



View/Hide Left Panel

Dynamic Networks



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 120
Dynamic Networks 120

OCR for page 120
Embedded Networked Sensing (Redux?) Deborah Estrin, University of California at Los Angeles FIGURE 1 DR. ESTRIN: When I looked at the audience and I didn’t recognize most of you, I added a question mark in my title, because maybe this isn’t something that you have heard discussed, or perhaps not by me. 121

OCR for page 120
FIGURE 2 Let me start off with some caveats. The first of this is that I don’t know if embedded network sensing or sensor networks are that interesting as networks. I do know that they are very interesting systems and data sources, and through this talk, recognize that I am an engineer so, most of what I focus on is the design of new and, hopefully, useful functionality and unlike the other examples here—power networks, communication networks, social networks, even, biological networks, neural networks— these networks are just now beginning to be prototyped. There are things that can retrospectively be called sensor networks that have been around for a long time, like our seismic grids. Sensor networks of the type I am talking about are relatively new. We have a science and technology center at the National Science Foundation Science and Technology Center. It started three years ago and the purpose was to develop this technology, and there are lots of active programs all over the country and the world of people developing this technology. As I said here, statements that begin sensor networks are, or more sensor networks, should always be called somewhat to question, because there aren’t very many sensor networks, which makes it difficult to talk about statistics on or of these things. One thing is that there clearly want to be statistics in these networks, as I hope will become clearer, in the sense that there are statistics in image processing, and there are statistics in a lot of visualization and data analyses and things that we do. This is a little bit of a mind shift from some of what I heard this morning, and I hope it will be of some use. If not, Rob knew all this when he invited me. So, 122

OCR for page 120
you can talk to him about it. FIGURE 3 FIGURE 3 Why embedded sensing? Remote sensing has been around for a long time, and generates lots of very interesting data that continues to be a real interesting source of information for scientists, and a source of interesting algorithmic challenges. In situ sensing is a complement to that, where, as everyone knows, a pixel in a remote sensing image represents an average over a very large area. FIGURE 4 123

OCR for page 120
There are many phenomena, particularly biological phenomena, that simply the average over a large area isn’t what you are interested in. What you are interested in is the particulars at the variations within that larger region. When that is the case you actually want to be able to embed sensing up close to the phenomenon, with the hope that it will reveal things that we were previously unable to observe. For us that means that embedded network sensing or sensor networks—I use that term, embedded network sensing—you already get a little bit of a clue that I don’t think it is so much about the network. FIGURE 5 It is important that it is networked, if that is actually a verb or an adjective or whatever. It is important that you have collections of these things that are networked. That is an important part of the system. What is really important about it is that this is a sensing system and a network system, more so than the details of the communications network. In any case, where this technology has been most relevant is where you actually have a lot of spatial variability and heterogeneity. If you don’t have variability, you don’t need to have embedded sensing because you can take a measurement at a single point, or take an average, and you learn a fair amount from that. If you don’t have heterogeneity, you can develop fairly good models that will allow you to estimate a value at a point where you are not actually measuring. So, embedded sensing is important where you can’t do a good job of that estimation until you understand the system better. In many contexts where we are building this apparatus, this instrumentation for people 124

OCR for page 120
now, it is for contexts in which they are trying to develop their science. It might not be that they end up deploying long lived permanent system places, rather, they are trying to develop a model for the physical phenomenon that they are studying. They need an instrument that gives them this level of spatial resolution and then, from that, they will develop appropriate models and won’t necessarily need to monitor continuously in all places. In many places early on these systems are of interest to scientists to develop better models and, as the technology matures, we expect it to be increasingly used for then the more engineering side of that problem. Initially, you have scientists who need to study what is going on with the run off that is increasing the nitrate levels in urban streams, that is leading to larger amounts of algal formation and things like that, understanding that whole dynamic, because it is actually a fairly complex problem, the same kind of story in the soils. Right now we are building instruments for scientists to be able to understand and model those processes. For the longer term the regulatory agencies will have models and know what levels of run off you are allowed to have from agriculture and from urban, and they will want to be able to put up systems that monitor for those threshold levels. When we talk about center networks, we are talking about both the design of those instruments initially to develop very detailed data sets, and in the longer term the ability to put out systems that last for long periods of time. Embedded network sensing is the embedding of large numbers of distributed devices, and what we mean by large has changed over time. These get deployed spatially, in a spatially dense manner, and in a temporally dense manner, meaning you are able to take measurements continuously, although there are many interesting systems where you might do this over a short period of time, go out and do a survey to develop a model, and you don’t necessarily need the system to be there and live for a very long period of time. Very important to these systems, as I will be describing is that the devices are networked, and you are not simply putting out a lot of wireless sensors and streaming all the data back to a single location. 125

OCR for page 120
FIGURE 6 The fact is that there is a tremendous amount of potential data, and there are many modalities that you end up deploying that causes these systems to be fairly interesting, even when we are not talking about tens of thousands, or necessarily even thousands of individual devices. Most of the examples I will be describing are from things we are doing now with scientists, although the expectation is that, over time, the engineering enterprise, and even health-related applications will end up dominating. FIGURE 7 126

OCR for page 120
The systems that we are building are ones that—our goal is to have them be programmable, autonomous—although I will talk about interactive systems as well—distributed observatories that, for now, address largely compelling science and environmental engineering issues. Where we are right now, sort of three, five, 10, depending on how many years into this, when you think it all began, is that we have really first generation technology, basic hardware and software, that lets you go out and deploy a demonstration sensor network. FIGURE 8 In particular, we have mature first generation examples of basic routing, of reliable transport, time synchronization, energy harvesting. Many of these nodes are operating based on batteries without any form of continuous recharge of that battery although, as I will be describing, in all the systems we deploy, we have some nodes that are larger and more capable, but energy harvesting largely refers to solar. We have basic forms of in-network processing, tasking and filtering. To some extent the systems are programmable in the sense that you can pose different queries to them, and have them trigger based on different thresholds and temporal patterns. We have basic tools for putting these systems together and doing development simulation testing. The things that aren’t in bold are things that are still not very mature, things like localization. What we mean by that is, when you are doing your data collection on a node, you actually want to time stamp each of those data points. You also want to know where it is collected in three 127

OCR for page 120
spaces. One of the problems that I remember in the early days, one of the things I would say is that, obviously, we are not going to be able to deploy these systems without some form of automated localization of the nodes, meaning a node has to know where it is in three space, so that it can appropriately stamp its data. Fortunately, I was wrong in the sense of identifying that as something that would be a non-starter if we didn’t achieve it, because doing automated localization is as hard as all the others. You have to solve all the sensor network problems in order to do automated localization. Although I was wrong about that, sometimes two wrongs make a right, because I was also wrong about the numbers and the rate at which we would very rapidly be deploying huge numbers of these things. Two wrongs make a right in the sense that the numbers of these things are still in the hundreds when you are doing a deployment. Going out with the GPS device, and identifying where this thing is, and configuring its location, the localization problem isn’t what is stopping us from going to bigger numbers. That is not the biggest thing in our way, but it is a very interesting problem to work on. That is why I don’t have that in bold because localization, while one of the most interesting problems, still isn’t one that has an off the shelf solution, although we are getting nice results from MIT and other places. It is getting closer to that. FIGURE 9 128

OCR for page 120
This is where we are, and a little slide on how we got there—Lots of work over time that was pretty small and scattered at the beginning, and then started to build up steam. One of the contexts in which it did that actually was with an NRC report. It seems appropriate to mention here, in about 2000, our Embedded Everywhere report. Then, various DARPA programs, while DARPA was still functional in this arena, and now lots of NSF programs in the area, and interesting conferences to go to and venues for people to publish, possibly too many venues for people to publish. I said before that our price to earnings ratio in terms of numbers of papers to numbers of deployment is a little frightening at times, so that is where we are. It is a field that seems to have captured people’s imaginations. People are doing prototyping of a small amount of industry activity, actually growing in the area. When I look back at what our early themes were, and what the themes were in terms of the problems to solve, this summarizes what I think are the primary changes. First of all, early themes, we talked a lot about the thousands—actually, I think I even had slides early on that said tens of thousands—of small devices, and the focus was absolutely on minimizing what every individual node, what each individual node had to do, and exploiting those large numbers, and making sure that the system was completely self configuring because, obviously, at that scale, you have to make the system self configuring. Hopefully, all of those will come back as being key problems to solve. They just aren’t necessarily the first problems to solve and, since I am in the business of actually building and deploying these things, it doesn’t help me to solve future problems before the current ones. FIGURE 10 129

OCR for page 120
Our focus has been, and has turned to, systems that are much more heterogeneous, and I will talk about a few types of heterogeneity, and particularly the capability of the nodes, having mobility in the system, and then the types of sensing that we do. Also, our systems that we are finding very interesting to design and use early on are not fully autonomous in many of the cases. They are interactive systems where you are providing a surround sound three dimension or N dimensional, when you think of all the different sensor modalities, view to the scientists. It is the ability of the scientists to actually be able to be in the field and combine their human observations, but also their ability to collect physical samples, or go around with more detailed analytical instruments, that is providing a very rich problem domain for us. I mention it here in particular because there are lots of interesting statistical tools that scientists want to be carrying around with them in the field. That is probably when I come back to why I put that in the title. It is because there is this shift from very large numbers of the smallest devices, in our experience, to a more heterogeneous collection of devices, and a shift from a focus on the fully autonomous to interactive systems. FIGURE 11 Let me say a little bit more about each of those things. First of all, what is important about this heterogeneity, and what are we doing to design for it, and what, if anything, would it have to do with, if you are somebody in statistics interested in this technology. All of our deployed systems and all the systems I know of contain several classes of nodes. At the smallest level you have little micro controller based devices that fit micro controllers that operate off of 130

OCR for page 120
often used steady state methods to determine Ki from which apparent Kds can be calculated. Such a simplified approach can also be used to study propagation of signal from the receptor into the cellular network. FIGURE 19 We looked at connectivity propagation going forward from the receptor. Each step signifies the formation of a link (edge) and represents a direct interaction. Such direct interactions may represent a single chemical reaction for noncovalent reversible binding interactions or 2-4 chemical reactions when the interaction is enzymatic. The numbers of links engaged as signal propagates from a many ligands that regulate the hippocampal neuron is shown in Figure 19. This is a complex plot with many ligands that affect the hippocampal neurons. At one end is the major neurotransmitter, glutamate. Signals from glutamate rapidly branch out and by 8 steps engage most of the network. At the other end is the fas ligand that causes apoptosis in neurons. Fas takes nearly 12 steps to engage most of the network. By the time we reach 10 or more steps, we can get about 1,000 links engaged, indicating that most of the network becomes interconnected. We then counted the number of links it takes going from ligand binding to a receptor to get to a component that produces a functional effects such as a channel or a transcription factor that turns on a gene. Eight is the average number for going from receptor to effector. When we are tracking paths from ligand-receptor interactions to channels or transcription factors we can identify the regulatory motifs that emerge as connectivity propagates through the network. This is shown for three important ligands for the hippocampal neuron: glutamate, norepinephrine and BDNF in Figure 20. 158

OCR for page 120
FIGURE 20 We counted feedback loops, sizes three and four, we then counted size three and four feedforward motifs. One thing we see is that, there are a lot more feed forward motifs than feedback loops in the cells. I think this represents the molecular basis for redundancy. Feedforward loops can arise from the presence of isoforms. The higher we go up in the evolutionaryse isoformshierarchy, there are more isoforms for many signaling proteins. The same protein comes in three, four or five different forms. They often have some what different connections, so we think of these proteins not quite as full siblings, but more like half brothers and sisters with some common connections and some unique connections. What we found most interesting, was non-homogeneous organization of the positive and negative motifs. As we start at the receptor, at the outside of the cell, the first few steps yield many more negative feedback and feed-forward loops, which would tend to limit the progression of information transfer. As you go deeper into the system, in each of these cases—with glutamate, norepinephrine, we pick up the positive feedback loops and positive feedforward motifs. It appears that, if the signal penetrates deep into the cell, it has much more chance of the signal being consolidated within the cell. This consolidation may trigger many of the memory processes by changing the cell state. This was our first big breakthrough that we got in understanding the configuration of the network and may satisfy my biology friends. 159

OCR for page 120
FIGURE 21 Two brief detours into the standard differential equation based modeling to illustrate the capabilities of the feedback and feed-forward regulatory motifs. Positive feedback loops work as switches. We have shown this a while ago for the MAP-kinase 1,2, system both initially from modeling analysis and subsequently by experiments in a model cell-culture system NIH-3T3 fibroblasts. A comparison of a model and experiments showing the input output relationship due to the presence of a feedback loop is shown in Figure 21. Positive feed forward motifs also give you extended output, and this is shown below in Figure 22. Although what is shown is a toy model, we can see that over a range of rates the presence of a feed-forward motif affects input-output relationships. So, the presence of these regulatory motifs can have real functional consequences. FIGURE 22 The next analysis was to simulate the formation of motifs as signals propagated from the receptor to an effector protein. Generally, in our initial analyses we started at the receptor and 160

OCR for page 120
allowed connectivity to propagate all over the cell. That does happen to some extent, but in most situations information flow is constrained by the input and output nodes. When we stimulate the hippocampal neurons, it results in changes in the activity of the channels or changes the activity of the transcription factors like CREB that results in altered gene expression. So, we decided to look at the system using a breadth-first of algorithm to go from receptor to effector with progressively increasing number of steps. This is shown in Figure 23. FIGURE 23 This analysis actually yielded the most satisfying part of our observations. It had been known for a long time that glutamate by itself would not allow this neuron to change state and get potentiated for an extended period without engaging the cAMP pathway. When we think about the cyclic AMP pathway, we think of the neurotransmitter norepinephrine, that in the hippocampus facilitates the glutamate dependent potentiation. When neurons become potentiated, they behave differently in response to stimuli. The pesudodynamic analyses showed that for norepinephrine, when the number of feed forward and feedback loops were counted for whether they were positive or negative, far more positive loops were engaged with increasing numbers of steps. The preponderance of positive feedback and feedforward loops from norepinephrine to CREB provides an explanation of why this route is so critical for the formation of memory processes. CREB is often called the memory molecule since its activity is crucial for the formation of memory in animal experiments. In contrast, for glutamate by itself, the numbers of positive and negative motifs are equal, and BDNF actually turned out to be a bonus. For neuronal communication there are always two cells, the presynaptic neuron and the postsynaptic neuron. Neurons can be potentiated by the actions that go on within themselves, and they also can be potentiated by changes in the 161

OCR for page 120
presynaptic neuron. It has been shown that that BDNF actually works in the pre-synaptic CA3 neuron, and in the postsynaptic CA1 neuron that we have modeled for the network analyses and the regulatory motifs for BDNF induced network evenly balance out. This statistical analysis gives us insight into how the configuration of motifs can affect state change in cells. If we engage more positive feedback and feed forward loops than negative loops we can induce state change (i.e plasticity). If the positive loops and negative loops are balanced then although signals propagate and acute effects are observed there is no state change. This is summarized in Figure 24. FIGURE 24 I want to make two further points. First, when we conducted this pesudodynamic analysis, we actually sampled the system for dynamic modularity. Modularity means different things for scientists in different fields. For those of us who come from a cell biology background, the word module actually means either a functional module, like components of one linear pathway, or it means a group of components in an organelle such as the proteins in the cell membranes, or the nucleus. In our analyses we used a functional approach going from receptor to effector protein. This analysis is shown in Figure 25. 162

OCR for page 120
FIGURE 25 As we follow the number of links that we engaged when we go from, the NMDA receptor an initial glutamate target, to the AMPA receptor, which is also a glutamate target and the final effector in the system, or from the NMDA receptor to the transcription factor we found that these lines were relatively linear. To validate this profile Avi came up with a clever trick in creating shuffled networks that could be used as controls for this biological networks. What he did was maintain the biological specificity of the first uppermost connection, which is from the ligand to the receptor, and maintained the biological specificity of the last connection that is the links to come into the AMPA channels or CREB. So, there may be 10 components that feed into CREB with seven or eight protein kinases that regulate it and those links did not change. He then randomized everything in the middle. When he tracked paths in these shuffled networks we either got paths that yielded plots that fit either to a power law or an exponential function. Two features of these paths are noteworthy. One is that the path is linear and, two, there are many more links engaged in the CA1 neuron network than in the shuffled networks even though only 10-20 percent of the links are engaged. I point that out to you because if you look at the scale, the number of links here is either 100 or 200 and if you go 10 steps without output constraint, without outward constraint, you engage about 1,000 links. So the input-output constraint and the number of steps (and if we use the number of steps as a surrogate for time) allows us to constrain the number of interactions that can occur starting from the point of signal entry to the final effector target. This boundary defines the module, and everything else on the outside becomes “separate” because one cannot engage these links within the time period defined by the number of steps. This type of analyses indicates that we are likely to have a series of dynamic functional modules. The properties of these functional modules are summarized in Figure 26. 163

OCR for page 120
FIGURE 26 Second, we were interested in figuring out what these highly connected nodes do as part of the network. Avi started out with a system of mostly unconnected nodes and then asked the question, what happens to the system if we add nodes with four, five, six links, iteratively. He then determined both the number of islands as a measure of networking and the motifs that are formed as the network coalesces. This approach is described below in Figure 27. FIGURE 27 Initially, at four links per node he had around sixty islands, and by the time he reaches 21 links per node, he was able to form one large island (i.e., a fully connected network). This is shown in Figure 28. 164

OCR for page 120
FIGURE 28 The surprise from this analysis was, none of the major players, the highly connected nodes, and ones that we know are biologically important were not needed to form the network. So what might the role of these biologically important highly connected nodes be? To answer this question we decided to determine what types of motifs are formed as these highly connected links come into play. The results from this analysis are shown in Figure 29. FIGURE 29 What we found is that the highly connected nodes disproportionately contribute to the formation of regulatory motifs. Eighty percent of the feedback loops and feed forward motifs occurred as these highly connected nodes come into play. For this network, it appears that the highly connected nodes are not needed for the structural integrity of the network, rather the highly connected nodes are required for the formation of the regulatory motifs that process information. Thus the psuedodynamic analysis has allowed us to move from thinking about individual 165

OCR for page 120
components to groups of components within these coupled chemical reaction networks. The location of these regulatory motifs within the network allows us to define areas within the networks that are capable of information processing. Maps specifying the density of motifs at specific locations and their relative positions with respect to receptors and the effector proteins are shown. A heat map representing the density of motifs as a function of steps from the receptor is shown in Figure 30. A detailed distribution of the various types of motifs in the interaction space between receptor and effector proteins is shown in Figure 31. FIGURE 30 FIGURE 31 Detailed analyses for the location of the various motifs indicate that the motifs are densely clustered around the functional center of the network as shown in Figure 31. The map in Figure 31 would suggest that there are no regulatory motifs between channels. Channels pose an interesting representation problem for interactions between each other. Often channels use membrane voltage and membrane resistance to interact with each other. But voltage and resistance are not represented as entities within this network and hence these motifs are not 166

OCR for page 120
“seen” in our network. So, there are some corrections we need to make for biological networks that use electrical and physical forces as entities. In spite of these limitations we can make a numbers of conclusions from the type of analyses we have conducted. These are summarized in Figure 32. FIGURE 32 The major features of the cellular network within hippocampal cells are 1) highly connected nodes can consolidate information by participating in regulatory loops 2) Early regulation is designed to limit signals and presumably filter spurious signals. As signals penetrate deep into the network the positive loops that are formed favor signal consolidation that leads to state change. Thus description of the statistics of networks for somebody like me who works at a cellular level, is very useful in providing an initial picture of the regulatory capabilities of the cellular network. It allows me to design experiments to test which of these regulatory motifs are operative and develop an overall picture that I would never get from a bottom up approach, if I was just studying one feedback loop or two feedback loops at a time. That is my experience with the statistics of networks. Thank you very much. [Applause.] 167

OCR for page 120
QUESTIONS AND ANSWERS DR. JENSEN: I am interested in the motif-finding aspect. Usually, with these algorithms you find all frequently occurring patterns, and then at some point you go on threshold. You say, you know things about the threshold, those are unexpected things, but because of the nature of them it is often difficult to do good hypotheses tests and say where we should draw that threshold. So, what approach did you use for saying these are motifs that seem big and interesting? DR. IYENGAR: Actually, I did not have a real initial statistical threshold, because I was going from a biological point of view. The protein that participated was known to have important biology. REFERENCES Bhalla, U.S., and R. Iyengar. 1999. “Emergent properties of networks of biological signaling pathways.” Science 283:5400. Bhalla, U.S., P.T. Ram, and R. Iyengar. 2002. “MAP kinase phosphatase as a locus of flexibility in a mitogen-activated protein kinase signaling network.” Science 297:5583. Eungdamrong, N.J., and R. Iyengar. 2007. “Compartment specific feedback loop and regulated trafficking can result in sustained activation of ras at the golgi.” Journal of Biophysics 92:808- 815. Jordan J.D., E.M. Landau, and R. Iyengar. 2000. “Signaling Networks: Origins of Cellular Multitasking” Cell 103:193-200. Ma’ayan, R., R.D. Blitzer, and R. Iyengar. 2005. “Toward Predictive Models of Mammalian Cells.” Annual Review of Biophysics and Biomolecular Structure 34:319-349. 168