Observing Weather and Climate from the Ground Up: A Nationwide Network of Networks (2009)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

5 Architecture for a Network of Networks This report so far has described several aspects of the national-level mesoscale observation network including the vision, scope, needs and tech- nologies. This chapter brings these aspects together in an architecture that can support all the functioning elements. The architecture recognizes that the national-level mesoscale network will be a network of networks (NoN), where the constituent networks embrace specific types of measurements or different geographical regions, many of which already exist. MEASUREMENT NETWORKS A new network of surface, above surface, and subsurface systems for mesoscale observing capability to meet multiple national needs should be defined ab initio, because very few systems have ever been planned and developed on the national scale. We have “national networks” such as ASOS, NEXRAD (WSR-88D), and the upper air rawinsondes, but they do not operate as true networks, because they (and most others) do not inter- act intelligently (i.e., adaptively and collaboratively) with other networks or even within their own network. The current state of surface networks in the United States is an u ­ ncoordinated set of local or regional deployments. There is no systematic national-scale establishment of surface network systems that serves all mesoscale observing needs. However, regional networks can be linked and integrated to form national-scale networks. Such networks already exist in other disciplines. A classic example is the cell-phone network within the United States that is deployed on a national scale without regional differ- 130 

ARCHITECTURE FOR A NETWORK OF NETWORKS 131 ences in the system. The system partition may be according to business units rather than regions. Current surface observation systems in the United States are quite numerous, as discussed in Chapter 2. Many of these networks result from some form of public enterprise, but the private-sector operates numerous networks as well. A well-known high-quality surface meteorological mea- surement system is the Oklahoma Mesonet (see Box 4.1). This network can be used as an example to denote the system technology (not necessarily the measurement technology) of such networks. The crucial communications component is handled by the Oklahoma Law Enforcement Telecommunica- tion System (OLETS) communications infrastructure. Thus this system has attributes in terms of data bandwidth, space and time scales of observa- tions, and the extent of observations (namely the State of Oklahoma). The Oklahoma Climatological Survey (OCS) receives the observations, verifies the quality of the data, and provides the data to mesonet customers. It only takes 5 to 10 minutes from the time the measurements are acquired until they become available to the public. It should be noted that this observation system consists of a suite of sensors that corresponds to specific technology that can be upgraded to future technologies. Quality assurance and calibration are important aspects of this system and must be supported by extensive quality control software and a calibra- tion laboratory. Thus this system has underlying technical and architectural aspects (McPherson et al., 2007). With the Oklahoma Mesonet, the meteorological network and the com- munication infrastructure are public enterprises. An alternate paradigm was used to develop the Helsinki testbed, a mesoscale observational network in Southern Finland. This network was heterogeneous from inception, and the measurement infrastructure is built around anchor systems such as the Vaisala ultra-high frequency dual-polarization radar. This public-private partnership consists of the Finnish Meteorological Institute, the Vaisala meteorological measurements company, and the University of Helsinki. The testbed provides information on observing systems and strategies, mesoscale weather phenomena, and applications in a coastal high-latitude environment (60-61N, 24-26E). Interest in the Helsinki testbed focuses on meteorological observations and forecasting directed towards meso-gamma scale phenomena that typically last from a few minutes to several hours. These weather events are often too small to be detected by traditional net- works. In coastal Finland, such weather events include temperature inver- sions, sea breeze, fog and low stratus, snow bands, urban heat islands, and convective storms. These and related phenomena such as lightning are often hazardous and cause substantial damage. For instance, fog causes consider- able disruption of land, sea, and air traffic. The sea breeze and its phases 

132 OBSERVING WEATHER AND CLIMATE FROM THE GROUND UP of development play an important role in the dispersion of atmospheric constituents. The Helsinki capitol area offers a representative study region for urban air quality research and boundary-layer modeling, especially in stable noc- turnal conditions, which are dominant in the area. A key feature of the Helsinki testbed is the heterogeneous nature of the measurements and the emphasis on urban weather. Thus one of the main differences between most U.S. networks and the Helsinki testbed is the emphasis on multiple users and urban systems. CONCEPTUAL ARCHITECTURE OF A NATIONAL MESOSCALE OBSERVING SYSTEM The concept of surface and subsurface measurement networks has been developing ad hoc, and the prognosis suggests rapid growth, especially aided by communication technology, customer demand, and the thirst of meso- scale models for data. It is anticipated that a vertical planetary boundary layer (PBL) component that serves the mesoscale will be developed as part of this system. This enterprise can benefit extensively from standards and protocols just as the communication industry has benefited from them. As described in Chapter 4, the technology of measurements is advanc- ing at a steady pace; however, we note that this advancement is not nearly as rapid as that of the weather measurement networks themselves. The indi- vidual regional networks have evolved according to a variety of paradigms, driven by local needs such as measurements in complex terrain or urban environments, as well as by application drivers such as transportation, agriculture, or homeland security. Thus the most important structure to be developed in a national-scale mesoscale network is a fundamental architec- ture that can create a “network of networks.” This concept is somewhat similar to that proposed for the National Ecological Observatory Network (NEON) project (http://www.neoninc.org) where the continent is divided into 20 regions. As emphasized in NEON, standards and architecture will form the most important aspects of a mesoscale network that satisfies mul- tiple national needs. A candidate national system serving multiple national needs will be one that can be considered a network of networks in an architectural sense. If current trends in technologies are a guide, many new instrumentation net- works will be composed of intelligent sensors that can be tasked to take measurements in a collaborative manner. Therefore it is envisioned that these networked sensors will respond to some feedback based on input from users and the prevailing environment. As an example, the Collaborative Adaptive Sensing of the Atmosphere (CASA) system’s network of sensors collaborates to jointly measure precipitation, and it adapts to the prevailing 

ARCHITECTURE FOR A NETWORK OF NETWORKS 133 weather conditions (Zink et al., 2005). The mode of operation is changed as the intelligence in the system responds to user needs, the prevailing weather, and the fault status of a neighboring sensor. The architecture describes the fundamental elements as well as the organizational and interfacial structures of the mesoscale network. It also describes the internal interfaces among the system’s components, and the interface between the system and its environment, especially the user. The various attributes of the architecture such as the space-time domain of the environment, type of measurements to be made, the multiple types of users, and the observational needs were described in Chapters 3 and 4. For example, the fundamental space-time structure of the phenomena being observed determines the frequency, scale, and density of observations, whereas the type of measurements determines the type of sensors. Another important feature within the architecture is the link to data assimilation and numerical models that may produce atmospheric structure on scales consid- erably smaller than the observational density. The various user sectors are accommodated in the architecture by its ability to support multiple sectors such as energy security, health, transportation, and homeland security. In addition, the deployment challenges in coastal areas and complex terrain and the microscale needs in the urban environment are also important. Sen- sor tasking addresses the adaptive and collaborative nature of the sensors whereas architecture addresses the requirements of metadata, storage, and policy infusion. Figure 5.1 provides a conceptual sketch of the candidate architecture of a national-scale mesoscale observing system. The architecture supports many important attributes. First, it is a user- driven system with sensor tasking, instead of a passive data-push system. Second, metadata structure and resource management, based on policy, are central features of the architecture. In addition the network supports links to analysis algorithms, such as real-time products and data assimilation systems, and to storage, query, and decision-support systems. Above all, this is a closed loop architecture, where the users have a structure that can support responses based on the observations. The architecture should provide a seamless ability to perform a Rolling Requirements Review (see Chapter 4), respond to the results of the review, and enable gap analysis to identify missing components of the system. It should also have the capability to support policy-based operation and to link to end users for decision making. The system attributes also include the ability to reconfigure itself to different modes of operation such as routine base operations, targeted modes, and event-driven modes. As an example, support of climate observations could result in systematic observations at very regular spatial and temporal intervals. In contrast, measurements could be targeted to monitor catchment areas of reservoirs or monitoring 

134 Mesobs System Candidate Architecture UP OBSERVING WEATHER AND CLIMATE FROM THE GROUND Sensor tasking based based Network reconfiguration . reconfiguration. on the environment . environment. resource Sensor allocation Tasking policy Multiple Environmental and National Metadata data attributes are are Users Repository “posted” in metadata “posted”in metadata Repository, 4-d grid of Repository ,aa4-d grid of the region. Data storage Data Assimilation Assimil Mesoscale network Mesoscale network and interface to Query / Access of of networks that can operate networks that can operate numerical models. i interface collaboratively and adapt toto collaboratively and adapt Analysis and the environment and user the environment and user algorithms run on data. run on data. needs. needs. FIGURE 5.1  Candidate architecture for a national-scale mesoscale observing s ­ ystem. SOURCE: Figure supplied by V. Chandrasekar. 5-2.eps industrial releases. Event-driven modes of operation require the network to 4 bitmap images reconfigure itself, along with the associated models, to, for example, pre- parecombined with vector images, type, rules, arrows for a tropical storm landfall or to assess the aftermath of an industrial accident or wildfires. INTEGRATION OF DISPARATE NETWORKS, STANDARDS, AND PROTOCOLS One of the main challenges to envision a network that serves multiple national needs is the concept of integrating networks and developing the necessary standards and protocols so that the private sector can manu- facture cost-effective sensor and network solutions. As a reference, one could refer to existing system partitions such as those in the National Weather Service and the International Telecommunication Union (ITU) or those planned for NEON. The network architecture will also describe the lower-level attributes for the instruments, including communication and calibration protocols, siting standards, instrument metadata standards, and procedures for adding new instruments. Similarly, observations and customer needs will prescribe the requirements for data bandwidth and storage, whereas the architecture will prescribe the level of redundancy. The attributes of the storage system include a distributed archival system, while the attributes of the access system include capability for space, time, and 

ARCHITECTURE FOR A NETWORK OF NETWORKS 135 application partitioning of the data. The information extraction subsystem includes the ability to incorporate quality control, data mining, visualiza- tion, and product generation capability while providing interface to data assimilation systems and mesoscale prediction models. Since the national-level network is expected to be a network of net- works, the architecture should support interoperability (the ability of diverse networks to work together), metadata across constituent networks, and the ability to add new networks and remove existing networks seam- lessly, ensuring scalability. Recommendation: A national design team should develop a well- a ­ rticulated architecture that integrates existing and new mesoscale networks into a national “network of networks.” This architecture should facilitate a thriving environment for data pro- viders and users by promoting metadata, standards, and interoperability, and enabling access to mesoscale data, analysis tools, and models. The effort must also include a process that continually identifies critical gaps and the evolving requirements of end users. The candidate architecture system encourages intelligent (collaborative and adaptive) sensors as well as responses to feedback from end users. Recommendation: Emerging technologies for distributed-collaborative- adaptive sensing should be employed by observing networks, especially scanning remote sensors such as radars and lidars. Some high-impact weather phenomena (e.g., tornadoes) of limited size and near-surface location can escape detection or be only poorly resolved by the current low-density network of weather radars. Collaborative and adaptive sensing and related technologies can efficiently enhance the detec- tion and monitoring of adverse weather for hazard mitigation and other applications, particularly for convective scales and in complex terrain and coastal and urban environments. Current state-of-the-art communication, computing, and remote sensing technologies facilitate this new paradigm for operation of networked instruments. Short-range, low-power weather radars, scanning lidars and ­radiometers, and other sensors can provide earlier detection and increased sampling of hazardous weather phenomena if the sensors communicate with each other and adapt to the ever-changing situation. Consider, as an example, improved quantitative precipitation estimation in an urban setting. Several short-range radars with overlapping coverage can work in concert to follow a convective cell moving across an urban area that is causing potential for flash flooding. Limiting the range to 20-30 km will avoid a major short- 

136 OBSERVING WEATHER AND CLIMATE FROM THE GROUND UP coming of the current WSR-88D radar network, that is, intersecting the 0° C melting layer with beams and/or overshooting the low-level clouds, which lead to significant errors in rainfall estimation. Additionally, tempo- ral resolution will increase when the need to scan everywhere is avoided when a small solid angle of scan will suffice. Increased spatial resolution is afforded by the short distances involved, substantial overlap between radars, reduction of random errors, and redundancy against instrument or communication failure. The above features should lead to improved accuracy of the fast-responding urban watershed by better locating heavy rainfall areas vis-a-vis the urban drainage patterns and infrastructure. Note that a new system would benefit from long-range surveillance capabilities, which are inherent to the current system. Given the capability for two-way communication with other ­ sensors, radar and lidar information in the urban environment will influence the frequency, timing, and/or location of other observations and vice versa. The Global Context: Global Earth Observing System of Systems Another example of an architectural challenge to organizing obser- vational systems is the GEOSS or Global Earth Observation System of S ­ ystems. GEOSS is an international effort to integrate the observing systems of all nations into a comprehensive and sustainable system whose goal is to “provide the right information, in the right format, to the right people, at the right time, to make the right decisions.” However, as the name implies, GEOSS is a distributed system of sys- tems, building on current cooperative efforts among existing observing and processing systems worldwide, while accommodating new components. Planning for GEOSS started in 2003 and is being developed by the intergovernmental Group on Earth Observations (GEO), which now con- sists of over 70 nations. The National Oceanic and Atmospheric Adminis- tration (NOAA) Administrator represents the United States and serves as a GEO co-chair along with representatives from South Africa, China, and the European Commission. The U.S. Group on Earth Observations (USGEO), a subcommittee of the President’s National Science and Technology Coun- cil, coordinates U.S. government participation. USGEO is supported by 15 federal agencies and three White House offices. The U.S. contribution to GEOSS is the Integrated Earth Observing System (IEOS), for which a strategic plan has been developed (see footnote for website). The goal is   From the GEOSS website at http://www.noaa.gov/eos.html; see also http://­earthobservations. org.   See http://usgeo.gov. 

ARCHITECTURE FOR A NETWORK OF NETWORKS 137 for GEOSS and IEOS to facilitate the sharing and applied usage of global, regional and local data from satellites, ocean buoys, weather stations and other surface and airborne Earth observing instruments. There are many common themes in both programs. For example, a USGEO Working Group on Architecture and Data Management is devel- oping a “Service Oriented Architecture” (SOA), an underlying structure that will support communications based upon loosely coupled connections among independent programs to create a scalable, extensible, interoperable, reliable, and secure framework. These attributes are similar to those of the Mesobs architecture for a NoN outlined above, except that we propose a comprehensive closed loop architecture that enables user feedback to the observing system. The national mesoscale NoN should be a vital component of GEOSS over the United States and could be thought of as one of the (very large) systems under its umbrella. This synergism will increase the value and pos- sibly the cost-effectiveness of both efforts.   Link from http://usgeo.gov/documents5dc2.html?s=docs.