Distributed Arrays of Small Instruments for Solar-Terrestrial Research: Report of a Workshop (2006)

Presentations and discussions at the workshop highlighted the complexity of issues related to the design, development, and implementation of the DASI concept. The following sections illustrate the range of topics that will require careful attention as plans for DASI move forward.

Successful implementation of the DASI program will require development of cyber infrastructure to enable the requisite coordination and communication among widely distributed sensors and research facilities. Sensors sample at different rates, are distributed at various locations, and are often operated by different groups and organizations. Furthermore, an analysis and data access and distribution program linked to modeling and computer simulation activities is necessary to provide closure between measurement and theory. In some cases, significant data processing is required to obtain the physical parameters required for assimilation in models. Many workshop participants emphasized that the development of tools that facilitate location, retrieval, and analysis of geophysical data from different instruments, locations, archives, and catalogs worldwide must be a central tenet of the DASI infrastructure requirements. These cyber tools also need to allow for real-time comparisons between simulations, models, and observations. When fully implemented, DASI should enable researchers to assimilate and interpret data in order to specify near-instantaneous space weather conditions, and possibly predict future conditions based on the “nowcast.”

The emergence of new data sets that result under DASI will add to the volume of experimental data that has been increasing significantly in recent years. Already there are new magnetometer chains, new polar orbiting satellites that allow a simultaneous view of the southern and northern polar regions, new ionospheric radars (SuperDARN, AMISR, and EISCAT), new instruments for making mesospheric/thermospheric wind measurements (via meteor radars, Fabry-Perot interferometers), and new digisondes to gather total ionospheric electron content data. Thus, workshop participants agreed that opportunities now exist to begin to create hardware and software cyber tools capable of studying the coupled elements of geospace as a single system. A framework is needed that can utilize DASI observations and other available geospace data and be flexible enough for the creation and inclusion of new data sets. Several general principles regarding key elements of an information system framework emerged from workshop discussions:

• Data must be simple to find, simple to query, and simple to download.

• Development of analysis tools must begin with the consideration of the needs of users.

• Ideally, the entire distributed instrument array should appear to a user as a single instrument, that is, as a DASI virtual observatory.

The solar and astronomical communities have spearheaded the virtual observatory concept, and that prior experience provides an excellent starting point and template for DASI. Virtual observatories can be built in several ways. A top-down approach has been most common to date. In this paradigm, specialized computer code enables access to diverse underlying data sources, providing a single user interface to all end users. A fundamental problem is that this approach does not scale well. A bottom-up approach starts with the development of standards for extracting geophysical measurements. These standards are defined as succinctly as possible, describing what data are available. As an example, the National Virtual Observatory for optical astronomy created a system in which the vast astronomical archives and databases around the world, together with analysis tools and computational services, are linked together into an integrated facility. Exploitation of the virtual observatory is facilitated by the development of standard protocols that, for example, allow searches of the observatory databases and return catalog entries for a specified location and search radius on the sky, or pointers to sky images given similar selection criteria.¹

Many lessons regarding database management from groups such as CEDAR will also be useful to the DASI program. For example, a very simple description of a standard file format is not sufficient. The standard must include well-defined measured parameters and a method for easily adding new parameters. Well-defined standards for specifying position and time, as well as standards for converting between frames of reference, are essential. The standard must expose summary data in a consistent way as well. While such a standard may seem burdensome at first, it is beneficial to data providers. The distributed instrument array is a natural platform for this data-standard approach. Thus, a major DASI infrastructure goal is to provide an easily accessible geophysical data set from combined instrumentation and arrays by incorporating standard tools such as

• Standard toolkits to expose data, for example, a Web site cyber toolkit; and

• Standard toolkits for exporting the data to other formats, for example, Matlab and Interactive Data Language.

Once data are acquired from various arrays, the utilization of the data by the broader geoscience community is the next concern. Many types of data are stored by different instrument operators in a variety of formats. Web technologies now are able to recognize the various data formats and obtain data from user-selected stations and then supply that data to the user or provide plots in a preferred digital format. Using a single Web portal to geophysical data, the user should be able to identify the type of data, network, or station location and to obtain the combined set of data in the specified format. There are a variety of virtual observatory projects in development to implement Web portals to do this.

The virtual observatory Web data portal will allow transparent and seamless integration of distributed related data sets into a systems view. This will be true of magnetometer networks, image networks, and global radar networks, among others. Rather than being another data archive, the virtual observatory will be a Web-based cyber tool that permits researchers to easily access data from multiple distributed databases with the following attributes:

• Development of a virtual repository for data and models as well as a network to facilitate collaborations;

• Collection and organization of data within a unified database structure;

• Standardization of the format for input of newly collected data;

• Testing and validation of new data with mass balance and statistical approaches;

• Utilization of visualization for quantitative understanding; and

• Integration of the cyber-networked infrastructure architecture to facilitate user access.

Grid technology is providing a powerful new open, Internet-based infrastructure that combines the resources of multiple sites and that includes unprecedented computing power and storage, as well as specialized data analysis and visualization resources, all of which are connected via a dedicated high-speed national network.² Although grid technology is still evolving, it should be open to all researchers since the Internet is now available to almost all scientists. The seamless sharing of data is one possibility and is potentially one of the main infrastructure goals of the DASI program. The creation of visualization tools that can utilize globally distributed data sets will push the limits of current technologies and could spark the creation of new grid functions. In addition, enabling the convergence of data and models is another strong goal of grid technology that is synergistic with DASI program goals. Grid technology has the potential to supply the underlying infrastructure for connecting the thousands of data-collecting instruments, analysis tools, and modeling computers that are integral to the DASI concept.

The science objectives of DASI will determine whether real-time communication is a necessity for all instruments at all sites. Instruments and sensors on the Internet or grid can be easily accessed and controlled from remote locations, and workshop participants discussed how this capability could form the backbone of the virtual observatory paradigm. However, participants noted that developing the capacity for Internet access to remote sensors will also be critical to the success of DASI.

Acquisition of near-real-time data from remote polar stations is already possible and is being accomplished by the Automated Geophysical Observatory team from its Antarctic stations.³ An Iridium-based data acquisition system is used, providing 20-megabytes/day data throughput (7 GB/year, 98 percent on duty cycle). This system could provide an immediate communication solution for remotely fielded DASI instruments. To address remote communications needs in the future, initiatives to develop communication capabilities on microsatellites may be necessary. Participants envisioned microsatellites with UHF transmission compatible with ARGOS,⁴ but with enhancements such as significantly higher bandwidth, two-way delay-tolerant communication, and Internet-Protocol-like message packaging.

WiMax is an emerging technology with a potential for enabling high-speed connections between clusters of instruments.⁵ A single WiMax base station with a connection to the Internet will be able to provide very-low-cost, high-speed connectivity to instruments within a 50-km radius. Base stations can be connected together to extend the size of the cluster even further.

A possibility for improving data coverage over the sparsely connected oceans would be an extension of the World Meteorological Organization’s Voluntary Observing Ships Scheme to include instruments such as GPS receivers.⁶ Currently more than 4000 ships worldwide transmit meteorological data on a regular basis using their own satellite communications equipment.

Workshop discussions highlighted the point that science goals will drive DASI infrastructure requirements related to the physical placement and distribution of instruments. The fundamental science grid size (spacing of instruments needed to achieve science goals) can vary greatly, as the examples in Table 4.1 illustrate.

Speakers also noted that any desired science grid size and spacing must be considered within the context of practical operational limits. The science benefit must be balanced against the logistics and overall operations budget as well. It is also important that planning for DASI not be done in isolation. In an effort to understand the global dynamical response, DASI should be well coordinated with existing distributed instrument arrays. (Included here should be Antarctic arrays monitoring the southern polar cap and auroral zones, CARISMA (previously CANOPUS) and MACCS in Canada, and the PENGUIn and the BAS/LPM Japanese stations in the south. Other aspects pertaining to existing and future arrays that workshop participants identified for further consideration included the following:

• Status of existing arrays and whether the arrays will be operational in the future,

• Resources needed for coordinating existing arrays,

• Scale sizes for different instruments and science drivers, and

• Communication infrastructure needed to coordinate subarrays of the DASI system.

The placement of instruments at populous locations worldwide can be easy and straightforward. Colleges and universities are usually willing to host experiments and provide instrument oversight and Internet access. Discussions at the workshop envisioned hundreds of DASI-related instruments throughout the United States and Canada. However, only a limited number of attended sites and/or towns in remote locations—in the Arctic and Antarctic—would be possible. The need for additional instrument sites together with the practical limitations of supporting additional attended observing stations at high geomagnetic latitudes has led to the development of remote observatory programs by many countries, including the United States, Britain, and Japan. The learning curves have been steep in the development of autonomous systems capable of operating successfully in the extreme polar environments, but such facilities should be available to DASI in the near future.

The polar region is one of the most important regions of geospace, because the direct connection between the solar wind and the magnetosphere can be measured within this region. The asymmetry with respect to Earth’s rotation axis introduces asymmetry in the distribution of various parameters and the characteristics of the two polar ionospheres even during equinox conditions. For that reason workshop participants noted that it is necessary to investigate both polar regions in an inter-hemispheric, coupled context in order to achieve the global understanding of the geospace system required for accurate modeling of space weather.

The workshop highlighted a great need for distributed autonomous Antarctic instrument arrays that can operate in the extremely cold and isolated polar environment. A technological challenge for the DASI program will be to deploy a sufficient number of autonomous systems in both polar regions at an affordable cost. Autonomous systems are obviously more expensive than instrument clusters placed at developed sites, because they must have special instrumentation for storing and generating electricity throughout the year. In addition, instrumentation has to use as little power as possible. Finally, deployment costs are much higher in such remote site locations. Because of the cost and danger of travel to remote sites for data retrieval, data links via satellite communication must be included in the design of polar instrument arrays.

Systems for remote environments should be designed for unattended operation for a period of at least 2 years and possibly longer, have real-time remote data retrieval, provide at least 2 years’ worth of on-site data backup, and be suitable for the environmental conditions in which they are intended to operate.