Review of NASA's Distributed Active Archive Centers (1999)

The Physical Oceanography DAAC (PO.DAAC) is hosted by the Jet Propulsion Laboratory (JPL). It manages data from a wide variety of ocean experiments and missions, including several done in collaboration with other countries, and it primarily serves the physical oceanography community. Although it has extensive experience with satellite data, the DAAC handles few data streams and will not be receiving data from the AM-1 platform. Consequently, the DAAC will have sufficient time to link its system with the EOSDIS Core System (ECS) (if and when it becomes available) and to scale up for future EOS missions.

The site visit showed that the DAAC is functioning well today, and it has the necessary strategic plans for operating successfully in the future. A primary

reason for its success is its location within the physical oceanography research group at JPL, a mutually beneficial arrangement that helps the DAAC understand how its data are used and the needs of researchers. This close working relationship, however, is jeopardized by recent trends to outsource DAAC functions, and the panel's main recommendation is that JPL should keep the DAAC intact and collocated with the oceanographers.

The Jet Propulsion Laboratory has been managing remote sensing data from the oceans since the Sea Satellite (SeaSat) Program in the early 1980s. These collective data activities formed the NASA Ocean Data System, which became the basis for the Physical Oceanography DAAC. The DAAC has existed since 1991 and is responsible for processing, archiving, and disseminating all of NASA's data related to physical oceanography (Box 7.1). The DAAC deals with data from many spacecraft, including several from foreign countries. Each instrument yields one discrete data set, except for the Ocean Topography Experiment (TOPEX/Poseidon) altimeters, which yield a data stream. PO.DAAC data vol-

umes are much lower than those of the other DAACs, so the data management problem is relatively tractable. As with the ASF DAAC, the PO.DAAC has been dealing with spacecraft data for quite some time; none of the other DAACs has had as extensive experience with an active satellite program.

The DAAC manages about 15 TB of data from a variety of ocean remote sensing missions (see Box 7.2). Future missions, which will add 4 to 5 TB of data to the DAAC each year, include SeaWinds 0/QuickSCAT, SeaWinds, Advanced Microwave Scanning Radiometer (AMSR), and Jason-1. These missions, which are scheduled to be launched in 1999 or 2000, are being flown in collaboration with other countries—SeaWinds and AMSR with Japan, and Jason-1 with France. None of these missions are related to the EOS AM-1 platform, and the DAAC's greatest challenge will be to ensure that the data become fully accessible through EOSDIS.

The Panel to Review the PO.DAAC held its site visit on January 8-9, 1998. The following report is based on the results of the site visit and e-mail discussions between the panel and the DAAC manager in June and July 1998.

The scientific value of the data sets for which the PO.DAAC has primary responsibility (see Box 7.2) is inestimable. Major discoveries are being made each year with the TOPEX/Poseidon data, which provide the first nearly global long-term sea surface height information. The wind data from NSCAT (see Figure 7.1) are better than wind data from any other data set or model analysis, and the QuickSCAT results are anticipated to be similarly spectacular. Finally, the Special Sensor Microwave Imager (SSM/I) Pathfinder information enables components of the fluxes between the ocean and atmosphere to be calculated, and the sea surface temperature information is important to a wide variety of ocean and climate studies.

Hierarchical Data Format (HDF) is the primary format used by the PO.DAAC, but the DAAC also maintains other formats, such as ASCII, for personal computer and Macintosh users who are unable to access the UNIX libraries needed for HDF. Some heritage data sets, such as SeaSat, are not being put in HDF for data distribution and exchange.

HDF is a subset of the standard EOSDIS format, HDF-EOS. Currently, none of the DAAC's data sets are in HDF-EOS format, and the DAAC will have to transition to the use of HDF-EOS over time if its data sets are to be used successfully in conjunction with other EOSDIS data sets.

DAAC staff were concerned about future support for HDF-EOS. A number of potential problems loom. First, the quality of the version of HDF used by the DAACs is not as good as the versions produced by the National Center for

Supercomputer Applications (NCSA). Second, the ECS contractor is evidently slow to fix bugs and provide support. Third, this version is not compatible with HDF 5.0, the version currently being implemented at NCSA. As a result, the DAAC will have to depend on the ECS contractor for the long-term development and maintenance of HDF-EOS. In the panel's view, NASA should consider contracting with NCSA to assume long-term maintenance and development of HDF-EOS. The long-term goal would be for NCSA to merge the capabilities of HDF-EOS into a future version of HDF 5.0.

Examination of the PO.DAAC's Web site indicates that on-line data sets are well documented. The DAAC archives ancillary data along with the data sets, and provides read software and detailed data set guides and user manuals for each data set. In addition, a reference list, sometimes quite extensive, is supplied with each data set. Some of the metadata provided by the DAAC (often from the flight projects) is more detailed than is required by EOSDIS guidelines. On the other

FIGURE 7.1. NSCAT-derived wind field for the period December 22 to December 24, 1996, showing both wind speed and pseudo-stream lines for the wind field. These data were taken by the NASA NSCAT instrument on the Japanese Advanced Earth Observing Satellite 1, and the image was produced by the PO.DAAC at JPL. SOURCE: PO.DAAC.

hand, documentation on the software is less complete, a problem acknowledged by the DAAC.

The PO.DAAC holds European Remote Sensing Satellite (ERS) data, whose use is restricted to ''approved'' NASA scientists under the terms of the MOU between NASA and the European Space Agency. The remaining holdings of the DAAC are available to users in any country.

The PO.DAAC does not follow the EOSDIS model of processing data products using Product Generation Executables (PGEs) provided by the instrument teams. Most standard data products are produced by the flight projects, then transferred to the DAAC for distribution and archive. In this sense, the PO.DAAC operates more like a data center than a DAAC (see Chapter 2, "DAAC Versus Data Center"). Data products produced by the DAAC, such as the TOPEX/ Poseidon Merged Geophysical Data Record, are not produced using instrument team PGEs. Rather, the DAAC develops the software and algorithms in-house, often with the oversight of members of the science team.

The DAAC plans to reprocess some data sets annually because it is more cost-effective to rework data sets regularly than to ignore them for a long time. Reprocessing NSCAT and TOPEX/Poseidon data sets is among the DAAC's top three priorities for FY 1998. The panel commends the DAAC for its wise approach to reprocessing.

The DAAC has developed a tool that allows HDF files to be spatially subsetted. Different tools have been developed for different data types (e.g., point, grid, swath), and the Web interface allows a user to specify the desired spatial subset. This makes it possible to reduce the amount of unwanted data returned to a user. PO.DAAC staff were quite proud of this subsetting tool, which is not scheduled to be provided by the ECS contractor until 1999. The panel applauds the DAAC's initiative to subset HDF.

Until NASA and NOAA develop a meaningful agreement on long-term archive of PO.DAAC data, the DAAC plans to maintain its data sets, provided that the budget and program continue on their present course.

The PO.DAAC's user community includes EOS science teams, ESDIS and other NASA program managers, data centers, physical oceanographers, K-12 educators, the private sector (e.g., fishermen), and the general public. The DAAC

places its highest priority on serving the needs of the science teams (the main repeat users of the DAAC), the flight projects (the main data providers), and ESDIS.

The DAAC keeps statistics on its users, such as the number of U.S. and non-U.S. users of a particular data set, but the statistics are not compiled in such a way as to enable full characterization of the user community. Such characterization would help the DAAC better understand and serve the diverse needs of its users. It would also facilitate development of a more accurate EOSDIS user model, thereby improving user satisfaction with the system as a whole. Developing and tracking metrics on who its users are and how they use the data should help the DAAC achieve both objectives.

The PO.DAAC has a close relationship with its User Working Group (UWG). The UWG meets twice a year, once at JPL to provide input on the DAAC's annual work plan and once off-site to discuss broader issues such as priorities on data product development and data acquisition. The UWG is pleased with the DAAC's responsiveness to its recommendations, and current and former members with whom the panel spoke are strong supporters of the DAAC. At the site visit, UWG members pointed out several DAAC initiatives that they believe have made the DAAC successful.

As noted above, the DAAC places a high priority on researchers who use and/or produce its primary data sets. Consequently, the DAAC's relationship with the scientific community is strong. Particularly impressive to the panel was the high level of interaction between the DAAC and the physical oceanography group at JPL, with which it is collocated. The DAAC scientist, Victor Zlotnicki, is a respected oceanographer, and this helps foster community trust in the DAAC.

Based on the large number of users (15,527), discussions with the User Working Group, and the CGED survey (Appendix D), it appears that the DAAC's relationship with oceanographers outside JPL is similarly strong. Indeed, it is not unusual for scientists to ask the DAAC to archive their data sets or develop software or tools. The panel notes that a close working relationship with researchers helps provide a scientific context for the DAAC's work, thereby helping it to fulfill its mission of facilitating research in ocean physics and air-sea interactions.

Scientists also provide an important and proven source of feedback and new ideas. In the panel's view, the DAAC's strong, ongoing interaction with scientists is a major factor in its success.

The DAAC believes that its real contributions are in the quality of the data produced and the scientific understanding that results from the use of these data. Consequently, user services are a high priority of the DAAC. It appeared to the panel that the DAAC is responsive to user requests for advice and suggestions, and a subsequent CGED survey (see Appendix D) confirmed this impression.

In addition, the DAAC has restructured its Web site and CD-ROMs to better serve its user communities. The Web site is easy to navigate and provides branches at the top that clearly direct various user groups appropriately (i.e., general public, educators, academic scientists). For example, the Web page feature on El Niño 1997–1998, which is kept up to date by the DAAC, is a stellar example of the use of Web technology to educate and inform the public at large. Similarly, the DAAC's educational CD-ROM products not only have provided a public service, but have also helped the DAAC expand its user base.

The PO.DAAC is located within a secure facility, but the necessary clearances can be obtained when users need to visit the center for several days or weeks. For example, the DAAC recently hosted a Chinese graduate student who needed to visit for a few weeks to produce a data set. In addition, the Web provides access to all DAAC holdings, thereby reducing the need for physical visits to the center.

The DAAC's system is currently based on a combination of SGI and Sun processors. The tape library is a 4-TB Metrum RSS-48 unit employing VHS tape technology. At the time of the site visit, the Metrum tape library was being replaced with a StorageTek 9710 unit based on digital linear tape (DLT) technology. Each cartridge has a capacity of 35 GB (uncompressed). The unit purchased is configured with six readers and has a total storage capacity of 20 TB (uncompressed), which the DAAC estimates will be sufficient for both its existing data sets and the data sets produced by future missions for which it is responsible. Unitree is used as the DAAC's hierarchical storage system. Although other DAACs (e.g., GSFC) have evidently not been happy with Unitree, the PO.DAAC has found that with the

proper amount of disk cache, it works quite well for the DAAC's data sets. Finally, the ECS contractor has purchased an EMASS unit for the DAAC, but the DAAC has no need for it or for the associated AMASS software, especially since the technology on which the EMASS unit is based is now obsolete.

The DAAC's investment in hardware is modest, and it seems to do a good job of bringing in new hardware in a timely fashion. Except for tertiary storage, the hardware is functionally partitioned by missions. That is, a dedicated computer (typically a small SGI multiprocessor) handles data product generation and processing requests for each data set maintained by the DAAC.

At the site visit, DAAC staff displayed an impressive awareness of the issues related to long-term archiving. Evidently, the task of restoring the SeaSat archive made clear how difficult a job maintaining a long-term archive is going to be. For a possible long-term archive media, the staff might explore the use of digital video disk (DVD) technology. As a "consumer" technology, it is likely that DVDs will exist for at least as long as CDs have (about 20 years). Although the archive media is only part of the problem (and, possibly, a small part), a stable archive media (i.e., a long period of commercial viability and a long shelf life so that the data remain readable over a long period of time) is a necessary requirement.

The DAAC software reflects careful and thoughtful engineering. Although some Level 0 and Level 1 data sets are maintained in their native format (or even in ASCII), the DAAC makes extensive use of HDF for storing data products (see Table 1.1 for a description of processing levels). The relational database system Ingres is used to keep track of data sets and data products. The archiving and product generation process is mostly a hands-off operation. Processing is controlled by a combination of PERL programs and shell scripts. The panel was impressed by the simplicity and effectiveness of the processing operation.

Because the Ingres product line is essentially in a maintenance mode, the DAAC will eventually have to switch to another database system. To minimize the impact of this eventual change when it occurs, DAAC staff have tried to avoid using Ingres-specific features to the maximum extent possible. When a switch occurs, the DAAC should consider the use of an object-relational database system with support for geospatial and temporal data (e.g., Informix Universal Server). This would significantly simplify the task of supporting requests for data sets covering a particular geographical region.

Although the PO.DAAC delivers some of its data sets and products via the Web, most large data sets are distributed using either CD-ROMs or tapes (8 mm or 4 mm.). DLT tapes are much denser (up to 35 GB uncompressed data), but their

relatively high cost is an impediment to widespread use. The DAAC is actively tracking DVD developments in the event that DVDs eventually become the standard distribution media.

In addition to DVDs, the DAAC is exploring the use of multicast and direct-broadcast satellite technology for distributing its data sets and products. Even though the use of direct-broadcast satellite technology to distribute large data sets is in its infancy, the Department of Defense (DOD) is making a major effort to use it to disseminate large geospatial, image, and video data sets. The DAAC should continue its exploration of the use of this technology since it may provide the most cost-effective distribution mechanism in the future. The panel would encourage the DAAC to consider a trial involving a commercial provider and a dozen key DAAC users.

At present, bandwidth appears to be sufficient for the acquisition and dissemination of the DAAC's data sets. Beginning with the QuickSCAT mission, however, both internal and external network capabilities will have to be upgraded to handle the larger volumes of data. The DAAC is using fiber-channel and gigabit technologies to upgrade its local area network for the QuickSCAT data. In addition, it is working with Goddard Space Flight Center to ensure that its external networks have sufficient capacity to support the flow of data from the down-link at the Wallops Flight Facility to the QuickSCAT Ground System. Additional bandwidth will be needed in the future to support the SeaWinds and AMSR missions, and the DAAC has already requested a network review to ensure that it has the needed capabilities before launch.

The PO.DAAC's philosophy is to meet user needs by developing a data management system incrementally and from the bottom up. The panel agrees with this responsive and flexible approach to data management. The DAAC manager, Donald Collins, listens to people (users and DAAC staff), gets along well with the DAAC scientist, and encourages his staff to take chances. Morale is good at the DAAC, although the recent trend toward outsourcing may be beginning to have a negative impact (see below).

On the other hand, the panel found the DAAC to be somewhat introspective, with a strong focus on the physical oceanography community and only a passing interest in the needs of users from other disciplines. Although the panel agrees that the primary focus of the DAAC should remain on physical oceanographers,

a broader multidisciplinary outlook could only enhance the position of the DAAC within the earth science community.

Staffing is one of the largest problems facing the PO.DAAC. The DAAC finds it difficult to retain staff or fill vacancies, in part because salaries offered to engineers and computer scientists by JPL are not competitive with salaries for similar positions in the greater Los Angeles area. In addition, ESDIS recently cut the DAAC budget and the DAAC chose to absorb the cut by leaving vacant positions unfilled. The number of staff was further reduced in March 1997, when JPL management decided to outsource some DAAC functions as part of an effort to reduce the total number of JPL employees (see "Relationship with JPL," below). Thus far, all programming and database staff, half of the user services staff, and most of the operations staff have been outsourced and moved off campus.

Another personnel issue has to do with the classic tension between operations and development. Although tensions were marked when the system was first being developed, the DAAC has partly solved the problem by separating the hardware for operations and development.

Finally, the DAAC has historically had a good relationship with its ECS liaisons and considers them DAAC staff. In fact, the two previous ECS science liaisons were hired by the DAAC.

The PO.DAAC's budget grew from $3.3 million in FY 1994 to $4.4 million in FY 1998 and is projected to reach $6.5 million by FY 1999 (Table 7.1). The growth in the budget reflects increases in operations and maintenance, and the acquisition of ECS hardware (e.g., FY 1997). Except for hardware acquisition years, the ECS-provided hardware, software, and personnel generally amount to less than 10% of the DAAC's budget. The overall development effort is anticipated to be less than 30% of the total budget for FY 1998 to FY 2002, the only years for which detailed DAAC-specific budgets were available.

The DAAC offered the panel several examples of its cost-effectiveness. First, the number of files, volume of data, users, and requests for data have all increased over the past three years, but costs have decreased, indicating that the DAAC has become more efficient. Second, building a system incrementally is cost-effective. Third, as the SeaSat data restoration shows, reworking data constantly is ultimately more cost-effective than ignoring them for a long time. Finally, the DAAC plans to adopt only part of the ECS, rather than the entire system (see "Strategic Plans," below), which may lead to large cost savings.

The ECS is one of the largest problems facing the DAAC, partly because of uncertainties in the modularity of the system and partly because the PO.DAAC is not on the near-term delivery schedule. The latter makes it difficult for the DAAC to make plans and to obtain accurate application program interfaces (APIs).

The DAAC has demonstrated that it can serve its users without the ECS. Linking its systems to the ECS will create work, but has several advantages. For example, the DAAC would like to use the ECS interoperability server to advertise its data sets. (The DAAC is doing a feasibility study to see whether it can adopt only this part of the system, and staff are confident they will succeed.) The DAAC also plans to use parts of the ECS for the Jason-1 data sets, particularly the ECS archive system and the means to distribute the data. The panel agrees that the DAAC's strategy of adopting only the parts of the ECS that it needs is sensible, given the context. However, by employing such a strategy, the DAAC risks losing coherence with the EOSDIS system.

Organizationally, the PO.DAAC is located within the Information Science Division at JPL, but it is physically embedded in the Physical Oceanography Research Element. The integration and feedback that have resulted have been mutually beneficial, and the panel felt that this has resulted in a center that serves its scientific community well. Unfortunately, this close working relationship may not last, in view of JPL management's decision to reduce staff by outsourcing some DAAC staff and moving them off campus. As a result of this decision, only a fraction of DAAC personnel would remain at the JPL site and be able to interact daily with researchers in oceanography. Although the use of contractors to perform various services is common at JPL (and, indeed, at other DAACs), the DAAC's perception is that this solution does not result in any savings and leads to morale problems and a less efficient operation. A subsequent e-mail discussion with Diane Evans, program scientist in JPL's Earth Science Program Office, suggests that morale has improved now that a contractor has been selected (Raytheon). Moreover, Charles Elachi, director of the Earth and Space Sciences Office of JPL, told the panel that he is committed to supporting data management activities and ensuring that the PO.DAAC meets the needs of the community. However, the panel's main concern—that outsourced DAAC staff will be physically separated from the JPL oceanographers—remains an issue.

According to Collins, the DAAC managers have been trying to strengthen their relationship with ESDIS. Although ESDIS knows it has communications problems with the DAACs, the DAACs believe that ESDIS' main communication problems are with the ECS contractor.

At the time of the review, the PO.DAAC was worried that the departed DAAC system manager, Gregory Hunolt, who had an operational focus, would be replaced with someone with a development focus, who would therefore not be philosophically attuned to the DAACs. The earlier departure of Dixon Butler, former operations director of the Data and Information Systems Division of Mission to Planet Earth, who displayed strong leadership and a vision for EOSDIS, was also a cause of concern to the DAAC.

The DAACs coordinate certain activities, such as priorities, procedures, and protocols, and they seem to communicate well at working levels (in particular the User Services Working Group and, to a lesser extent, the Operations Working Group and Systems Engineering Working Group). Greater cooperation with other DAACs, however, is not a high priority for the PO.DAAC, mainly because there are no instrument interdependencies. The panel, on the other hand, notes that communication and interoperability with other DAACs are necessary for helping users locate and combine disparate data sets, thereby fostering multidisciplinary research. This is a major objective of EOSDIS.

The DAAC has participated in ECS design reviews, but feels that the ECS contractor has not been responsive to its specialized needs. The DAAC's perception is that the ECS contractor is building to the specifications of the contract, which can be modified only by ESDIS. The ECS contractor has now stopped soliciting input for development and is focusing on delivery of the system to the AM-1 DAACs. Thus, the opportunity for input has passed.

The PO.DAAC is a well-run, well-functioning data center that has provided value-added services (e.g., production and distribution of data sets, development of tools and algorithms) to the oceanographic community since the early 1990s. Its success can be attributed to several factors, including (1) a vision of doing whatever is necessary to satisfy the needs of its users; (2) a sensible strategy for keeping its data sets active through regular reprocessing; (3) a flexible, incremental approach to system development; and (4) a commitment to long-term archiving. The first may well be the most important, and the DAAC could serve its users better if it developed and tracked statistics on what and how data are being used.

The DAAC has an excellent relationship with its highest-priority users, physical oceanographers. This relationship is strengthened by the influence of the User Working Group in the DAAC's activities and by the collocation of DAAC staff with oceanographers at JPL. This mutually beneficial relationship helps the DAAC provide appropriate data, software, and tools, which in turn help the scientists conduct their research. A recent trend by JPL management to outsource

DAAC functions, however, could result in a loss of synergy with the scientific community and could ultimately undermine the oceanographic research objectives of the Earth Science Enterprise.

Although the DAAC serves the physical oceanography community well, it does not expend much effort to serve researchers in other disciplines. Some of these researchers will seek to combine disparate data sets from a variety of sources, including the PO.DAAC. Indeed, facilitating the creation of such multidisciplinary data sets is a primary goal of EOSDIS and was a reason for creating the DAACs and the ECS (see Chapter 1). Changing technology has reduced the necessity for the ECS, and as long as the DAAC is able to fully link its systems with the other DAACs, the goals of EOSDIS can likely be achieved without the adoption of the full ECS.