Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 93
Space Studies Board Annual Report-1994 (Appendix A.1)
Space Studies Board
Annual Report—1994
Appendix:
Reports of the Panel to Review EOSDIS Plans
On November 22, 1991, NRC Chair Frank Press received a letter from
NASA Administrator Richard H. Truly requesting an assessment of the Earth
Observing System (EOS) Data and Information System (EOSDIS) in the context
of a recently completed restructuring of the flight elements of the program. In
response, the NRC Commission on Physical Sciences, Mathematics, and
Applications assembled a study group, the Panel to Review EOSDIS Plans,
chaired by Mr. Charles A. Zraket. The activities of this panel, which was a
collaboration of the Space Studies Board, the Computer Science and
Telecommunications Board, and the Board on Earth Sciences and Resources,
were managed by the Commission office.
Because the EOSDIS was the subject of a government procurement at
the time that the study was initiated, the panel first addressed those issues that
REPORT MENU
could be analyzed without the detailed system design information being
NOTICE
developed by the industrial proposal efforts under way. Preliminary conclusions
FROM THE CHAIR
on these issues were presented to NASA in an interim report delivered on April 9,
CHAPTER 1
1992, under letters from Dr. Press and Mr. Zraket. These letters and the interim
CHAPTER 2
report are reproduced in Appendix A.1.
CHAPTER 3
CHAPTER 4
CHAPTER 5
Based on agency responses to this initial assessment, panel Chair Zraket
APPENDIX A.1
prepared and submitted a second (letter) report on September 28, 1992; this
APPENDIX A.2
letter report is reprinted here as Appendix A.2. By mutual agreement with the
APPENDIX A.3
agency, the panel then suspended its activities pending completion of the
EOSDIS procurement.
After the selection of the vendor for EOSDIS, the panel resumed its work
to assess the design presented in the winning proposal. The results of this
second phase were documented in a third report, delivered to NASA on January
11, 1994. This third report is reproduced, with a cover letter from NRC Chair
Bruce M. Alberts, in Appendix A.3.
file:///C|/SSB_old_web/an94ap.htm (1 of 18) [6/18/2004 10:43:31 AM]
OCR for page 94
Space Studies Board Annual Report-1994 (Appendix A.1)
A.1 Interim Report of the Panel to Review EOSDIS
Plans
On April 9, 1992, the Panel to Review EOSDIS Plans completed the first
of three reports and submitted it to NASA Administrator Daniel S. Goldin. Two
cover letters accompanied the report. The first cover letter was from NRC Chair
Frank Press.
Enclosed is an interim report by the National Research Council on
NASA's plans for EOSDIS as well as a transmittal letter from the Chair of the
Panel that prepared this report. As you know, EOSDIS is a very complex
program, and the demands on the Panel that prepared this interim report were
extraordinary—in understanding the program, in coping with a demanding
schedule, and in reaching judgements. At the same time, my colleagues and I
appreciate the importance of EOSDIS. To quote from the attached report: "If
EOSDIS fails, so will EOS, and so may the U.S. Global Change Research
Program."
It was against such an understanding that the National Research Council
accepted this task, believing that we are obliged to assist the government, even
when the time is short, the amount of information to be marshalled great, and the
imperative to provide judgements urgent.
I believe the Panel that prepared this report has done an exceptional job,
ably assisted by the people of NASA. At the same time, the judgements as well
as the limits of this interim report should be clear. While the Panel supports the
schedule for procuring a contractor for the EOSDIS Core System, it finds major
shortcomings in the actual plans for EOSDIS, and provides substantial
recommendations for implementing the program that the Panel believes will help
ensure its success. Therefore, this report cannot be construed as an
endorsement of NASA's current plans for EOSDIS, but rather a substantial
critique of flaws, which, if addressed, will in the Panel's judgement help ensure a
strong and responsive program over the long term. The Panel believes that the
terms of the contract as stated in the Request for Proposal are sufficiently flexible
to accommodate its recommendations.
The limits of the report should also be plain. It is an interim report,
provided in response to requests from NASA and other interested parties for an
early alert as to the Panel's views of EOSDIS plans. The Panel's final report this
August will offer detailed analyses for these interim judgements, and will also
respond directly to the specific issues as posed in the Terms of Reference for this
task.
I look forward to your comments on this interim report. And the Panel
looks forward to a discussion with NASA officials involved in EOSDIS planning on
this report and any further issues to be considered in preparing the final report.
We are arranging for your colleagues at NASA with responsibility for the EOSDIS
file:///C|/SSB_old_web/an94ap.htm (2 of 18) [6/18/2004 10:43:31 AM]
OCR for page 95
Space Studies Board Annual Report-1994 (Appendix A.1)
Project to be briefed by the Panel next week, and intend to release it publicly on
April 17th.
Signed by
Frank Press
Chair, National Research Council
The second cover letter to Administrator Goldin for the April 9, 1992,
interim report was from panel Chair Charles A. Zraket.
I am pleased to submit the interim report of the National Research
Council's Panel to Review Earth Observing System Data and Information System
(EOSDIS) Plans. This contains the panel's preliminary observations and
recommendations on the current plans for EOSDIS, based on the information
provided. The panel looks forward to an early opportunity to discuss these
recommendations with NASA and other interested parties, as well as to issuing
its final report in August 1992.
On behalf of the panel, I wish to thank all of those at NASA who
responded quickly and professionally to our very substantial requests for
information and to our many and often difficult questions. We could not have
done our work without their full and ready cooperation.
I also wish to express our gratitude for the splendid cooperation from the
staff of the National Research Council that enabled the panel's work on this
interim report to be completed in less than two months.
Signed by
Charles A. Zraket
Chair, Panel to Review EOSDIS Plans
Panel to Review EOSDIS Plans
Interim Report
This interim report identifies several issues regarding NASA's plans for
developing the Earth Observing System Data and Information System (EOSDIS)
and offers a number of recommendations that NASA should consider as it
proceeds with procuring a contractor to build the system. This report does not
respond in detail to the items in the terms of reference—that will be the subject of
the panel's final report. Given the short time available for the panel's initial
assessment, it has not been able to pursue the issues it identified to the depth it
would like. The panel hopes, nevertheless, that NASA will find its interim
conclusions and recommendations useful in the negotiations that will take place
file:///C|/SSB_old_web/an94ap.htm (3 of 18) [6/18/2004 10:43:31 AM]
OCR for page 96
Space Studies Board Annual Report-1994 (Appendix A.1)
with the selected contractor to define the ongoing work plans for the EOSDIS
Project.
The appendices of this report include NASA's letter of request for this
study, the terms of reference for the task, a list of the members of the panel and
brief biographies, the work done and the meetings held to enable the panel to
write this interim report, a brief description of EOSDIS for readers not familiar with
the Project, and a brief description of the U.S. Global Change Research Program
and its objectives. [These items are not provided in this annual report.]
The panel was selected to have the competencies demanded by its
charge—in understanding the needs of those who will use EOSDIS (including
both EOS and non-EOS investigators), in the computer science and technology
underlying EOSDIS, in the creation and implementation of large data systems,
and in the recent history of large space-based data systems. The fact that the
procurement for the EOSDIS Core System was concurrent with the panel's work
required extreme care to avoid either the reality or perception of conflict of
interest. Thus, in addition to following the National Research Council's standard
procedures for dealing with bias and conflict of interest, the panel—and those
who provided it information and briefings—took pains to consider only publicly
available information. The panel, to the best of its knowledge, has not been
provided with nor has it considered any proprietary information related to the
procurement.
OBJECTIVES AND MAJOR FINDINGS
In combination with other programs of the U.S. Global Change Research
Program, the Earth Observing System (EOS) is intended to reduce the current
uncertainties about global climate change. Its Data and Information System
(EOSDIS) is essential to the success of EOS. If EOSDIS fails, so will the Earth
Observing System and so may the U.S. Global Change Research Program. The
panel has been told repeatedly by responsible government officials that EOS is
critical to the larger, global change program—one involving many agencies of
government, and other national and international participants—and that EOSDIS
offers a unique opportunity to begin building a national, and eventually,
international, information system for global change research.
To achieve these aspirations, EOSDIS will have to evolve to meet the
changing needs of global change research over the next two decades and
beyond. The panel believes that the recommendations offered in this report are
necessary to ensure that growth and evolution. Specifically, the panel offers its
judgments in terms of the following objectives it believes essential to the success
of EOSDIS:
EOSDIS must facilitate the integration of data related to the aims of
the U.S. Global Change Research Program. Without this integration, the
file:///C|/SSB_old_web/an94ap.htm (4 of 18) [6/18/2004 10:43:31 AM]
OCR for page 97
Space Studies Board Annual Report-1994 (Appendix A.1)
multidisciplinary and interdisciplinary research objectives of the U.S. Global
Change Research Program will not be achieved. The EOSDIS program must be
structured and managed to facilitate interactions with the other agencies involved
in the U.S. Global Change Research Program so that existing data and future
data collected by NASA and by other national and international
organizations—using research and operational satellites as well as in situ
sources—are available to all global change research scientists.
EOSDIS must serve a large and broad set of users to facilitate the
aims of the U.S. Global Change Research Program in supporting a community
concerned with understanding the earth as a system. To serve that larger
community, EOSDIS must provide its information in a manner that is simple,
transparent, and inexpensive; it also must assure availability of its data to both
the earth science community and the larger scientific community.
EOSDIS must ensure that service to current users—including those
involved with Version 0—will not be interrupted as the development of the system
proceeds, and that Version 1 and subsequent versions will be implemented as
soon as possible to meet the needs of the users, both in the EOS program and in
the larger U.S. Global Change Research Program.
EOSDIS, as it evolves, must maintain the flexibility to build rapidly on
relevant advances in computer science and technology, including those in
databases, scalable mass storage, software engineering, and networks. Doing so
means that EOSDIS should not only take advantage of new developments, but
also should become a force for change in the underlying science and technology
where its own needs will promote state-of-the-art developments. Flexibility also
requires organizational and management structures and processes that can
respond to evolving requirements and implement the means for meeting them.
EOSDIS needs substantive user participation in the design and
development of the system, including involvement in the decisions on data
acquisition and archiving, standard or ad hoc product generation, and interfaces
that directly affect science users.
The structure of the EOSDIS management organization and the
attention it gives to the project should reflect the importance of the program in
terms of its role as one of the major and most costly programs NASA has ever
undertaken as well as its central role in the U.S. Global Change Research
Program.
The EOS program was recently restructured from a mission consisting of
two large, orbiting platforms containing a total of 30 instruments to a series of six
smaller spacecraft containing a total of 20 instruments. The amount of data
expected to be collected from EOS, however, has decreased only slightly: from
330 gigabytes/day to 240 gigabytes/day. The estimate for the total amount of
processed data (from the EOS spacecraft and the other missions and
instruments that will be flown) that will be managed by EOSDIS changed from
file:///C|/SSB_old_web/an94ap.htm (5 of 18) [6/18/2004 10:43:31 AM]
OCR for page 98
Space Studies Board Annual Report-1994 (Appendix A.1)
1300 gigabytes/day to about 1100 gigabytes/day, a reduction of only 15 percent.
Furthermore, the capabilities of the EOSDIS System are tied to the existence of
the seven Distributed Active Archive Centers (DAACs) and the data they contain,
rather than to the flight rates. Although the panel will certainly examine this issue
further for the final report, it appears that the recent restructuring of the EOS flight
program has had little effect on the requirements for EOSDIS and thus does not
affect the preliminary conclusions of this interim report.
In general, the panel does not see any serious risk to the EOSDIS
program due to unavailable or inadequate technology. The panel believes that
the prototyping plans of the EOSDIS Project Office, to be implemented after the
contractor is selected, should be accelerated in order to assure that Version 1 is
completed in accord with design objectives.
There are risks, however, in two aspects of the planning for EOSDIS. One
area of risk derives from the scale and pace of changes in computer and data
management technology that can be expected over the long-term life of the
program, and from the great diversity of users who must interface with EOSDIS.
NASA needs to focus immediate attention on planning how EOSDIS will evolve to
continue to be a useful system as the scientific needs and the technology change
over time.
Another area of risk concerns the management structure of EOSDIS.
EOSDIS is an exceptionally large and complicated project that will cost several
billion dollars, involve thousands of people, and continue for many years. The
management will involve a complex mix of government, contractors, and a
scientific community that is diverse and spread around the world. Each has an
important role to play, and each will interact in a variety of ways with the other
elements. In its recommendations in this interim report the panel has attempted
to provide a number of mechanisms and approaches that it believes will help
define these roles and interactions.
NASA, of course, must have the ultimate responsibility for implementing
EOSDIS. To do so effectively, however, NASA should first ensure proper internal
management attention and also should use its own personnel in earth science
and computer science, who can contribute significantly to the successful design
of the system. Secondly, NASA needs to bring the scientific user community into
the project as a partner, rather than regarding users simply as customers. Finally,
NASA must accept the leadership role necessary to provide the essential unity
among the user community (including other federal agencies and international
participants), DAAC elements (management and scientific), and contractors. The
complexity of this project demands that a structure be developed to ensure that
all interests are properly integrated into the design of EOSDIS.
The panel believes that NASA can proceed prudently with the
procurement process for EOSDIS, provided the agency builds in the flexibility to
make the adjustments necessary to ensure the success of the project. The
conclusions and recommendations offered in this interim report can help NASA to
incorporate that flexibility into work plans during the contract negotiations that will
file:///C|/SSB_old_web/an94ap.htm (6 of 18) [6/18/2004 10:43:31 AM]
OCR for page 99
Space Studies Board Annual Report-1994 (Appendix A.1)
soon take place. This flexibility can be accommodated within the scope of the
current procurement as long as it is planned ahead of final contact negotiations
and the contract terms are compatible with this approach. The panel believes that
its recommendations should not materially affect the EOSDIS schedule and that
they can be implemented in work plans resulting from the pending contract
negotiations. It is important to all users that EOSDIS implementation proceed as
closely as possible to the planned schedule.
The panel has divided its assessment into three parts: user interactions,
EOSDIS architecture, and EOSDIS management. The recommendations for each
area offer actions that NASA should consider in order to meet the objectives of
the program described above without halting the current procurement. The panel
also recognizes that requirements may change over time and that NASA may
have to adjust its work plans over the life of the project.
In order to be of service to NASA during this important stage of
negotiating with the selected contractor, the panel believes that it is necessary to
provide this advice now, in this interim report. The final report will expand on the
issues discussed in this interim report and will respond in detail to the terms of
reference.
CONCLUSIONS AND RECOMMENDATIONS
The following are the panel's judgments concerning the user interaction,
architecture, and management issues that it believes must be addressed if
EOSDIS is to meet the objectives integral to its success. In each instance, the
panel points to strengths and weaknesses in the program, and offers
recommendations.
User Interactions
Strengths
NASA has stated its intention to incorporate user feedback throughout
EOSDIS development and evolution. The panel applauds this approach. The
ability of EOSDIS to serve the broad spectrum of users will be the final measure
of EOSDIS success. In this context, it should be acknowledged that NASA has
led other agencies in developing the Global Change Master Directory, which will
be a comprehensive description of all global change data sets. The panel also
commends NASA for its plan to share software code and toolkits with users who
wish to import them for their own systems.
Panel Concerns
file:///C|/SSB_old_web/an94ap.htm (7 of 18) [6/18/2004 10:43:31 AM]
OCR for page 100
Space Studies Board Annual Report-1994 (Appendix A.1)
In its review, the panel has identified several areas in which an
augmentation or strengthening of critical user interactions could substantially
improve the likelihood for success of the EOSDIS program. Areas of concern are
NASA's Science Data Plan, links with other agencies, use of Pathfinder data
sets, treatment of operational and historical data, long-term archiving,
involvement of nontraditional communities, and the ability to provide customized
data sets.
Science Data Plan. Version 0 science data requirements are being compiled into
a Science Data Plan by the EOSDIS Project through regular interactions with the
user community. The intent is to solicit regular review of these requirements from
the science community to make certain that evolving needs are adequately
reflected in the EOSDIS Project planning. Care must be taken to ensure that the
Science Data Plan continues to emphasize the links between global change
research objectives and the acquisition of individual data sets. A clearer picture of
base-level requirements can be achieved by a continuing assessment of science
objectives, existing holdings that might meet the objectives, and requirements for
future data streams.
The panel recommends that the Science Data Plan identify the
links between global change research objectives and existing and
planned data sets.
Interagency Links. The research priorities of the U.S. Global Change Research
Program cut across the missions of individual federal agencies. The distribution
of current holdings as well as data to be acquired underscores the need for
interagency interoperability and cooperation. NASA has been an active
participant in interagency efforts for the U.S. Global Change Research Program
through a variety of working groups, and is currently a full partner in developing a
tri-agency (NASA, NOAA, USGS) data and information implementation plan, of
which EOSDIS is a critical component. The panel endorses the efforts of these
agencies to work cooperatively.
The Global Change Master Directory is an excellent first step in helping
users to identify relevant data sets for global change research. A similar effort is
needed in achieving interoperability for access to the data. Success will require
both technical developments and leadership in order to integrate and provide
broad access to disparate data types currently distributed throughout the
agencies. The panel believes that NASA is the logical agency to initiate this step
in the context of EOSDIS. Moreover, EOSDIS will be much more effective in
broadening its user base if it serves as the vehicle for integrating data.
The panel recommends that NASA expand its efforts to increase
interagency links by assuming an active leadership role among
the agencies in achieving interoperability not only at the level of
the Global Change Master Directory, but also at the level of
providing access to the actual data.
file:///C|/SSB_old_web/an94ap.htm (8 of 18) [6/18/2004 10:43:31 AM]
OCR for page 101
Space Studies Board Annual Report-1994 (Appendix A.1)
Pathfinder Data Sets. Prototyping has been a routine component of EOSDIS
planning and Version 0 implementation by the Project Office. NASA has been
successful in establishing prototype earth science data systems that are currently
acquiring, processing, distributing, and archiving pre-EOS data. Lessons from
such prototyping activities can identify problems associated with the manipulation
and distribution of extremely large data sets.
Pathfinder data sets provide an early means to evaluate the handling of
large data sets, the development of products, and the distribution of data and
products. NASA and NOAA are cooperating in a Pathfinder data program for
selected satellite data. This program will be extremely valuable to the U.S. Global
Change Research Program and to the prototyping of various functions of the
overall data and information system.
The panel recommends that NASA develop ways to integrate the
efforts of existing data centers and centers of data supported by
NSF, DOE, and USGS with the NOAA/NASA Pathfinder activities.
Further, the Pathfinder data program now under way should be
accelerated.
Operational and Historical Data. Data from past and currently operating
satellites already are being provided to several DAACs. NASA has shown
considerable foresight in recognizing the importance of data streams from NASA,
NOAA, DOD, and foreign satellites in establishing long-term data sets for global
change research. Although the EOSDIS Request for Proposal addresses data
management of NASA's EOS platform instruments as well as NASA's
commitment to maintaining data sets acquired by pre-EOS sensors, the panel
wishes to emphasize the need for the accessibility of non-EOS instrument data
streams to EOSDIS users.
The panel believes that the full benefit of EOSDIS to the U.S. Global
Change Research Program will not be realized until an effort similar to that for
EOS data is undertaken to manage the immense collection of historical data
related to global change research already collected through operational
observing systems. This collection includes the routine data from the space-
based and surface-based observing systems of NOAA and DOD, as well as the
routine and special data collected by USGS, USDA, EPA, DOE, NSF, and the
Census Bureau. Integration, interpretation, and synthesis of such data, as part of
a modern data and information system for long-term operational measurement,
are critical to the goals of the U.S. Global Change Research Program and the
interpretation of EOS measurements.
The panel recommends several ways to address the issue of
integrating the operational and research data from other agencies
into EOSDIS:
a. NASA should articulate a plan for incorporating operational and non-
EOS instrument data streams into EOSDIS. Where EOS and non-EOS
file:///C|/SSB_old_web/an94ap.htm (9 of 18) [6/18/2004 10:43:31 AM]
OCR for page 102
Space Studies Board Annual Report-1994 (Appendix A.1)
instruments have similar functions, NASA should develop a strategy to enhance
the use of both data streams. This strategy should also include consideration of
cross-calibration between basic radiometric data and higher-level products of an
EOS instrument with a non-EOS instrument.
b. To test the interoperability of EOSDIS and to integrate the critical long-
term operational data that now exist at Affiliated Data Centers into a global
change data and information system, NASA should perform a full-function test of
the EOSDIS architecture and software on some of the Affiliated Data Centers, in
particular, centers with holdings (such as long-term satellite or in situ data
records) critical to the U.S. Global Change Research Program and to the
synthesis and interpretation of data from EOS instruments.
c. NASA should articulate its policy on how Affiliated Data Centers will
move up through the different levels of interoperability that are specified for
linkage with EOSDIS.
Long-Term Archiving. Long-term archiving of EOS data is an issue that has not
been addressed. Long-term commitment to maintaining data collected as part of
EOSDIS is a critical component of the U.S. Global Change Research Program.
NASA, in its response to questions from the panel, correctly pointed out that the
issue of maintaining long-term archives is one that must be addressed by all
participating federal agencies. Without a concrete plan and agency coordination
for establishing permanent data archives, however, the overall objectives of EOS,
and, therefore, of the U.S. Global Change Research Program, are jeopardized.
As in the case of increasing interagency links, the panel believes that NASA can
provide the leadership in addressing this need.
The panel recommends that NASA develop an adequate plan and
technology for long-term data archiving in conjunction with the
other federal agencies participating in the U.S. Global Change
Research Program.
Involvement of Nontraditional Communities. NASA has identified ways for
broadening the user community and providing information about EOSDIS to
those unfamiliar with the system through professional journals and newsletters.
Such publications may be adequate for reaching users in certain disciplines but
may be ineffective for those in other fields, particularly in the nonphysical
sciences. For example, one of the science priorities identified in the U.S. Global
Change Research Program is to assess the human dimensions of global change.
A detailed plan for involving potential user communities beyond the traditional
disciplines associated with the earth and environmental sciences has not been
clearly delineated for the panel.
Many approaches could be taken to encourage users from nontraditional
communities (e.g., legal, educational, political, and social). A useful approach
could include the distribution of sample products that would allow users to
become familiar with the various types of data sets available and to judge
file:///C|/SSB_old_web/an94ap.htm (10 of 18) [6/18/2004 10:43:31 AM]
OCR for page 103
Space Studies Board Annual Report-1994 (Appendix A.1)
whether those data would be helpful to their research.
The panel recommends that NASA take an active role in
facilitating access to EOSDIS by other, nontraditional disciplines
through a program that includes representatives from those
disciplines in NASA's user advisory groups and develops products
useful to them.
Customized Data Sets. NASA clearly recognizes the importance of involving the
user community in the development of EOSDIS. An approach to encourage
active user participation is to provide customized data integration and synthesis
of various products. The availability of software tools that conform to standards in
an open architecture environment would facilitate participation by active users.
For example, these tools might enable a user to assemble a customized set of
specific time- and/or space-averaged data that could not otherwise be assembled
without the user having to develop new software.
The panel recommends that NASA encourage broad user
participation by providing greater opportunities to create
customized data sets.
EOSDIS Architecture
Strengths
The panel in its several lengthy discussions with EOSDIS technical staff
was impressed by the staff's competence and motivation. The staff has devised a
process for designing the EOSDIS Core System that would rely on open
systems, including multiple levels of interoperability for both users and the
DAACs as well as the ability to handle evolving international standards. These
two approaches—use of an open system and adoption of standards even though
they will change over the lifetime of EOSDIS—will strengthen the program.
The Project plans to deliver EOSDIS in incremental stages (via Versions
1 to 6 and Data Product Levels 0 to 6) that are expected to provide the flexibility
necessary to meet user needs, to respond to budget uncertainties over the next
decade, and to adjust to EOS flight schedules.
Panel Concerns
Design Control. Any large software system requires design criteria that are set
by project management and articulated clearly and precisely throughout the
project hierarchy. This is particularly true for EOSDIS because of four reasons:
(1) the unprecedented size of the system's storage and processing capacity; (2)
the extraordinary heterogeneity of both user computation systems and user
file:///C|/SSB_old_web/an94ap.htm (11 of 18) [6/18/2004 10:43:31 AM]
OCR for page 104
Space Studies Board Annual Report-1994 (Appendix A.1)
requirements; (3) the large variation in scale of both the mass stores and the
granules of data to be simultaneously managed; and (4) the high degree of
evolution expected in the system. The combination of these factors will make the
design, implementation, and evolutionary control of the system a substantial
architectural challenge.
Although NASA has assured the panel that EOSDIS will serve the needs
of global change researchers, the EOSDIS Core System Statement of Work and
the Functional and Performance Requirements documents of the Request for
Proposal seem to be based on the management of data holdings resident with or
owned by NASA or the DAACs and the created data products related to those
holdings. It is entirely likely that data and/or data archives that are not within the
exclusive purview of NASA or the DAACs will need to be made accessible to
users through EOSDIS, without changing ownership of the data or the autonomy
of the data repository. In anticipation of the need for accessibility, EOSDIS
software should be built in the form of modular components with open,
configuration-controlled interfaces so that other national and international
agencies will be able to link with the system and provide products and services to
the broader global change research community.
The panel believes that responsibility for the design criteria and for their
enforcement to guide the system architecture must reside with the government.
The government must assure that the contractor's detailed architecture and
implementation decisions follow the directions given by the government system
architects.
The panel recommends that NASA produce a clear, concise
statement of the design criteria for EOSDIS that focuses on
facilitating global change research and that NASA communicate
these criteria throughout the Project hierarchy.
The panel recommends that NASA strengthen its internal system
architecture team by acquiring additional experienced people and that it give
them the responsibility, authority, and budget to ensure that the design criteria
are met as the system design and implementation proceed. A technical project of
the magnitude and complexity of EOSDIS should have the very best system
architecture team possible. NASA should make every effort to acquire such
talent.
Logically Distributed System. The research that will be possible through the
resources provided by EOSDIS is difficult to characterize at present. Some
research will focus on narrow disciplinary questions, while other work will be
interdisciplinary. Since we cannot, indeed should not, attempt to specify the
future directions that earth science research will take, EOSDIS must be flexible
enough to respond to a wide variety of approaches. Furthermore, EOSDIS will be
only a part, albeit a major one, of the efforts directed at managing data and
information for global change research.
file:///C|/SSB_old_web/an94ap.htm (12 of 18) [6/18/2004 10:43:31 AM]
OCR for page 105
Space Studies Board Annual Report-1994 (Appendix A.1)
The EOSDIS development plan provides for centralized control over the
specification and implementation of the system. Each DAAC will implement an
Information Management System that will be centrally developed by a single
contractor. Although a centralized system is desirable for the management,
operation, and control of the satellite and its instruments, the data will be
distributed and dispersed among geographically separate and discipline-specific
DAACs. Achieving the proper balance between the common elements that
should be developed centrally and those that should be developed in a
distributed fashion is critical to the success of the overall U.S. Global Change
Research Program. At present, it appears as though the EOSDIS development
plan is too heavily oriented toward a centralized approach.
The panel recommends that the EOSDIS Project adapt its
development plan to ensure a more logically distributed system,
including:
a. Designing EOSDIS so that all users (EOS and non-EOS investigators,
DAACs, other data centers) can easily build selectively on top of EOSDIS
components. EOSDIS should not constrain local implementation of diverse
functions by users and DAACs. The development plan should reflect a
philosophy that it is "easy to interact with EOSDIS" with minimum loss of
autonomy. EOSDIS must be able to tolerate different versions of functionality and
partial sharing of the components and toolkits it exports.
b. Identifying those areas of interdisciplinary research that will require
special interfaces among discipline-specific products and formats. The Project
should specify the interfaces, build prototypes, and run simulations to exercise
them, permitting users to evaluate them prior to developing final specifications
and proceeding to full implementation. A contractor team that resides at each
DAAC and works closely with the DAAC as well as the contractor's "central core"
team should facilitate the development of these prototypes.
This type of distributed development can be accomplished within the
scope of the current procurement as long as it is planned ahead of final contract
negotiation, and contract terms are compatible with this approach.
Incremental Prototyping. The current EOSDIS development plan closely ties
the availability of the distributed archive and product generation functions to the
EOS flight schedule. There is much work that should be done, however, prior to
the first scheduled launch of EOS instruments in 1998 to strengthen prototyping
efforts already under way. For example, there are both existing archives and data
expected from pre-EOS satellites that will be invaluable to the U.S. Global
Change Research Program. Although the EOSDIS Project team has initiated the
early prototyping effort for Version 0, more can and should be done to benefit
current global change research and to enhance user feedback for final system
design.
The panel recommends that EOSDIS Project management extend
file:///C|/SSB_old_web/an94ap.htm (13 of 18) [6/18/2004 10:43:31 AM]
OCR for page 106
Space Studies Board Annual Report-1994 (Appendix A.1)
its incremental development plan so that all user interfaces, all
toolkits, and the end-to-end network system are:
a. Specified in detail early in the development of Version 1 and prototyped
or simulated sufficiently, and
b. Evaluated in depth by users and DAACs prior to full implementation in
Version 1. This will require a system network simulation and sufficient testing
tools for users to assess and validate the specified functionality.
Usability Evaluation. Prudent practice in the design of complex data
management systems ordinarily includes a means of measuring the usability of
the data. To the extent possible, such measures should be quantitative. Early
evaluation exercises should be designed to measure ease of use, quality of
interface specifications, and convenience of interoperability of heterogeneous
system components. These exercises should ensure that individual users and
data archivers can acquire piecemeal both functional capabilities and data sets. It
is also prudent practice to involve independent judgment by having this
evaluation performed by a group other than those responsible for developing the
system.
The panel recommends a usability evaluation program starting as
soon as possible that involves:
Selecting key functions, interfaces, and system behavior attributes for
evaluation;
Defining a set of metrics and expected values of those metrics for
each parameter to be evaluated;
Creating prototypes, simulations, and test suites to stress aspects of
usability;
Using the evaluations to guide final specification of system
components; and
Implementing this program so that most of the evaluation and
validation is done by groups other than the prime contractor.
EOSDIS Management
Strengths
NASA is to be commended for developing the plans for EOS as its
file:///C|/SSB_old_web/an94ap.htm (14 of 18) [6/18/2004 10:43:31 AM]
OCR for page 107
Space Studies Board Annual Report-1994 (Appendix A.1)
flagship for U.S. participation in global climate change research. NASA and the
EOS Project are further to be commended for their dedication to producing an
adequate data system for EOS and for its user community. The unprecedented
level of funding allocated for EOSDIS and the high level of planned contingency
funding are evidence of the commitment NASA has made to this important
national research effort. The panel is impressed with the degree of dedication
and commitment of the EOSDIS Project team. The team is working diligently and
competently toward both prototyping key system and subsystem capabilities and
planning for the procurement of the full EOSDIS system.
Panel Concerns
Visibility and Management Attention. Although EOSDIS appears to receive
substantial attention from management at NASA Headquarters, in the panel's
view, EOSDIS lacks the attention of senior management at the Goddard Space
Flight Center. The EOS Project is the largest single development effort the
Goddard Center has undertaken. Even without the flight hardware components,
EOSDIS by itself probably satisfies that description. EOSDIS is an extremely
complex interdisciplinary science project and must integrate the most advanced
data and system technologies. EOSDIS also contains both the flight operations
segment and the ground data system. The fact that schedules overlap and that
the prime contractor probably will use different groups of personnel to implement
these two very different elements will amplify the government's oversight and
management challenge. Yet the panel has heard substantial evidence that from
the management standpoint, EOS and EOSDIS are treated like an ordinary
project within the Goddard Center. For example, the Project Manager for
EOSDIS is two management levels down within the Flight Operations Directorate,
which is only one of ten directorates at the Goddard Center. In addition, the
Project Office is quite small for the task at hand, with plans for only 45
government employees when fully staffed. This small core of dedicated staff
provides inadequate programmatic and managerial depth and expertise in the
development of large, distributed data systems and in computer science and
technology.
Given the preeminent position of EOS and EOSDIS in the U.S. Global
Change Research Program, the panel believes that it is essential to increase the
level of management visibility of the Project and the size and skills of the Project
staff. In addition to learning from other government agencies that have had
experience in the development and operation of large distributed data handling
systems, NASA could, as needed, add to the Project experienced systems
development personnel from other parts of the government.
The panel suggests that greater flexibility in defining success criteria and
in using the process for setting award fees for direct feedback from the Project
Manager to senior-level contractor management would help to assure that the
contractor will do an outstanding job on EOSDIS. The panel commends NASA for
including users in its performance board for contract evaluation and urges the
active participation of users in setting award fees.
file:///C|/SSB_old_web/an94ap.htm (15 of 18) [6/18/2004 10:43:31 AM]
OCR for page 108
Space Studies Board Annual Report-1994 (Appendix A.1)
The panel recommends that the EOSDIS Project Manager have
higher management visibility within Goddard Space Flight Center.
The staff authorizations and skills should be sized to the scope
and complexity of the Project. Further, the Project could augment
its staff with experienced personnel from other parts of the
government in addition to NASA.
The panel recommends that the EOSDIS Project use the award fee
process to best advantage through greater differentiation of success and failure
criteria for evaluating contractor performance and by involving users in
determining award fees.
Scientific Involvement at Goddard Space Flight Center. The Goddard
Center's in-house earth scientists have a very limited role in the management
and operations aspects of the EOSDIS Project. Although NASA has established
a variety of science advisory and data working groups, such groups cannot
replace the continuing and even daily involvement of the external scientific
community and the Goddard Center staff to ensure that the eventual system is
responsive to user needs.
Likewise, the nation's computer science community currently has very
limited involvement in the Project, despite the fact that EOSDIS, to be successful,
must implement the latest advances in scientific data management technology
and, in some cases, stimulate the development of new technologies. The
development of EOSDIS would benefit from substantive use of expertise in
systems design and exploitation of information processing technology. Because
underlying technologies, such as storage density, processor speeds, and
transmission rates, are doubling roughly every three years, EOSDIS must be able
to exploit rapidly expanding capabilities during its lifetime of a generation or more.
EOSDIS will also stretch the limits of what can be done by a mammoth
database management system shared by a very diverse and demanding user
community. Certainly, many of the underlying technologies such as storage will
evolve on their own. Other technologies, however, will have to be encouraged,
such as large-scale data management, visualization, and integration of
heterogeneous information. Possible ways to stimulate technology include
establishing an intramural computer science research capability comparable to
those in other sciences, supporting and using the external computer science
community, and using DAACs to establish formal and informal links with the
computer science research community in their neighboring universities.
The panel recommends that NASA involve Goddard Space Flight
Center earth scientists to a greater degree in the management
and operations of EOSDIS and also involve computer scientists
both inside and outside of NASA to explore research and
technology in those areas where EOSDIS will stress the state of
the art in science and technology and where EOSDIS will evolve
most rapidly.
file:///C|/SSB_old_web/an94ap.htm (16 of 18) [6/18/2004 10:43:31 AM]
OCR for page 109
Space Studies Board Annual Report-1994 (Appendix A.1)
DAAC Involvement. The DAACs are not well integrated into the EOSDIS
management structure, particularly during the development phase. The DAAC
managers do not have well-defined authority or accountability in building
EOSDIS. DAACs should be involved early, in contrast to the current plan, in
which their primary role appears to be to operate the hardware and software at
their sites after delivery, and to deliver data products to users.
There should be mechanisms for feedback on scientific utility and
operational effectiveness from the individual DAACs and associated archive
centers to the central Project since the DAACs will be the primary sites for user
interaction. There should be a coherent overall development, management, and
science advisory structure that includes the DAACs. The panel understands that
DAAC managers and scientists are involved in advisory roles. Advisory roles,
however, are not sufficient for developing capabilities for and at the DAACs.
Overall, the centralized management of the design and implementation of
EOSDIS functions at each DAAC is not conducive to active user involvement and
responsiveness to changing technology. What is needed is a structure that
strengthens the local role of each DAAC beyond the present DAAC advisory
group and thus enhances the responsiveness of each DAAC in meeting the
needs of its user community, gives the DAAC some control over its destiny, and
yet ensures that an interoperable system is developed to meet the requirements
of EOSDIS.
The panel recommends that NASA create, at each DAAC, a
Development Team of full-time staff and active science users to
address DAAC and user concerns. These teams should evaluate
EOSDIS planning and implementation, including architecture,
DAAC interface definitions, and other deliverables essential to
ensuring that the DAACs will be responsive to user needs and that
the EOSDIS system will be interoperable. In accomplishing these
tasks, the teams should monitor the contractor's activities on
behalf of user communities and prepare test data sets to verify
system interfaces. Each DAAC Development Team should
validate that DAAC's operational capability to use the evolving
EOSDIS system as each of the program releases is implemented.
Finally, NASA should provide the DAACs with modest funding to
respond to specific user needs so that the DAACs will be able to
parallel the evolution of the user community's ability to manipulate,
integrate, and model data.
file:///C|/SSB_old_web/an94ap.htm (17 of 18) [6/18/2004 10:43:31 AM]
