6
The Role of Archival Data
Although the committee was not directly charged with considering archival aspects, it deemed this matter highly relevant and crucial for the success of any near Earth object (NEO) survey mission. The committee was also concerned that new ground- and space-based systems may collect such large volumes of data that they could overwhelm the archives and the systems used to access them.
Observations made both from the ground and from space retain value long after they are obtained because they are valuable in providing orbit improvement and characterization information. Hence, the proper archiving and hosting of any NEO survey data is crucial for the survey’s success and its legacy value. NEO survey data include image data, source and object catalogues, as well as calibration products that are necessary to fully understand the performance of the survey system.
Image archives of the sky obtained at visual and infrared wavelengths are valuable because they may contain previously unrecognized detections of NEOs discovered by the space-based survey. Positions obtained from archived images often enable a significant improvement in the accuracy of an object’s orbit after its initial discovery. For example, discovery observations of the ~500-meter-diameter, potentially hazardous, asteroid Apophis indicated a potential impact in 2029. However, shortly after the recognition of its potentially hazardous nature, archived observations taken 3 months prior to its discovery were used to better calculate its orbit and rule out the possibility of an impact. Access to such “precovery data” requires both archival and public access to these data.
Finding a precovery image is not simply a matter of finding images that should have been found originally, but rather of being able to search an image to lower signal-to-noise ratio. This is made possible because if it is known that an object is within, perhaps, 10 resolution elements of a given position, then a signal-to-noise ratio of 3, or even less, is adequate to identify an object compared with the original search having to identify the object at a signal-to-noise ratio of approximately 5 among 109 resolution elements. In addition, when searching for a precovery image, one knows the direction and rate of motion of the object and so can perform a moving co-add, which significantly increases the signal-to-noise ratio. Finally, newer, more accurate star catalogues (e.g., Gaia), enable old positions to be remeasured to higher accuracy than at the time they were taken.
Catalogue data, including photometric source catalogues that tabulate brightness measurements taken over time, represent a higher-level product that is generated from the image data. The publication of such catalogues enables, for instance, the derivation of absolute magnitudes, which crucially support the data provided by a mid-infrared survey system by enabling the estimation of albedos and improving diameter estimates based on thermal models.
Typically, researchers put a focus on the archiving aspect, which includes the storage and backup of data. However, recent technological developments enabled the real-time browsing and search through archival image and catalogue data (e.g., the Spitzer Heritage Archive1), and the Solar System Object Image Search Tool by the Canadian Astronomy Data Centre2). Such tools have been proven immensely useful in the identification of precovery data as well as additional serendipitous observations to further constrain the physical and orbital properties of asteroids.
Finding: Effective archiving will facilitate improved knowledge of the distribution of NEO physical properties, leading to more appropriate model parameters and increased accuracy of derived properties, such as size.
Finding: Research using archival data can play and has played an important role in future threat evaluation and NEO science by the general research and planetary defense community. Archiving all data and images to support future improved thermal modeling, searching for serendipitous precovery observations, and other types of studies not considered during the survey mission is critical to detecting and characterizing NEOs.
Currently, archiving of NEO-related data is mainly accomplished through three available services in the United States: the NASA Planetary Data System (PDS),3 the International Astronomical Union Minor Planet Center (MPC),4 and the NASA Infrared Science Archive (IRSA).5 PDS is NASA’s main hub for data archival of planetary data and mainly consists of isolated archive files; the MPC is the official clearinghouse for solar system objects and provides large databases of small-body observations and orbital properties; IRSA includes image and catalogue data from the Spitzer and Wide-Field Infrared Survey Explorer (WISE) space telescopes.
The modus operandi for accessing data from these services varies significantly and often impedes the use of the data that is stored. A common problem is that data are isolated and cannot be queried in a meaningful way. For instance, archive files from the PDS and MPC have to be downloaded and information has to be extracted and combined locally. While IRSA provides a state-of-the art user interface, NEO flux data from the Near Earth Object Wide-Field Infrared Survey Explorer (NEOWISE) have to be extracted from source tables, involving highly time-consuming queries using other services.
Furthermore, most current asteroid survey programs report only astrometry and, in some cases, photometry for some select observations to the MPC, making it hard to use the existing data for further characterization efforts. The publication of additional data products would be invaluable for the physical and orbital characterization of asteroids and hence relevant to this study. To the knowledge of this committee, there is currently no uniform NASA policy in place to track whether or how NEO survey data have to be archived.
Finding: The current system for archiving NEO data is not optimized for accessing data and analyzing data in an automated fashion, and there is no consistent NASA policy on archiving NEO survey data.
The committee heard from experts on data archiving and was reassured that storage capacity for data will not be a problem in the future, given regular and reasonable upgrades to storage systems. However, storage alone is insufficient. In order to leverage the legacy value of NEO survey missions, it is mandatory that all data products generated by the survey are properly archived and made public in a timely manner. The publication of such data enables not only the generation of higher-level data products that are relevant to threat evaluation, but also an independent verification of the results through the scientific community. Support for such efforts should continue to be provided by NASA, as is currently being done through the Near Earth Objects Observation (NEOO) program
___________________
1 California Institute of Technology, 2018, “Spitzer Heritage Archive,” February 23, https://sha.ipac.caltech.edu/applications/Spitzer/SHA/.
2 Canadian Astronomy Data Centre, 2019, “Solar System Object Image Search,” modified January 28, http://www.cadc-ccda.hia-iha.nrccnrc.gc.ca/en/ssois/.
3 University of Maryland, 2019, “NASA Planetary Data System: Small Bodies Node,” https://pds-smallbodies.astro.umd.edu/.
4 Harvard and Smithsonian Center for Astrophysics and NASA, 2019, “The International Astronomical Union Minor Planet Center,” https://minorplanetcenter.net.
5 California Institute of Technology, 2019, “NASA/IPAC Infrared Science Archive,” https://irsa.ipac.caltech.edu/frontpage/.
elements of Research Opportunities in Earth and Space Sciences (ROSES). Furthermore, it is mandatory for NASA to continue support for ground-based astrometric and photometric follow-up observations to improve orbital elements and provide additional characterization information.
Recommendation: All observational data, both ground- and space-based, obtained under NASA funding supporting the George E. Brown, Jr. Near-Earth Object Survey Act, should be archived in a publicly available database as soon as practicable after it is obtained. NASA should continue to support the utilization of such data and provide resources to extract near Earth object detections from legacy databases and those archived in future surveys and their associated follow-up programs
