Establish a dual recording stream for all data from mail-in, telephone, or handheld devices: one file to go to the contractors and one to be retained by the Census Bureau. In the event of catastrophic failure by a contractor or a serious challenge to the results, it will be important to have all the raw data in the hands of the Census Bureau.
It is practical to develop simple programs, written and run by Census Bureau personnel, that will search large data files for patterns of interest. In this way, unexpected or curious results can be efficiently discovered and checked, and this can contribute to the validation and verification effort.
Related to points (1) and (2), the Census Bureau should develop quantitative validation metrics, a priori, to check for data set self-consistency and comparison of redundant data.
Other important general operational measures that we recommend for the 2010 census, either to determine whether any security breaches have occurred or to prove that the 2010 Census was secure (and which are probably already carried out), include:
Retention of an archive of all raw data with date and time stamps. In the event of serious software failure, it would be important to be able to “replay the census” from these raw data.
Use, by the Census Bureau and contractors, of dedicated processing systems that run no other applications and have highly secured network connections and secure accounts.
Use of periodic system checkpoints to monitor and analyze software systems for intrusions or unauthorized manipulations of data.
Strict control over handheld devices, including their inventory, individual device identification, and permission to operate (turn them on, turn them off, enable data transfer, disable data transfer, etc.).
Use of methods to prevent and detect bogus data streams, including data that impersonate handheld devices.
With respect to concerns about configuration control of the management information system of the 2010 census, the processing history of the dress rehearsal could be retained and the software systems intended for use in 2010 could be used to “replay” the dress rehearsal soon before the 2010 census to identify any systems that fail to interoperate. That is, assuming that the management information system for the dress rehearsal functions well, saving the processing history would then provide a means for determining whether modifications or updates of components of the management information system between 2008 and 2010 had raised any interoperability problems. (This is referred to as regression testing.) In addition, all information system errors encountered during the dress rehearsal should be captured in a form that allows them to be used during the software development work between the dress rehearsal and the start of the 2010 census.
RECOMMENDATION 9: The Census Bureau should use dual-recording systems, quantitative validation metrics, dedicated processing systems, periodic system