tracking software requirements and specifications. In beginning the task of improving its software practices, though, the Census Bureau must recognize that the effort is a difficult one, requiring high-level commitment in the same manner as architecture reengineering.
On a related note, and consistent with the Bureau’s broader efforts to improve software engineering practices, we urge the Census Bureau to assess its standards and planning assumptions related to hardware and software experience. It is well known in the software development community that it is vastly more expensive to detect bugs and operational errors when hardware and software have been fielded than to catch those bugs during prerelease testing; this lesson has also been learned by other survey organizations as they have moved increasingly into computer-assisted interviewing methods (National Research Council, 2003b). For the census—a technologically intensive survey of grand scale with a strict timeline—catching software errors early is particularly important to smooth operations.
Recommendation 6.5: The Census Bureau should evaluate and improve its protocols for hardware and software testing, drawing on expertise from the computer science and software development communities. Rigorous hardware and software testing should be factored into census operational schedules, in addition to the field testing performed in the 2006 proof-of-concept test, the 2008 dress rehearsal, or such other formal census tests as may arise.