These quite precise estimates would be highlighted for users who want to know what they can confidently learn from the data and are daunted by the array of table cells with large margins of error.
In addition, the Census Bureau should review its guidance for calculating standard errors for user-constructed estimates of sums and differences. The documentation should provide many more examples for a range of applications to make clear how the guidance can be used. It should also emphasize more strongly when the guidance is not readily applicable.
Recommendation 7-6: The Census Bureau, in consultation with data users and statistical methodologists, should evaluate its presentation of sampling errors of estimates that are published on the ACS web site and also its descriptions of methods for computing approximate estimates of sampling errors for estimates for which sampling errors are not published. Steps should be identified to improve the usability and ease of comprehension of information on sampling errors.
In addition to monitoring basic quality measures, a major continuing survey such as the ACS requires periodic, in-depth assessments of data quality on a wide range of dimensions across time and among population groups and geographic areas. The benefits of such assessments accrue not only to data users, who can gain deeper understanding of the value and challenges of the data, but also to survey managers who require information to help them identify areas for methodological research and subsequent survey improvement.
A comprehensive survey evaluation is referred to as a quality profile. Such a document brings together and analyzes the magnitudes of and contributions to sampling and nonsampling errors from various survey component processes for estimates from a survey, generally, and, when possible, for specific questionnaire items. A quality profile also typically includes comparisons of selected survey estimates with estimates from other surveys or administrative records. Examples of quality profiles include those developed for the American Housing Survey (Chakrabarty, 1996); the Residential Energy Consumption Survey (Energy Information Administration, 1996); the Schools and Staffing Survey (Kalton et al., 2000); and the Survey of Income and Program Participation (U.S. Census Bureau, 1998).
A quality profile for the ACS would be complex to prepare and require