Letter Report addressed to Dr. Martha Farnsworth Riche, Director, U.S. Bureau of the Census (1997)

COMMISSION ON BEHAVIORAL AND SOCIAL SCIENCES AND EDUCATION

2101 Constitution Avenue Washington, D.C. 20418

November 10, 1997

Dear Dr. Riche:

The Panel to Evaluate Alternative Census Methodologies has been reviewing the plans for the 2000 census since mid-1995. We have concluded that the Census Bureau's plans to design the census to achieve a modern, efficient, integrated, and accurate approach to counting the U.S. population are moving in the right direction to ensure more reliable data; that sampling procedures are necessary for significantly improving the accuracy and cost efficiency of the census; and that there is no reasonable alternative for reaching these goals (see National Research Council, 1996, 1997). In addition, much of the Bureau's extensive program of research, testing, and evaluation of its methods in anticipation of the 2000 census is in accord with findings and recommendations of this panel and those of two predecessor panels (National Research Council, 1994, 1995).

We write at this time regarding the proposed mailing and unduplication procedures for the 2000 census. We believe that certain elements of the proposed plan merit special attention during the planning, execution, and evaluation of the 1998 census dress rehearsal. The dress rehearsal provides the last opportunity to test and fine tune the components of the plan for the 2000 census--nearly all of which have been tried and tested previously--to confirm that they operate successfully together in a near-census environment.

The Bureau's original plan for the mail survey procedures during the first phase of the 2000 census were thoroughly tested to maximize mail response. That plan called for contacting households up to four times by mail:

• a prenotification letter to households heralding the arrival of the census form in the mail in the next few days;

• the census form;

• a reminder postcard to thank those who had already completed the form and to urge completion by those who had not;

• a replacement census form to those who had not yet responded.

The Bureau's research has demonstrated that a prenotification letter, reminder postcard, and replacement questionnaire all increase response significantly. The approach was also shown to be very cost-beneficial because the increased response rates produced savings in follow-up costs that outweighed the additional expenditures for mail contacts. This conclusion was based on consideration of both direct costs (for example, printing and postage) and indirect costs (for example, unduplication costs resulting from some households returning both the original and the replacement forms).

Subsequent to this research and the resulting plan for the mail survey procedures, however, the Census Bureau determined that, under the time restrictions and volume requirements of the actual census, it cannot execute the plan to target the replacement forms to only those households who have not yet responded. The Bureau learned that the sheer volume of forms involved in the actual census makes it impossible to selectively address and mail the expected number of replacement forms in the very short time available between receiving the first wave of mail responses and mailing the replacement. The delay required for a targeted replacement mailing would be so great as to far outweigh the benefits of sending such a replacement. Because its well-researched mailout plan cannot be implemented in a cost-effective way, the Census Bureau has altered the plan to include a blanket (or nontargeted) mailing of replacement forms. That way, the replacement forms can be printed and addressed at the same time as the original forms are done. With this change, all households will receive four pieces of mail:

• a prenotification letter;

• the first census form;

• a reminder postcard, to thank those who had already completed the form and to urge completion by those who had not; and

• a replacement census form to all households, regardless of whether or when they have returned the original census questionnaire.

The expected benefit of increased mail response appears likely to outweigh the cost of printing sufficient replacement forms and mailing one to each household.

This change may have unintended, noneconomic consequences on the response. Even if those consequences are small in percentage terms, their potential impact in a census-sized project can produce significant costs or delays and affect data quality. The panel concludes, therefore, that the change in the mailout procedure warrants very careful attention in the dress rehearsal. The two major elements that need study are public perception and finding and resolving cases in which more than one census form has been filed (“unduplication”).

With the introduction of a blanket mailing of replacement forms, public misunderstandings regarding the efficiency and effectiveness of the census could arise, possibly negatively affecting the mailback response rate. In an operation that depends on

de facto voluntary response, public perceptions are critical. For the first time in the 2000 census, the Bureau will be using a paid advertising and communications campaign. The panel supports the Bureau's intention to work with its advertising and communications experts to develop strategies to offset or prevent public misperception of the blanket replacement mailing strategy and other census procedures. In developing those strategies, convening focus groups prior to the dress rehearsal could help the Bureau anticipate public perception issues in regard to blanket replacement plans. Subsequently, quantitative assessment should be a part of the advertising and communications campaign in both the 1998 dress rehearsal and the 2000 census.

The unduplication process is particularly important. In the plans for a redesigned census, the Bureau is making every effort to make it easy for households to respond. These efforts include: a Be Counted program, which makes unaddressed forms available in public places; non-English forms to be distributed in targeted areas together with English forms; and a limited capability for obtaining census information by telephone response--in addition to the replacement census forms to be mailed to all households. Consequently, almost all households will have access to several census forms and, for a variety of reasons, many may return more than one.

On the basis of its experience in the 1995 and 1996 census tests, the Bureau believes that field and computer procedures designed to rectify these situations can easily handle the volume of duplicate and erroneous forms that would be expected in a census. But the 1995 and 1996 tests used targeted mail replacement forms, not blanket replacement forms for all households. Thus, it is not at all clear that the experience from the earlier tests is applicable to the situation of the 1998 dress rehearsal and the 2000 census. The Census Bureau will attempt to minimize confusion about multiple census forms through the messages in the reminder postcard and in the letter accompanying the replacement form, messages that clearly specify that only one form need be returned. However, in many households there will be an opportunity for different household members to return the original and replacement forms. There is also the possibility that the arrival of a second form at so many households that have already submitted the first form might induce unanticipated patterns or mechanisms of duplication, which would result in a need for changes in a variety of census procedures. Research to date does not allow for accurate assessment of the likelihood or size of these possibilities.

The Bureau staff should pay specific attention in the dress rehearsal to the impact of the blanket replacement mailing on the unduplication process. The panel believes that the Census Bureau should put in place a mechanism for quickly evaluating the efficiency and effectiveness of the mailout plan and be prepared to alter the 2000 mailout plans if the blanket replacement procedure proves to be ineffective, inefficient, or unduly disruptive to the census process.

Recommendations

The panel recommends that the Bureau examine in further detail the cost-benefit evaluation of the blanket replacement mailing (including both direct and broad indirect costs) to assess the sensitivity of savings estimates to changes in response rates or unduplication workloads.

The panel recommends that the Census Bureau immediately focus on developing and implementing dress rehearsal evaluation procedures for issues surrounding the mailout and unduplication plans. The Bureau should complete this evaluation of the mailout and unduplication procedures as soon as possible during and after the dress rehearsal.

The panel recommends that the Census Bureau carefully examine (and measure if possible) the public reaction to the replacement mailing strategy as well as any effects on response and data quality during the dress rehearsal.

The Census Bureau has demonstrated that it has a strong, open, and effective design process and internationally recognized expertise in data collection and statistical methodology. The panel reiterates that the 2000 census planning process is headed in the right direction and that the Bureau is developing procedures that can accomplish the redesign goals: reducing costs, reducing the time and effort required of respondents, doing a better job of counting traditionally undercounted groups, and improving data quality. Our recommendations in this letter are offered in the spirit of maintaining the high standards the Bureau has set.

National Research Council 1994 Counting People in the Information Age. D.L. Steffey and N.M. Bradburn, eds., Panel to Evaluate Alternative Census Methodologies, Committee on National Statistics, National Research Council. Washington, D.C.: National Academy Press

1995 Modernizing the U.S. Census. B. Edmonston and C. Schultze, eds., Panel on Census Requirements in the Year 2000 and Beyond, Committee on National Statistics, National Research Council. Washington, D.C.: National Academy Press

1996 Sampling in the 2000 Census: Interim Report I. A.A. White and K.F.Rust, eds., Panel to Evaluate Alternative Census Methodologies, Committee on National Statistics, National Research Council. Washington, D.C.: National Academy Press.

1997 Preparing for the 2000 Census: Interim Report II. White, A.A., and K.F. Rust, eds., Panel to Evaluate Alternative Census Methodologies, Committee on National Statistics, National Research Council. Washington, D.C.: National Academy Press.