fleshing out more specific study plans for the ideas that are ultimately selected for experimentation in the coming months.

We also think that the Census Bureau needs to increase its in-house expertise in experimental design regarding census experimentation. The panel has seen evidence in the past that some experiments, in both censuses and test censuses, have not been fully consistent with accepted principles of experimental design. This includes the use of preliminary assessments of which factors might affect a response of interest, the use of controls and blocking for meaningful comparisons (see, e.g., National Research Council, 2006:Rec. 6.8), and the simultaneous varying of test factors (including use of orthogonal designs, factorial designs, and fractional factorial designs) for greater effectiveness of test panels. Also, often not enough attention is paid in advance to the statistical power of tests. Certainly some of this can be attributed to the fact that the primary function of a census or a census test is an opportunity to assess the full census operation with the embedded experiments having to make do with various limitations. However, it is important for the Census Bureau to improve its application of experimental design techniques for its experiments, both to reduce the costs of the experimentation and to increase the information contained in the results.