Engagement Supplements were rotated each November (or September)—reduces the effective sample size on an average annual basis by half. To maintain confidence interval widths, data pooling would have to encompass a time period twice as long. And it obviously would not be possible to estimate year-to-year changes, even nationally or at the state level.
The characteristics of interest, and the way they change temporally or vary spatially, create opportunities for sample design tradeoffs and experimentation with the periodicity with which questions appear on a module. Particularly with a combined November supplement, as described above, it may not be optimal to have identical content each year, and it is important to assess which information would suffer least from less frequent collection. For phenomena that do not change rapidly, less frequent sampling is not a bad tradeoff to exploit. If, for example, patterns of volunteering abroad (question S15) do not change quickly, that question could be a candidate to be fielded every other year, which would open up survey space for other questions. If there is not much demand to do research on short time interval trends in participation, voting, and other phenomena, there is less need for annual data collection. On the other hand, if one wanted to track erosion in a population’s confidence in a rapidly changing political climate for purpose of anticipating social unrest, an infrequent survey is an ineffective option (indeed, even a more frequent survey might not be the best way to tap into such feelings). If measuring trends is a priority (as is the case, for example, for survey data on which monthly unemployment rates are based), adequate sample size becomes important for establishing statistical significance.
The 2-year cycle framework suggests a core set of questions to be asked each year and another set that might be asked every 2 years, or even less frequently, thereby clearing space for additional biannual questions. Core questions would be reserved for items where change and granularity, to the extent it exists among the current topics, are needed (or one or the other); where neither is needed, questions become candidates for less frequent inclusion, on a rotating basis.
Another issue that supports our recommendation for a combined supplement—as opposed to separate, biannual Volunteer and Civic Engagement Supplements—involves the way the overall CPS survey sample is rotated. Currently, analysts can take advantage of the fact that the sample overlaps from year to year because respondents are in the sample for 4 months, out for 8 months, then back in for 4 additional months. This sample rotation format means that half the sample respondents in any given month were also present in the sample 1 year earlier; therefore, more precise estimates of annual change can be obtained, a feature that would be lost if only 2-year change estimates were possible.