The 2014 Redesign of the Survey of Income and Program Participation An Assessment (2018) / Chapter Skim
Currently Skimming:
5 Methodological Enhancements
5 Methodological Enhancements
Pages 59-82
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 59... ...
These enhancements are discussed in this chapter. IMPUTATION METHODOLOGY In Reengineering the Survey of Income and Program Participation, the National Research Council panel recommended, "the Census Bureau should move to replace hot-deck imputation routines for missing data .
|
From page 60... ...
Model-Based Imputation Creating Topic Flags The imputation process developed for the 2014 SIPP combines both model-based and hot-deck methods. A model-based approach is used to impute topic flags.
|
From page 61... ...
using information from available household, parent, and spouse characteristics, as well as from administrative records. In most cases this is a modelbased imputation, but on occasion a logic-based imputation is possible.2 Topic flags are imputed in the first round.
|
From page 62... ...
provides a listing of the topics for which topic flags were produced and the percentage of the in-universe respondents that were imputed. Addressing administrative data, the table also compares respondents that were imputed, by topic, with those who responded, based on whether the Census Bureau could link the individual to Social Security data.
|
From page 63... ...
72,065 6.3 69.5 91.2 Dependent Care 20,835 2.1 72.0 92.9 TANF 41,011 4.9 61.7 91.0 WIC 58,367 3.4 70.5 90.9 NOTES: OASDI = Old-Age, Survivors, and Disability Insurance; SNAP = Supplemental Nutrition Assistance Program; SSA = U.S. Social Security Administration; SSI = Supplemental Security Income; TANF = Temporary Assistance for Needy Families; WIC = Supplemental Nutrition Program for Women, Infants, and Children.
|
From page 64... ...
Benefit Topic Flag Response and Receipt by 2012 Earnings from Administrative Records (in %) "Yes" Received Conditional "Yes" Household SNAP in 2013 In-Universe for SNAP In-Universe for SNAP Income from Administrative SIPP SIPP SIPP SIPP SIPP SIPP Records Reported Imputed Reported Imputed Reported Imputed No Earnings 35.3 16.4 20.5 6.7 23.5 22.4 >$0 but <$25,000 43.0 45.9 24.9 25.0 23.5 19.8 ≥$25,000 21.7 37.7 54.6 68.4 5.3 6.7 Total 100.0 100.0 100.0 100.0 SOURCE: Benedetto et al.
|
From page 65... ...
Column 2 of Table 5-3 presents the results for earnings data from administrative records, also categorized by SIPP survey respondents and nonrespondents. (Again, the administrative data are for 2012, the year prior to the survey reference year, as 2013 data were not TABLE 5-3 Job Line 1 Topic Flag Compared to 2012 Administrative Record Employment Overall percentages for cases where SIPP respondent answered the first question about jobs held (94.5% of in-universe respondents)
|
From page 66... ...
considered the rate of the "Yes" response for five topic flags (Job Line 1, SNAP, WIC, Disability, and Education Enrollment) , broken down by imputed versus reported for each quintile of the predicted propensity of response for that topic.
|
From page 67... ...
2. While the topic flag imputations demonstrate a potential major step forward, the topic with the highest incidence of missing data -- assets -- continues to be imputed using hot-deck methods alone.
|
From page 68... ...
2. In Appendix A, the study panel presents the statistical form of the joint distribution of the topic flags.
|
From page 69... ...
• Move to replace hot-deck imputation routines for missing data in SIPP with modern model-based imputations, implemented mul tiple times to permit estimating the variability due to imputation; imputation models for program participation and benefits should make use of program eligibility criteria and characteristics of ben eficiaries from administrative records so that the imputed values reflect as closely as possible what is known about the beneficiary population.5 • Request the Statistical and Science Policy Office in the U.S. Office of Management and Budget to establish an interagency working group on uses of administrative records in SIPP.
|
From page 70... ...
In this context, "Social Security" refers more specifically to OASDI, but these three distinct programs are more widely known collectively as "Social Security." These studies demonstrated, among other things, that some respondents whose administrative records indicated that they received OASDI but not SSI reported that they received SSI benefits -- sometimes with and sometimes without also reporting receipt of OASDI benefits. Other respondents whose administrative records indicated that they received SSI but not OASDI benefits reported that they received only OASDI benefits or both OASDI and SSI (Huynh et al., 2002)
|
From page 71... ...
When Census Bureau researchers linked SSA administrative records to the first wave of the 2014 panel, they found that nearly half of those respondents who reported receiving SSI had no administrative records confirming their receipt of SSI benefits. The misreporting of SSI receipt was of two general types: double reporting and program swapping.
|
From page 72... ...
These efforts should include, among others, exploiting the Internal Revenue Service Form 1099-R data on pension income and IRA withdrawals, Internal Revenue Service Form 1099-INT and 1099-DIV data on interest and dividend income, and the SSA Detailed Earnings Record data on tax deferred pension contributions. RECOMMENDATION 5-2: The Census Bureau should continue to investigate effective use of administrative records, such as Internal Revenue Service data mentioned in this report, to enhance the quality of data collected in the Survey of Income and Program Participation.
|
From page 73... ...
For SIPP, the Blaise CAPI instrument controlled the CARI recordings, with instrument coders being able to select which parts of the interview would be recorded. The study panel was particularly interested in listening to selected interviews to help understand outliers and identified data problems.
|
From page 74... ...
• Most critical for the study panel's work, the software plugin that ran the EHC module was not set up to allow CARI recordings in any of the EHC sections.6 This was most unfortunate because this plugin encompasses most of the program participation and job sections for which the panel wanted to examine seam effects. The study panel initially had a goal to behavior-code items related to program participation, looking for evidence of behaviors that may contribute to seam effects.
|
From page 75... ...
INCENTIVE EXPERIMENTS IN THE 2014 REDESIGNED SIPP A fourth area of methodological enhancements employed in the 2014 SIPP is the continued experimentation with incentives7 to strengthen survey response. We begin by providing some background on incentive experiments more generally and then talk about the specific experiments conducted by the Census Bureau using the reengineered SIPP.
|
From page 76... ...
Lessons Learned from Other Federal Surveys The Census Bureau has a rich history of using incentives to improve response rates and reduce attrition, including experiments in SIPP and the Survey of Program Dynamics. This point also holds for a host of other federal surveys including the Medical Expenditure Panel Survey (MEPS)
|
From page 77... ...
. Incentive Experiments in SIPP Earlier Experiments and Results The SIPP program conducted incentive experiments in 1996 and 2001 to address rising nonresponse rates and incorporated incentives for all cases in 2004.
|
From page 78... ...
. The $20 incentive also led to statistically significantly reduced item nonresponse rates relative to no incentive.
|
From page 79... ...
Both subgroups showed higher response rates compared to the Group 1 control (no incentive) in wave 2, 14 The study panel's description here of the Census Bureau experiments on incentives is based on (1)
|
From page 80... ...
The study panel was given Census Bureau plans for wave 3 incentives but had seen no results as of the time of this report. The plans were to target incentives to those predicted in waves 1 and 2 to have low response rates and the highest increase in response based on an incentive.
|
From page 81... ...
The panel commends the Census Bureau and SIPP staff for their research on incentives and encourages continued work to devise an effective long-term plan. RECOMMENDATION 5-5: The Census Bureau should continue to research and utilize incentive programs in the Survey of Income and Program Participation to promote response and reduce nonresponse bias with the goal of devising an effective long-term plan.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.