National Academies Press: OpenBook
« Previous: Item Nonresponse
Suggested Citation:"Reporting Errors." National Research Council. 1991. Improving Information for Social Policy Decisions -- The Uses of Microsimulation Modeling: Volume II, Technical Papers. Washington, DC: The National Academies Press. doi: 10.17226/1853.
×
Page 33
Suggested Citation:"Reporting Errors." National Research Council. 1991. Improving Information for Social Policy Decisions -- The Uses of Microsimulation Modeling: Volume II, Technical Papers. Washington, DC: The National Academies Press. doi: 10.17226/1853.
×
Page 34
Suggested Citation:"Reporting Errors." National Research Council. 1991. Improving Information for Social Policy Decisions -- The Uses of Microsimulation Modeling: Volume II, Technical Papers. Washington, DC: The National Academies Press. doi: 10.17226/1853.
×
Page 35
Suggested Citation:"Reporting Errors." National Research Council. 1991. Improving Information for Social Policy Decisions -- The Uses of Microsimulation Modeling: Volume II, Technical Papers. Washington, DC: The National Academies Press. doi: 10.17226/1853.
×
Page 36

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

DATABASES FOR MICROSIMULATION: A COMPARISON OF THE MARCH CPS AND SIPP 33 imputed, and the mean value of the imputed face amount was twice that of the reported amount ($20,893 versus $9,198). Although relatively few low- to moderate-income households with vehicles had the value imputed, the imputed amounts were higher on average than the reported amounts. Undoubtedly, the current income and asset income imputation procedures in the SIPP distort at least somewhat the picture of the population participating in programs such as food stamps. Allin and Doyle (1990) compared program participants from the 1984 SIPP panel whom they simulated to be eligible for food stamp benefits with participants whom they simulated to be ineligible because of excessive incomes or asset holdings. (Seemingly ineligible participants accounted for 11% of total participants.) They found that only 5 percent of the eligible participants but 28 percent of the ineligible participants had some or all income or asset values imputed. Reporting Errors In addition to errors that are inadequately compensated for or actually introduced as a result of imputing for nonresponse, there are ma ny kinds of reporting errors—known, speculated about, and unknown—that affect data quality in the CPS and SIPP. (Reporting errors can originate from the designated respondent, a proxy respondent, or the interviewer.) Aggregate comparisons of CPS and SIPP data with estimates from independent sources can indicate the magnitude of differences although they cannot pinpoint the sources of error. Such comparisons for income indicate that the combined effect of underreporting, overreporting, and misreporting errors (for example, reporting AFDC benefits as general assistance) is almost always a net shortfall. Tables 4 and 5 show comparisons of SIPP quarterly and CPS annual income estimates by source with independent estimates for 1983–1984. The CPS obtains higher estimates of aggregate earnings (99% of the benchmark compared with 93–95% in SIPP), but lower estimates of SSI (85 versus 90–99%), social security (92 versus 100–102%), AFDC (76 versus 76–86%), food stamps (71 versus 83–90%), and veterans' compensation (63 versus 75–80%). CPS estimates of aggregate interest, after implementation of a new imputation procedure that makes use of matched IRS data, are currently more complete than SIPP estimates, but SIPP provides more complete estimates of aggregate dividends.10 10 Caution is needed in interpreting benchmark comparisons because of conceptual differences between the surveys and the independent estimates and because the latter may also have errors. Comparisons for asset income, such as dividends and interest, are particularly problematic. With regard to conceptual differences, the personal income accounts (part of the National Income and Product Accounts), which are often used as benchmarks, include interest and dividends received by nonprofit organizations in addition to households. SOI data from tax returns are more comparable but not completely so: for example, until recently, taxpayers were not required to report interest from nontaxable sources such as municipal bonds. The available independent benchmarks are also subject to errors of various kinds—for example, there are economic incentives to conceal income from the IRS—while the quality of household-sector estimates of property income in the national accounts likely suffers from the fact that the estimates are calculated as the residual remaining after the income accruing to other sectors is accounted for. See Vaughan (1988) for discussion of these points.

DATABASES FOR MICROSIMULATION: A COMPARISON OF THE MARCH CPS AND SIPP 34 TABLE 4 A Comparison of SIPP and CPS Estimates With Independent Estimates of Income Recipients and Amounts for Selected Income Types Income Source SIPP as a Percentage of SIPP as a Percentage of CPS as a Percentage of the Independent Estimate the Independent Estimate the Independent Estimate of Average Monthly of Quarterly Income of Annual Income (1983) Recipients Wage and salary 3rd Quarter 1983 N.A. 95.0 99.0 4th Quarter 1983 N.A. 94.3 1st Quarter 1984 N.A. 93.2 2nd Quarter 1984 N.A. 94.4 3rd Quarter 1984 N.A. 95.2 4th Quarter 1984 N.A. 94.5 Federal supplemental security income 3rd Quarter 1983 92.0 89.8 84.9 4th Quarter 1983 91.3 93.5 1st Quarter 1984 94.8 96.4 2nd Quarter 1984 98.2 97.4 3rd Quarter 1984 98.3 98.6 4th Quarter 1984 98.1 99.2 Social security 3rd Quarter 1983 99.2 99.6 91.7 4th Quarter 1983 96.3 100.6 1st Quarter 1984 97.3 100.5 2nd Quarter 1984 97.7 101.1 3rd Quarter 1984 97.5 101.3 4th Quarter 1984 97.5 101.6 Aid to Families with Dependent Childrena 3rd Quarter 1983 78.5 76.2 76.0 4th Quarter 1983 79.2 78.5 1st Quarter 1984 84.5 85.3 2nd Quarter 1984 86.0 86.0 3rd Quarter 1984 82.0 80.2 4th Quarter 1984 80.7 78.8 Food stamps 3rd Quarter 1983 89.5 90.1 71.2 4th Quarter 1983 91.0 83.1 1st Quarter 1984 90.8 85.2 2nd Quarter 1984 90.5 86.2 3rd Quarter 1984 90.3 84.6 4th Quarter 1984 91.7 83.6 Veterans' compensation or pensiona 3rd Quarter 1983 89.2 78.9 63.3 4th Quarter 1983 89.7 79.9 1st Quarter 1984 90.6 78.0 2nd Quarter 1984 90.8 74.5 3rd Quarter 1984 89.8 76.3 4th Quarter 1984 93.3 79.7 NOTES: N.A., not available. SIPP estimates are from the 1984 panel; CPS estimates are from the March 1984 income supplement. aRecipients exclude dependents covered by payments. SOURCE: Jabine, King, and Petroni (1990: Table 10.1).

DATABASES FOR MICROSIMULATION: A COMPARISON OF THE MARCH CPS AND SIPP 35 TABLE 5 SIPP and CPS Estimates Compared With Independent Estimates of Aggregate Interest and Dividend Income, 1983–1984 and Calendar Year 1984 (in billions of dollars) 1984 Income Source 1983–1984 (NIPA-Based) NIPA-Based Reported to IRS Interest Independent estimate $239.1 $254.6b $176.4 Survey estimate SIPP $115.7 $115.4 CPS Original imputation $104.7 $109.2 Revised imputationa $129.5 $138.7 Survey estimate as a % of the independent estimate SIPP 48.4 45.3 65.4 CPS Original imputation 43.8 42.9 61.9 Revised imputation 54.2 54.5 78.6 Dividends Independent estimate $63.6 $66.5b $50.6c Survey estimate SIPP $38.3 $40.3 CPS $29.1 $30.7 Survey estimate as a % of the independent estimate SIPP 60.1 60.7 79.7 CPS 45.8 46.1 60.6 aThe revised imputation corrects for bias in the pre-1983 imputation procedure by making use of matched IRS information on interest income. b Personal income aggregate from the National Income and Product Accounts (NIPA) adjusted to the survey universe based on observed relationship between NIPA aggregate and independent estimate for the CPS universe in 1983. cTotal domestic and foreign dividends received. SOURCE: Jabine, King, and Petroni (1990: Table 10.3). The new processing system just introduced for the March 1989 CPS produces somewhat higher aggregate income estimates than previously obtained (by 1% overall and 3% for unearned income), because imputations are now based on the 49 separate income types that are collected rather than on 11 combined income types (see Welniak, 1990). As the Census Bureau admits, the introduction of the new processing system, which also made other improvements in the imputation methodology, is “long overdue” (Bureau of the Census, 1990). (The expanded income questionnaire was introduced in the March 1980 CPS,

DATABASES FOR MICROSIMULATION: A COMPARISON OF THE MARCH CPS AND SIPP 36 but the processing system to handle the additional detail was not implemented until last year.) Only one recent study has analyzed reporting errors in the March CPS. Coder (1990) examined reporting of wage and salary income in the March 1986 CPS for married couples who were successfully matched with IRS records.11 He found (pp. 32–33): First, nonsampling error is much larger than that evidenced by the aggregate measures that are generally used to characterize it. The 2–3 percent underestimate in the aggregate masks widespread over- and underreporting of amounts and imputation procedures that account for a small proportion of the bias introduced by nonresponse. Second, the most serious error problems seem more concentrated at the bottom and top of the distribution. Third, in spite of these errors, the distribution of wage and salary income from the CPS for the matched universe was very similar to that derived from the IRS data…. This does not say that the estimates of the characteristics of persons at specified points in the distribution, notably the tails, are not biased in some way by these errors. Estimates for one universe in particular, the working poor, may be especially at risk. A number of studies have investigated some types of errors in SIPP. One important study involves checking reported amounts for various transfer payments against administrative records of amounts actually paid out. The SIPP record check study compared data from the first two waves of the 1984 panel with administrative data from four states for eight programs—AFDC, SSI, food stamps, social security, veterans' compensation and pensions, federal civil service retirement, unemployment insurance benefits, and workers' compensation (Marquis, Moore, and Huggins, 1990). The results show negatively biased participation rates for most programs: that is, net underreporting of participation, although there are overreports as well as underreports. For most programs, there appears to be relatively little bias in reporting of benefit amounts for those 11 This study is the first in a series of planned evaluations by the Census Bureau of the quality of current income reporting in the March CPS and SIPP, using administrative data and other sources. In turn, this work is part of a broader project to investigate ways to develop improved income statistics by combining data from the CPS, SIPP, and administrative records. Previous studies that have been conducted of reporting errors in the CPS have concentrated on earnings and labor force status. Some years ago, Herriot and Spiers (1975) used matched CPS and IRS tax data to evaluate the quality of reporting of earnings in the March 1973 CPS. The results showed good consistency of reporting for people with low to moderate levels of earnings, but underreporting for people with high levels of earnings. (David et al. [1986] obtained similar results with the 1981 exact-match file.) More extensive, although far from complete, analysis of reporting errors in the CPS has been conducted of the labor force information from the core questionnaire. Brooks and Bailar (1978) summarized the available evidence as of that time on the nature and extent of reporting errors, from sources such as rotation group bias, in the CPS estimates of employment and unemployment. More recently, Fuller and Chua (1985) used reinterviews to examine the quality of the CPS labor force estimates.

Next: Sample Size »
Improving Information for Social Policy Decisions -- The Uses of Microsimulation Modeling: Volume II, Technical Papers Get This Book
×
Buy Paperback | $100.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

This volume, second in the series, provides essential background material for policy analysts, researchers, statisticians, and others interested in the application of microsimulation techniques to develop estimates of the costs and population impacts of proposed changes in government policies ranging from welfare to retirement income to health care to taxes.

The material spans data inputs to models, design and computer implementation of models, validation of model outputs, and model documentation.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!