Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
1 Introduction The most exciting thing going on in social science in the 1980s; the most significant statistical survey in four decades; . . . the most important data available in the 1980s for research on American families and individuals; . . . a survey that . . . fillips] a major void and benefits] many agencies.... (Hunt, 1985:99-100, 148) The object of these glowing words the Survey of Income and Program Participation (SIPP) began operations in the fall of 1983, when interview- ers of the Bureau of the Census fanned out across the country to ask resi- dents of 20,000 households a set of detailed questions about their social and economic circumstances. At 4-month intervals ("waves") over the next 2-1/2 years, the interviewers returned to each household in the 1984 SIPP panel to obtain updated information. The survey did not stop with one panel: be- ginning in February 1985 and each year thereafter, Census Bureau inter- viewers queried a new sample of households, revisiting each of them at 4- month intervals over a period of about 2-1/2 years. What is this survey and why were people so enthusiastic about its prospects?) SIPP IN BRIEF As its name implies, SIPP was designed to improve information on the income distribution and economic well-being of the population and on par- ticipation and eligibility for a wide range of government social welfare 1The set of comments quoted above about SIPP are from Charles Lininger, an economist who directed developmental work on SIPP at the U.S. Department of Health and Human Services for several years; Joseph Duncan of Dun and Bradstreet, formerly chief statistician of the U.S. government; Guy Orcutt, an economist at Yale University (recently retired); and Bruce Chapman, director of the Census Bureau in the early 1980s. 13
4 THE SURVEY OF INCOME AND PROGRAM PARTICIPATION programs for example, Aid to Families with Dependent Children (AFDC), food stamps, social security, unemployment compensation, Medicare, and Medicaid. Other continuing surveys, including the Current Population Sur- vey (CPS) March income supplement, which since the mid-1940s has sup- plied most of the available statistics about household income, could not meet the growing needs for information to support federal social welfare program planning and socioeconomic research. Within this broad framework, the following specific goals of SIPP and some of the design features that resulted from those goals were identified: · to improve the reporting of family and personal income, both cash and in-kind, by source by asking more questions and by obtaining reports more frequently than once a year; · to obtain detailed information, comparable to administrative data, on program participants, including multiprogram participants, and on the dy- namics of participation over time by asking for monthly information at each interview, with more detailed questions and relevant explanatory vari- ables, and by following the same people to observe program entries and exits; · to obtain information necessary to determine program eligibility, in- cluding data on assets, and to characterize participants in comparison with eligible nonparticipants; · to provide an opportunity to obtain timely information on emerging concerns of social welfare policy, broadly defined-by including special sections of questions (topical modules) on subjects of current policy interest (e.g., disability, child support, day care, health status, and use of health care); · to maintain the quality of annual income and poverty statistics and other cross-sectional estimates developed from the longitudinal SIPP data- by starting a new SIPP panel every year with a fresh sample of households; and . to improve both participant and income-by-source information by comparing survey reports with venous administrative files. The first SIPP panel that was introduced in October 1983 included about 21,000 households. Because of budget restrictions, the sample sizes of subsequent panels have varied from 12,500 to 23,500 households, and some panels have had fewer than the originally planned eight interview waves. The sample for each panel includes adults 15 years of age and older who were living in the household at the time of the first interview; they are followed if they move to new addresses during the panel's life. For chil- dren under 15 and adults who reside in a household containing an original sample adult during the life of a panel, data are collected only if they continue to reside with an original sample adult.
INTRODUCTION 15 The SIPP questionnaire contains two sections. The core section in- cludes questions about income sources and amounts, program participation, and labor force activity: it is asked in every 4-month interview wave. The topical module section, which is also asked in all waves, includes one or more modules on selected topics. "Fixed" topical modules, which are asked of each panel once or twice in its life, cover assets and liabilities, income taxes paid, annual income, program eligibility, and personal histories. "Variable" topical modules, for which there is competition to appear in SIPP, have ranged over a large number of topics, such as child care expenses, health status and use of the health care system, housing costs and financing, and child support. SIPP was long in the malting: planning and development activities spanned most of the decade of the 1970s. And when SIPP was originally scheduled to become operational (January 1981), it appeared that the survey would be stillborn: all funds for SIPP were deleted from the federal budget in 1980 and again in 1981. A rescue effort mounted by the newly appointed director of the Census Bureau and other staff in the executive branch and Congress persuaded the administration and Congress in the summer of 1982 to restore full funding for SIPP in the budget of the Census Bureau. (The original plan had been to have the survey sponsored by the Social Security Administration and conducted by the Census Bureau, with costs divided between them.) The restoration of funds permitted the survey to get under way in 1983. It is currently funded at about $31 million annually. Now, after nearly 9 years of operation, the Census Bureau has initiated a thoroughgoing reassessment of SIPP. The survey has been functioning long enough for users both inside and outside the Census Bureau to develop experience in working with the data. In addition, results are available on many aspects of SIPP operations and data quality from the extensive meth- odological research program that has been part of SIPP since the beginning. Hence, there is the basis for an in-depth review and consideration of changes that could enhance the utility and cost-effectiveness of the program in the future. A review at this juncture is also timely because of the 10-year cycle- centering around each decennial census-that typically characterizes evalu- ation and redesign of the continuing household surveys that the Census Bureau conducts. A new sample design for SIPP, based on data from the 1990 census about the geographic distribution and other characteristics of the population, will be implemented beginning with the 1995 panel. It is convenient to make any other major changes in the survey that appear desir- able at the same time. Such changes could affect the design and content of the SIPP questionnaire, features of the survey design (e.g., length of panels or frequency of interviews), strategies for data collection and processing, and publications and other data products.
6 THE SURVEY OF INCOME AND PROGRAM PARTICIPATION As part of the redesign effort, the Census Bureau asked the Committee on National Statistics (CNSTAT) at the National Research Council to ap- point a study panel to conduct a wide-ranging review of SIPP. The panel, which began its work in summer 1990, was asked to pay particular attention to ways to make more use of the longitudinal data from SIPP and to use data from SIPP and other related surveys and administrative records to improve the nation's statistical series on the distribution of income. The panel was also asked to consider how the views of users could best feed into the planning and conduct of the survey. SIPP TO DATE Achievements There is no question, in our view, that SIPP is clearly established as a major continuing survey that provides important information for federal policy making and social science research. A few years ago, this conclusion might have been in doubt, given the rocky childhood that followed SIPP's difficult birth. As noted, repeated budget cutbacks forced the Census Bureau to reduce the sample size for most panels and, for some panels, to reduce the number of interview waves as well. The Census Bureau then experienced difficulties in processing the large volume of data generated by the stream of interviews from the field. Users also experienced problems in working with the complex data sets from the survey and, consequently, only slowly began to exploit the richness of the information. At present, funding is sufficient for the Census Bureau to operate the survey at the originally planned level. Even more important, the survey has developed a growing community of committed users who have used the data for a range of policy analyses arid research studies. In this section we highlight just a few examples of important new insights from SIPP that are relevant to social welfare policies and programs and to research. Part-Year Poverty and Program Participation Federal and state assistance programs such as AFDC and food stamps are designed to help people who experience short periods of hardship, as well as those in need for longer periods. SIPP provides information that was previously unavailable on part-year periods of low income and on the pro- portion of program recipients who rely on benefits for temporary assistance in comparison with the proportion who depend on them over the longer term. Using data from the 1984 SIPP panel, Ruggles and Williams (1987: Table 1) found that fully 26 percent of the population experienced at least
INTRODUCTION 17 1 month of income below the poverty line in a year, although relatively few people about 6 percent were poor every single month. These rates var- ied dramatically across family types. For example, only 3 percent of people in marred-couple families were poor every month of the year; in contrast, 26 percent of people in female single-parent families were poor every month. Ruggles (1989:Table 1) estimated from the 1984 SIPP panel that the median duration for receipt of AFDC was about 11 months, providing a different picture of the program from previous analyses using annual data. For example, Bane and Ellwood (1983), working with the Panel Study of Income Dynamics (PSID), estimated that the median duration of-AFDC was about 2 years. Although the reasons for the differences in estimated spell length are not definitely established, it seems likely that the SIPP monthly data pick up short spells of AFDC that are omitted or merged into fewer, longer spells in the PSID annual data.2 In Ruggles's study, people most likely to stop receiving AFDC in less than a year were the previously mar- ried or previously employed (60-65 percent exited AFDC within a year). In contrast, only 40 percent of never-married recipients exited the program within a year, and another 40 percent were still receiving AFDC after 2 years. Multiple Program Participation The number and scope of federal and state assistance programs have grown enormously since the 1960s. The annual data from the March CPS income supplement can only show how many people receive benefits from more than one program at some time during the year. SIPP can distinguish among intrayear patterns of multiple program participation, specifically, whether people receive multiple benefits concurrently or follow a sequential process of program receipt. Doyle and Long (1988:Tables D-1-D-6) found complex patterns of pro- gram participation in the first 12 months of the 1984 SIPP panel. In the initial month, 23 percent of the population participated in one or more of the following programs: social security, Supplemental Security Income (SSI), public assistance (including AFDC and general assistance), and food stamps. Of program recipients, 24 percent participated in more than one program. The most popular combinations were public assistance and food stamps (70% of all multiple program participants), social security and food stamps (9%), and social security and SSI (8%~. During the next 11 months, about 23 percent of initial program recipients experienced at least one tran- sition to a different program combination or ended their participation. 2In an analysis of AFDC receipt from the 1984 and 1985 SIPP panels, Fitzgerald (1992) obtained the some results as Ruggles.
18 THE SURVEY OF INCOME AND PROGRAM PARTICIPATION Effect of Assets on Program Eligibility and Poverty Public assistance programs typically place a low ceiling on the value of assets that people can hold and still be eligible to receive benefits. More generally, assets that people can "spend down" provide a cushion against periods of low income. SIPP, in contrast to the March CPS, provides suffi- cient information to assess the role that capitalizing on assets can play in maintaining adequate income and, hence, consumption levels. In a study with the 1984 SIPP panel, Ruggles and Williams (1989:Table 6) found that simulating the spend-down of financial assets eliminated 35- 40 percent of all the periods of poverty that were observed over a 32-month period. However, the median duration of the remaining periods was slightly longer than when assets were not taken into account. Doyle and Trippe (1989:Table 15) found that a simulation of the food stamp program for August 1984 based on SIPP data produced a lower esti- mate of households eligible for benefits and hence a higher participation rate in the program-than did a simulation based on March CPS data. A primary reason was that the more extensive asset data in SIPP (in compari- son with the CPS) resulted in disqualifying a larger number of households from eligibility for food stamp benefits because they failed to meet the asset test. Health Insurance Coverage Public and private spending for health care in the United States currently accounts for one-eighth of the gross national product, yet many Americans lack health care insurance. Issues of insurance coverage and affordability of health care are at the forefront of public policy debate. SIPP provides data that can inform policy makers about the extent to which loss of health insurance coverage is a short-term or long-term phenomenon and whether proposed public policies, such as mandated employer health insurance ben- efits, are effectively targeted at the problem. Using data from the 1984 SIPP panel for adults aged 18 and over, Swartz and McBride (l990:Table 1) estimated that one-half of periods with- out health insurance lasted less than 5 months and two-thirds lasted less than 9 months. However, 25 percent lasted longer than 1 year, and 15 percent lasted more than 2 years. McBride and Swartz (1990:26) found that people with longer uninsured periods (lasting 9 months or more) were more likely to be unemployed or out of the labor force, have low monthly family incomes, and work in a service occupation, in comparison with people with shorter spells. Moffitt and Wolfe (1990) found significant relationships between ex
INTRODUCTION 19 pected health care benefits and the work-or-welfare participation decisions of low-income female-headed families in the 1984 SIPP panel. An index of the expected value of Medicaid benefits was negatively related to the likeli- hood of employment, while an index of the expected value of private health insurance benefits showed a strong positive association. Behavioral Dynamics SIPP has made important contributions to social science research, particu- larly to increased understanding of the short-term dynamics of- individual and family behaviors. A few examples of studies in this area include: · Speare, Avery, and Goldscheider (1990), who examined the interrela- tionship of leaving home and other characteristics in the 1984 SIPP panel: they found that young women were more likely to leave their parents' home than young men, that young men who had left were more likely to return, and that the parents' income had a negative association and the young person's employment, income, and education had a positive association with nest leaving; · Koo and Gogan (1988), who documented the extensive amount of change experienced by households in their membership over a 9-month period, using data from the 1979 research panel of the Income Survey De- velopment Program (the predecessor to SIPP); and · Fitzgerald (1991), who explored marriage prospects and the duration of periods of welfare in the 1984 SIPP panel, finding that spouse availabil- ity (as measured by the ratio of employed single men to all single men) exhibited a positive association with the likelihood that a woman would exit a spell of welfare. Researchers have also developed and refined analytical methods and con- cepts through their use of SIPP data. For example, a study by Hagstrom (1991) of the work effort of husbands and wives in relation to their decision to participate in the food stamp program involved the specification of a complex three-way model of choices. Overall, SIPP has provided the grist for a wide range of policy analysis and research studies. The number of papers and articles based on SIPP data increased appreciably from 1984 to 1990 (see Appendix A). Topics covered in these studies include: income and poverty, jobs and work-welfare deci- sions, program participation, the elderly, family change and living arrange- ments, assets and wealth, child care and children, disability, health care and health insurance, race and ethnicity, long-term care, migration, education, veterans, and the rural population.
20 THE SURVEY OF INCOME AND PROGRAM PARTICIPATION Problems Clearly, SIPP is an important survey for many areas of public policy and research interest. However, there are problems in SIPP problems that have kept the survey from proving as useful as it could have been in the past and that, if not adequately addressed, could handicap its usefulness in the future. SIPP has one of the most extensive programs for data quality research and improvement of any federal survey. On many dimensions of data qual- ity, SIPP has registered signal improvements over the March CPS income supplement.3 However, weaknesses many of which SIPP shares with other surveys remain, including: incomplete coverage of the population, par- ticularly young minority men; high nonresponse rates for some types of income and assets; timing errors in reporting receipt of benefits from pro- grams, along with errors due to confusion among program names; and loss of sample cases (i.e., attrition or dropping out from a panel after the first interview), particularly among low-income people, minorities, movers, rent- ers, and single young adults. The SIPP design has achieved success in generating detailed data for analyzing the intrayear dynamics of income and program participation. However, some aspects of the design that had broad acceptance at the outset have not worked well or are now widely seen as limiting the usefulness of the survey for important kinds of policy analysis. For example, the introduction of new panels every year, when coupled with content changes, has contributed to delays in data processing, with the result that few analyses have been able to benefit from combining panels. The length of each panel-32 months- limits the ability of the survey to provide information on such increasingly important policy concerns as welfare dependency over the longer term. Also, the survey lacks information for people who become institutionalized and, in many instances, for children who move to another household. Of course, the grave compromises to the original design necessitated by the imposition of budget cuts on the Census Bureau namely the reductions in sample size and number of interviews for most panels fielded to date have materially affected the usefulness of the information. Along with data quality and design limitations, users have been troubled by problems with the data products from SIPP. There have been successes- for example, the useful series of publications from the topical modules but there have also been significant failures, including: long lags in releasing 3See Citro (1991); Jabine, King, and Petroni (1990); Singh, Weidman, and Shapiro (1988); and Vaughan (1988) for comparative analyses of the quality of SIPP data.
INTRODUCTION 21 microdata files;4 inadequate documentation and user support services; a period of several years when no publications were issued from the core data on income and program participation; and limitations in the data files and reports that provide longitudinal measures from SIPP. Over the last few years, the Census Bureau has worked hard, and with appreciable success, to alleviate such problems as delays in producing data products and the lack of a publication series for the core information. How- ever, these improvements have come at a price that reduces the survey's flex- ibility namely, the imposition of a freeze on the content of the core question- naire. SIPP is certainly not unique among federal surveys in experiencing prob- lems, particularly given that it has been in a start-up phase. Moreover, it has achieved many successes and served many important policy and research needs. Yet the Census Bureau faces an especially heavy burden in striving to remain on top of SIPP and to find cost-effective ways to improve it in the future. SIPP is indeed unique in the following respect: it is arguably the most complex continuing household survey considering the range and detail of questionnaire items (many of them on complicated and sensitive topics), the number and frequency of interviews and introduction of new panels, and the large sample size-that the federal government has ever fielded. Although it is certainly possible to simplify some aspects of SIPP so as to reduce the data collection and processing burden (and this report discusses appropriate ways to accomplish this), there is only so much to be gained in this direc- tion. SIPP will remain inherently complex, given the complexity of the real-world behavior that it is trying to measure. As such, SIPP requires the highest level of quality in every component of its operation. There is little room for mistakes, particularly in implementing innovations, given that SIPP's size and complexity will rapidly compound any problems that occur. SIPP is also unique from the Census Bureau's perspective in that the Bureau both sponsors and operates SIPP, unlike all other household surveys, for which it collects data on behalf of another statistical or policy analysis agency. There is no "Center for Income and Program Statistics" that is the counterpart of the Bureau of Labor Statistics, the National Center for Edu- cation Statistics, or the National Center for Health Statistics. Hence, the Census Bureau must cope with a much broader range of issues and bring to bear a much broader range of expertise involving data content and analy- sis as well as collection and processing than for any of its other household surveys. 4Most analyses conducted to date have used the original 1984 SIPP panel, both because it was the only panel available for a period of time and because of its large sample size (the largest of any panel until 1990).
22 THE SURVEY OF INCOME AND PROGRAM PARTICIPATION LOOKING TO THE FUTURE Roles for SIPP In considering the environment in which SIPP will operate in the future, we have identified both challenges and opportunities for the survey and the Census Bureau. SIPP, with its focus on income and program participation and its longitudinal design, clearly has the opportunity to make even more important contributions to policy analysis and research in the future than it has to date. Public policy concern with questions such as the following is growing and likely to continue to grow as the United States enters a period of difficult economic and social adjustment: · To what extent has the distribution of income and poverty become less equal and what will be the future trends in inequality of well-being? . Do welfare programs lead to long-term dependency, and what can be done to reduce dependency? · What can be done to help the nation's children, many of whom, particularly in single-parent families, appear to face increasingly bleak prospects? · As the population ages, how severe will be the problems of long- term care for the elderly, particularly for the "sandwich generation" that must care for parents as well as children? SIPP should be well positioned to shed light on these and related issues, particularly if its design is modified in some respects. At the same time, social and economic changes pose challenges for cost-effective data collec- tion in SIPP (and other panel surveys).s Increasingly, fewer and fewer households in the United States are "survey fnendly": that is, fewer house- holds contain families, all of whose members live together and, if they move, stay together and are easy to trace. More and more common are situations in which one spouse becomes difficult to trace after a divorce; children of divorced parents shuttle back and forth between parents; and the relevant economic unit for understanding a child's financial circumstances includes the custodial parent, the noncustodial parent, the new spouse of one or both parents, and the child's grandparents. Also increasingly com- mon are households in which several unrelated people reside and who share some, all, or none of the living expenses. Low-income, inner-city areas r.ontin~,f~ to node. cliff~lt nrnblems of data collection, particularly to obtain SSee Bianchi (1990) for a review of changing patterns of family composition and other socioeconomic trends that involve children, many of which have implications for survey data collection. See Fein (1989) for a review of problems in obtaining complete coverage of young minority men and other population groups in censuses and surveys; see also Shaw and Guthrie (1992).
INTRODUCTION 23 complete coverage of young men, many of whom "float" among several households maintained by their relatives or friends. More and more people are at risk of entering institutions, including long-term care facilities and prisons: understanding the economic situation of their families requires information on the institutionalized member. Whatever the family struc- ture, sources of economic support whether in the form of new or modified government programs, new financial instruments, or new fringe benefits- continue to proliferate, and they present problems for accurate data collec- tion. Recent developments in questionnaire design research, survey collec- tion and processing technology, and longitudinal analysis methods afford opportunities for improving the timeliness and quality of data from SIPP (and other panel surveys). Better understanding of how respondents answer questions-obtained through cognitive laboratory expenments, focus groups, and related methods-can lead to improved question wording and, conse- quently, more accurate responses obtained with less burden. Computer- assisted personal and telephone interviewing (CAPI and CATI) can facili- tate the collection of high-quality data on complex topics in a manner that minimizes the requirements for subsequent coding or editing and maximizes the capacity to change the questionnaire as needed. The use of modern database management systems can support an integrated approach to data processing that makes it easier to link data for families and individuals across interview waves and to improve data quality (e.g., by using all avail- able information from prior waves to supply values for missing data). Im- proved methods for analysis of behavioral changes over time (e.g., periods of program participation and poverty), which take account of incomplete observations and shifting family characteristics, can produce useful longitu- dinal statistics for reports. At the same time, these methods and technolo- gies pose substantial implementation challenges, particularly in the context of an ongoing survey program of the size and complexity of SIPP. The Panel's Report We begin our detailed assessment with two chapters that focus on the goals for SIPP. No survey program, no matter how large, can or should be all things to all people; hence, it is critical to set out clearly the main goals for the survey as a necessary precursor to considering changes in content, de- sign, or other features. In Chapter 2 we review the development of SIPP and its goals. We summarize the conclusions reached by the Committee on National Statistics from a preliminary assessment of SIPP in 1989; we summarize the views of current users about goals for SIPP; and we present our conclusions and recommendation about appropriate future goals for SIPP in light of policy
24 THE SURVEY OF INCOME AND PROGMM PARTICIPATION and research needs and budget and operational constraints. We reaffirm the CNSTAT recommendation that SIPP concentrate on improving data on in- come and Program participation, with a particular focus on the population ~~~~~~ ~~- r--o~ -- I- rat that is economically at risk. In Chapter 3 we identify priority areas for improvement within each of SIPP's goals and address their broad implications for the content of the survey. We also consider the relationship of SIPP to the March CPS and to administrative records from the perspective of developing improved income statistics. In Chapter 4 we consider the basic design of SIPP. We review a range of design parameters including panel length, recall length of the interview, frequency with which new panels are introduced, sample size and design- that have important implications for the quality and utility of the data and for ease of survey operation. We discuss the pros and cons of alternative designs and present the panel's preferred design, which includes extending the length of each panel to 4 years and having new panels introduced every other year rather than annually (to contain costs and reduce processing burden). Several of the changes that we recommend to the survey's design and content should greatly improve SIPP's ability to provide needed infor- mation for addressing the increasingly important issue of welfare depen- dency. These changes include lengthening each SIPP panel; keeping open the option of further extending panel length to obtain additional observa- tions for population subgroups of policy interest; and improving the retro- spective information that is collected on respondents' previous program participation and family background. Next, in Chapter 5 we consider operational alternatives for SIPP data collection and processing, including the use of computer-assisted interview- ing and database management systems. We conclude that major improve- ments are needed in data processing at the Census Bureau to enable SIPP to run efficiently and at the same time have a capability for flexible response to changing circumstances. In Chapter 6 we change focus from issues of data input to those of data output. Widespread dissemination of the information from SIPP through regular publication series and well-documented, timely microdata files is essential to the cost-effectiveness of the survey. We consider the dimen- sions of the publication program and other products and services that should be provided to users of the SIPP data. We pay particular attention to the issues involved in appropriate publication and analysis of the rich longitudi- nal data in SIPP. In Chapter 7 we consider priority areas for SIPP methodological re- search and evaluation, including research on the potential of new question- naire formats to improve data quality. Such evaluation and research studies are needed to supply important information to current users, to inform the
INTRODUCTION 25 upcoming redesign of SIPP, and to provide the basis for the next major reassessment and possible redesign of the survey, which should occur no later than 2005. Finally, in Chapter 8 we consider issues related to the oversight, coordi- nation, and management of SIPP. We note that the Census Bureau has a number of means of obtaining outside input to the survey program, and we discuss ways in which the SIPP advisory mechanisms could be made even more effective. The preliminary evaluation of SIPP by the Committee on National Sta- tistics (1989:75) recommended that our panel review the management of SIPP within the Census Bureau. An effective management structure is a key component to the success of any survey and is particularly critical for SIPP because of the inherent complexity of the program and the fact that the Census Bureau has a greater range of responsibilities for SIPP than it does for its other household surveys. Moreover, the many changes that are proposed as part of the upcoming redesign of SIPP-including changes in content, survey and sample design, data collection and processing, and data products- will place especially heavy burdens on management over the next few years. We propose ways to strengthen the management of SIPP that we believe are vitally important to the smooth implementation of the redesign and, more broadly, to the successful operation of the Census Bureau's income statistics program in future years.