|
Items in cart [0] |
Questions? Call 888-624-8373 |
PAPERBACK PDF BOOK Free Resources |
||||||||||||||||
|
||||||||||||||||
|
|
|||||||||||||||
| ||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||
| Copyright © 2009. National Academy of Sciences. All rights reserved. Terms of Use and Privacy Statement |
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 1
Summary
The Survey of Income and Program Participation (SIPP) is a major continu-
ing household survey, conducted by the Bureau of the Census, on people's
economic well-being and their receipt of assistance from a wide range of
government programs. Beginning in 1983, new samples of households have
been selected for the survey every year. The adults in these households
become panel members who are interviewed at 4-month intervals over a 32-
month period. The questionnaire for each interview wave includes a core
questionnaire about income, employment, and program participation and
one or more supplemental modules on related topics.
Funding cutbacks hindered the development of this highly complex sur-
vey in its early years, but SIPP is now clearly established as an important
source of information for federal policy making and social science research.
The survey has a growing community of users in federal agencies, academic
institutions, and other organizations. Analysts have used the data for new
knowledge about such topics as part-year poverty and program participa-
tion, multiple program participation, the effect of asset holdings on program
eligibility and poverty, patterns of health insurance coverage, and the short-
term behavioral dynamics of individuals and families. Despite its suc-
cesses, however, SIPP has experienced problems that have kept it from
being as useful as it could have been in the past and that, if not adequately
addressed, could affect its usefulness in the future.
After nearly 9 years of operation, the Census Bureau has under way a
comprehensive reassessment of SIPP A new sample design, using informa
OCR for page 2
THE SURVEY OF INCOME AND PROGRAM PARTICIPATION
lion from the 1990 census, will be implemented for SIPP beginning with the
1995 panel. At that time, the Census Bureau will make other changes to
enhance the utility and cost-effectiveness of the program.
As part of the evaluation and redesign effort, the Bureau asked the
Committee on National Statistics (CNSTAT) to convene a study panel to
conduct an independent review of SIPP. The panel drew on the work of an
interim assessment of SIPP, performed by CNSTAT in 1989, which focused
on federal agency uses of the data; consulted widely with users both inside
and outside federal agencies; and conducted its own assessments of SIPP.
Our report covers the following aspects of SIPP: the survey's goals and
their implications for content and the relationship of SIPP to other surveys
and administrative record data sources; survey and sample design; data col-
leciion and processing; publications and other data products; analytical methods
for using the complex longitudinal data from SIPP; methodological research
and evaluation needed to plan and evaluate the SIPP redesign; and the
management and oversight of the SIPP program. The first part of the Sum-
mary presents our overall conclusions and recommendations on those as-
pects; the second part presents all the panel's detailed recommendations.
GOALS
Over the course of SIPP's history, many people involved with the survey
have wanted to expand it in one or another way to provide detailed informa-
tion for their fields of concern. To satisfy these varied interests, SIPP
would need to be an all-encompassing survey in the area of social welfare
policy. We believe that SIPP cannot and should not be viewed as such.
Rather, it is essential for the cost-effective operation of the program that it
focus on a core set of major goals.
The two primary goals for SIPP should be, as its name implies, to
provide detailed information on the distribution of income and other eco-
nomic resources and on eligibility for and participation in government as-
sistance programs. Within these two goals, the survey should pay most
attention to improving information for people who are economically at risk:
poor people and near-poor and middle-income people who, if they experi-
enced an event such as loss of a spouse or parent or job, would be at risk of
economic deprivation and in need of assistance. As an added but secondary
goal, SIPP should continue and strengthen its capability to respond to cur-
rent policy needs for data in topical areas that are related to its core sub-
jects, such as support for children and use of health care.
We have identified several ways in which the data from SIPP should be
enhanced to better serve the two primary goals. In the area of income-
related measures, the priorities for improving information are: enhancing
the quality of those measures that are relevant to program eligibility and
OCR for page 3
SUMMARY
3
participation; developing measures of taxes and after-tax income; and de-
veloping measures that take account of in-kind benefits and that reflect
changing family characteristics. In the area of program-related measures,
the priorities for improving information are obtaining more complete and
frequent data with which to determine program eligibility and developing
adequate measures of periods or spells of both eligibility and participation.
Finally, it is important that SIPP keep up to date with respect to new and
changing sources of income and types of programs.
How SIPP best achieves its goals raises the issue of its relationship to
other surveys and administrative records. SIPP was developed-to provide
added information and remedy deficiencies in the March income supple-
ment to the Current Population Survey (CPS), which for decades has been
the primary source of the nation's income and poverty statistics. SIPP's
design enables it to collect more detailed information than is possible in the
March CPS (e.g., intrayear and cross-year in addition to annual measures).
Also, SIPP has achieved improvements in data quality (e.g., less item nonresponse)
that would be difficult to match in the March CPS. However, to date, such
problems as small sample size and lack of timeliness have limited SIPP's
ability to provide regular income statistics. Changes that are implemented
as part of the redesign should alleviate these problems. We urge the Census
Bureau to set a target date by which time SIPP will be able to serve as the
primary source of annual and other measures of income and poverty. (Some
information on income should of course continue to be collected in the CPS
for use in analyses of the labor force data that are the prime focus of that
survey.)
The use of administrative records (e.g., program case records and tax
returns) can be helpful to SIPP in many ways. These records can provide
additional information on sample persons, furnish the means to obtain addi-
tional samples for groups of policy interest, and provide the basis for evalu-
ating and improving the quality of the survey responses. However, the use
of administrative records poses technical and operational problems that will
need to be addressed. Also, some uses raise concerns about the confidenti-
ality of the information, which must be adequately protected.
SURVEY DESIGN
We evaluated several alternative designs for SIPP-varying in panel length,
frequency of introduction of new panels, length of recall period, and total
sample size all of which were constrained to have the same number of
annual interviews as provided for by the current SIPP budget. Each design
has its own strengths and weaknesses relative to the current design and the
other alternatives.
We conclude that the current design is not optimal to the needs of SIPP
OCR for page 4
4
THE SURVEY OF INCOME AND PROGRAM PARTICIPATION
users for timely, high-quality, and relevant data for cross-sectional and lon-
gitudinal applications. We recommend that the length of each SIPP panel
be increased from 32 to 48 months, a change that will make SIPP more
suitable for analysis of spells of poverty and program participation and the
dynamics of poverty and program entrances and exits. We also recommend
that panels be introduced every 2 years rather than annually, a change that
should reduce the operational complexity of the survey and facilitate timely
data processing without compromising data quality. We recommend retain-
ing the 4-month recall length, which means that each panel under the new
design will have 12 interviews. However, we urge the Census Bureau to
conduct research on 6-month versus 4-month recall periods, since an in-
crease in recall length-if there were no adverse effects on the quality of
the intrayear information- would permit longer and larger panels. Under
the design that we propose, the total sample size of each panel would in-
crease from 20,000 to 27,000 households, and it could increase further if
savings are achieved through the introduction of new data collection and
processing technology.
DATA COLLECTION AND PROCESSING
The Census Bureau is planning to convert SIPP data collection from paper-
and-pencil methods to computer-assisted personal interviewing (CAPI). The
Bureau is also planning to install a modern relational or other database
management system for processing SIPP. We support both of these initia-
tives, which should lead to improvements in data quality and timeliness of
data products. We encourage the Census Bureau to identify CAPI and
database management systems that incorporate all of the features necessary
to handle the complex SIPP data. For example, the systems chosen should
permit data from a previous interview to be used in the current interview
and in the imputation of missing items. The data processing performed by
the database management system should be fully and carefully integrated
with the SIPP CAPI system. Because of the necessity for thorough testing
of CAPI and database management technology in conjunction with other
proposed changes in SIPP, we urge the Census Bureau to consider fielding a
somewhat smaller dress rehearsal panel in 1995 as a means of working out
operational problems. Under this scheme, full implementation of the new
design would occur in 1996.
DATA PRODUCTS AND THEIR USE
To earn a high return on the investment in a survey as rich and complex as
SIPP, it is critical that the responsible agency have an active dissemination
OCR for page 5
SUMMARY
s
program that includes timely and accessible published reports, computer-
readable data products, and associated explanatory materials. In the past,
there have been weaknesses in the data dissemination effort for SIPP, in-
cluding the absence of a regular publication series for the core content on
income and program participation.
We support the Census Bureau's plans to publish regular, comprehen-
sive descriptive reports from the SIPP core data. These reports should
include longitudinal statistics on the dynamics of income, poverty, and pro-
gram status and cross-sectional measures and should be issued on a frequent
schedule. In addition, we encourage the Bureau to establish-a research
report series to include in-depth analytical studies of topics related to in-
come and programs. The Bureau should also continue publications from the
topical modules and should produce reports from SIPP on the dynamics of
major demographic and employment transitions.
Development of statistics from the SIPP core data gives rise to difficult
conceptual and measurement issues, particularly in the case of longitudinal
measures. These issues include developing statistics that appropriately re-
late to policy needs, specifying analysis units (e.g., household or person),
characterizing change in contextual variables (e.g., marital status changes),
constructing equivalence scales (i.e., income measures that are adjusted for
household characteristics), measuring the duration of spells of poverty and
program participation, and treating missing interviews. Because the issues
involved in developing appropriate statistics from SIPP are so complex, we
urge the Census Bureau to ensure that the analysis staff are able not only to
prepare publications, but also to undertake an ongoing research program
and keep up to date with regard to relevant analytical techniques and policy
concerns.
In addition to published reports, the Census Bureau should develop
improved microdata products from SIPP to support policy analysis and so-
cial science research. Priorities for improvement include moving toward a
goal of releasing core data files within 6 months after the end of data
collection and producing calendar-year files that combine two panels. Im-
provements in documentation and related user information services are also
necessary and important for SIPP.
METHODOLOGICAL RESEARCH AND EVALUATION
One strength of SIPP, going back to the days of its predecessor, the Income
Survey Development Program, has been the extent of research on data qual-
ity and ways to improve both the quality and the efficiency of the survey.
SIPP should continue that focus on methodological research and evaluation.
In the immediate future, the Census Bureau will need the results of method-
ological studies to develop the details of the proposed redesign. Research
OCR for page 6
6
THE SURVEY OF INCOME AND PROGRAM PARTICIPATION
should be conducted on proposed questionnaire changes, using small-scale
administrative record-check studies and taking full advantage of the find-
ings of current experiments with cognitively based interviewing procedures
and questionnaire formats. Research should also be conducted on other
aspects of the redesign, including alternative methods of oversampling the
low-income population and the best combination of personal and telephone
interviews in an environment of computer-assisted interviewing. Research
should be carried out as well on issues of estimation and data use in light of
the redesign, including ways to improve weighting and imputation. In the
longer term, the Census Bureau will need information on the impact of the
redesign to identify successes and to respond to problems. Finally, it will
be important for the Bureau to monitor error levels in present and future
SIPP panels on a continuing basis and to provide regularly updated infor-
mation to users.
MANAGEMENT AND OVERSIGHT
An effective management structure is a key component to a survey's suc-
cess. In the case of SIPP, the Census Bureau has leadership, analysis, and
dissemination responsibilities that are not typical of the household surveys
that it conducts for other agencies. These responsibilities, together with
SIPP's complexity, argue for a management structure for SIPP that is stron-
ger and more focused than the Bureau's structure for its other household
surveys. We recommend that the Bureau establish a high-level position of
project director with full responsibility for its income statistics program,
including both SIPP and the March CPS income supplement. The project
director needs to combine relevant substantive expertise with strong survey
management skills and have a sufficiently large staff to guide the program,
prepare reports and analyses, and address analytical concepts and methods.
Finally, the Census Bureau has made commendable efforts to seek out-
side advice about SIPP, which is essential to keeping the survey oriented to
user needs and up to date with improvements in survey design and analysis.
We urge the Bureau to strengthen its advisory mechanisms, particularly in
seeking input from users in the academic community, and in obtaining ex-
pert advice on analytical methods that is comparable to the advice it cur-
rently obtains on survey methodology and data products.
RECOMMENDATIONS
The text of all of the panel's recommendations are presented here, keyed to
the chapters in which they appear in the report.
OCR for page 7
SUMMARY
Goals for SIPP
Recommendation 2-1: The two primary goals of SIPP should be to provide
improved information on the distribution of income and other economic
resources for people and families and on eligibility for and participation in
government assistance programs. Within these two goals, most attention
should be paid to improving the information for people who are economi-
cally at risk. A third important but subordinate goal is for SIPP to have a
capability to respond to current policy needs for data in topical areas that
are related to the core subjects of SIPP.
Achieving SIPP's Goals
Recommendation 3-1: Priorities for improved income and related measures
from SIPP should include:
· enhancing the quality of income and related measures that are rel-
evant to program eligibility and participation;
· developing measures of taxes and after-tax income;
· valuing (or otherwise taking account of) in-kind benefits; and
· constructing income and related measures that take account of fam-
ily characteristics and changes in families over time.
SIPP should also keep up to date with respect to new sources of income
and other economic resources.
Recommendation 3-2: The measurement of asset-related information in
SIPP should be reassessed in light of SIPP's focus on programs and the
economically at-risk population. The collection of asset items should be
redesigned and simplified, if possible, to reduce respondent burden and
improve the quality of the data needed to serve SIPP's primary goals.
Recommendation 3-3: SIPP should develop, on an experimental basis,
selected measures of economic security against risk, such as access to credit.
Recommendation 3-4: Priorities for improved measures of program par-
ticipation and eligibility from SIPP should include improving the range and
frequency of information needed to determine eligibility for major assis-
tance programs and providing adequate measures of spells of both eligibil-
ity and participation. SIPP should also keep up to date with respect to
newly important programs and program changes.
Recommendation 3-5: The topical module component of SIPP should con-
tinue and be strengthened by:
· obtaining input from both government agencies and the social sci-
ence research community about topics related to SIPP's core goals to con-
sider for modules;
OCR for page 8
8
THE SURVEY OF INCOME AND PROGRAM PARTICIPATION
· streamlining the content development process so that timely infor-
mation can be collected on emerging policy and research issues; and
.
using some topical modules as a means for the Census Bureau's
analysis staff to conduct research on expanded and alternative measures of
income and programs.
Recommendation 3-6: SIPP should become, over time, the primary source
of the nation's income statistics in place of the March CPS income supple-
ment. SIPP should receive priority for major investments to develop im-
proved income measures. As there will necessarily be a transition period
during which SIPP and CPS income statistics are both published, every
effort should be made to increase user understanding of differences and
similarities and to effect incremental improvements as appropriate in both
surveys.
Survey Design
Recommendation 4-~: SIPP should be redesigned as an ongoing panel
survey in which each panel lasts for 4 years and has 12 4-month interviews,
with a new panel introduced every 2 years. The sample size for each panel
should be increased over that for the current design.
Recommendation 4-2: The Census Bureau should conduct research on the
data quality effects of 6-month versus 4-month reference periods in SIPP, so
that information is available to consider other possible design changes at a
later date, including the possibility of further extending the length of SIPP
panels beyond 4 years.
Recommendation 4-3: The use of a monthly rotation group structure should
be retained for SIPP. The Census Bureau should consider cost-effective
means to obtain the core data for the last calendar year of each panel that
will otherwise be missing for some months for some groups. The Census
Bureau should also investigate ways to minimize the loss of mover house-
holds that may result in part from the closeout of follow-up at the end of
each month.
Recommendation 4-4: The Census Bureau should investigate alternative
methods of oversampling the low-income population in SIPP, including the
use of screening interviews as a possible complement to or substitute for an
approach based on using information from the 1990 census.
Recommendation 4-5: The Census Bureau should take steps to ensure that
it will be possible to extend the length of SIPP panels for selected sub-
groups of interest or to follow them up at a later date, should such options
be desired to obtain additional sample size and longitudinal information
OCR for page 9
SUMMARY
9
Recommendation 4-6: SIPP panels should treat all children who reside in
interviewed households at the first wave and also children born during the
course of a panel to original sample mothers as original sample members,
who are followed if they move into households without an original sample
adult. SIPP panels should also continue to follow and collect data for both
original sample adults and children if they move into institutions.
Data Collection and Processing
Recommendation 5-1: We strongly support the Census Bureau's goal to
convert SIPP to computer-assisted personal interviewing (CAPI). Since the
Bureau's current CAPI software system (QUISC) does not appear to meet
the data collection requirements for SIPP, the Census Bureau should give
high priority to investigating other available CAPI systems and determine
the most appropriate system for SIPP.
Recommendation 5-2: We strongly support the Census Bureau's plans to
adopt a new database management system for SIPP. The Census Bureau
should use the capabilities of a DBMS to the fullest in seeking to make
improvements in all aspects of processing, analyzing, and documenting the
data from SIPP. The processing performed by the database management
system should be fully integrated with the SIPP CAPI system.
Recommendation 5-3: In view of the major advances that continue to
occur in computing hardware and software, the Census Bureau should de-
vote significant resources to continued education and training of its data
processing staff. In particular, the SIPP processing staff should take advan-
tage of the experience of other data processing facilities outside the Census
Bureau that deal with longitudinal surveys.
Recommendation 5-4: The Census Bureau should make every effort to
ensure smooth implementation of CAPI and an improved database manage-
ment system for SIPP under the new design of 4-year panels introduced
every 2 years. One option that the Census Bureau should consider is to
field a large-scale dress rehearsal panel in 1995 as a way to work out any
operational problems. Under this scheme, full implementation of the SIPP
survey redesign would occur in 1996.
Data Products and Their Use
Recommendation 6-1: The Census Bureau should move forward with its
plans for regular, comprehensive series of descriptive reports on income,
programs, and related topics from the core data in SIPP. Longitudinal
statistics (e.g., on the dynamics and correlates of transitions in income,
OCR for page 10
10
THE SURVEY OF INCOME AND PROGRAM PARTICIPATION
poverty, and program status) should be published; cross-sectional statistics
should also be issued on a frequent schedule.
The Census Bureau should also establish a research report series to
include in-depth analytical and methodological studies of special topics re-
lated to income and program participation. Data sources for these studies
could include in addition to SIPP the March CPS income supplement
and other surveys and administrative records.
Finally, the Census Bureau should continue publications from the topi-
cal modules and also establish a regular series of summary and in-depth
reports from SIPP on the dynamics and correlates of major demographic
and employment transitions (e.g., marriage, retirement).
Recommendation 6-2: The Census Bureau should ensure that its analysis
staff, in addition to preparing the regular publications from SIPP, are able to
undertake an ongoing program of research and development into effective
means of analyzing and presenting SIPP statistics and are able to stay well-
versed in relevant policy issues arid analytical techniques.
Recommendation 6-3: The Census Bureau should continue to develop im-
proved microdata products from SIPP to support policy analysis and social
science research. Priority improvements include:
· moving toward a goal of releasing core data files within 6 months
after the end of data collection;
.
producing calendar-year files that combine panels, in addition to wave
and panel files;
.
determining, in consultation with users, changes and additions to the
file contents that would assist their analyses; and
· developing additional ways of delivering SIPP microdata products to
users, such as by means of high-storage capacity compact disks (CD-ROM)
and an improved on-line data extraction system.
Recommendation 6-4: The Census Bureau should work to improve docu-
mentation and related user information services for SIPP. Priority improve-
ments include:
· making use of CAPI and database management system technology to
fully integrate documentation (including frequency counts for variables)
and data;
· developing documentation for recoded variables and the types of
imputations that are performed for missing data in SIPP;
.
developing means to update key explanatory documents, such as the
SIPP Users' Guide, on a more frequent basis;
· restoring and expanding information and training programs, such as
training sessions, working papers, and compilations of professional society
presentations; and
OCR for page 11
SUMMARY 1 1
· maintaining effective channels of communication for users to feed
back problems and suggestions and learn of the Bureau's response, and for
users to be informed of new developments in the survey and its data prod-
ucts.
Methodological Research and Evaluation
Recommendation 7-1: The Census Bureau should support methodological
research and evaluation for SIPP leading up to and following the survey
redesign. The research program should include the following components:
· research to improve the format and wording of the questionnaire,
making use of record-check studies and, to the extent possible, of findings
from the current program of cognitively based questionnaire expenmenta-
tion;
.
research targeted to other aspects of the current redesign (and to
possible design changes later on), including the length of the recall period,
screening techniques to obtain larger sample sizes for subgroups of interest,
and data collection modes (the best combination of computer-assisted per-
sonal and telephone interviewing and the possible role of centralized tele-
phoning);
· research on issues of estimation and data use, taking into account the
features of the redesign and including ways to improve cross-sectional and
longitudinal weights, imputation procedures, and population coverage;
· research to evaluate the success of major elements of the redesign
(e.g., the attrition effects of longer panels); and
· a quick-response capability to address unanticipated problems with
the implementation of the redesign.
Recommendation 7-2: The Census Bureau should undertake continuous
monitoring of error levels in present and future SIPP panels and regularly
provide information on errors to users, in periodic updates of the SIPP
Quality Profile and other publications.
Recommendation 7-3: We strongly support the Census Bureau's program of
cognitively based research and experimentation with the SIPP questionnaire,
which could contribute to questionnaire improvements for the current redesign
and perhaps, in the future, to a major revision of the questionnaire and inter-
viewing procedures. The Bureau should subject the cognitive work to rigor-
ous evaluation, including record-check studies to evaluate data quality.
Management and Oversight
Recommendation 8-1: To be as effective as possible in carrying out its
responsibilities to produce timely, comprehensive, relevant, high-quality,
OCR for page 12
2
THE SURVEY OF INCOME AND PROGRAM PARTICIPATION
and analytically appropriate statistics on income and program participation,
the Census Bureau should establish a senior-level position of project direc-
tor for the Bureau's income surveys, SIPP and the March CPS income
supplement. That position should include full management and budgetary
authority for the income statistics program and sufficient resources to ob-
tain the level of analysis staff that is needed to provide substantive guidance
to the program, prepare reports, conduct analyses, and evaluate analytical
concepts and methods. The person who fills this position should have
recognized substantive expertise in topics related to income, poverty, and
assistance programs, combined with strong survey management skills.
Recommendation 8-2: We support the Census Bureau's efforts to obtain
outside advice about the SIPP program and encourage the Bureau to further
strengthen its advisory mechanisms. The Bureau should regularly seek
advice about the content, overall design, and goals of SIPP from federal
agency users and from other users, including academic researchers. The
Bureau should also regularly seek advice about technical matters from ex-
perts in the field. Working groups should be formed or continued in three
main areas: (1) survey methods and evaluation of ways to improve data
quality; (2) conceptual and analytical issues in the development of appropri-
ate income and program statistics from complex longitudinal data; and (3)
microdata products, documentation, and means of data access.
Representative terms from entire chapter:
program participation
