4

The Questionnaire:
Instrumentation and Flow

The 2014 Survey of Income and Program Participation (SIPP) uses an electronic questionnaire in which both the content and instrumentation have been redesigned. This chapter provides a fairly detailed description of the new instrumentation, the questionnaire flow, and the handling of preloaded data in that process.

INSTRUMENTATION

With the 2014 panel, the SIPP electronic instrument (questionnaire) was converted from an obsolete survey programming language to the Windows-based Blaise software, which is widely used within survey organizations for survey instrumentation. Many questions, instructions, and answer spaces are presented to the interviewer on a “standard” Blaise screen in a sequential fashion consistent with the 2008 SIPP questionnaire instrument. Other questions appear on a newly designed event history calendar (EHC) screen.¹ The EHC screens are created by an external module that is inserted into the Blaise-based instrument. Descriptions of both screens appear below.

___________________

¹ The EHC implemented in the SIPP questionnaire differs from those described in the early literature on this topic. First, the EHC is used exclusively as a tool for interviewers and is not used as a visual aid for respondents. Second, the respondent is not asked about “major events” in his life that are then placed in the calendar to “bound” the later substantive questions. The SIPP EHC is used to display, for the interviewer, the collective spells of activities (employment, unemployment, and program participation) reported by the respondent. The interviewer is expected to observe patterns in these spells, looking for gaps and overlaps between activities that are questionable, and then try to resolve these questionable spells with the respondent.

An important point is that only the interviewers see these screens. Respondents do not interact with either the standard screen or the EHC screen because SIPP is conducted through an interviewer, either in person or over the telephone.

Standard Interviewing Screen

Figure 4-1 shows the layout of a typical standard screen. The question appears at the top left-hand corner of the screen. Below the question, in blue font, are instructions for the interviewer. The middle part of the screen is for entry of precoded answers. It shows radio buttons (if any) associated with potential answers. The bottom third of the screen presents answers to previously answered, related questions, and a place to enter free-form answers. The interviewer may also use this space to record precoded answer codes instead of selecting a radio button.

The Event History Calendar Screen

These screens are new to the 2014 SIPP. While a standard screen shows one question at a time sequentially, this alternative type of screen focuses on a calendar that assists the interviewer/respondent in accurately capturing participation spells² for selected programs. For each program, the calendar can display one or more participation spells across the reference year and subsequent months up to the month of the interview. The calendar can also display multiple programs, some of which are interdependent (such as employment and unemployment).

Figure 4-2 shows the layout of a typical EHC screen. Spells reported earlier in the interview are shown as horizontal red bars, with the current questionnaire section highlighted in yellow. Question and answer fields appear on the far right side of the screen, and are restricted to asking whether there was participation in the designated activity or program. There is a separate box for reporting the beginning and ending months of spells.

The EHC offers a potentially significant advantage over the standard questionnaire approach by allowing respondents and interviewers to move back and forth across programs and activities, reporting events that are linked in time (such as the ending of a job spell and the start of a subsequent unemployment insurance spell), rather than being forced to follow the sequential instrumentation. However, the study panel learned that this potential advantage is not maximized because interviewers only infre-

___________________

² By a spell, the report means a continuous block of time, with designated start and end months, during which a household member participated in an activity, program, or institution about which SIPP collects data.

quently use the calendar screens in this way. The interviewers may not be comfortable with using the “nonscripted” interviewing that this process requires, and appear to follow a linear path through the questions in the EHC (as is required for more structured questions outside the EHC). Possible explanations for this pattern include (1) interviewer training in how to use the EHC was insufficient; (2) navigating from an identified EHC spell and then immediately to a series of supplemental Blaise questions, the interconnectedness of spells within and across timelines can easily be overlooked. The report returns to this point in Chapter 7.

QUESTIONNAIRE FLOW

The questionnaire for the reengineered SIPP is long and complex. Questions are grouped by topic, which this report will refer to as “sections.” Some sections are asked only of the designated household respondent, but most sections are asked separately of each current household member who is 15 years of age and older.³

Rosters

The initial section of the instrument presents questions to a designated household respondent, asking that person to verify a prelisted address and to specify tenure. She is also asked to help build two household rosters: the current household roster (those household members living in the household during the interview-month), and a second roster, called the Type 2 roster. This second roster includes those household members who lived in the household sometime during the reference year but who live elsewhere at the time of the interview. The household respondent is asked for basic demographic and relationship information about each individual on either roster. In wave 2 and subsequent waves, any original household member from wave 1 will be followed to his new household and interviewed there.

Type 2 roster members are not interviewed, but a record extension⁴ is

___________________

³ Proxy records are generated for children under 15 years of age and for individuals who were household members at some point during the reference year but do not live in the household at the time of the interview.

⁴ Limited income information is collected for Type 2 roster members and is included on the person file of the interviewed individual who lived with them during the reference year. Information about the Type 2 members is included on each record of persons interviewed with whom the Type 2 individual lived during the reference period. For example, if a mother and child lived with the grandmother (of the child) part of the year (in the selected household) but then moved in with a stepfather before the interview, the Type 2 information on mother and child is attached to the grandmother’s person record. Type 2 members are not followed in subsequent waves.

created for each of them, and the household respondent is asked for an estimate of the annual income for each Type 2 roster member. This approach to capturing information on former household members who are not present at the time of the interview is new to the 2014 SIPP and is a direct response to the longer reference period. In earlier SIPP panels,

• No data were collected on persons who were present for part of wave 1 but not present at the time of the interview for the current wave.
• Except for infants born during the reference period, all persons present at the time of the wave 1 interview were considered to have been present for the entire 4-month reference period.
• In waves 2 and later, SIPP attempted to obtain proxy interviews for all nonoriginal-sample members (15 years and older) who were present for part of the reference period but not present at the time of the interview. Failing this, the data were imputed.
• Original sample members who left the household would have been interviewed at their new households, if possible, with data being collected for the entire reference period. Only limited data would have been collected on these individuals from the households that they left. If the attempt to interview these individuals at their new households was unsuccessful, their data during the time that they were present in the households that they left would have been imputed.

The procedures with respect to original sample members have not changed with the new design. With the earlier SIPP design, the purpose of attempting to collect full information after wave 1 on former household members during the time they lived with original sample members—and imputing the data when it could not be collected—was to provide SIPP data users with a full picture of the households in which original sample members lived in every month they remained part of the sample. With the new design, this goal is sacrificed in view of the difficulty of collecting such information over the new survey’s much longer reference period—plus the months between the end of that reference period and the current wave’s interview. The panel reiterates, however, that the practices regarding original sample members who leave their households have not changed. SIPP continues to follow and, where possible, interview these persons.

Each current household member (15 years and older) is then interviewed across an extensive set of questionnaire sections. The standard screen format is used initially in the interview to ask individual household members about marriage as well as some income screeners to identify lower-income households.

Program Participation

The questionnaire then goes through a series of sections that address participation in various programs or institutions, using a combination of the standard screen format and the EHC screen format. The 2014 SIPP questionnaire is focused on accurately capturing “spells of participation.” The EHC module and its screens are geared to assist the interviewer in visualizing multiple spells across multiple programs and thereby assist in improving the reporting of those spells.

When participation in a program has been established and the “spell” recorded, the SIPP questionnaire asks a number of detailed questions about that participation. For these questions, the instrument shifts back to the standard screen, with each question-and-answer appearing sequentially on separate screens. Box 4-1 shows the sections related to program participation that are incorporated into the EHC module and the types of questions that follow (in the standard format screens) the identification of a spell.

The EHC screen is used for the sections listed in Box 4-1, but space on the calendar is a limiting factor. Thus several sections involving program participation are not displayed as part of the EHC module. Box 4-2 shows the sections dealing with participation that are not included in the EHC screens. These sections use only the standard screens to establish participation, define the spells of participation, and ask questions related to that participation.

Assets and Debts

Another major part of the SIPP questionnaire deals with assets, income from assets, and debts. Only one spouse is asked about jointly owned assets. Box 4-3 shows the sections dealing with these assets and debts. For each asset, the questionnaire asks about ownership and total value of the asset. When appropriate (such as for bonds and money market accounts) the questionnaire asks for income from the asset. The questionnaire also asks for associated debt for such assets as real estate and automobiles.

Multiple sets of questions collect information on real estate and business ownership as well as rental housing. There are a number of questions about an owned primary residence: mortgage principal owed, length of loan, interest rate, association fees, and payments for utilities. For rented residences, the questionnaire asks about rent and utilities. For both owned and rented residences, it asks which household member paid the most toward these expenses. For other real estate, the questionnaire asks about rental income and existing debt. For reported businesses, the questionnaire asks about percentage of ownership, value of the business, and debt of the business.

Lastly, there are questions about other sources of debt. These include retail and credit card accounts, medical bills, and personal loans.

Disability

The section on disability is short. It contains nine questions about whether the respondent has difficulty in performing routine activities in daily life. These include three questions focused on child-specific limitations.

Biological Lineage and Relationship with Children in Household

Several sections are related to children. In one section, the respondent is asked to create a roster of his biological children and biological parents. There is a section of questions on child care for children in the household: its arrangements, costs, and whether the household receives any child care assistance. This section also asks the respondent whether he has had any difficulties in working because of issues with child care. A related section asks about the time that the respondent spends with household children and whether the family eats meals together. A third related section asks questions about the children’s experiences at school, such as homework, special classes, sports, or any disciplinary issues.

Safety and Well-being

The final section of the SIPP questionnaire deals with the safety and well-being of the household. Several questions address potential persistent health/safety issues with the principal residence, such as holes in the floor,

rats, or plumbing that does not work. The section also asks whether the household has had difficulty paying rent or utility bills. It also asks about food insecurity.

HANDLING OF PRELOADED DATA

As with prior SIPP panels, the 2014 redesign makes use of dependent interviewing techniques,⁵ primarily to combat seam effect/bias (Moore, 2008), although it also may reduce respondent burden (U.S. Census Bureau, 2016). With the addition of the EHC format in the redesigned instrument, the format of dependent interviewing in the 2014 SIPP also changed from the format used in the 2008 SIPP.

Preloaded Data Used in Wave 2 and Beyond

The 2014 SIPP wave 1 interviews were conducted in 2014 with a reference year of 2013. During a wave 1 interview, program participation information was collected for both the reference period (calendar year 2013) as well as any subsequent days (within 2014) up to the date of the wave 1 interview. For example, if the wave 1 interview took place on April 1, there would be 3 months of data collected for calendar year 2014 during the wave 1 interview. These 3 months would overlap with the reference period for wave 2, which was calendar year 2014.

Data for this overlap period (consisting of the early months of the wave 2 reference period) were preloaded⁶ from the wave 1 interview into the wave 2 instrument as “spells of participation” (see Figure 4-3). Spells of participation that ended prior to the wave 1 interview were preloaded into the wave 2 instrument as closed spells; spells ongoing at the time of the wave 1 interview were preloaded into the wave 2 instrument as open spells. The interviewers for wave 2 were instructed to largely ignore closed spells. Open spells were either extended and/or closed as part of the wave 2 interview. Additional spells of program participation could also be added that occurred later in the wave 2 reference period. Figure 4-4 shows examples of closed and open spells on an EHC screen.

___________________

⁵ “Dependent interviewing is a method of scripting computer-assisted survey questionnaires, in which information about each respondent known prior to the interview is used to determine question routing and wording. This method of personalizing questionnaires can be used to reduce respondent burden and measurement error. The prior information can be incorporated reactively, for in-interview edit checks, or proactively, to remind respondents of previous answers” (Lavrakas, 2008, p. 186).

⁶ Data from the previous wave is only preloaded with consent from the respondent, which is obtained in wave 1. See the next section, “Respondent Consent to Use Data in Subsequent Interviews.”

For programs that are not collected via the EHC (Box 4-2 lists these programs), more conventional dependent interviewing techniques are used in subsequent waves, for example

The interviewer then proceeds to ask about dates of receipt and amounts received, as usual.

While the use of dependent interviewing can reduce seam effects/bias and respondent burden, the changes are largely in one direction. Only “yes” responses are preloaded; those individuals who were not participating in a program do not have that lack of participation preloaded. The reason for this feature is that analysis of data from previous SIPP panels found that respondents are often more likely to report (i.e., not conceal) their program participation as they become more familiar with the interviewer and the interview process. Thus, SIPP administrators do not want to provide any barrier to a respondent answering questions more honestly in subsequent waves, even if they failed to report program participation in earlier waves.

Respondent Consent to Use Data in Subsequent Interviews

The Census Bureau obtains respondent consent before her data is used in subsequent interview waves, either as feedback (preloaded data) and/or dependent interviewing. The Census Bureau reported that 98.2 percent of 2014 SIPP wave 1 respondents consented to the preloading of their data from wave 1 for use in the wave 2 interview. Similarly, 98.9 percent of wave 2 respondents consented to the preloading of their data from wave 2 to be used in the wave 3 interview. In the case of a proxy response, the consent is set to the answer provided by the individual providing the proxy response, as per his personal interview.

SUMMARY OF QUESTIONNAIRE DESIGN AND FLOW

The 2014 SIPP electronic questionnaire is delivered through an interviewer. It is newly programmed using the Blaise software. The instrument uses both standard interviewing screens (which direct a linear flow through questions and answers) and an EHC screen that displays multiple activities and spells on a single screen. The EHC makes it easier to see gaps and questionable overlaps in spells of various activities, and the interviewer is expected to identify and resolve these problems using unscripted interviewing techniques. The EHC is programmed as a separate module attached to the Blaise software.

After preliminaries, the general flow of the questionnaire begins within the EHC for a particular activity (i.e., employment or participation in a particular program) in order to identify the spell(s) for that activity. Once the spells for a single activity are identified, the flow takes the interviewer and respondent out of the EHC and into a series of detailed questions about the activity, moving from question to question in a linear flow, using structured interviewing from standard interviewing screens. At the end of those questions, the questionnaire goes back into the EHC to identify spells for the next activity.

The EHC screen is limited in size and therefore spell data for about half of the activities are collected outside the EHC. For wave 2 and beyond, some spell data are preloaded (with respondent consent) from the previous wave of data collection.

The purpose of this chapter was to lay out the fundamentals about the questionnaire structure and flow. In later chapters of this report the study panel discusses some related concerns including the system used to record interviews (Chapter 5, “Computer Audio-Recorded Interviews”) does not record within the EHC; and interviewers may not fully utilize the flexibility with the EHC to identify issues with reported spells (Chapter 7, “Using the EHC to Capture Spells”).

This page intentionally left blank.