4
Implications for a New Study
In addition to broad perspectives on the concept of social mobility and a focused consideration of an array of measurement issues, the workshop also gave attention to possible data sources and the design of a new survey. These topics were the subject of both detailed presentations and general discussion.
Possible data sources for a new study of social mobility include followups of existing surveys, linkages to administrative records, and a new stand-alone survey. The different instruments have different strengths and weaknesses and also might be used in combination. John Robert Warren of the Minnesota Population Center at the University of Minnesota reviewed many options in his presentation, “Potential Data Sources for a New Study of Social Mobility in the United States.” Warren began provocatively by presuming that “the implicit sentiment in the room is that we need to do a completely new data collection. My prior is that that would be the last option. It is the most expensive and in many ways least practical. If we can avoid it, it would be nice to avoid.”
To weigh the different possible data sources for a new study of social mobility, Warren first elaborated eight parameters by which to evaluate existing and potential surveys. He then reviewed nine smaller surveys, two large surveys, linkages to administrative records, and the possibility of a new stand-alone survey, assessing costs and benefits along the param-
eters. In closing, he considered several possible ways forward, sparking further discussion of data sources.
Parameters for Evaluating Data Sources
Before considering the advantages and disadvantages of each data source, Warren first elaborated the eight parameters he used in reaching his assessments.
(1) Population coverage and definition. Warren’s first parameter was population coverage. In particular, it would be important to include the incarcerated and other institutionalized populations who are generally omitted in surveys. Warren acknowledged that this lacuna “may or may not be a problem with respect to biasing estimates and social mobility, but it is certainly something that is stratified by social and economic characteristics.” With the tremendous increase in incarceration rates, this has also changed over time, particularly since the 1962 and 1973 Occupational Changes in a Generation (OCG) Surveys.
Regarding definition of the population to be studied, Warren considered whether samples of different generations would be representative. In studying how social origins affect people’s adult outcomes, a cross-section of American adults could be designated Generation 0 (G0). This sample would be representative. Observations would be made of the attributes of their parents, in Generation-1 (G-1). People included in G-1 would not be a representative cross-section of Americans, as those with several surviving children would be overrepresented, while those without children or whose children did not survive would be underrepresented. Observations would also be made of the offspring of G0, designated Generation +1 (G+1). These individuals would also not be a representative cross-section of Americans. Specifically, immigrants would be excluded. If the goal of a new study of mobility is to determine “how the distribution of social and economic position changes across generations more broadly,” then proper samples of both the parent and offspring generations would be required.
(2) Sample size. Regarding sample size for a study of social mobility, Warren argued that it should be large enough to support separate analyses of social and demographic subgroups. This might include analysis by nativity, ethnicity, or geographical location. Sufficiently large samples of smaller population subgroups might be obtained by strategic oversampling or by use of large-scale administrative data.
(3) Topical coverage. Warren quickly reviewed a series of topics to be covered in a survey of social mobility. These began with the core topics of education, occupation, and income for parents and children. Race, gender, ethnicity, and immigration history would also be relevant. His
list continued with family structure, health, incarceration experiences, neighborhood characteristics, school quality, voting behavior, and a host of possible mediating factors.
(4) Temporal issues. As it has been several decades since the last major study of mobility, and because there have been such profound social, demographic, and economic changes since then, it is essential that a new study capture characteristics of the contemporary United States, Warren suggested. He also said it would be valuable to study more than one birth cohort so as to make valid comparisons over time.
(5) Spatial issues. Another parameter on which to evaluate existing or potential survey instruments is their inclusion of spatial issues. Understanding the impact of place on mobility would entail attention to neighborhoods, states, and other geographic aggregations, urban, rural, and suburban. Previous work has considered the ways in which rural or farm residence, life in segregated neighborhoods in central cities, and residence in particular geographic regions of the country have shaped opportunity structures within and across generations. The design of a new study of mobility should permit such geographic comparisons. Spatial data would also facilitate cross-national comparisons.
(6) Sustainability. Declaring “I don’t want to come back here in 10 or 20 years and do this all over again,” Warren proposed that “it would be ideal if whatever we decide to do is the first iteration of an ongoing process of monitoring social mobility in the United States.”
(7) Financial expense. In order to address relative financial costs, Warren laid out several options. Other than a new stand-alone data collection effort, options include supplements to ongoing data collection efforts or utilizing administrative record data. These latter approaches would compromise control over specification of study population, design of sample, execution of fieldwork, focus, and breadth of measures. As Warren observed, “Some trade-off is always involved.”
Warren considered the trade-offs among these different options along three dimensions. For financial cost, “the most expensive option is to collect new data, and the least expensive option is administrative record data, with supplementing existing surveys in the middle.” Regarding sample size, a new survey would probably study a much smaller sample, administrative record data would have the largest sample size, and supplementing existing surveys is again in the middle. Regarding topical coverage, a new survey might be quite restricted due to costs, and administrative data includes a very narrow range of topics. On this dimension, supplementing existing surveys might yield the greatest gain. Assessing the cost of the data requires looking at these multiple dimensions simultaneously, and “each of these kinds of options has different pluses and minuses.”
(8) Privacy and data access. Warren’s final parameter addressed respondents’ privacy and researchers’ access. Again, the issues differ across the different kinds of data sources reviewed. “In general,” Warren summarized, “the more sensitive the data we collect, the greater the risk to the subjects, and thus the greater restrictions on its use. If we have data that are perfect in every respect, but it takes each of us three years to get permission to analyze it, and only a small number of people can ever analyze it, that is not every useful.”
Warren then used these eight parameters to evaluate nine existing smaller-scale surveys, two larger surveys, linkages with administrative record data, and prospectively a new stand-alone survey.
Nine Existing Smaller-Scale Surveys
Nine existing surveys might conceivably become the basis of a new study of social mobility. Warren qualified that these surveys are deemed “small” only as compared to the much larger sample sizes of such instruments as the Current Population Survey (CPS) or the American Community Survey (ACS). The nine existing smaller-scale surveys are
- General Social Survey (GSS),
- Health and Retirement Study (HRS),
- High School and Beyond,
- National Longitudinal Survey of Adolescent Health,
- National Longitudinal Survey of Youth 1979,
- National Longitudinal Survey of Youth 1997,
- Panel Study of Income Dynamics,
- Project Talent, and
- Survey of Income and Program Participation (SIPP).
“The real question,” as Warren framed it, “is whether it would be worth basing a new study of social mobility on an extension of one of these smaller-scale surveys.” To consider this question, Warren utilized the eight parameters.
With respect to population and definition, GSS, SIPP, and HRS are samples of household-based adults, and thus exclude institutionalized adults. All nine surveys are representative only of a cross-section of Americans in G0. People in G-1 are not observed at all if they have no surviving children, and are overrepresented if they have multiple surviving children. Immigrants are not included in G+1. Addressing sample size, Warren noted that none of the surveys are large enough to permit detailed subgroup analysis.
For topical coverage, all of the surveys include observations of education, occupation, and income, although not for all three generations. To varying degrees, they include information about other social and economic circumstances. Turning to temporal concerns, Warren noted that any new study would ideally include people from a wide range of birth cohorts, including very recent ones. In this regard, the GSS and planned 2014 SIPP panel are most promising. All nine of the surveys include geographical information but generally permit access to that information only through restricted data use agreements. To varying degrees, they facilitate cross-national comparisons.
In assessing sustainability, Warren asserted that only two of the existing surveys, GSS and SIPP, are suitable targets of opportunity. They are likely to continue to include new cross-sections of Americans born across multiple birth cohorts and could perhaps become vehicles to monitor social mobility in the United States into the future. Warren acknowledged that the financial expense is difficult to gauge but considered it safe to assume that adding topical modules on existing ongoing surveys (SIPP, GSS) would be far less expensive than fielding large new data collection efforts. On the last parameter of privacy and data access, Warren confirmed that data from each of these nine surveys are generally freely available, except for the detailed geographical information available only through restricted use data agreements.
Warren’s “bottom line” regarding the nine existing smaller-scale surveys is that GSS and SIPP would be the best candidates to serve as the basis for a new study of social and economic mobility. Nonetheless, the topical content of both is limited, neither includes institutionalized people, and neither allows for research on how distributions of social and economic resources are reproduced or transformed across generations by demographic processes.
Two Larger-Scale Surveys
Warren then turned his attention to existing larger-scale surveys—the CPS and ACS—to consider whether they might be data sources for a new study of social mobility.
The CPS was the basis for the earlier OCG I and II Surveys. Could the CPS serve this purpose once again? Warren proposed two different designs that might enable the CPS to be used for a new study. One, which he termed OCG III, would field a follow-on survey after the CPS and have a sample size of about 65,000. The other option, a February Supplement to the CPS, would be a new topical supplement as part of the CPS’s rotation of topical supplements and would have a sample size of about 140,000.
With reference to the eight parameters identified, Warren considered
these designs. He made some distinctions between the two. For example, OCG III would probably have greater flexibility for expanding topical coverage than the February Supplement. Overall, Warren found several benefits to using either of these designs. In his assessment, either would produce new data resources for studying contemporary mobility patterns. As CPS is an ongoing effort, either design could be implemented just once or repeatedly, going forward. On a per respondent basis, either design would also probably be less expensive than collecting new data in a stand-alone survey. Warren further presumed that an initiative connected to a federal survey would probably generate higher response rates than a stand-alone survey conducted by another entity, while acknowledging this was an empirical question.
Either of the CPS designs would also have shortcomings. Their topical content would be limited compared to what might be included in a new survey. Both would continue to exclude institutionalized people. Finally, neither would allow for research on how distributions of social and economic resources are reproduced or transformed across generations by demographic processes.
The other large-scale existing survey that Warren considered as the basis for a new study of social mobility is ACS. Using this instrument would involve conducting a separate post-ACS follow-up interview with a sample of ACS respondents, following the precedent of the National Survey of College Graduates but asking questions related to mobility rather than education. The ACS could also be linked to administrative data records.
With reference to the eight parameters, Warren noted that although population coverage rates in the ACS are not perfect, the survey does include institutionalized people. Because the ACS is so large, the followup would be of a subsample and selective oversampling of particular groups of interest would be feasible. The topical content of ACS is already broad for generation G0; the subsequent survey could expand the list of topics and gather parallel information for G-1 and G+1. The design would provide data on a contemporary cross-section of Americans. Currently, publicly available ACS data include information on respondents’ state, metropolitan area, and urban, rural, or suburban residence; more detailed data are made available for restricted use.
Considering sustainability, Warren noted the ACS will continue into the foreseeable future, and the follow-up design might be repeated more than once. Because the follow-up design would benefit from pre-existing ACS sampling and fieldwork operations, per respondent cost would be lower than in a new stand-alone survey. The large ACS sample would also allow for efficient subsampling. Once steps are taken to prevent disclosure, ACS data are routinely made available to the public. Warren
further noted that, because of its affiliation with a federal government agency, the follow-up design might elicit a higher response rate than a new stand-alone survey.
Warren’s “bottom line” is that a follow-up survey to the ACS would produce new data on contemporary mobility patterns, at lower cost than a new stand-alone survey. However, the content would be limited compared to what might be covered in a new stand-alone survey, and would still not allow for research on how distributions of social and economic resources are reproduced or transformed across generations by demographic processes.
Administrative Record Data
Warren then considered the value of linkages to administrative record data, such as the Internal Revenue Service (IRS), the Social Security Administration (SSA), the National Death Index, the National Student Clearing House, and similar sources. Warren began by sharing his “bias” that such data could be useful as a supplement to, but would never be a substitute for, a full stand-alone study of social mobility. That said, he nonetheless gave consideration to a hypothetical that he referred to as “the Warren/Grodsky Scheme,” which would link 1990 U.S. Census records to ACS records for a particular birth cohort, particularly the cohort born between 1974 and 1981 who lived in the United States in 1990.
Warren proceeded to evaluate this prospect along the eight parameters, generating a final summation of its strengths and weaknesses. In his view, a linked 1990 Census-ACS design would produce rich data on contemporary mobility patterns for one cohort. It would comprise a richer set of measures than the alternatives, include institutionalized people, have a large sample, and be relatively inexpensive. However, he noted, the topical content would be limited in comparison to a new survey. Because of privacy concerns, restrictions on using the data would be so high as to make analysis very difficult or perhaps impossible. Finally, he said, “it is not clear that one could ever get permission to do it.”
New Stand-Alone Survey
The last option Warren considered is a stand-alone survey, a new data collection operation that would make possible an updated assessment of rates and patterns of social mobility in the United States. The new data could potentially be linked to a range of administrative data from the SSA, IRS, Census Bureau, as well as such records as school transcripts. Multiple modes would be utilized in fielding the survey.
As Warren considered the eight parameters for this option, he offered
several observations. The sample could be representative of a cross-section of people in G0, G-1, and G+1. People in institutions could be included (although cost might limit coverage). The survey would ask about people in G0, their parents’ generation, and their children’s generation, and perhaps also information about their siblings, spouses, and spouse’s parents and cousins. Although the sample size would be more modest than that of CPS and ACS, strategic oversampling would be possible. A comprehensive range of topics could be covered, including about factors hypothesized to serve as mechanisms in the process of status attainment. The survey would generate a representative cross-section of people in the United States and could collect substantial geographic details. It could be designed to be replicated, rather than a unique undertaking. The costs of the endeavor would undoubtedly be high. With the exception of linkages to administrative records, it should not raise privacy concerns that would result in restricted access to the data.
How does this option compare to the others? Warren said he sees many benefits. These include the wide range of measures, and even the possibility of exploring hypothetical mechanisms of status attainment. Linkages to administrative data are also valuable, as that would validate and enrich the new data. Further, the possibility of employing strategic sampling designs would expand the type of analyses that could be performed on the data. Warren also noted that, from the outset, the survey could be designed so as to facilitate comparative work over time and across countries.
The option also presents a number of shortcomings, chief among which is cost. On a per respondent basis, a stand-alone survey is certainly the most expensive way to pursue a new study of social mobility. The costs would result in trade-offs in sample size and content coverage. In Warren’s view, “The cost would have real implications for how sophisticated and complicated and rich the data would be.” He also presumed that response rates to such a survey would probably be lower than for a follow-up connected to an ongoing federal survey.
Possible Next Steps
Having evaluated these different data sources along the eight parameters and weighed their costs and benefits, Warren said he was left with a clear conclusion: “There is no perfect solution.” Nonetheless, for pursuing a new study of social mobility, Warren proceeded to identify five “especially promising options” which, he noted, were not mutually exclusive. Warren also shared some of his preferences regarding these options, leading to further discussion.
1. GSS: Investment in the GSS could be continued. This might include improved occupation coding, more detailed information on occupations, better measures of income and wealth of G0, and better measures of the attributes of parent generations. Warren proposed, “Perhaps it is time to field a new topical module like the 1994 GSS topical module on social mobility.”
2. SIPP: Relevant content could be added to the new SIPP panel. This would include social and economic attributes of a respondent’s parents but would require improving measures relative to the 1986 and 1988 SIPP Module. Warren suggested, “I think this is worth at least thinking about.”
3. CPS: The CPS could be supplemented via either the OCG III or the February Supplement, as discussed above.
4. ACS: Surveys could supplement or link the ACS. Follow-up surveys and linkages of the ACS to administrative data records are discussed above.
5. Collect new data: A new stand-alone survey, despite its many virtues, would probably be so costly as to limit both topical coverage and size and complexity of sample.
In Warren’s judgment, the best choices among these five options would be to continue to invest in the GSS and to supplement or link the ACS. He expressed particular support for the latter option. Workshop discussion ensued regarding the reliability of data from the CPS as compared to the ACS. One participant pointed out that the GSS is an individual rather than a household survey, which makes it less suited to the purpose of a new social mobility study. The tremendous value of administrative data was also emphasized, particularly for capturing intergenerational correlations. Warren concurred and noted that, among existing surveys, “the ACS would be the easiest to link to IRS or Social Security records because the data are collected in a way that allows for this.” In response to a question about data accessibility, Warren clarified that, in contrast to linking two federally mandated data sources, a follow-up survey of people already in the ACS “does not raise disclosure issues.”
Any new data-gathering effort on social mobility will face a number of design issues involving selection of competing measures, timeframes, units of analysis, and overarching goals. Florencia Torche of New York University engaged these issues in her presentation, “How Social Mobility Is Modeled and Measured.” Torche began by confirming that social mobility “captures the strength and pattern of association between the socioeconomic standing of parents and their adult children.” She said she
would cover the following questions: How and when is that association best measured? What are some implications of different measurements? What is the appropriate unit of analysis? How would data-gathering efforts differ depending on whether the goal of the study is to describe or to explain mobility?
Measures and Their Implications
Socioeconomic standing is captured by different measures, including social class, occupational status, annual earnings, hourly wage, and total family income. Before exploring these measures, Torche affirmed, “The strength and the pattern of association and the way we measure depend on the measure of socioeconomic standing used. This is not irrelevant because the findings vary depending on the measure of socioeconomic standing and the measure of association.”
Torche reviewed some standard measures of socioeconomic standing with attention to their reliability and validity. Occupation-based measures of socioeconomic standing include social class and occupational status. Social class is captured by variables, such as job title, employment status, supervisory status, number of supervised workers, firm size, and industry. According to Torche, retrospective report of parental information by adult children is considered reliable. The relevant variable for occupational status is job title; retrospective report of parental information by adult children is also considered reliable. Another approach to assessing socioeconomic standing utilizes economic measures. These include individual earnings, hourly wages, and total family income (with due attention to defining the family unit). For these, retrospective report by adult children is inadequate. Other more reliable routes to information on parental economic standing include panel surveys and merging of datasets.
Occupation-based measures of mobility and economic measures of mobility may yield different conclusions. The discrepancies become particularly evident in country rankings. While the United States consistently ranks as the least mobile of advanced countries in terms of economic mobility as measured by earnings or income, it exhibits considerably more fluidity in terms of class mobility as measured by occupation. “The discrepancy,” Torche observed, “to some extent is expected. We are not measuring the same thing. Occupation is not the same as income or earnings.” The different indicators measure different aspects of well-being although, Torche noted, the implicit assumption is that they are capturing the same latent concept. Yet, she acknowledged, “to the extent that they capture different phenomena, we have not agreed on what these phenomena are.”
Other discrepancies may be introduced by the timing at which measures are undertaken. For occupation, a single point-in-time measure may be sufficient or perhaps two time-point measures. This is insufficient for capturing permanent well-being when measuring earnings, income, and wage.
Generally, measures of both parents’ and children’s economic standing are sought at around age 40, in order to minimize age-related errors in the variables and lifecycle bias. Additional retrospective measures of parental earnings and income are problematic, as retrospective reports of these indicators by adult children are not reliable. To Torche, “This causes the first conundrum. Do we want to do something retrospective and just drop economic measures of well-being, or do we want to collect very highquality measures of parental economic weight? The data requirements are different. The costs are different. If economic mobility will be addressed in an optimal manner, that means we need to collect parental economic information.” Torche mentioned alternative methodological strategies for meeting the challenge, including the two-sample instrumental variable, synthetic parental cohorts, direct-merging of datasets, and panel surveys.
The chosen unit of analysis also has implications for data collection. Torche observed that “from a class perspective, we tend to claim that the family is the unit of stratification, but because of data constraints, we have mostly measured the class position by the main head of household, although that is starting to change.” The unit of analysis needs reconsideration. Torche suggested reviewing whether the relevant unit of analysis should be the individual, and attributes of individuals aggregated, or whether more thought should be given to “family-level variables.”
While declaring that she lacked an answer to that question, Torche did offer some observations about data collection in this regard. She affirmed the need to measure the individual socioeconomic attributes of each member of the household in both generations—“admittedly a very tall data requirement.” She said it is also imperative to collect detailed information about family structure and change over time that involves generations. Torche foresees, at the least, the need for a clear family roster in both generations. As an aside, Torche proposed measuring sibling correlations as another route to studying “the extent to which factors associated with social origins, not only parental standing, shape how well you do in life.”
Describing, Explaining, and Monitoring Mobility
Torche acknowledged the plethora of measures that are relevant to the study of mobility as possible “mediators” of that process. These may be helpful in explaining the mechanisms of mobility. However, Torche cautioned, when a great many indirect effects are added to structural
equation models, they do not necessarily capture the causal effects of interest. “That is the way it is,” Torche lamented, although analysts may be “very quick to move to the language of causation.” While the “attempt to understand mechanisms in mobility is certainly extremely important,” Torche questioned whether it is wise “to keep adding mediators to tell a story that cannot be anything close to causal.”
Perhaps, she proposed, it is better to stick to a clear task of describing, rather than explaining, the phenomenon of social mobility. That is, it might be “better to spend energy measuring less and just humbly staying at the bivariate level but measuring it well.” Acknowledging that the optimal would be to pursue both goals of describing and explaining mobility, Torche nonetheless recognized necessary trade-offs between the two goals in terms of data gathering. Choices need to made regarding “what and how often and how to ask a question.” Furthermore, “the data requirements are very different in terms of variable need and sample size and even representativeness of sample.” Decisions on data gathering will be different depending on the goal pursued.
Torche also considered the possibility of monitoring mobility as the goal of a new study. This would put resources toward tracking the mobility of subgroups of interest, such as particular minority groups or immigrants. The data requirements for this task would again be distinct from those of either explanation or overall description. Torche noted that standard approaches to capture mobility across the population fail to capture distinctions across subgroups with respect to the entire distribution.
The theme of description and explanation as different possible goals of a new study of social mobility sparked further discussion among participants at the close of the workshop. Chandra Muller cautioned against a too-ambitious study aimed at explaining mobility, noting, “In my experience in putting together studies, starting big initially is often grounds for not succeeding in the long run.” She proposed starting small, “piggybacking” on existing studies to “eke out” whatever was possible, and then addressing new topics incrementally.
Sean Reardon similarly expressed support for “doing one thing really well and that is being able to describe the trends in social mobility over time in some simple way.” Reardon acknowledged that while he would like to have explanations and mechanisms, straightforward description, with “some secondary kind of nuanced description” of subpopulations, might be the better way to proceed. Reardon thought it reasonable to presume that “lots of people will think about ways to use the data to help test some explanatory theories,” but that it would be unwise “to build a design to test some subset of possible explanatory theories partially and maybe not do a great job of doing the straight-up stylized facts descrip-
tive stuff that is at the core.” Reardon concluded, “I want to make sure we keep our eye on that ball.”
Robert Mare cast doubt on this approach because, he said, simple description may not be so simple when the object to be described is itself complex and changing. Mare reminded the workshop of the many changes in social institutions that had been discussed, and the lack of clarity over even the proper unit of analysis. To Mare, “It all depends on what we think mobility is,” and “if we are not measuring this over appropriate units, then we do not have anything.”
Michael Hout recalled that the first point of emphasis of his presentation was that social mobility is in many ways the wrong object of study and that the focus should instead be on the transmission of social circumstances across generations—that is, “are people today doing better or worse than their families when they were growing up?” Making any progress on that question requires truly wrestling with “What do we mean by family and what do we mean by worse?”
