3.1. Value of the SWB Module Data to Date
It is still early to gauge the research and policy value of data emerging from the ATUS SWB module. Even so, the kinds of research described above provide a preliminary indication of the insights that can be drawn from the ability to combine time-use information (as it links to specific activities) and self-assessments of well-being during those periods, which have relevance to policies ranging from commuting and home production to eldercare and maintaining good health. Without established and consistent historical data that combine time use and emotional experience, researchers would be limited to analyzing trends in evaluated time use that are difficult to tie to specific determinants.
Several characteristics of the SWB module data contribute to its value:
• Its status as the only national data source on subjective well-being that is linked to activities and time use.
• Its Day Reconstruction Method (DRM)-like capability, unavailable with most other data sources on subjective well-being.
• Its large enough sample sizes (especially if pooled over multiple survey years) to accommodate analyses of important subgroups of the population.
• Its ability to facilitate research to begin solving difficult measurement and conceptual issues that have historically plagued work on subjective well-being.
The fact that the ATUS SWB module is the only federal government data source of its kind gives it a potentially very high value. In particular, its approximation of the DRM is unique.12 As described above, linking of emotional states to daily experience may be the most directly relevant dimension of subjective well-being to policy. It is important to know how people feel when they are working, commuting, taking care of the old and the young, etc. In addition, identifying the context in which such activities take place, and asking respondents to rate well-being in that context (in the case of the ATUS, of the previous day) has the advantage of eliciting specific memories and, in turn, reducing bias associated with respondent recall.
More generally, there has been enough progress in research on the measurement of subjective well-being to pinpoint specific policy domains and questions for which such data are useful. For example, cross-sectional data have proven important for research assessing the
________________
12The day reconstruction method is itself an approximation of more time-consuming experience sampling and ecological momentary assessment methods; however, the day reconstruction method captures information about episodes while the ecological momentary assessment method typically captures information about moments (Christodoulou, Schneider, and Stone, 2012). Simplified versions of the experience sampling and ecological momentary assessment methods—which, in some, sense represent the gold standard since they involve repeated assessment in real time of people’s current hedonic well-being—are necessitated by burden, time, and intrusiveness constraints in surveys. Though research is under way on the issue, it is still an open question how well, and under what conditions, the day reconstruction method approximation is adequate and useful.
relative impact on people of income and unemployment13 and marriage and marital dissolution (Deaton, 2011, p. 50) and, more generally, on the effect of policies where large nonmarket components are involved (e.g., standard of living during end-of-life medical treatment). Data on subjective well-being have the potential to augment information in any situation in which market data are unavailable or not relevant and policy makers require criteria for choosing one course of action among two or more alternatives. In these cases, a range of evidence—revealed preference, stated preference, and subjective well-being measures—can usefully be drawn upon. And wellbeing measures that are tied to specific activities add a great deal of subtlety to these analysis; for example, while perhaps unemployed persons are able to engage more in activities they like to do (spend time with friends or relatives, rest, watch television, etc.), perhaps they enjoy each of those activities less relative to the employed.
It will be a task for this Panel’s final report to provide an assessment of the extent to which subjective measures—including both global, evaluative measures and the more experiential measures that are the focus of this module—can or should be used to guide policy. Collecting data within the context of the ATUS has the potential to help researchers and policy makers evaluate whether these measures can be used in this way.
3.2. Cost of Discontinuing the Module
The cost of discontinuing the module could be large since—if the value of such data became more apparent at some point in the future—restarting the survey would likely entail repeating start-up tasks and drawing again on political capital to make it happen. More importantly, the data continuity that is now being established (with the 2010 and 2012 waves and the proposed 2013 wave) would be lost, affecting the ability of researchers to draw inferences from trends in reported time use and well-being.
On the budget side, the marginal financial cost of adding the developed module to ATUS is relatively modest—about $178,000.14 That said, it would be useful to perform a full accounting to assess the quality of survey results and any effects that the addition of the SWB module may have on the quality of the overall CPS and ATUS. At least in terms of respondent burden and response rates, these concerns would seem to be modest for the former and unfounded for the latter. Indeed, by design, the ATUS is asked of those who have rotated out of the CPS, and modules are asked after the core ATUS is completed. This design element prevents modules from impacting response to the core ATUS and CPS.15 Because the SWB questions are the last
___________________
13One could reasonably conclude that addressing the recent high rate of unemployment was made even more urgent by findings from research on subjective well-being showing that, in terms of individuals’ utility, more was involved than simply an income effect. As Krueger and Mueller (2012) note, unemployment takes an emotional toll on people even while they are engaged in leisure activities. This calls into question an earlier conclusion by economists that people’s decreases in well-being because of unemployment may be partially compensated by increases in leisure.
14The monetary cost of the 2012 module was higher ($273,000) as it included cognitive testing, data editing, interviewer training, and call monitoring activities by BLS.
15If ATUS interviewers indicated that the survey will take 5 minutes longer, addition of the module could affect people’s willingness to participate (unit response rates). ATUS response
thing the respondent hears, the impact on the core ATUS is expected to be minimal. Similarly, the SWB module cannot, by design, bias the core diary responses. On the respondent burden question, for the 2012 SWB module, average time spent was approximately 5 minutes, which adds up to an estimated 1,100 hours for the 12,800 respondents (Federal Register).
A third wave of data collection will add significant information beyond what has been collected so far. Most obviously, another year for the survey means an increased capacity for researchers to enlarge samples by pooling data across years. For some purposes—for example, to look at well-being effects associated with changes in employment during recessions (only a small percentage of the population is unemployed) or to investigate differences across population subgroups—the number of observations needed to make valid statistical inferences well exceeds the annual sample size. This is especially true for comparing self-reported well-being score across smaller population subgroups. Almost all of the research to date using ATUS—which covers a wide range of topics, from household production, to work and leisure patterns, to childcare issues—has pooled data across years to increase the robustness of the statistical estimates.16 The need to enlarge samples (pool data) will be true for research applications that rely on the SWB module of the ATUS as well.
Crucially, the 2012 module (the second wave) is only the first version of the survey that asks the overall life satisfaction (evaluative) well-being questions. In order to begin looking at sensitivity of measures and changes over time in these questions, at least one additional round of the survey—and ideally several more—are needed. A 2013 module would effectively double the sample size of respondents who have answered the evaluative well-being questions.
Fielding another round of the SWB module will also add to the accumulating evidence needed to determine the value of incorporating it into the ATUS (and possibly elsewhere) on something more than an experimental basis. More generally, continuing the module will encourage discussion of how measures of subjective well-being can play a useful role in assessing the effects of public policies. On the research side, a third wave of data may shed light on unanswered questions about survey issues, data quality, and reliability (e.g., nonresponse bias, question ordering, context effects). Other technical issues that could be studied include mode of administration effects (is reported well-being lower in face-to-face interviews than for telephone or internet modes?); activation/valence (are positive and negative affect two ends of the same bipolar dimension or are they separable unipolar dimensions? scaling (do populations from difference cultures or age groups systematically respond differently? and memory bias (e.g., are negative events reported more or less frequently than positive events?).
A third wave of the survey could also be used to explore opportunities for experimentation designed to move toward an optimal survey structure, should the module become a permanent biannual ATUS supplement. Although it is unlikely that major changes could be made for a 2013 module, in the longer term it is certainly worth considering whether
__________________
rates have ranged from 52.5 to 57.8 percent. The response rates for 2010 (the first year of the SWB Module) was 56.9 percent.
16A bibliography of research that has used ATUS data can be found at http://ideas.repec.org/k/atusbib.html(accessed August 7, 2012).
modifications could be made to increase its value. Examples of possible modifications to consider include
• Split sample surveys—one half the respondents could receive one question while the other half gets another; this would be useful for testing such things as sensitivity to different scales and question wording.17
• Finding the optimal number of activities to ask about. It is not obvious that three activities is the optimal number of activities to include on the module. It may be useful to ask about hedonic well-being associated with more activities in order to increase the reliability of daily estimates. Importantly, sampling more episodes increases the power to examine activity-specific effects, which may be particularly valuable for addressing policy questions. Doubling or even tripling the number of episodes may be cost-effective, although that benefit would have to weighed against considerations of participant burden and the potential impact on response rates.
• Selecting the “right” positive and negative emotion adjectives for module questions. Research supports the separation of positive and negative states but, more generally, should the module be focused more on suffering or happiness. The module could experiment with different adjectives and how interpretation varies across populations.
• Expanding coverage to pain and other sensations. There are no good conceptual criteria for differentiating between sensations and “pure” emotional states or for how the two link together. Intuitively, sensations are principally physiological states, in contrast to such feelings as anxiety, stress, and joy, which are principally subjective states.
• Additional or replacement questions for consideration. A possible example is adding a question or two about sleep, such as: “How many hours of sleep do you usually get during the week?” or “How many hours of sleep do you usually get on weekends?” The objective of such questions would be to find out if respondents’ reports about behaviors/emotions—feeling happy, tired, stressed, sad, pain—are influenced by (chronic) sleep deprivation or other sleep patterns.18 A methodological question is how well do people recall the previous night’s sleep?
• Selecting among competing evaluative measures. Is the current Cantril approach, which is perhaps the most remote from affect measures, optimal? Alternative versions of the evaluative measure are common in the literature.
It would also be interesting to make modifications to the SWB module so that day-of-week effects could be tested for different domains—health, education, transportation, etc.
__________________
17In its well-being survey, the United Kingdom’s Office of National Statistics has used, or plans to use, split trials to test for such things as sensitivity to different scales, question wording, and order and placement of questions.
18This idea was raised by Mathias Basner, of the University of Pennsylvania School of Medicine, who noted that self-assessments of habitual sleep time overestimate physiological sleep time and that estimates of habitual sleep time based on ATUS overestimate self-assessments of habitual sleep times found in other population studies. Therefore, he suggested that it would be “very elucidating” to compare self-assessments of sleep time for the two questions above against estimates based on ATUS responses for the day before the interview day.
The merits of retaining some fraction of the sample for experimental work should be strongly considered, presumably not for 2013 but for subsequent years. One such experiment would be to determine sample sizes needed for subgroup analyses (e.g., day reconstruction method questions, which rely some recall, are systematically answered differently by older and younger populations; in an aging society, it is important to be cognizant of these effects).
The ATUS SWB questions could be the model for a standard set of questions that could be added to other surveys. With effective data linking, this could yield a rich set of findings about the relation to SWB of a wide range of covariates. If such a strategy were adopted, the experience of the ATUS SWB module will provide insights about how questions might perform on health, economic, and other kinds of surveys; and for determining candidate surveys such as the National Health Interview Survey and the National Health and Nutrition Examination Survey, administered by the National Center for Health Statistics, and the Survey of Income and Program Participation, administered by the U.S. Census Bureau for adding modules. As noted above, there are potentially major advantages in having similar questions embedded across multiple surveys, especially as linking of microdata (including administrative) records becomes increasingly feasible.
In light of changing budgets and priorities and emerging alternative data sources (e.g., private label, digital, Web-based), the nation’s statistical agencies have already begun to reexamine the content, modes, and structure of their surveys and data programs more intensively than ever before. New scrutiny of what trends in society are important to measure (such as those recommended by the Commission on the Measurement of Economic Performance and Social Progress; Stiglitz, Sen, and Fitoussi, 2009) may give rise to new opportunities to refocus statistical program coverage (and the surveys on which they are built) and to move into new research areas surrounding SWB. Smaller-scale studies and data collections, such as the ATUS SWB module, are needed to help judge the value and feasibility of embarking on production of national-level SWB statistics, such as those under development in the United Kingdom. Moreover, determination of the place of measures of subjective well-being in monitoring the economy and society cannot be done without the data. The question of whether self-reported measures of well-being should one day be reported alongside more standard economic statistics, such as those for income and employment and for financial markets, is as yet unanswered.
A careful assessment of the data emerging from ATUS and the SWB module may help avoid mistakes if self-reported well-being statistics are ever produced on a larger scale. To the extent that evidence can be accumulated on the research and policy value of such data, a better basis for making these data collection and statistical program decisions can be established. The fact that the United States has a decentralized statistical system makes coordinating of the survey content related to subject well-being a greater challenge than in countries with centralized statistics systems. However, it also affords the option of targeting development in the areas that are identified as the most relevant for policy and measurement—such as health, employment, or education—for which the argument is strongest for adding this kind of content. In light of these arguments, it is the view of the panel that the cost of the proposed 2013 SWB module is quite modest given its potential to inform decisions about potentially much larger statistical system investments.
