People’s engagement in society, their associations and networks, and the characteristics of their communities profoundly affect their quality of life. The attributes commonly discussed under the rubric “social capital”—political participation; engagement in community organizations; connectedness with friends and family and neighbors; and attitudes toward and relationships with neighbors, government, and groups unlike one’s own—are often associated with positive outcomes in many areas of life, including health, altruism, compliance with the law, education, employment, and child welfare. It has also been observed that civic engagement, social cohesion, and other dimensions of social capital are sometimes related to negative outcomes. Under certain circumstances these actions and processes may contribute to social tension and community fragmentation; in others to social cooperation and integration.
Recognizing the value of understanding these relationships, the Corporation for National and Community Service (CNCS) requested that the Committee on National Statistics create a panel “to identify measurement approaches that can lead to improved understanding of civic engagement, social cohesion, and social capital—and their potential role in explaining the functioning of society.” The statement of task called for the panel to consider conceptual frameworks, definitions of key terms, the feasibility and specifications of relevant indicators, and the relationship between these indicators and selected social trends. It also called on the panel to weigh the relative merits of surveys, administrative records, and nongovernment and nonsurvey data sources, and to assess the appropriate role of the federal statistical system.
To fulfill its charge, the panel assessed the role of the Civic Engagement and Volunteer Supplements of the Current Population Survey (CPS), conducted by the U.S. Census Bureau and currently the most visible federal survey with questions about social capital. The panel also considered the broader contextual questions implied in its charge
- Which social capital variables (dimensions) are most relevant to policy, research, and general information needs—and which are measureable?
- What are the most promising approaches—survey and nonsurvey, government and nongovernment—for collecting this information?
- What should be the role of the federal statistical system, recognizing a rapidly changing data collection environment?
- How might disparate data sources—including administrative data and unstructured digital data (that is, the vast range of information produced on an ongoing basis, and usually for purposes other than statistics and research)—be exploited?
CONCLUSION 1: Data on people’s civic engagement, their connections and networks, and their communities—aggregated at various levels of demographic and geographic granularity—are essential for research on the relationships between a range of social capital dimensions and social, health, and economic outcomes, and for understanding the directions of those effects. This research in turn informs policies that seek to maximize beneficial outcomes and minimize harmful ones.
The panel emphasized the importance of data collection and measurement of social capital dimensions on the basis of (1) evidence connecting them to specific, measurable outcomes in domains such as health, crime, education, employment, and effectiveness of governance; (2) their value in providing descriptive information capable of generating insights about society; and, relatedly, (3) their research and policy value.
KEY MEASUREMENT CONCEPTS
Though the relevant literature is extensive, there is no universally agreed-upon definition of social capital or taxonomy of its components. The first key term referenced in the study charge, “civic engagement,” is, according to Ehrlich (2000, p. vi), comprised of individual activities oriented toward making “a difference in the civic life of…communities and developing the combination of knowledge, skills, values and motivation
to make that difference. It means promoting the quality of life in a community, through both political and nonpolitical processes.” Volunteerism is one defining characteristic of civic engagement in that most if not all such activities are discretionary.
The second key term in the charge, “social cohesion,” can be viewed as having multiple dimensions, including: belonging or isolation, inclusion or exclusion, participation or noninvolvement, recognition or rejection, and legitimacy or illegitimacy (Jensen, 1998). By implication, as articulated by Forrest and Kearns (2001, p. 2128), “a society lacking cohesion would be one which displayed social disorder and conflict, disparate moral values, extreme social inequality, low levels of social interaction between and within communities and low levels of place attachment.” Specification of the geographic unit of analysis (spatial scale) is an essential dimension of social cohesion. Neighborhoods, states, or other groups can be in conflict with one another while demonstrating strong internal social cohesion. Portes (1998, p. 6) emphasizes the capacity of personal and group connections and other support resources to affect “the ability of actors to secure benefits by virtue of their membership in social networks or other social structures.”
Civic engagement and social cohesion are often viewed as components of the charge’s third key term—social capital. Francis Fukuyama (2002, p. 27) describes social capital as “shared norms or values that promote social cooperation, instantiated in actual social relationships.” He emphasizes the role of certain subjective states and attitudes, such as trust, which “…acts like a lubricant that makes any group or organization run more efficiently” (Fukuyama, 1999, p. 16). Putnam (2003) introduces two types of social capital: bridging and bonding. The former is exemplified by voluntary associations and horizontal ties based on common interests that transcend differences of ethnicity, religion, and socioeconomic status in communities; the latter refers to social ties built around homogeneous groups that do not span “diverse social cleavages.”
The key terms in the study charge are constructs with uncertain boundaries.
CONCLUSION 2: Because the terms “social capital,” “civic engagement,” and “social cohesion” refer to broad and malleably-defined concepts that take on different meanings depending on the context, they are not amenable to direct statistical measurement. However, dimensions of these broad constructs—the behaviors, attitudes, social ties, and experiences—can be more narrowly and tangibly defined and are thus more feasibly measured.
Measures of social capital can also be differentiated in terms of those that are behaviors (e.g., participating in a political campaign), those that capture attitudes (e.g., trust in neighbors or political representatives), and those that are experiences (e.g., being discriminated against). Many of these are rooted at the individual level, though they may typically be studied as properties aggregated at group levels ranging from families, to neighborhoods, to communities, to regions, to nations. Others, such as voter identification laws or school segregation, are inherently group concepts. And the relevant unit of observation can be suggestive of the appropriate data collection mode. If one is interested in total voter turnout or total membership in associations, administrative and other nonsurvey data sources may be adequate. If the focus is attributes of individuals engaged in various behaviors or with specific attitudes, microdata are essential.
PRIORITIZING MEASURES, DATA COLLECTION STRATEGIES
Studies of social capital have covered a broad range of topics in the social, health, and economic policy domains, including:
- personal connectedness and employment outcomes;
- effects of social cohesion, self-reported “trust,” and other dimensions of neighborhood social capital on crime and public safety;
- cohesion and community resiliency;
- home ownership and civic engagement;
- social connections and self-reported well-being;
- isolation and health effects;
- social capital and mental illness;
- social relationships and health mechanisms; and
- social capital and child outcomes.
Depending on the question of interest, a given dimension of social capital may be seen as a mechanism whereby change can be affected (i.e., through policy levers) or as the primary focus itself. For example, reducing social isolation or improving trust in a neighborhood may be tools to improve health and reduce crime, or they may be the policy objectives in and of themselves.
CONCLUSION 3: For data collection related to social capital, the theoretical or policy issue of interest is critical for identifying clearly defined components and developing instruments (survey or otherwise) designed to measure these components.
Empirical research has produced valuable insights and advanced understanding of a range of phenomena related to social capital. However—with some exceptions, such as social isolation as a risk factor for health—to date, it has produced only sketchy evidence of causal relationships between social capital and outcomes of policy interest or, conversely, of how a given indicator is predictive of changes in the level of social capital (e.g., the link between home ownership and extent of participation in the community). Even so, data collected from large population surveys are still essential because of their value in providing descriptive information and because evidence continues to accumulate that phenomena described as social capital play an important role in the functioning of communities and the nation.
CONCLUSION 4: The study of social capital, though a comparatively young research field, is sufficiently promising to justify investment in data on the characteristics of communities and individuals in order to determine what factors affect their condition and progress (or lack thereof) along a range of dimensions. Improved measurement, additional data, and resulting research findings are likely to find uses in policy making.
Although there are difficult challenges of demonstrating causation, this (along with wrestling with vague concepts) is familiar in nearly all social science research fields, especially early in their development. Studies based on highly granular, ongoing, and multisource datasets appear to offer the greatest promise for untangling the circularity of causal pathways—for example, to what extent does deterioration of job growth in a city weaken social ties and lead to group conflict over scarce resources, and vice versa? To what extent does interaction and trust among neighbors contribute to reductions in crime, and to what extent do reductions in crime encourage greater neighborhood connectedness?
With these and other research questions in mind, statistical agency programs may prioritize (1) improvement in the near-term data collection, focusing primarily on existing survey vehicles, or (2) longer term visions that anticipate the potential of combining government surveys with one another, with administrative data, and with unstructured digital data generated as the by-product of day-to-day business, communication, social, and other activities.
RECOMMENDATION 1: For data collection in areas of social capital, a multipronged strategy should be pursued in which large population surveys conducted by the federal statistical system play a role, but one that is increasingly complemented
and supplemented by new, innovative, experimental alternatives. The greatest promise lies in specific-purpose surveys such as those focused on health, housing, and employment issues (especially those that have a longitudinal structure) and in the exploitation of nonsurvey sources ranging from administrative data (e.g., local-level incident-based crime reports) to digital communications and networking data that are amenable to community-level analyses. Many of the surveys will continue to be conducted or funded by the federal government, while many of the nonsurvey sources will originate elsewhere.
The quality of the nation’s information and its research capacity will in large part be determined by the effectiveness with which these increasingly disparate data sources can be exploited and coordinated by the statistical agencies and users of their products.
THE CPS SUPPLEMENTS
That the government collects data about civic engagement signals that these topics are important to the nation. The purpose of the CPS Civic Engagement Supplement—fielded in 2008, 2009, 2010, 2011 and, with a half sample, 2013—as stated in justification documentation prepared by CNCS for the U.S. Office of Management and Budget (2011, p. 3), is to
…collect data for the Civic Health Assessment, an annual report mandated by the Serve America Act that is produced in partnership with the National Conference on Citizenship (NCoC). The Civic Engagement Supplement provides information on the extent to which American communities are places where individuals are civically active. It also provides information on the number of Americans who are active in their communities, communicating with one another on issues of public concern, and interacting with public institutions and private enterprises.
At national and state levels, the Supplement fulfills several elements of this mandate for descriptive information.
CONCLUSION 5: Current Population Survey (CPS) supplements, which offer only a limited amount of survey space (about 10 minutes is allotted for a given monthly supplement), are most appropriate for collecting data on variables that (1) can be estimated from a small set of questions, (2) deal with people’s behaviors, (3) would be difficult to ascertain through nonsurvey methods, and (4) need to be correlated with personal attributes that are also captured on the survey in order to study how they inter
relate for groups such as the elderly, minorities, or immigrants. Also critical is that the CPS data are useful when the research and policy questions of interest require information aggregated at the federal-, state-, or (in some cases) metropolitan-area level.
By these criteria, the Civic Engagement and Volunteer Supplements are well suited for generating statistics on a subset of well-defined activities. Volunteering is a particularly important form of engagement because, unlike “memberships,” which are also often asked about, it requires action.
CONCLUSION 6: Information about the population’s political participation and voting activities can be adequately captured with a small number of questions. Likewise, the Current Population Survey (CPS) has proven useful for understanding volunteering rates and patterns—especially when linked with data from the survey’s time use (American Time Use Survey) module. Thus, the CPS Volunteer (September) and Civic Engagement (November) Supplements are best focused on political and civic participation.
The CPS Supplements are less useful for generating data on dimensions of social capital such as social cohesion, connectedness, and trust.
CONCLUSION 7: Although even a short module can generate useful information, the Current Population Survey does not offer a comparative advantage for data collection on complex behaviors and attitudes indicative of social cohesion, individual and group connectedness, and civic health generally. These phenomena cannot be satisfactorily characterized by data collected from a small set of questions.
Rich and detailed datasets are needed to capture the complexities of social capital, particularly since many of these phenomena take place most intensively as community-level social processes. Examples of this research model include the Kasinitz et al. (2008) study of immigrants in New York City and the Project on Human Development in Chicago Neighborhoods (Sampson et al., 1997, 2002, 2012b). These studies were designed to generate insights about the links between neighborhood characteristics, social organizations, community level factors, and broader social phenomena. They utilize a wide range of methodologies ranging from experimental designs to systematic observational approaches that benchmark data on neighborhood social processes.
Determining the appropriate scope of the Civic Engagement and Volunteer Supplements begins by recognizing what can and cannot be measured well within the structure of the survey; budget realities also factor in. During planning for the 2013 supplements, CNCS was called on to consider cost-cutting options, which included (1) combining the civic engagement and volunteer supplements, with a reduced number of questions on each topic, in order to field both each year; (2) moving to a rotating schedule in which each is fielded as is, but only in alternating years; or (3) cutting sample sizes in order to field both supplements annually.
RECOMMENDATION 2: Due to the importance of substate and subgroup analyses, under a cost-reduction scenario the panel favors a combined civic engagement and volunteer supplement to the Current Population Survey (CPS) even though it would require reducing the number of questions in each category. Question streamlining would be accomplished by (1) narrowing the subject matter now covered in the Civic Engagement Supplement based on assessment of what information can and cannot be collected effectively in a short survey module; (2) identifying and eliminating redundancies across the CPS Civic Engagement and Volunteer Supplements; and (3) identifying and eliminating questions for which comparable data can be found in other government surveys or elsewhere, while recognizing there is analytic value in having both volunteering and civic engagement data, along with covariate information, for the same respondents.
BEYOND THE CPS
Developing a comprehensive data collection strategy requires consideration of other survey vehicles; the CPS supplements should not be evaluated in isolation. Although few surveys specialize exclusively on social capital, many include at least a few questions that relate to the context on which they focus. The primary focus of the CPS is the labor force. The American Time Use Survey (also a CPS supplement conducted by the U.S. Census Bureau for the Bureau of Labor Statistics) captures volunteering and is also important for studying time spent in various other nonmarket activities. The Health and Retirement Study (conducted by the Institute for Social Research at the University of Michigan) asks about people’s support networks in the context of health among older Americans. The Panel Study of Income Dynamics Study (also conducted by the Institute for Social Research at the University of Michigan) asks about organizational memberships and contacts in the context of caregiving and
well-being. And the National Longitudinal Survey of Youth (conducted by NORC at the University of Chicago for the Bureau of Labor Statistics) asks about volunteerism, religious affiliation, and political attitudes in the context of education and work.
The new Neighborhood Social Capital Module—part of the American Housing Survey (conducted by the Census Bureau for the Department of Housing and Urban Development)—is a promising initiative that focuses on neighborhood effects. Data are collected on shared expectations for social control, social cohesion, trust within neighborhoods, and neighborhood organizational involvement. Further work will be needed to determine the precision of the small area estimates and statistical properties, but the survey sample size is considerably larger than the CPS—and it includes a longitudinal component.
RECOMMENDATION 3: The Corporation for National and Community Service should establish a technical (research and evaluation) working group tasked with systematically investigating the content of, and redundancies or overlap in, federal surveys in areas related to social capital measurement. A good place to start is with the Current Population Survey (CPS) Civic Engagement Supplement and the Neighborhood Social Capital Module of the American Housing Survey. Other candidates are the CPS Volunteer Supplement and the American Time Use Survey and the CPS Voting and Registration Supplement and other national election administration and voting surveys. The technical working group should be charged with finding effective ways to coordinate the content of these options.
Longer term aspects of the data collection strategy identified above involve looking beyond traditional survey vehicles. Measurement of the more complex components of social capital, in particular, requires multimodal data collections that include intensive and sustained research models.
RECOMMENDATION 4: For measuring relationships between such phenomena as social cohesion and neighborhood environment on one hand, and health, social and economic outcomes on the other, statistical and funding agencies should take an experimental approach, sponsoring studies at the subnational level and in-depth and longitudinal pilot data collections. This suggests that additional research and testing will be needed before committing to the content and structure of specific survey instruments. The statistical agencies’ advisory groups may
be especially helpful in thinking creatively about what kinds of research and survey projects offer the most promise.
In considering alternative measurement approaches and strategies for a rapidly changing data world, it has become increasingly necessary to statistical agencies to monitor developments taking place beyond the traditional government survey world.
Statistical information about the United States and its subpopulations will increasingly be assembled from an interconnected data system. Building a capacity to link across survey sources as well as administrative and other kinds of records is an obvious strategy for maximizing the value of resources. The value added stems from two factors: First, merging data sets allows for a broadening of covariates that may be correlated with measures of outcomes. Combining individual-level survey information with other sources can also provide contextual data about the geographic unit of interest. Second, and especially relevant to assessment of the CPS Civic Engagement Supplement, is that sample sizes associated with national-level population surveys are not typically adequate to support local-area analyses. Modelling methods can often take advantage of survey data augmented with additional records for the purpose of producing small area estimates that are essential to measuring neighborhood and community phenomena.
The panel recognizes and endorses linking work already pioneered by the Census Bureau and other government agencies and the ongoing and more intensive efforts underway. The panel also recognizes the conceptual problems that must be solved and the resources needed to undertake this work.
CONCLUSION 8: The Current Population Survey (CPS) cannot provide all the variables and the level of geographic detail necessary for research on social capital, social cohesion, and civic engagement. It is therefore essential that design strategies for the CPS be conceptualized with the presumption that this data source will need to be linked (even if only at the group level) to other data from the federal government and beyond. The national-level data collected on a regular basis should complement other sources, both government and nongovernment, for use by researchers. Research data centers operated by the federal statistical agencies can create opportunities for these kinds
of coordinated efforts, which must comply with respondent confidentiality and privacy requirements.
Nonsurvey Data Collection
Multimodal data collection, involving complements and substitutes for traditional government surveys, is necessitated by the fact that much of what is interesting and important about social capital takes place at the level of neighborhoods or communities, where general population surveys need to be augmented or, in some cases, replaced by data sources that allow for more targeted studies.
It has become commonplace to emphasize the potential—for solving problems in government, the private sector, and in scientific research—of the ever-growing volume of data created and captured digitally. Some kinds of information, such as the structure and density of people’s online relationships and connections or their patterns of cellphone communication, are next to impossible to discern using conventional survey methods. However, while alternative data collection and analysis methods are no doubt flourishing, establishing the statistical validity of estimates based on “big data” sources is in its infancy. In addition, most unstructured digital data are generated by and owned by private sector entities where models for methodological transparency and privacy and confidentiality protection are undeveloped. These are but two reasons, among several, that a survey-centric approach—for which problems of data accuracy, quality, representativeness, and confidentiality have largely been contained or solved—will continue to play a central role in social science research for the foreseeable future.
Beyond social media, private-sector data generated by people’s purchasing and other online activities and by automated payroll systems has created private-sector alternatives (or, in some cases, complements) to such key economic indicators as the Consumer Price Index (e.g., the Web-based MIT Billion Prices Index) and employment statistics (e.g., ADP employment reports). The emergence of big data, coupled with advances in computational science analytic techniques, raises the possibility of developing indicators of citizens’ civic engagement and other social behaviors and attitudes that are less burdensome than surveys.
The statistical agencies are of course aware of the changing data landscape and are considering measures to adapt and take advantage to modernize their programs. Even so, the magnitude of upcoming changes argues that the statistical agencies be involved even more closely in these developing areas and engage in parallel data studies.
RECOMMENDATION 5: Under the leadership of the U.S. Office of Management and Budget, the federal statistical system should accelerate (1) research designed to understand the quality of statistics derived from alternative data—including those from social media, other Web-based and digital sources, and administrative records; (2) monitoring of data from a range of private and public sources that have potential to complement or supplement existing measures and surveys; and (3) investigation of methods to integrate public and private data into official statistical products.
The research agenda outlined above is not simple. The U.S. statistical system is decentralized, comprised of more than 50 entities, about 15 of which are defined as principal statistical agencies. One of the drawbacks of such a system is the difficulty of generating critical mass for the purpose of major research undertakings that are broader than the mandates or needs of any one agency and that require a coordinated approach to be successfully pursued.
RECOMMENDATION 6: In mapping the way forward for the integration and exploitation of new data sources, the U.S. Office of Management and Budget should coordinate the exploration of alternatives for developing the necessary research capability across the federal statistical system. Among the alternatives are extensions of the current partnership between the Census Bureau and the National Science Foundation and the creation of a federally funded research and development center for this work.
Such a center for statistics, for which there is precedent, would allow a much needed focus to be placed on research topics that are common to the entire statistical system and not unique to one agency.
The measurement areas described in this report represent only a portion of those that factor into social science, urban planning, public health, and other research areas. But the nature of the activities, attitudes, and behaviors encompassed, along with the multiple geographic levels of interest and the role of group and individual interactions, make it an illuminating case study of the growing need for multimode data collection to underpin modern research and policy. And, because the study of social capital is a relatively new strand of social science inquiry, where methods are not as entrenched as elsewhere, it is a good testing ground for development of experimental measurement approaches that exploit the rapidly evolving data landscape. Because data users have fewer pre-
conceived notions of what the underlying statistical framework (and official statistics in the area) should look like, measurement of social cohesion, civic engagement, and other dimensions of social capital is a good place for statistical agencies to begin developing cutting-edge techniques for blending traditional survey data with new, nonsurvey data into integrated measurement programs.