Research: Next Steps
Our report discusses the challenges of measuring racial discrimination in a range of social and economic domains. Establishing that overt or subtle forms of discrimination have occurred and the consequent effects on outcomes requires careful and thorough analysis to rule out or limit alternative explanatory factors. In much research to date, the data and analytical methods make it difficult to justify the assumptions of the underlying theoretical model. Moreover, many statistical and survey-based analyses never articulate an explicit model, which makes it difficult to judge the adequacy of the data and analysis to support the study findings. Laboratory experiments, while often better justified, cannot in and of themselves measure the contribution of discrimination to differential outcomes in a real-world setting.
Although it is difficult to measure racial discrimination, it is possible to conduct important, appropriate research in this area that adds to our knowledge. Some laboratory and field experiments, statistical analyses of observational data, evaluations of natural experiments, and survey measures of discriminatory attitudes and reported experiences of discrimination have produced useful results pertaining to particular types of possible discrimination within a domain or process. To make further progress, we believe it will be necessary for funding and program agencies to support studies that cut across disciplinary boundaries, make use of multiple methods and types of data, and analyze racial discrimination as a dynamic process rather than as a point-in-time event. It will also be necessary for program and research agencies to identify priority areas for which research on the possible role of
racial discrimination is most needed and to further the development of useful data sources for measurement purposes.
Our efforts as a panel concentrated on an in-depth exploration of concepts and methodological approaches to measuring racial discrimination. Within the scope of our charge and resources, we could not take the next step of developing a detailed agenda in any domain for further research to inform policy making and public understanding. What we undertake in this short concluding chapter is to suggest ways in which program and research agencies might build a research agenda that is directed to priority needs for measuring racial discrimination.
Program agencies that are charged to monitor and investigate discrimination complaints, such as the U.S. Department of Education Office of Civil Rights, the U.S. Equal Employment Opportunity Commission, and others, have a direct interest in the measurement and understanding of racial discrimination. These agencies could benefit most directly from improved data and research in relevant domains of interest to them. Other agencies that design and operate programs that may be directly affected by the presence of discrimination and by antidiscrimination laws and regulations should also have an interest in discrimination research (such agencies exist in the U.S. Departments of Health and Human Services, Housing and Urban Development, Justice, Labor, and others).
Priority Research Topics
An initial step in furthering useful research for program needs is for agencies to identify the subset of outcomes and processes in which racial discrimination may occur that are of most importance from the agency’s policy perspective. This is not a trivial task. It is crucial, however, to framing a cost-beneficial research agenda, given the substantial time and effort that would likely be required to obtain appropriate data and conduct useful analyses on even a single topic.
Because resource limitations will necessarily constrain research and data collection, program agencies should subject their list of priority research areas to careful evaluation regarding feasibility and costs. We strongly urge that agencies not limit their determination of feasible priority projects to a particular disciplinary perspective or type of analytical method or data. Narrowing a methodological focus too early could well lead to conclusions that do not stand up when subjected to other kinds of analyses.
As a hypothetical example, consider racial discrimination in the employment domain, which clearly presents many questions of policy and pub-
lic interest. A review of the labor market literature, as well as an analysis of program agency data on discrimination complaints, could help identify priority topics. For instance, some research has indicated (see Chapter 11) that equally qualified nonwhite and white college graduates are hired at similar starting salaries but that nonwhites become increasingly disadvantaged with regard to earnings over time. These results suggest that research on employer decision processes related to job training and promotion could merit greater attention in the near future than, say, replication of studies of factors in initial hiring decisions.
The next step is to develop a detailed research plan. In Chapter 7, we argued that statistical information on racial gaps in outcomes will rarely be adequate to support conclusions about the role of racial discrimination in the absence of a detailed understanding of the decision processes of decision makers, including information on what knowledge is available to them and what knowledge they bring to bear in making particular types of decisions. In the labor market example, this would mean understanding the processes by which hiring or promotion occurs and the information available to employers in making employment or promotion decisions. Focused case studies of employer decision processes may be needed to provide the requisite depth of understanding of employer behavior to permit subsequent statistical analysis.1
To be most useful and cost-effective, focused studies of decision-making processes should be informed by theoretical models of the ways in which discrimination might occur. Especially because discrimination may take subtle, as well as overt, forms, it is important to have a theoretical framework to guide the data to be collected in case studies. In the labor market example, such a framework could help determine which actors in a firm to interview; what kinds of institutional practices, policies, and procedures to learn about; and what other information to collect.
In developing a theoretical framework, researchers could usefully review the existing literature of laboratory experiments about discriminatory attitudes and behaviors and the kinds of situations in which attitudes are most likely to lead to race-based discriminatory treatment. For instance, an economics approach to studying discrimination could be enhanced by psychological insights derived from empirical results of laboratory experiments, as well as from psychological concepts about the functioning and sources of discriminatory attitudes and behavior. If laboratory results are not suffi-
ciently focused on the decision making relevant to an agency, then additional experiments could be commissioned to fill in gaps. Such experimentation would require that laboratory researchers develop methods for obtaining participation from people in the work world and other venues outside of academia.
Consideration of appropriate concepts and review of pertinent laboratory results should help suggest the types of data that are most needed for informative analyses of observational data with statistical models. For example, case studies might justify adding questions to cross-sectional and longitudinal surveys on self-reports of discrimination, or they might suggest collecting information on specific characteristics related to the decision process, such as (again, a labor market example) whether the employee was recommended by another employee for an open position, what kinds of testing and interviews were required, and so on.
With data in hand, and with well-developed models of decision processes, agencies would be poised to create a research agenda around questions they would like to present to researchers well versed in statistical analysis methods that are appropriate for assessing evidence of discrimination and its impacts. Program agencies should also consider the possibilities of field studies that bring scientific evaluation techniques to real-world decision-making examples. The use of audit studies within the U.S. Department of Housing and Urban Development, for example, has helped support the claim that ongoing housing market discrimination occurs in the housing search process, which suggests the importance of ongoing enforcement of open-housing policies. Agencies in other domains should consider the possibility of field or audit studies in their own areas of interest.
The work we outline above would require collaboration of scholars from multiple disciplines, including economists, sociologists, social psychologists, and survey researchers. In some situations, ethnographers and cultural anthropologists could also contribute much-needed expertise for designing and conducting the case studies of employers or other decision makers to obtain the richest data possible.
Facilitating Data Access and Use
Another way in which program agencies could facilitate a cost-effective agenda for research on the possible role of racial discrimination in domains of interest concerns the provision of data. Agencies should first analyze the research potential of their own administrative records, identifying low-cost changes to record requirements that would facilitate analytical use of the data. Concurrently, agencies could work to develop arrangements for reasonably ready access to the data by qualified researchers.
In practice, the development of suitable administrative records data for
analysis of racial discrimination, particularly if the research goal is to compare administrative records with survey reports of discrimination events, is likely to present difficult problems. There could well be problems in obtaining access to administrative records, understanding agency reporting systems, and protecting the confidentiality of the data. Nonetheless, because administrative data are maintained for recordkeeping purposes by enforcement agencies and thereby provide a low-cost resource for research, it seems worthwhile to conduct feasibility studies to determine their potential for analytical use, alone and linked to survey data. Such use could not only add to knowledge but also help agencies design more informative records systems for their own enforcement and education programs.
Program agencies could also provide input to the federal statistical system regarding data items that would be useful to include in ongoing household cross-sectional and longitudinal data systems run by statistical agencies. Major longitudinal surveys of cohorts of individuals exist in the domains of labor market experience, education, and health. Such surveys are prime candidates to review to identify cost-effective additions or modifications of questions that would support research on discrimination. Statistical agencies can contribute to the provision of useful data for analysis of discrimination by sponsoring research, as we recommended in Chapter 10, on best practices for obtaining data on racial and ethnic classifications.
Finally, program agencies can play a valuable role in facilitating research evaluation of natural experiments consequent to policy and regulatory changes, by modifying or augmenting administrative records systems, as appropriate. With suitable data, natural experiment evaluations can compare differences in outcomes over time and between individuals affected and not affected by these changes, in ways that can illuminate the possible role of racial discrimination.
We suggest that research funding agencies, such as the National Science Foundation, the National Institutes of Health, and private foundations, can best leverage their resources by addressing areas of research on racial discrimination that are less apt to be considered by program agencies. They also have a comparative advantage in supporting more basic research and data infrastructure, including support for rich longitudinal data collections.
Within-Domain and Across-Domain Cumulative Effects Studies
Research funding agencies are better positioned than program agencies to support innovative, cross-disciplinary, multimethod research on cumulative disadvantage and the roles that current and past discrimination—
whether in a particular stage of a process or in other domains—may play in causing a set of differential outcomes. They are also better positioned to support innovative studies of the possible role of discrimination in cumulative disadvantage over a lifetime and across generations. Our discussion in Chapter 11 of cumulative disadvantage described the need for and the difficult nature of such studies, of which there are very few examples to date.
To move cumulative effects research forward, it could be useful to build on studies of the possible role of discrimination in differential outcomes in one stage of a process to develop insights about the role of past and current discrimination for a subsequent stage of the same process or for another domain. As one example, field experiments in housing and labor markets might provide a basis for work on subsequent outcomes in those domains, by identifying geographic areas or types of firms for which experimental results suggest particularly strong effects of discrimination at an entry level (seeking an apartment or home, seeking a job). These areas or firms could possibly be further studied to consider the cumulative effects of the initial-stage discrimination on outcomes at subsequent stages (e.g., ability to obtain home equity loans or refinancing, access to training and promotion opportunities).
Research funding agencies could also consider supporting studies of the effects of discrimination in one domain, such as housing, on processes in another domain, such as access to schools. They could consider supporting studies of longer-term discrimination over lifetimes and generations. Such cross-process, cross-domain, cross-generation types of research will necessarily require bringing together researchers from multiple disciplines and perspectives and using various data sets and methods—for example, using laboratory experiments to develop theoretical constructs for paths and mechanisms by which cumulative disadvantage could occur; using case studies and ethnographic research to obtain very rich data on perceptions and experiences of discrimination in a particular population group or community; and using rich panel data to follow population cohorts over time.
There are examples of rich, multidisciplinary, cross-domain research in other areas of inquiry that discrimination researchers might look to for guidance. In particular, a number of multifaceted studies have been conducted in recent years of changes in the well-being of low-income populations following major changes in welfare policies (see National Research Council, 2001b). These studies have combined national surveys, surveys of specific cities and neighborhoods, and in-depth ethnographic research to understand the factors that contributed to a range of social and economic outcomes for low-income families in a period of rapid policy and economic change. Survey data have come from repeated cross-sectional interviews and longitudinal panels and from interviews of welfare case workers in addition to welfare recipients, people leaving the welfare rolls, and other
low-income families. Some studies have included field experiments of alternative welfare policies; other studies have taken advantage of natural experiments provided by major changes in national welfare policy and variations in implementation by states.
We have stressed in several chapters the need for rich longitudinal data sets that follow individuals over time and hence permit studies of cumulative disadvantage, as well as studies that delineate paths by which disadvantage—and possible discrimination—occurs. Statistical agencies fund some of the major panel surveys, such as the National Longitudinal Surveys of Labor Market Behavior of the Bureau of Labor Statistics, but many panel surveys are funded by public and private research agencies. These surveys represent significant components of the data infrastructure for social science research. Public and private research agencies interested in facilitating studies of racial discrimination, particularly over long periods of time, can usefully consider ways to augment ongoing and new panel surveys to provide relevant data.
Our report has documented the strengths and weaknesses that various methodologies and data sources can bring to the table for measuring racial discrimination. The difficulties of analysis in this area make it daunting for program and research funding agencies to develop focused, cost-effective agendas for research and data collection. We have suggested some strategies for developing future research plans. We urge that research on racial discrimination, whether focused on program agency priorities for analysis of a particular domain or more basic research on cumulative disadvantage, bring multiple perspectives to bear and use multiple methods and data sources. Although current and even past racial discrimination may be only part of the explanation for persistent racial gaps in important domains of social and economic life, it is important for public policy and public understanding to carry out research on the role of discrimination among all of the factors that shape American society today.