Part II
Methods
In Part I, we defined the concepts of race and racial discrimination from a social science research perspective. The history of legal and institutionalized racial discrimination in the United States and the existence of widespread racial disparities in outcomes across domains prompt our review of methods for assessing the extent to which discrimination continues to affect historically disadvantaged racial and ethnic groups. Our definition of racial discrimination includes overt and subtle discriminatory behaviors and processes. If, as some have suggested, modern forms of discrimination are less likely to be direct and explicit and more likely to be indirect and ambiguous than in the past, it will be increasingly difficult to measure the effects of discrimination on various outcomes.
Our goal in Part II is to move from a descriptive analysis of existing disparities (association) to consider methods of inferential analysis (causation), with a focus on determining the circumstances in which a racial disparity may be attributed, in whole or in part, to racial discrimination. The core measurement issues in which we are interested include the following:
-
measuring the incidence, causes, and effects of racial discrimination;
-
identifying appropriate units of analysis (individual or aggregate level);
-
identifying explanatory mechanisms that lead to discriminatory behaviors and institutional processes;
-
identifying mediating factors and processes that affect observed disparities;
-
measuring the extent or magnitude of discrimination within a do-
-
main, across domains, and over time; and
-
determining how much of an observed disparity is an effect of discrimination.
Our discussion of these issues is limited by our charge to focus on the measurement of racial discrimination. However, much of the discussion can be readily applied to measurement of closely related topics, such as gender or age discrimination.
There are many different methods for measuring racial discrimination. We review three types of methods that are widely used in various literatures: controlled laboratory experiments and field experiments; analysis of observational data and natural experiments; and measures of reported perceptions and experiences of discrimination from surveys and administrative records. It is important to note that no one method allows researchers to address all of the measurement issues listed above.
For example, laboratory experiments help researchers to identify the mechanisms that may lead to different forms of racial discrimination and the factors that mediate the expression of discriminatory attitudes and behaviors. Because of experimental control over relevant variables, researchers are able to identify whether race or the interaction of race and other factors triggers an expression of racial discrimination. Laboratory experiments are useful for drawing causal inferences at the individual level and important for identifying subtle mechanisms of discrimination; however, they do not directly address disparities in the aggregate. That is, laboratory effects do not often generalize to the broader population and can rarely tell us the extent to which naturally observed disparities are the result of discrimination.
The results of field experiments, on the other hand, are often more generalizable than the results of laboratory experiments. Although field experiments may involve less experimental control, researchers can use them to measure the extent of discrimination in a particular domain, such as the housing or labor market. For instance, audit studies in the housing or employment arena can provide useful information about the possible occurrence of discrimination by real estate agents against homebuyers or by employers against job applicants from disadvantaged racial groups.
Some ability to generalize may also be gained by using nonexperimental approaches. Researchers can use statistical modeling and estimation to analyze observational data and draw causal inferences. Statistical models are useful for identifying associations between race and different outcomes while controlling for other factors that may explain the observed outcomes. Simply identifying an association with race, however, is not equivalent to measuring the magnitude of racial discrimination or its contribution to differential outcomes by race. In most observational settings, the lack of ex-
perimental control and the inability to manipulate “treatment” variables make it difficult to dismiss alternative explanations of causation without relying on strong and often untestable assumptions.
One problem is that observational data often contain only a small set of characteristics and may not include variables that are important for explaining an observed effect or for modeling the process by which discrimination could occur. For instance, a finding of a large discriminatory effect within a domain (e.g., differential treatment in hiring leading to wage disparities) may be erroneous if it is not possible to control for other explanatory variables, such as motivation or skill level,1 or to develop an accurate statistical model of the decision process. Alternatively, a finding that discrimination at a certain point within a domain contributes little to an observed disparity may ignore the possible effects of how earlier discrimination may have accumulated over time; for example, discrimination in secondary education can affect skill levels and thereby affect subsequent wages (see Chapters 4 and 11).
Surveys also provide observational data to measure racial attitudes and reported experiences and perceptions of racial discrimination. But again, these data are rarely sufficient to establish causality, statistically or substantively. The most detailed observational studies are collections of case studies, which contain large amounts of information on small numbers of individuals or organizations. Such collections of case studies can produce the kinds of information on underlying behavioral processes needed to draw valid causal inferences, although their results may be limited in generalizability. Longitudinal survey data can be particularly helpful for understanding trends in racial attitudes and reported experiences and perceptions of discrimination and the extent to which racial disparities are a function of discrimination that occurs over time and across domains.
Determining how much of an observed outcome is an effect of racial discrimination is difficult. Translating effects from experimental data to what is observed in real situations is not easy. Moreover, it is much easier to assess the occurrence of discrimination at one point in a process than to identify effects of discrimination that occur earlier in a process or across relevant domains. A feasible solution to these difficulties may be to combine methods, using data and results from multiple sources. In the following five chapters, we describe issues and methods for research design, measurement, and analysis that together may allow researchers to identify and assess racial discrimination. When appropriate, each chapter contains conclusions and recommendations.
We begin in Chapter 5 by introducing a general framework for inferring causation between race-based discrimination and outcomes of interest. Racial disparities are often substantial and widely observed, but only rarely do researchers directly observe discriminatory behavior. To establish a causal relationship between race and discrimination, one would ideally vary the race of a single person and measure any differences in outcomes. Because doing so is impossible, researchers typically observe a disparate outcome and trace back through the process that generated it to determine whether racial discrimination had a causal effect. In other words, they attempt to answer retrospectively the counterfactual question of whether the outcome for a nonwhite individual would have been different if he or she had been white. In Chapter 5, we discuss how accumulated scientific evidence from both experimental and observational research may support causal conclusions and allow researchers to determine whether racial discrimination contributes causally to an observed racial disparity.
Controlled experiments, whose strengths are direct manipulation of experimental conditions and randomization, are ideal for drawing causal inferences and come closest to addressing the above counterfactual question. Chapter 6 describes two types of experimental methods—laboratory and field experiments—and their application to the measurement of racial discrimination. We describe the design, use, strengths, and limitations of experimental methods and provide key examples of laboratory and field studies used to measure racial discrimination.
In Chapter 7, we critically review the issues that must be addressed to draw valid causal inferences about racial discrimination from analyses of observational data. We first review the primary descriptive approach used in the literature on racial discrimination—decomposition of racial disparities. We then discuss the limitations of such descriptive analysis and the challenges of moving from description to inference by using statistical models (particularly regression models). We focus on the assumptions underlying statistical models and possible approaches to the problems of using such models to infer discrimination, one of which is to take advantage of natural experiments, which occur when a policy change targets discrimination in a particular domain or set of domains.
Our primary intent in Chapter 8 is to consider the use of observational data from surveys, in-depth interviews, and administrative records to measure the occurrence of racially discriminatory attitudes and behaviors and people’s reported perceptions of and experiences with discrimination. We provide extended reviews of major large-scale surveys of racial attitudes and reported perceptions and experiences of racial discrimination by white and black Americans.
Finally, in Chapter 9 we provide an example of the challenges of measuring racial discrimination in an important area of current concern, which
is racial profiling by law enforcement officials in which race alone or in combination with other variables is used to select individuals for further investigation (e.g., to stop motorists to search for illegal drugs). Profiling is a form of statistical discrimination. We discuss some of the methodological challenges of determining when racial profiling may be occurring, although these challenges are such that we are not able to identify the best measurement approaches.