National Academies Press: OpenBook

Measuring Racial Discrimination (2004)

Chapter: PART II: METHODS

« Previous: 4 Theories of Discrimination
Suggested Citation:"PART II: METHODS." National Research Council. 2004. Measuring Racial Discrimination. Washington, DC: The National Academies Press. doi: 10.17226/10887.
×

Part II
Methods

In Part I, we defined the concepts of race and racial discrimination from a social science research perspective. The history of legal and institutionalized racial discrimination in the United States and the existence of widespread racial disparities in outcomes across domains prompt our review of methods for assessing the extent to which discrimination continues to affect historically disadvantaged racial and ethnic groups. Our definition of racial discrimination includes overt and subtle discriminatory behaviors and processes. If, as some have suggested, modern forms of discrimination are less likely to be direct and explicit and more likely to be indirect and ambiguous than in the past, it will be increasingly difficult to measure the effects of discrimination on various outcomes.

Our goal in Part II is to move from a descriptive analysis of existing disparities (association) to consider methods of inferential analysis (causation), with a focus on determining the circumstances in which a racial disparity may be attributed, in whole or in part, to racial discrimination. The core measurement issues in which we are interested include the following:

  • measuring the incidence, causes, and effects of racial discrimination;

  • identifying appropriate units of analysis (individual or aggregate level);

  • identifying explanatory mechanisms that lead to discriminatory behaviors and institutional processes;

  • identifying mediating factors and processes that affect observed disparities;

  • measuring the extent or magnitude of discrimination within a do-

Suggested Citation:"PART II: METHODS." National Research Council. 2004. Measuring Racial Discrimination. Washington, DC: The National Academies Press. doi: 10.17226/10887.
×

main, across domains, and over time; and

  • determining how much of an observed disparity is an effect of discrimination.

Our discussion of these issues is limited by our charge to focus on the measurement of racial discrimination. However, much of the discussion can be readily applied to measurement of closely related topics, such as gender or age discrimination.

There are many different methods for measuring racial discrimination. We review three types of methods that are widely used in various literatures: controlled laboratory experiments and field experiments; analysis of observational data and natural experiments; and measures of reported perceptions and experiences of discrimination from surveys and administrative records. It is important to note that no one method allows researchers to address all of the measurement issues listed above.

For example, laboratory experiments help researchers to identify the mechanisms that may lead to different forms of racial discrimination and the factors that mediate the expression of discriminatory attitudes and behaviors. Because of experimental control over relevant variables, researchers are able to identify whether race or the interaction of race and other factors triggers an expression of racial discrimination. Laboratory experiments are useful for drawing causal inferences at the individual level and important for identifying subtle mechanisms of discrimination; however, they do not directly address disparities in the aggregate. That is, laboratory effects do not often generalize to the broader population and can rarely tell us the extent to which naturally observed disparities are the result of discrimination.

The results of field experiments, on the other hand, are often more generalizable than the results of laboratory experiments. Although field experiments may involve less experimental control, researchers can use them to measure the extent of discrimination in a particular domain, such as the housing or labor market. For instance, audit studies in the housing or employment arena can provide useful information about the possible occurrence of discrimination by real estate agents against homebuyers or by employers against job applicants from disadvantaged racial groups.

Some ability to generalize may also be gained by using nonexperimental approaches. Researchers can use statistical modeling and estimation to analyze observational data and draw causal inferences. Statistical models are useful for identifying associations between race and different outcomes while controlling for other factors that may explain the observed outcomes. Simply identifying an association with race, however, is not equivalent to measuring the magnitude of racial discrimination or its contribution to differential outcomes by race. In most observational settings, the lack of ex-

Suggested Citation:"PART II: METHODS." National Research Council. 2004. Measuring Racial Discrimination. Washington, DC: The National Academies Press. doi: 10.17226/10887.
×

perimental control and the inability to manipulate “treatment” variables make it difficult to dismiss alternative explanations of causation without relying on strong and often untestable assumptions.

One problem is that observational data often contain only a small set of characteristics and may not include variables that are important for explaining an observed effect or for modeling the process by which discrimination could occur. For instance, a finding of a large discriminatory effect within a domain (e.g., differential treatment in hiring leading to wage disparities) may be erroneous if it is not possible to control for other explanatory variables, such as motivation or skill level,1 or to develop an accurate statistical model of the decision process. Alternatively, a finding that discrimination at a certain point within a domain contributes little to an observed disparity may ignore the possible effects of how earlier discrimination may have accumulated over time; for example, discrimination in secondary education can affect skill levels and thereby affect subsequent wages (see Chapters 4 and 11).

Surveys also provide observational data to measure racial attitudes and reported experiences and perceptions of racial discrimination. But again, these data are rarely sufficient to establish causality, statistically or substantively. The most detailed observational studies are collections of case studies, which contain large amounts of information on small numbers of individuals or organizations. Such collections of case studies can produce the kinds of information on underlying behavioral processes needed to draw valid causal inferences, although their results may be limited in generalizability. Longitudinal survey data can be particularly helpful for understanding trends in racial attitudes and reported experiences and perceptions of discrimination and the extent to which racial disparities are a function of discrimination that occurs over time and across domains.

Determining how much of an observed outcome is an effect of racial discrimination is difficult. Translating effects from experimental data to what is observed in real situations is not easy. Moreover, it is much easier to assess the occurrence of discrimination at one point in a process than to identify effects of discrimination that occur earlier in a process or across relevant domains. A feasible solution to these difficulties may be to combine methods, using data and results from multiple sources. In the following five chapters, we describe issues and methods for research design, measurement, and analysis that together may allow researchers to identify and assess racial discrimination. When appropriate, each chapter contains conclusions and recommendations.

1  

Goldsmith et al. (2000) provide a counterexample of research that explicitly includes measures of motivation.

Suggested Citation:"PART II: METHODS." National Research Council. 2004. Measuring Racial Discrimination. Washington, DC: The National Academies Press. doi: 10.17226/10887.
×

We begin in Chapter 5 by introducing a general framework for inferring causation between race-based discrimination and outcomes of interest. Racial disparities are often substantial and widely observed, but only rarely do researchers directly observe discriminatory behavior. To establish a causal relationship between race and discrimination, one would ideally vary the race of a single person and measure any differences in outcomes. Because doing so is impossible, researchers typically observe a disparate outcome and trace back through the process that generated it to determine whether racial discrimination had a causal effect. In other words, they attempt to answer retrospectively the counterfactual question of whether the outcome for a nonwhite individual would have been different if he or she had been white. In Chapter 5, we discuss how accumulated scientific evidence from both experimental and observational research may support causal conclusions and allow researchers to determine whether racial discrimination contributes causally to an observed racial disparity.

Controlled experiments, whose strengths are direct manipulation of experimental conditions and randomization, are ideal for drawing causal inferences and come closest to addressing the above counterfactual question. Chapter 6 describes two types of experimental methods—laboratory and field experiments—and their application to the measurement of racial discrimination. We describe the design, use, strengths, and limitations of experimental methods and provide key examples of laboratory and field studies used to measure racial discrimination.

In Chapter 7, we critically review the issues that must be addressed to draw valid causal inferences about racial discrimination from analyses of observational data. We first review the primary descriptive approach used in the literature on racial discrimination—decomposition of racial disparities. We then discuss the limitations of such descriptive analysis and the challenges of moving from description to inference by using statistical models (particularly regression models). We focus on the assumptions underlying statistical models and possible approaches to the problems of using such models to infer discrimination, one of which is to take advantage of natural experiments, which occur when a policy change targets discrimination in a particular domain or set of domains.

Our primary intent in Chapter 8 is to consider the use of observational data from surveys, in-depth interviews, and administrative records to measure the occurrence of racially discriminatory attitudes and behaviors and people’s reported perceptions of and experiences with discrimination. We provide extended reviews of major large-scale surveys of racial attitudes and reported perceptions and experiences of racial discrimination by white and black Americans.

Finally, in Chapter 9 we provide an example of the challenges of measuring racial discrimination in an important area of current concern, which

Suggested Citation:"PART II: METHODS." National Research Council. 2004. Measuring Racial Discrimination. Washington, DC: The National Academies Press. doi: 10.17226/10887.
×

is racial profiling by law enforcement officials in which race alone or in combination with other variables is used to select individuals for further investigation (e.g., to stop motorists to search for illegal drugs). Profiling is a form of statistical discrimination. We discuss some of the methodological challenges of determining when racial profiling may be occurring, although these challenges are such that we are not able to identify the best measurement approaches.

Suggested Citation:"PART II: METHODS." National Research Council. 2004. Measuring Racial Discrimination. Washington, DC: The National Academies Press. doi: 10.17226/10887.
×

This page intentionally left blank.

Suggested Citation:"PART II: METHODS." National Research Council. 2004. Measuring Racial Discrimination. Washington, DC: The National Academies Press. doi: 10.17226/10887.
×
Page 71
Suggested Citation:"PART II: METHODS." National Research Council. 2004. Measuring Racial Discrimination. Washington, DC: The National Academies Press. doi: 10.17226/10887.
×
Page 72
Suggested Citation:"PART II: METHODS." National Research Council. 2004. Measuring Racial Discrimination. Washington, DC: The National Academies Press. doi: 10.17226/10887.
×
Page 73
Suggested Citation:"PART II: METHODS." National Research Council. 2004. Measuring Racial Discrimination. Washington, DC: The National Academies Press. doi: 10.17226/10887.
×
Page 74
Suggested Citation:"PART II: METHODS." National Research Council. 2004. Measuring Racial Discrimination. Washington, DC: The National Academies Press. doi: 10.17226/10887.
×
Page 75
Suggested Citation:"PART II: METHODS." National Research Council. 2004. Measuring Racial Discrimination. Washington, DC: The National Academies Press. doi: 10.17226/10887.
×
Page 76
Next: 5 Causal Inference and the Assessment of Racial Discrimination »
Measuring Racial Discrimination Get This Book
×

Many racial and ethnic groups in the United States, including blacks, Hispanics, Asians, American Indians, and others, have historically faced severe discrimination—pervasive and open denial of civil, social, political, educational, and economic opportunities. Today, large differences among racial and ethnic groups continue to exist in employment, income and wealth, housing, education, criminal justice, health, and other areas. While many factors may contribute to such differences, their size and extent suggest that various forms of discriminatory treatment persist in U.S. society and serve to undercut the achievement of equal opportunity.

Measuring Racial Discrimination considers the definition of race and racial discrimination, reviews the existing techniques used to measure racial discrimination, and identifies new tools and areas for future research. The book conducts a thorough evaluation of current methodologies for a wide range of circumstances in which racial discrimination may occur, and makes recommendations on how to better assess the presence and effects of discrimination.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!