Audit Studies and the Assessment of Discrimination
The Housing Discrimination Study (HDS) uses primarily audit studies to estimate overall-level discrimination against ethnic/racial groups. Overall-level discrimination occurs when there is ethnic/racial discrimination averaged across realtors, applicants (i.e., auditors), and circumstances. By realtor is meant housing realtors and other purveyors of housing; by circumstances is meant the circumstances of the contact between the auditor and the realtor. Overall-level discrimination is of course different from individual-level discrimination, in which particular auditors are discriminated against because of ethnic status. To simplify the following discussion, the term “black” is used to encompass racial minorities.
Discrimination here refers to adverse market discrimination, which can occur in two ways. Discrimination against blacks occurs when race = black is a direct reason for the realtor to produce a negative outcome or when race = black is used by the realtor as a surrogate for unobserved measures of (renter/buyer) suitability that vary in distribution across the racial groups. The former is direct discrimination, and the latter is statistical discrimination.
Realtors should strive to ascertain all qualification measures directly, including those that vary in distribution across racial groups. Realtors are
S.A. Murphy works for the Department of Statistics and Institute for Social Research at the University of Michigan.
not to use race as a surrogate for qualification measures. More work is needed to assist well-intentioned realtors (with an imperfect ability to assess qualifications) in avoiding the use of race as a surrogate for qualifications.
Since individual-level discrimination and overall-level discrimination are easily confused when discussing the utility of the audit model, the two are discussed in turn below.
THE AUDIT MODEL AND INDIVIDUAL-LEVEL DISCRIMINATION
Sometimes audit results are reported by audit pair (averaged over realtors and circumstances of audit visits). Audit results reported for a particular pair may be misconstrued as estimates of net or gross individual-level discrimination. However, even under extremely optimal conditions, audit results for a particular pair cannot be interpreted as estimates of individual-level discrimination. Suppose the audit coordinator is successful in matching the audit pair on all possible qualifications (that vary in distribution by race). One can expect there to be many individual characteristics that do not vary across the two racial groups (black/white), yet are used in assessing renter qualifications. These characteristics may or may not be truly indicative of one's qualifications. Moreover, since the characteristics are equivalently distributed across racial groups, their use does not constitute discrimination between racial groups. The individuals in the audit pair may not be equal on these characteristics; thus the realtor may treat the individuals in the audit pair unequally, yet no discrimination occurs. Fallibility in assessing qualifications may lead to this situation. To emphasize this observation more strongly, even if both of the members of the audit are white, it may be expected that audit results for one pair (averaged over realtors and circumstances of audit visits) would result in nonzero “estimates” of individual-level measures of discrimination.
The conclusion is that audit results presented by audit pair do not represent individual-level discrimination unless an extremely strong assumption holds. This assumption is that the auditors are matched on all qualifications and individual characteristics regardless of whether these qualifications/characteristics vary by race. This assumption appears impractical and unnecessary when the goal is to ascertain overall-level discrimination. It is difficult to see how this level of matching can be achieved in practice. Indeed, this level of matching (i.e., the test coordinator has
matched the members of the pair not only on all possible qualifications that vary in distribution by race, but also on all possible individual characteristics that vary in distribution by race) will not hold even in optimal settings. The economic agent may treat the members of the audit pair differently because of differing individual characteristics that do not vary by race. Of course, if the audit results for one audit pair are extremely gross, one may be inclined to believe that there are no individual characteristics/ qualifications that could have produced such a gross effect.
THE AUDIT MODEL AND OVERALL-LEVEL DISCRIMINATION
As long as audit pairs are matched on all qualifications that vary in distribution by race, audit results averaged over realtors, circumstances of the visits, and auditors can be viewed as an unbiased estimate of overall-level discrimination (i.e., average level of adverse market discrimination). This is because one averages overall individual auditor characteristics (as opposed to the case of individual-level discrimination, in which audit results are to be averaged only over realtors and circumstances of the visits).
The following is a quantitative explanation in which:
f is used to denote densities.
e is race (black/white).
ea is realtor (realty, etc.).
r(ea) is the realtor's legitimate applicant requirements/qualifications that vary in distribution by race (income requirements, credit requirements, etc.).
ic is individual auditor characteristics that do not vary in distribution by race. That is, f(ic|e) = f(ic).
ice is individual auditor characteristics that may vary in distribution by race and are not in r(ea). That is, f(ice|e,r(ea),ic) is not identically equal to f(ice|r(ea),ic). There is a dependence of f on r(ea) since applicant requirements may vary by realtor, and thus r(ea) may include more or less of the set of all individual auditor characteristics varying by race.
c is circumstances of the audit visit unconnected with the particular individual auditor (e.g., the apartment was rented in the meantime).
X is 1 if the realtor says the advertised apartment is available, 0 otherwise. X is indexed by all factors that contribute to a result of 1 or 0. X is an unknown nonparametric function: X(e,r(ea),ice, ic,ea,c).
Technically, X = X(e,r(ea),ice,ic,ea,c) may be viewed as a nonparametric structural equation in the terminology of Pearl (2000).
Suppose the audit pair is composed of one white and one black. The two individuals are assigned the same realtor (same ea) and are assigned or possess the same values as the realtor's legitimate requirements, r(ea). They have different individual characteristics, denoted by (ice′, ic′) and (ice′′, ic′′), respectively. Also, since they must visit the agent at different times, the circumstances will be different as well (c′ and c′′).
The net average or overall-level discrimination estimate is then found by averaging
X(e = w, r(ea),ice′,ic′,ea,c′) – X(e = b,r(ea),ice′′,ic′′,ea,c′)
over circumstances, realtors, and audit pairs. This results in an estimator for
Êƒ(X(w,r(ea),ice,ic,ea,c) f(ice|w,r(ea),ic) (1)
–X(b,r(ea),ice,ic,ea,c) f(ice|b, r(ea),ic)) f(c,ea) f(ic) d(ea,c,ice,ic),
where f(c,ea) is determined by the selection of advertisements and the visit times of the auditors (this density is the same for both members of the audit pair since they ask about the same advertisement, and the order of the visits is randomized); where f(ic) is determined by the selection of the auditors; and where f(ice|w,r(ea),ic) is determined by the realtor's suitability requirements and the selection of the auditors. In addition, f(ice|w,r(ea),ic) ≠ f(ice|b,r(ea),ic) since there are individual characteristics that vary in distribution between the racial groups.
The above difference is zero if X(e,r(ea),ice,ic,ea,c) is constant in both e and ice, that is, if the realtor's judgment of suitability does not directly employ race or employ race as a surrogate for individual characteristics (not in r(ea)) that vary in distribution by race. This is exactly what would be expected from a measure of overall discrimination.
Audit pairs need only be matched on all qualifications that vary in distribution by race to provide an unbiased estimate of average or overall discrimination. Audit pairs may be matched on other characteristics (to improve statistical power), but this is not necessary to produce an unbiased
estimator of net average or overall discrimination. Moreover, to accumulate information about discrimination, one needs many audit pairs. Multiple audits by audit pairs can be used to hold costs down, but one does not thereby accumulate information about discrimination. Information about discrimination is not accumulated because the members of the audit pair may differ in ic, individual characteristics that are distributed equally in the two races. Differential treatment of the members of the auditor pair due to ic is not adverse discrimination.
Improving the Matching on All Qualifications that Vary in Distribution by Race
Estimation of net overall discrimination requires that the audit pair be matched on all suitability qualifications that vary in distribution by race. It can be expected that these qualifications will vary by economic agent. Furthermore, it is unrealistic to expect the HDS to be aware of all of realtors' suitability qualifications. The HDS can assign qualifications to an audit pair in a way that depends on the realtors. First, the customary qualifications/requirements (such as creditworthiness, income, and employer/occupation) are assigned. Then, after audit pairs have been randomized to realtors and individuals within the audit pair randomized by order, the first auditor should record all questions asked and answers given that are not part of the assigned customary requirements. The test coordinator can then use the additional information requested of the first auditor to form a closer match by maintaining a consistent life story for the second auditor. Thus the realtor determines which individual characteristics (beyond the customary suitability requirements and race) will be matched.
Determining Whether the Overall Discrimination Effect Is a “Market-Level” Discrimination Effect
It is of low utility to discover that agents handling homes for which most blacks could not reasonably qualify are discriminatory. It is much more useful to discover that agents handling homes well within the reach of many blacks are discriminatory.
It appears that the distribution of the prices of the homes used in the HDS should match the distribution of creditworthiness, income, and employer/occupation among blacks rather than an overall average distribution of creditworthiness, income, and employer/occupation. This point is in
line with the idea that it is important to ascertain discrimination in situations in which blacks are qualified buyers. This is not the same as saying that the impact of discrimination is most important at the realties blacks choose to use, but that the impact of discrimination is most important at the realties that sell homes blacks are qualified to buy.
The above point can be seen from display (1), in which the differences
(X(w,r(ea),ice,ic,ea,c) f(ice|w,r(ea),ic) –
are weighed by the density f(c,ea). Thus large differences can be paired with a small f(c,ea) weight and vice versa, and so the definition of overall-level discrimination changes with the distribution of realtors, ea.
How difficult would it be to match the market to the qualifications of blacks; that is, to choose home advertisements (i.e., realtors) with probabilities proportional to the “appropriate” segment of the black population?
2000 Causality: Models, Reasoning, and Inference . New York : Cambridge University Press .