Measuring Housing Discrimination in a National Study: Report of a Workshop (2002)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

5 Developing a Model of Housing Discrimination CORE TREATMENT VARIABLES HDS 2000 is collecting data on core treatment variables with which analysts will measure change since the 1989 study (see Table 5-1). These variables include measures of terms and conditions, housing availability, and general sales effort on the part of the agent. Data on additional treat- ment variables collected from the sales audits are intended to capture changes in the housing market. These variables provide objective measures of how the auditor was treated during the housing transaction and are used to measure racially disparate treatment during the audit. HDS 2000 also includes new variables, not measured previously, re- garding financing assistance offered by the agent. This addition is intended to capture a shift in the housing market whereby real estate agents appear to be playing a much greater role in providing borrowers with mortgage infor- mation. Many agents prequalify borrowers for the type and amount of mortgage they can receive instead of referring the applicant to a lending institution. Since this process is completed before the agent shows the potential buyers available housing that meets their needs or desires, it pro- vides increased opportunity for racially disparate treatment. USE OF GROSS AND NET MEASURES In the discussion of the HDS model, Arthur Goldberger, Department of Economics, University of Wisconsin, suggested a modification of the 25 

26 MEASURING HOUSING DISCRIMINATION IN A NATIONAL STUDY TABLE 5-1 Core Treatment Variables Rental Only Rental and Sales Sales Only Terms and Conditions Housing Availability Financing Assistance • Application fee • Access denied: no • Assistance with financing required appointment or no volunteered • Special rental incentives unit available • Auditor told he/she is not offered • Advertised unit available qualified • Rent includes extra • Units similar to • Auditor told fixed-rate amenities advertised unit available conventional financing available • Auditor told adjustable-rate conventional financing available Sales Effort Sales Effort • Questions asked about • Follow-up phone call income • Questions asked about reasons for need to move • Invitation to call back NOTE: This is not an exhaustive list of core treatment variables in the 2000 HDS. SOURCE: The variables in the table are those collected in both the 1989 HDS audit and the 2000 HDS audit. model. When looking at population data rather than experimental data, researchers are interested in observing the frequency of adverse treatment for minorities in the housing market. The researchers proxy this quantity with a Bernoulli variable, Yit, that measures adverse treatment for minority and white auditors for auditor race i and test t: 1 if the individual was treated favorably  Y it =   0 if the individual was treated unfavorably The race-specific average of Yit gives the proportion of tests, for each race, in which auditors were treated favorably. The gross measure of discrimination is the proportion of tests in which the minority auditor was treated unfavorably and the white auditor was treated favorably, P10. The net measure of discrimination is the difference 

DEVELOPING A MODEL OF HOUSING DISCRIMINATION 27 between the gross measure (P10) and the proportion of tests in which the white auditor was treated unfavorably compared with the minority auditor, P01 (i.e., net measure = P10 – P01) (see also Chapter 4.) P01 is a proxy for the frequency of adverse treatment incidences against minorities that are unre- lated to race. P01 may be a poor proxy, however, if it includes deliberate reverse discrimination, which is subtracted out of the net measure. The discussion frequently returned to the need to clearly define the concept of discrimination. To find the correct measure of discrimination, participants contemplated a conceptual experiment in which auditors are matched perfectly on all observable characteristics and encounter com- pletely identical circumstances during their visit to a housing agent. Under these circumstances, the researchers believe the correct measure of the inci- dence of disparate treatment discrimination is the gross measure. One could measure both reverse racial discrimination (P01) and racial discrimi- nation (P10), although the latter is the quantity of interest. The Urban Institute researchers noted, however, that this conceptual experiment is unachievable. Additional discussion centered on the standard for housing market transactions, more specifically, the solutions for the joint probabilities in Table 5-2 in the absence of housing discrimination. Workshop partici- pants suggested that the Urban Institute should consider the solutions for Pij and their implications for the net and gross measures of adverse treat- ment. These solutions for varying levels of housing discrimination would help the Urban Institute assess whether the gross and net measures are adequately capturing discrimination in the market. While the discussion addressed this issue, of major concern was the measurement of discrimina- tion in the context of the population of interest and a clear definition of TABLE 5-2 Proportion of Auditors Receiving Favorable Treatment Minority White Favorable Unfavorable Favorable P11 P10a Unfavorable P01 P00 a Gross measure; The net measure is = P10 – P01. 

28 MEASURING HOUSING DISCRIMINATION IN A NATIONAL STUDY discrimination. Each of these issues is presented in separate sections of this report. Some participants expressed their preference for the net measure since it captures the difference in unfavorable treatment of minority and white testers. The gross measure will reveal the number of instances of discrimi- nation against minorities and may appear high; however, the frequency of these instances may be equivalent to that for whites. The net measure will capture this by calibrating the magnitude of the discrimination. Charles Manski, Board of Trustees Professor, Department of Econom- ics, Northwestern University, and Susan Murphy, Associate Professor, Sta- tistics Department, and Senior Associate Research Scientist, Survey Re- search Center, University of Michigan, also commented on the breadth of methodological issues in the 2000 HDS and the implications of these is- sues for measuring discrimination in the national housing market. Their comments included a discussion of the strengths and weaknesses of the current methodology and some alternative methodologies that could be applied. Manski’s discussion addressed measuring the severity or magnitude of discrimination rather than just the occurrence of discrimination. For ex- ample, the extent to which the characteristics of minority households must be altered so they appear more qualified than white households could serve as a measure of the magnitude of discrimination. During his comments, Manski also proposed that by collecting richer data, researchers could dis- tinguish between statistical and prejudicial discrimination. FACTORS AFFECTING HOUSING DISCRIMINATION The HDS focuses predominantly on economic and family-size charac- teristics. These attributes of the individual are expected to drive housing needs and thus the units shown or suggested to the auditor. The initial model posits that disparate treatment is due to the individual’s race and observable circumstances that could arise during the tester’s visit. During the workshop, Urban Institute researchers acknowledged an inability to match auditors on the myriad of possible unobservable characteristics. They stated that their goal was different: to structure a study that could test whether those unobservable characteristics really matter in racially differen- tial treatment of the auditors. Some participants raised questions about the power of the statistical tests being performed and the need to control for covariates even if the 

DEVELOPING A MODEL OF HOUSING DISCRIMINATION 29 paired-testing methodology appears to control for them. One argument for the use of covariates is that favorable or unfavorable treatment by hous- ing agents may depend on the sector of the housing market or type of transaction observed. The audit methodology results in identical agents observing auditors with similar characteristics. Including the covariates in the model would allow the researchers to observe how the estimated mar- ginal probabilities in Table 5-2 respond to this methodology. Another ques- tion raised during the workshop was whether the discussion of power for the statistical tests and the need to control for covariates is necessary in the absence of a clearly defined population. An appropriate model may be one that accounts for the measurement of outcomes that represent a mix of different measured phenomena. CHARACTERISTICS OF TESTER PAIRS Several participants expressed concern about the “actual” characteris- tics of auditors—those not assigned by the test coordinator—and their po- tential effect on the validity of the test. More specifically, participants asked how test coordinators ensure that the audit pair are believable potential renters or purchasers of the advertised housing unit. The discussion en- compassed whether auditors appear able to afford a particular housing unit, as well as how close an auditor’s actual residence is to the test site. An additional concern of workshop participants was heterogeneity among white testers, given that two such testers of differing ancestry may receive very different treatment by a housing agent. Participants suggested that the test coordinator be mindful of this heterogeneity when pairing white with minority testers. Otherwise, the result of the test may reflect not solely minority-white differences, but also the housing agent’s percep- tions, based on ethnicity or other factors, of a white applicant’s attractive- ness as a buyer or renter. In contrast with previous audits, testing agencies participating in HDS 2000 collect actual tester characteristics, such as income, level of education, employment experience, and testing experience. Tester training and test protocols are designed to limit the effect of variation among tester pairs. Participants stressed the importance of addressing the issue of heterogene- ity among the auditors, the housing units, and the housing agents. Hetero- geneity in any of these elements may have an impact on both the gross and net measures of adverse treatment. Sanders Korenman, Center for the Study of Business and Government, 

30 MEASURING HOUSING DISCRIMINATION IN A NATIONAL STUDY Baruch College, City University of New York, commented that auditors’ assigned characteristics should reflect the legal definition of discrimination. Researchers should control for attributes that provide legally allowable rea- sons to deny housing. Researchers may want the minority or white auditor to represent the subset of the minority or white population possessing those allowable characteristics. Korenman believes it would then be unnecessary to control for other differences correlated with race (e.g., language) if these differences are irrelevant to the housing transaction. Joseph Altonji, Department of Economics, Northwestern University, presented the following model for dealing with the above issues: y it = f ( x it , ε it , z it , ν it ; R i ) where i denotes the auditor, and t denotes the test. In this model, yit is the outcome measure representing favorable or unfavorable treatment (e.g., whether the auditor was shown the unit). The variable xit is a vector con- taining the characteristics of the auditor that are observed by or known to the researchers and are used to match audit pairs. It includes both assigned and nonassigned attributes, the latter having been collected by the re- searcher during the application process. The variable εit is a vector of char- acteristics of the auditor that are relevant to the agent’s assessment of the suitability of the auditor for the unit and are observed by the agent but not used to match audit pairs. The elements of εit vary across auditors and over time for a given auditor. Both xit and εit are limited to factors that are legitimate indicators of the suitability of the auditor for the housing unit and may legally be used by the auditor to make judgments. The variables zit and vit represent observed or known and unobserved or unknown charac- teristics of the unit that determine how the agent weighs the characteristics xit and εit of the auditor. Finally, the variable Ri denotes the race of the auditor. In terms of the model, a natural benchmark for discrimination is the situation in which race, Ri plays a role in the agent’s decision function given the characteristics of the unit z and v and the characteristics of the auditor x and ε. R will play a role in the auditor’s decisions if there is (1) institutional discrimination or racial preference, whether conscious or subconscious, on the part of the agent; and/or (2) the agent uses the race of the auditor to draw inferences about the suitability of the auditor for the unit, such as ability to pay the rent, maintain the unit, or get along with neighbors, or the degree of interest in the unit. Note that the housing provider may draw inferences about the auditor’s 

DEVELOPING A MODEL OF HOUSING DISCRIMINATION 31 suitability for the unit on the basis of the characteristics x and ε. However, if the housing provider uses race to draw any inferences about characteris- tics that are relevant to the housing transaction, he or she is discriminating. The audit methodology is to send auditors with the same value of xit to inquire about a housing unit. The fraction of times the outcome is favor- able for whites but not for non-whites is sometimes interpreted as a mea- sure of discrimination against non-whites. The fraction of times the out- come is favorable for non-whites but not for whites is sometimes interpreted as a measure of discrimination against whites. The sum of these two frac- tions is referred to as the gross discrimination rate. The difference between these two fractions is a measure of net discrimination against non-whites. The problem with the gross measure of discrimination is that random variation across testers in εit, differences in the distribution of εit that are related to race, and random variation in z and v between testor visits to a particular unit will lead to differences in the outcomes even though the audit pairs have been matched on xit. (Variation in z and v may arise, for example, from situational changes in the housing provider that occur be- tween the two audit visits, or different weights placed by a particular agent on the characteristics x and ε in the event the auditors see different agents.) That is, the gross measure of discrimination will be positive even if there is no discrimination, and R plays no role in the decision of any of the agents. Note that the variation in zit or in elements of εit that is observed by the researchers could be accounted for in analyzing the results of the audits. The problem with the net measure of discrimination against non-whites is that it will overstate discrimination to the extent that the values of the uncontrolled auditor characteristics εit are systematically related to race. The design and analysis of the audit studies should account for the differences among the auditors and housing providers that are reflected in εit, zit, and vit in the above model. Altonji offered four comments on how the Urban Institute could address heterogeneity in the study. First, re- searchers could look for differences in the outcomes of auditors of the same race who have visited similar housing units. This method would assess treatment outcomes within racial groups. Second, researchers could have individual auditors perform multiple tests involving similar units. This method would provide information about the influence of variation across auditors in εit on the distribution of outcomes. Third, auditors could per- form sandwich tests, in which auditors are sent on a test in triples, rather than pairs. The fourth comment is that more information should be gath- ered about the auditors even if it is not used to form matched pairs. Addi- 

32 MEASURING HOUSING DISCRIMINATION IN A NATIONAL STUDY tional steps should also be taken to gather preferences and characteristics relevant to housing providers. While the 2000 HDS has started to collect these data, more information could be gathered. From this information, the audit researchers could assess which characteristics are most important in matching auditors and assigning attributes. Researchers have considered using the information on the treatment of whites in all the audits to im- prove estimates of the treatment of whites. These estimates would increase the precision of the net adverse treatment measure. Murphy’s discussion of the methodological aspects of the 2000 HDS also addressed the interaction of auditor characteristics and the structure of audit pairs. She commented that, given the number of audit pairs and the number of visits per audit pair, researchers would not accumulate informa- tion within an audit pair because individual characteristics, which may not vary by race, persist across audit pairs. The resulting estimate of discrimi- nation obtained for these audit pairs may be due to individual characteris- tics that are equally distributed across race or due to discrimination. Pro- vided that researchers have matched testers on characteristics that matter to the housing providers, researchers can obtain better estimates of discrimi- nation by looking across tester pairs. APPLICATION OF SAMPLING WEIGHTS TO A MEASURE OF HOUSING DISCRIMINATION A secondary objective of the workshop was for participants to discuss the notion of preserving probability in the selection of advertisements by sampling with probabilities proportional to the size of the audit site. The Urban Institute uses classical population sampling to draw inferences about a population. It is not clear that application of these methods is necessary, however, since the study will not draw the usual theoretical inferences about population parameters. Rather than estimating a known population pa- rameter, the researchers are trying to estimate an underlying phenomenon that exists within the population. The underlying universe encompasses this conceptual model of discrimination and the character or prevalence of discrimination activities that occur in the interaction between two hypo- thetical individuals. There was considerable discussion during the workshop about the rel- evance of sampling weights to the analysis. For certain statistical analyses, weighting is important; however, many participants do not believe sample survey weights are relevant for the type of analysis the Urban Institute is 

DEVELOPING A MODEL OF HOUSING DISCRIMINATION 33 performing. The researchers argued for maintaining weights because ad- vertisements are stratified by weeks. During high-volume weeks, fewer tests are performed. If discriminatory agents represent a large proportion of advertisements during high-volume weeks, they will also be overrepre- sented in the sample. Not allowing for weighting of the advertisements will ignore the potential bias in the estimate. Altonji offered another suggestion for addressing weights. He sug- gested the Urban Institute weight the results using not the advertisements, but the characteristics of the housing unit. The audit results could then be compared with a national database containing the distribution and charac- teristics of the housing stock in the United States, namely occupancy or vacancy rates. The audit results could be weighted to reflect the expected availability of different housing stock in the market at a particular point in time. It was noted that if weighting is appropriate, approximate weights for the correct population are preferred over equal probability weights that are generated for the incorrect population. Workshop participants discussed the use of multiple newspapers in the original sampling frame instead of just in the pilot phase. Researchers from the Urban Institute expressed doubt about whether they had placed too much emphasis on the potential overlap in advertising and the fact that a single unit may be advertised in multiple newspapers. Analysts noted that the use of multiple newspapers could not be applied because the Phase I analysis of the 2000 HDS must remain comparable to the 1989 analysis. In discussing potential changes in the design of Phase II, workshop partici- pants suggested the analysts merge all newspaper advertisement sources. Fienberg noted that once the sample has been obtained, analysts can per- form the calculation two ways: (1) reweighting according to the sampling probabilities and (2) not reweighting or disregarding the potential overlap. Participants also discussed the feasibility of providing separate estimates for subsets of newspaper sources or for a clearly defined population of newspa- pers—for example, having the ability to estimate the likelihood of dis- crimination for the major newspapers in a particular area without concern for drawing inferences about the U.S. housing market. Several variations could be explored, including oversampling of underrepresented housing unit types. A recurring theme throughout the workshop was characterization of the housing market. Specification of the population of housing units has implications for the inferences drawn, as well as the appropriate weighting scheme. Workshop participants proposed that while the U.S. housing mar- 

34 MEASURING HOUSING DISCRIMINATION IN A NATIONAL STUDY ket is a candidate for the population, it may not reflect the true population of interest to the researchers. More specifically, if researchers are interested in discrimination against minority households, the population might be restricted to housing units in which this subgroup would be interested. The entire U.S. housing market may be the housing choice set of minority groups, or that set may be restricted to particular housing types. One proposal for restricting the housing market was to segment it by housing costs or affordability. METHODOLOGICAL IMPLICATIONS OF THE PHASE II DESIGN Tom Louis of the RAND Corporation addressed methodological im- plications of the Phase II design. He discussed the importance of identify- ing a set of primary goals for the study in a nonstatistical way. For instance, if the design includes the whole population, however defined, what sum- maries will be obtained, and what will they mean? Without being con- cerned with sample weights or statistical tests, what do the estimates mean, and do they provide the information needed? Once the proper estimates have been obtained and their meaning understood, the problem can be designed with the appropriate weights and statistical model. A premise of the audit design is that the survey design and weights can be extrapolated to a population. Inherent in the variables of interest is that these extrapola- tions capture contrasts in the population. The design should serve the objective of comparing treatment between white and minority home seek- ers. The weights will provide metropolitan-area estimates based on the distribution of advertisements within the sample relative to the population. Louis also discussed the importance of weights applied to the sample of advertisements. If the contrast in white and minority treatment mea- sured by some metric (e.g., the difference or odds ratio) has either no or low interaction with attributes used to form strata or sampling frames, the within-sample weights are adjusted. Louis addressed the design of later study phases in view of the findings from earlier phases. He suggested Phase II could serve the objective of providing reasonable estimates of the variance components associated with auditors, housing providers, and ad- vertisement sources. The later phases of the study would rely on explora- tion of the interactions between audit pairs and other methodological con- cerns identified in earlier phases. 

DEVELOPING A MODEL OF HOUSING DISCRIMINATION 35 Louis suggested that a more appropriate primary goal of the HDS might be to better understand transactions in the general housing market rather than to conduct a definitive study representing the population of housing market transactions. These statistical and policy-related decisions on the study design and objectives will determine how samples are allo- cated. Louis’s remarks also addressed matching of audit pairs and its impli- cations for the interpretation of audit results. He expressed concern about the large variance component for the matched pairs on the one hand and the inability to properly model tester heterogeneity on the other. He sug- gested that matching auditors on the wrong attributes—characteristics that have high variance components—could be worse than not attempting to match auditors at all. He did not suggest abandoning the matching of auditor pairs. Rather, he stressed matching on important attributes and formulating a model that would allow for the specification of covariance adjustments. As noted earlier, sandwich tests, in which two auditors of the same race view the advertised unit—one prior to and the other after the minority tester—can provide important information about differential treatment in housing transactions. Louis noted that similar information could be ob- tained without performing an actual sandwich test. By combining infor- mation within racial groups across audits for similar housing units, research- ers could explore variation within racial groups, particularly for matched characteristics. Analyses across audit sites could also provide information needed in low-population sites, such as underserved communities. For some sites, the definition of an underserved community restricts the study to small sample sizes. Louis proposed that a mix of design- and model- based analyses that incorporates results from various test sites could help in obtaining estimates within smaller sites. Participants did not offer defini- tive ways of addressing these issues, but noted the importance of raising them. Korenman presented several methodological implications of the Phase II design. He emphasized the need to assess the quality of an estimator with respect to how the researchers and other members of the housing community will use the measure. He also mentioned the importance of having a definition of discrimination and identifying what the study at- tempts to measure. He reiterated two uses of the latter: providing a bench- mark for racial discrimination in U.S. housing markets and identifying target communities for enforcement audits. 

36 MEASURING HOUSING DISCRIMINATION IN A NATIONAL STUDY Returning to an issue discussed earlier, Korenman also addressed which measure—gross or net adverse treatment—is most appropriate for estimat- ing discrimination. He noted that while the objective is not to provide one measure, but rather various components of an overall benchmark, each component should be a credible and reliable estimate. He noted the im- portance of having the gross and net measures capture the desired phenom- enon and move in directions consistent with what is known about housing discrimination from other sources. One aspect of this issue is the need to measure adverse treatment relative to the legal definition of discrimination or adverse treatment. Korenman stated that, consistent with the legal defi- nition, researchers could assign profiles and match testers on attributes that constitute legal bases for differential treatment. Korenman also expressed the need for a better understanding of the processes that generate variation across time and space in the measurement of housing discrimination. He did not propose that such analysis be added to the scope of the HDS, but observed that the issues involved are impor- tant and call for some caution in interpreting results. Korenman commented as well on the proposed remedies for selection bias in the newspaper sampling methodology. In addition to underrep- resented areas, the sampling frame may underrepresent housing unit types (e.g., rent control units). The modified sampling frame would still miss some unit types. Participants discussed capturing available housing stock by linking vacancy rates with actual rentals or turnovers to buttress the newspaper selection methodology. Korenman commented on the screen- ing call, in which a white tester calls about the housing unit to determine whether it is still available. He asked what information is retained from such calls and whether researchers could test to see whether the race of the auditor making the initial screening call matters. Finally, participants discussed the implications of changes in demo- graphics for the legal definition of discrimination and the audit methodol- ogy. Some participants commented on the basis of casual observations that discrimination against whites may be more prevalent in some high-minor- ity housing markets. Also, in some housing markets where whites are a small minority of the population, white-minority testing may not make sense; rather, it may be more appropriate to pair a second- or third-genera- tion Hispanic or Asian auditor with an African American auditor. These multiracial and multiethnic pairs may be more reflective of the actual hous- ing search pattern in these types of communities. The 2000 census repre- sents the first time respondents could multiply identify on race and 

DEVELOPING A MODEL OF HOUSING DISCRIMINATION 37 ethnicity on a full national scale. Data obtained from the census may indicate potential modifications to the paired-testing methodology. Par- ticipants raised the issues of (1) how to measure discrimination in housing markets with changing demographics, and (2) whether sending individual auditors as opposed to pairs of auditors representing a household would better capture the housing market.