National Academies Press: OpenBook

Measuring Housing Discrimination in a National Study: Report of a Workshop (2002)

Chapter: 3 Defining the Population of Interest

« Previous: 2 Overview of the 2000 Housing Discrimination Study
Suggested Citation:"3 Defining the Population of Interest." National Research Council. 2002. Measuring Housing Discrimination in a National Study: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10311.
×

Page 15

3

Defining the Population of Interest

A significant portion of the workshop discussion focused on identifying a probabilistic econometric model for identifying the population of interest consistent with HUD's objectives. Participants suggested that before assessing whether the current model is appropriate for the parameters of interest, some important questions need to be addressed:

  • What is the model for the outcomes?

  • What is actually being sampled?

  • Given the sample, what is the role of sampling weights? How does the analysis move from paired events to a universe of populations?

  • What is the population to which one can generalize from such events?

  • How does that universe relate to the housing market?

TARGET POPULATION

During the workshop, the sponsors defined the target population as the housing market in the test sites. Participants noted that the target population differs from the population suggested by the sampling frame, and posed the question, What is the correct definition for the population given the sampling frame? The researchers responded that the actual population is the housing stock served by advertisements appearing in a site's major metropolitan newspaper on Sunday.

Suggested Citation:"3 Defining the Population of Interest." National Research Council. 2002. Measuring Housing Discrimination in a National Study: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10311.
×

Page 16

Participants suggested an alternative sampling frame that would resemble the target population more closely. This sample would include all known newspapers with housing advertisements circulating in the test site. With this method, researchers would lose a priori knowledge of the selection probability for a given advertisement. However, this limited knowledge may not be necessary since the true probabilities can be computed and weighted appropriately. This alternative sampling frame would yield unequal probabilities of selection, but participants did not view this as a major limitation.

Another alternative, raised by Tom Louis of the RAND Corporation, would be to abandon the sample survey goal in favor of an alternative unbiased selection procedure. The objective would be to select neighborhoods or communities in which advertising is prevalent without the goal of selecting a random sample. This method would ignore the sampling procedure and would not be concerned with sample survey weights or statistical comparisons within samples. Louis also noted that the reliance on advertisements produces selection bias since part of the market is missed (e.g., availability that is “advertised” through word of mouth). The researchers responded that pilot tests in the current study design include nonadvertisement sampling that begins to address this issue by identifying areas that are either protected (e.g., gated communities) or neglected (e.g., small communities that do not typically advertise available housing units in newspapers). Field reconnaissance in these areas provides researchers with available housing stock from which a secondary sample is drawn.

Participants discussed several ways to make the sampling frame more realistic. For example, one could look at the income and asset distribution of the minority population in individual metropolitan areas and sample housing units on the basis of that distribution. Urban Institute researchers pointed out that this sampling frame is quite different from one in which the sample of units is based on the distribution of where minorities currently live—an alternative approach offered during the discussion of auditing in underserved communities (see Chapter 6). The researchers expressed concern about this latter modification because it would institutionalize outcomes that may be the result of discrimination.

Participants were asked to discuss ways of making the sample more representative of the target populations, but in a neutral manner with respect to potential variants existing in the housing market. There was substantial agreement that this would be extremely difficult to do unless HUD were able to clarify the population to which it wants to generalize. The

Suggested Citation:"3 Defining the Population of Interest." National Research Council. 2002. Measuring Housing Discrimination in a National Study: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10311.
×

Page 17

alternatives to the sampling frame discussed by participants included some that represent an attempt to overcome the limitations of the paired-testing methodology and allow alternative forms of generalization. Participants agreed that the current methodology poses many difficulties when applied to underserved communities, in which the paradigm does not appear to work well given differences in housing market structures. For example, it is highly unlikely a white individual would seek housing in some small, predominantly minority community.

POTENTIAL BIAS IN TEST SITE SELECTION

Workshop participants noted that many larger metropolitan areas have newspapers targeted to particular ethnic communities. The housing search patterns for people in some communities, especially minority groups, may not include advertisements in the major Sunday newspaper. In such cases, the initial sampling frame would not encompass advertisements more likely to be read by the subpopulation of interest, a limitation that would have implications for the credibility of the point of entry into the housing market that the researchers were trying to establish. Furthermore, omitting certain newspapers from the sampling frame could compromise the ability to draw inferences about the population the auditors represent.

The researchers noted that the sampling frame will be modified during the pilot phase of the study to include alternative advertisement sources, such as small neighborhood papers. For five specific sites, two types of enhancements to the usual newspaper advertisement sample are being used.

The first type of enhancement involves exploring the overall distribution of newspaper advertisements independently for rentals and sales, with the audit researchers looking at the relative distribution of those advertisements across neighborhoods within an audit site. Researchers will assess the multisource enhancements used in the pilot studies and develop a sampling frame for selecting advertisements in Phase II of the study. In Phase II, researchers will draw from a multiframe sampling source, using multiple neighborhood or community newspapers. However, sampling issues arise from multiple newspaper advertisement sources. In particular, the sponsors asked the workshop participants to discuss the implications for the sampling frame of the potential overlap in housing units—units that are sampled more than once because they are advertised in several papers.

Some participants questioned why multiple newspaper sources were not included in the Phase I study. The researchers responded that they

Suggested Citation:"3 Defining the Population of Interest." National Research Council. 2002. Measuring Housing Discrimination in a National Study: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10311.
×

Page 18

wished to preserve known probabilities of selection for the advertisements. Devising a method to calculate the sampling probabilities would have been daunting given multiple newspapers and the fact that on a given day, advertisements must be drawn from 22 sites. In addition, the researchers wanted to mirror the 1989 sampling procedure. They acknowledged the limitations of this method.

Researchers noted further that the Phase I sampling frame was chosen to be comparable with the 1989 HDS. The newspaper methodology used for the 1989 and 2000 HDS to select advertised housing units does not reach the entire housing market. In fact, many minorities often do not seek housing opportunities in areas that do not receive a large amount of newspaper coverage. In addition, there are issues within minority neighborhoods and other types of areas regarding obtaining access to the housing stock that is available for both rental and sales. In some instances, it is unlikely a majority individual would seek housing in certain minority neighborhoods. In other instances, housing units may not be advertised in mainstream or community neighborhoods. Participants also discussed issues involved in testing in smaller metropolitan areas, some of which are joint rural-type counties.

The second enhancement of the sample involves obtaining census estimates for the proportion of available rental and sales housing for each test site. During the initial visit, auditors will ask the housing agent for the addresses of similar units. The resulting information will yield an auxiliary set of addresses to be used in identifying underrepresented areas that never reach the major metropolitan newspapers. Census housing market data will be matched with the auxiliary addresses. The researchers will then sample from these underrepresented areas—areas with fewer advertisements than available housing units—which are likely to be in minority communities.

DRAWING INFERENCES TO THE POPULATION OF INTEREST

In assigning tester characteristics, researchers guarantee that auditors are qualified for the housing units. Workshop participants suggested that by allowing for individuals who are on the margin financially, analysts may miss the discrimination of interest. Specifically, some participants believe that pairs of clearly qualified and clearly unqualified home seekers of differing races would receive similar treatment (either acceptance or rejection), but that on the margin, housing agents would make more subjective judg

Suggested Citation:"3 Defining the Population of Interest." National Research Council. 2002. Measuring Housing Discrimination in a National Study: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10311.
×

Page 19

ments and exercise greater discretion by offering additional assistance or compensation to the nonminority tester. By representing only eminently qualified testers, therefore, the audit results may understate the real degree of housing discrimination in the market. The sponsors responded that they believe discrimination may occur for both marginal and overqualified individuals and that both forms of discrimination are important to measure. The strategy of the study, however, is to identify clear and convincing disparate treatment when less discretion on the part of housing agents is allowed.

Participants discussed whether the population from which the auditors are drawn is a fair representation of the people who are seeking housing, in other words, whether the audit findings can be used to generalize about the discrimination a typical couple would face in the housing market. Participants suggested that unobservable characteristics of the testers may represent differences from the population of home seekers and that these characteristics may not be eliminated through training or protocols. However, as Stephen Ross noted, audit pairs conduct more than one audit. Since many of the pairs are fixed in terms of the people who make them up, their experiences in different settings allow researchers to test for whether unobservable tester characteristics matter. This issue is discussed further in Chapter 5.

Ross noted that the researchers have a difficult problem in dealing with very small sample sizes. HUD wants not only a national measure of housing discrimination and changes in the national incidence of discrimination, but also individual metropolitan measures. The audit methodology will produce only 70 samples. Accordingly, the research team will explore different options for obtaining exact estimates, such as permutation-type estimators or estimators that are corrected for small sample sizes. Such methods were not discussed during the workshop, but Ross (2000) notes that because permutation tests rely on frequency data, weights cannot be used.

WEIGHTING ADVERTISED UNITS

Participants noted it may be impractical to sample every newspaper in a large metropolitan area, but, as noted earlier, researchers are exploring the use of multiple papers and other advertisement resources in smaller areas, where a multiframe sampling source is more practical. The Urban Institute can use data gathered on the relationship between the sample and the popu

Suggested Citation:"3 Defining the Population of Interest." National Research Council. 2002. Measuring Housing Discrimination in a National Study: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10311.
×

Page 20

lation for these smaller areas to provide information about potential differences in the incidence of discrimination across advertisement sources.

Some participants were unclear on the appropriate weight that should be applied to the results, but noted that it should reflect the universe of available housing stock. Weighting audit results by housing unit size may not be particularly relevant, but weights should incorporate the various sources of information on housing availability, including major metropolitan newspapers, community newspapers, Internet-related databases, the Realtors® database (or MultiList), and other sources. Participants discussed the implications of accessing a fairly comprehensive database for available sales housing, such as the MultiList available to real estate agents. Perhaps such a list would serve as a base against which researchers could sample units and create approximate weights. Although participants did not know of the feasibility of this option, one stated it is more appropriate to obtain approximate weights for the right population than precise equal probability weights for the wrong population. Each specification has associated tradeoffs that should be considered before results are reported.

Suggested Citation:"3 Defining the Population of Interest." National Research Council. 2002. Measuring Housing Discrimination in a National Study: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10311.
×
Page 15
Suggested Citation:"3 Defining the Population of Interest." National Research Council. 2002. Measuring Housing Discrimination in a National Study: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10311.
×
Page 16
Suggested Citation:"3 Defining the Population of Interest." National Research Council. 2002. Measuring Housing Discrimination in a National Study: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10311.
×
Page 17
Suggested Citation:"3 Defining the Population of Interest." National Research Council. 2002. Measuring Housing Discrimination in a National Study: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10311.
×
Page 18
Suggested Citation:"3 Defining the Population of Interest." National Research Council. 2002. Measuring Housing Discrimination in a National Study: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10311.
×
Page 19
Suggested Citation:"3 Defining the Population of Interest." National Research Council. 2002. Measuring Housing Discrimination in a National Study: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10311.
×
Page 20
Next: 4 Defining Housing Discrimination »
Measuring Housing Discrimination in a National Study: Report of a Workshop Get This Book
×
Buy Paperback | $29.00 Buy Ebook | $23.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Federal law prohibits housing discrimination on the basis of seven protected classes including race. Despite 30 years of legal prohibition under the Fair Housing Act, however, there is evidence of continuing discrimination in American housing, as documented by several recent reports. In 1998, the Department of Housing and Urban Development (HUD) funded a $7.5 million independently conducted Housing Discrimination Survey (HDS) of racial and ethnic discrimination in housing rental, sales, and lending markets (Public Law 105-276). This survey is the third such effort sponsored by HUD. Its intent is to provide a detailed understanding of the patterns of discrimination in housing nationwide.

In 1999, the Committee on National Statistics (CNSTAT) of the National Research Council (NRC) was asked to review the research design and analysis plan for the 2000 HDS and to offer suggestions about appropriate sampling and analysis procedures. The review took the form of a workshop that addressed HUD's concerns about the adequacy of the sample design and analysis plan, as well as questions related to the measurement of various aspects of discrimination and issues that might bias the results obtained. The discussion also explored alternative methodologies and research needs. In addition to addressing methodological and substantive issues related specifically to the HDS, the workshop examined broader questions related to the measurement of discrimination.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!