 |
Questions? Call 888-624-8373 |
|
|
|
|
|
 |
|
|
|
The following HTML text is provided to enhance online
readability. Many aspects of typography translate only awkwardly to HTML.
Please use the page image
as the authoritative form to ensure accuracy.
Page 15
3
Defining the Population of
Interest
A significant portion of the workshop discussion focused on
identifying a probabilistic econometric model for identifying the
population of interest consistent with HUD's objectives.
Participants suggested that before assessing whether the current
model is appropriate for the parameters of interest, some important
questions need to be addressed:
TARGET POPULATION
During the workshop, the sponsors defined the target population
as the housing market in the test sites. Participants noted that
the target population differs from the population suggested by
the sampling frame, and posed the question, What is the correct
definition for the population given the sampling frame? The
researchers responded that the actual population is the housing
stock served by advertisements appearing in a site's major
metropolitan newspaper on Sunday.
|
|
|
|
|
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 15
Page 15
3
Defining the Population of
Interest
A significant portion of the workshop discussion focused on
identifying a probabilistic econometric model for identifying the
population of interest consistent with HUD's objectives.
Participants suggested that before assessing whether the current
model is appropriate for the parameters of interest, some important
questions need to be addressed:
What is the model for the outcomes?
What is actually being sampled?
Given the sample, what is the role of sampling
weights? How does the analysis move from paired
events to a universe of populations?
What is the population to which one can generalize
from such events?
How does that universe relate to the housing market?
TARGET POPULATION
During the workshop, the sponsors defined the target population
as the housing market in the test sites. Participants noted that
the target population differs from the population suggested by
the sampling frame, and posed the question, What is the correct
definition for the population given the sampling frame? The
researchers responded that the actual population is the housing
stock served by advertisements appearing in a site's major
metropolitan newspaper on Sunday.
OCR for page 16
Page 16
Participants suggested an alternative sampling frame that would
resemble the target population more closely. This sample would include
all known newspapers with housing advertisements circulating in the
test site. With this method, researchers would lose a priori knowledge
of the selection probability for a given advertisement. However, this
limited knowledge may not be necessary since the true probabilities
can be computed and weighted appropriately. This alternative sampling
frame would yield unequal probabilities of selection, but participants
did not view this as a major limitation.
Another alternative, raised by Tom Louis of the RAND Corporation,
would be to abandon the sample survey goal in favor of an alternative
unbiased selection procedure. The objective would be to select
neighborhoods or communities in which advertising is prevalent without
the goal of selecting a random sample. This method would ignore the
sampling procedure and would not be concerned with sample survey
weights or statistical comparisons within samples. Louis also noted
that the reliance on advertisements produces selection bias since part
of the market is missed (e.g., availability that is
“advertised” through word of mouth). The researchers
responded that pilot tests in the current study design include
nonadvertisement sampling that begins to address this issue by
identifying areas that are either protected (e.g., gated communities)
or neglected (e.g., small communities that do not typically advertise
available housing units in newspapers). Field reconnaissance in these
areas provides researchers with available housing stock from which a
secondary sample is drawn.
Participants discussed several ways to make the sampling frame more
realistic. For example, one could look at the income and asset
distribution of the minority population in individual metropolitan
areas and sample housing units on the basis of that distribution.
Urban Institute researchers pointed out that this sampling frame is
quite different from one in which the sample of units is based on the
distribution of where minorities currently live—an alternative
approach offered during the discussion of auditing in underserved
communities (see Chapter 6). The researchers
expressed concern about this latter modification because it would
institutionalize outcomes that may be the result of discrimination.
Participants were asked to discuss ways of making the sample more
representative of the target populations, but in a neutral manner with
respect to potential variants existing in the housing market. There
was substantial agreement that this would be extremely difficult to do
unless HUD were able to clarify the population to which it wants to
generalize. The
OCR for page 17
Page 17
alternatives to the sampling frame discussed by participants included
some that represent an attempt to overcome the limitations of the
paired-testing methodology and allow alternative forms of
generalization. Participants agreed that the current methodology poses
many difficulties when applied to underserved communities, in which
the paradigm does not appear to work well given differences in housing
market structures. For example, it is highly unlikely a white
individual would seek housing in some small, predominantly minority
community.
POTENTIAL BIAS IN TEST SITE SELECTION
Workshop participants noted that many larger metropolitan areas
have newspapers targeted to particular ethnic communities. The
housing search patterns for people in some communities, especially
minority groups, may not include advertisements in the major Sunday
newspaper. In such cases, the initial sampling frame would not
encompass advertisements more likely to be read by the
subpopulation of interest, a limitation that would have
implications for the credibility of the point of entry into the
housing market that the researchers were trying to establish.
Furthermore, omitting certain newspapers from the sampling frame
could compromise the ability to draw inferences about the
population the auditors represent.
The researchers noted that the sampling frame will be modified
during the pilot phase of the study to include alternative
advertisement sources, such as small neighborhood papers. For five
specific sites, two types of enhancements to the usual newspaper
advertisement sample are being used.
The first type of enhancement involves exploring the overall
distribution of newspaper advertisements independently for rentals
and sales, with the audit researchers looking at the relative
distribution of those advertisements across neighborhoods within an
audit site. Researchers will assess the multisource enhancements
used in the pilot studies and develop a sampling frame for
selecting advertisements in Phase II of the study. In Phase II,
researchers will draw from a multiframe sampling source, using
multiple neighborhood or community newspapers. However, sampling
issues arise from multiple newspaper advertisement sources. In
particular, the sponsors asked the workshop participants to discuss
the implications for the sampling frame of the potential overlap in
housing units—units that are sampled more than once because
they are advertised in several papers.
Some participants questioned why multiple newspaper sources were
not included in the Phase I study. The researchers responded that
they
OCR for page 18
Page 18
wished to preserve known probabilities of selection for the
advertisements. Devising a method to calculate the sampling
probabilities would have been daunting given multiple newspapers and
the fact that on a given day, advertisements must be drawn from 22
sites. In addition, the researchers wanted to mirror the 1989 sampling
procedure. They acknowledged the limitations of this method.
Researchers noted further that the Phase I sampling frame was chosen
to be comparable with the 1989 HDS. The newspaper methodology used for
the 1989 and 2000 HDS to select advertised housing units does not
reach the entire housing market. In fact, many minorities often do not
seek housing opportunities in areas that do not receive a large amount
of newspaper coverage. In addition, there are issues within minority
neighborhoods and other types of areas regarding obtaining access to
the housing stock that is available for both rental and sales. In some
instances, it is unlikely a majority individual would seek housing in
certain minority neighborhoods. In other instances, housing units may
not be advertised in mainstream or community neighborhoods.
Participants also discussed issues involved in testing in smaller
metropolitan areas, some of which are joint rural-type counties.
The second enhancement of the sample involves obtaining census
estimates for the proportion of available rental and sales housing for
each test site. During the initial visit, auditors will ask the
housing agent for the addresses of similar units. The resulting
information will yield an auxiliary set of addresses to be used in
identifying underrepresented areas that never reach the major
metropolitan newspapers. Census housing market data will be matched
with the auxiliary addresses. The researchers will then sample from
these underrepresented areas—areas with fewer advertisements
than available housing units—which are likely to be in minority
communities.
DRAWING INFERENCES TO THE POPULATION OF
INTEREST
In assigning tester characteristics, researchers guarantee that
auditors are qualified for the housing units. Workshop participants
suggested that by allowing for individuals who are on the margin
financially, analysts may miss the discrimination of interest.
Specifically, some participants believe that pairs of clearly
qualified and clearly unqualified home seekers of differing races
would receive similar treatment (either acceptance or rejection),
but that on the margin, housing agents would make more subjective
judg
OCR for page 19
Page 19
ments and exercise greater discretion by offering additional
assistance or compensation to the nonminority tester. By representing
only eminently qualified testers, therefore, the audit results may
understate the real degree of housing discrimination in the market.
The sponsors responded that they believe discrimination may occur for
both marginal and overqualified individuals and that both forms of
discrimination are important to measure. The strategy of the study,
however, is to identify clear and convincing disparate treatment when
less discretion on the part of housing agents is allowed.
Participants discussed whether the population from which the auditors
are drawn is a fair representation of the people who are seeking
housing, in other words, whether the audit findings can be used to
generalize about the discrimination a typical couple would face in the
housing market. Participants suggested that unobservable
characteristics of the testers may represent differences from the
population of home seekers and that these characteristics may not be
eliminated through training or protocols. However, as Stephen Ross
noted, audit pairs conduct more than one audit. Since many of the
pairs are fixed in terms of the people who make them up, their
experiences in different settings allow researchers to test for
whether unobservable tester characteristics matter. This issue is
discussed further in Chapter 5.
Ross noted that the researchers have a difficult problem in dealing
with very small sample sizes. HUD wants not only a national measure of
housing discrimination and changes in the national incidence of
discrimination, but also individual metropolitan measures. The audit
methodology will produce only 70 samples. Accordingly, the research
team will explore different options for obtaining exact estimates,
such as permutation-type estimators or estimators that are corrected
for small sample sizes. Such methods were not discussed during the
workshop, but Ross (2000) notes that because permutation tests rely on
frequency data, weights cannot be used.
WEIGHTING ADVERTISED UNITS
Participants noted it may be impractical to sample every newspaper
in a large metropolitan area, but, as noted earlier, researchers
are exploring the use of multiple papers and other advertisement
resources in smaller areas, where a multiframe sampling source is
more practical. The Urban Institute can use data gathered on the
relationship between the sample and the popu
OCR for page 20
Page 20
lation for these smaller areas to provide information about potential
differences in the incidence of discrimination across advertisement
sources.
Some participants were unclear on the appropriate weight that should
be applied to the results, but noted that it should reflect the
universe of available housing stock. Weighting audit results by
housing unit size may not be particularly relevant, but weights should
incorporate the various sources of information on housing
availability, including major metropolitan newspapers, community
newspapers, Internet-related databases, the Realtors® database (or
MultiList), and other sources. Participants discussed the implications
of accessing a fairly comprehensive database for available sales
housing, such as the MultiList available to real estate agents.
Perhaps such a list would serve as a base against which researchers
could sample units and create approximate weights. Although
participants did not know of the feasibility of this option, one
stated it is more appropriate to obtain approximate weights for the
right population than precise equal probability weights for the wrong
population. Each specification has associated tradeoffs that should be
considered before results are reported.
Representative terms from entire chapter:
housing market
Items in cart [0]
