Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.

Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 48

48
of Table 14. As with the raw data, both the median and mean mended that utilization also be calculated for prime contracts
standardized DBE goals--overall and race-conscious--are and subcontracts combined in order to provide a fuller picture
larger among those state DOTs that used disparity or availabil- of DBE participation relative to all contract and subcontract
ity studies to help in the goal-setting process. spending.167
It is also worth noting that for both FFY 2006 and FFY 2007, Some studies did not have complete subcontractor data.
no state DOT that relied upon a disparity or availability study The usefulness of their utilization (and disparity) statistics is
to assist in its DBE goal setting had an entirely race-neutral correspondingly reduced.
DBE goal.163 Few studies provided any utilization statistics by NAICS
sector or subsector; omitting such detailed industry analy-
sis is a practice that has been viewed negatively by some
State DOT Utilization Analyses
courts.168
Recommended Approach. A high-quality utilization
and disparity analysis requires a well-constructed project
State DOT Disparity Analyses
database detailing several years of past contract and subcon-
tract activity. It is important that the database capture sub- Recommended Approach. A disparity analysis of state
contracting activity for both DBEs and non-DBEs.164 DOT spending is simply a comparison of DBE utilization to
Utilization and disparity statistics are most informative when DBE availability. Therefore, the preceding discussions of mar-
they are disaggregated along the following four key dimen- ket definition, availability measurement, and utilization sta-
sions, including: tistics are all relevant to the ability to produce a useful disparity
analysis. The primary analysis should be conducted on proj-
· Race, ethnicity, and gender, as well as more aggregated ects or business activities that were generally not subject to
groupings--minorities, white women, and all DBEs; race-conscious contracting requirements. This means state
· Contracts with DBE goals versus contracts without DBE DOT contracts without DBE goals, or the utilization of DBEs
goals;165 in the surrounding economy or, preferably, both. Proper test-
· Major procurement categories (i.e., construction and ing for substantive and statistical significance must be per-
construction-related professional services); and formed in order to identify whether disparities are large and
· NAICS sectors (two-digit) and subsectors (three-digit), whether the observed disparities could have arisen due to ran-
and industry groups (four-digit). dom chance alone.
Disaggregating by highway districts can also be useful,
Data Impacted by Race-Conscious
depending on the state DOT's needs.
Contracting Requirements
All the disparity studies reviewed provided utilization statis-
tics by race, ethnicity, and gender as well as for the aggregated As already explained, performing a disparity analysis on
groupings of MBE, WBE, and DBE.166 Most studies provided contracting and subcontracting dollars that were subject to
comparisons of federally assisted contracts to non-federally race-conscious affirmative action requirements, as is often
assisted contracts or otherwise attempted to provide compar- the case for state DOTs, is of limited value in strict scrutiny
isons of DBE utilization on projects with goals to projects analysis. Much more informative is disparity analysis on con-
without goals. All of the studies examined utilization by major tracting and subcontracting dollars that were not subject to
procurement categories. affirmative action requirements, at the state DOT, economy-
Most studies conducted separate utilization analyses for wide, or both.
prime contracts versus subcontracts. In such cases, it is recom- Some studies performed disparity analyses only on state
DOT contracts and subcontracts that were subject to race-
163See line 9 in Table 14, above. This changed in FFY 2008 with the introduction
of a completely race-neutral goal at NDOT subsequent to completion of its dis-
parity study. 167See 49 C.F.R. § 26.45(a) (goals are set on DOT-assisted contracts, not just the
164See Step A--Create a Database of Representative, Recent, and Complete State subcontract portion of DOT-assisted contracts); Northern Contracting, Inc. v.
DOT Projects, for an extended discussion of how to build such a database. Appen- Illinois Department of Transportation, 2005 U.S. Dist. LEXIS 19868, *73 (Sept. 8,
dix A also provides an discussion regarding the collection of subcontract data. 2005) (Northern Contracting II) ("At no point do the Regulations limit the appli-
165See Chapter Three, Economy-Wide Disparity Analysis for the Relevant Mar- cation of DBE goals to the subcontracting portion of contracts. To the contrary,
kets, supra, for discussion of different methods for making this comparison. the Regulations expressly provide that the goals requirements are imposed on
166The one availability study that included a utilization analysis (NDOR, 2000) prime contractors.").
did not disaggregate by race, ethnicity, and gender. 168See, e.g., fn. 191.

OCR for page 48

49
conscious contracting requirements.169 These studies included type of significance addresses the size of a given disparity.
no comparison of DBE participation on projects with goals to The second is "statistical" significance. This type of significance
DBE participation on projects without goals and also lacked addresses whether a given disparity could have arisen due to
any economy-wide analysis of business disparities. When random chance alone. Both types of significance need to be
studies such as these find "overutilization" in certain cate- considered in disparity analyses.
gories, policy makers may be misled to conclude that there is For example, suppose DBE participation in a given category
an absence of discrimination. (not subject to race-conscious contracting requirements) was
Several courts have recognized that this is incorrect. The 13.9% and DBE availability in that same category was 14.0%.
plaintiffs in Concrete Works III argued that "overutilization" The disparity ratio would be 0.139 ÷ 0.140 = 0.99. While it is
on projects subject to race-conscious contracting require- possible that this disparity might be statistically significant
ments indicated an absence of discrimination. The court (if, e.g., a large number of contracts and subcontracts was
rejected this argument, concluding that the more pertinent involved), it is difficult to imagine such a small disparity being
inquiry should focus on M/WBE participation on projects a source of concern to anyone. In contrast, sometimes sub-
without goals.170 stantively significant disparities may not be statistically signif-
Disparity analyses should strive to compare DBE partici- icant. This often occurs when statistics are based on a relatively
pation and DBE availability on projects that were not subject small sample of contracts.
to race-conscious contracting requirements, either in the The Equal Employment Opportunity Commission has
public sector, economy-wide, or both. This has been accom- adopted a standard for substantive significance in the employ-
plished in several ways, described below:
ment discrimination setting. This so-called "80% rule" has
· Comparing disparity ratios on contracts subject to goals to been endorsed as a relevant way to gauge legally meaningful
those on contracts not subject to goals. In states where racial or gender disparities rather than simply testing for statis-
there is no M/WBE program for state-funded projects, this tical significance alone. According to Meier, Sacks, and Zabell
can be accomplished by comparing disparities on federally (1986, 32):
assisted contracts with those on state-funded contracts.
· Selecting certain contracts to be "control" or "zero goals" Properly understood, the 80% rule has the potential to rational-
contracts and comparing disparity ratios on these con- ize much of the case law to date. . . . As a practical matter, even
when statistically significant differences have been noted, the
tracts to those on contracts with goals.
courts have been reluctant to find adverse impact when the dif-
· Comparing disparity ratios on state DOT projects subject ferences lack what is variously described as "practical," "substan-
to goals to disparity ratios from other public entities in the tive," or "constitutional" significance. And conversely, substantial
relevant market that do not use goals. disparities have been found insufficient to establish a prima facie
· Comparing disparity ratios on state DOT projects subject case when the sample sizes are so small as to make statistical sig-
to goals to disparity ratios from private entities in the rele- nificance unlikely. The 80% rule appears to be a reasonable
articulation of a statistical criterion to determine whether statis-
vant market.
tically significant differences are substantial enough to warrant
legal liability.
Importance of Significance Testing
Another important aspect to disparity analysis is significance The 80% rule states that for statistical disparities to be
testing. There are two dimensions to assessing significance. The taken as legally dispositive in the discrimination context, they
first is "constitutional" or "substantive" significance.171 This should be (a) statistically significant and (b) "substantively"
significant. Substantive significance is taken to mean, for
example, a DBE utilization measure that is less than or equal
169
CDOT (2001), GDOT (2005), OH (2001), NJ (2005). The OH (2001) study to 80% of the corresponding DBE availability measure. The
included some limited analysis of private sector disparities but still treated
"overutilization" of M/WBEs by state agencies subject to affirmative action
distinction between these two types of "significance" has
requirements as evidence of the absence of discrimination ("American Indian sometimes been a source of confusion to courts (Gastwirth
[firms at Ohio DOT] are overutilized at rates that are statistically significant. . . . 1988, 248).
Accordingly, race- and gender-conscious remedies are . . . [not] recommended . . .
Among the studies we reviewed, four indicated whether dis-
for American Indians.").
170
Concrete Works III, 321 F.3d at 98485; see also Western States, 407 F.3d at 992; parity ratios were at or below the 80% threshold but did not
Northern Contracting II, 2004 U.S. Dist. Lexis at *3738; Hershell Gill Consulting perform any statistical significance testing on the disparity
Engineers, Inc. v. Miami-Dade County, Florida, 333 F.Supp.2d 1305, 1318 (S.D. ratios. As Meier, Sacks, and Zabell (1986) note, a proper use
Fla. 2004) ("[The court] will keep the potential effect of the MWBE programs in
mind when analyzing the evidence presented by the County."). of the 80% rule requires disparity ratios to be at or below 80%
171
See also fns. 395 and 201 and the accompanying discussion in the text. and also be statistically significant.

OCR for page 48

50
Three of the studies we reviewed employed the 80% rule and drawn from a normal probability distribution and their dif-
tested for statistical significance. Three more employed statis- ference is greater than or equal to two standard deviations, we
tical significance testing but did not employ the 80% rule. can infer that the observed difference in the numbers is due
However, in all six of these studies, the significance test used is to more than just random chance. If we were to draw these
only correct under the assumption that contracts and subcon- two numbers repeatedly, we would observe that this inference
tracts are all the same size. If each contract and subcontract was correct, on average, only 19 out of every 20 draws. In
were the same size, the correct t-statistic would be: other words, it would be correct 95% of the time and incor-
u-a rect 5% of the time.
(1) t= One other type of significance testing was observed in
a (1 - a ) three of the studies we reviewed--Monte Carlo simulation
studies. Starting from the project database of contracts and
where u is the ratio of DBE contract and subcontract dollars
subcontracts, all with differing dollar sizes, these studies
to total contract and subcontract dollars, and a is the ratio of
simulate the award process by programming a computer to
DBE firms to all firms.
randomly assign contract awards to the several types of DBEs
However, contract and subcontract values vary greatly in
as well as to non-DBEs, based on their estimated availability
amount. Consequently, the correct t-statistic is:172
percentage. For example, if black-owned firms had esti-
u-a mated availability of 5%, then the computer would ran-
(2) t=
a (1 - a ) ci2 domly pick 5% of the contracts and subcontracts and assign
( c i )2 them to black-owned firms. The total value of the randomly
assigned awards would then be totaled and compared to
where ci is the dollar award or payment amount for contract availability to assess whether there was a disparity between
or subcontract i.173 This statistic is employed to do disparity utilization and availability. The simulation exercise is then
testing in only three of the studies we reviewed. repeated a large number of times. If utilization fell below
Not all data are normally distributed, but many probabil- availability in 95% or more of the runs, that disparity is sta-
ity distributions are approximately normal and samples from tistically significant.
many other distributions are approximately normal. The t- Monte Carlo studies are typically used in cases where sam-
distribution is one such distribution. It is often employed in ple sizes are small and the underlying distribution is not
practice to model data based on the assumption that the sam- known. In large samples, however, sample mean statistics tend
ple they are drawn from is approximately normal. to converge toward a normal distribution, and the t-statistic
In any particular application, the absolute value of the t- provided in equation (2) can be used. In the three studies we
statistic resulting from equation (2) can be compared to a reviewed, the Monte Carlo simulations were performed only
table of critical values for Student's t distribution to deter- on the full overall sample of contracts and subcontracts. In
mine whether the result is statistically significant.174 In dis- one study we reviewed, simulations were performed once for
crimination cases, the courts have usually required p-values all federally funded construction and engineering contracts
of 5% or less to establish statistical significance in a two-sided statewide over the entire study period and once for all state-
case. The analogous p-value for the one-sided case is half of funded construction and engineering contracts statewide over
the two-sided p-value, or 2.5%. the entire study period. Both of these samples were quite large
A two-sided p-value of 5% corresponds approximately to and the t-test from equation (2) could have been easily imple-
two standard deviations. The relevance of "two standard mented instead. No explanation was provided of the choice to
deviations" in statistics and the law is that it corresponds to a use Monte Carlo simulations.
95% confidence interval around a normal distribution. In the However, these three studies also presented dozens of sep-
simplest terms, in normally distributed data, approximately arate tables of disparity statistics disaggregated along several
95% of the values lie within two standard deviations (a 95% dimensions, for example, agency, funding source, major pro-
confidence interval) above or below the mean, and 5% lie curement category, contractor type, time period, and geo-
outside this range. Therefore, if we compare two numbers graphic region. Several of these tables therefore had only a
small number of contracts and subcontracts included. Dis-
parity statistics were presented in each of these tables, but no
significance testing was done. These tables would have been
172TX (1994, 8890). excellent candidates for Monte Carlo simulations that, as we
173In implementing this test statistic, prime contractor award or payment have said, can be helpful for assessing the significance of dis-
amounts must be a net of subcontract award or payment amounts.
174A table of critical values for the t-distribution can be found in any college-level parity ratios in cases with small samples. No explanation was
statistics textbook. provided of the choice not to use Monte Carlo simulations.