National Academies Press: OpenBook

Analyzing Information on Women-Owned Small Businesses in Federal Contracting (2005)

Chapter: 4 Disparities and Representation

« Previous: 3 The Federal Small Business Contracting Process
Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×

4
Disparities and Representation

Disparity, underrepresentation, and discrimination are different concepts. Disparity is simply a measured difference between two groups on an outcome of interest, such as differences in average earnings between men and women. Underrepresentation is a disparity in which the difference goes against a particular group (for example, lower earnings for women than men). Discrimination involves differential (adverse) treatment of a group compared with others based on membership in the group. Discrimination may also involve differential treatment on the basis of other factors that results in an adverse outcome for a particular group. Disparities may be due to any number of factors, including, but not necessarily and not limited to, discriminatory practices and behaviors.

This chapter focuses on the definition and measurement of disparities and underrepresentation of women-owned small businesses in federal contracting. It thereby responds to the committee’s charge to review the preliminary study, completed in late 2002, by the Office of Federal Contract Assistance for Women Business Owners (CAWBO) of the Small Business Administration (SBA). Chapter 5 considers the challenging task of inferring discrimination as a possible reason for any observed adverse disparities or underrepresentation. Our recommendations about the CAWBO study are presented in Chapter 6.

DISPARITY

Evidence of large and persistent differences, or disparities, in economic outcomes among men and women in the United States is not hard to find.

Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×

For example, for as long as wages have been measured, women have been estimated to earn less than men, although the gap has lessened over time and among some groups (see, e.g., Bureau of Labor Statistics, 2002). By occupation, adult women currently outnumber adult men in such occupations as nurses, elementary school teachers, social workers, bank tellers, and librarians, whereas men outnumber women in such occupations as purchasing managers, dentists, carpenters, firefighters, and mail carriers. By industry, more women are employed than men in the health care and education sectors and more men are employed in agriculture, construction, and mining (U.S. Census Bureau, 2003:Tables 615, 619). Properly measured, these examples illustrate disparities on the basis of gender.

The Small Business Reauthorization Act of 2000, Section 811(m), required the SBA to determine industries in which women-owned small businesses were “underrepresented” and “substantially underrepresented.” In other words, SBA was required to identify industries that evidenced disparities or differentials between women-owned small businesses and other businesses in which the disparity was adverse to women-owned small businesses.

Disparity Ratio

Congress did not indicate how to measure disparities, adverse or otherwise. Studies of preferential contracting programs commonly use a measure termed the “disparity ratio,” D. Calculating the disparity ratio begins by calculating the values of two shares for a target group, in this case women-owned small businesses. The two shares are a utilization share, U, and an availability share, A.1 The utilization share looks at an outcome of interest, which in this case is winning government contracts, measured in number of contracts or dollars awarded. The utilization share measures contracts (or dollars) awarded to women-owned small businesses, Cw, as a share of total contracts (or dollars) awarded, Ct. The availability share looks at the available universe or pool for contracting, measured as numbers of businesses or gross dollar receipts. The availability share measures women-owned small businesses (or their gross receipts), W, as a share of total businesses (or total gross receipts), T. Taking the two shares as a ratio gives an estimated value for the disparity ratio:

1  

The terms “utilization” and “availability” are defined in the disparity ratio literature, which has addressed primarily minority contracting disparities (see, e.g., Enchautegui et al., 1997; Marcus Weiss & Affiliates, 1990; Mason Tillman Associates, 1998).

Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×

D = U / A, where

U, or utilization = Cw / Ct and

A, or availability = W / T.

(1)

If D is 1.00, then there is no disparity for women-owned small businesses: their actual share, U, of contracts is the same as their expected share, A, based on their representation in the total business population. If the ratio is less than 1.00, then there is an adverse disparity or underrepresentation of women-owned small businesses among successful government contractors relative to the total business population. If the ratio is more than 1.00, then women-owned small businesses are overrepresented among successful government contractors relative to the total business population.

A more statistically tractable and interpretable alternative widely used in the legal and academic analysis of employment discrimination takes the difference between U and A as a measure of disparity. Thus, a difference of zero would indicate no disparity; a difference of less than zero (i.e., U is a smaller proportion than A) would indicate underrepresentation of the target group for the outcome of interest relative to the total population; a difference of greater than zero (i.e., U is a larger proportion than A) would indicate overrepresentation of the target group.

In the contracting arena, the availability share for such a target group as women-owned small businesses can be expected to vary across industries and other characteristics of businesses and contracts. For this reason, it is critical to use disparity ratios (or differences) to measure representation and not simple counts or percentages of utilization. For example, if industry A has 10 percent women-owned small businesses and industry B only 2 percent women-owned small businesses, a comparison of counts or percentages of contracts awarded would be misleading because one would not expect women-owned small businesses to win as many contracts in industry B as in industry A. Differences among industries also suggest the importance of examining each industry separately and not lumping industries together in the calculation of a single disparity ratio (or difference). Separate ratios should be calculated instead. The example in Table 4-1 makes this point clear.

Constituent Elements

Critical to appropriate calculation of disparity ratios is an agreed-on definition and measurement of each element of the ratio—the target group, the outcome of interest, and the total population—and of any other variables to be included in the analysis, such as industry. We first discuss issues of defining and measuring the constituent elements of the disparity ratio

Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×

TABLE 4-1 Hypothetical Calculation of Disparity Ratios for Two Industries

 

Industry A

Industry B

Total

a. Women-owned federal contractors

25

50

75

b. Total federal contractors

1,000

1,000

2,000

c. Women-owned firms

1,000

1,000

2,000

d. All firms

10,000

50,000

60,000

Utilization share, U = (a) / (b)

0.025

0.05

0.038

Availability share, A = (c) / (d)

0.10

0.02

0.033

Disparity ratio, D = (U / A)

0.25

2.50

1.15

NOTE: Calculating the disparity ratio for the two industries combined would lead to the conclusion that women-owned small businesses were overrepresented in federal contracting, when the data indicate that they are underrepresented in industry A.

and then summarize and assess the approach to these issues in the SBA CAWBO study and other studies.

Target Group

The target group of interest for the congressionally mandated SBA study comprises women-owned small businesses. There is a clear formal definition in the legislation governing the SBA of businesses that are “women-owned” (see Box 2-2). In implementing the definition, errors of classification can occur because of differences between the SBA definition and the definition employed in a relevant data source. For example, the 1997 Survey of Women-Owned Business Enterprises conducted by the U.S. Census Bureau as part of the 1997 Economic Census provided separate tabulations for firms with 50-50 male-female ownership and those with 51 percent or greater female ownership, but earlier surveys in the series counted some equally owned businesses as women-owned. This change reduced the number of women-owned businesses in the 1997 survey by an estimated 37 percent (U.S. Census Bureau, 2001:12).2

2  

This survey and the Survey of Minority-Owned Business Enterprises have been renamed the Survey of Business Owners and Self-Employed Persons; the most recent round is for 2002.

Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×

In addition, there may be efforts to game the status of being woman-owned, which could affect not only the quality of the data on which the targeting of a set-aside program depends, but also its effectiveness. Businesses may fraudulently claim to be women-owned according to the official definition, or they may transfer ownership but not actual control to a woman. Front organizations may also be formed for the purpose of bidding on contracts that are then substantially subcontracted to other firms that do not meet the definition. In each such case, the estimated number of women-owned small businesses exceeds the true number and contracts are diverted from legitimate women-owned small businesses.

The SBA also operates with a formal definition of “small,” or, rather, it has a set of definitions, which vary by industry in terms of the variable used for measurement (e.g., number of employees, gross receipts) and the threshold value (e.g., maximum of 100 employees for a retail trade business, maximum of $3 million average annual receipts for a travel agency) (see Box 2-2). Errors of classification can occur because of differences between the SBA definition and the definition employed in a relevant data source, advertent or inadvertent reporting errors for measures of firm size, fluctuations in firm size, differences in organizational form, and other factors. Contracting preferences for small businesses create an incentive to maintain status as an SBA-defined small business.

Outcome of Interest

The outcome of interest for the SBA study is federal contract awards, which can potentially be defined in terms of numbers of awards or dollar amounts of awards. The latter measure is most commonly used in the literature, which reflects the fact that the yearly goals for use of women-owned and other types of small businesses established by Congress and negotiated between the SBA and individual contracting agencies represent percentages of total awarded dollars, usually separately for prime contracts and subcontracts. In addition, data on dollar amounts awarded are readily accessible from the Federal Procurement Data System (FPDS), whereas data on number of contract awards require combining FPDS records for individual actions (new awards, modifications, etc.) into records for individual awards, or else using contract actions as a rough proxy for awards. Data on total size of awards for individual contracts also require combining FPDS individual action records.

Although defining the outcome measure in terms of dollar amounts awarded makes sense for estimating utilization shares to respond to the congressional mandate, it would also be useful to examine utilization shares in terms of contract awards, or actions, in order to better understand the contracting picture for women-owned small businesses. Table 3-1 in Chap-

Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×

ter 3 shows that shares in terms of dollars and actions are similar for many cabinet departments, but there are exceptions. Notably, the Department of Energy in fiscal year 2003 exhibited a relatively high share of contract actions going to women-owned small businesses (15 percent), but the share of its contract dollars awarded to women-owned small businesses was only 0.5 percent. This finding presumably reflects the fact that this department has a small number of very large contracts for its laboratories, which are neither bid on nor won by small businesses.

Because the distribution of contract awards by dollar value may be highly skewed for some agencies, time periods, or industries, a careful analysis of utilization estimates should determine their sensitivity to outliers and whether some contract awards should be excluded from the estimation. For example, the distribution of prime contract dollars awarded by the Defense Department for fiscal year 2004 (data not yet available) may be skewed upward by the small number of very large contracts awarded for military support operations and reconstruction in Iraq.

With regard to the universe of awards to include, it would be desirable to examine separately awards between $2,500 and $100,000, as procurements in this dollar range are designated for small businesses (to the extent feasible). Utilization shares for awards under $100,000 would be informative regarding the relative success of women-owned small businesses compared with other small businesses in obtaining these small-size contracts, whereas the shares for larger awards would be informative regarding the success of women-owned small businesses in obtaining contracts that larger businesses may bid on as well.

Because the congressionally mandated study of women-owned small businesses requires estimating disparity ratios separately by industry, the universe of contract actions must of necessity be limited to those over $25,000. The FPDS does not include smaller awards, and only very limited information, not including industry classification, is maintained for small contracts by the General Services Administration (see Chapter 3).3 The detailed information in the FPDS for awards over $25,000 permits estimating utilization shares for smaller and larger award size categories. Also, with the currently available data, disparity ratios can be calculated only for prime contract awards but not for subcontracts. The many small businesses that bid on and win subcontracts are not recorded in the FPDS.

3  

Contracting officers may fill out SF-279, which is input to the FPDS, for contracts as small as $10,000, but most such contracts are combined and aggregate information for them entered onto the SF-281.

Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×
Total Population

The total population of interest for the SBA study concerns businesses operating in the United States. The population can potentially be defined in terms of numbers of businesses or total business receipts. Typically, only the first definition has been used in the literature for estimating availability shares. Whichever measure is used (numbers or dollars), it is important that it is consistent with the measure used for the outcome of interest (see “Consistency of Elements,” below). Specification of an appropriate population measure must also consider the universe of firms to include, specifically, the treatment of very small firms and other firms that may not be ready, willing, and able to bid on federal procurements.

Very small firms Small businesses are defined to include not only corporations and partnerships, but also sole proprietorships (see Box 2-2). Consequently, a broad definition of the business universe is likely to include a large number of very small firms defined by some metric.

One available measure of smallness is a business having no paid employees. Thus, data from the 1997 Survey of Women-Owned Business Enterprises, which included in its universe all firms operating in the United States with $1,000 or more in gross annual receipts, showed that 75 percent of the estimated total number of 20.8 million firms had no paid employees. By gender of ownership, 84 percent of the estimated number of 5.4 million women-owned firms had no paid employees, as did 67 percent of 3.6 million equally male and female-owned firms, 45 percent of 11.4 million male-owned firms, and 33 percent of 0.4 million publicly held, foreign-owned, and nonprofit firms, which are not classified by gender of ownership (U.S. Census Bureau, 2001:12).

These percentages may be misleading, however, regarding the capabilities of small businesses to handle federal contracts. Some businesses without paid employees may in fact be much larger operations because they use such mechanisms as hiring independent contractors as needed instead of having salaried employees.

Another, perhaps more appropriate, measure of smallness is sales volume. For 1997, 20 percent of all firms with $1,000 or more gross receipts had less than $5,000 in gross receipts; the comparable figure for women-owned businesses was 33 percent. These data suggest that significant fractions of businesses may represent occasional efforts of people who are fully employed as wage and salary workers and do not intend to grow their “business.” An example would be a full-time employed manager or teacher who reported a couple of thousand dollars of income from honoraria for presenting papers at a few meetings.

Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×

Given that available contracting data limit the estimation of utilization shares to contract awards over $25,000, then the universe of firms for estimating availability shares should be limited in some appropriate manner. For example, the universe could be limited to firms that reported a specified minimum amount of gross receipts in the 1997 Survey of Women-Owned Business Enterprises. Determining the minimum amount of gross receipts would require careful analysis of the distribution of gross receipts for small businesses and could perhaps be informed by data from the Central Contractor Registration (CCR) on the size distribution of small businesses that have registered to do business with the federal government. (The CCR obtains data on number of employees and 3-year average annual receipts.)

If separate utilization shares are calculated for smaller and larger contract awards (say, $25,000 to $100,000 and $100,000 or more), as suggested above, then the universe of firms should be appropriately classified for calculating availability shares. For smaller contract awards, the universe could be limited to small businesses as defined by the SBA (perhaps excluding some very small businesses), while, for larger awards, it could include both small and larger businesses.

Other criteria for defining “ready, willing, and able” A related issue that arises in determining an appropriate universe of firms for estimating availability shares is whether to use criteria in addition to firm size to further limit the universe to firms that are likely bidders on federal contracts. This decision depends largely on the intended use of the disparity ratios. Possible uses are to help justify remedial preferential contracting programs for women-owned small businesses in court, to help refine the design of such programs, or to be part of a broad analysis of barriers that women may face in business ownership and growth.

If disparity ratios are to be used in a comprehensive analysis of barriers that women may face in business ownership and growth, which could, in turn, produce underrepresentation in federal contracting, then a broad definition of the total population of firms for estimating availability shares is likely to be appropriate. If, however, disparity ratios are to be used in court to justify remedial preferential contracting programs for women-owned small businesses, then a broad definition is likely to be inappropriate. Case law in this area has put forward a standard for the total population universe of firms, which includes those that are ready, willing, and able to be federal contractors (see Chapter 2). The notion is that remedial programs are justified when there is evidence of possible discrimination by the federal contracting process against a class of firms in the “ready, willing, and able” universe. Remediation is not necessarily justified, according to case law, when the evidence pertains to a broader universe of

Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×

firms, which may indicate societal barriers to business development and growth by a class of firms, but not additional discrimination in the contracting process.

Not only would a “ready, willing, and able” standard for procurements over $25,000 probably exclude very small businesses from the total population universe, but it would also probably exclude other businesses that are too new or are not technically qualified for one or another reason in bidding on federal contracts. Some studies of disparities in state and local contracting have used restricted business lists, such as contractor registries and survey respondents that report interest in or efforts to obtain contracts, to estimate availability shares. The Department of Commerce used a regression methodology with several data sources to estimate a ready, willing, and able population of firms for estimating availability shares for federal contracting (see “Disparity Ratios in the Literature,” below).

At present, the CCR maintained by the Department of Defense for the entire federal government could be used for this purpose. Beginning October 1, 2003, every business that intends to bid on a federal contract must register on the CCR. Still, it could be argued that the CCR is too limited a list and that other ready, willing, and able firms exist that, for one or another reason, are not registered on the CCR.

Ideally, one would be able to estimate availability shares and calculate disparity ratios with broader and narrower definitions of the total population. Constructing and comparing multiple measures would provide a fuller picture of the role of women-owned small businesses in economic activity, generally, and in federal contracting specifically. It may also facilitate an investigation of where in the business development and contracting process disparities arise.

Consistency of Elements

A statistically defensible disparity ratio for women-owned small businesses (or another target group) in federal contracting requires consistency among its elements. In particular, if the utilization share is defined in terms of contract award dollars, as is most commonly done, then the availability share should be defined in monetary terms as well, such as annual gross receipts.

Most often, however, disparity studies have used inconsistently defined elements, such as contract award dollars for the utilization share and number of firms for the availability share. The extreme implicit assumption is that every business in the available pool is equally ready, willing, and able to bid on and perform every contract. Some studies have used consistent measures, such as number of contract awards and number of firms. We did not find examples of studies that have used contract award dollars and

Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×

gross receipts. A Department of Commerce study used contract award dollars and estimated dollar value of capacity for federal contracting (see “Disparity Ratios in the Literature,” below).

One reason for not using gross receipts may be the belief that the availability shares for women-owned small businesses would be systematically smaller because of the very large gross earnings of a small number of very large firms. To the extent this distortion occurs, then the resulting disparity ratios would be closer to 1.0, perhaps undercutting the rationale for preferential contracting programs. More substantive is a concern that the current size, capabilities, and resources of a firm may themselves be influenced by past discrimination. Yet an inconsistent calculation that bases utilization shares on contract dollars awarded and availability shares on number of firms could well produce downwardly biased disparity ratios, thereby overstating the need for preferential contracting programs. The reason is the heavily skewed distribution of U.S. firms by size. Small women-owned businesses make up a disproportionately large percentage of firms compared with their share of business receipts.

Publicly held corporations that account for the majority of business receipts have no official gender, although their shareholders do. The gender of ownership of a business is only specified in the case of proprietorships, partnerships, and Schedule C businesses. These types of businesses are generally considerably smaller in scale than corporations. If the women-owned share of businesses is calculated only among businesses for which the gender of ownership is known, then, for consistency, the women-owned share of contracting should be calculated only among contracts awarded to such businesses.

A careful analysis will look at the results of several different types of disparity ratios. To the extent that different measures—such as receipts-based measures, numbers-based measures, measures that vary the lower limit of receipts for inclusion in the universe of firms, and measures that exclude outliers (e.g., very large contract awards, very large businesses in terms of receipts)—tell a similar story, then the justification for (or against) preferential contracting programs will be strengthened. To the extent that different measures tell very different stories (e.g., the large percentage of contract actions compared with the small percentage of contract dollars awarded to women-owned small businesses by the Department of Energy), then additional analysis will be required to make a case for preferring a particular measure or set of measures among all those calculated.

In addition to consistency among the elements of the disparity ratio, the reference periods for the utilization and availability shares should be as close in time as possible. The number of women-owned small businesses has grown rapidly in the recent past, with an estimated 16 percent increase over the period 1992 to 1997 (37 percent among firms with paid employ-

Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×

ees), compared with a 6 percent growth rate in the same time period for all businesses, with or without paid employees (U.S. Census Bureau, 2001:12). As a consequence, if the time period for the availability share lags the time period for the utilization share by more than a year or so (as is likely to be the case given data availability), then the disparity ratio will probably be biased upward, indicating a better position for women-owned small businesses in federal contracting compared with their position in the business sector than is probably true.

Industry Breakdowns

The congressionally mandated study of women-owned small businesses in federal contracting requires estimates of representation by industry. The current standard set of industry codes, the North American Industry Classification System (NAICS), has five levels of classification. At the broadest level, NAICS consists of 20 industry sectors, identified by the first two digits of the individual 6-digit industry codes (1,170 in all). Subsectors, industry groups, NAICS industries, and U.S. industries are identified by the first three digits, four digits, five digits, and six digits, respectively, of the specific industry codes. The NAICS system, developed jointly by Canada, Mexico, and the United States, was first issued in 1997 and most recently updated in 2002. It replaced the previous 4-digit Standard Industrial Classification (SIC) system, last updated in 1987, which emphasized the manufacturing sector. NAICS focuses on new and emerging industries, industries using new technology, and service industries. The 2002 NAICS update made substantial revisions in the construction and wholesale trade sectors and minor revisions in the retail and information sectors (see www.census.gov/epcd/www/naics/html [December 2004]).

A key issue for calculating disparity ratios (or differences) by industry is which level of NAICS codes to use. The 20 industry sectors (defined by the first two digits in the NAICS coding scheme) appear too broad to be used as the basis of disparity ratios to inform understanding of the role of women-owned small businesses in federal contracting and what kinds of preferential treatment may be indicated. At the other extreme, there are so many specific industries identified by 6-digit NAICS codes that the resulting disparity ratios are likely to be adversely affected by sampling variability and also by errors of classification.

Ideally, the level of NAICS codes used for calculation of disparity ratios will be chosen to optimize two criteria. First, each industry-specific disparity ratio should meet a specified reliability standard (see “Data Quality Issues,” below). Second, the level of industry detail should be as disaggregated as the data will support so that preferential treatment programs (if

Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×

any) are targeted to relevant industries and exclude industries for which such programs do not appear to be warranted. With this approach, disparity ratios could be calculated for 3-digit categories when the industries in a 3-digit grouping are reasonably homogeneous (e.g., in scale of operations), or when there is insufficient sample size for further disaggregation. Given sufficient sample size and heterogeneity in a 3-digit category, then disparity ratios could also be calculated separately for two or more 4-digit groupings.

UNDERREPRESENTATION

Congress did not define underrepresentation or substantial underrepresentation—that is, how small the disparity ratio must be to single out an industry for instituting preferential treatment programs for women-owned small businesses in federal contracting. In the literature, a disparity ratio of 0.80 or smaller has been used to denote industries in which women-owned or minority-owned small businesses are underrepresented in contracting (see, e.g., Enchautegui et al., 1997). Adoption of the threshold value of 0.80 may in part stem from its use by the Equal Employment Opportunity Commission as a rule of thumb for defining underrepresentation in enforcing antidiscrimination employment laws (see Chapter 2). The 0.80 threshold value also has the advantage, compared with a higher value, of reducing classification errors due to sampling variability or other sources of error in the underlying data.

No threshold, formal or informal, has been set for identifying industries in which women-owned small businesses are considered substantially underrepresented in contracting. The SBA CAWBO preliminary study reviewed below adopted a threshold of 0.50 or smaller to denote substantial underrepresentation.

Ultimately, the selection of threshold is a matter of judgment and face validity. Scientific methods can help narrow the range of reasonable values but cannot determine a specific threshold value to denote underrepresentation or substantial underrepresentation in federal contracting. In order to arrive at a scientifically defensible definition of “underrepresentation,” one needs to specify an underlying model of the processes that would generate such underrepresentation—see the discussion of theory-based models of discrimination in Chapter 5.4

4  

Formal theory-based models make it possible to estimate statistically the probable effects of discriminatory factors as an explanation for observed disparities (see Chapter 5).

Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×

DATA QUALITY ISSUES

Whether the purpose of calculating disparity ratios is to support the use of preferential contracting programs or to be informative about the role of women-owned small businesses in federal contracting, it is critical that good practices be followed with regard to selecting suitable data sources, estimating the elements of the disparity ratios, evaluating sampling and nonsampling errors in the estimates, testing the sensitivity of the estimates to alternative data sources and methods, and documenting the procedures used in every step of the process. Good documentation is particularly important to permit outside review.

There are no perfect data sources for estimating disparity ratios by industry for women-owned small businesses in federal contracting, particularly in the case of availability shares. Better-suited data are available for estimating utilization shares, except that detailed information is not available about contracts under $25,000, and no information is available about subcontracts. The lack of data for subcontracts is unfortunate, because one would expect subcontracting to be an important means by which women-owned and other types of small businesses participate and gain experience in federal contracting.

Generally, careful evaluation is required of the strengths and weaknesses of alternative data sources. Table 4-2 summarizes key features of the FPDS for estimating utilization shares for prime contract awards over $25,000. It also summarizes key features of two possible sources for estimating availability shares—the Census Bureau’s Survey of Women-Owned Business Enterprises conducted every 5 years (SWOBE, renamed the Survey of Business Owners) and the CCR. Because of changes in these sources over time, comparable time series of utilization and availability shares for past years are not easy to construct.

Once a data source is selected for use in constructing disparity ratios for women-owned small businesses, then it is important to evaluate several dimensions of data quality and carefully document the findings. At least five quality features should be considered: sampling variability, nonsampling errors, outliers, consistency of reference period, and consistency of concept.

Sampling variability Some data sources, such as the SWOBE, are sample surveys, for which estimates are subject to sampling variability. Such variability is important to take into account when determining the level of detail of industry classifications for preparing separate disparity ratios and when selecting threshold values for underrepresentation and substantial underrepresentation. Other data sources are virtually a census of a universe—for example, the FPDS is a census of actions involving contracts over $25,000 for almost all federal contracting agencies. However, utiliza-

Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×

tion shares estimated from the FPDS may not be reliable in a broader sense if the number of contracts on which the shares are based is small for a particular industry and year. At the limit, if there is only one contract, the utilization share will be 1 or zero. In such instances, it may be advisable to pool data across years and industries for estimation.

Nonsampling errors In a survey such as the SWOBE, nonsampling errors include nonresponse to the questionnaire or to specific items on the questionnaire, underreporting or overreporting of such items as gross receipts and number of employees, and errors introduced by sample weighting and nonresponse imputation procedures. In an administrative records system like the FPDS, nonsampling errors may include inadvertent failure to file forms for some contract actions, late filing, and misreporting of some information for a contract action.

It is usually difficult to estimate the magnitude and effects of nonsampling errors on estimates from the data. At a minimum, users should ask the producing agency for information about nonsampling errors and about procedures that are used to assess and correct for errors (for example, auditing procedures for the data in the FPDS, respondent reinterviews for a survey). Because nonsampling errors may have important effects that are hard to estimate, users should plan to conduct sensitivity analyses to determine how much variability in estimates of disparity ratios results under different assumptions about the quality of the data.

Outliers Careful examination of the distribution of the data on important dimensions, such as contract award size or firm size, should be part of the process of estimating disparity ratios. It may be that outliers or extreme values are present that distort the estimates for a given industry, contracting agency, or time period. Examples include the likely spike in contracting dollars awarded in fiscal year 2004 because of the war in Iraq and such unique, very large contracts as the Department of Energy national laboratories, requiring highly specialized capabilities. Graphical display techniques, such as box plots, can help identify outliers. Whether to exclude them from the estimation will not always be clear, but the decision should be carefully documented and the rationale explained. Again, sensitivity analysis of the effects on the estimates of excluding or including various outliers will be helpful in determining and justifying an appropriate decision.

Outliers are a particular problem for the contracting application, in contrast to the much more developed case of employment discrimination. In the employment application, the unit of analysis is an individual man or woman applying for a particular position. In the contracting application, however, firms vary widely in such measures of capacity as revenues and

Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×

TABLE 4-2 Features of Relevant Data Sources for Study of Use of Women-Owned Small Businesses in Federal Contracting

Name and Sponsor Agency

Survey of Business Owners and Self-Employed Persons (SBO); combines Surveys of Women-Owned and Minority-Owned Business Enterprises (SWOBE/SMOBE); conducted by the U.S. Census Bureau

Central Contractor Registration (CCR); contractor-operated on behalf of the U.S. General Services Administration, Office of Management and Budget, Small Business Administration, and Department of Defense

Federal Procurement Data System (FPDS); being upgraded as FPDS-Next Generation (FPDS-NG); contractor-operated on behalf of the U.S. General Services Administration

Universe

As of 2002, all business enterprises with gross receipts of $1,000 or more, including sole proprietorships, partnerships, subchapter S-corporations, and a small sample of C-corporations; all businesses with the same owner treated as single business regardless of how many tax returns filed

As of October 1, 2003, all vendors and prospective vendors (must be registered prior to submitting a bid)

Individual Contract Action Reports database (ICAR) for prime contracting actions reported on form DD-350 (defense) and SF-279 (civilian agencies) for every contract over $25,000 (and some smaller contracts); also produces reports from aggregate data (e.g., for groups of contracts under $25,000 reported on form 281); excludes Congress, GAO, federal courts, FAA, TVA, FDIC, CIA, and NSA

Sample Size and Design

Information on sole proprietorships obtained from IRS tax returns; other types of businesses included in survey with complex, stratified design

N.A.

N.A.

Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×

Frequency and Timeliness

Every 5 years, part of Economic Census, data released 2-3 years after collection; most recent survey provides data for 2002, collected in 2003, released in 2005; special analyses on request

Updated daily; no regular reports; accessible by federal agencies and contractor registrants

Updated quarterly; annual reports produced beginning for fiscal year 2000; special analyses on request

Content for Each Data Record

Percentage ownership by gender, Hispanic origin, and race, number of owners, whether did business with federal government, type of business (e.g., sole proprietorship), number of employees, annual payroll, annual gross revenues, number of establishments, places of operation, industries of operation

DUNS Number, Tax ID or SSN, corporate web page URL, physical address, mailing address, business start date, number of employees, 3-year average annual revenue, fiscal year close date, whether accept federal contracts or grants or both, type of organization (e.g., sole proprietorship), where incorporated, business types (e.g., government entity by type, educational entity by type, business by field), socioeconomic factors (e.g., small business, woman-owned), socioeconomic certification, party performing certification, up to six NAICS codes, up to six SIC codes, financial and contact information for billing

ICAR records: reporting agency, contract identification, action date, kind of contract action, dollars obligated or deobligated, principal product or service code, principal NAICS code, contractor identification, principal place of performance, whether bundled, type of contract or modification (e.g., fixed-price), solicitation procedure (e.g., set-aside), authority for other than open competition, number of offers received, extent competed, type of contractor (e.g., SDB), whether women-owned, HUBZone, SDB, or other preference program, price evaluation adjustment percent difference, whether subcontracting plan, whether subject to labor statutes,

Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×

 

 

 

estimated contract completion date, contractor’s taxpayer ID, parent company’s taxpayer ID, whether part of small business competitiveness demonstration program (selected agencies), number of employees category (small businesses only), average annual gross revenue category (small businesses only)

Historical Information

First conducted in 1972; prior to 1997 had only a small sample of C-corporations (none prior to 1992) and treated each business tax return as a separate business; identification of sole proprietorships with employees revised in 1997 (reduced estimated number relative to 1992 method); identification of women-owned businesses revised in 1997 (reduced estimated number of women-owned businesses by separately identifying 50-50 men-women-owned)

Prior to October 1, 2003, covered vendors for the Defense Department, Transportation Department, Treasury Department, and NASA only; as of January 1, 2004, integrates the SBA PRO-Net database of certified small businesses

 

NOTE: N.A. = not applicable. CIA = Central Intelligence Agency; DUNS = Data Universal Numbering System number provided by Dun & Bradstreet; FAA = Federal Aviation Administration; FDIC = Federal Deposit Insurance Corporation; GAO = Government Accountability Office; IRS = Internal Revenue Service; NAICS = North American Industry Classification System; NASA = National Aeronautics and Space Administration; NSA = National Security Agency; SDB = small disadvantaged business; TVA = Tennessee Valley Authority.

SOURCE: U.S. Census Bureau (2001); www.ccr.gov [December 2004]; www.fpds.gov [December 2004]; see also Eagle Eye Publishers, Inc. (2004b).

Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×

number of employees, and contracts vary widely in such characteristics as technical requirements and dollar value.

Consistency of reference period Given that utilization shares or availability shares or both may exhibit dynamic changes over time, it is important that the reference period be as consistent as possible. Take an example in which the share of women-owned small businesses increases, but their share of contracts does not. If the availability share is computed from the 1997 SWOBE (because 2002 SWOBE data are not yet available) and the utilization share from the 2003 FPDS as the year of interest, the result would be to underestimate the current availability share and overestimate the disparity ratio. Such an outcome could lead to an erroneous conclusion that little disparity was evident because the estimated disparity ratio was higher than the true disparity ratio. If data sources with consistent reference periods are not available, then consideration could be given to adjusting one or another source to achieve greater consistency—for example, trend indicators from other sources might be used to project the 1997 SWOBE data forward in time.

Consistency of concept As discussed above, the concept underlying the utilization and availability shares must be as consistent as possible for appropriate estimation of disparity ratios. Thus, if data limitations dictate a universe of federal contracts over $25,000 for estimating utilization, then the universe of businesses for estimating availability should probably exclude very small businesses that are not realistically likely to bid on or be qualified for such contracts.

DISPARITY RATIOS IN THE LITERATURE

In the years immediately following City of Richmond v. J.A. Croson Co., 488 U.S. 469 (1989), many state and local governments commissioned studies of disparities in government contracting for minority-owned and, less often, women-owned small businesses using a variety of measures of utilization and availability. The Urban Institute subsequently conducted a meta-analysis of these studies (Enchautegui et al., 1997).5 In the late 1990s, the Department of Commerce performed analyses to identify industries in which ready, willing, and able minority-owned businesses were underrepresented in federal contracting. Finally, CAWBO conducted industry-specific analyses of disparities in federal contracting for women-owned

5  

In meta-analysis, the researcher produces statistics that combine and summarize the results of more than one study on an outcome variable of interest, selecting studies that meet specified criteria for content and quality.

Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×

small businesses pursuant to the Small Business Reauthorization Act of 2000, using 1997 SWOBE data on businesses with paid employees to calculate availability shares.

We review each of these efforts below. With the exception of the Urban Institute meta-analysis, the possible usefulness of the various study results is severely impaired by the poor quality of documentation of the methods and data sources employed.

State and Local Disparity Studies

Over 100 jurisdictions have conducted studies of the contracting experience of minority-owned small businesses and, in some instances, women-owned small businesses. These studies have been criticized as too often relying on anecdotal evidence of discrimination provided by minority and women owners or on a general history of discrimination in the locality and not on statistical evidence of disparities (see, e.g., La Noue, 1992, 1995).

We were not able to review any of the individual studies, which, in any case, deal with state and local contracting, not federal contracting. We reviewed the Urban Institute meta-analysis (Enchautegui et al., 1997), which combined results from many of these studies to estimate statistical disparities across states and localities in contracting for minority-owned and women-owned small businesses in broad industry groups. The Urban Institute study was conducted for the U.S. Department of Justice as part of an evaluation in the mid-1990s of the need for federal preferential contracting programs (see Chapter 2). Its approach and findings provide useful insights for the estimation of valid, informative disparity ratios.

Urban Institute Meta-Analysis

The Urban Institute collected reports of 95 state and local government-commissioned disparity analyses and reviewed them to screen out those that did not provide relevant or statistically defensible results. Studies were included only if they satisfied all four of the following criteria: they provided disparity ratios or the data to construct ratios and not just anecdotes; they reported findings separately by industry category; they reported the number of contracts used in the analysis or the statistical significance of the calculated ratios; and they included more than 80 contracts in total. In addition, some studies that satisfied these criteria were dropped because of significant inconsistencies in their results or poor documentation for key calculations. After the screening, 58 studies remained for the Urban Institute meta-analysis.

These 58 studies varied in their measures of utilization and availability for constructing disparity ratios. Most commonly, utilization was measured

Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×

in monetary terms, either as the dollar amount of contract awards or the dollar amount actually paid out. Some studies also defined utilization in terms of numbers of contract awards. Measures of availability were expressed in numerical terms (numbers of firms) and never in monetary terms (e.g., gross receipts), and they differed in the universe definition. The six most common universe definitions, listed in order from a narrow to a broad definition of availability, were previous award winners (firms on vendor lists), firms that bid on past contracts or appeared on lists to receive information about procurements, firms certified as minority owned or women owned, firms that expressed interest in government contracting work in response to a survey, all firms with paid employees, and all firms. Some studies used more than one definition. Availability measures based on all firms or all firms with paid employees used data from the 1987 SWOBE.

To combine results across studies, the Urban Institute first averaged the disparity ratios for each study (jurisdiction) that reported more than one ratio (most did) and then took the median of the study averages in order to minimize the effects of outliers. The averaging was performed separately for groups defined by ownership status (black, Hispanic, Asian, American Indian, women-owned) and industry category (construction, construction subcontracts, goods, professional services, other services).6

The Urban Institute estimated an overall disparity ratio of 0.29 for women-owned businesses in state and local contracting. This ratio was estimated to differ significantly from a chance result, using a 0.05 probability test. Estimated disparity ratios for women-owned businesses by industry category varied from 0.17 for professional services to 0.77 for construction subcontracting (the latter estimate was not statistically significantly different from 0.80 or 1.00).

The Urban Institute tested the sensitivity of the results to several methodological features of the various studies. Disparity ratios were calculated separately for studies with large numbers of contracts or high levels of availability of minority-owned or women-owned businesses and all other studies. Separate ratios were also calculated for each of three research firms that conducted multiple disparity studies for state and local governments and all other studies. These breakdowns did not alter the picture conveyed by the overall results. Disparity ratios for women were low, no matter how they were computed—that is, women-owned businesses received a very small share of contracts and contract dollars compared with their share of the business population (Enchautegui et al., 1997:Tables 2.4, 2.9).

6  

The results are not specific to minority-owned or women-owned businesses that are small businesses as defined by the SBA. Data from the 1997 SWOBE indicate that a very high percentage of women-owned businesses (99 percent) would probably be classified as small by the SBA (U.S. Census Bureau, 2001:Table 9).

Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×

The Urban Institute also tested the sensitivity of the results to the universe definition for the measure of availability, specifically, whether the measure used SWOBE data on all firms or all firms with paid employees to define the universe, or, instead, used a measure that could be construed as “ready, willing, and able” (e.g., defining the universe in terms of registered bidders). Using SWOBE data could present several problems (see Enchautegui et al., 1997:70-71). Some of these problems, if not corrected, would probably generate overestimates of availability—for example, the fact that SWOBE includes very small businesses and prior to 1992 excluded C-corporations (primarily those with more than 35 shareholders). Other problems would probably generate underestimates of availability—for example, known undercounts of Hispanic-owned and Asian-owned businesses in SWOBE. Underestimation might also occur if the survey reference year preceded the estimation year in a particular study. Yet a list of bidders or registrants might be out of date or exclude some capable and interested firms. Finally, respondents to a survey of interest in government contracting might not accurately represent the entire sample because of differences between respondents and nonrespondents, or the sampling frame for the survey might have excluded some businesses.

As it turned out, across all studies, the median disparity ratios for women-owned businesses differed little by whether the universe of available firms was defined broadly or more narrowly (Enchautegui et al., 1997:Table 2.5). Given that the measure of utilization was held constant, this finding suggests that, on average, women-owned firms were neither overrepresented nor underrepresented on vendor or bidder lists of state and local governments compared with their share of all firms. In contrast, disparity ratios for minority-owned firms were significantly higher when the universe of available firms was defined broadly than when it was defined more narrowly. This finding suggests that, in the states and localities studied, minority-owned firms were overrepresented on vendor or bidder lists compared with their share of all firms.

As noted above, almost all 58 studies calculated disparity ratios that were internally inconsistent because the utilization measure was in terms of contract dollars awarded and the availability measure was in terms of number of firms in a specified universe. Some studies also calculated disparity ratios consistently, in that the utilization measure was in terms of numbers of contracts awarded, not dollars. No jurisdiction compared shares of contract dollars awarded with shares of gross business receipts.

Comparing the consistently and inconsistently calculated ratios for women-owned small businesses for jurisdictions for which both types of ratios were reported suggests few differences between them. The exception was construction subcontracting, for which the median disparity ratio for the consistent calculations (numbers of contracts and firms) was 0.52 points

Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×

higher than that for the inconsistent calculations (contract dollars and numbers of firms). This finding suggests that women-owned businesses received a higher percentage of construction subcontracts than of the subcontract dollars awarded.7

The Urban Institute meta-analysis is not directly useful to the SBA to respond to the congressional charge to estimate disparity ratios by industry for women-owned small businesses in federal contracting. The data pertain to state and local contracting for those jurisdictions that chose to conduct disparity studies, and they are out of date. The study approach is useful, however, because it demonstrates attention to detailed documentation of data and methods and careful explication of limitations of the analysis and the sensitivity of the results to factors that could have biased them. The data for each study included in the analysis are provided in the report, so that other researchers can examine them. One useful addition to the study would have been a display of the data points for each industry using a box plot, which would graphically allow conclusions about the spread in the data.

Department of Commerce “Ready, Willing, and Able” Analyses

In 1996, the Department of Justice put forward a plan to revamp the federal government’s preferential contracting programs for small disadvantaged businesses (see Chapter 2). One provision of the revised acquisition regulations provided that small disadvantaged businesses could receive a price evaluation adjustment, or bid-credit, to level the playing field with larger businesses. However, the bid-credit could be used only for contracts in industries designated by the Department of Commerce as falling below an industry-specific benchmark limit with regard to utilization of small disadvantaged businesses. The Justice Department intended these benchmarks to represent the “level of minority contracting that one would reasonably expect to find in a market absent discrimination or its effects.” The Commerce Department’s Office of the Chief Economist and Office of Policy Development in the Economics and Statistics Administration released the results of its first benchmark study in 1998, which pertained to contracting in fiscal year 1996, and updated those results in 1999. No further studies have been conducted (U.S. Office of Management and Budget, 1998, 1999).

The Commerce Department’s study has been harshly criticized for the lack of documentation of data sources, analytical methods, and limitations

7  

Estimates were computed by the steering committee staff, averaging results for each jurisdiction reporting more than one consistent, or inconsistent, disparity ratio, and taking the median of consistent (inconsistent) ratios across studies in an industry group.

Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×

of the results; for failure to develop meaningful industry groupings for a study of federal contracting; and for the lack of a theory of discrimination underlying the study (La Noue, 2000). We agree with the criticisms about the deficiencies of documentation, which make it difficult to assess the value of the approach for possible use by the SBA, and about the superficial approach to defining industries. Articulating a theory of discrimination requires defining the universe of available firms—whether it is appropriate to use an “all firms” definition from a source such as the Survey of Business Owners, or whether a definition of “ready, willing, and able” firms is more appropriate and what should be the precise definition of “ready, willing, and able.” We consider the suitability of various definitions in light of the intended uses of the calculated disparity ratios, but do not recommend a specific definition for the universe of available firms (see Chapter 6).

The only available documentation for the Commerce Department’s benchmark study is contained in a brief appendix to a Federal Register notice (U.S. Office of Management and Budget, 1998:35716-35717). It summarizes the methodology in the briefest terms.

Creating a Data Set

The Commerce Department first sought to assemble a data set of firms that were “ready and willing” to supply the federal government. The department rejected using the 1992 Survey of Minority-Owned Business Enterprises, presumably because it was out of date for a study of contracting experience in fiscal year 1996 and also because of uncertainty about how to screen out firms that were not interested in or capable of supplying the federal government. Instead, the department used three data sets for fiscal year 1996:

  1. Bidders from a sample of competitive procurements over $25,000. These firms were identified from a survey of federal contracting officers, who were asked to provide names of bidders for a sample of 16,616 new, competitive procurements stratified by industry and, for construction industries, by the 9 census geographic divisions. The survey had a 76 percent response rate.

  2. All firms that won sole-source or other noncompetitive procurements over $25,000.

  3. All firms certified by the SBA as active and eligible for Section 8(a) contracts, whether or not the firms won new contracts in fiscal year 1996.8

8  

The addition of all firms on the 8(a) list was criticized on the grounds that most of them do not bid in open competitions (La Noue, 2000:95).

Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×

The department matched the firms in these three data sets by taxpayer identification number and federal contracting arena (that is, major industry grouping corresponding to a 2-digit SIC category or, sometimes, a 1-digit SIC category) to eliminate duplication. It then added measures of size (annual payroll), age in years of existence, and profit-nonprofit status for each firm by matching with the Census Bureau’s 1995 Standard Statistical Establishment List (SSEL).9 It also resolved cases of inconsistent reporting of small disadvantaged business status. Finally, the department added a utilization measure for each successful bidder in the integrated data set by matching with the FPDS to obtain total net prime contract obligations over $25,000 awarded to a firm in fiscal year 1996.

Smaller contracts were excluded from the analysis because of the lack of detailed information about them. The department found that small disadvantaged businesses received a smaller percentage of small awards compared with awards over $25,000, so that excluding small awards would somewhat overestimate utilization.

Measuring “Ready, Willing, and Able”

The next step in the Commerce Department’s methodology was to measure the capacity of the ready and willing firms in the integrated data set. A dollar value for capacity was assigned to each firm equal to the geometric mean value of federal contracting work for contractors of a given size and age in a given industry group. The capacity values were estimated through regression equations for each industry group, using those firms in the integrated data set that were successful bidders in fiscal year 1996. The dependent variable for each such firm was the log of the utilization measure (amount of federal contract dollars); independent variables included the log of the number of years since the firm first appeared in the SSEL, a dummy variable if the firm first appeared before 1975, the log of 1995 payroll, interaction terms between the payroll and age variables, and a dummy variable if the firm certified that it met the SBA’s definition of a small business in the contracting arena. For firms missing one or more of the independent variables for the equations and for nonprofit and government establishments, the mean log of utilization was computed separately within industry group.

9  

The SSEL has been renamed the Business Register; it is confidential and was used under an arrangement by which Department of Commerce staff were sworn as special census agents and worked on-site at Census Bureau headquarters in Suitland, Maryland.

Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×
Calculating Utilization and Availability Shares

Finally, the Commerce Department used the integrated data set to measure utilization and availability shares for small disadvantaged businesses in major industry groups. Utilization shares were calculated as the total value of prime contract dollars awarded in fiscal year 1996 to small disadvantaged businesses in the data set divided by the total value of prime contract dollars awarded to all firms in the data set. Availability shares were calculated as the total capacity value assigned to all small disadvantaged businesses in the data set divided by the total capacity value assigned to all firms in the data set.

Based on the results, the Department of Commerce determined that small disadvantaged firms were eligible for a 10 percent price evaluation adjustment, or bid-credit, in 71 industry groups, and for 3 construction industries, within each of 9 geographic divisions. By comparison, the SIC includes 83 2-digit industry categories in all. An updated analysis, released in 1999, limited the bid-credit to small disadvantaged businesses in 51 industry groups plus each of 9 geographic divisions for 3 construction industries. No documentation was published of the threshold level of disparity used by the department in determining eligible industry groups.

Assessment

On the positive side, the Department of Commerce study exhibited diligent and innovative work to attempt to develop a defensible measure by which to classify firms as ready, willing, and able to supply federal procurement needs for prime contracts over $25,000. Utilization shares and availability shares were also consistently defined in dollar terms.

Greatly impairing the usefulness of the study results and methodology, however, is the lack of documentation for key components in the estimation, particularly for the regression equations that were used to predict capacity values. For example, there is no description of the values of the estimated coefficients for the independent variables, no information on standard errors or how much of the variance in the dependent variable is explained by the independent variables, no analysis of the distribution of the residuals (that is, the differences between the predicted and reported values of the dependent variable for each observation) to look for possible prediction biases, no discussion of alternative functional forms of the regressions that were tested (if any), and no reports of outliers and their effects on the estimated coefficients. Analyses of these kinds are essential to verify that a regression equation performs well for the observations for which it is estimated.

Also important for evaluation is to cross-validate the analysis by find-

Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×

ing ways to compare the regression predictions with a set of target or “true” values that were not used to develop the equations (see National Research Council, 2000:Ch. 6). In this application, it would have been possible to estimate the equations on a random half-sample of the observations for successful bidders, use the equation results to predict capacity for other successful bidders in the integrated data set, and compare the predicted values with the reported values of contract dollars awarded for the other successful bidders. Whether such analysis was done for the equations used by the department or alternative forms of those equations is not known.

SBA CAWBO Study

The preliminary CAWBO study of women-owned businesses in federal contracting was summarized for the committee in the open session of its spring 2004 workshop. Later on in the project, the committee members were able to receive a copy and examine it. The committee does not comment on specific estimates in the document, but only on general methodological points.

Methodology

The basic methodology used by CAWBO followed that of many of the state and local contracting studies reviewed by the Urban Institute, and it refers to the Urban Institute analysis as the basis for a number of methodological decisions (Enchautegui et al., 1997). Like many of the state and local studies, CAWBO used inconsistent definitions for the utilization share, U, and the availability share, A, in calculating disparity ratios within industry group. This difference may have affected the estimates of disparity ratios for industries, although whether the effects were small or large is not known by the committee.

For the utilization share (U), the CAWBO study defined both the numerator and the denominator in monetary terms. The numerator of U was defined as total dollars of federal prime contracts over $25,000 awarded in fiscal year 1999 to women-owned small businesses as defined by the SBA. The denominator was defined as total dollars of federal prime contracts over $25,000 awarded in fiscal year 1999 to all firms, including women-owned small businesses, nonwomen-owned small businesses, and larger businesses. The data source was the FPDS, which contains detailed information for all federal prime contracts over $25,000, accounting for over 90 percent of the $200 billion spent in fiscal year 1999. Contracting data for fiscal year 1999 were used instead of the most recent data available at the

Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×

time of the study (for fiscal year 2000), in order to be more comparable to the reference period for the SWOBE data used to measure availability.

For the availability share (A), the CAWBO study defined both the numerator and the denominator as counts. The numerator of A was defined, not in dollar terms similar to the numerator of U above, but as the number of women-owned businesses (firms) with paid employees identified in the 1997 SWOBE. The denominator of A was defined as the total number of business firms with paid employees, including women-owned businesses (assumed to be primarily small businesses), other small businesses, and larger businesses, from the same 1997 survey.

Defining underrepresentation as a disparity ratio of 0.80 or less, the CAWBO study estimated that women-owned small businesses were underrepresented in all but 5 of 71 industry groups (2-digit SIC categories) for which disparity ratios were calculated. Defining substantial underrepresentation to be a disparity ratio of 0.50 or less, the study estimated that women-owned small businesses were substantially underrepresented in 56 industry groups. The CAWBO study stated that it chose 0.80 as the level between representation and underrepresentation to allow for the possibility of measurement error. That level is also congruent with the Urban Institute meta-analysis and the rule of thumb used by the Equal Employment Opportunity Commission in employment discrimination cases. The CAWBO study stated that it chose 0.50 as the level between underrepresentation and substantial underrepresentation to be conservative, noting that the few studies that offered a level set it at higher than 0.50.

Assessment

A major limitation of the CAWBO study is the same as that of the Department of Commerce study reviewed above—namely, incomplete and unclear documentation of data sources and estimation methods and the lack of any published sensitivity analysis that would indicate the robustness of the estimated disparity ratios to alternative measures of utilization and availability. Tables and graphs are not clearly labeled, and the steps followed to evaluate and select data sources and construct estimates are not clearly described. The presentation is in the form of a series of questions and answers (for example, “Why use FY 1999 data when the FY 2000 data are available for contracting?”). Such a presentation could be useful to provide as a supplement for stakeholders, but it does not substitute for a clear, ordered description of estimation techniques and procedures. Finally, although the document mentions many studies that were consulted, it provides only two references.

The CAWBO document indicates that a variety of data sources were examined for measuring availability prior to selecting the 1997 SWOBE

Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×

data on firms with paid employees to use in the analysis. Specifically, the study examined seven data sets: (1) the SBA PRO-Net database of small businesses registered to do contracting with the federal government, which carries a self-designation of women-owned status; (2) the 1997 SWOBE tabulation for all firms; (3) the 1997 SWOBE restricted to firms with paid employees; (4) a special tabulation of the 1997 SWOBE for all firms in which some firms reporting 50-50 male-female ownership were assigned to the women-owned category (as was done in the 1992 SWOBE); (5) the same special tabulation restricted to firms with paid employees; (6) the Department of Defense CCR; and (7) the Department of Commerce database of firms active in federal prime contracting in fiscal year 1999, including contract winners and bidders from a survey of contracting officers.

The CAWBO document does not present disparity ratios estimated using these various sources. It presents a summary justification for selecting the 1997 SWOBE data for firms with paid employees, but it does not indicate whether that justification applied to all industries. According to the document, this choice was conservative in that the only data set indicating that women-owned small businesses were a smaller share of all businesses than the SWOBE data on firms with paid employees was the Department of Commerce file of successful and unsuccessful bidders.

CAWBO declined to use the Department of Commerce data set (which may be subject to significant sampling and nonsampling errors), given its belief that more firms are ready, willing, and able to be federal contractors than those on the list of active bidders. To assess the validity of this belief, CAWBO looked at a sample of industries to determine if only the largest firms could handle the typical federal contract. The document summarizes the findings, stating that the majority of contracts in an industry are below the average size contract (a small number of large contracts pull up the average), and that women-owned small businesses have obtained contracts at the average size, indicating that firms with employees in an industry should be able to handle the typical contract. CAWBO noted that small firms can subcontract, engage in joint ventures, hire temporary staff, or take other steps to expand their capacity as needed.

CAWBO rejected the idea of a regression analysis to estimate the capacity of available firms to handle federal contracting on the grounds of lack of time and resources to assemble the necessary firm-specific data. CAWBO also said that the SBA was not charged to explain observed disparities in federal contracting within industries, but simply to measure them.

Finally, the CAWBO document summarizes an analysis of the success of women-owned small businesses in the private sector compared with the government sector. Marketplace disparity ratios were calculated for every industry—the method is not explained but may have involved comparing

Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×

shares of numbers of firms with shares of business receipts for women-owned businesses. The marketplace ratios were then compared with the federal contracting disparity ratios. Women-owned small businesses were underrepresented in the private market as well, but not as much as in the government market, a finding that CAWBO believes supports the conclusion that capable women-owned small businesses are not being well used in federal contracting. This and the other analyses summarized in the CAWBO document indicate the hard work that the analysts put into the study. The analyses would be more useful and convincing if the detailed results were provided, as was done in the Urban Institute meta-analysis. Our recommendations regarding the CAWBO study and the development of disparity ratios to respond to the SBA’s congressional mandate are presented in Chapter 6.

Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×
Page 38
Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×
Page 39
Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×
Page 40
Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×
Page 41
Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×
Page 42
Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×
Page 43
Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×
Page 44
Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×
Page 45
Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×
Page 46
Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×
Page 47
Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×
Page 48
Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×
Page 49
Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×
Page 50
Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×
Page 51
Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×
Page 52
Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×
Page 53
Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×
Page 54
Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×
Page 55
Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×
Page 56
Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×
Page 57
Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×
Page 58
Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×
Page 59
Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×
Page 60
Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×
Page 61
Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×
Page 62
Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×
Page 63
Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×
Page 64
Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×
Page 65
Suggested Citation:"4 Disparities and Representation." National Research Council. 2005. Analyzing Information on Women-Owned Small Businesses in Federal Contracting. Washington, DC: The National Academies Press. doi: 10.17226/11245.
×
Page 66
Next: 5 Measuring Discrimination »
Analyzing Information on Women-Owned Small Businesses in Federal Contracting Get This Book
×
Buy Paperback | $40.00 Buy Ebook | $31.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

It has been clear for at least 50 years the disadvantages that small businesses face in competing for U.S. government contracts. The Small Business Act of 1953 created the Small Business Administration (SBA), an independent agency in the executive branch that counsels and assists specific types of small businesses including firms owned by minorities and other socially and economically disadvantaged individuals and firms owned by women. Women-owned small businesses, however, are underrepresented or substantially underrepresented in some industries.

In 2002, the SBA Office of Federal Contract Assistance for Women Business Owners (CAWBO) organized a draft study containing a preliminary set of approximations of the representation of women-owned small businesses in federal prime contracts over $25,000 by industry. Because of the past legal challenges to race- and gender-conscious contracting programs at the federal and local levels, the SBA asked the Committee on National Statistics of the National Academies to conduct an independent review of relevant data and estimation methods prior to finalizing the CAWBO study.

The Steering Committee on Women-Owned Small Businesses in Federal Contracting was created and charged with holding a workshop to discuss topics including the accuracy of data and methods to estimate the use of women-owned small businesses in federal contracting and the definition of "underrepresentation" and "substantial underrepresentation" in designating industries for which preferential contracting programs might be warranted. Analyzing Information on Women-Owned Small Businesses in Federal Contracting presents the committee's report as well as the recommendations that committees have made.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!