4
Disparities and Representation

Disparity, underrepresentation, and discrimination are different concepts. Disparity is simply a measured difference between two groups on an outcome of interest, such as differences in average earnings between men and women. Underrepresentation is a disparity in which the difference goes against a particular group (for example, lower earnings for women than men). Discrimination involves differential (adverse) treatment of a group compared with others based on membership in the group. Discrimination may also involve differential treatment on the basis of other factors that results in an adverse outcome for a particular group. Disparities may be due to any number of factors, including, but not necessarily and not limited to, discriminatory practices and behaviors.

This chapter focuses on the definition and measurement of disparities and underrepresentation of women-owned small businesses in federal contracting. It thereby responds to the committee’s charge to review the preliminary study, completed in late 2002, by the Office of Federal Contract Assistance for Women Business Owners (CAWBO) of the Small Business Administration (SBA). Chapter 5 considers the challenging task of inferring discrimination as a possible reason for any observed adverse disparities or underrepresentation. Our recommendations about the CAWBO study are presented in Chapter 6.

DISPARITY

Evidence of large and persistent differences, or disparities, in economic outcomes among men and women in the United States is not hard to find.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 38
Analyzing Information on Women-Owned Small Businesses in Federal Contracting 4 Disparities and Representation Disparity, underrepresentation, and discrimination are different concepts. Disparity is simply a measured difference between two groups on an outcome of interest, such as differences in average earnings between men and women. Underrepresentation is a disparity in which the difference goes against a particular group (for example, lower earnings for women than men). Discrimination involves differential (adverse) treatment of a group compared with others based on membership in the group. Discrimination may also involve differential treatment on the basis of other factors that results in an adverse outcome for a particular group. Disparities may be due to any number of factors, including, but not necessarily and not limited to, discriminatory practices and behaviors. This chapter focuses on the definition and measurement of disparities and underrepresentation of women-owned small businesses in federal contracting. It thereby responds to the committee’s charge to review the preliminary study, completed in late 2002, by the Office of Federal Contract Assistance for Women Business Owners (CAWBO) of the Small Business Administration (SBA). Chapter 5 considers the challenging task of inferring discrimination as a possible reason for any observed adverse disparities or underrepresentation. Our recommendations about the CAWBO study are presented in Chapter 6. DISPARITY Evidence of large and persistent differences, or disparities, in economic outcomes among men and women in the United States is not hard to find.

OCR for page 38
Analyzing Information on Women-Owned Small Businesses in Federal Contracting For example, for as long as wages have been measured, women have been estimated to earn less than men, although the gap has lessened over time and among some groups (see, e.g., Bureau of Labor Statistics, 2002). By occupation, adult women currently outnumber adult men in such occupations as nurses, elementary school teachers, social workers, bank tellers, and librarians, whereas men outnumber women in such occupations as purchasing managers, dentists, carpenters, firefighters, and mail carriers. By industry, more women are employed than men in the health care and education sectors and more men are employed in agriculture, construction, and mining (U.S. Census Bureau, 2003:Tables 615, 619). Properly measured, these examples illustrate disparities on the basis of gender. The Small Business Reauthorization Act of 2000, Section 811(m), required the SBA to determine industries in which women-owned small businesses were “underrepresented” and “substantially underrepresented.” In other words, SBA was required to identify industries that evidenced disparities or differentials between women-owned small businesses and other businesses in which the disparity was adverse to women-owned small businesses. Disparity Ratio Congress did not indicate how to measure disparities, adverse or otherwise. Studies of preferential contracting programs commonly use a measure termed the “disparity ratio,” D. Calculating the disparity ratio begins by calculating the values of two shares for a target group, in this case women-owned small businesses. The two shares are a utilization share, U, and an availability share, A.1 The utilization share looks at an outcome of interest, which in this case is winning government contracts, measured in number of contracts or dollars awarded. The utilization share measures contracts (or dollars) awarded to women-owned small businesses, Cw, as a share of total contracts (or dollars) awarded, Ct. The availability share looks at the available universe or pool for contracting, measured as numbers of businesses or gross dollar receipts. The availability share measures women-owned small businesses (or their gross receipts), W, as a share of total businesses (or total gross receipts), T. Taking the two shares as a ratio gives an estimated value for the disparity ratio: 1   The terms “utilization” and “availability” are defined in the disparity ratio literature, which has addressed primarily minority contracting disparities (see, e.g., Enchautegui et al., 1997; Marcus Weiss & Affiliates, 1990; Mason Tillman Associates, 1998).

OCR for page 38
Analyzing Information on Women-Owned Small Businesses in Federal Contracting D = U / A, where U, or utilization = Cw / Ct and A, or availability = W / T. (1) If D is 1.00, then there is no disparity for women-owned small businesses: their actual share, U, of contracts is the same as their expected share, A, based on their representation in the total business population. If the ratio is less than 1.00, then there is an adverse disparity or underrepresentation of women-owned small businesses among successful government contractors relative to the total business population. If the ratio is more than 1.00, then women-owned small businesses are overrepresented among successful government contractors relative to the total business population. A more statistically tractable and interpretable alternative widely used in the legal and academic analysis of employment discrimination takes the difference between U and A as a measure of disparity. Thus, a difference of zero would indicate no disparity; a difference of less than zero (i.e., U is a smaller proportion than A) would indicate underrepresentation of the target group for the outcome of interest relative to the total population; a difference of greater than zero (i.e., U is a larger proportion than A) would indicate overrepresentation of the target group. In the contracting arena, the availability share for such a target group as women-owned small businesses can be expected to vary across industries and other characteristics of businesses and contracts. For this reason, it is critical to use disparity ratios (or differences) to measure representation and not simple counts or percentages of utilization. For example, if industry A has 10 percent women-owned small businesses and industry B only 2 percent women-owned small businesses, a comparison of counts or percentages of contracts awarded would be misleading because one would not expect women-owned small businesses to win as many contracts in industry B as in industry A. Differences among industries also suggest the importance of examining each industry separately and not lumping industries together in the calculation of a single disparity ratio (or difference). Separate ratios should be calculated instead. The example in Table 4-1 makes this point clear. Constituent Elements Critical to appropriate calculation of disparity ratios is an agreed-on definition and measurement of each element of the ratio—the target group, the outcome of interest, and the total population—and of any other variables to be included in the analysis, such as industry. We first discuss issues of defining and measuring the constituent elements of the disparity ratio

OCR for page 38
Analyzing Information on Women-Owned Small Businesses in Federal Contracting TABLE 4-1 Hypothetical Calculation of Disparity Ratios for Two Industries   Industry A Industry B Total a. Women-owned federal contractors 25 50 75 b. Total federal contractors 1,000 1,000 2,000 c. Women-owned firms 1,000 1,000 2,000 d. All firms 10,000 50,000 60,000 Utilization share, U = (a) / (b) 0.025 0.05 0.038 Availability share, A = (c) / (d) 0.10 0.02 0.033 Disparity ratio, D = (U / A) 0.25 2.50 1.15 NOTE: Calculating the disparity ratio for the two industries combined would lead to the conclusion that women-owned small businesses were overrepresented in federal contracting, when the data indicate that they are underrepresented in industry A. and then summarize and assess the approach to these issues in the SBA CAWBO study and other studies. Target Group The target group of interest for the congressionally mandated SBA study comprises women-owned small businesses. There is a clear formal definition in the legislation governing the SBA of businesses that are “women-owned” (see Box 2-2). In implementing the definition, errors of classification can occur because of differences between the SBA definition and the definition employed in a relevant data source. For example, the 1997 Survey of Women-Owned Business Enterprises conducted by the U.S. Census Bureau as part of the 1997 Economic Census provided separate tabulations for firms with 50-50 male-female ownership and those with 51 percent or greater female ownership, but earlier surveys in the series counted some equally owned businesses as women-owned. This change reduced the number of women-owned businesses in the 1997 survey by an estimated 37 percent (U.S. Census Bureau, 2001:12).2 2   This survey and the Survey of Minority-Owned Business Enterprises have been renamed the Survey of Business Owners and Self-Employed Persons; the most recent round is for 2002.

OCR for page 38
Analyzing Information on Women-Owned Small Businesses in Federal Contracting In addition, there may be efforts to game the status of being woman-owned, which could affect not only the quality of the data on which the targeting of a set-aside program depends, but also its effectiveness. Businesses may fraudulently claim to be women-owned according to the official definition, or they may transfer ownership but not actual control to a woman. Front organizations may also be formed for the purpose of bidding on contracts that are then substantially subcontracted to other firms that do not meet the definition. In each such case, the estimated number of women-owned small businesses exceeds the true number and contracts are diverted from legitimate women-owned small businesses. The SBA also operates with a formal definition of “small,” or, rather, it has a set of definitions, which vary by industry in terms of the variable used for measurement (e.g., number of employees, gross receipts) and the threshold value (e.g., maximum of 100 employees for a retail trade business, maximum of $3 million average annual receipts for a travel agency) (see Box 2-2). Errors of classification can occur because of differences between the SBA definition and the definition employed in a relevant data source, advertent or inadvertent reporting errors for measures of firm size, fluctuations in firm size, differences in organizational form, and other factors. Contracting preferences for small businesses create an incentive to maintain status as an SBA-defined small business. Outcome of Interest The outcome of interest for the SBA study is federal contract awards, which can potentially be defined in terms of numbers of awards or dollar amounts of awards. The latter measure is most commonly used in the literature, which reflects the fact that the yearly goals for use of women-owned and other types of small businesses established by Congress and negotiated between the SBA and individual contracting agencies represent percentages of total awarded dollars, usually separately for prime contracts and subcontracts. In addition, data on dollar amounts awarded are readily accessible from the Federal Procurement Data System (FPDS), whereas data on number of contract awards require combining FPDS records for individual actions (new awards, modifications, etc.) into records for individual awards, or else using contract actions as a rough proxy for awards. Data on total size of awards for individual contracts also require combining FPDS individual action records. Although defining the outcome measure in terms of dollar amounts awarded makes sense for estimating utilization shares to respond to the congressional mandate, it would also be useful to examine utilization shares in terms of contract awards, or actions, in order to better understand the contracting picture for women-owned small businesses. Table 3-1 in Chap-

OCR for page 38
Analyzing Information on Women-Owned Small Businesses in Federal Contracting ter 3 shows that shares in terms of dollars and actions are similar for many cabinet departments, but there are exceptions. Notably, the Department of Energy in fiscal year 2003 exhibited a relatively high share of contract actions going to women-owned small businesses (15 percent), but the share of its contract dollars awarded to women-owned small businesses was only 0.5 percent. This finding presumably reflects the fact that this department has a small number of very large contracts for its laboratories, which are neither bid on nor won by small businesses. Because the distribution of contract awards by dollar value may be highly skewed for some agencies, time periods, or industries, a careful analysis of utilization estimates should determine their sensitivity to outliers and whether some contract awards should be excluded from the estimation. For example, the distribution of prime contract dollars awarded by the Defense Department for fiscal year 2004 (data not yet available) may be skewed upward by the small number of very large contracts awarded for military support operations and reconstruction in Iraq. With regard to the universe of awards to include, it would be desirable to examine separately awards between $2,500 and $100,000, as procurements in this dollar range are designated for small businesses (to the extent feasible). Utilization shares for awards under $100,000 would be informative regarding the relative success of women-owned small businesses compared with other small businesses in obtaining these small-size contracts, whereas the shares for larger awards would be informative regarding the success of women-owned small businesses in obtaining contracts that larger businesses may bid on as well. Because the congressionally mandated study of women-owned small businesses requires estimating disparity ratios separately by industry, the universe of contract actions must of necessity be limited to those over $25,000. The FPDS does not include smaller awards, and only very limited information, not including industry classification, is maintained for small contracts by the General Services Administration (see Chapter 3).3 The detailed information in the FPDS for awards over $25,000 permits estimating utilization shares for smaller and larger award size categories. Also, with the currently available data, disparity ratios can be calculated only for prime contract awards but not for subcontracts. The many small businesses that bid on and win subcontracts are not recorded in the FPDS. 3   Contracting officers may fill out SF-279, which is input to the FPDS, for contracts as small as $10,000, but most such contracts are combined and aggregate information for them entered onto the SF-281.

OCR for page 38
Analyzing Information on Women-Owned Small Businesses in Federal Contracting Total Population The total population of interest for the SBA study concerns businesses operating in the United States. The population can potentially be defined in terms of numbers of businesses or total business receipts. Typically, only the first definition has been used in the literature for estimating availability shares. Whichever measure is used (numbers or dollars), it is important that it is consistent with the measure used for the outcome of interest (see “Consistency of Elements,” below). Specification of an appropriate population measure must also consider the universe of firms to include, specifically, the treatment of very small firms and other firms that may not be ready, willing, and able to bid on federal procurements. Very small firms Small businesses are defined to include not only corporations and partnerships, but also sole proprietorships (see Box 2-2). Consequently, a broad definition of the business universe is likely to include a large number of very small firms defined by some metric. One available measure of smallness is a business having no paid employees. Thus, data from the 1997 Survey of Women-Owned Business Enterprises, which included in its universe all firms operating in the United States with $1,000 or more in gross annual receipts, showed that 75 percent of the estimated total number of 20.8 million firms had no paid employees. By gender of ownership, 84 percent of the estimated number of 5.4 million women-owned firms had no paid employees, as did 67 percent of 3.6 million equally male and female-owned firms, 45 percent of 11.4 million male-owned firms, and 33 percent of 0.4 million publicly held, foreign-owned, and nonprofit firms, which are not classified by gender of ownership (U.S. Census Bureau, 2001:12). These percentages may be misleading, however, regarding the capabilities of small businesses to handle federal contracts. Some businesses without paid employees may in fact be much larger operations because they use such mechanisms as hiring independent contractors as needed instead of having salaried employees. Another, perhaps more appropriate, measure of smallness is sales volume. For 1997, 20 percent of all firms with $1,000 or more gross receipts had less than $5,000 in gross receipts; the comparable figure for women-owned businesses was 33 percent. These data suggest that significant fractions of businesses may represent occasional efforts of people who are fully employed as wage and salary workers and do not intend to grow their “business.” An example would be a full-time employed manager or teacher who reported a couple of thousand dollars of income from honoraria for presenting papers at a few meetings.

OCR for page 38
Analyzing Information on Women-Owned Small Businesses in Federal Contracting Given that available contracting data limit the estimation of utilization shares to contract awards over $25,000, then the universe of firms for estimating availability shares should be limited in some appropriate manner. For example, the universe could be limited to firms that reported a specified minimum amount of gross receipts in the 1997 Survey of Women-Owned Business Enterprises. Determining the minimum amount of gross receipts would require careful analysis of the distribution of gross receipts for small businesses and could perhaps be informed by data from the Central Contractor Registration (CCR) on the size distribution of small businesses that have registered to do business with the federal government. (The CCR obtains data on number of employees and 3-year average annual receipts.) If separate utilization shares are calculated for smaller and larger contract awards (say, $25,000 to $100,000 and $100,000 or more), as suggested above, then the universe of firms should be appropriately classified for calculating availability shares. For smaller contract awards, the universe could be limited to small businesses as defined by the SBA (perhaps excluding some very small businesses), while, for larger awards, it could include both small and larger businesses. Other criteria for defining “ready, willing, and able” A related issue that arises in determining an appropriate universe of firms for estimating availability shares is whether to use criteria in addition to firm size to further limit the universe to firms that are likely bidders on federal contracts. This decision depends largely on the intended use of the disparity ratios. Possible uses are to help justify remedial preferential contracting programs for women-owned small businesses in court, to help refine the design of such programs, or to be part of a broad analysis of barriers that women may face in business ownership and growth. If disparity ratios are to be used in a comprehensive analysis of barriers that women may face in business ownership and growth, which could, in turn, produce underrepresentation in federal contracting, then a broad definition of the total population of firms for estimating availability shares is likely to be appropriate. If, however, disparity ratios are to be used in court to justify remedial preferential contracting programs for women-owned small businesses, then a broad definition is likely to be inappropriate. Case law in this area has put forward a standard for the total population universe of firms, which includes those that are ready, willing, and able to be federal contractors (see Chapter 2). The notion is that remedial programs are justified when there is evidence of possible discrimination by the federal contracting process against a class of firms in the “ready, willing, and able” universe. Remediation is not necessarily justified, according to case law, when the evidence pertains to a broader universe of

OCR for page 38
Analyzing Information on Women-Owned Small Businesses in Federal Contracting firms, which may indicate societal barriers to business development and growth by a class of firms, but not additional discrimination in the contracting process. Not only would a “ready, willing, and able” standard for procurements over $25,000 probably exclude very small businesses from the total population universe, but it would also probably exclude other businesses that are too new or are not technically qualified for one or another reason in bidding on federal contracts. Some studies of disparities in state and local contracting have used restricted business lists, such as contractor registries and survey respondents that report interest in or efforts to obtain contracts, to estimate availability shares. The Department of Commerce used a regression methodology with several data sources to estimate a ready, willing, and able population of firms for estimating availability shares for federal contracting (see “Disparity Ratios in the Literature,” below). At present, the CCR maintained by the Department of Defense for the entire federal government could be used for this purpose. Beginning October 1, 2003, every business that intends to bid on a federal contract must register on the CCR. Still, it could be argued that the CCR is too limited a list and that other ready, willing, and able firms exist that, for one or another reason, are not registered on the CCR. Ideally, one would be able to estimate availability shares and calculate disparity ratios with broader and narrower definitions of the total population. Constructing and comparing multiple measures would provide a fuller picture of the role of women-owned small businesses in economic activity, generally, and in federal contracting specifically. It may also facilitate an investigation of where in the business development and contracting process disparities arise. Consistency of Elements A statistically defensible disparity ratio for women-owned small businesses (or another target group) in federal contracting requires consistency among its elements. In particular, if the utilization share is defined in terms of contract award dollars, as is most commonly done, then the availability share should be defined in monetary terms as well, such as annual gross receipts. Most often, however, disparity studies have used inconsistently defined elements, such as contract award dollars for the utilization share and number of firms for the availability share. The extreme implicit assumption is that every business in the available pool is equally ready, willing, and able to bid on and perform every contract. Some studies have used consistent measures, such as number of contract awards and number of firms. We did not find examples of studies that have used contract award dollars and

OCR for page 38
Analyzing Information on Women-Owned Small Businesses in Federal Contracting gross receipts. A Department of Commerce study used contract award dollars and estimated dollar value of capacity for federal contracting (see “Disparity Ratios in the Literature,” below). One reason for not using gross receipts may be the belief that the availability shares for women-owned small businesses would be systematically smaller because of the very large gross earnings of a small number of very large firms. To the extent this distortion occurs, then the resulting disparity ratios would be closer to 1.0, perhaps undercutting the rationale for preferential contracting programs. More substantive is a concern that the current size, capabilities, and resources of a firm may themselves be influenced by past discrimination. Yet an inconsistent calculation that bases utilization shares on contract dollars awarded and availability shares on number of firms could well produce downwardly biased disparity ratios, thereby overstating the need for preferential contracting programs. The reason is the heavily skewed distribution of U.S. firms by size. Small women-owned businesses make up a disproportionately large percentage of firms compared with their share of business receipts. Publicly held corporations that account for the majority of business receipts have no official gender, although their shareholders do. The gender of ownership of a business is only specified in the case of proprietorships, partnerships, and Schedule C businesses. These types of businesses are generally considerably smaller in scale than corporations. If the women-owned share of businesses is calculated only among businesses for which the gender of ownership is known, then, for consistency, the women-owned share of contracting should be calculated only among contracts awarded to such businesses. A careful analysis will look at the results of several different types of disparity ratios. To the extent that different measures—such as receipts-based measures, numbers-based measures, measures that vary the lower limit of receipts for inclusion in the universe of firms, and measures that exclude outliers (e.g., very large contract awards, very large businesses in terms of receipts)—tell a similar story, then the justification for (or against) preferential contracting programs will be strengthened. To the extent that different measures tell very different stories (e.g., the large percentage of contract actions compared with the small percentage of contract dollars awarded to women-owned small businesses by the Department of Energy), then additional analysis will be required to make a case for preferring a particular measure or set of measures among all those calculated. In addition to consistency among the elements of the disparity ratio, the reference periods for the utilization and availability shares should be as close in time as possible. The number of women-owned small businesses has grown rapidly in the recent past, with an estimated 16 percent increase over the period 1992 to 1997 (37 percent among firms with paid employ-

OCR for page 38
Analyzing Information on Women-Owned Small Businesses in Federal Contracting ees), compared with a 6 percent growth rate in the same time period for all businesses, with or without paid employees (U.S. Census Bureau, 2001:12). As a consequence, if the time period for the availability share lags the time period for the utilization share by more than a year or so (as is likely to be the case given data availability), then the disparity ratio will probably be biased upward, indicating a better position for women-owned small businesses in federal contracting compared with their position in the business sector than is probably true. Industry Breakdowns The congressionally mandated study of women-owned small businesses in federal contracting requires estimates of representation by industry. The current standard set of industry codes, the North American Industry Classification System (NAICS), has five levels of classification. At the broadest level, NAICS consists of 20 industry sectors, identified by the first two digits of the individual 6-digit industry codes (1,170 in all). Subsectors, industry groups, NAICS industries, and U.S. industries are identified by the first three digits, four digits, five digits, and six digits, respectively, of the specific industry codes. The NAICS system, developed jointly by Canada, Mexico, and the United States, was first issued in 1997 and most recently updated in 2002. It replaced the previous 4-digit Standard Industrial Classification (SIC) system, last updated in 1987, which emphasized the manufacturing sector. NAICS focuses on new and emerging industries, industries using new technology, and service industries. The 2002 NAICS update made substantial revisions in the construction and wholesale trade sectors and minor revisions in the retail and information sectors (see www.census.gov/epcd/www/naics/html [December 2004]). A key issue for calculating disparity ratios (or differences) by industry is which level of NAICS codes to use. The 20 industry sectors (defined by the first two digits in the NAICS coding scheme) appear too broad to be used as the basis of disparity ratios to inform understanding of the role of women-owned small businesses in federal contracting and what kinds of preferential treatment may be indicated. At the other extreme, there are so many specific industries identified by 6-digit NAICS codes that the resulting disparity ratios are likely to be adversely affected by sampling variability and also by errors of classification. Ideally, the level of NAICS codes used for calculation of disparity ratios will be chosen to optimize two criteria. First, each industry-specific disparity ratio should meet a specified reliability standard (see “Data Quality Issues,” below). Second, the level of industry detail should be as disaggregated as the data will support so that preferential treatment programs (if

OCR for page 38
Analyzing Information on Women-Owned Small Businesses in Federal Contracting small businesses pursuant to the Small Business Reauthorization Act of 2000, using 1997 SWOBE data on businesses with paid employees to calculate availability shares. We review each of these efforts below. With the exception of the Urban Institute meta-analysis, the possible usefulness of the various study results is severely impaired by the poor quality of documentation of the methods and data sources employed. State and Local Disparity Studies Over 100 jurisdictions have conducted studies of the contracting experience of minority-owned small businesses and, in some instances, women-owned small businesses. These studies have been criticized as too often relying on anecdotal evidence of discrimination provided by minority and women owners or on a general history of discrimination in the locality and not on statistical evidence of disparities (see, e.g., La Noue, 1992, 1995). We were not able to review any of the individual studies, which, in any case, deal with state and local contracting, not federal contracting. We reviewed the Urban Institute meta-analysis (Enchautegui et al., 1997), which combined results from many of these studies to estimate statistical disparities across states and localities in contracting for minority-owned and women-owned small businesses in broad industry groups. The Urban Institute study was conducted for the U.S. Department of Justice as part of an evaluation in the mid-1990s of the need for federal preferential contracting programs (see Chapter 2). Its approach and findings provide useful insights for the estimation of valid, informative disparity ratios. Urban Institute Meta-Analysis The Urban Institute collected reports of 95 state and local government-commissioned disparity analyses and reviewed them to screen out those that did not provide relevant or statistically defensible results. Studies were included only if they satisfied all four of the following criteria: they provided disparity ratios or the data to construct ratios and not just anecdotes; they reported findings separately by industry category; they reported the number of contracts used in the analysis or the statistical significance of the calculated ratios; and they included more than 80 contracts in total. In addition, some studies that satisfied these criteria were dropped because of significant inconsistencies in their results or poor documentation for key calculations. After the screening, 58 studies remained for the Urban Institute meta-analysis. These 58 studies varied in their measures of utilization and availability for constructing disparity ratios. Most commonly, utilization was measured

OCR for page 38
Analyzing Information on Women-Owned Small Businesses in Federal Contracting in monetary terms, either as the dollar amount of contract awards or the dollar amount actually paid out. Some studies also defined utilization in terms of numbers of contract awards. Measures of availability were expressed in numerical terms (numbers of firms) and never in monetary terms (e.g., gross receipts), and they differed in the universe definition. The six most common universe definitions, listed in order from a narrow to a broad definition of availability, were previous award winners (firms on vendor lists), firms that bid on past contracts or appeared on lists to receive information about procurements, firms certified as minority owned or women owned, firms that expressed interest in government contracting work in response to a survey, all firms with paid employees, and all firms. Some studies used more than one definition. Availability measures based on all firms or all firms with paid employees used data from the 1987 SWOBE. To combine results across studies, the Urban Institute first averaged the disparity ratios for each study (jurisdiction) that reported more than one ratio (most did) and then took the median of the study averages in order to minimize the effects of outliers. The averaging was performed separately for groups defined by ownership status (black, Hispanic, Asian, American Indian, women-owned) and industry category (construction, construction subcontracts, goods, professional services, other services).6 The Urban Institute estimated an overall disparity ratio of 0.29 for women-owned businesses in state and local contracting. This ratio was estimated to differ significantly from a chance result, using a 0.05 probability test. Estimated disparity ratios for women-owned businesses by industry category varied from 0.17 for professional services to 0.77 for construction subcontracting (the latter estimate was not statistically significantly different from 0.80 or 1.00). The Urban Institute tested the sensitivity of the results to several methodological features of the various studies. Disparity ratios were calculated separately for studies with large numbers of contracts or high levels of availability of minority-owned or women-owned businesses and all other studies. Separate ratios were also calculated for each of three research firms that conducted multiple disparity studies for state and local governments and all other studies. These breakdowns did not alter the picture conveyed by the overall results. Disparity ratios for women were low, no matter how they were computed—that is, women-owned businesses received a very small share of contracts and contract dollars compared with their share of the business population (Enchautegui et al., 1997:Tables 2.4, 2.9). 6   The results are not specific to minority-owned or women-owned businesses that are small businesses as defined by the SBA. Data from the 1997 SWOBE indicate that a very high percentage of women-owned businesses (99 percent) would probably be classified as small by the SBA (U.S. Census Bureau, 2001:Table 9).

OCR for page 38
Analyzing Information on Women-Owned Small Businesses in Federal Contracting The Urban Institute also tested the sensitivity of the results to the universe definition for the measure of availability, specifically, whether the measure used SWOBE data on all firms or all firms with paid employees to define the universe, or, instead, used a measure that could be construed as “ready, willing, and able” (e.g., defining the universe in terms of registered bidders). Using SWOBE data could present several problems (see Enchautegui et al., 1997:70-71). Some of these problems, if not corrected, would probably generate overestimates of availability—for example, the fact that SWOBE includes very small businesses and prior to 1992 excluded C-corporations (primarily those with more than 35 shareholders). Other problems would probably generate underestimates of availability—for example, known undercounts of Hispanic-owned and Asian-owned businesses in SWOBE. Underestimation might also occur if the survey reference year preceded the estimation year in a particular study. Yet a list of bidders or registrants might be out of date or exclude some capable and interested firms. Finally, respondents to a survey of interest in government contracting might not accurately represent the entire sample because of differences between respondents and nonrespondents, or the sampling frame for the survey might have excluded some businesses. As it turned out, across all studies, the median disparity ratios for women-owned businesses differed little by whether the universe of available firms was defined broadly or more narrowly (Enchautegui et al., 1997:Table 2.5). Given that the measure of utilization was held constant, this finding suggests that, on average, women-owned firms were neither overrepresented nor underrepresented on vendor or bidder lists of state and local governments compared with their share of all firms. In contrast, disparity ratios for minority-owned firms were significantly higher when the universe of available firms was defined broadly than when it was defined more narrowly. This finding suggests that, in the states and localities studied, minority-owned firms were overrepresented on vendor or bidder lists compared with their share of all firms. As noted above, almost all 58 studies calculated disparity ratios that were internally inconsistent because the utilization measure was in terms of contract dollars awarded and the availability measure was in terms of number of firms in a specified universe. Some studies also calculated disparity ratios consistently, in that the utilization measure was in terms of numbers of contracts awarded, not dollars. No jurisdiction compared shares of contract dollars awarded with shares of gross business receipts. Comparing the consistently and inconsistently calculated ratios for women-owned small businesses for jurisdictions for which both types of ratios were reported suggests few differences between them. The exception was construction subcontracting, for which the median disparity ratio for the consistent calculations (numbers of contracts and firms) was 0.52 points

OCR for page 38
Analyzing Information on Women-Owned Small Businesses in Federal Contracting higher than that for the inconsistent calculations (contract dollars and numbers of firms). This finding suggests that women-owned businesses received a higher percentage of construction subcontracts than of the subcontract dollars awarded.7 The Urban Institute meta-analysis is not directly useful to the SBA to respond to the congressional charge to estimate disparity ratios by industry for women-owned small businesses in federal contracting. The data pertain to state and local contracting for those jurisdictions that chose to conduct disparity studies, and they are out of date. The study approach is useful, however, because it demonstrates attention to detailed documentation of data and methods and careful explication of limitations of the analysis and the sensitivity of the results to factors that could have biased them. The data for each study included in the analysis are provided in the report, so that other researchers can examine them. One useful addition to the study would have been a display of the data points for each industry using a box plot, which would graphically allow conclusions about the spread in the data. Department of Commerce “Ready, Willing, and Able” Analyses In 1996, the Department of Justice put forward a plan to revamp the federal government’s preferential contracting programs for small disadvantaged businesses (see Chapter 2). One provision of the revised acquisition regulations provided that small disadvantaged businesses could receive a price evaluation adjustment, or bid-credit, to level the playing field with larger businesses. However, the bid-credit could be used only for contracts in industries designated by the Department of Commerce as falling below an industry-specific benchmark limit with regard to utilization of small disadvantaged businesses. The Justice Department intended these benchmarks to represent the “level of minority contracting that one would reasonably expect to find in a market absent discrimination or its effects.” The Commerce Department’s Office of the Chief Economist and Office of Policy Development in the Economics and Statistics Administration released the results of its first benchmark study in 1998, which pertained to contracting in fiscal year 1996, and updated those results in 1999. No further studies have been conducted (U.S. Office of Management and Budget, 1998, 1999). The Commerce Department’s study has been harshly criticized for the lack of documentation of data sources, analytical methods, and limitations 7   Estimates were computed by the steering committee staff, averaging results for each jurisdiction reporting more than one consistent, or inconsistent, disparity ratio, and taking the median of consistent (inconsistent) ratios across studies in an industry group.

OCR for page 38
Analyzing Information on Women-Owned Small Businesses in Federal Contracting of the results; for failure to develop meaningful industry groupings for a study of federal contracting; and for the lack of a theory of discrimination underlying the study (La Noue, 2000). We agree with the criticisms about the deficiencies of documentation, which make it difficult to assess the value of the approach for possible use by the SBA, and about the superficial approach to defining industries. Articulating a theory of discrimination requires defining the universe of available firms—whether it is appropriate to use an “all firms” definition from a source such as the Survey of Business Owners, or whether a definition of “ready, willing, and able” firms is more appropriate and what should be the precise definition of “ready, willing, and able.” We consider the suitability of various definitions in light of the intended uses of the calculated disparity ratios, but do not recommend a specific definition for the universe of available firms (see Chapter 6). The only available documentation for the Commerce Department’s benchmark study is contained in a brief appendix to a Federal Register notice (U.S. Office of Management and Budget, 1998:35716-35717). It summarizes the methodology in the briefest terms. Creating a Data Set The Commerce Department first sought to assemble a data set of firms that were “ready and willing” to supply the federal government. The department rejected using the 1992 Survey of Minority-Owned Business Enterprises, presumably because it was out of date for a study of contracting experience in fiscal year 1996 and also because of uncertainty about how to screen out firms that were not interested in or capable of supplying the federal government. Instead, the department used three data sets for fiscal year 1996: Bidders from a sample of competitive procurements over $25,000. These firms were identified from a survey of federal contracting officers, who were asked to provide names of bidders for a sample of 16,616 new, competitive procurements stratified by industry and, for construction industries, by the 9 census geographic divisions. The survey had a 76 percent response rate. All firms that won sole-source or other noncompetitive procurements over $25,000. All firms certified by the SBA as active and eligible for Section 8(a) contracts, whether or not the firms won new contracts in fiscal year 1996.8 8   The addition of all firms on the 8(a) list was criticized on the grounds that most of them do not bid in open competitions (La Noue, 2000:95).

OCR for page 38
Analyzing Information on Women-Owned Small Businesses in Federal Contracting The department matched the firms in these three data sets by taxpayer identification number and federal contracting arena (that is, major industry grouping corresponding to a 2-digit SIC category or, sometimes, a 1-digit SIC category) to eliminate duplication. It then added measures of size (annual payroll), age in years of existence, and profit-nonprofit status for each firm by matching with the Census Bureau’s 1995 Standard Statistical Establishment List (SSEL).9 It also resolved cases of inconsistent reporting of small disadvantaged business status. Finally, the department added a utilization measure for each successful bidder in the integrated data set by matching with the FPDS to obtain total net prime contract obligations over $25,000 awarded to a firm in fiscal year 1996. Smaller contracts were excluded from the analysis because of the lack of detailed information about them. The department found that small disadvantaged businesses received a smaller percentage of small awards compared with awards over $25,000, so that excluding small awards would somewhat overestimate utilization. Measuring “Ready, Willing, and Able” The next step in the Commerce Department’s methodology was to measure the capacity of the ready and willing firms in the integrated data set. A dollar value for capacity was assigned to each firm equal to the geometric mean value of federal contracting work for contractors of a given size and age in a given industry group. The capacity values were estimated through regression equations for each industry group, using those firms in the integrated data set that were successful bidders in fiscal year 1996. The dependent variable for each such firm was the log of the utilization measure (amount of federal contract dollars); independent variables included the log of the number of years since the firm first appeared in the SSEL, a dummy variable if the firm first appeared before 1975, the log of 1995 payroll, interaction terms between the payroll and age variables, and a dummy variable if the firm certified that it met the SBA’s definition of a small business in the contracting arena. For firms missing one or more of the independent variables for the equations and for nonprofit and government establishments, the mean log of utilization was computed separately within industry group. 9   The SSEL has been renamed the Business Register; it is confidential and was used under an arrangement by which Department of Commerce staff were sworn as special census agents and worked on-site at Census Bureau headquarters in Suitland, Maryland.

OCR for page 38
Analyzing Information on Women-Owned Small Businesses in Federal Contracting Calculating Utilization and Availability Shares Finally, the Commerce Department used the integrated data set to measure utilization and availability shares for small disadvantaged businesses in major industry groups. Utilization shares were calculated as the total value of prime contract dollars awarded in fiscal year 1996 to small disadvantaged businesses in the data set divided by the total value of prime contract dollars awarded to all firms in the data set. Availability shares were calculated as the total capacity value assigned to all small disadvantaged businesses in the data set divided by the total capacity value assigned to all firms in the data set. Based on the results, the Department of Commerce determined that small disadvantaged firms were eligible for a 10 percent price evaluation adjustment, or bid-credit, in 71 industry groups, and for 3 construction industries, within each of 9 geographic divisions. By comparison, the SIC includes 83 2-digit industry categories in all. An updated analysis, released in 1999, limited the bid-credit to small disadvantaged businesses in 51 industry groups plus each of 9 geographic divisions for 3 construction industries. No documentation was published of the threshold level of disparity used by the department in determining eligible industry groups. Assessment On the positive side, the Department of Commerce study exhibited diligent and innovative work to attempt to develop a defensible measure by which to classify firms as ready, willing, and able to supply federal procurement needs for prime contracts over $25,000. Utilization shares and availability shares were also consistently defined in dollar terms. Greatly impairing the usefulness of the study results and methodology, however, is the lack of documentation for key components in the estimation, particularly for the regression equations that were used to predict capacity values. For example, there is no description of the values of the estimated coefficients for the independent variables, no information on standard errors or how much of the variance in the dependent variable is explained by the independent variables, no analysis of the distribution of the residuals (that is, the differences between the predicted and reported values of the dependent variable for each observation) to look for possible prediction biases, no discussion of alternative functional forms of the regressions that were tested (if any), and no reports of outliers and their effects on the estimated coefficients. Analyses of these kinds are essential to verify that a regression equation performs well for the observations for which it is estimated. Also important for evaluation is to cross-validate the analysis by find-

OCR for page 38
Analyzing Information on Women-Owned Small Businesses in Federal Contracting ing ways to compare the regression predictions with a set of target or “true” values that were not used to develop the equations (see National Research Council, 2000:Ch. 6). In this application, it would have been possible to estimate the equations on a random half-sample of the observations for successful bidders, use the equation results to predict capacity for other successful bidders in the integrated data set, and compare the predicted values with the reported values of contract dollars awarded for the other successful bidders. Whether such analysis was done for the equations used by the department or alternative forms of those equations is not known. SBA CAWBO Study The preliminary CAWBO study of women-owned businesses in federal contracting was summarized for the committee in the open session of its spring 2004 workshop. Later on in the project, the committee members were able to receive a copy and examine it. The committee does not comment on specific estimates in the document, but only on general methodological points. Methodology The basic methodology used by CAWBO followed that of many of the state and local contracting studies reviewed by the Urban Institute, and it refers to the Urban Institute analysis as the basis for a number of methodological decisions (Enchautegui et al., 1997). Like many of the state and local studies, CAWBO used inconsistent definitions for the utilization share, U, and the availability share, A, in calculating disparity ratios within industry group. This difference may have affected the estimates of disparity ratios for industries, although whether the effects were small or large is not known by the committee. For the utilization share (U), the CAWBO study defined both the numerator and the denominator in monetary terms. The numerator of U was defined as total dollars of federal prime contracts over $25,000 awarded in fiscal year 1999 to women-owned small businesses as defined by the SBA. The denominator was defined as total dollars of federal prime contracts over $25,000 awarded in fiscal year 1999 to all firms, including women-owned small businesses, nonwomen-owned small businesses, and larger businesses. The data source was the FPDS, which contains detailed information for all federal prime contracts over $25,000, accounting for over 90 percent of the $200 billion spent in fiscal year 1999. Contracting data for fiscal year 1999 were used instead of the most recent data available at the

OCR for page 38
Analyzing Information on Women-Owned Small Businesses in Federal Contracting time of the study (for fiscal year 2000), in order to be more comparable to the reference period for the SWOBE data used to measure availability. For the availability share (A), the CAWBO study defined both the numerator and the denominator as counts. The numerator of A was defined, not in dollar terms similar to the numerator of U above, but as the number of women-owned businesses (firms) with paid employees identified in the 1997 SWOBE. The denominator of A was defined as the total number of business firms with paid employees, including women-owned businesses (assumed to be primarily small businesses), other small businesses, and larger businesses, from the same 1997 survey. Defining underrepresentation as a disparity ratio of 0.80 or less, the CAWBO study estimated that women-owned small businesses were underrepresented in all but 5 of 71 industry groups (2-digit SIC categories) for which disparity ratios were calculated. Defining substantial underrepresentation to be a disparity ratio of 0.50 or less, the study estimated that women-owned small businesses were substantially underrepresented in 56 industry groups. The CAWBO study stated that it chose 0.80 as the level between representation and underrepresentation to allow for the possibility of measurement error. That level is also congruent with the Urban Institute meta-analysis and the rule of thumb used by the Equal Employment Opportunity Commission in employment discrimination cases. The CAWBO study stated that it chose 0.50 as the level between underrepresentation and substantial underrepresentation to be conservative, noting that the few studies that offered a level set it at higher than 0.50. Assessment A major limitation of the CAWBO study is the same as that of the Department of Commerce study reviewed above—namely, incomplete and unclear documentation of data sources and estimation methods and the lack of any published sensitivity analysis that would indicate the robustness of the estimated disparity ratios to alternative measures of utilization and availability. Tables and graphs are not clearly labeled, and the steps followed to evaluate and select data sources and construct estimates are not clearly described. The presentation is in the form of a series of questions and answers (for example, “Why use FY 1999 data when the FY 2000 data are available for contracting?”). Such a presentation could be useful to provide as a supplement for stakeholders, but it does not substitute for a clear, ordered description of estimation techniques and procedures. Finally, although the document mentions many studies that were consulted, it provides only two references. The CAWBO document indicates that a variety of data sources were examined for measuring availability prior to selecting the 1997 SWOBE

OCR for page 38
Analyzing Information on Women-Owned Small Businesses in Federal Contracting data on firms with paid employees to use in the analysis. Specifically, the study examined seven data sets: (1) the SBA PRO-Net database of small businesses registered to do contracting with the federal government, which carries a self-designation of women-owned status; (2) the 1997 SWOBE tabulation for all firms; (3) the 1997 SWOBE restricted to firms with paid employees; (4) a special tabulation of the 1997 SWOBE for all firms in which some firms reporting 50-50 male-female ownership were assigned to the women-owned category (as was done in the 1992 SWOBE); (5) the same special tabulation restricted to firms with paid employees; (6) the Department of Defense CCR; and (7) the Department of Commerce database of firms active in federal prime contracting in fiscal year 1999, including contract winners and bidders from a survey of contracting officers. The CAWBO document does not present disparity ratios estimated using these various sources. It presents a summary justification for selecting the 1997 SWOBE data for firms with paid employees, but it does not indicate whether that justification applied to all industries. According to the document, this choice was conservative in that the only data set indicating that women-owned small businesses were a smaller share of all businesses than the SWOBE data on firms with paid employees was the Department of Commerce file of successful and unsuccessful bidders. CAWBO declined to use the Department of Commerce data set (which may be subject to significant sampling and nonsampling errors), given its belief that more firms are ready, willing, and able to be federal contractors than those on the list of active bidders. To assess the validity of this belief, CAWBO looked at a sample of industries to determine if only the largest firms could handle the typical federal contract. The document summarizes the findings, stating that the majority of contracts in an industry are below the average size contract (a small number of large contracts pull up the average), and that women-owned small businesses have obtained contracts at the average size, indicating that firms with employees in an industry should be able to handle the typical contract. CAWBO noted that small firms can subcontract, engage in joint ventures, hire temporary staff, or take other steps to expand their capacity as needed. CAWBO rejected the idea of a regression analysis to estimate the capacity of available firms to handle federal contracting on the grounds of lack of time and resources to assemble the necessary firm-specific data. CAWBO also said that the SBA was not charged to explain observed disparities in federal contracting within industries, but simply to measure them. Finally, the CAWBO document summarizes an analysis of the success of women-owned small businesses in the private sector compared with the government sector. Marketplace disparity ratios were calculated for every industry—the method is not explained but may have involved comparing

OCR for page 38
Analyzing Information on Women-Owned Small Businesses in Federal Contracting shares of numbers of firms with shares of business receipts for women-owned businesses. The marketplace ratios were then compared with the federal contracting disparity ratios. Women-owned small businesses were underrepresented in the private market as well, but not as much as in the government market, a finding that CAWBO believes supports the conclusion that capable women-owned small businesses are not being well used in federal contracting. This and the other analyses summarized in the CAWBO document indicate the hard work that the analysts put into the study. The analyses would be more useful and convincing if the detailed results were provided, as was done in the Urban Institute meta-analysis. Our recommendations regarding the CAWBO study and the development of disparity ratios to respond to the SBA’s congressional mandate are presented in Chapter 6.