Analyzing Information on Women-Owned Small Businesses in Federal Contracting (2005)

Fostering the development of small businesses has been a concern of the federal government since World War II. The charter of the U.S. Small Business Administration (SBA), established in 1953, provides that it will ensure small businesses a “fair proportion” of federal government contracts and sales. Repeatedly, legislation has charged the SBA to oversee efforts by federal contracting agencies to award specified percentages of federal contracting dollars to small businesses, including those owned by women. The Small Business Reauthorization Act of 2000 provided for set-aside contracting programs for eligible women-owned small businesses in industries in which they were underrepresented or substantially underrepresented as determined by the SBA.

In 2002 the SBA Office of Federal Contract Assistance for Women Business Owners (CAWBO) prepared a draft study containing a preliminary set of estimates of representation of women-owned small businesses in federal prime contracts over $25,000 by industry. Because of the history of legal challenges to race- and gender-conscious contracting programs at the federal and local levels, the SBA asked the Committee on National Statistics of the National Academies to conduct an independent review of relevant data and estimation methods prior to finalizing the CAWBO study.

The charge to the Steering Committee on Women-Owned Small Businesses in Federal Contracting was to hold a workshop to discuss such topics as the accuracy of data and methods to estimate the use of women-owned small businesses in federal contracting; the definition of “underrepresentation” and “substantial underrepresentation” for purposes of designating industries for which preferential contracting programs might be warranted;

and appropriate statistical methods and data needed to help understand gender disparities in federal contracting. The workshop was held April 30-May 1, 2004; this report is based primarily on the workshop materials and discussion.

From our review of the data and methods used, we conclude that the disparity ratio estimates from the CAWBO preliminary study are not adequate to identify industries in which women-owned small businesses are underrepresented (or substantially underrepresented) in federal prime contracting. For that reason, the estimates should not be used to designate industries in which to permit the use of preferential contracting programs.

The CAWBO preliminary disparity ratio estimates were developed for industry categories (defined by 2-digit Standard Industrial Classification or SIC codes) by dividing the utilization share for each industry by the availability share. Utilization was defined as the share accruing to women-owned small businesses of the total dollar amount of contract actions for federal prime contracts over $25,000 in fiscal year 1999 for the particular industry. Availability was defined as the share of women-owned businesses with paid employees among all businesses with paid employees in the particular industry from the 1997 Survey of Women-Owned Business Enterprises.

We find the CAWBO study to be problematic in several respects, including that the documentation of data sources and estimation methods is inadequate for evaluation purposes. The CAWBO study does not provide sufficient justification for the definition and data used to measure the availability of women-owned small businesses. In particular, it does not adequately justify the decision to include in the availability measures all women-owned businesses with paid employees as “ready, willing, and able” to perform federal contracting, even though “ready, willing, and able” is the judicial standard that has been invoked in lawsuits against preferential contracting programs. In addition, the CAWBO study uses an inconsistent definition for the disparity ratio (comparing dollars of contract awards with numbers of businesses), different years for estimating utilization and availability in a period of rapid growth of women-owned small businesses, and 2-digit industry SIC categories instead of the more realistically delineated 3-digit or 4-digit categories in the newly devised North American Industry Classification System (NAICS). Finally, the CAWBO estimates, which pertain to the period 1997-1999, are now out of date.

The committee recommends that, instead of using the CAWBO preliminary estimates of representation of women-owned small businesses in federal contracting by industry, the Small Business Administration should estimate disparity ratios with more recent data and revised, fully documented methods.

This recommendation addresses eight specific issues, as follows.

We agree with CAWBO’s decision to use data from the Federal Procurement Data System (FPDS) on prime contract actions over $25,000 to measure utilization shares for women-owned small businesses in federal contracting. The available information on smaller government contracts is not contract-specific and cannot be readily analyzed. Moreover, while smaller contracts are a large share of total contract actions in a fiscal year, their dollar value is a small share of the total dollars awarded.

CAWBO should use FPDS data for a reference period that corresponds to the reference period for the data used to measure availability shares. For example, 2002 FPDS data would be appropriate to use with data from the 2002 Survey of Business Owners (SBO).

CAWBO should assess the accuracy and completeness of the FPDS data to the extent feasible. In addition, CAWBO should examine the distribution of contract awards by size and assess the likely effects on utilization estimates of extreme values.

Selecting a data source and universe definition for measuring the availability of women-owned small businesses for federal contracting is challenging. Given limitations of existing data and resource constraints on data collection and modification, there is no single data source, or combination of data sources, that is wholly satisfactory for measuring availability with a specific universe definition. Moreover, given different views about an appropriate universe, there is no single availability measure that is likely to satisfy all stakeholders.

In order to construct disparity ratio estimates for more narrowly as well as more broadly defined universes of businesses, we recommend that CAWBO make use of two sources for measuring availability: the Central Contractor Registration (CCR) for 2004 and the 2002 Survey of Business

Owners. CAWBO should evaluate both sources on data quality to the extent feasible, refine each source as appropriate for comparison with a utilization measure that is based on contract awards over $25,000, and examine the distribution of eligible contractors and businesses by size and the likely effects on availability estimates of extreme values.

The CCR, as of October 1, 2003, is supposed to contain government-wide information about all current vendors and prospective bidders on federal prime contracts and grants. Use of the CCR data would provide availability measures for a limited universe of firms—those that have actually won federal prime contracts or are interested in bidding on them. Results from the 2002 Survey of Business Owners will be available in 2005. Use of the 2002 SBO data would provide availability measures for a broader universe of firms than those that are registered with the CCR. For both the CCR and the SBO, careful evaluation should be conducted of the quality of the data and to determine whether to include all or a subset of firms in the universe for measuring availability. Because the universe for measuring utilization is prime contracts over $25,000, it is possible that the universe for measuring availability should exclude some very small firms or those with other characteristics that limit their ability to compete for contracts of this size.

The preliminary CAWBO study calculated utilization shares in monetary terms (share of total prime contract dollars awarded), but it calculated availability shares in numeric terms (share of total firms with paid employees). This inconsistent approach has been used in many disparity studies, but it inappropriately mixes apples and oranges and should not be used.

We recommend that CAWBO calculate consistent disparity ratios of two main types. First, CAWBO should calculate monetary ratios as the women-owned small business share of federal prime contract dollars for contracts over $25,000, divided by their share of total business receipts. CAWBO should also calculate numeric ratios as the women-owned small business share of the number of federal prime contract awards over $25,000, divided by their share of businesses. Separate ratios should be calculated for contract awards classified by size (e.g., $25,000 to $100,000, over $100,000).

Monetary ratios are critical to compute because the legislatively mandated goals for small business contracting are specified as percentages of contract dollars awarded, not percentages of contracts awarded. Moreover, dollar value is critical to business success, not awards per se. While not as useful as monetary ratios, numeric ratios are simple to understand and may

be helpful when thinking about possible discrimination in the process for awarding contracts because they compare numbers of decisions. We think it is important to calculate both monetary and numeric disparity ratios separately by size of contract award.

The CAWBO preliminary study used 2-digit SIC categories for estimating industry-specific disparity ratios. The SIC system focused on manufacturing and had not changed in basic structure and concept for over 60 years. The NAICS represents a significant reorganization, redefinition, and differentiation of the SIC categories that provides a more coherent and detailed classification system for business activity. For example, the NAICS treats accommodation and food services as a separate sector, not included with retail trade as in the SIC. We recommend that CAWBO use NAICS codes for all of its estimates. The level of industry detail should be as disaggregated as the data will support. Thus, estimates could be developed for 3-digit NAICS subsectors and for 4-digit NAICS industry groups within subsectors to the extent that further disaggregation is substantively meaningful, statistically reliable, and feasible.

The designation of specific values of disparity ratios to serve as thresholds for underrepresentation and substantial underrepresentation is ultimately arbitrary. Science cannot establish specific threshold values, which must be a matter of reasoned judgment. We conclude that CAWBO’s decision to define the two thresholds as less than or equal to 0.80 for underrepresentation and less than or equal to 0.50 for substantial underrepresentation is a reasonable way to present its results. The threshold of 0.80 for underrepresentation, whereby the utilization share for women-owned small businesses in an industry is less than or equal to 80 percent of the corresponding availability share, follows past practice in the field. It also allows for errors in data and estimation that, with a higher threshold, might lead to an erroneous conclusion of disparity when there was in fact no disparity. The threshold of 0.50 for substantial underrepresentation appears sufficiently below 0.80 and sufficiently higher than zero to distinguish substantial from less substantial underrepresentation.

Because almost any data source and measure of disparity will be subject to errors and because stakeholder views of appropriate disparity measures

may differ according to their views on the usefulness and appropriateness of preferential contracting programs, it is unlikely that a single disparity measure will go unchallenged. We recommend that CAWBO identify industry groups for which more than one disparity measure finds underrepresentation using a disparity ratio of 0.80 or less. The disparity measures should employ as recent data as possible.

Four types of measures that could satisfy these criteria are (1) monetary and (2) numeric disparity ratios calculated using fiscal year 2002 FPDS contracting data for utilization shares and 2002 SBO data for availability shares; and (3) monetary and (4) numeric ratios calculated using fiscal year 2004 FPDS contracting data for utilization and 2004 CCR data for availability.

The SBA could defensibly designate for a new preferential contracting program for women-owned small businesses those industry groups that exhibit underrepresentation on all or most of this group of measures. In determining how to designate an industry group that exhibits underrepresentation on some but not all measures, greater weight should be given to monetary measures over numeric measures. Also, greater weight should be given to disparity ratios that are specific to the smaller size awards for which small firms could reasonably compete.

With multiple measures, there will be industries that neither clearly underrepresent nor clearly overrepresent women-owned small businesses in federal contracting. We recommend that CAWBO single out industries for which a clear determination of representation is not easily made for further analysis and possible designation at a later date.

Clear, complete documentation and evaluation of data sources, methods, and the strengths and weaknesses of alternative measures are essential for credible analysis and to permit assessment and replication. Adequate documentation and evaluation are particularly important for analyses, such as the CAWBO study, that are intended to inform federal policies and practices that have economic consequences. The revised SBA study of women-owned small businesses in federal contracting should conform to scientific standards of evaluation, documentation, and reproducibility. All definitions should be clearly specified, the attributes and strengths and weaknesses of alternative data sources and alternative disparity measures should be clearly described, and the results of internal and external evaluations should be presented.

We recommend that the SBA work with the General Services Administration, other relevant agencies, and interested stakeholders to design and implement informative, regularly produced tables and analyses from the Federal Procurement Data System and the Central Contractor Registration on trends in federal contracting. Tables should be designed to provide readily interpretable information for use by policy makers, researchers, and stakeholders.

Although a priority for the SBA is to revise the CAWBO study of women-owned small businesses in federal prime contracting as recommended above, an important longer term agenda is for the SBA to work with appropriate agencies to develop data to assess the use of women-owned and other types of small businesses in subcontracting on federal prime contracts. Subcontracting is an important arena for small businesses to gain experience and a track record that could enhance their capabilities to handle larger prime contracts, or to develop a substantial subcontracting business. Both surveys and administrative records systems could be useful vehicles for providing data on subcontracting.

The steering committee found that almost all of the work to date on use of women-owned and other types of small businesses in federal contracting has been in response to court decisions or legislation about preferential contracting programs. Until recently, complete, detailed information on women-owned businesses and their contracting experience has not been available for analysis on a regular basis. Now that more useful information is available from such sources as the CCR, we recommend that the SBA proactively develop a research agenda for analyzing the role of women-owned and other types of small businesses in federal contracting. Research on subcontracting should be included as soon as feasible.

The research agenda should identify issues of concern to policy makers, contracting agencies, small businesses, and other stakeholders and identify priorities for data collection and analysis. Academic researchers in the field

should be involved in the design of the program. The program should include studies not only of disparities, but also of the many variables that may explain observed disparities, including the possible role of discrimination in various stages of the contracting process and, to the extent feasible, in the processes of small business formation and development.

Findings from carefully specified case studies and statistical analyses could help refine a regular series of disparity ratio estimates and contribute to analyses of discrimination and other factors as possible explanations for observed disparities. Such findings could also help the SBA refine its assistance and mentoring programs for small businesses and help contracting agencies improve their support for small businesses that seek to enter and be successful in the federal contracting market.