Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 79
Analyzing Information on Women-Owned Small Businesses in Federal Contracting 6 Conclusion and Recommendations From our review of data, methods, and prior assessments of the use of women-owned small businesses in government contracting, we draw one conclusion and make four major recommendations. We conclude that the disparity ratio estimates developed by the Office of Federal Contract Assistance for Women Business Owners (CAWBO) are not adequate to identify industries in which women-owned small businesses are underrepresented in federal prime contracting. Our recommendations cover: (1) data and methods for producing revised disparity ratio estimates, (2) development of more useful reports on trends in federal contracting, (3) collection of data on subcontracting, and (4) development of a research agenda for analyzing the role of women-owned and other types of small businesses in federal contracting, including studies of disparities and the possible role of discrimination in the contracting process. CONCLUSION The committee concludes that the disparity ratio estimates from the SBA Office of Federal Contract Assistance for Women Business Owners preliminary study (completed in late 2002) are not adequate to identify industries in which women-owned small businesses are underrepresented (or substantially underrepresented) in federal prime contracting. As discussed in Chapter 4, the CAWBO study has the following problems: it does not provide sufficient justification for the definition and data used to measure the availability of women-owned small businesses, in particular the decision to include in the availability measures all women-owned busi-
OCR for page 80
Analyzing Information on Women-Owned Small Businesses in Federal Contracting nesses with paid employees as “ready, willing, and able” to perform federal contracting; it uses an inconsistent definition for the disparity ratio (comparing dollars of contract awards with numbers of businesses); it uses different years for estimating utilization and availability in a period of rapid growth of women-owned small businesses; it uses Standard Industrial Classification (SIC) 2-digit industry categories instead of North American Industry Classification System (NAICS) 3-digit or 4-digit categories; and it provides inadequate documentation of the source data and estimation methods. Finally, the CAWBO estimates are now out of date. For these reasons, the CAWBO estimates should not be used to designate industries in which to permit the use of preferential contracting programs for women-owned small businesses. RECOMMENDATION 1—REVISE THE CAWBO PRELIMINARY ESTIMATES The committee recommends that, instead of using the CAWBO preliminary estimates of representation of women-owned small businesses in federal contracting by industry, the Small Business Administration should estimate disparity ratios with more recent data and revised, fully documented methods. New data have become available since the CAWBO preliminary study was completed, and revisions to the basic approach are needed. It is also critically important that the revised study clearly and comprehensively document and base its identification of target industries (those in which women-owned small businesses are underrepresented) on carefully evaluated alternative measures of utilization and availability. This recommendation addresses eight specific issues: (1) data for measuring utilization, (2) data for measuring availability, (3) types of disparity ratios, (4) industry classification detail consistent with substantive meaning and precision requirements, (5) choice of levels of the disparity ratio for distinguishing underrepresentation and substantial underrepresentation, (6) identification of industries that clearly underrepresent women-owned small businesses on the basis of multiple measures of disparity with more recent data, (7) identification of industries for further analysis, and (8) documentation and evaluation. 1-1 Data for Measuring Utilization Selecting a data source for measuring utilization shares for women-owned small businesses in federal contracting requires deciding the size of contracts to include and the reference year. The data source must also be carefully evaluated for completeness and quality and to determine outliers and their possible effects on estimates.
OCR for page 81
Analyzing Information on Women-Owned Small Businesses in Federal Contracting We agree with CAWBO’s decision to use data from the Federal Procurement Data System (FPDS) on prime contract actions over $25,000. The available information on smaller government contracts is not contract-specific and cannot be readily analyzed (see Chapter 3). Moreover, while smaller contracts are a large share of total contract actions in a fiscal year, their dollar value is a small share of the total dollars awarded. For example, in fiscal year 2003, contract actions reported on SF-281 (the government form used to aggregate data for smaller contracts—see Chapter 3) represented 91 percent of 11.5 million total contract actions, including new awards, modifications, and others; however, in the same year, the dollar value of SF-281 contract actions represented only 5 percent of $305 billion total net dollars awarded (see www.fpds.gov). CAWBO should use FPDS data for a reference period that corresponds to the reference period for the data used to measure availability shares. For example, 2002 FPDS data would be appropriate to use with data from the 2002 Survey of Business Owners (SBO).1 CAWBO should assess the accuracy and completeness of the FPDS data to the extent feasible. In addition, CAWBO should examine the distribution of contract awards by size and assess the likely effects on utilization estimates of extreme values. A useful analysis would link Central Contractor Registration (CCR) records for nonwomen-owned small businesses with FPDS records for initial competitive contract awards to determine a threshold dollar value above which small businesses never or hardly ever are successful bidders. Some or all of the contracts over that threshold value might then be excluded from the FPDS for calculating utilization shares. The reason to perform the analysis for nonwomen-owned small businesses is to avoid excluding contract size classes in which women-owned small businesses may not be successful. Whether and which contracts above the threshold value to exclude from the estimation of disparity ratios would require further analysis. Some large contracts, such as the Department of Energy procurements to operate the national laboratories or Department of Defense procurements for major weapons systems, may be appropriate to exclude because they could not realistically be structured to enable small businesses to bid successfully. Other large contracts may, however, represent bundling of work that could have been separated into several smaller contracts and, consequently, should 1 Should CAWBO wish to revise its earlier estimates to produce a time series, we recommend that it use fiscal year 1997 utilization data from the FPDS to compare with the 1997 Survey of Women-Owned Business Enterprises (SWOBE) data on availability (see Chapter 4).
OCR for page 82
Analyzing Information on Women-Owned Small Businesses in Federal Contracting be retained in the analysis. While a full evaluation of the characteristics of contracts that may affect the expected success rate of women-owned small businesses is likely to be beyond the SBA’s resources, some sensitivity analyses should be performed of the effects on disparity ratios of removing contracts above specified size limits. Other aspects of data quality in the FPDS concern the accuracy of classification of contractors by type of business (see the “Data Quality” section of Chapter 5) and the accuracy of classification of the type of work by NAICS industry code. Finally, there may be industries for which few contract awards or other contracting actions occur in a given year. In such instances, consideration should be given to pooling FPDS data for more than one year, providing such pooling does not obscure important time trends. 1-2 Data for Measuring Availability Selecting a data source and universe definition for measuring the availability of women-owned small businesses for federal contracting is a challenging task. Given limitations of existing data and resource constraints on data collection and modification, there is no single data source, or combination of data sources, that is wholly satisfactory for measuring availability with a specific universe definition. Moreover, given different views about an appropriate universe, there is no single availability measure that is likely to satisfy all stakeholders. In order to construct disparity ratio estimates for more narrowly as well as more broadly defined universes of businesses, we recommend that CAWBO make use of two sources for measuring availability: the Central Contractor Registration for 2004 and the 2002 Survey of Business Owners. CAWBO should evaluate both sources on data quality to the extent feasible, refine each source as appropriate for comparison with a utilization measure that is based on contract awards over $25,000, and examine the distribution of eligible contractors and businesses by size and the likely effects on availability estimates of extreme values. The CCR, as of October 1, 2003, is supposed to contain government-wide information about all current vendors and prospective bidders on federal prime contracts and grants, including small businesses certified by the SBA as eligible for various preferential contracting programs. Annual updating is required to maintain active status in the file; the required information includes 3-year average revenues and number of employees, as well as status as a small business and type of ownership (woman-owned, veteran-owned, etc.). At present, the CCR includes about 351,000 active vendors (see www.ccr.gov).
OCR for page 83
Analyzing Information on Women-Owned Small Businesses in Federal Contracting Use of the CCR data would provide availability measures for a limited universe of firms—those that have actually won federal prime contracts or are interested in bidding on them. Careful evaluation should be conducted of the quality of the registry data—for example, matching the CCR with the SBA PRO-Net database on small businesses to look for missing, incomplete, or incorrect CCR records. Careful evaluation should also be conducted to determine whether to include all or a subset of firms in the universe for measuring availability. Because the universe for measuring utilization is contracts over $25,000, it is possible that some very small businesses in the CCR should be excluded from the availability universe. That decision could be informed by matching the CCR for nonwomen-owned businesses with the FPDS data to determine the characteristics of successful vendors for contracts at or near the $25,000 threshold. For example, if firms with fewer than a certain number of employees or average revenues hardly ever win contracts in the range of, say, $25,000 to $50,000, then such firms should probably be excluded from the CCR data in calculating availability shares. In this and other instances of evaluation, the analysis should be conducted separately by industry group. Results from the 2002 Survey of Business Owners will be available in 2005, so CAWBO could measure availability from that source as well. Use of the 2002 SBO data would provide availability measures for a broader universe of firms than those that are registered with the CCR. However, some kinds of firms in the SBO may not be serious prospects for seeking federal work because they are very small, lack specific technical capabilities, or for some other reason. The suggested analysis of CCR and FPDS data to determine characteristics of firms that win relatively small contracts could inform the decision of whether to exclude from the SBO firms that fall below some size threshold on the basis of number of employees or size of revenues. Comparing the characteristics of firms that register with the CCR with those of firms in the 2002 SBO could possibly suggest additional screening criteria to use—for example, it may be that in certain industries firms in certain areas of the country never or hardly ever register to bid. We understand that SBA must respond in a timely fashion to the congressional mandate for estimates of disparity ratios for women-owned small businesses in federal contracting by industry. Thus, we do not anticipate that CAWBO would undertake the kinds of extensive and intensive analyses that would be appropriate for a longer term research agenda on disparities and discrimination in federal contracting (see Recommendation 4). However, we think that some analysis is warranted, such as that suggested using the CCR, to determine screening criteria in addition to size (e.g., age of business, geographic location) to use on the 2002 SBO data (and the CCR data). Doing so would permit disparity ratios to be calculated for
OCR for page 84
Analyzing Information on Women-Owned Small Businesses in Federal Contracting universes of women-owned small businesses that are closer to the judicially enunciated concept of “ready, willing, and able.” 1-3 Types of Disparity Ratios The preliminary CAWBO study calculated utilization shares in monetary terms (share of total prime contract dollars awarded), but it calculated availability shares in numeric terms (share of total firms with paid employees). This inconsistent approach has been used in many disparity studies, but it inappropriately mixes apples and oranges and should not be used. We recommend that CAWBO calculate consistent disparity ratios of two main types. First, CAWBO should calculate monetary ratios as the women-owned small business share of federal prime contract dollars awarded for contracts over $25,000 divided by their share of total business receipts. CAWBO should also calculate numeric ratios as the women-owned small business share of the number of federal prime contract awards over $25,000 divided by their share of businesses. Separate ratios should be calculated for dollars awarded for contracts classified by size (e.g., $25,000 to $100,000, over $100,000). Point 1-6 below lays out how these variously derived ratios would be used to designate industries by their representation of women-owned small businesses in federal prime contracting. Monetary ratios are critical to compute because the legislatively mandated goals for small business contracting are specified as percentages of contract dollars awarded, not percentages of contracts awarded. Moreover, dollar value is critical to business success, not awards per se. Because contracting agencies have discretion in determining how to package agency requirements (see Chapter 3), numbers of contracts may bear scant relationship to dollar value: one contract may be worth the sum of dozens or hundreds of smaller contracts. The use of monetary ratios may well result in a reduction of the number of industries in which evidence of disparities is found in comparison with numeric ratios or inconsistently calculated ratios in which utilization is computed in monetary terms and availability in numeric terms. The reason is that women-owned businesses are a relatively small share of gross business receipts even as they are a relatively large share of businesses. However, concern about such a result should not influence the choice of an appropriate disparity ratio. While not as useful as monetary ratios, numeric ratios can also add information. Because one contract may be modified many times over its course, we recommend that numeric ratios be calculated on the basis of initial contract awards. An advantage of numeric ratios is that they are
OCR for page 85
Analyzing Information on Women-Owned Small Businesses in Federal Contracting simple to understand. Also, the computation of statistical significance levels is more straightforward and requires fewer assumptions for numeric ratios than for monetary ratios. In addition, when thinking about possible discrimination in contract award decision making, numeric ratios have the advantage of comparing numbers of decisions. We think it is important to calculate both monetary and numeric disparity ratios separately by size of contract award. Size categories could be $25,000 to $100,000 (contracts in this category are reserved for small businesses whenever possible), and $100,000 and over. Examination of the distribution of contract awards by size could inform the specification of additional categories as seems appropriate. 1-4 Industry Classification The CAWBO preliminary study used 2-digit SIC categories for estimating industry-specific disparity ratios. The SIC system focused on manufacturing and had not changed in basic structure and concept for over 60 years. The NAICS represents a significant reorganization, redefinition, and differentiation of the SIC categories that provides a more coherent and detailed classification system for business activity. For example, the NAICS treats accommodation and food services as a separate sector, not included with retail trade as in the SIC. We recommend that CAWBO use NAICS codes for all of its estimates. The level of industry detail should be as disaggregated as the data will support. Thus, estimates could be developed for 3-digit NAICS subsectors and for 4-digit NAICS industry groups within subsectors to the extent that further disaggregation is substantively meaningful, statistically defensible, and feasible. Substantive meaning has to do with the heterogeneity within 3-digit subsectors (which are the rough equivalent of the 2-digit SIC categories for industry divisions used in the CAWBO study) and the consequent implications for federal contracting. For example, NAICS subsector 485 for transit and ground passenger transportation includes industry groups 4851 for urban transit systems, 4852 for interurban and rural bus transportation, 4853 for taxi and limousine service, 4854 for school and employee bus transportation, and 4855 for charter buses. The scale of operations for these groups varies widely (one person may run a taxi service, but a large enterprise is required to run a subway system), and analysis may determine that disparity ratios vary widely as well. Statistical defensibility has to do with sample size calculations—how many sample cases must be available for estimates that are reliable at a specified level of precision, such as 95 percent. Although relevant administrative record data sources, such as the CCR and the FPDS, represent censuses, not samples like the SBO, it is still important to determine mini-
OCR for page 86
Analyzing Information on Women-Owned Small Businesses in Federal Contracting mum numbers of cases for categorization by industry and whether pooling of data among industries or over a period of several years is indicated. For doing this, statisticians often rely on a superpopulation argument—in this case, while the contracting data appear to be a population, they can be assumed to represent one of many theoretical possible outcomes of a process for a given industry group in a given time period. Feasibility involves available resources for the effort. By the terms of the legislation, the SBA may call on any government agency for needed data, so, for example, the Census Bureau could perform precision calculations for the 2002 SBO and, working with CAWBO staff, identify heterogeneous 3-digit subsectors that may warrant separately calculated disparity ratios by 4-digit industry group. 1-5 Disparity Ratio Thresholds The designation of specific values of disparity ratios to serve as thresholds for underrepresentation and substantial underrepresentation is ultimately arbitrary. Science cannot establish specific threshold values, which must be a matter of reasoned judgment. We conclude that CAWBO’s decision to define the two thresholds as less than or equal to 0.80 for underrepresentation and less than or equal to 0.50 for substantial underrepresentation is a reasonable way to present its results. The threshold of 0.80 for underrepresentation (the contracting utilization share for women-owned small businesses in an industry is less than or equal to 80 percent of the corresponding availability share) follows past practice (see Chapters 2 and 4) and allows for errors in data and estimation that, with a higher threshold, might lead to an erroneous conclusion of disparity when there was in fact no disparity. The threshold of 0.50 for substantial underrepresentation appears sufficiently below 0.80 and sufficiently higher than zero to distinguish substantial from less substantial underrepresentation. Nonetheless, plausible arguments can be made for different values of these thresholds. 1-6 Clear Cases of Underrepresentation Because almost any data source and measure of disparity will be subject to errors and because stakeholder views of appropriate disparity measures may differ according to their views on the usefulness and appropriateness of preferential contracting programs, it is unlikely that a single disparity measure will go unchallenged. We recommend that CAWBO identify industry groups for which more than one disparity measure finds underrepresentation using a disparity ratio of 0.80 or less. The disparity measures should employ as recent data as possible. Four types of measures that could satisfy these criteria are (1) monetary
OCR for page 87
Analyzing Information on Women-Owned Small Businesses in Federal Contracting and (2) numeric disparity ratios calculated for categories defined by size of initial contract award, using fiscal year 2002 FPDS contracting data for utilization shares and 2002 SBO data for availability shares; and (3) monetary and (4) numeric ratios calculated for categories defined by size of initial contract award, using fiscal year 2004 FPDS contracting data for utilization and 2004 CCR data for availability. All measures should be the final measures after any adjustments of the utilization and availability data (for example, excluding firms with revenues below a threshold or very large contracts) and after any pooling of data by years or industry. The SBA could defensibly designate for a new preferential contracting program for women-owned small businesses industry groups that exhibit underrepresentation on all or most of this group of measures. In determining how to designate an industry group that exhibits underrepresentation on some but not all measures, greater weight should be given to monetary measures over numeric measures (see Point 1-3 above). Also, greater weight should be given to disparity ratios that are specific to the smaller size awards for which small firms could reasonably compete. 1-7 Cases for Further Analysis With multiple measures, there will be industries that neither clearly underrepresent nor clearly overrepresent women-owned small businesses in federal contracting. We recommend that CAWBO single out industries for which a clear determination of representation is not easily made for further analysis and possible designation at a later date. 1-8 Documentation and Evaluation Clear, complete documentation and evaluation of data sources, methods, and the strengths and weaknesses of alternative measures are essential for credible analysis and to permit assessment and replication. Adequate documentation and evaluation are particularly important for analyses, such as the CAWBO study, that are intended to inform federal policies and practices that have economic consequences. The revised SBA study of women-owned small businesses in federal contracting should conform to scientific standards of evaluation, documentation, and reproducibility. All definitions should be clearly specified, the attributes and strengths and weaknesses of alternative data sources and alternative disparity measures should be clearly described, and the results of internal and external evaluations should be presented. Appropriate editorial techniques, such as putting technical material in appendixes and including an executive summary and listing of key findings, can be used to make the documentation accessible to nontechnical as well as technical readers.
OCR for page 88
Analyzing Information on Women-Owned Small Businesses in Federal Contracting RECOMMENDATION 2—PRODUCE MORE USEFUL REPORTS ON FEDERAL CONTRACTING The General Services Administration, the Office of Management and Budget, and the Small Business Administration have worked with other federal agencies and private contractors to develop and improve several databases about contractors and contract actions on the Internet. These detailed databases, including the CCR and FPDS, not only support more efficient procurement and contract administration, but also have the potential to inform interested stakeholders and to support policy-relevant research on disparities, possible discrimination, and other facets of federal contracting. However, at present, only limited, cryptic reports are regularly produced from these databases (see Chapter 3). The SBA Office of Advocacy has a program of regular reports and analyses on small businesses and their contributions to the economy and has worked to develop data files for such analysis (see, e.g., Armington, 2004; U.S. Small Business Administration, 1998, 2003). However, the SBA does not sponsor regular reports that would inform Congress, other interested parties, and the public about trends in federal contracting disaggregated by such characteristics as type and size of business, agency, and region. It annually publishes goals for the use of various types of small businesses in federal contracting, but it does not regularly publish tabulations or analyses related to contracting even though federal contracts are an important source of business for many small firms. We recommend that the SBA work with the General Services Administration, other relevant agencies, and interested stakeholders to design and implement informative, regularly produced tables and analyses from the Federal Procurement Data System and the Central Contractor Registration on trends in federal contracting. RECOMMENDATION 3—COLLECT DATA ON SUBCONTRACTING A priority for the SBA in responding to the 2000 Small Business Reauthorization Act is to revise the CAWBO study of women-owned small businesses in federal contracting as recommended above. That study, given the limitations of available data, pertains to prime contracting. An important longer term agenda is for the SBA to work with appropriate agencies to develop data to assess the use of women-owned and other types of small businesses in subcontracting on federal prime contracts. Subcontracting is an important arena for small businesses to gain experience and a track
OCR for page 89
Analyzing Information on Women-Owned Small Businesses in Federal Contracting record that could enhance their capabilities to handle larger prime contracts, or to develop a substantial subcontracting business. One way to obtain useful data on subcontracting could be to conduct periodic surveys of prime contractors about their use of subcontractors. Surveys of small businesses about their subcontracting experience could also be helpful. Such surveys could start with small businesses that are registered in the CCR, but they should also include other small businesses as well, as there is no registration requirement for firms that are interested in subcontracting but not also prime contracting. Another way to obtain useful data on subcontracting is to explore the development of administrative records systems that could provide relevant information. Such data development could not happen quickly because of the need for extensive planning and testing prior to full implementation to minimize unnecessary record-keeping and ensure high-quality responses. Yet, in time, an investment in an appropriate administrative records system on subcontracting could have a large payoff in terms of improved understanding of the sizeable fraction of federal contracting dollars that accrue to subcontractors. RECOMMENDATION 4—DEVELOP A RESEARCH AGENDA ON WOMEN-OWNED SMALL BUSINESS CONTRACTING The steering committee found that almost all of the work to date on use of women-owned and other types of small businesses in federal contracting has been in response to court decisions or legislation about preferential contracting programs. Until recently, complete, detailed information on women-owned businesses and their contracting experience has not been available for analysis on a regular basis. Now that more useful information is available from such sources as the CCR, we recommend that the SBA proactively develop a research agenda for analyzing the role of women-owned and other types of small businesses in federal contracting. Research on subcontracting should be included as soon as feasible. The research agenda should identify issues of concern to policy makers, contracting agencies, small businesses, and other stakeholders and identify priorities for data collection and analysis. Academic researchers in the field should be involved in the design of the program. The program should include studies not only of disparities, but also of the many variables that may explain observed disparities, including the possible role of discrimination in various stages of the contracting process and, to the extent feasible, in the processes of small business formation and development. Relevant studies could include the following:
OCR for page 90
Analyzing Information on Women-Owned Small Businesses in Federal Contracting Regular updating and refinement of basic disparity ratio calculations. Analysis of the relative success of women-owned small businesses compared with other small businesses in winning federal contracts, eliminating larger and publicly held companies (none of the latter is identified by the gender of shareholder owners). Investigation of business ownership patterns and how modifications to the definition of women-owned businesses (e.g., inclusion of businesses owned equally by men and women) affect analyses of disparities and discrimination. Capability analysis for women-owned small businesses similar to, but much more extensive than, the Department of Commerce study and what is likely feasible for the recommended revision of disparity ratio estimates (see Recommendation 1 above). Such analysis should be accompanied by comprehensive evaluation and documentation, outside review, and data collection, if possible, to obtain additional relevant information. For example, in addition to measures of size (number of employees and gross receipts), it would be useful to have measures of equipment and facilities, bonding and insurance, technical capabilities, and access to credit to consider as variables in statistical analysis using regression or matching methods. Development of performance measures, when feasible, that compare women-owned small business contractors and other contractors. For example, measures of on-time, undamaged delivery could probably be developed for mailing and shipping contracts. Such measures, together with contracting officials’ perceptions about performance, could enter into analyses of the use of women-owned small businesses in federal contracting. Conduct of case studies of the contracting experience of women-owned small businesses in specific industries and contracting agencies, using field interviews of businesses, contracting agencies, and relevant trade associations augmented by survey and administrative records data. Case studies are important to further the development of theory-based statistical models for analysis of discrimination (see Chapter 5). In particular, as our limited case study of Defense Department contracting practices illustrated (see Chapter 3), the contracting process is highly dependent on the discretion of the acquisition team with regard to outreach, development of a particular procurement mechanism, and other key decisions. Case studies are an appropriate way to investigate possible discrimination by such gatekeepers. Use of case studies, surveys, and administrative records data to analyze the possible role of discrimination at various stages in the federal contracting process with state-of-the-art statistical methods.
OCR for page 91
Analyzing Information on Women-Owned Small Businesses in Federal Contracting Findings from case studies and statistical analyses could help the SBA not only refine a regular series of disparity ratios and contribute to analyses of various types of discrimination as possible explanations for observed disparities, but also refine its assistance and mentoring programs for small businesses and help contracting agencies improve their support for small business bidders and potential bidders. These types of analysis can thereby contribute to improvements in the contracting process that are valuable to taxpayers as well as to all businesses. A contracting process that discriminates against women-owned small businesses may discriminate against any business. Moreover, a contracting process that discriminates against certain bidders prevents the government from contracting at the lowest price and obtaining the best product because better qualified or lower priced firms are passed over in order to award contracts to the preferred firms. An examination of contracting with women-owned small businesses may thus improve fairness and efficiency for all.
Representative terms from entire chapter: