Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Appendix C Proposed Pilot Tests of Compensation Data Collection In Chapter 6, the panel recommends that the Equal Employment Op- portunity Commission (EEOC) sponsor a pilot study to be conducted by an independent contractor that would provide much more reliable information about the costs and benefits of the proposed collection than the panel can provide based on existing evidence. In this Appendix, we outline two pos- sible approaches to conducting an independent pilot study. MICRODATA PILOT This approach would seek to gain an understanding of the availability, sensitivity, and reliability of the employer earnings data for their employees. The pilot would be conducted by an independent contractor. This recom- mendation that EEOC should use an independent contractor to conduct the pilot is based on the belief that an independent contractor could gain access to information that might not be possible to collect if the data were to be used for enforcement purposes. The process of contracting with an independent contractor would also give EEOC a strong incentive to make the plan for data use sufficiently comprehensive so that the potential con- tractors for the pilot would develop competitive study designs and plans for analysis of the results. The first priority of the microdata pilot test would be to assess the availability and retrievability of data items of interest for individual em- ployees. This would likely require conducting an employer records check of emÂ loyers in different industries and size classes. The records check would p focus on questions of interest and would parallel the questions that have 131
132 COLLECTING COMPENSATION DATA FROM EMPLOYERS been posed in the Office of Federal Contract Compliance Programs (Â FCCP) Advanced Notice of Proposed Rulemaking (ANPRM) (OFCCP, O 2011). In particular, at the establishment level, how do employers record and maintain compensation data? What internal actions would employers need to take to assemble earnings and earnings-related data? What catego- ries of compensation-related data are maintained in computer-accessible personnel and payroll systems? How easy is it for employers to cross-haul data between the payroll record systems (earnings and hours measures) and their human resource management systems (employee characteris- tics and work histories)? How do these systems vary by type and size of company? How costly is it on the margin to retrieve and report these data? How much time is spent in retrieving and reporting these data (for purposes of quantifying the response burden to be reported to the Office of Management and Budget under the Paperwork Reduction Act)? Second, it would be important to validate the availability and reliability of variables that would help to gain an understanding of the role of earn- ings in affirmative action and antidiscrimination enforcement. In addition to annual and hourly wages, the pilot would collect a number of core demographic variables by the present EEO-1 categories using an annual wage measure in order to test targeting firms for enforcement purposes. To the extent it is possible, the pilot should also collect additional variables that could help to explain the equal opportunity environment in the estab- lishment and the possible influence of these variables on wage differences that may be observed. These variables might include birth date, entry level wage, and hire date. Such variables could assist in refining indicators that could help identify the possibility of discrimination based on age or senior- ity. These new earnings-related variables should be audited, on a random basis, to ensure that they appropriately reflect the data that reside in the employerâs data systems. The pilot could also test various earnings definitions, such as that used in the Bureau of Labor Statistics Occupational Employment Survey. In de- veloping the test, the public responses to the OFCCP ANPRM could well be instructive. With the results of the pilot data collection in hand, the contractor could turn to assessing the quality of the data. The contractor should com- pute earnings means with measures of dispersion for the estimates. With these measures in hand, the contractor should analyze the data as if the data were used by EEOC to select cases for further investigation (e.g., does earnings data assist in identifying potential discrimination cases and does it support a reasonable âthreshold analysisâ to determine whether or not an employer should be subject to further review and evaluation), and also as if the data were the subject of litigation (e.g., how well do the earnings data meet tests of reliability and appropriateness). The power of tests (as
APPENDIX C 133 illustrated in Chapter 4 of this report) to detect discrimination can be calcu- lated to shed light on the size of the groups required to have a good chance of detecting discrimination. Another useful test would be to use data from the Social Security Administration, the Internal Revenue Service, the states, and the Census Bureau (as discussed in Chapter 2)âor even from other data systems such as state driverâs license files, passports, and visasâto explore the develop- ment of an ongoing data quality assessment tool through which the EEOC earnings data would be benchmarked against other sources, at fine levels of granularity, to assess the closeness of their match to the EEOC earnings data. There is precedent for gaining such access to otherwise confidential tax and administrative data sources. For example, the Internal Revenue Service (IRS) Statistics of Income statistical sample files have been provided to compliance offices in the IRS for statistical analysis related to their com- pliance (audit) needs in a manner that does not adversely compromise the statistical integrity of those sample files (see discussion in Appendix B). The results of the pilot test could be anonymized to permit a peer re- view of the independent findings. SIMPLIFIED AGGREGATED-DATA PILOT An alternative, simplified pilot could be performed by an independent contractor using a revised EEO-1 report format. The design would be a simplified version of the pilot described below. The contractor would pre- pare prototype EEO-1 reports that contained sufficient wage information to permit the EEOC to calculate grouped-data test statistics for differences in the mean standardized earnings across race/ethnicity and gender groups. The standardized wage rates (full-year earnings, not actual earnings) would have to be integrated with the other data used to produce the EEO-1 report. Audited formulas for computing the average and the relevant vari- ances would be developed for the data within EEO-1 occupation and race/ ethnicity or gender group. Audited formulas for computing all relevant test statistics would also be developed. These could be based on existing statisti- cal analysis software, or simply vetted using such software. In prototyping a report that permitted statistical screening using grouped data techniques, the contractor would also be directed to experi- ment with tabulations that controlled for birth and hire date. Once again, the goal would be to produce standardized enhancements to the EEO-1 re- port that properly integrated the relevant data on standardized wage rates, birth dates, and hire dates with the other data used to compute the EEO-1 report. Once the data have been integrated, the report would generate vali- dated sufficient statistics for the grouped data comparison of conditional means, given birth date and hire date. Audited formulas for computing
134 COLLECTING COMPENSATION DATA FROM EMPLOYERS all relevant test statistics based on the conditional means would also be developed. Once again, these could be computed using existing statistical analysis software or simply vetted with such software. Because of the complexity of these calculations, and the difficulty of in- terpreting the raw report data, the contractor would be used to develop an electronic reporting format that the agency could then use for preliminary screening of the EEO-1 reports. The electronic reporting format, encom- passing the audited formulas, could then be implemented by integration into payroll and human resource management software reporting systems, just as the option to produce the current EEO-1 report has been incorpo- rated into such products.