Fairness in Employment Testing Validity Generalization, Minority Issues, and the General Aptitude Test Battery (1989) / Chapter Skim
Currently Skimming:
6 The Theory of Validity Generalization
6 The Theory of Validity Generalization
Pages 119-133
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 119... ...
In medicine, meta-analyses are becoming increasingly important as a technique to systematize the results of clinical trials (Proceedings of the Workshop on Methodological Issues in Overviews of Randomized Clinical Trials, 1987) , to collect research results in particular areas (the Oxford Database of Perinatal Medicine)
|
From page 120... ...
Meta-analysis attempts to provide a formal mechanism for doing so. By combining information from different studies, meta-analysis increases the precision with which effects can be estimated (or increases the power of statistical tests of hypotheses)
|
From page 121... ...
The questions remain: how are the observed validities to be used to estimate the distribution of true validities across a population of jobs, and thus what is the probable range of values that can be generalized to a new job? There are a number of ways in which the correlation coefficient obtained in any given study of the relation of test scores to job performance is affected by situational factors, so that the validity estimate differs from the true validity of the test for a new job: 'The criterion-related validity of a test is a measure of the relationship between the test score and a criterion of job performance (e.g., supervisor ratings)
|
From page 122... ...
For example, in a highly selective job, range restriction occurs so that nearly all workers will have a narrow high range of test scores, and the true validity will be lower than that for an unselected applicant group. If the applicant and worker distributions can be estimated, it is possible to correct for range restriction.
|
From page 123... ...
Thus for sample sizes of 100, the variance is about .01 and the standard deviation is .1; we expect the observed validity to differ by .1 from the true validity. Corrections for Sampling Error To illustrate how corrections for sampling error fit into the estimation of the distribution of true validities in a population of jobs, we offer a hypothetical example.
|
From page 124... ...
We use these observed validities to estimate properties of the original distribution of true validities. The mean true validity is estimated by the mean of the sample validities, 29.
|
From page 125... ...
Thus if a worker group is thought to have a standard deviation only half that of the applicant group, then the restriction ratio is one-half, and the validity of the test for the applicant group is close to twice that of the worker group. The main problem in determining the correction for restriction of range is identifying the appropriate population of applicants for a particular job and estimating the variance of test scores for those applicants.
|
From page 126... ...
How can the observed validities be corrected for restriction of range? The standard procedure is as follows: for each job studied, the restriction ratio-the ratio of standard deviations of test scores for applicants and workers is estimated.
|
From page 127... ...
Similarly, if the restriction ratios have a large variance, a reduction will occur in estimating the variance of true validities compared with the observed variance of sample validities. The model and calculations are as follows: Sample validity = restriction ratio x true validity + error
|
From page 128... ...
Lack of adequate reliable data about the variance of test scores in realistically defined applicant populations is a major problem in validity generalization from the GATB validity studies. The absence of direct data is so pronounced that the committee has chosen the conservative response of making no corrections for range restrictions in its analysis of GATB validities.
|
From page 129... ...
Just as with the restriction ratio, the validity of test score with observed rating is divided by the reliability correction to become a validity of test score with true rating. The main effect of the reliability correction is to increase the estimate of average true validity.
|
From page 130... ...
The first is that, if most of the variability in the observed validities can be accounted for by the artifacts of sampling error, unreliability of test and criterion, and restriction of range, then it is
|
From page 131... ...
The reason is that because such omnibus procedures are sensitive to many kinds of departures from absolute consistency among studies, they are not optimal for detecting a specific pattern. To put this argument more precisely, the omnibus statistical test that tests for any difference among validities does not have as much power to detect a particular difference between groups of studies as does a test designed to detect that specific, betweengroup contrast.
|
From page 132... ...
3. Lack of adequate, reliable data about the variance of test scores in realistically defined applicant populations appears to be a major problem in validity generalization from the GATB validity studies.
|
From page 133... ...
Connecting the Sample to the Population The generalization of validities computed for 500 jobs in some 750 USES studies to the population of 12,000 jobs in the Dictionary of Occupational Titles is justified only to the degree that these jobs are similar to the other jobs not studied. Thus a necessary component of validity generalization for the GATB is to establish links between the jobs studied and the remainder.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.