Cover Image

PAPERBACK
$108.00



View/Hide Left Panel
Click for next page ( 149


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 148
7 INTRODUCTION An Assessment of the Dictionary of Occupational Titles as a Source of Occupational Information In the preceding chapter, procedures used to compile the most recent edition of the Dictionary of Occupational Titles are described, and several concerns are raised about the quality and characteristics of the fourth edition in light of the way it was produced. To fulfill the committee's charge to make recommendations about whether future editions of the DOT should be produced and what kinds of occupational research should be conducted to produce them, an evaluation of the quality and characteristics of the DOT is presented in this chapter. The results of this assessment, coupled with knowledge about use, have helped to inform us as to how well the data contained in the DOT meet the purposes for which they are intended and/or used. This assessment is also a basis for the committee's recommendations about whether data collection and analysis activities used in compiling future editions of the DOT should differ substantially from what has been done in the past. Establishing the quality and characteristics of data contained in the DOT is not a straightforward task. First, as already mentioned, data collection procedures were not well documented. As a result the possibilities are Pamela S. Cain had primary responsibility for the preparation of this chapter. 148

OCR for page 148
An Assessment of DOT as a Source of Occupational Information 149 limited for systematic secondary analysis of the procedures themselves or of their implications for the resulting data. Second, most of the data contained in the DOT are unique, so no readily available bench marks exist against which to compare and assess them. In fact, a great deal of occupational research takes the DOT as the bench mark or standard of comparison, a fact that makes the assessment of DOT data even more important. In this chapter we present the results of several analyses that were designed to explore in detail and systematically the nature of the process by which the DOT was produced and the quality and characteris- tics of the resulting data. SAMPLING PROCEDURES As described in chapter 6, the industry designations developed by the occupational analysis program provide the "sampling frames" from which establishments are selected for on-site visits. The underlying assumptions of the procedure are that jobs vary by industry, by region, and by size (i.e., number of employees) and that these criteria provide the soundest basis for achieving reasonable coverage of all jobs and for discovering significant variations among jobs within occupations. Within the establishments chosen, emphasis is put on analyzing those jobs that appear to be unique to the work performed in establishments of the type that the selected one represents. No bench mark data on the "population" of jobs exist, and the procedures by which specific choices were made about which jobs to study are not well documented. Consequently, it is not possible to establish whether the DOT provides comprehensive and representative information about jobs in the U.S. economy. Nevertheless, certain aspects of the procedures and their outcomes raise serious questions about the success in attaining representative coverage. A total of 232 industry designations are used to delineate the "universes" from which sample establishments are chosen. As we have noted in chapter 2, several of these, notably the designation clerical and kindred workers, are not in fact industries, and their use carries the implicit assumption that such occupations do not vary significantly in content among establishments of different types. As a consequence of this treatment of a number of nonproduction occupations the majority of the 232 industry designations that provide the universes from which establish- ments are selected are in the manufacturing sector. In contrast, the current version of the Standard Industrial Classification denotes 1,005 industries at its most detailed level, and less than half are in manufacturing. Viewed in this context then, the DOT cannot be said to be based on job analyses

OCR for page 148
lso WORK, JOBS, AND OCCUPATIONS conducted in establishments representing the entire spectrum of U.S. industry types. Comparable establishment-level data for the DOT and the U.S. economy can be used to yield a crude indicator of the direction in which the job analysis efforts for the fourth edition DOT were channeled. In themselves these data do not constitute an evaluation of the DOT'S coverage, since the critical issue, under the assumptions of the procedure currently used, is the variety of types of establishments rather than the number of establishments (or the number of employees). Nevertheless, comparison of the two distributions reinforces the impression of a disproportionate emphasis on manufacturing. Data for DOT establishments were obtained from a set of staffing schedules that were recently computerized and made available to us by the national office of the Division of Occupational Analysis. As noted in chapter 6, in the course of fourth edition production, staffing schedules were not prepared for all establishments entered or for all jobs analyzed. Furthermore, computerization of the schedules had not yet been com- pleted at the time of the committee's study. Thus the data employed in our analysis cover only 2,063 establishments; schedules for an estimated 1,100 to 1,200 establishments are still outstanding.) The characteristics of establishments in which staffing schedules were not completed or of establishments whose schedules had not yet been computerized cannot be determined. As far as we can ascertain, there is no reason to believe that there are marked differences between the characteris- tics of establishments for which data are and are not available, especially since analysts were supposed to complete staffing schedules for every establishment in which they analyzed a significant number of jobs. Given the procedures by which staffing schedules were filled out and their purpose, however, we conjecture that analysts may have been more likely to complete the schedules in larger, more bureaucratic establishments, especially those with personnel offices. Data on the national population of establishments were obtained from tables in County Business Patterns, 1974 (U.S. Bureau of the Census, 1977~. This publication is compiled by the Census Bureau using data from the administrative records of the Internal Revenue Service and the Social Security Administration. Information is available on establishments, payroll, and employment by industrial classification, size class, and county for all types of employment covered by the Federal Insurance Contribu- tions Act. In 1974 these data covered approximately 90 percent of U.S. This information was obtained through personal communication with staff at the national office and the North Carolina field center.

OCR for page 148
An Assessment of DOT as a Source of Occupational Information 151 establishments and 75 percent of the employed population. Not covered were some government employees; self-employed persons; and certain types of farm, domestic service, and railroad workers. In order to compare the DOT data with the published data on the national population of establishments, the staffing schedules were recoded to the categories used in County Business Patterns. The DOT establishments in public administration (N = 59) were excluded from tabulations, as were establishments for which data were missing. These exclusions resulted in the loss of 113 establishments and a final total of 1,950 establishments in the DOT sample. Table 7-1 presents a comparison of the percentage distribution of DOT and U.S. establishments by sac major industry division. The two distributions exhibit marked dissimilarities. The largest discrepancy occurs in the manufacturing category: 67 percent of the DOT establishments are in manufacturing industries, although this category accounts for only 8 percent of all U.S. establishments and for 32 percent of total employment. Underrepresentation is most pronounced in the retail trade and services divisions. Retail trade accounts for a mere 4 percent of the DOT establishments, although nationally, it includes 29 percent of establish- ments and employs 20 percent of the labor force. Only 7 percent of the DOT establishments are in the services division, an industry division that accounts for 27 percent of all U.S. establishments and for 20 percent of U.S. employment. Both retail trade and services include establishments engaged in a great variety of activities. It seems highly improbable that the disparity in coverage between these major industry divisions and the manufacturing division reflects a real difference in the heterogeneity of occupations. As previously noted, the wide disparity between the two distributions cannot be interpreted as conclusive evidence; but it does suggest that the procedures used to select establishments for the fourth edition DOT resulted in an overrepresentation of establishments in manufacturing industries. This overrepresentation occurred primarily at the expense of the retail trade and service industries, which include 40 percent of all workers. Moreover, the comments and observations of field center personnel lend additional support to the general impression that job analysis activities have tended to place emphasis on manufacturing industries. Size was another important criterion of establishment selection accord- ing to the occupational analysts, one for which national data are also available from County Business Patterns, 1974 (U.S. Bureau of the Census, 1977~. In Table 7-2 the percentage distribution of establishments by size class (number of employees) is presented for the DOT and for the U.S.

OCR for page 148
152 WORK, JOBS, AND OCCUPATIONS TABLE 7-1 Percentage Distribution of Establishments by SIC Industry Division: Comparison of DOT Sample a and U.S. Labor Force b Establishments U.S. Labor DOT,U.s.,Force, c sac DivisionDOT, Npercentagepercentagepercentage Agricultural services, forestry, fisheries1618.30.90.3 Mining271.40.61.1 Contract construction522.49.16.2 Manufacturing1,30967.27.632.1 Transportation and utilities954.93.56.4 Wholesale trade402.18.77.0 Retail trade824.229.019.6 . . . . finance, insurance, real estate442.39.06.8 Services1407.226.819.6 Nonclassifiable ~00.04.80.9 TOTAL1,950100.0100.0100.0 a DOT data taken from establishment staffing schedules. For purposes of comparison with U.S. data, establishments in public administration were eliminated from tabulation. bSOURCE: Coun~BusinessPatterns,1974(U.S.BureauoftheCensus,1977:TablelB). C Workers employed in the establishments covered, not the employed civilian labor force. Included in this category are establishments that could not be classified because of insufficient information. Typically, these were new businesses. population of establishments. This comparison also reveals discrepancies between the DOT sample and the national population; the discrepancy is particularly large in the smallest size class. Establishments employing one to four workers made up 59 percent of all U.S. establishments but only 6 percent of the DOT establishments. Generally, small establishments with fewer than 20 employees were underrepresented in the DOT sample, while intermediate (20 to 249 employees) and large (250 or more employees) establishments were overrepresented in relation to the U.S. distribution of establishments. There is a rather close correspondence, however, between the DOT distribution of establishments and the distribution of U.S. employment. Once again, we point out that the implications of these results for the

OCR for page 148
An Assessment of DOT as a Source of Occupational Information 153 TABLE 7-2 Percentage Distribution of Establishments by Employment-Size Class: Comparison of DOT Sample a and U.S. Labor Force b Establishments U.S. Labor DOT,U.S., Force, c Size DOT, Npercentagepercentage percentage 1-4 1256.458.7 7.2 5-9 1497.618.0 8.2 10-19 20010.311.3 10.4 20-49 36718.87.5 15.3 50-99 27714.22.4 11.4 100-249 33817.31.4 13.6 250-499 21611.10.4 9.6 500-999 1206.20.2 8.3 1,000+ 1588.10.1 16.0 TOTAL 1,9501 ~.01 00.0 1 00.0 a DOT data taken from establishment staffing schedules. For purposes of comparison with U.S. data, establishments in public administration were eliminated from tabulation. bSOURCE: Coun~Business Patterns, 1974(U.S. Bureau ofthe Census, 1977: Table 1B). C Workers employed in the establishments covered, not the employed civilian labor force. coverage of jobs are not straightforward. If the assumption that industry type is the proper basis for sampling establishments is correct, then an important first step might be to revise the industry list so that it provides coverage of all unit items in the sac. In this frame of reference the number of establishments in each industry would not be relevant, since the objective would be to obtain adequate minimum coverage for each separate type of establishment. On the other hand, if jobs in manufacturing are more diverse than those in other sectors, then oversampling of manufac- turing enterprises is quite appropriate. The DOT analysts would be expected to devote more of their attention to establishments (and presumably jobs) in these industries. Furthermore, if jobs tend to be similar in large and small establishments, undersampling small establish- ments and oversampling large estabishments would be justified on grounds of cost effectiveness. The difficulty is that there is no evidence at all regarding the relationship between type of establishment and the variability of job content. We do not

OCR for page 148
154 WORK, JOBS, AND OCCUPATIONS know whether manufacturing jobs are more heterogeneous than other jobs or whether jobs in small establishments differ from ostensibly similar jobs in large establishments or in other small establishments. In addition to considering the types and sizes of establishments providing the base data for the DOT, it is also possible to compare the distribution of occupational units in the DOT with the distribution of workers. This approach also has very obvious limitations, since some occupational units include large numbers of workers and others include relatively few. Nevertheless, the data presented in Table 7-3, in which DOT coverage and labor force employment by major occupational category are shown, reveal very marked discrepancies. Some 60 percent of all base titles fall in the processing, machine trades, and benchwork categories, although these categories include only about 12 percent of the labor force. Taken in conjunction with the finding (documented in Table 7-5 below) that a substantial proportion of occupational titles are supported by one (or no) job analysis schedule, the skewness of the distribution in Table 7-3 raises the conjecture that the choice of jobs for analysis has a major impact on the number of occupations identified and that therefore the concentration of attention on manufacturing establishments has an important impact on the entire classification structure. To state this more explicitly, if there is a strong tendency for each job analysis to result in the identification of a separate occupation (as Table 7-5 seems to imply), the selection of job analysis sites and of the jobs to be analyzed at these sites becomes the crucial decision of the occupational analysis program. As noted above, the procedures for selecting sites for job analysis were not carefully developed. Analysts drew heavily on the third edition DOT to guide their job analysis activities. This practice might well have led them to concentrate more on jobs in established manufacturing industries (which were well represented in earlier editions) and to devote less attention to jobs in newly emerging or rapidly growing sectors of the economy, such as services or retail trade. In addition, it was clear to us in talking with the analysts that many were oriented almost exclusively toward the study of production jobs. Undoubtedly, this orientation is a historical outgrowth of the program, rooted in tradition, but other reasons may be salient, such as the ease of access to manufacturing establishments. Similarly, the emphasis on large establishments may have come about because of the relative efficiency of analyzing many jobs in a few large establishments versus a few jobs each in many small ones. For whatever reasons the concentration on manufacturing and relatively large establishments came about, and whatever its implications are for the coverage of jobs, the results of the foregoing comparisons raise questions about exactly how sampling for the DOT should proceed. Previous practices were relatively unsystematic, virtually uninfonned by empirical

OCR for page 148
An Assessment of DOT as a Source of Occupational Information 155 TABLE 7-3 Comparison of Percentage Distributions of DOT Titles and Labor Force by DOT Occupational Categories Percentage of Base Titles Percentage of DOT Occupational Category(N = 12,099) Labor Force Professional, technical, and managerial12 25 Clerical and sales8 25 Service4 16 Agriculture, fishing, and forestry2 4 Processing23 2 Machine trades18 6 Benchwork19 4 Structural work7 9 Miscellaneous7 8 TOTAL100 99 SOURCE: Labor force data derived from April 1971, Current Population Survey; sample (N = 60,441) includes currently employed workers and experienced unemployed for whom a census code could be assigned. Excluded are 12 percent of sample for whom WT codes could not be assigned. Data on distribution of DOT titles by category provided by the Department of Labor occupational analysis program. data, and resulted in relative inattention to several sectors that include large proportions of workers. The distributions of workers or of establish- ments that we have had to use as crude indicators are not the basic relevant criteria, of course; a more desirable goal would be the iden- tification of the types of organizations that have unique types of jobs, with at least minimum coverage of these unique types of jobs. A sampling strategy that would ensure adequate coverage of the job content of the American economy will not be easy to develop, but it is essential that work on this problem be initiated immediately if the DOT is to serve the many demands that are made of it. SOURCE DATA Chapter 6 observes that the amount and type of source data supporting DOT titles and definitions vary and that the quality of the data appears to be uneven. These conclusions were based on examination of the source data, on reports from analysts involved in writing definitions, and on findings of the Booz, Allen & Hamilton, Inc. (1979) management review. In this section a more systematic and detailed inquiry into the quality of

OCR for page 148
156 WORK, JOBS, AND OCCUPATIONS source data is undertaken to determine the extent to which departures from standard procedures occurred and whether such departures vary by period or across certain types of jobs. As is evident from the discussion in chapter 6, there are numerous points at which departures could have occurred. The nature of these departures is important to the extent that they have deleteriously affected the quality and comparability of the data in the DOT. To assess the quality of DOT documentation, we used a set of data collected by Booz, Allen & Hamilton as part of its management review. Because the only information available on the procedures by which the DOT was produced is anecdotal and impressionistic, Booz, Allen & Hamilton conducted a special study of DOT source data in November 1978. Analysts at the North Carolina field center were requested to record information on the documentation available for a sample of 307 DOT base titles. The sample was systematically selected by choosing every fortieth title in the DOT. However, there was an occasional departure from this procedure. If the title selected was not a base title, a substitution was made, but the procedure by which this was done is unclear. Even though the sample is slightly unsytematic, the difficulties of conducting another similar study justify the use of these data to get an idea of the quality of DOT documentation. As a check on the Booz, Allen & Hamilton sample, the percentage distribution of base titles by DOT major occupational categories for the sample was compared with that of the DOT. The comparison, in Table 7-4, reveals that the two distributions are very similar. Hence on this criterion at least, the sample appears to be quite representative of the population from which it was drawn. The distribution of DOT titles by the kind of documentation available for each is shown in Table 7-5. The summary information at the end of the table shows that 11 percent of the DOT titles had no supporting documentation other than the third edition definition, which was based on job analyses conducted prior to 1965. Seventy-one percent of titles were supported by job analysis schedules only, 8 percent by schedules and occupational code requests, and the remaining 10 percent by other combinations of data. Thus job analysis schedules constituted the bulk of the data base for the DOT, other types of information making up a relatively small percentage of the source data. The quality of the definitions for the 11 percent of titles lacking any sort of documentation other than the third edition is particularly questionable, . .. . ~ . . ~ ~ , . since there Is no way ot knowing whether and to what extent changes in the content of these jobs occurred between the third and fourth editions. The quality of definitions based solely (5 percent) or in part (14 percent) on information other than job analysis schedules may also be questionable.

OCR for page 148
An Assessment of DOT as a Source of Occupational Information 57 TABLE 7-4 Percentage Distribution of DOT Titles by Major Group: The DOT versus the Booz, Allen & Hamilton Sample Category DOT Booz, Allen & Hamilton Sample 0-11213 289 345 4 52321 61818 71919 878 976 TOTAL1~1~ N(12,099)(307) Occupational code requests, for example, are essentially employers' job orders, which are taken over the phone and may not be verified on site. As a result the job specifications contained in code requests probably reflect hiring requirements rather than the functional requirements of jobs, as would have been determined via on-site analysis. Similarly, information obtained through letters from trade associations (which are, in part, advocacy groups) is perhaps more likely to depict the ideal job than the average or typical one. For both sources of information, skill and other requirements of the job may be inflated or biased upward, in relation to what would have been determined through on-site analysis. If these data continue to be used to support DOT definitions, steps should probably be taken to determine their properties and possible biases and their comparability to data obtained via on-site observations and interviews. Table 7-5 shows the distribution of titles by the number of job analysis schedules available for each. Sixteen percent of DOT occupations are unsupported by job analysis schedules (11 percent of these are completely unsupported, and 4.5 percent are supported by other types of information). Of the total number of occupations an additional 29 percent are supported by only one schedule, 19 percent by two schedules, and the remaining 37 percent by three or more schedules. The small number of jobs analyzed per title raises additional questions about the inclusiveness and accuracy of the occupational information

OCR for page 148
158 WORK, JOBS, AND OCCUPATIONS TABLE 7-5 Percentage Distribution of DOT Titles by Number and Type of Supporting Documentation Documentation Percentage Number of job analysis schedules (]AS) o 2 4 6 8+ TOTAL Number of occupational code requests (OCR) o 3+ TOTAL Number of othera sources o 1 2 3+ TOTAL All forms of documentation None JAS only OCR only Other only WAS and OCR JAS and other JAS, OCR, and other TOTAL TOTAL N 16 29 19 4 2 13 101 90 6 2 2 100 89 8 2 1 100 1 1 4 8 101 307 a Other includes comments from trade associations, job descriptions from employees, etc. SOURCE: Tabulated using data from Booz, Allen & Hamilton study of DOT documentation.

OCR for page 148
An Assessment of DOT as a Source of Occupational Information 185 TABLE 7-13 (continued) Factor Variables 1 2 3 4 s 6 Eigenvalue 10.86 4.98 2.18 1.20 1.09 0.63 Percentage variance 49.30 22.60 9.90 5.40 4.90 2.90 Cumulative percentage 49.30 72.00 81.90 87.30 92.20 95.10 a Factor loadings greater than or equal to .4 are in boldface. b Where necessary, scores on variables were reflected so that high scores represent high levels of the trait. relationship in the DOT between the substantive complexity of occupations and their managerial responsibilities. The fifth and sixth factors account for 5 and 3 percent of the shared variance in the matrix, respectively. Factor 5, which is composed of only 4 items, might be labeled "interpersonal skills." An inspection of the items' content reveals that this dimension involves working with feelings and ideas and sensory or judgmental criteria and that it involves influencing people and dealing with their social welfare. The sixth factor, although it accounts for only 3 percent of the variance, is readily interpretable as reflecting undesirable aspects of the working conditions of occupations. By and large, the results of this factor analysis are straightforward. Several variables did load on more than one factor: as noted, there is some overlap between factors 1 and 4; factors 1 and 2 also share two items in common. Only five variables (COLORDIS, PUS, COLD, WET, and NOISE), failed to load significantly on any of the factors. Of these five variables, all but COLORDIS are dichotomous variables with limited variance. The variable COLORDIS (occupations requiring an aptitude for color discrimina- tion) appears to tap a unique occupational dimension. Presumably, many occupations require similar special aptitudes, but since each aptitude is probably required of only a few occupations, it would be preferable to include such information as part of the occupational definition. These results can be interpreted in two ways. The most straightforward interpretation is simply that there is a great deal of redundancy among DOT indicators. Alternatively, the factor patterns just presented could result from the procedures used in making DOT ratings. In rating occupations for these traits, occupational analysts might have forced consistency among them. It is true that many of the functions and traits appear to tap nearly identical phenomena (e.g., GED and INTELL). However, it is also the case that the way in which the ratings were made-

OCR for page 148
186 WORK, JOBS, AND OCCUPATIONS TABLE 7-14 Factor Analysis of Fourth Edition DOT Occupational Characteristics: Items and Loadings for Six Major Factors Variable Label description Loading Factor 1: substantive complexity, 49.3 percent a GED general educational development svP specific vocational preparation INTELL DATA REPCON NUMER VERBAL ABSTRACT MVC CLERICAL SPATIAL PEOPLE FORM TALK pop VARCH DATACOM intelligence b complexity of functioning with data b repetitive or continuous processes numerical aptitude b verbal aptitude b abstract and creative versus routine, concrete activities measurable or verifiable criteria clerical perception b spatial perception b complexity of functioning with people b form perception b talking direction, control, and planning variety and change communication of data versus activities with things Factor 2: motor skills, 22.6 percent a FINGDEX finger dexterity b MOTOR motor coordination b MANDEX manual dexterity b THINGS FORM SPATIAL SEE REACH STS MACHINE complexity of functioning with things b form perception b spatial perception b seeing reaching set limits, tolerances, or standards activities involving processes, machines versus social welfare Factor 3: physical demands, 9.9 percept a LOCATION outside working conditions STOOP stooping, kneeling, crouching, crawling EYEHAND CLIMB STRENGTH eye-hand-foot coordination b climbing, balancing lifting, carrying, pulling, pushing Factor 4: management, 5.4 percept a DEPL dealing with people pop direction, control, planning PEOPLE complexity of functioning with people b TALK talking .86 .86 .83 .81 .81 .78 .76 .68 .64 .64 .55 .47 .46 .44 .43 .42 .41 .69 .68 .67 .66 .52 .47 .43 .42 .37 .33 .67 .53 .52 .49 .48 .78 .74 .70 .64

OCR for page 148
An Assessment of DOT as a Source of Occupational Information 187 TABLE 7-14 (continued) Variable Label Description Loading SCIENCE DATA TANGIBLE activities resulting in tangible satisfaction versus prestige scientific, technical activities versus business contact DATACOM communication of data versus activities with things complexity of functioning with data b Factor 5: interpersonal skills, 4.9 percept a sac FIF IN FLU sensory or judgmental criteria feelings, ideas, facts influencing people MACHINE activities involving processes, machines versus social welfare Factor 6: undesirable working conditions, 2.9 percent a HAZARDS hazardous conditions ATMOSPHR fumes, odors, dust, poor ventilation HEAT extreme heat 63 57 .49 .44 .51 .41 .41 37 .52 .42 .37 a Percentage of common variance explained. b Sign reflected on this variable. all ratings assigned at one time by a single analyst-could have inflated the degree of consistency among the scores for each occupation and hence the degree of correlation between variables measured over occupations. This is called a "halo effect," the tendency of one judgment to be affected by another. It is well known that when several ratings are made at a single time by a single judge, they tend to be more consistent than when the ratings are made independently of one another (Selltiz et al., 1959:351- 352~. Evidence that the rating procedure itself is an important source of the high degree of interrelationship among the DOT variables is offered by the results of a similar factor analysis performed by using third edition data (Barker, 1969~. For the third edition, different analysts rated each of the traits: one analyst rated occupations for aptitudes, another for tempera- ments, etc., a procedure that would mitigate the tendency to force consistency among the ratings. In an analysis of third edition ratings, Barker found that 11 factors emerged and that the factor loadings, commonalities, and percentage of common variance explained were all much lower than the estimates presented here. Although other reasons could account for the differences between his findings and ours (e.g., differences in the underlying distribution of occupations), the suspicion is strong that the differences are attributable to the change in the rating

OCR for page 148
188 WORK, JOBS, AND OCCUPATIONS procedures from the third to fourth edition, that is, that the high covariation among the worker functions and worker traits is an artifact at least in part of the procedures used to rate DOT occupations. If this is so, these findings suggest that a modification of current rating procedures is needed along with a careful examination of the content of the items themselves. These results suggest that the more reliable indicators of the features of occupations tapped by the worker traits and worker functions variables could be created by developing factor-based multiple-item scales to represent the various dimensions revealed by the factor analysis. Such scales would have the advantage of greater internal reliability and consistency than single indicators or scales created by simple summing of items without knowledge of their factor structure. In Appendix F we present scores for scales constructed in this way for the categories of the 1970 U.S. Census detailed occupational classification. SEX BIAS IN THE RATING OF OCCUPATIONS Recently, the DOT has come under attack for alleged sex bias. It has been claimed that in the third edition DOT both the occupational descriptions and the ratings of occupational characteristics undervalued jobs held mainly by women (Wits and Naherny, 1975~. In particular, it has been asserted that third edition ratings of the complexity of work in relation to data, people, and things reflect traditional stereotypes regarding the relative complexity of the kinds of jobs typically held by women and those typically held by men (Wits and Naherny, 1975~. Consideration of a few examples is sufficient to legitimate the charge of sex bias in the third edition. In it the DATA, PEOPLE, and THINGS variables included as the lowest response level a judgment that an occupation had "no significant relationship" to data, people, or things. Typist, a job held mainly by women, was coded as having no significant relationship to things, whereas Typesetting-Machine Tender, a job held mainly by men, was coded at a higher level of complexity. Such jobs as Nursery School Teacher and Practical Nurse were coded as having minimal or no significant relation- ship to data, people, and things, while such jobs as Dog Pound Attendant were rated as functioning at a higher level of complexity. According to informants in the national office the no significant relationship category for the worker functions was dropped in the fourth edition in response to the charge of sex bias in the third edition. Occupations that had been scored at the lowest complexity levels in the third edition were assigned new worker function scores. In addition, in

OCR for page 148
An Assessment of DOT as a Source of Occupational Information 189 some instances other scores were changed, presumably to reflect changes in job content or to correct other errors in the third edition. In order to document the changes made between the third and fourth editions and to determine whether the ratings of occupations commonly pursued by women had been upgraded as claimed, we conducted an analysis of third and fourth edition worker function ratings. This was done by utilizing the April 1971 Current Population Survey (cPs) of a representative sample of the labor force. This data set contains, among other variables, both the third and fourth edition DOT codes for the job held at the time of the survey and the sex of each worker. The cPs data set includes data for 60,441 members of the labor force. Third edition DOT codes were assigned to each occupational response by trained occupational analysts in the occupational analysis field centers. Fourth edition codes were subsequently added to the data, using a map prepared by the Division of Occupational Analysis that related fourth edition DOT codes to third edition codes. By comparing third and fourth edition scores on the DATA, PEOPLE, and THINGS variables separately for men and women, we can determine the effect of scoring changes between the third and fourth editions on the relative status of male and female workers. Note that our sample for this analysis is composed of workers, not jobs. However, neither workers nor jobs changed, only the classification of jobs in the DOT scheme and hence the scoring of the worker function variables. An analysis of the nature of these changes permits an indirect inference about the extent of sex bias remaining in the fourth edition DOT. We begin by considering the labor force as a whole (see Table 7-15~. In 1971, about a third of both the male and female labor force were in occupations that were judged in the third edition to have no significant relationship to data. In contrast, a much larger proportion of men than women were in occupations having no significant relationship to people, and a much larger proportion of women than men were in occupations with no significant relationship to things. The second line of the table, which gives the mean fourth edition score for occupations with "no significant relationship" in the third edition, shows what happened to these occupations in the fourth edition. On average, the occupations held by men and those held by women were assigned similar scores on the DATA and PEOPLE variables, but on the THINGS variable the occupations held by women were judged to be more complex than the occupations held by men. In short, the major effect of the abolition of the no significant relationship category was to upgrade substantially the complexity in relation to things of occupations held by women. This conclusion is also evident in the "difference in means" row, which shows the difference in the average score between the third and fourth editions. Since a low score

OCR for page 148
l9o WORK, JOBS, AND OCCUPATIONS means greater complexity, the fact that all the numbers in the row are negative indicates an average upgrading of complexity levels between the third and fourth editions. The only change of substantive importance, however, is the upgrading of occupations held by women on the THINGS variable. The remaining point to note concerning the total labor force is that except for changes required by the abolition of the no significant relationship codes, there were few changes in ratings between the third and fourth editions. More than 90 percent of the scores remained unchanged between the two editions, as perhaps was to be expected, given the way in which DOT occupational data were generated. Inspection of the second section of Table 7-15 allows us to identify a major source of change in the THINGS ratings: the upgrading of clerical and sales jobs held by women. Most clerical and sales jobs (whether held by men or women) were identified in the third edition as having no significant relationship to things. However, the occupations held by women were coded substantially differently on the THINGS variable in the fourth edition from those held by men; on average, the clerical and sales occupations held by women were judged as having much greater complexity than those held by men. No doubt this reflects the greater propensity of female clerical and sales workers than male clerical and sales workers to operate office machines. Whereas in the third edition the task of typing was rated as not involving a significant relationship to things (level 8), in the fourth edition it was rated as involving the "operating- controlling" of things (level 2~. The same sort of coding change was made for a large number of positions involving the operation of office machines. Hence while both clerical and sales occupations held by women and those held by men tended to be upgraded in the fourth edition, the upgrading was much greater for the jobs held by women. Thus on the basis of fourth edition scores the average female clerical and sales worker is scored as doing more complex work in relation to things than the average male clerical and sales worker. In contrast to the clerical and sales sector the service and benchwork sectors included here because they are also large employers of women do not exhibit radically different patterns of upgrading for jobs held by men and those held by women, although they do show significant differences in the proportion of occupations in the third edition with no significant relationship to data, people, and things. What do these results tell us about sex bias in the fourth edition DoT? Although no definitive judgment is possible in the absence of an external criterion of job complexity against which to assess the DOT ratings, the relative similarity in the mean scores for male and female workers is

OCR for page 148
An Assessment of DOT as a Source of Occupational Information 191 certainly consistent with an inference that these variables are largely bias free. For the total labor force, the means for the DATA variable vary by only about half a point, and the means for PEOPLE and THINGS by even less. Although the means are lower for men, indicating that they work in occupations with greater complexity than those held by women, the size of the differences is within what would be expected from well-known patterns of occupational segregation by sex. Hence there is no reason to believe that the kink! of work women do is undervalued in the fourth edition DOT, at least with respect to the worker function ratings. Of course, the possibility exists that the work that women do is overvalued and that if unbiased scores were available, the mean difference between male and female workers would be even greater. However, this is unlikely, given otter evidence demonstrating that men and women are equally well educated on the average and hold jobs with similar average prestige (Treiman and Terrell, 1975a, b), that the average GED levels of the jobs held by men and by women are virtually identical (the means are 3.14 and 3.20), and that the average svP levels of the jobs held by men and by women differ by only about a half a point (the means are 4.70 and 4.14~. These results imply that the worker function ratings in the fourth edition-but not the third edition-can be used to assess sex differences in occupational attainment without undue distortion (see chapter 4 for a discussion of such analyses). CONCLUSION This chapter deals with two major issues, the adequacy of the source data used to create the DOT and the adequacy of the data on occupational characteristics created in conjunction with the DOT. These issues are, of course, not unrelated, since the adequacy of the source data determines, in part, the adequacy of the resulting occupational characteristics scales. Still, it is useful to consider them separately. The chapter documents the very uneven coverage of the labor force in the basic data collection process. First, the DOT includes many more production process occupations, relative to the number of individuals in the labor force employed in such occupations, than clerical, sales, and service occupations. While it may be that production process occupations are, in fact, more finely differentiated in the economy than are other occupations, there is no evidence that this is so. An equally plausible explanation is that DOT data collection procedures, which tend to concentrate on manufacturing plants, create a bias toward more detailed coverage of production process occupations than of other types of work. At present, there is no way of resolving this question, since there exist no principles for determining the boundaries of occupations and hence no

OCR for page 148
192 o ._ ._ LO so ._ a' a' 3 CQ By Cal LO o o of ._ so o Cal Cal Cal 1 C- O LL1 ~ a' m ~ EM o Ct Lie V] V ~ ~Ct :: ~ o :L 6 E~ ro an ~ _.. Cto a, Ct Ct Cal 0\ V) rid -, At ~ in, '.,) -~ As, ~ To ~ 1 _. en ~ ~ ~ rid ~ ~ rat oo so ON ~ - , ~ ~ ~ ~ 1- ~ rid ~ ~OF Go No ~ ~No r ~ oo ~ ~ ~ ~ O ~O oo - ~ ~ ~ 1 oo - . . . . . . . _ r~ ~ ~ \0 ~ I ~I ~ _ ~O ~ 0 r~ oo ~ ~ ~, _ - - O r~ oo oo c~ ~oo O - oo _ _ ~ oo ~ O ~r ~_ O _ _ v, ~ 00 ~O r ~v) oo ~ r~ _ ~ ~ ~_ l- _ oo ~ _ ~ (~, _ 00 ~O O t - ~ ~ 1 ~ oo oo =: =: 3 ._ .o ~o o ~o :: v, .o: - ~ o ~ ~ =^ o - s ~ - c o .o s ~ - s ~ ~ ~ s s ~ ~ ~ ~ ~ s ~ D - 9', ~ D - ~y D~ 9'. e - s

OCR for page 148
193 OO ~ ~ rot red cr 1 Can ... 1 ~ ... , Cry 1 ~ red ... , ~ 1 red~ oo .. 1~ 11- o ~ - C33 .- (, ._ ~ Cal _`~ To .~ s V' .= ._ ~u, 3 .= O ~ ~0 0 It,) ~ _ ~ C_, _ ~ O ~ a=^= Q) .= . = ~ .P ~ S ~ O ~ ~ .0 .O O O O O O ~ ~ Hi 0 0 0 ~ Is) 1 O Cal i: . U. ~0 . U. O 0^ at;} 11 ~ X a, o D ~O Pi ~O U. - ~ P. - , - ~ . - o s ~ == a' ~ l_ O sp o-s ~ .- ~ ~ cd .= Op-= o o ~ ~L, ~ ~ . - ~S O '~ ~ 3 3 ^- 0' Ce . =. ~ . ~ 8 C , , , ~ o oo o o 1 1 ~ o ~ 1 o. ~ ~ ~ ~ oo ~, o oo 0- 1 - 3 o ~: - O ~ =^ O O ._ ~ ~ - a' O . . ~ ._ S .= ,= p. ~l, ~ ~ O O ~ .0 .~, _ ~ ~ 1 3 5 t; _ ~ r c-; ~ ~ _ 4_ u, a o _ _ '_ O e_ 1. 1 o o c~ ~ .~= oo ~ ~ ~ =3 1 1 .= O3 o o ~ S m V) v z :r o 00 1 o o a o 00 o c ~00 00 0 00 ~ _ ~ ~ ~ ~ ~ 00 ~ ~ ~ ~ _ oo 1 00 ~ ~ C~ ~ ~ ~ ~ ~ _ 00 ~ _ ~ ~ ~ _ oo ~1

OCR for page 148
194 WORK, JOBS, AND OCCUPATIONS unambiguous procedures for aggregating jobs into occupations. The development of such principles and of procedures for using them in the data collection process should be given high priority in preparation for future editions of the DOT. Second, some occupations in the fourth edition DOT were analyzed many times, while others were not analyzed at all. Given the heterogeneity of jobs included within a single occupational category (which is confirmed by the substantial "job description" effect on the reliability of worker trait and worker function ratings), procedures need to be developed to ensure a more even sampling of jobs within occupations in order to be certain that each occupational description is based on data froin a sufficient number of job analyses to produce representative data. What constitutes an "occupation" and how much heterogeneity in the content of a set of jobs justifies a single occupational title is a difficult question. Historically, the DOT has tended to define occupations by their titles rather than by their content. Jobs with similar titles have been grouped unless the evidence strongly indicated that they differed in content, and occupations with different titles have been defined as being different, regardless of similarity in content. At the same time, each job analysis tends to produce a new DOT occupation, while jobs with titles similar to titles already existing in the DOT tend not to be analyzed at all, making it impossible to determine their degree of similarity. Occupational titles are also used inconsistently in the DOT to define very specific or very heterogeneous groups of jobs. Branch manager, for example, describes a wide variety of jobs, all of which involve coordination and control functions but vary enormously in terms of the specific tasks performed. Tool and Die Maker, by contrast, describes basically the same job regardless of where tool and die makers are employed. Consideration should be given to developing a clear and unambiguous way of defining occupations. The analysis in this chapter also raises serious questions regarding the adequacy of the worker trait and worker function variables. First, it is unclear whether the 46 variables on which data are collected adequately represent the kind of information needed by users within and outside the Employment Service. Our conjecture is that they do not. Many of the DOT variables, especially the aptitudes, interests, and temperaments, are not heavily used, as we have seen in chapters 3 and 4. Oddly, other information collected on job analysis schedules but never subsequently recorded, i.e., information on promotion ladders and lateral transfer routes, is often mentioned by users outside the Employment Service as a major lack in the DOT. Obviously, consideration should be given to the inclusion of such information in the DOT occupational descriptions. More

OCR for page 148
An Assessment of DOT as a Source of Occupational Information 95 generally, a careful conceptual review should be undertaken of the sort of information needed for matching workers with jobs (e.g., data on the transferability of skills), for counseling job applicants about occupational requirements, for assessing the comparability of occupations for the resolution of equal employment opportunity disputes (better data on the responsibilities entailed in occupational performance, for example), and for occupational research of various kinds. Once the major dimensions of occupations on which data are needed are identified, scales measuring these dimensions should be developed following standard psychometric practices. In particular, consideration should be given to the development of factor-based multiple-item scales, the use of which would go a long way toward overcoming the reliability problems identified in Appendix E and summarized in this chapter. Despite the deficiencies in the fourth edition worker trait and worker function variables identified here, they remain the most comprehensive set of occupational characteristics currently available. As such, their use should be encouraged. To facilitate this use, Appendix F provides data on eight DOT variables aggregated to match the categories of the 1970 U.S. Census detailed occupational classification and four factor-based scales derived from the DOT variables. Researchers should find these data a useful supplement to data on the average characteristics of workers that can be derived from census occupational statistics. Moreover, one potential major threat to the usefulness of these data can be discounted on the basis of our analysis: so far as we can tell, the fourth edition worker function variables do not undervalue occupations held mainly by women as the third edition worker function variables apparently did.