This chapter discusses the use of occupational information for determination of Social Security disability benefits. Although O*NET was envisioned as a replacement for the Dictionary of Occupational Titles (DOT), the Social Security Administration (SSA) continues to rely on the DOT when making disability determinations. The chapter first reviews the history of the use of occupational information in the process of disability determination. It then discusses prior interagency efforts between the Department of Labor (DOL) and SSA to develop an occupational information database suited to the process of disability determination and prior research on the use of O*NET for disability determination. The third section evaluates the potential use of O*NET data vis à vis the specific types of occupational information required in the disability determination process, and the final section presents the panel’s conclusions and recommendations in this area.
OCCUPATIONAL INFORMATION NEEDS OF THE SOCIAL SECURITY ADMINISTRATION
The Social Security Act (Section 223(d)(2)) establishes that disability determination requires that an individual’s physical or mental impairment is of such severity that she or he is not only unable to do his or her previous work but cannot, considering his or her age, education, and work experience, engage in any other kind of substantial gainful work that exists in the national economy. “Work that exists in the national economy” is defined as work that exists in significant numbers in either the region where the
individual lives or in several regions in the country. To answer the question of whether or not “work exists in significant numbers” in the national economy, the SSA took administrative notice of the DOT. That is, under the assumption that only occupations that existed in significant numbers were reflected in the DOT, this O*NET predecessor served as a primary tool for determining whether a Social Security claimant had the capacity to work.
In 1996, SSA requested that the Institute of Medicine, in collaboration with the National Research Council’s Committee on National Statistics, conduct an independent review of the statistical design and content of its research plan for redesigning the disability decision process. The study committee concluded that the DOT replacement (i.e., O*NET), “will not meet SSA’s needs to define the functional capacity to work without major reconstruction” (Institute of Medicine, 2002, p. 9). The report continues:
Barring some resolution, SSA will be left with no objective basis upon which to justify decisions concerning an individual’s capacity to do jobs in the national economy. SSA might be cast back into the era when it relied extensively on the testimony of “vocational experts” or their written evaluations. [emphasis in the original]
Given that occupational information is critical for use in disability determination, our panel invited Sylvia E. Karman, a representative of SSA, to make a presentation on this issue. SSA appears to think that O*NET is not able to fulfill the needs of vocational experts and disability adjudicators involved in the process of disability determination. In a letter to administrators of disability determination services, SSA advises disability adjudicators and reviewers not to use O*NET when making disability decisions (Social Security Administration, Office of Disability, 1999). Sylvia Karman (2009) indicated that this view is widely shared, presenting a list of four reports (Government Accounting Office, 2002a, 2002b; Social Security Advisory Board, 2001; Institute of Medicine, 1998) which, she said, either state that “both SSA and DOL acknowledge that O*NET cannot be used in SSA’s process” and/or “recommend that SSA investigate other alternatives.”
THE FUTURE OF OCCUPATIONAL INFORMATION FOR DISABILITY DETERMINATION
Having ruled out the use of O*NET for disability determination purposes, SSA has begun taking steps to develop its own occupational information system. In December 2008, the commissioner of social security established the Occupational Information Development Advisory Panel. The advisory panel was charged with providing independent advice and recommendations on plans and activities to replace the DOT currently used in the SSA disability determination process (Astrue, 2008). The panel’s report,
issued in September, 2009, recommends the creation of a new “Social Security Administration Occupational Information System” for use in disability determination (Social Security Administration Occupational Information Advisory Panel, 2009). The panel recommends development of an initial, empirically derived work taxonomy and data elements for the content model; research to examine various job classification methods; creation of internal and external research capacity at SSA; basic and applied research on work-side and person-side job attributes and demands; development of scales and measures for the dimensions of the taxonomy; and communication with users, the public, and the scientific community.1
SSA has concluded that, in its current form, O*NET does not fulfill the needs of the disability determination process. At the same time, DOL has concluded that O*NET fulfills its needs for occupational information; other chapters of this report show that O*NET meets many other occupational information needs. However, disability determination was an important use of the DOT and because O*NET was created to replace the DOT, it seems fair to conclude that O*NET has failed to replace the DOT in this particular usage.
Given public demand for budgetary restraint and efficient government, which acquire additional importance in times of economic recession and slow economic growth, duplication in government functions should be prevented. Therefore, the development of parallel, possibly redundant, occupational information systems, one for general purposes termed O*NET and the other tailored to the needs of SSA, is of concern to taxpayers. In addition, dual data collection processes would seem unnecessarily expensive.
The panel is not advocating the adoption of O*NET by SSA or the development of a hybrid O*NET-Disability system in the disability determination process. However, we conclude that a considerably modified and expanded O*NET may be capable of informing the disability determination process. There are also some potential economies of scale to be derived from the development of a single occupational information system to be used by both agencies, which may allow cost-sharing of resources in such functions as data collection and system maintenance.
An occupational information system that facilitates the process through which individuals with disabilities obtain gainful employment would help relieve the financial pressure on the SSA system and also contribute to the mental health of those who become productive members of society.
Not all stakeholders share the opinion that O*NET cannot be amended to meet the needs of those involved in disability determination. In fact, the Committee to Review the Social Security Administration’s Disability Decision Process called for interagency collaboration (Institute of Medicine,
1998). Its 1998 report encouraged SSA to explore some interagency agreement “to initiate a version of O*NET that would collect information on minimum as well as average job requirements to better serve SSA’s needs to assess ability to engage in substantial gainful activity” (p. 24).
We found evidence suggesting that these calls for collaboration between DOL and SSA were heeded. In 2000, vocational rehabilitation professionals initiated discussions with DOL and SSA which led to the creation of the Inter-Organizational O*NET Taskforce with representatives of 16 associations of physicians, psychologists, therapists, counselors, insurers and educators (Cannelongo, 2009). The group met for 4 years and proposed development of a modified version of O*NET called O*NET–D (for Disability) that would incorporate occupational information gathered in the field by rehabilitation professionals trained in job analysis, using standardized questionnaires. A pilot study of the feasibility of training rehabilitation professionals to conduct job analyses funded by DOL yielded promising results (Lechner, Cannelongo, and Keener, 2002).2
At around the same time, SSA commissioned the American Institutes for Research (AIR) to examine the suitability of O*NET for the disability determination process (Gustafson and Rose, 2003). Based on an analysis of the initial O*NET database (the “occupational analyst” database), the AIR research team found that reliability, definitional, and anchoring issues could lead to problems if O*NET data were used for disability determination. At the same time, however, the authors identified specific steps for addressing these problems. For example, they suggested that a disability decision maker could use O*NET task lists and other descriptive information to help determine the activities of claimants’ current jobs and described an approach to using selected O*NET descriptors that would adjust for the positively skewed distributions of ratings of these descriptors. Gustafson and Rose (2003, p. 15) concluded that “SSA could implement into the [disability determination process] a version of O*NET that is legally defensible and acceptable to decision-makers and claimants alike.”
Another piece of evidence, suggesting the continued possibility of collaboration between DOL and SSA, is the testimony provided by former O*NET director, James Woods, to the Social Security Advisory Board on January 13, 2009 (Woods, 2009). In his address, he regretted that earlier efforts to accommodate the SSA needs into O*NET did not bear fruit; however, he remained hopeful that O*NET
may provide a basis to help SSA focus on a specific set of data needs and to organize data within the O*NET framework—for SSA’s specific needs. O*NET, or at least the lessons learned in developing the O*NET system, may provide a starting point rather than SSA starting from scratch.
In spite of such past interagency efforts, communication and collaboration between DOL and SSA regarding a common occupational database now appears quite limited. An inspection of their most recent communications suggests that both agencies have reached the implicit conclusion that DOL will not modify O*NET to accommodate disability determination users, and that SSA will build an entirely different occupational information system for its purposes. The fact that SSA’s Occupational Information Development Advisory Panel does not include a DOL liaison suggests that the development of an SSA-sponsored system may proceed relatively independent of O*NET.
It is important to consider why SSA and other stakeholders deem O*NET inadequate for disability determination purposes. In the next sections, we present the primary issues that preclude the use of O*NET in the disability determination process in the eyes of stakeholders, the available evidence, and our conclusions regarding the type and the extent of the O*NET modifications called for by each of the issues.
MEASUREMENT OF FUNCTIONAL REQUIREMENTS
In recent years, the concept of disability has shifted its focus from diseases, conditions, and impairments to the functional limitations caused by these factors (Institute of Medicine, 2002, p. 4). A key element implicit in the contemporary view of the disability determination process is the need for an increased understanding of the physical and social factors in the work environment that may impact the Social Security claimant’s functional capacity.
There are a number of mental, physical, and psychosensory disabilities that affect the capacity to work. In the context of disability determination, the role of occupational analysis is to determine the important job behaviors and the type and level of ability that is required to perform them.
In spite of their importance for job performance, physical abilities have not received as much attention as cognitive abilities have in the occupational analysis literature (Guion and Highhouse, 2006). Much of what is known about physical abilities appears to have its origins in the work of Fleishman and his associates (Fleishman and Reilly, 1992). An overall summary of this work (Hogan, 1991) suggests that physical abilities can be subsumed into three general fitness factors: (1) muscular strength, or the ability to apply or
resist force by contracting muscles; (2) cardiovascular endurance, or aerobic capacity; and (3) coordination, or quality of movement.
Physical abilities are not the only type of ability impacted by disability. Cognitive, psychomotor, and sensory/perceptual abilities can also be impaired. Fleishman’s approach to abilities is particularly important because his measures of ability requirements—including cognitive, physical, psychomotor, and sensory abilities—were adopted in O*NET (Fleishman and Quaintance, 1984). For example, the 52 ability scales used in O*NET were drawn almost verbatim from Fleishman and Reilly (1992). Even though all of these 52 abilities conceivably have implications for disability determination, the panel heard from stakeholders who questioned their utility in the disability determination process.
Comparing the O*NET and SSA Approaches
Sylvia Karman pointed out a series of shortcomings related to disability determination in the O*NET approach to the measurement of physical requirements (Karman, 2009). These shortcomings, along with a critical examination of their rationale, are reviewed below.
Physical Abilities Versus Functional Capacity
Fleishman’s physical and sensory-motor measures use rating scales to assess occupational requirements along each physical ability dimension. As incorporated into the O*NET content model, these scales provide a definition of the ability as well as examples of tasks or job behaviors situated at various points on the scale.
This approach is most useful for a construct or a criterion-related approach to the validation of measures of physical requirements that are used for selection purposes (Hogan, 1991). For example, a number of tests are available to assess each of these abilities that can be used to identify capable individuals and can be validated in criterion-related studies (Fleishman and Reilly, 1992). However, from the point of view of disability determination, these abilities represent nonspecific, psychologically worded, or unobservable constructs that cannot be easily tied to specific disabilities or specific groups of muscles, such as those involved in lifting, kneeling, etc.
Consider, for example, the O*NET ability, Static Strength, which is defined as “the ability to use muscle force in order to lift, push, pull, or carry objects. It is the maximum force that one can exert for a brief period of time using the hand, arm, back, shoulder, or leg” (National Center for O*NET Development, no date; see Figures 4-1 and 4-2). Clearly, this physical ability construct cuts across very different muscle groups and
different body limbs. In contrast to this type of definition of a physical construct, the SSA disability determination process relies on the notion of Residual Functional Capacity (RFC), which measures the ability to perform specific physical tasks, such as lifting 20 pounds with hands and arms.3 In the RFC assessment of the claimant, the focus is on specific and observable functions or behaviors related to lifting, standing, sitting, and pushing, as well as similarly verifiable (medically and otherwise) postural limitations regarding balancing, crouching, and crawling (Form SSA-4734-BK, 08-2008).
There is a series of O*NET work context descriptors related to how much time the occupation requires sitting, standing, climbing, walking or running, and keeping one’s balance (items 34 through 39 in the work context O*NET questionnaire—National Center for O*NET Development, no date) that have conceptually parallel items in the RFC assessment, even though the anchors placed at the various points of these scales are quite different in the two approaches. For example, SSA uses specific time ranges (e.g., “about 6 hours in an 8-hour workday”), whereas O*NET uses relative scales (e.g., “about half the time”). The panel also observes that some O*NET descriptors, such as item 38 in the work context questionnaire, collapses occupational requirements across posturing, such as kneeling-crouching-stooping-crawling, whereas the RFC assessment breaks down each one of these postural limitations.
Sensory and Perceptual Abilities
In contrast to the noticeable differences found in the domain of physical abilities, the panel identified smaller differences between the O*NET and SSA approaches with regard to sensory and perceptual abilities. Note for example the almost perfect equivalence between the O*NET descriptors of near vision, far vision, visual color discrimination, and depth perception and the RFC assessment (descriptors of near acuity, far acuity, color vision, and depth perception). The scales and definition of scale points, however, are still quite different between the two scales.
The RFC assessment involves an evaluation of the claimant’s ability to sustain environmental factors, such as extreme heat, extreme cold, wetness, humidity, noise, vibration, and others, using scales ranging from “unlimited” to “avoid all exposure.” Although the panel did not find perfectly
equivalent descriptors in O*NET, the O*NET Work Context domain involves a number of conceptually similar items related to exposure to either very hot or very cold temperatures (item 23), such contaminants as gases and dust (item 25), and whole-body vibration (item 27). The anchors in these O*NET scales range from “never” to “every day,” and the anchors in the RFC range from “unlimited” to “avoid all exposure.”
Use of Behavioral Anchors
O*NET uses the scale format known as Behaviorally-Anchored Rating Scale (BARS), in which behavioral anchors representing differing degrees of a construct are placed along the scale continuum (see Chapter 4). The various degrees of the continuum represented by these scales are illustrated through “anchors” situated at the corresponding scale points. These anchors are short statements describing tasks purportedly representing the level of the construct: “light a candle” is placed next to the scale point 2 in the Ability scale termed Arm-Hand Steadiness,” “thread a needle” is placed next to the scale point 4 in the same scale, and “cut facets on a diamond” is placed next to the point 6 in the scale.
Clearly, there are variations in the degree of arm-hand steadiness lying between any of these pairs of proximal anchors. If the O*NET occupational unit score on arm-hand steadiness is 3 (requiring a level of arm-hand steadiness between 2–light a candle and 4–thread a needle), it seems nearly impossible to determine the type of task that a potential claimant should be able to complete to be deemed capable of performing work in this occupation unit. For example, two Social Security disability claimants, both of them capable of lighting a candle and unable to thread a needle, may or may not be able to perform an occupation with a score of 3 on this ability. This could be true because the two claimants have different limitations in their degree of arm-hand steadiness, despite the fact that both of them are unable to thread a needle.
Still another issue related to the behavioral anchors employed in the O*NET physical ability scales is whether ability requirements are scaled at the level of the ability required by the most demanding task or the typical (average) task. For example, a Social Security claimant may be capable of performing the occupation because she or he has enough arm-hand steadiness to thread a needle, so long as threading a needle represents the maximal level of ability that would be required on the job; however, that same claimant may be unable to perform all the work in the occupation if threading a needle represents the level required by the typical, average, or everyday task.
Source of Physical Requirements Information
Another concern is about the source of the ability requirement information. Karman viewed the National Center for O*NET Development’s use of trained occupational analysts to judge ability requirements, using solely a written description of the occupation, as an obstacle to relying on this information for disability determination purposes (Karman, 2009). This concern may be accentuated when physical requirements are being determined, because many of them lend themselves to job observation. The process of disability determination can be quite litigious, and those in charge of making the determination prefer to minimize the risk of legal challenges by relying on occupational information gathered directly by trained vocational or job analysts. As noted in Chapter 1, trained occupational analysts gathered information directly from job incumbents for inclusion in the DOT.
Level of Aggregation in Occupational Categories
According to Karman (2009), the number of occupations included in O*NET is too small for disability determination purposes, because each occupation involves multiple, heterogeneous jobs that may have different physical and education requirements. If her assertion is correct that there is a wide range of physical and education requirements of jobs within the same O*NET occupation, then SSA would find it nearly impossible to determine whether or not a given disability precludes a claimant from performing a specific job in the occupation.
The process used to create and write descriptions for the 1,122 original O*NET occupations, referred to as “occupational units” was complex, according to a report of the National Center for O*NET Development (1998). It entailed the use of the occupational classification system adopted by the Bureau of Labor Statistics to administer the Occupational Employment Survey, the development of crosswalks to DOT title codes, cluster analyses of DOT data, analysis and aggregation of DOT task statements, and multiple reviews by subject matter experts. As described in this report, even though DOT titles and task data contributed to the original formation of these occupational units, the occupational units were not the outcome of a simple clustering of DOT titles, nor were they meant to represent simple aggregations of DOT titles. It is not unreasonable to conclude that, at the end of this process, each occupational unit had its own identity independent of—though partially informed by—DOT titles and task content.
Since that time, the O*NET occupational classification system has been revised several times, so that the current O*NET-SOC 2009 occupations may exhibit less within-occupation variability than did the occupational units created by the research team in the late 1990s. Nevertheless, the
reduction from over 12,000 occupational titles in the DOT to the current 1,102 occupations in O*NET-SOC 2009 will inevitably be accompanied by some increase in within-occupation variability.
Karman presented a chart indicating wide variability in education and physical requirements across 553 DOT titles that she said were clustered into a single O*NET occupational unit (51-9198, Helpers-Production Workers) (Karman, 2009). Harvey (2009) presented similar data and drew similar implications regarding what in his view constituted excessive aggregation in the original O*NET occupational units for the purpose of disability determination. However, as noted above, the occupational units were not intended to be merely aggregated DOT titles. The question of the extent of variability in current O*NET occupations deserves further study.
CONCLUSIONS AND RECOMMENDATIONS
The SSA’s disability determination process currently relies on assessment of the residual functional capacity of a claimant, focusing on physical functions or behaviors and postural limitations as well as on mental functions, if indicated. Matching the results of the RFC to the descriptors of physical ability and occupational context employed in O*NET is inherently difficult. Nevertheless, there are commonalities in the descriptors used in these two systems, even though substantial differences remain in the level of detail, specificity, and types of scales employed to measure them. The evidence indicates that occupational descriptors involving exposure to unusual environmental demands, such as heat or cold, exist in both O*NET and the RFC assessment used by SSA. However, there is no clear, one-to-one correspondence between the two types of environmental descriptors, because some environmental factors are defined and grouped quite differently in the two models. Taken together, the differences and similarities suggest that continued collaboration between DOL and SSA is in the interest of efficient use of government resources.
Recommendation: SSA and DOL should create an interagency task force to study the viability of potential modifications of O*NET to accommodate the needs of SSA with regard to disability determination. Before implementing these or similar modifications, however, we recommend that the task force conduct (1) an in-depth needs analysis of the occupational information required by the current disability determination process and (2) an interagency cost-benefit and cost-sharing analysis of the additional resources that would be needed to make O*NET suitable to the disability determination process.
The reduction from over 12,000 occupational titles in the DOT to the
current 1,102 occupations in O*NET-SOC 2009 has been accompanied by some increase in within-occupation variability in the physical and mental requirements of the work included in these two different types of occupational categories. Because the extent of this variability has important implications for the usefulness of O*NET in disability determination, it should be studied.
Recommendation: As part of the research on the occupational classification system recommended in Chapter 3, the Department of Labor should commission research to determine whether and to what extent O*NET occupations represent excessively heterogeneous clusters of jobs (in terms of their physical and cognitive requirements) for the purpose of disability determination. This research should include gathering evidence from firsthand observations regarding physical requirements and verifiable survey responses from well-informed sources capable of assessing cognitive requirements.
Astrue, M.J. (2008, December 23). Establishment of the Occupational Information Development Advisory Panel. Federal Register, 73(247), 78864. Available: http://www.socialsecurity.gov/oidap/Documents/federal_register/federal_register.htm [accessed November 2009].
Cannelongo, J.M. (2009). Statement to the Social Security Advisory Board. Available: http://www7.nationalacademies.org/cfe/Social%20Security%20Advisory%20Board%20Presentation.pdf [accessed September 2009].
Fleishman, E.A., and Quaintance, M.K. (1984). Taxonomies of human performance. Orlando, FL: Academic Press.
Fleishman, E.A., and Reilly, M.E. (1992). Handbook of human abilities. Palo Alto, CA: Consulting Psychologists Press.
Government Accounting Office. (2002a, July 11). SSA disability programs: Fully updating disability criteria has implications for program design. Available: http://www.gao.gov/new.items/d02919t.pdf [accessed November 2009].
Government Accounting Office. (2002b, August). SSA and VA disability programs reexamination of disability criteria needed to help ensure program integrity. Available: http://www.gao.gov/new.items/d02597.pdf [accessed November 2009].
Guion, R.M., and Highhouse, S. (2006). Essentials of personnel assessment and selection. Mahwah, NJ: Lawrence Erlbaum.
Gustafson, S.B., and Rose, A. (2003). Investigating O*NET’s suitability for the Social Security Administration’s disability determination process. Journal of Forensic Vocational Analysis, 6,3-15.
Harvey, R.J. (2009). The O*NET: Do too abstract titles + unverifiable holistic ratings + questionable raters + low agreement + inadequate sampling + aggregation bias = (a) validity, (b) reliability, (c) utility, or (d) none of the above? Paper prepared for the Panel to Review the Occupational Information Network (O*NET). Available: http://www7.nationalacademies.org/cfe/O_NET_RJHArvey_Paper1.pdf [accessed July 2009].
Hogan, J. (1991). Structure of physical performance in occupational tasks. Journal of Applied Psychology, 76, 495-507.
Institute of Medicine. (1998). The Social Security Administration’s decision process: A framework for research, 2nd interim report. Committee to Review the Social Security Administration’s Disability Decision Process Research, G.S. Wunderlich and D.P. Rice (Eds.). Washington, DC: National Academy Press.
Institute of Medicine. (2002). The dynamics of disability: Measuring and monitoring disability. Washington, DC: The National Academies Press.
Karman, S. (2009). Definition of disability: What compels SSA to use DOT. Presentation to the Panel to Review the Occupational Information Network (O*NET). Available: http://www7.nationalacademies.org/cfe/Karman%20Power%20point.pdf. [accessed June 2009].
Lechner, D., Cannelongo, J., and Keener, J. (2002). Feasibility of internet-based training and reliability of internet-trained analysts. Submitted by Aguirre International under Department of Labor Contract BB-12266-02-03. Washington, DC: Aguirre International.
National Center for O*NET Development. (no date). Questionnaires. Available: http://www.onetcenter.org/questionnaires.html [accessed July 2009].
National Center for O*NET Development. (1998). Appendix D: The development of the Occupational Information Network (O*NET) analyst database. Raleigh, NC: Author. Available: http://www.onetcenter.org/reports/appendix_d.html [accessed November 2009].
Social Security Administration. (2008). Physical residual functional capacity assessment. Form SSA-4734-BK (08-2008). Washington, DC: Author.
Social Security Administration, Office of Disability. (1999, June 30). Policy instruction DDSAL-507. Washington, DC: Author.
Social Security Administration Occupational Information Advisory Panel. (2009). Content model and classification recommendations for the Social Security Administration Occupational Information System. Available: http://www.socialsecurity.gov/oidap/Documents/Occupational%20Information%20Development%20Advisory%20Panel.pdf [accessed November 2009].
Social Security Advisory Board. (2001, January). Charting the future of Social Security’s disability programs: The need for fundamental change. Available: http://www.ssab.gov/ Publications/Disability/disabilitywhitepap.pdf [accessed November 2009].
Woods, J. (2009, January 13). Testimony to the Social Security Advisory Board. Available: http://www7.nationalacademies.org/cfe/Woods%20Testimony%20to%20SSA.pdf [accessed January 2010]. Unpublished document provided to the Panel to Review the Occupational Information Network.