7
Findings and Recommendations

VALUE AND UTILITY OF THE CURRENT LISTINGS

Task 1 Statement: “… the committee will consider … [t]he value and utility of the current Listings for all users (claimants, SSA, health care professionals, state offices, and officials involved in the adjudication process).


The Listings were originally created as a case-screening tool that would allow the Social Security Administration (SSA) to rapidly identify allowance cases without having to perform a comprehensive analysis of an individual’s capacity to work. By setting the severity standard of the Listings at a higher level (inability to engage in any gainful activity) than its fundamental disability standard (inability to engage in substantial gainful activity), SSA is able to identify a significant number of allowance cases and to have some degree of confidence that these cases would be allowed if they were subject to a more comprehensive disability assessment. By relying heavily on diagnostic and clinical criteria in the Listings, SSA also ensures that disability decisions based on the Listings have a medical basis, as required by the law. By using identical and specific Listings criteria for all applicants, SSA provides equal treatment for all. So, at least on a conceptual level, using the Listings as an administrative screening tool provides significant value and utility to a variety of different constituencies:

  • For SSA itself, the Listings help reduce the burden of processing millions of disability claims each year, reducing administrative costs and case-processing time.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 89
Improving the Social Security Disability Decision Process 7 Findings and Recommendations VALUE AND UTILITY OF THE CURRENT LISTINGS Task 1 Statement: “… the committee will consider … [t]he value and utility of the current Listings for all users (claimants, SSA, health care professionals, state offices, and officials involved in the adjudication process). The Listings were originally created as a case-screening tool that would allow the Social Security Administration (SSA) to rapidly identify allowance cases without having to perform a comprehensive analysis of an individual’s capacity to work. By setting the severity standard of the Listings at a higher level (inability to engage in any gainful activity) than its fundamental disability standard (inability to engage in substantial gainful activity), SSA is able to identify a significant number of allowance cases and to have some degree of confidence that these cases would be allowed if they were subject to a more comprehensive disability assessment. By relying heavily on diagnostic and clinical criteria in the Listings, SSA also ensures that disability decisions based on the Listings have a medical basis, as required by the law. By using identical and specific Listings criteria for all applicants, SSA provides equal treatment for all. So, at least on a conceptual level, using the Listings as an administrative screening tool provides significant value and utility to a variety of different constituencies: For SSA itself, the Listings help reduce the burden of processing millions of disability claims each year, reducing administrative costs and case-processing time.

OCR for page 89
Improving the Social Security Disability Decision Process For individuals involved in case adjudication (SSA and Disability Determination Services [DDSs] personnel), the Listings reduce the effort and expense of processing applications by separating easy-to-approve cases from cases that require more in-depth review. For applicants (and their families, representatives, and health care providers), the Listings allow faster decisions in clearcut cases of inability to engage in any gainful activity and promote fairness (by providing consistent criteria and procedures). For the general public, the Listings promote efficiency as well as fairness in adjudication. In addition, over several decades of use, the Listings have become an established part of the SSA disability process. They are understood and well accepted by both the legislative and judicial branches of government. Using the Listings as an administrative screening tool to identify and quickly allow obviously disabled applicants provides significant value and utility to a variety of different constituencies, including claimants, DDSs and SSA, and the general public. The committee could find no compelling reason why SSA should abandon the Listings process. However, the fact that the Listings have value and utility as a decision-making tool does not mean that they cannot and should not be improved. Validity of the Listings as a Measure of Disability If the purpose of the Listings is to identify quickly a subset of legitimately eligible claimants (i.e., individuals so medically impaired that they cannot be expected to achieve substantial earnings), they can be evaluated and perhaps improved as a diagnostic test whose performance is measured by its sensitivity, specificity, predictive values, and similar performance criteria (for example, examining their receiver operator characteristic curves).1 The evaluation of the test is affected somewhat by the fact that it is the screening phase in a two-phase decision process. The goal of the overall decision process is to distinguish individuals who meet the Social Security definition of disability from those who do not, so that the people who should be awarded benefits (i.e., true positives) are allowed benefits and people who should not receive benefits (i.e., false positives) are denied. Ideally, the overall decision process has high sensitivity (i.e., ability to identify and award true positives) and high specificity (i.e., ability to identify and 1 Receiver operator characteristic curves, developed in the 1950s as a statistical method of detecting radar and sonar signals contaminated by noise, are very useful for exploring the tradeoff between the sensitivity and specificity of diagnostic and screening tests and comparing two or more tests.

OCR for page 89
Improving the Social Security Disability Decision Process deny false positives). Because the political costs of denying benefits to applicants who meet the SSA disability standard are high, while the political and economic costs of awarding benefits to applicants who do not meet the standard are also high, SSA is willing to spend a substantial amount of time and resources to increase the sensitivity and specificity of the decision process. This, however, subjects SSA to criticism for taking too long to decide claims. The Listings step was introduced to increase the speed of the process, even though it may reduce accuracy because it relies on less information. In theory, the accuracy problem is mitigated, because SSA uses a stricter standard (i.e., any gainful activity), which should increase specificity (i.e., reduce the number of false positives). Although this diminishes sensitivity, SSA can tolerate having more truly disabled claimants fail the screen, because they are not denied. Rather, they are considered in the next phase of the process (steps 4 and 5). Because these latter steps are less strict, they should be awarded benefits (this assumes that steps 4 and 5 function effectively, an issue that this committee did not address). The use of the Listings relies on an assumption that a subgroup of individuals exists with medical impairments and associated functional limitations so severe that they are unable to work and, therefore, merely ascertaining the existence of such impairments justifies awarding them benefits. (In practice, the standard is slightly less severe and therefore more practical, because some people work despite impairments of listing-level severity—e.g., quadriplegia, total blindness; the actual standard is a medical condition so severe that the individual cannot reasonably be expected to work.) This means that the degrees of medical severity described in the Listings should be strongly correlated with not working. However, such a correlation has never been clearly established for most conditions in the Listings, and little work has been done to establish the extent to which the Listings are a valid measure of work disability. With the exception of studies done during the development of the adult mental disorders listings implemented in 1985 (Pincus et al., 2001; reviewed in detail in Kennedy, 2002), the committee is not aware of any studies, data, or other information that establish the extent to which the Listings are a valid measure of work disability. The only validation of the decision to award benefits on the basis of a listing is the review for quality assurance of a random sample of a few percent of allowances and denials, which determines if the criteria were met, not whether they are the right criteria. As noted above, SSA began an effort to validate the Listings in 2001. Although this initial work on validation was completed over five years ago, there does not seem to have been any follow-through. This is not to suggest that the approach proposed by the Disability Research Institute is the only possible approach or that it should be adopted. Rather, it merely

OCR for page 89
Improving the Social Security Disability Decision Process illustrates that there are ways to evaluate the validity of the Listings, even in the absence of an objective “gold standard” for what constitutes disability for SSA purposes. The lack of a gold standard—i.e., a way to identify true positives and negatives independent of SSA’s process—hampers the ability of SSA to assess its decision-making process, including the effectiveness of the Listings in expediting awards at a reasonable cost. SSA should support research and perform analyses of its own program data to approximate a gold standard and better evaluate how well the Listings (and the entire decision process) work. Because there is no external objective gold standard against which the Listings can be measured, the technique of latent class analysis may be a helpful approach to refine the Listings and document their performance. Latent class analysis is a statistical technique for finding groupings of related cases (i.e., “latent classes”) underlying observed categorical or nominal data (Pepe, 2003:197). One use of latent class analysis is to evaluate diagnostic tests in the absence of a gold standard, by providing estimates of sensitivity, specificity, and other properties of the tests.2 Inappropriate Allowances Based on the Listings One important aspect of assessing the validity of the Listings is the extent to which they identify as disabled individuals who actually meet SSA’s disability criteria (i.e., the “true positives”). An equally important issue is the extent to which they incorrectly identify as disabled those individuals who do not meet SSA’s disability criteria (i.e., “false positives”). There are reasons to minimize false positives in the Listings. Most obviously, they constitute extra costs to the taxpayers in the form of benefits that should not be provided. Benefit entitlement also tends to serve as a disincentive for eventual return to work and discourages those receiving benefits from risking reentry into the labor market (primarily because of the length of time required to be out of work while qualifying for the program and the fact that applicants cannot receive vocational rehabilitation services until they are allowed cash benefits). To the extent that benefits are awarded to individuals who could otherwise work, this may remove capable workers from the labor force. It may also make it more likely that other individuals who could work will drop out of the labor force and apply for benefits, thus increasing application rates, reducing em- 2 Examples of the use of latent class analysis in evaluating diagnostic tests in the absence of a gold standard include Albert et al., 2001; Boelaert et al., 2004; Ferraz et al., 1995; Moayyedi et al., 2004; and Young et al., 2003. Statistical methodology sources include Albert and Dodd, 2004; Hui and Zhou, 1998; Pepe, 2003; Pepe and Alonzo, 2001; Qu et al., 1996; Walter and Irwig, 1988; and Yang and Becker, 1997.

OCR for page 89
Improving the Social Security Disability Decision Process ployment, and leading to even more individuals being added to the Social Security Disability Insurance (SSDI) rolls. For society, this would mean the loss of the goods and services these workers could have produced as well as the imposition of additional taxes to pay for their benefits. Nevertheless, given the nature of the sequential evaluation process and the Listings’ “screen in” role in that process, false positives are an inevitable cost of the administrative efficiency that the Listings allow. SSA staff and executives who discussed these matters with the committee clearly recognized this fact. They also understood that false positive and false negative rates can be manipulated, to some extent, by the choice of Listings criteria. Calibrating the Listings to a fairly low degree of impairment severity will tend to increase the number of desirable true positives (i.e., screen in more applicants who deserve to receive benefits)—but at the cost of also increasing undesirable false positives. Calibrating the Listings to a higher degree of impairment severity will tend to decrease the number of undesirable false positives—but at the cost of also decreasing the desirable true positives. (SSA already does the latter to some extent by deliberately trying to set the Listings criteria to a severity level higher than the disability standard, i.e., inability to engage in any gainful activity, instead of inability to engage in any substantial gainful activity.) If SSA makes the Listings too strict, their purpose of saving time and attendant resources is diminished. Establishing the correct balance— determining what is an acceptable rate of false positives (representing unwarranted program expenditures) as a trade-off for a desired level of true positives (representing administrative cost savings in terms of faster processing time)—is a policy issue that SSA must decide. However, to decide this SSA needs to know the false positive rate and its corresponding program costs, as well as the amount of administrative cost savings. In discussions with the committee, SSA staff noted an isolated instance in which SSA actively investigated whether a listing was producing false positive results. In August 1999, SSA revised the Listings to remove Listing 9.09, Obesity. In explaining its removal, SSA wrote: … we reviewed a small group of cases in which individuals were found disabled based on a finding that their impairments met or equaled listing 9.09 … [and we found] that, in a significant number of cases, we would not have found the individuals disabled under other listings or at step 5 of the sequential evaluation process. This was a very limited investigation into a single listing. But it is an example of the kind of investigation that could shed light onto the performance of the Listings, especially in terms of understanding the extent to which individual listings are producing inappropriate allowances.

OCR for page 89
Improving the Social Security Disability Decision Process Allowance Cases Not Identified by the Listings The strict criteria of the Listings serve to make it highly specific, i.e., unlikely to identify and allow many false positives. Specificity is gained at some expense to sensitivity, i.e., it is likely to miss more true positives than a less strict screen. Truly disabled individuals who fail the screen and are not allowed at step 3 of the initial decision process are still considered in steps 4 and 5. If they meet the SSA definition of disability, the DDS should find they cannot do their usual work and any other work and allow them. This is not to say that there are no undesirable consequences of an incorrect finding that an individual does not meet a listing, or that there is no reason to minimize these kinds of outcomes. Failure to identify a disabling impairment at step 3 represents a lost opportunity for adjudicative efficiency, and every such lost opportunity reduces the value and utility of the Listings. SSA should also ensure that the false negative rate at steps 4 and 5 is as low as possible, subject to the trade-off that SSA is willing to make between the sensitivity and specificity of those steps. Modifying steps 4 and 5 of the process to identify a high percentage of truly disabled (i.e., increasing sensitivity) is likely to also increase the rate of false positives (i.e., lower specificity). As with the Listings, the underlying rates of true disability and the sensitivity and specificity of the medical-vocational steps of the decision process are not known. A decade ago, SSA asked the Institute of Medicine (IOM) to conduct an independent review of SSA’s plans to redesign its disability decision process, a project that SSA had begun in 1994. After the committee deliberated and issued an interim report highly skeptical of the new plan, SSA decided to no longer pursue the new decision process; instead, it rededicated itself to improving the existing decision-making process, including “a concentrated effort to update and improve the Listings.” In its 2002 final report to SSA (IOM and NRC, 2002:129-131), the committee had several observations and recommendations for SSA with respect to the Listings. Regarding SSA’s plans to update the Listings, the committee observed: It appears likely that the agency’s agenda for reform in this area will be driven as much by internal and external anecdotal concerns, including general perceptions of which Listings are the most outdated, as by any long-range strategy. Nevertheless, the committee believes that a successful process of Listings revision must be based on a systematic approach to evaluation, design, and testing. The committee has not seen any indication of a plan for determining the specificity and sensitivity parameters for any existing or proposed Listing. Developing such parameters seems critical to both the scientific and the political validation of the Listings as a decisional tool.

OCR for page 89
Improving the Social Security Disability Decision Process Because the Listings screen is meant to be used to identify clear cases of disability, one would expect this screen to be devised such that it is highly specific (seldom identifies false positives) and relatively sensitive (identifies some substantial number of true positives). The question for SSA is how specific and how sensitive. In order to undertake meaningful research on the validity of any medical listing, SSA must be able to specify the acceptable level of specificity and sensitivity by which it can validate the screen against those criteria. SSA provided the committee with a list of ongoing projects designed to update the Medical Listings and improve their performance. The committee, however, has no information suggesting that baseline criteria were established at the outset or that any method was developed for validating the existing and proposed new Listings against those criteria. These are serious and difficult issues. As SSA moves forward to incrementally revise and reform the current decision process, it must be able to determine whether or not changes are improving the accuracy of the process. Indeed, it has to be able to make these determinations prior to the time that changes are implemented on a national basis. Whether or not specific Listings need to be improved and the direction of that improvement must await the results of the baseline evaluation and subsequent reevaluation. The committee went on to restate and reinforce a recommendation that it had made in its interim report from years earlier: SSA should conduct the necessary research, prior to making changes in the Listings, to (1) determine whether or not the current Listings satisfy the agency’s goals for specificity and sensitivity, (2) determine whether or not these goals are satisfied consistently across the Listings for the different body groups or conditions, and (3) evaluate the options to correct the problems detected by these evaluations, as it develops any new list of medical impairments. That IOM committee also cited, and supported, similar recommendations that had been previously made to SSA in 1996 by the Disability Policy Panel of the National Academy of Social Insurance (NASI, 1996:22): Experts on SSA’s medical criteria [i.e., the Listings] report that there is considerable variation among the criteria used for different body systems in terms of the severity of impairments that are presumed to constitute work disability. To date, no systematic research has been done to evaluate the consistency of the presumptions underlying the medical criteria for different body systems. Research of this kind should have a high priority. The current committee believes that these observations and recommendations are as valid today as they were when they were first made.

OCR for page 89
Improving the Social Security Disability Decision Process Declining Use of the Listings as a Screening Tool As noted previously, SSA is concerned that the Listings, which once formed the basis for the vast majority of the disability allowance decisions, now account for a significantly lower percentage of total allowances. While the Listings originally accounted for more than 90 percent of allowances, they are currently the basis for barely 50 percent. The agency seeks to make the Listings a more effective screening tool by increasing the rate at which allowance cases are identified by the Listings. The extent to which the new quick disability determination (QDD) process being piloted as part of SSA’s Disability Service Improvement Plan in the New England region increases the ability of the Listings to expedite cases remains to be seen. QDD should tend to increase speed by pulling allowances that can be decided quickly into a faster process of decision making, performed by more experienced adjudicators. On the other hand, the QDD process picks cases to expedite using factors in addition to medical criteria and looks for quick denials as well as quick allowances, which may slow the adjudication of cases meeting the Listings. Ultimately, the correct rate at which allowances are made at the listings step has to be determined by carefully analyzing the actual outcomes of the disability determination process. Whether the best rate is 90 percent or 50 percent or some other figure is an empirical question. There does not appear to be any basis for assuming that the high percentage of cases identified as allowances by the Listings in the 1950s and 1960s (which unquestionably made the decision process more expeditious) were “correct” allowances, any more than it is safe to assume that it is possible to increase the rate at which allowance cases are now identified by the Listings without undesirable consequences. In discussing these matters with the committee, SSA executives referred to a common perception that there was a change in the Listings from “medical” to “functional” standards in the mid-1980s, suggesting that this change may have significantly contributed to their declining use as a basis for allowance. These changes included a major revision to the adult mental disorders criteria that took effect in August 1985 and changes in most of the other body systems that took effect in January 1986. However, the data reviewed by the committee do not support a cause-and-effect relationship between these changes and use of the Listings. As can be seen in Figure 7-1, use of the Listings as the basis for allowance has been in decline since about 1983, when the Listings were used in 82 percent of initial allowances. Following a decline from 82 percent in 1983 to 72 percent in 1985, use of the Listings went up in 1986 and 1987, when the revised Listings were in effect, then resumed falling until the present day. After revisions in the Listings promulgated in 1985, SSA undertook no

OCR for page 89
Improving the Social Security Disability Decision Process FIGURE 7-1 SSDI allowances meeting/equaling the Listings: 1975-2004 (percentages). NOTE: Based on initial state agency determinations for SSDI-only and concurrent claims (excludes Supplemental Security Income [SSI]-only allowances and allowances made on the basis of the Listings at subsequent stages of the appeal process). SOURCE: SSA tables. other significant modifications until 1993 and 1994, when it revised the respiratory and cardiovascular listings, and then none again until 1999. Yet despite continuity in the Listings between 1985 and 1993, the steady decline in use of the Listings continued unchecked. The decline continued at a steady rate throughout the 1990s, with no indication that the changes in 1993 and 1994 had any effect. In addition, the sudden surge in Listings usage in 1986 is the opposite of what would be expected if the 1985 revision had been a cause of the overall decline in their usage. Therefore, it does not seem reasonable to attribute the decline in Listings usage to any changes in the Listings themselves, either as a result of a perceived change in basic approach (functional vs. medical) or as a result of specific new Listings criteria. There are many other possible explanations for the decline. One factor might be gradual advances in medical treatment and outcomes, resulting in fewer and fewer applicants exhibiting

OCR for page 89
Improving the Social Security Disability Decision Process the profound manifestations (clinical or functional) of disorders reflected in the Listings. Another factor might be the major change in the evaluation of vocational factors SSA implemented in 1979 with the publication of the medical/ vocational rules in Appendix 2 to Subpart P of 20 CFR Part 404 (the so-called “grid” rules), followed by several years in which SSA published many major policy rulings explaining the use of those new rules (especially in 1982, 1983, and 1985). Another factor could be increasing filing rates throughout the 1980s and 1990s, with a corresponding increase in the percentage of cases involving less obvious disability. However, regardless of the reasons for the change in the rate at which the Listings are used as the basis for allowance decisions, the correct rate can only be determined through a comprehensive assessment of their performance and effectiveness. RECOMMENDATION 1. SSA should continue to use the current Listings as a screening test in its disability decision process, but should increase their value and utility by closely examining and monitoring their performance, conducting research to evaluate and improve their effectiveness in expediting awards in obvious cases, and making timely changes in response to these evaluations. CONCEPTUAL MODELS FOR ORGANIZING THE LISTINGS Task 2 Statement: “ … the committee will consider … [c]onceptual models for organizing the Listings, beyond the current ‘body systems’ model specified in federal regulations.” The Listings are based on a medical model, which is not ideal, given what is known about the factors causing disability. However, a better model does not exist at this time. The bases for the listings are anatomic, diagnostic, and functional, the mix of which varies from listing to listing and body system to body system. As medical treatments and assistive technologies advance, the anatomic and diagnostic bases for the Listings will become less and less useful as markers of disability. SSA should begin now to look for better methods of determining disability for Social Security benefits. One alternative to a body-system-based list of diagnoses and impairments would be to develop and apply a generic functional evaluation that does not attempt to assess the severity of impairment but goes directly to the functional capacities of claimants to engage in substantial gainful activity (SGA). Of course, to meet SSA statutory requirements, such an

OCR for page 89
Improving the Social Security Disability Decision Process evaluation would still have to establish a connection between the functional limitations and a “medically determinable impairment” that causes those limitations. This would be more suited to the nature of employment disability as it is understood conceptually today, i.e., not equal to a person’s degree of impairment or even of functional limitation but rather the result of the interaction of a person’s functional capacities with characteristics of the workplace and other external factors, such as support systems. SSA considered developing such a screening tool in the 1990s and eventually dropped the effort (for an extensive discussion of that effort, see IOM and NRC, 2002:Chapter 6). The opinion of the committee is that a screening tool equal to the complexity of disability does not exist at this time. A potential model for a functionally based screening tool is the current process that SSA uses to identify children applying for SSI whose impairments are “functionally equal” to the Listings. For children applying for SSI, the disability standard is stricter than the standard used for adults (under either the SSI or SSDI programs). Children must have impairments that are of listing-level severity. Because the Listings focus largely on medical criteria for many impairments, SSA developed a process by which it considers the functional consequences of a child’s impairments in the context of the Listings. As with adults, SSA begins by evaluating whether the child’s impairment meets any of the criteria in the Listings. If not, it next considers whether the child’s impairment is equivalent to the Listings based on the medical factors, as it does for adults. However, if the impairment does not meet or medically equal the severity of the Listings, SSA then goes on to make one additional assessment for children using the Listings that it does not make for adults—functional equivalence. With functional equivalence, SSA performs an overall functional assessment of the child using six broad domains.3 The domains are: acquiring and using information, attending and completing tests, interacting and relating with others, moving about and manipulating objects, caring for yourself, and health and physical well-being. Each domain is scored by an SSA pediatrician on a four-point scale: no limitation, less than marked, marked, or extreme. Functional equivalence is achieved with marked limitations in two domains or an extreme limitation in one domain. In effect, this establishes a universal “functional” listing that applies regardless of the nature of the child’s impairment. A similar approach could be developed for adults, although it would most likely be a substantial undertaking for SSA to develop and validate appropriate functional criteria for adults. Established norms for the physical, mental, and social growth exist for children but not for adults. 3 The childhood functional equivalence process is described in SSA’s regulations at 20 CFR 416.926a.

OCR for page 89
Improving the Social Security Disability Decision Process making. The most obvious example of this is the lack of feedback to DDS adjudicators on the results of appeals to the Office of Hearings and Appeals of their decisions. The committee suggests that SSA closely examine ways of using program information to identify listings that are not being applied consistently or to indicate possible changes in the effectiveness of listings. RECOMMENDATION 3. SSA should develop a management information system that combines a balanced quality assurance process to promote consistency and reliability of individual allowances and denials and a program of analysis of aggregate patterns to evaluate consistency and reliability of the Listings. SSA should also develop feedback processes to inform adjudicators and program managers of decision results, including those found by subsequent adjudicators to meet or equal the Listings. External Input Affecting the Listings No matter how reliable and valid the Listings may be at any given moment, they are constantly affected by external developments. These include changes in disease patterns, advances in scientific knowledge and medical practice, advances in assistive technologies, and changes in the workplace affecting workers in terms of job requirements and potential sources of injury. The most common devices that government agencies use to ensure that evidence-based regulations are kept current are: feedback from the regulatory process staff research external advisory committees SSA has expanded regulatory feedback in recent years by sponsoring policy conferences and using advanced notices of proposed rulemaking (ANPRMs). At policy conferences, medical specialists present the latest research and medical practices and interact with beneficiaries, advocates, and SSA disability officials. ANPRMs solicit suggestions from all interested parties on how the Listings should be revised. The committee supports these efforts to incorporate more public and professional input into the Listings revision process. The Office of Medical Policy, the staff component of the SSA Office of Disability Programs, is responsible for maintaining the Listings. Currently, the Office of Medical Policy has seven medical officers, who are charged with keeping abreast of the medical literature, such as the results of clinical trials, research on outcomes, and practice guidelines. Five are physicians,

OCR for page 89
Improving the Social Security Disability Decision Process with expertise in psychiatry, physical medicine and rehabilitation, neurology, and pediatrics. In addition, there is a speech and language pathologist and a psychologist. This office is small and does not have experts in all the major specialties (although it can draw on the advice of specialists in the federal DDS), so its ability to supply the necessary medical expertise to the Listings revision process is limited. RECOMMENDATION 4. SSA should ensure that its Office of Medical Policy has the expertise relevant to the full range of listed impairments and has the resources to stay knowledgeable concerning new developments in medicine and rehabilitation, for example, by conducting systematic literature reviews on a periodic basis. Use of Medical Advisory Committees External advisory committees are common in federal agencies. They are a way to tap expertise in the scientific and medical research community, and they also increase understanding and acceptance of government policies and programs among affected constituency groups. Throughout its history, SSA has used a variety of different advisory committees, advisory councils, and similar groups as a source of expert advice and recommendations on difficult issues (SSA, 2001). SSA formed an external medical advisory committee in 1955 to create the original Listings. More informal medical advisory groups were used into the 1980s and were a part of the efforts to incorporate consideration of pain in the Listings, revise the cardiac listings, and revise and expand the mental listings. During that time, SSA would assemble a panel of experts from both inside the agency (including DDSs) and outside (including medical experts in the appropriate fields). The use of medical advisory groups was stopped when the Federal Advisory Committee Act (FACA) was instituted, and advocacy groups asked for representation. SSA management decided that consumer representation on medical advisory committees was not appropriate and discontinued the medical advisory committee. Despite the strictures of FACA, other agencies with similar responsibilities to develop and apply scientifically based regulations—such as the Centers for Medicare and Medicaid Services, Food and Drug Administration, and Environmental Protection Agency—use expert advisory committees extensively to obtain advice on when and what regulatory changes are needed. They establish and operate their advisory committees under FACA. These external committees are typically balanced not just among areas of expertise but also among viewpoints, including representatives of consumers and, in some cases, affected industries as well as the relevant clinical specialties. SSA’s disability programs would benefit from external advice

OCR for page 89
Improving the Social Security Disability Decision Process from clinical and other experts on disability determination criteria and procedures, and the involvement of external advisory groups would also increase the understanding and acceptance of SSA disability determination requirements in the medical community. RECOMMENDATION 5. SSA should re-establish a medical advisory committee under the Federal Advisory Committee Act to advise the commissioner on when scientifically based regulations, especially the Listings, should be revised to keep them up to date. Ad hoc advisory committees should be established under the auspices of the medical advisory committee to advise on the revision or addition of specific listings or body systems. The medical advisory committee should be multidisciplinary and include representation from all appropriate constituencies. Responding to Advances in Medical Practice and Technology Medicine evolves in ways that may result in the need to revise a listing, but it also makes more incremental advances that affect the way the Listings are applied. In such cases, the standard of severity in the listing does not change but the method of evaluating it may. One way SSA has tried to make the Listings better adapt to these kinds of changes is to make the Listings themselves more generic. In the past, listings criteria have been quite specific; for example, naming particular tests, such as x-rays or diagnostic criteria, or referring to a specific edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM). SSA has begun taking a more generic approach; for example, specifying “imaging studies” rather than x-rays or MRIs, and referring to “the latest edition” of the DSM, rather than a specific edition. This approach is helpful and should be pursued, but it is fairly limited in scope. The greater concern is how SSA can revise the Listings fast enough to keep pace with medical advances. SSA executives expressed frustration with the formal federal rulemaking process and the length of time it takes to develop and implement new rules, and asked the committee to propose a streamlined process that would be legally acceptable. While acknowledging that the Administrative Procedures Act (APA) places significant burdens on federal agencies to ensure that agency rules are developed in full view of the public and with full public participation, the committee fully supports the APA process. SSA has already adopted some modifications to its internal regulatory development process to expedite the development of new Listings, and the committee encourages SSA to continue those efforts, as well as its efforts to make the Listings more adaptable by providing generic rather than specific guidelines. Another way to respond to continuing improvements in methods of

OCR for page 89
Improving the Social Security Disability Decision Process treating and evaluating impairments is to use agency rulings (Social Security Rulings, or SSRs) to provide adjudicators with up-to-date methods of obtaining and evaluating evidence and up-to-date guidance in applying the Listings. The committee believes that these efforts, in conjunction with the establishment of a medical advisory committee, can help SSA keep the Listings up to date while maintaining the important protections embodied in the APA. Also, when medical advances or other changes necessitate a new SSR, the need to revise the affected listing itself should be assessed. RECOMMENDATION 6. SSA should continue to develop procedures for expediting development of new regulations and fully involving the public and external experts in the regulatory process governed by the Administrative Procedures Act. Agency rulings, which are published in the Federal Register and made available on SSA’s website, should continue to be used to explain and clarify the substantive rules developed in the full regulatory process. ADAPTABILITY OF THE LISTINGS Task 5 Statement: “… the committee will consider … [a]daptability of the Listings, including methods to account for variable access to health care services (including diagnostics and pharmaceuticals) in determining whether an individual’s condition meets or equals the Listings.” The criteria in some of SSA’s listings are designed to apply only if an applicant is undergoing, or has undergone, specific medical treatment(s) and such medical treatment has been unsuccessful. Most listings require evidence that specific clinical or laboratory findings persist despite treatment to meet the listing. This is especially true of listings for cardiovascular impairments. For example: 4.10 Aneurysm of aorta or major branches, due to any cause (e.g., atherosclerosis, cystic medial necrosis, Marfan syndrome, trauma), demonstrated by appropriate medically acceptable imaging, with dissection not controlled by prescribed treatment (see 4.00H6). [Emphasis added] Unfortunately, appropriate and necessary medical care and treatment for serious disorders is not readily available to everyone. For individuals who do not receive treatment, these listings may not apply. As SSA notes in the preface to the cardiovascular listings, “If you do not receive treatment, you cannot show an impairment that meets the criteria of most of these listings.”4 4 See section 4.00B.3. of Appendix 1, subpart P, 20 CFR part 404, published January 13, 2006 at 71 FR 2335.

OCR for page 89
Improving the Social Security Disability Decision Process Variable access to quality health care services throughout the country is an unfortunate fact, but it is beyond the capacity of SSA to remedy. Observation of a patient’s response to medical treatment is a standard medical practice and a legitimate way for SSA to evaluate impairment severity in its rules. The fact that all applicants may not be able to document impairment severity this way does not make it any less valuable as a method to assess impairment severity in those who can, especially given that the Listings are only a screening tool to identify obvious allowances. Ideally, individuals applying for disability benefits would be evaluated and receive the medical, vocational rehabilitation, and employment services that would enable them to resume working gainfully. Instead, in the current system, many individuals with remediable work limitations are not eligible for medical care or vocational rehabilitation until after they have completed the process of qualifying for cash benefits. At that point, they may become eligible for Medicaid if they are SSI recipients (unless they have already qualified under other criteria, such as those for the Children with Special Health Care Needs program). SSDI beneficiaries must wait for two years to be eligible for Medicare. Only then may these individuals be able to obtain the medical care they need. This requirement obviously disadvantages poor people and others without adequate health care coverage, but any unfairness is the result of the social and political system that created these inequities, not SSA’s Listings, which is meant to be the most efficient method available for easily identifying obvious allowance cases. RECOMMENDATION 7. The committee recommends against attempting to consider variable access to health care at the Listings step in determining disability. It is not strictly a medical issue that can be incorporated easily in Listings criteria. Although medical evaluation is involved in gauging the severity and functional impacts of an untreated condition, the circumstances limiting access to health care and assistive technology should be considered separately. EVALUATING MULTIPLE IMPAIRMENTS Task 6 Statement: “… the committee will consider … [m]ethods to assess and quantify the effects of multiple impairments that may not individually cross the eligibility threshold.” It is possible for claimants to have several medical conditions that together prevent them from engaging in any gainful activity, although no single impairment is of listing-level severity. SSA has a provision for con-

OCR for page 89
Improving the Social Security Disability Decision Process sidering the combined effects of multiple impairments. It is one of three circumstances in which an individual’s impairment(s) may be found “medically equivalent” in severity to a listed impairment, even though no listing is met. Finding a combination of impairments equal in severity to a listed impairment would justify an allowance at step 3. However, evaluating medical equivalence to a listing, as its name implies, is a medical judgment. It requires a medical assessment of whether the findings related to an individual’s impairments are “at least of equal medical significance to those of a listed impairment.”5 As such, it does not take into account the functional consequences of the combined impairments. In practice, determining whether multiple impairments combine to equal listing-level severity medically, i.e., considering symptoms, signs, and laboratory findings (but not age, education, and work experience) across several body systems, is very difficult and quite subjective. In some cases at least, the impairments are so different—for example, back pain, depression, and a heart condition—that combining the signs, symptoms, and laboratory tests does not provide clear guidance. Given the complexity and subjectivity of this process, and the absence of any scientific analysis of the process, it is a matter of conjecture whether adjudicators’ judgments on equivalence for combinations of impairments are valid or reliable. The committee believes that, unless SSA is able to establish scientifically the value of medical judgments on equivalence in cases involving combinations of impairments, the prudent and most practical way of deciding such cases is to assess the net functional impact that the impairments have at steps 4 and 5 of the decision process. Even where the impairments are in the same body system or otherwise related, it is difficult to determine their combined impact without reference to functional consequences. Despite the difficulty assessing medical severity of multiple impairments, it is possible that, given a sufficiently comprehensive study of common comorbidities (which might include, for example, putting a sample of cases through medical equivalency evaluations at step 3 and residual functional capacity evaluations at steps 4 and 5 and comparing the results), some recurring patterns might occur showing frequently occurring combinations of impairments that could be described in listings that addresses those combinations. Other potential approaches to assessing combinations of impairments could include more training of adjudicators using common comorbidities as examples, mandatory review by a second medical consultant, or referral to an expert group with the appropriate specialties. As discussed earlier, for children applying for SSI there are two different approaches to equivalence—a medical approach and a functional approach. 5 The rules on medical equivalence are contained in SSA’s program rules at 20 CFR 404.1526 and 416.926.

OCR for page 89
Improving the Social Security Disability Decision Process For functional equivalence in children, SSA assesses the net functional impact of all impairments combined. There is no similar process for adults, and developing such an approach may not be feasible. RECOMMENDATION 8. Generally, SSA should not try to determine the medical equivalency of multiple impairments to the Listings at step 3. Such cases should be analyzed functionally. However, research may determine that some common comorbidities, each of a certain degree of severity or more, are typically allowed after going through steps 4 and 5, and these sets of conditions could be incorporated in the Listings to expedite decisions. INTEGRATING FUNCTIONAL ASSESSMENT IN THE LISTINGS Task 7 Statement: “… the committee will consider … [a]dvisability of and methods for integrating functional assessment into the Listings.” When discussing functioning in the Listings, it is important to first define terms. The 2006 Listings use the term “function” and its different forms (e.g., functioning, functional, dysfunction, and so forth) almost 500 times. In general, the term is used in one of two distinct ways. First, there are many references to function at the body/organ level. That is, in describing how a particular organ or body part functions (e.g., “muscle function,” “labyrinthine-vestibular function,” “pulmonary function,” “ventricular dysfunction,” “cardiac dysfunction,” “myocardial function,” “function of the stoma,” “liver dysfunction,” “pancreatic dysfunction,” “renal function,” “hyperfunction of the adrenal cortex,” “major dysfunction of a joint(s)”). References such as these to organ/body-level function are components of a clinical assessment rather than an assessment of the disabling functional consequences of the disorders. In addition, the Listings contain references to functioning at the person level. The best example may be the musculoskeletal Listings, in which functional loss is introduced and defined in terms of the effect of the impairment on a person’s normal functioning. The Listings state that “… functional loss for purposes of these listings [musculoskeletal] is defined as the inability to ambulate effectively on a sustained basis for any reason….” Similarly, in the description of performing fine and gross movements, function is again defined at the person level, i.e., “… sustaining such functions as reaching, pushing, pulling, grasping, and fingering to be able to carry out activities of daily living.” It is the person-level functioning that appears to be the concern of SSA, especially to the extent that it may resemble the residual functional

OCR for page 89
Improving the Social Security Disability Decision Process capacity evaluation that SSA performs at the later stages of the sequential evaluation process for individuals whose impairments are not of listing-level severity. As discussed in Chapter 6, SSA’s concerns underlying this specific task have to do with the perceived evolution of the Listings from supposedly objective, measurable clinical criteria (including organ/body-level functioning) that can be easily and quickly applied to more subjective, ill-defined functional criteria (functioning at the person level) that are difficult and time-consuming to apply. Without a doubt, the earlier listings criteria were more clinical, shorter, and easier to apply. However, the committee is not aware of any data showing that these earlier versions of the Listings were any more accurate as a screening tool than more recent versions. And, at least on an intuitive level, it would seem that Listings criteria that measure impairment severity on a functional basis would have a better chance of correlating with functional work disability than would Listings criteria that measure impairment severity on a clinical basis. As the Listings evolved from a short list of catastrophic conditions that were intended to correlate closely with inability to work, incorporating functional criteria was a natural development. As one recent historian of the Listings notes, function has been part of them for a long time, at least from the 1960s (Cowles, 2005:2). Examples of functional criteria in the 1967 Listings are cited in Chapter 6. Recently, however, SSA has begun to incorporate function into the Listings in a more systematic way. This is clear in the 2002 musculoskeletal revision, which some see as a model for revising all the listings. The 2002 revision introduced a standardized approach to considering the functional consequences of musculoskeletal disorders, using two broad criteria, each representing the same degree of functional deficit—“inability to ambulate effectively on a sustained basis for any reason” and “inability to perform fine and gross movements effectively on a sustained basis for any reason”— as the ultimate criteria for meeting the Listings for disorders of the spine and joints, fractures of the extremities, amputation, and soft tissue injuries such as burns. As SSA noted in response to public comments expressing concern about these new functional criteria, the intent was to establish a clearer and more consistent functional standard that would “promote greater consistency in decision making.”6 The question remains whether it is possible to identify valid functional Listings criteria that correlate highly with inability to work. Although there is no global or generic functional assessment tool that could be used for all applicants, functional assessment tools have been developed and validated 6 See Revised Medical Criteria for Determination of Disability, Musculoskeletal System and Related Criteria; Final Rule. 66 FR 58023 (November 19, 2001).

OCR for page 89
Improving the Social Security Disability Decision Process for some conditions. Currently, however, functional evaluation in the Listings is not systematically based on research on such tools. RECOMMENDATION 9. SSA should undertake a comprehensive assessment of the performance of both medical and functional listings, and should consider function at the Listings step when it can be shown to correlate with inability to perform substantial gainful activity (SGA). SSA should both monitor and sponsor current research regarding the extent to which medical and functional criteria are highly correlated with limitations on performance of SGA. RESEARCH TO SUPPORT THE LISTINGS SSA has a disability research and demonstration program, currently consolidated in a unit of the Office of Disability and Income Support Programs (OSDIP). Moreover, SSA has authority to fund demonstrations affecting SSDI from the Social Security trust funds rather than the administrative budget, which comes from the discretionary budget. Currently, ODISP’s research is focused on return-to-work demonstrations. It would be appropriate for the program also to include research to improve the evidence base of the Listings and experiments to evaluate alternative listing criteria and procedures. This report gives examples of some experiments that could be done to improve the Listings. For example, a proposed revised listing could be tested against a sample of claims going through the regular process to compare results. This could include independent evaluations of each file, perhaps in conjunction with medical evaluations of the individuals (subject to human subjects guidelines), who would be allowed or denied under the current rules and thus not otherwise affected. SSA could also the test the effects on decision making of using functional assessment tools validated for use in particular conditions or body systems. Such testing would support the effort called for in Recommendation 9 to integrate function in the Listings to the extent functional assessments are found to be predictive of inability to engage in substantial gainful activity. SSA is understandably concerned about the implication of a listing change and whether it will be useful for at least several years. The development of the Listings, like any regulation potentially affecting large numbers of people, is a major undertaking that takes time. There is also a benefit to having stable criteria over time, so that once a listing is revised, it is not likely to be revised again for several years. It is possible, through the use of experimentation and pilot projects, to test new regulations before they are finally adopted and applied throughout the system.

OCR for page 89
Improving the Social Security Disability Decision Process RECOMMENDATION 10. SSA, with input from the medical advisory committee, should develop a program of research to support the disability decision process, including the Listings. The research program should include experiments that test revised listings before they are adopted. The Listing of Impairments is a screening test whose effectiveness and utility is measured by its sensitivity, specificity, predictive values, and other criteria. These in turn depend on knowledge of the validity of the test, which calls for a gold standard for comparison, and knowledge of the prevalence of listing-level disorders in the population. SSA should support research on error rates in applying the Listings, such as latent class models, which are often used when there is no gold standard measure or the results of a gold standard test are imperfect (Qu et al., 1996). SSA should also support research on the prevalence of common disorders that meet the Listings (10 impairment codes account for more than 49 percent of the applications for Social Security disability). SSA could supplement existing surveys, such as the National Health and Nutrition Examination Survey (NHANES) or the Behavioral Risk Factor Surveillance Survey. NHANES has the advantage of using mobile examination laboratories to examine the health status of survey participants directly. RECOMMENDATION 11. SSA, in conjunction with other agencies that assist persons with disabilities, should conduct a periodic, nationally representative sample survey of the population to determine the distribution and extent of severe impairments that might meet the Listings. This might be done by supplementing existing surveys. The results would be useful for program planning and for evaluating the effectiveness of the Listings and other aspects of the disability decision process. REFERENCES Albert, P.A., L.M. McShane, J.H. Shih, and the US NCI Bladder Tumor Marker Network. 2001. Latent class modeling approaches for assessing diagnostic error without a gold standard: With applications to p53 immunohistochemical assays in bladder tumors. Biometrics, 57:610-619. Albert, P.S., and L.E. Dodd. 2004. A cautionary note on robustness of latent class models for estimating diagnostic error without a gold standard. Biometrics, 60:427-435. Boelaert, M., S. Rijal, S. Regmi, R. Singh, B. Karki, D. Jacquet, F. Chappuis, L. Campino, P. Desjeux, D. Le Ray, S. Koirala, and P. Van der Stuyft. 2004. A comparative study of the2004. A comparative study of the effectiveness of diagnostic tests for visceral leishmaniasis. American Journal of Tropical Medicine Hygiene, 70(1):72-77. Cowles, A. 2005. A history of the disability listings. March 2, 2005. Available: www.ssa.gov/history/DisabilityListings.html (accessed November 3, 2005).

OCR for page 89
Improving the Social Security Disability Decision Process Ferraz, M.B., S.D. Walter, R. Heymann, and E. Atra. 1995. Sensitivity and specificity of different diagnostic criteria for Behcet’s Disease according to the latent class approach. British Journal of Rheumatology, 34(10):932-935. Hui, S.L., and X.H. Zhou. 1998. Evaluation of diagnostic tests without gold standards. Statistical Methods in Medical Research, 7:354-370. IOM and NRC (Institute of Medicine and National Research Council). 2002. The dynamics of disability: Measuring and monitoring disability for Social Security programs, edited by G.S. Wunderlich, D.P. Rice, and N.L. Amado. Washington, DC: National Academy Press. Kennedy, C. 2002. SSA’s disability determination of mental impairments: A review toward and agenda for research. In The dynamics of disability. Measuring and monitoring disability for Social Security programs, edited by G.S. Wunderlich et al. Washington, DC: National Academy Press. Pp. 241-280. Moayyedi, P., J. Duffy, and B. Delaney. 2004. New approaches to enhance the accuracy of the diagnosis of reflux disease. Gut, 53(4):55-57. NASI (National Academy of Social Insurance). 1996. Balancing security and opportunity: The challenges of disability income policy. Summary and Overview, Report of the Disability Policy Panel. Available: www.nasi.org/usr_doc/Balancing_Summary.pdf (accessed October 5, 2006). Pepe, M.S. 2003. The statistical evaluation of medical tests for classification and prediction. New York: Oxford University Press. Pepe, M.S., and T.A. Alonzo. 2001. Comparing disease screening tests when true disease status is ascertained only for screen positives. Biostatistics, 2:249-260. Pincus, H.A., C. Kennedy, S.J. Simmens, H.H. Goldman, P. Sirovatka, and S. Sharfstein. 1991. Determining disability due to mental impairment: APA’s evaluation of Social Security Administration guidelines. American Journal of Psychiatry, 148(8):1037-1043. Qu, Y., M. Tan, and M.H. Kutner. 1996. Random effects models in latent class analysis for evaluating accuracy of diagnostic test. Biometrics, 52:797-810. SSA (Social Security Administration). 2001. Research Note #13: Listing of Social Security advisory councils & commissions. Updated November 6, 2001. Available: www.ssa.gov/history/councils.html (accessed October 16, 2006). Walter, S.D., and L.M. Irwig. 1988. Estimation of test error rates, disease prevalence, and relative risk from misclassified data: A review. Journal of Clinical Epidemiology, 41:923-937. Yang, I., and M.P. Becker. 1997. Latent variable modeling of diagnostic accuracy. Biometrics, 53:948-958. Young, J., H. Bucher, P. Tschudi, P. Periat, C. Hugenschmidt, and A. Welge-Lussen. 2003. The clinical diagnosis of acute bacterial rhinosinusitis in general practice and its therapeutic consequences. Journal Clinical Epidemiology, 56(4):377-384.