3
A Framework for Research on the Disability Decision Process

Too many discussions of Social Security reform start with a set of proposals before a framework for assessing alternatives has been established. Any effort to understand why the Social Security system is the way it is, and to develop thoughtful judgments regarding potential reforms, can benefit greatly by starting from a set of principles. Principles provide a set of criteria against which particular features of a program or proposed reforms can be judged as “better” or “worse”, “well designed” or poorly designed.'' Quite simply, if any system is to be made better—however vague that goal may at first appear—there must be some standards against which the goal can be measured. In many cases, there is no simple answer, as different principles compete with each other and require compromise. Nonetheless, a set of principles provides a framework for thinking about the issue and allows us to honestly assess the inevitable trade-off in a rational and rigorous way.

(Steuerle and Bakija, 1997, p. 38)

The committee endorses the Social Security Administration's (SSA) efforts to undertake a variety of research activities related to the revision of the disability decision process. Nevertheless, research planned and currently being conducted appears to be somewhat disjointed. The committee suggests the need for an overall framework for a research plan.

Program evaluation research begins with an initial identification of a problem followed by one or more policy-related questions and a series of activities to answer the questions. Defining a problem often is, at least in part, a political process the outcomes of which may not totally flow from an assessment of available information (Berk and Rossi, 1990). Proposals for change in programs could arise because of perceived dissatisfaction with the current state of affairs; because the program may not be performing as it was meant to, costs for the program may have spiraled upwards, or the size of the program may have increased. Any or all of these complaints can lead to a decision that a problem exists.

The following policy issues or questions need to be addressed in the context of the problems associated with SSA's research design for redesigning the disability decision process:



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 11
The Social Security Administration's Disability Decision Process: A Framework for Research, Second Interim Report 3 A Framework for Research on the Disability Decision Process Too many discussions of Social Security reform start with a set of proposals before a framework for assessing alternatives has been established. Any effort to understand why the Social Security system is the way it is, and to develop thoughtful judgments regarding potential reforms, can benefit greatly by starting from a set of principles. Principles provide a set of criteria against which particular features of a program or proposed reforms can be judged as “better” or “worse”, “well designed” or poorly designed.'' Quite simply, if any system is to be made better—however vague that goal may at first appear—there must be some standards against which the goal can be measured. In many cases, there is no simple answer, as different principles compete with each other and require compromise. Nonetheless, a set of principles provides a framework for thinking about the issue and allows us to honestly assess the inevitable trade-off in a rational and rigorous way. (Steuerle and Bakija, 1997, p. 38) The committee endorses the Social Security Administration's (SSA) efforts to undertake a variety of research activities related to the revision of the disability decision process. Nevertheless, research planned and currently being conducted appears to be somewhat disjointed. The committee suggests the need for an overall framework for a research plan. Program evaluation research begins with an initial identification of a problem followed by one or more policy-related questions and a series of activities to answer the questions. Defining a problem often is, at least in part, a political process the outcomes of which may not totally flow from an assessment of available information (Berk and Rossi, 1990). Proposals for change in programs could arise because of perceived dissatisfaction with the current state of affairs; because the program may not be performing as it was meant to, costs for the program may have spiraled upwards, or the size of the program may have increased. Any or all of these complaints can lead to a decision that a problem exists. The following policy issues or questions need to be addressed in the context of the problems associated with SSA's research design for redesigning the disability decision process:

OCR for page 11
The Social Security Administration's Disability Decision Process: A Framework for Research, Second Interim Report What are the goals and purposes of the research? What are the nature, extent, and potential etiologies of the problems with the current disability decision process? Based on the problem analysis, what options might be developed to alleviate the problems? Will a proposed redesigned disability decision process be workable? Will a proposed redesigned decision process alleviate the current problems; might it create other problems? These and other questions provide an initial conceptual framework of issues and methods for a research plan to assess a proposed redesigned disability decision process as a workable solution to current problems (see Table 3-1). Table 3-1 Issues and Methods To Be Addressed in a Framework for a Research Plan for a New Disability Decision Process Question Research Steps Research Methods 1. What is the nature and extent of the problem with the disability decision process? Needs assessment research • Special surveys and analytic studies • Assembly of existing internal and external data • Satisfaction surveys • Analysis of data from studies using established evaluative criteria • Focus groups 2. What alternative solutions might address these problems? Identify alternative options Small-scale testing Field evaluation • Review and analysis of research literature • Specially targeted research • Laboratory research and pilot studies and demonstrations • Field tests • Focus groups • Process engineering assessments • National surveys 3. Will the proposed disability decision process be workable, and will it alleviate the problems? Program evaluation and transition to implementation • Clinical trials • Simulation • Evaluation studies of the proposed decision process using the established criteria • Cost-effectiveness studies • Tests of the new decision process • in selected sites

OCR for page 11
The Social Security Administration's Disability Decision Process: A Framework for Research, Second Interim Report As described in the previous chapter, SSA's goal for the research is to develop a new decision process that is simple to administer; that facilitates consistent application of rules at each decision level; that provides accurate and timely decisions; and that the public perceives as straightforward, understandable, and fair. In addition, SSA expects the new decision-making approach to focus the decision on the functional consequences of an individual's medically determinable impairment(s) (SSA, 1994a). The remainder of this chapter briefly explains what is entailed in addressing each of the questions outlined in the above table. WHAT IS THE NATURE AND EXTENT OF THE PROBLEM WITH THE DISABILITY DECISION PROCESS? This question calls for what is referred to in social science research as “needs assessment.” Such assessment of the need for redesigning the disability decision process also helps to define the problems and in turn leads to the identification of goals. Needs assessment research should be undertaken at the outset as soon as problems have been identified. Answers are needed regarding the nature, size, and distribution of the problems with the disability decision process, and the factors contributing to the problems. An adequate design of a public program, such as SSA's proposed disability decision process, and projection of its effectiveness and efficiency require as a start, solid information on the nature and extent of the problems, both perceived and real, with the current program. Research efforts involved in this step can be relatively inexpensive and simple, involving collating and assessing existing data from government reports, published and unpublished studies, special analyses, internally generated data, policy papers, results of satisfaction surveys and other selected data gathering efforts, and other documents. On the other hand, if adequate information is not available and the issue is an important one, large-scale surveys may need to be conducted and the data from the surveys analyzed. That can be expensive and lengthy (Raizen and Rossi, 1981). Complaints, whether ultimately substantiated or not, often suggest that a program should be evaluated and improved. However, in order to assess the validity of the complaints, objective evaluative criteria should be established apriori, so that the various complaints about the program can be evaluated and the program's performance can be measured. To determine if a redesigned disability decision process would lead to improvements, one or more studies need to be conducted to provide information on how the current program is working relative to the established criteria. Analysis of data from such studies would identify the gaps between performance and the goals of the program. WHAT ALTERNATIVE SOLUTIONS MIGHT ADDRESS THESE PROBLEMS? Answers to this question depend largely on how much is understood about the problems and what variables can be changed to rectify them. Review and analysis of relevant existing literature and targeted research on the various components of the program should be undertaken to understand enough about the current program and its problems to identify appropriate alternative solutions. In general, creative energies should be aimed at developing a range of

OCR for page 11
The Social Security Administration's Disability Decision Process: A Framework for Research, Second Interim Report potential options or solutions for the problems identified. Given resource and time constraints, however, priorities need to be established. Moving from identifying alternative solutions or prototypes to proposing viable solutions requires iterative small-scale testing using available knowledge, pilot testing the prototypes, and conducting simulation studies prior to refining the prototype decision process so that it will work well with real subjects. WILL A PROPOSED REDESIGNED DECISION PROCESS BE WORKABLE AND WILL IT ALLEVIATE THE PROBLEMS? Once a viable prototype is developed to replace the existing disability decision process, two questions need to be answered: (1) is the prototype workable in the real world? and (2) will it alleviate the problems identified with the current process? Following successful pilot testing of the prototype, the next step involves field evaluation. Such evaluation includes field tests and national surveys that enables the agency to make needed changes and to answer questions, such as what is the best way to decide if a person is sufficiently “disabled” to be eligible for benefits. Randomized controlled experiments and clinical trials in addition to field evaluation sometimes serve a useful purpose. Assessing if a proposed prototype can be implemented is only one part of the solution. A necessary step is to examine if a proposed redesigned decision process will produce the improvements it is meant to deliver and at what cost. Research methods that are often used include simulation, modeling, and analyses of data using information from small-scale tests, field evaluation, program evaluation, and surveys conducted in-house or by other organizations, such as the Bureau of the Census and the National Center for Health Statistics. Prospective cost-effectiveness and cost-benefit studies to measure efficiency may seem ideal intellectually, but often they are impractical in terms of costs and timeliness. Practical approaches must take into account constraints of time and available resources. Process engineering research with the aim to control costs of the proposed decision process may also be useful. At this stage of the research the proposed prototype should be tested in a real life environment and evaluated relative to the same criteria that were initially applied to assessment of the current disability decision process. The findings then should be compared with those obtained from the existing program. The various prototypes tested will have different strengths and weaknesses, and the selection of the prototype for implementation will involve, at least to some extent, political programmatic decisions concerning priorities and budget. The next chapter reviews, in the context of the framework for a research design outlined in Table 3-1, the research plan developed by SSA, the individual projects under this plan that are completed and underway, the proposed sequence and timeline for completion of the individual projects, and the implications of each project for the overall plan. The chapter also identifies the gaps to be filled with proposed additional research.