National Academies Press: OpenBook

The Social Security Administration's Disability Decision Process: A Framework for Research, Second Interim Report (1998)

Chapter: 3 A Framework for Research in the Disability Decision Process

« Previous: 2 Background
Suggested Citation:"3 A Framework for Research in the Disability Decision Process." Institute of Medicine. 1998. The Social Security Administration's Disability Decision Process: A Framework for Research, Second Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/6194.
×

3
A Framework for Research on the Disability Decision Process

Too many discussions of Social Security reform start with a set of proposals before a framework for assessing alternatives has been established. Any effort to understand why the Social Security system is the way it is, and to develop thoughtful judgments regarding potential reforms, can benefit greatly by starting from a set of principles. Principles provide a set of criteria against which particular features of a program or proposed reforms can be judged as “better” or “worse”, “well designed” or poorly designed.'' Quite simply, if any system is to be made better—however vague that goal may at first appear—there must be some standards against which the goal can be measured. In many cases, there is no simple answer, as different principles compete with each other and require compromise. Nonetheless, a set of principles provides a framework for thinking about the issue and allows us to honestly assess the inevitable trade-off in a rational and rigorous way.

(Steuerle and Bakija, 1997, p. 38)

The committee endorses the Social Security Administration's (SSA) efforts to undertake a variety of research activities related to the revision of the disability decision process. Nevertheless, research planned and currently being conducted appears to be somewhat disjointed. The committee suggests the need for an overall framework for a research plan.

Program evaluation research begins with an initial identification of a problem followed by one or more policy-related questions and a series of activities to answer the questions. Defining a problem often is, at least in part, a political process the outcomes of which may not totally flow from an assessment of available information (Berk and Rossi, 1990). Proposals for change in programs could arise because of perceived dissatisfaction with the current state of affairs; because the program may not be performing as it was meant to, costs for the program may have spiraled upwards, or the size of the program may have increased. Any or all of these complaints can lead to a decision that a problem exists.

The following policy issues or questions need to be addressed in the context of the problems associated with SSA's research design for redesigning the disability decision process:

Suggested Citation:"3 A Framework for Research in the Disability Decision Process." Institute of Medicine. 1998. The Social Security Administration's Disability Decision Process: A Framework for Research, Second Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/6194.
×
  • What are the goals and purposes of the research?

  • What are the nature, extent, and potential etiologies of the problems with the current disability decision process?

  • Based on the problem analysis, what options might be developed to alleviate the problems?

  • Will a proposed redesigned disability decision process be workable?

  • Will a proposed redesigned decision process alleviate the current problems; might it create other problems?

These and other questions provide an initial conceptual framework of issues and methods for a research plan to assess a proposed redesigned disability decision process as a workable solution to current problems (see Table 3-1).

Table 3-1 Issues and Methods To Be Addressed in a Framework for a Research Plan for a New Disability Decision Process

Question

Research Steps

Research Methods

1. What is the nature and extent of the problem with the disability decision process?

Needs assessment research

• Special surveys and analytic studies

• Assembly of existing internal and external data

• Satisfaction surveys

• Analysis of data from studies using established evaluative criteria

• Focus groups

2. What alternative solutions might address these problems?

Identify alternative options

Small-scale testing

Field evaluation

• Review and analysis of research literature

• Specially targeted research

• Laboratory research and pilot studies and demonstrations

• Field tests

• Focus groups

• Process engineering assessments

• National surveys

3. Will the proposed disability decision process be workable, and will it alleviate the problems?

Program evaluation and transition to implementation

• Clinical trials

• Simulation

• Evaluation studies of the proposed decision process using the established criteria

• Cost-effectiveness studies

• Tests of the new decision process

• in selected sites

Suggested Citation:"3 A Framework for Research in the Disability Decision Process." Institute of Medicine. 1998. The Social Security Administration's Disability Decision Process: A Framework for Research, Second Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/6194.
×

As described in the previous chapter, SSA's goal for the research is to develop a new decision process that is simple to administer; that facilitates consistent application of rules at each decision level; that provides accurate and timely decisions; and that the public perceives as straightforward, understandable, and fair. In addition, SSA expects the new decision-making approach to focus the decision on the functional consequences of an individual's medically determinable impairment(s) (SSA, 1994a).

The remainder of this chapter briefly explains what is entailed in addressing each of the questions outlined in the above table.

WHAT IS THE NATURE AND EXTENT OF THE PROBLEM WITH THE DISABILITY DECISION PROCESS?

This question calls for what is referred to in social science research as “needs assessment.” Such assessment of the need for redesigning the disability decision process also helps to define the problems and in turn leads to the identification of goals. Needs assessment research should be undertaken at the outset as soon as problems have been identified. Answers are needed regarding the nature, size, and distribution of the problems with the disability decision process, and the factors contributing to the problems. An adequate design of a public program, such as SSA's proposed disability decision process, and projection of its effectiveness and efficiency require as a start, solid information on the nature and extent of the problems, both perceived and real, with the current program. Research efforts involved in this step can be relatively inexpensive and simple, involving collating and assessing existing data from government reports, published and unpublished studies, special analyses, internally generated data, policy papers, results of satisfaction surveys and other selected data gathering efforts, and other documents. On the other hand, if adequate information is not available and the issue is an important one, large-scale surveys may need to be conducted and the data from the surveys analyzed. That can be expensive and lengthy (Raizen and Rossi, 1981).

Complaints, whether ultimately substantiated or not, often suggest that a program should be evaluated and improved. However, in order to assess the validity of the complaints, objective evaluative criteria should be established apriori, so that the various complaints about the program can be evaluated and the program's performance can be measured.

To determine if a redesigned disability decision process would lead to improvements, one or more studies need to be conducted to provide information on how the current program is working relative to the established criteria. Analysis of data from such studies would identify the gaps between performance and the goals of the program.

WHAT ALTERNATIVE SOLUTIONS MIGHT ADDRESS THESE PROBLEMS?

Answers to this question depend largely on how much is understood about the problems and what variables can be changed to rectify them. Review and analysis of relevant existing literature and targeted research on the various components of the program should be undertaken to understand enough about the current program and its problems to identify appropriate alternative solutions. In general, creative energies should be aimed at developing a range of

Suggested Citation:"3 A Framework for Research in the Disability Decision Process." Institute of Medicine. 1998. The Social Security Administration's Disability Decision Process: A Framework for Research, Second Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/6194.
×

potential options or solutions for the problems identified. Given resource and time constraints, however, priorities need to be established. Moving from identifying alternative solutions or prototypes to proposing viable solutions requires iterative small-scale testing using available knowledge, pilot testing the prototypes, and conducting simulation studies prior to refining the prototype decision process so that it will work well with real subjects.

WILL A PROPOSED REDESIGNED DECISION PROCESS BE WORKABLE AND WILL IT ALLEVIATE THE PROBLEMS?

Once a viable prototype is developed to replace the existing disability decision process, two questions need to be answered: (1) is the prototype workable in the real world? and (2) will it alleviate the problems identified with the current process? Following successful pilot testing of the prototype, the next step involves field evaluation. Such evaluation includes field tests and national surveys that enables the agency to make needed changes and to answer questions, such as what is the best way to decide if a person is sufficiently “disabled” to be eligible for benefits. Randomized controlled experiments and clinical trials in addition to field evaluation sometimes serve a useful purpose.

Assessing if a proposed prototype can be implemented is only one part of the solution. A necessary step is to examine if a proposed redesigned decision process will produce the improvements it is meant to deliver and at what cost. Research methods that are often used include simulation, modeling, and analyses of data using information from small-scale tests, field evaluation, program evaluation, and surveys conducted in-house or by other organizations, such as the Bureau of the Census and the National Center for Health Statistics. Prospective cost-effectiveness and cost-benefit studies to measure efficiency may seem ideal intellectually, but often they are impractical in terms of costs and timeliness. Practical approaches must take into account constraints of time and available resources. Process engineering research with the aim to control costs of the proposed decision process may also be useful.

At this stage of the research the proposed prototype should be tested in a real life environment and evaluated relative to the same criteria that were initially applied to assessment of the current disability decision process. The findings then should be compared with those obtained from the existing program. The various prototypes tested will have different strengths and weaknesses, and the selection of the prototype for implementation will involve, at least to some extent, political programmatic decisions concerning priorities and budget.

The next chapter reviews, in the context of the framework for a research design outlined in Table 3-1, the research plan developed by SSA, the individual projects under this plan that are completed and underway, the proposed sequence and timeline for completion of the individual projects, and the implications of each project for the overall plan. The chapter also identifies the gaps to be filled with proposed additional research.

Suggested Citation:"3 A Framework for Research in the Disability Decision Process." Institute of Medicine. 1998. The Social Security Administration's Disability Decision Process: A Framework for Research, Second Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/6194.
×
Page 11
Suggested Citation:"3 A Framework for Research in the Disability Decision Process." Institute of Medicine. 1998. The Social Security Administration's Disability Decision Process: A Framework for Research, Second Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/6194.
×
Page 12
Suggested Citation:"3 A Framework for Research in the Disability Decision Process." Institute of Medicine. 1998. The Social Security Administration's Disability Decision Process: A Framework for Research, Second Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/6194.
×
Page 13
Suggested Citation:"3 A Framework for Research in the Disability Decision Process." Institute of Medicine. 1998. The Social Security Administration's Disability Decision Process: A Framework for Research, Second Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/6194.
×
Page 14
Next: 4 SSA's Research Plan for a Redesigned Disability Decision Process »
The Social Security Administration's Disability Decision Process: A Framework for Research, Second Interim Report Get This Book
×
Buy Paperback | $47.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF
  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!