National Academies Press: OpenBook

Informing Social Security's Process for Financial Capability Determination (2016)

Chapter: 5 Methods and Measures for Assessing Financial Competence and Performance

« Previous: 4 Abilities Required to Manage and Direct the Management of Benefits
Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×

5

Methods and Measures for Assessing Financial Competence and Performance

Building on the conceptual framework of financial capability presented in the previous chapter, this chapter begins by providing an overview of the assessment of financial capability. This is followed by a description of the role and important characteristics of instruments for assessment of financial competence and performance, a review of instruments currently available, and a summary of their uses and limitations. Considerations and challenges entailed in financial capability assessment are then described.

OVERVIEW OF ASSESSMENT OF FINANCIAL CAPABILITY

The goal of financial capability assessment is to evaluate—as objectively as possible—an individual’s abilities to manage or direct the management of his or her funds in a way that routinely meets the person’s basic needs of food, shelter, and clothing. Assessment of financial capability involves the collection, integration, and interpretation of relevant information from a variety of sources. These sources of information may include

  • interviews with the individual;
  • behavioral observations of the individual;
  • formal financial capability assessment instruments (e.g., structured interviews);
  • records from physicians, psychologists, nurses, social workers, professional counselors, occupational therapists, rehabilitation counselors, and other health care professionals; and
Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×
  • communication with and from knowledgeable third parties (e.g., family members, friends, nonlicensed professionals).

Assessments can be either direct or indirect. Direct assessments are those based on a sample of the individual’s self-report or actual performance, whether it be in an office setting (i.e., interviews, observations, assessment instruments) or a real-life setting (observations). Indirect assessment refers to the collection of information from records or third parties (e.g., record review, interviews with individuals knowledgeable about the person’s financial performance).

The setting in which an assessment takes place (e.g., a natural environment or a controlled setting such as a clinician’s office) is important. As described in Chapters 1 and 4, the committee’s conceptualization of financial capability distinguishes between financial competence (i.e., the financial knowledge and financial judgment one possesses, demonstrated in a controlled [e.g., office or clinical] setting) and financial performance (i.e., one’s degree of success in handling financial demands in the context of the stresses, supports, contextual cues, resources, and opportunities in one’s actual environment). Certain independent activities of daily living, such as shopping, managing one’s finances, and arranging for transportation, are indicative of successful financial performance (Harvey et al., 2013; McKibbin et al., 2004). Direct assessment of a person’s ability to meet the financial demands of his or her everyday environment, taking into account task complexity, compensatory abilities, and environmental supports, would be optimal. However, except for social workers, case workers, and other professionals who perform home assessments or otherwise interact with clients in their natural environment, professionals generally must rely on client self-report, collateral informants such as family members, or medical records from therapists or rehabilitation counselors in making judgments about financial performance. For this reason, formal instruments that provide valid and reliable information about an individual’s financial capability could be useful in helping to inform the U.S. Social Security Administration’s (SSA’s) capability determinations.

INSTRUMENTS DESIGNED TO ASSESS FINANCIAL CAPABILITY

As part of its statement of task, SSA asked the committee to consider the use of assessment tools that could be employed in SSA’s capability determination process. Many of the available instruments assess financial competence (financial knowledge and/or financial judgment) rather than real-world financial performance. The underlying assumption governing the use of these instruments is that the lack of the basic knowledge and skills required to identify and count currency or to employ basic arithmetic

Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×

operations such as addition or subtraction will undermine a person’s success in handling the financial demands of his or her everyday environment (Marson et al., 2000; Sherod et al., 2009). Similarly, if a person is unable to make financial choices that advance his or her best interests in response to hypothetical scenarios, the assumption is that the person also will be unable to do so in real life. Instruments used to assess financial knowledge and/or financial judgment collect information directly from the person’s self-report; indirectly from collateral informants, such as family members or friends; or, increasingly, through direct observation of the ability of the person to perform calculations (Gerstenecker et al., 2015; Marson et al., 2000), work with actual currency (Marson et al., 2000), or make financial decisions in response to hypothetical scenarios (Marson et al., 2000).

Important Characteristics of Financial Capability Assessment Instruments

In its review of existing financial capability assessment instruments, the committee considered several important characteristics, including the instruments’ reliability, validity, generalizability of performance, susceptibility to reporter biases, sensitivity and specificity, administration properties, and generalization to individuals with different disorders and of diverse ethnic and cultural backgrounds. Each of these characteristics is described in turn below.

Reliability

Reliability refers to the consistency of the results obtained from an assessment instrument. If a financial capability assessment instrument does not yield consistent results, the results are unreliable and cannot be interpreted meaningfully. Four types of reliability generally are assessed:

  • test-retest—consistency of test scores over time (stability, temporal consistency);
  • interrater—consistency of ratings or test scores across independent judges;
  • parallel- or alternative-forms—consistency of scores across different forms of the test (stability and equivalence); and
  • internal consistency—coherence of different items intended to measure the same thing within the test (homogeneity), a special case of which is split-half reliability, where scores on two halves of a single test are compared, and the result of this comparison can be converted into an index of reliability.

A number of factors can affect the reliability of a test’s scores. These include the time elapsed between two administrations of the test, which

Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×

affects test-retest and alternative-forms reliability; the scales of measurement applied (e.g., nominal versus interval); and similarity of content and expectations of subjects regarding different elements of the test, which affect alternative-forms and internal consistency (including split-half) reliability. Reliability also is affected by changes in subjects that occur over time and are introduced by physical ailments, emotional problems, or the subject’s environment, as well as test-based factors such as poor test instructions, subjective scoring, and guessing. It is important to note that a test can generate reliable scores in one context and not in another, and that the inferences that can be drawn from different estimates of reliability are not interchangeable (Geisinger, 2013; IOM, 2015).

Validity

Validity is another important characteristic of assessment instruments. Historically, three primary types of validity have been recognized (Sattler, 2014; Sireci and Sukin, 2013):

  • construct validity—the degree to which an instrument measures the theoretical concept it is designed to measure;
  • content validity—the degree to which the instrument’s content (typically reviewed by experts in the field) represents the targeted subject matter and supports use of the instrument for its intended purposes; and
  • criterion-related validity—the degree to which the instrument’s score correlates with other measurable, reliable, and relevant variables (i.e., criteria) thought to measure the same construct, which includes concurrent validity (high correlation with existing validated measures) and predictive validity (the extent to which an instrument’s scores predict scores or outcomes on some criterion measure, such as future financial performance).

Generalizability of Performance

As previously noted, performance on a particular financial capability assessment instrument may differ in a laboratory or office setting and in the real world. For example, one might demonstrate the ability to count currency or make change for a purchase in a quiet clinical setting only to have difficulty in the real world at a busy supermarket. In such cases, performance on the instrument in a controlled setting cannot be generalized to performance in an individual’s actual environment. If, as discussed in Chapter 4, financial performance is the most important component of financial capability for the purpose of determining the need for a representative

Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×

payee, generalizability of performance is a critical property for a financial capability assessment instrument.

Susceptibility to Reporter Biases

Another characteristic of assessment instruments is susceptibility to reporter biases. To the extent that an instrument relies on information reported by the subject or a third-party informant, it is open to reporter biases. Persons can be poor reporters of their own financial capacities by virtue of having a memory or other cognitive impairment related to their condition (e.g., Alzheimer’s dementia, schizophrenia), being unaware of their deficits related to brain injury (anosognosia), or minimizing their deficits for secondary gain (e.g., desiring autonomy in decision making). Third-party informants who know the person well can be helpful if they have ample opportunity to observe him or her in a variety of real-world situations, have the cognitive and psychological ability to make a proper assessment, and are motivated to convey accurate information. As discussed later in this chapter, however, not all informants are equally good reporters of financial performance, nor are all health care providers in a position to render accurate judgments about financial capability, given their lack of training in this area and their limited time and opportunities to assess financial capability in the individuals they see.

Sensitivity and Specificity

To be effective, any instrument used to determine deficits in financial capability needs to identify correctly a high percentage of persons who are truly impaired (sensitivity) while correctly excluding a high percentage of persons who are not impaired (specificity). For example, an instrument with a sensitivity of 85 percent and a specificity of 80 percent would correctly identify 85 percent of true cases of impairment but would miss 15 percent of true cases; and while 80 percent of nonimpaired cases would be correctly identified, 20 percent of cases would be falsely identified as impaired. It is important to note, however, that sensitivity and specificity are merely properties of assessment instruments. To determine the utility of an instrument, one needs to assess its true positive predictive value, which depends on its sensitivity and the actual base rate of impairment in the population, as well as its negative predictive value, which depends on its specificity and the population base rate.

Administration Properties

Ease and time of administration are important characteristics of any instrument used to determine financial knowledge, financial judgment, or

Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×

financial performance. Instruments that take too long to administer may be impractical in clinical settings, where time is limited for practitioners and their staff. For some disorders, neuropsychological impairments that are characteristic of the disorder itself may mean that administration of even a relatively straightforward instrument may take a substantial amount of time. For example, the Financial Capacity Instrument (FCI) (Griffith et al., 2003; Marson et al., 2000) can take 40 to 50 minutes or longer to administer in a person with Alzheimer’s disease. Other important administration properties include the nature and amount of training required to learn how to administer and score the test; whether the test relies on individual self-report or collateral information or is clinician rated; and how structured the instrument is (semistructured interviews require more training and judgment on the part of the interviewer relative to more structured instruments). Instruments whose administration requires trained individuals need to have clear scoring guidelines; the degree of structure of the scoring is another administration property.

Generalizability to Individuals with Different Disorders and of Diverse Ethnic and Cultural Backgrounds

Another characteristic that needs to be evaluated for each available instrument is external validity, or generalizability to individuals with different types of disorders and of diverse ethnic and cultural backgrounds. Given the diversity of current SSA beneficiaries with respect to cultural and language background as well as underlying medical, neurological, and neuropsychiatric conditions, it is important that any instruments used to assess financial capability be broadly generalizable.

Overview of Instruments Available for Assessing Financial Capability

The committee identified and reviewed eight instruments developed specifically to evaluate aspects of financial capability. Annex Table 5-1 at the end of this chapter summarizes information about each of these instruments, including its psychometric properties. Given the conceptual framework of financial capability described in Chapter 4, the committee examined each instrument in terms of the components of financial capability it is used to evaluate (as summarized in Table 5-1).1 It should be noted that the committee is unaware of any instruments designed to assess individuals’ ability to direct someone else to manage their funds.

___________________

1 Because the instruments were developed independently of the conceptual framework proposed in Chapter 4, the committee inferred the components of the framework that may be assessed by the instruments from the type of data collected by each.

Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×

TABLE 5-1 Components of Financial Capability Measured by Assessment Instruments

Component of Financial Capability Assessment Instruments
Financial Performance Clinician Assessment of Financial Incapability (CAFI) (Black et al., 2014)
Financial Incapability Structured Clinical Assessment done Longitudinally (FISCAL) (Lazar et al., 2015)
Money Mismanagement Measure (MMM) (Conrad et al., 2006)
Timeline Historical Review of Income and Financial Transactions (THRIFT) (Black et al., 2013)
Financial Knowledge Financial Capacity Assessment Instrument (FCAI) (Kershaw and Webber, 2008)
Financial Capacity Instrument (FCI) (Griffith et al., 2003; Marson et al., 2000)
Financial Capacity Instrument-Short Form (FCI-SF) (Gerstenecker et al., 2015)
Financial Judgment Assessment of Capacity for Everyday Decision-making (ACED) (Lai and Karlawish, 2007)
CAFI
FCAI
FCI
FISCAL
MMM
THRIFT

Assessment of Financial Performance

In contrast to financial knowledge and financial judgment, which can be measured in an office or clinical setting, financial performance represents the actual, real-world performance (or success) of an individual in handling financial demands in the context of the stresses, supports, contextual cues, and resources in his or her actual environment. Of the instruments reviewed by the committee, four appear to measure financial performance. The Financial Incapability Structured Clinical Assessment done Longitudinally (FISCAL) (Lazar et al., 2015) is focused on financial performance—particularly whether individuals have been spending their funds in a way that does not meet their basic needs or results in harm to them. The FISCAL incorporates information from the beneficiary’s medical records, health care providers, and/or family members and information

Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×

from a semistructured interview with the beneficiary (Rosen et al., 2015). The assessors reconcile any discrepancies among different sources of information, which helps offset the limitations of self-reported information (e.g., misrepresentation of one’s circumstances or behavior). The FISCAL explicitly allows (but does not require) the use of contextual information to inform the determination of (what the instrument refers to as) “capability.” The Clinician Assessment of Financial Incapability (CAFI) (Black et al., 2014) also appears to tap into financial performance, as it asks clinicians to rate how well the individual has been meeting his or her basic needs. Both instruments have showed good psychometric properties in relation to other assessment methods. The Money Mismanagement Measure (MMM) (Conrad et al., 2006), which was developed to help assess the effectiveness of representative payee systems, also relies on self-report to assess financial judgment and performance. Similarly, the Timeline Historical Review of Income and Financial Transactions (THRIFT) (Black et al., 2013) uses a timeline follow-back method to elicit information from individuals about their income, “in-kind” payments or exchanges (e.g., letting a friend stay in one’s apartment in return for the friend’s paying for food), expenditures, and debts over the past month, which can be used to evaluate the individuals’ financial performance.

Assessment of Financial Knowledge

As described in Chapter 4, financial knowledge is possession of the declarative and procedural knowledge required to manage one’s finances, including, for example, the concept of money, values of currency, making change, check writing, use of ATMs (automated teller machines), and online banking procedures. Three of the instruments listed in Table 5-1 can be used to assess financial knowledge, as measured by structured or semistructured questions that ask the individual to demonstrate knowledge or skills needed for managing finances or through observation of the individual carrying out financial or money-related tasks. Assessment of the kinds of knowledge measured by these instruments appears to require at least some level of clinical training, although a nonclinician arguably could be trained to administer them. Of the instruments reviewed, the Financial Capacity Instrument-Short Form (FCI-SF) (Gerstenecker et al., 2015) requires the least time to administer (approximately 15 minutes), as it is focused fairly narrowly on financial knowledge and calculations. The FCI is significantly longer (its administration to an individual with Alzheimer’s disease, for example, can take 40 to 50 minutes), as it taps a broader range of financial skills, as well as financial judgment. The Financial Capacity Assessment Instrument (FCAI) also can be used to assess financial knowledge. The time required for its administration is not described (Kershaw and Webber, 2008).

Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×

Assessment of Financial Judgment

As discussed in Chapter 4, financial judgment is possession of the abilities needed to make financial decisions and choices that serve the individual’s best interests. The FCI, which has been used primarily to assess financial knowledge in older adults with cognitive impairment, includes several items that can be used to assess financial judgment. These items are focused on the ability to detect mail and telephone fraud, schemes commonly used to exploit older adults. The FCAI (Kershaw and Webber, 2008) also purports to encompass financial judgment as one of the assessed domains; however, the committee found it difficult to determine whether this is actually the case. The FISCAL may implicitly measure financial judgment in that it provides a means of assessing whether the individual has been making decisions that serve his or her best interests, but it does not directly assess individuals’ financial judgment, for example, in terms of their ability to make appropriate decisions on hypothetical scenarios. As indicated previously, it is more focused on actual performance. The MMM purports to measure financial judgment; as a self-report instrument, however, it is suited more for use as a screening tool than as a definitive measure of an individual’s ability to make appropriate financial judgments. The THRIFT, through its use of the calendar timeline follow-back method, may tap into financial judgment (e.g., an individual may recall spending that was not in his or her best interests). However, it is difficult to determine how well this instrument can detect people’s ability to protect their own best interests if they do not report any problems with their spending. The CAFI, designed as a clinician-rated instrument, assesses judgment through questions intended to determine whether an individual is at risk for financial victimization. Again, however, without direct observation of the individual, this instrument relies on the clinician’s perspective and (in turn) on the individual’s self-report.

The Assessment of Capacity for Everyday Decision-making (ACED) (Lai and Karlawish, 2007; Lai et al., 2008) was designed to help clinicians evaluate older adults’ everyday decision-making capacity—particularly those with cognitive impairment whose ability to function independently at home may be in question. Issues commonly of concern in this population include the ability to manage one’s finances. Therefore, the authors developed a specific version of the ACED’s structured questionnaire to assess financial judgment—and possibly financial performance. Its format is modeled on the MacArthur tools developed by Appelbaum and Grisso to assess decision-making capacity for treatment (and later, for clinical research) (Lai et al., 2008). It has not been tested in individuals with serious mental illnesses such as schizophrenia.

Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×

General Functional Capacity Assessment Instruments with Financial Knowledge Subscales

A number of existing instruments are designed to assess functional capacity across a broad range of domains in adults with such conditions as schizophrenia, bipolar disorder, schizoaffective disorder, and major neurocognitive disorders (attributable to such conditions as Alzheimer’s disease, Parkinson’s dementia, diffuse Lewy body disease, frontotemporal dementia, head trauma, and vascular dementia)—conditions that may be present in Social Security beneficiaries being evaluated for capability. In reviewing the literature, the committee found a number of instruments designed to assess functional domains that may include or overlap with financial capability. Although these instruments were not designed specifically to assess financial performance and competence, several of them have validated financial knowledge subscales that can be used to assess such skills as identifying and counting currency, writing a check, and balancing a checkbook.

Two such instruments commonly used are the Independent Living Scales (ILS) (Loeb, 1996) and the Kohlman Evaluation of Living Skills (KELS) (Thomson, 1992). Both have established reliability and validity. The ILS is a standardized, individually administered assessment of adults’ competence in independent activities of daily living, including managing money. The money management subscale assesses the individual’s ability to count money, perform monetary calculations, pay bills, and take precautions with money. The items are both knowledge focused (e.g., inquiring about the cost of a loaf of bread) and performance focused (e.g., observing the person writing a check) within an office or clinical setting. Although originally developed for use in older adults, the ILS can be used with a variety of clinical populations. The money management subscale has good reliability (0.88), and its criterion and concurrent validity have been established. The KELS is a standardized, individually administered assessment of a person’s possession of basic living skills, including money management. Originally developed for use in short-term psychiatric facilities, it can be used across populations. Specific items include making change, filling out bank forms, and paying bills.

More recently, the financial subscale of the UCSD Performance-based Skills Assessment (UPSA) (Patterson et al., 2001) and UPSA Brief (UPSA-B) (Mausbach et al., 2010, 2011) has been used to assess individuals with bipolar and schizoaffective disorders. This subscale enables direct assessment of such functional tasks as counting currency, making change for a purchase, and balancing a checkbook. Its advantage is that it has been shown to correspond to neurocognitive testing, has good discriminative validity, and has been used with different ethnic and cultural groups (Mausbach et al., 2010, 2011). Like the ILS and KELS, however, the UPSA subscale

Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×

provides information only on financial competence as demonstrated in a test environment and does not capture real-world performance.

The functional tests of the financial subscale of the Everyday Functioning Battery (Heaton et al., 2004) are more advanced than the UPSA subscale. First developed to assess the financial skills of adults diagnosed with HIV infection, it was used more recently in the Valero study for persons with schizophrenia (Harvey et al., 2011). This instrument assesses higher-order functional skills such as preparing checks and bank deposits to pay bills, organizing specific payments, and setting aside specific amounts of money in a bank account.

The financial subscale of the information version of the Independent Living Skills Survey (ILSS) (Wallace et al., 2000) has been widely used among informants of persons with schizophrenia and other neuropsychiatric disorders. Questions focus on real-world performance as measured by the frequency of successfully performing acts requiring the assessed skills in the past 30 days; scores range from “never” to “always” (on a 0-4 point scale). In addition, the extent to which the individual performed the skilled acts without prompting is rated. Examples of the 10 items of the financial subscale include (1) paying bills for rent or utilities, (2) budgeting money and planning where money should be spent, (3) making a deposit or withdrawal from a bank, (4) cashing a paycheck or Supplemental Security Income (SSI) check, (5) purchasing essential items before luxury items, and (6) purchasing prescribed medication.

Because all of the above instruments were designed to assess functional capacity across a broad range of domains, none permits the comprehensive assessment of financial competence and performance enabled by instruments designed specifically for that purpose, despite having validated financial subscales. It should also be noted that, as with the latter instruments, none of these subscales directly taps the ability of individuals to manage their benefits directly, although the ILSS includes an item for informants to rate the degree to which a person contacted someone responsible for financial support and asked that individual relevant questions.

USES AND LIMITATIONS OF AVAILABLE ASSESSMENT INSTRUMENTS

The committee concludes that at present, because of their limitations, no single assessment instrument can be recommended for routine use with beneficiaries. Although the characteristics of the instruments vary, as a group they lack data demonstrating one or more of the following characteristics: construct and content validity for assessment of the ability to manage benefits to meet basic needs; ability to measure financial performance in a real-world (as opposed to office or clinical) setting; efficiency in administration;

Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×

ability to be administered reliably by a range of professionals; confirmation of reliability and validity when used by persons other than their developers; and validity for use with individuals with different disorders and of diverse ethnic and cultural backgrounds. Several of the instruments are also limited by reliance on self-report by the person being assessed.

Two of these limitations warrant emphasis. First, most of the available instruments were not developed specifically to evaluate the ability of SSA beneficiaries to manage their benefits to meet their basic needs. Hence, they may include items that relate to other financial functions and yield data that are not responsive to the question at hand. A particular risk in such situations is that beneficiaries will be deemed incapable based on assessment of tasks not directly related to the management of their benefits, or will have relevant impairments overlooked because they are not the focus of the instrument being used.

Second, most of the instruments were designed for use with individuals with specific disorders and therefore, without additional validation, cannot be generalized for use with individuals with different disorders or of diverse ethnic and cultural backgrounds. Given the diversity of conditions and paths that may lead to deficits in financial performance, it is reasonable to question whether a test developed for one condition is equally valid for another. For example, is a test developed for use in people with Alzheimer’s disease or other neurocognitive conditions equally valid for persons with schizophrenia, bipolar disorder, major depression, or substance abuse? Moreover, many measures developed for U.S.-born, English-speaking individuals may not be relevant or appropriate for non-native English speakers or individuals of different cultural groups. As noted above, it is important that any instruments used to assess financial capability be broadly generalizable because current SSA beneficiaries represent a highly diverse population with respect to culture, language, and underlying conditions. The committee found little evidence that any of the available instruments has to date been sufficiently tested or validated in diverse populations.

When using these instruments, one must also take assessor bias into consideration. The assessment of financial judgment, in particular, can be affected by value judgments embedded in the design and scoring of the instrument being used. Bias can be reflected, for example, in the instrument’s cut-off scores. Likewise, different perceptions of what is valuable can impact the assessment of financial judgment and performance. For instance, an individual may value setting aside a weekly allotment for a hair appointment even if it means having less money with which to buy food, while the assessor may deem weekly hair appointments excessive and unnecessary if they mean the person will not have enough food.

Although the committee currently cannot recommend any of the available instruments for routine use, ongoing study of existing instruments

Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×

may in the future demonstrate their reliability and validity for assessments of beneficiaries’ capability to manage or direct the management of their benefits. Even given the limitations of current instruments, however, the committee recognizes that individual clinicians or other assessors may find one or another instrument (particularly when validated for the population of which the person being assessed is a member) to be helpful in informing their judgments about individuals’ capability.

CONSIDERATIONS AND CHALLENGES IN ASSESSMENT OF FINANCIAL COMPETENCE AND PERFORMANCE

Sources of Evidence

As noted at the beginning of this chapter, assessment of financial capability may be direct and/or indirect. The strength of direct assessment is that it captures the individual’s actual ability in a given setting, but its primary limitation is that some capabilities (e.g., judgment, pursuit of one’s best interests) cannot be assessed by direct observation. Direct assessment methods are subject to a number of other limitations as well. With respect to self-reported information, it is well established that people often are poor reporters of their true functional status and tend to overestimate their actual abilities (Bowie et al., 2007; Gould et al., 2015). Respondents’ abilities to self-report accurately depend on a number of factors, including what they are reporting (the specific domain of functioning), whether the function is observable to others, and what health conditions the respondent may have that could affect reporting (i.e., conditions affecting insight or cognition). Direct observations also may be affected by observer bias or the setting in which the assessment takes place. For example, math tests in a formal, office setting may make some people anxious, which may result in underperformance in that setting.

Indirect assessment entails record review and/or collection of collateral information from third parties. Collateral reports from individuals knowledgeable about the beneficiary’s financial performance in meeting his or her basic needs are especially useful when individuals, including those with significant psychiatric or cognitive disorders, cannot accurately provide direct information about their financial capability (e.g., they may provide inaccurate self-assessment or be unable to participate in direct assessment). Collateral informants (such as family members, neighbors, members of the clergy, and others who interact frequently with the beneficiary) can provide information based on their observations of the individual’s financial knowledge, judgment, and performance in the real world. Recent work, however, has found that the quality of information provided by collateral informants varies (Kershaw and Webber, 2008; Sabbag et al., 2011). Some individuals

Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×

may lack involved family members or friends who can provide reliable information about their financial performance. Some informants do not have the opportunity to observe financial performance (e.g., the person’s making a purchase), while others spend insufficient time with the individual to assess his or her performance accurately. In addition, some collateral informants may under- or overestimate the individual’s financial abilities. A relative might underestimate or underreport an individual’s financial abilities in the hope of gaining access to the person’s funds or overestimate the person’s abilities so as not to alienate him or her.

Medical professionals often are asked to render judgments about financial capability (see, e.g., SSA’s Form 787 in Appendix C). Medical professionals are trained diagnosticians, but as discussed in Chapter 4, diagnosis alone is seldom sufficient for making a judgment about financial capability. In addition, the majority of medical professionals lack specific training in how best to assess financial capability (Widera et al., 2011). Interviews with the individual during a clinical encounter may lead to inaccurate judgments about his or her financial capability. There is ample evidence that for many providers who do not know the client well, biased self-reports by beneficiaries may lead to inaccurate judgments (Loewenstein et al., 2001; Marson et al., 2006; Sabbag et al., 2011). Because diagnosis and medical evidence are less important than actual knowledge of a person’s financial capabilities, medical professionals, including consultative examiners, who lack current information about the individual’s real-world financial performance, who do not know the person well, or who lack access to good collateral informants or relevant records may not provide the most useful information to SSA about the person’s financial capability.

Many other health and social service professionals, such as social workers, occupational therapists, caseworkers, and rehabilitation counselors, work regularly with clients and their families and therefore may have superior information about the client’s financial knowledge and skills, financial judgment, and available environmental supports. Importantly, professionals who work more directly and regularly with clients can assess the consistency of their abilities to meet financial demands in the real world. Financial performance in real-world situations can best be captured by informants who have current knowledge of and experience with how the individual functions in his or her actual environment and have sufficient opportunities to observe the individual in that environment.

Social workers, for example, provide health, mental health, and substance abuse services in such positions as case manager, psychotherapist, rehabilitation counselor, medical social worker, behavioral analyst, and counselor. Social workers are employed in various service settings, such as primary care, specialty mental health care, community clinics, rehabilitation and recovery services, subsidized housing programs, and skilled nursing

Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×

facilities that may give them access to direct observations of beneficiaries. Other relevantly trained professionals, such as nurses, physical therapists, occupational therapists, marriage and family counselors, and members of the clergy, often have direct knowledge of how individuals with whom they have regular contact function in their actual environment, as well as the supports they may have and the challenges they may face.

Given the potential difficulties with the use of medical evidence to inform determinations of financial capability, the committee endorses SSA’s current requirement to consider what it calls “lay evidence” as part of its capability determination process. However, the committee notes that the strength of such evidence can vary depending on its source and that professionals with knowledge of individuals’ performance may have insights beyond those attainable from nonprofessional informants (e.g., family members, neighbors). For this reason, it is important for SSA to distinguish between the weight given to “lay evidence” that comes from professionals (e.g., social workers, professional counselors, clergy) and that acquired from relatives, friends, landlords, and other untrained individuals.

Strength of Evidence

To help address the concerns outlined in the previous section, it is important that evidence of an individual’s financial capability specify how well and for how long the informant has known the individual and the nature of their relationship.2 It is also important to specify the extent to which (1) the informant’s judgment is based on observed behavior; (2) the informant’s judgment is based on the individual’s self-report; (3) the informant’s judgment is based on information from collateral informants, and the perceived quality of these informants; and (4) in the case of professionals, the judgment is based on the individual’s medical record and the assessments of other health care professionals (including other physicians, psychologists, social workers, and nurses). Such specification of the basis for the evidence provided will allow for greater understanding of the quality of the evidence as support for a judgment regarding financial capability.

In addition, because most informants, including professionals, are not trained specifically in assessment of financial competence and performance, they would benefit from robust direction as to the type of information that is helpful in making a determination of financial capability. Providing such detailed guidance to professional and lay informants could be expected to improve the strength and quality of the evidence they provide. SSA recognizes that a face-to-face interview with the beneficiary is valuable for

___________________

2 This would include, to the extent possible, the beneficiary’s perspective on the relationship as well as the informant’s.

Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×

determining capability and currently provides field office employees with sample questions to help guide them through the areas they may want to explore in an assessment of the beneficiary’s capability (see Box 2-2 in Chapter 2). This same type of guidance would be helpful not just for SSA employees but for anyone being asked to provide information about an individual’s financial performance. The FISCAL provides another source of questions that could be used to guide informants in acquiring and providing information about financial performance.

Evidence of financial competence may be needed to inform capability determinations when evidence of a beneficiary’s real-world financial performance is very limited or unavailable, either because the person has had no funds to manage recently or because no reliable informant with such knowledge can be identified. Evidence of financial competence also could help to corroborate, refute, or explain evidence acquired of beneficiaries’ financial performance. As with evidence of financial performance, detailed guidance and sample questions (see Box 5-1), with requests that the basis for informants’ answers be specified, would assist informants in providing relevant information about beneficiaries’ financial competence (knowledge and judgment).

The Need for Periodic Reassessment

An additional, important consideration emerged from the committee’s review of methods and measures for assessing financial capability. As discussed in Chapter 4, many psychiatric and cognitive conditions are

Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×

characterized by progressive or fluctuating changes over time in the presence, severity, and nature of symptoms. Such changes make it difficult to assess capability as a static trait. (See Chapter 4 for a detailed discussion of this issue.) Without some mechanism for periodic reassessment, therefore, beneficiaries with fluctuating, deteriorating, or improving financial capability are more likely to be misclassified relative to those with more stable conditions. One possible approach to addressing this issue would be to incorporate reassessments of financial capability into the current process for continuing disability reviews (CDRs). For disability beneficiaries, SSA procedures call for periodic CDRs. Yet while CDRs provide a prime opportunity for capability (re)assessments, their purpose is to identify any changes (improvements) in the medical basis for beneficiaries’ disability award. Thus, even if the CDRs were to occur on schedule, they would not fully serve the purpose of reassessment of financial capability.3 SSA could apply the same principle used in the CDR process to develop an analogous process for recognizing and responding to changes in capability over time. Reassessments initially could be targeted toward (1) beneficiaries who had been determined to be incapable but who might improve over time as their condition or environmental supports changed; and (2) beneficiaries who, although capable, were at risk for becoming incapable as their condition progressed or their environment changed. As screening criteria or other systematic methods for identifying people at high risk for financial incapability were developed, they might be used to broaden the target population for periodic reassessment.

In addition, beneficiaries, family members, representative payees, and professionals who were likely to come into contact with beneficiaries could be alerted systematically to notify SSA if they believed that beneficiaries’ capability had changed so as to warrant redetermination. SSA might also implement a process to survey payees and/or beneficiaries periodically, similar to that of the U.S. Office of Personnel Management, integrating screening questions that could trigger the need to further investigate the beneficiary’s financial capability.

___________________

3 For example, there are beneficiaries whose condition precludes their ability for substantial gainful activity but does not (yet) affect their financial competence or performance. When such a condition invariably will worsen, a CDR is required only every 5-7 years because the individual is not expected to regain the ability to work. As some of the conditions worsen, however, they may affect the individual’s financial capability. Such cases are among those that are important for SSA to reevaluate.

Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×

SUMMARY

The chapter has provided a general overview of the uses and limitations of direct and indirect methods for assessing financial competence and financial performance. Of particular note are concerns about the reliability of self-reported information (e.g., due to an individual’s lack of awareness of his or her impairment or deliberate efforts to conceal it) and of information about individuals’ financial performance that is provided by third-party informants. Some informants lack the opportunity to observe financial performance, while others spend insufficient time with the individual to assess his or her performance accurately. Informants also may under- or overestimate the individual’s financial abilities for a variety of reasons. In addition, most informants, including medical and nonmedical professionals, are not trained specifically in the assessment of financial performance and would benefit from robust direction as to the type of performance information that is helpful to SSA in making a determination of financial capability.

In principle, assessment instruments could be helpful to medical and other professionals in gathering evidence of beneficiaries’ financial performance. However, half of the instruments identified by the committee are designed to assess financial competence in an office or clinical setting. Although four of the instruments appear to measure financial performance in a real-world setting, most rely primarily on self-reported behavior. While some of these instruments show good psychometric properties in relation to other assessment methods, sufficient data on reliability and validity across populations are not yet available to warrant recommending their routine use.

Although the committee currently cannot recommend any of the available instruments for routine use, ongoing study of existing instruments may in the future demonstrate their reliability and validity for assessment of beneficiaries’ capability to manage or direct the management of their benefits. Even given the limitations of current instruments, however, the committee recognizes that individual clinicians or other assessors may find one or another instrument (particularly when validated for the population of which the person being assessed is a member) to be helpful in informing their judgments about individuals’ capability.

REFERENCES

Black, A. C., K. L. Serowik, K. M. Ablondi, and M. I. Rosen. 2013. Timeline historical review of income and financial transactions: A reliable assessment of personal finances. The Journal of Nervous and Mental Disease 201(1):56-59.

Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×

Black, A. C., T. J. McMahon, R. A. Rosenheck, S. A. Ball, R. K. Ries, D. Ames, and M. I. Rosen. 2014. Development of the Clinician Assessment of Financial Incapability (CAFI). Psychiatry Research 215(3):784-789.

Bowie, C. R., E. W. Twamley, H. Anderson, B. Halpern, T. L. Patterson, and P. D. Harvey. 2007. Self-assessment of functional status in schizophrenia. Journal of Psychiatric Research 41(12):1012-1018.

Conrad, K. J., M. D. Matters, D. J. Luchins, P. Hanrahan, D. L. Quasius, and G. Lutz. 2006. Development of a money mismanagement measure and cross-validation due to suspected range restriction. Journal of Applied Measurement 7(2):206-224.

Geisinger, K. F. 2013. Reliability. In APA handbook of testing and assessment in psychology, Vol. 1, edited by K. F. Geisinger, B. A. Bracken, J. F. Carlson, J. C. Hansen, N. R. Kuncel, S. P. Reise, and M. C. Rodriguez. Washington, DC: American Psychological Association. Pp. 21-42.

Gerstenecker, A., A. Eakin, K. Triebel, R. Martin, D. Swenson-Dravis, R. C. Petersen, and D. Marson. 2015. Age and education corrected older adult normative data for a short form version of the financial capacity instrument. Psychological Assessment [Epub ahead of print].

Gould, F., L. S. McGuire, D. Durand, S. Sabbag, C. Larrauri, T. L. Patterson, E. W. Twamley, and P. D. Harvey. 2015. Self-assessment in schizophrenia: Accuracy of evaluation of cognition and everyday functioning. Neuropsychology 29(5):675-682.

Griffith, H. R., K. Belue, A. Sicola, S. Krzywanski, E. Zamrini, L. Harrell, and D. C. Marson. 2003. Impaired financial abilities in mild cognitive impairment: A direct assessment approach. Neurology 60(3):449-457.

Harvey, P. D., T. Raykov, E. W. Twamley, L. Vella, R. K. Heaton, and T. L. Patterson. 2011. Validating the measurement of real-world functional outcomes: Phase I results of the VALERO study. The American Journal of Psychiatry 168(11):1195-1201.

Harvey, P. D., L. Stone, D. Lowenstein, S. J. Czaja, R. K. Heaton, E. W. Twamley, and T. L. Patterson. 2013. The convergence between self-reports and observer ratings of financial skills and direct assessment of financial capabilities in patients with schizophrenia: More detail is not always better. Schizophrenia Research 147(1):86-90.

Heaton, R. K., T. D. Marcotte, M. Rivera Mindt, J. Sadek, D. J. Moore, H. Bentley, J. A. McCutchan, C. Reicks, and I. Grant. 2004. The impact of HIV-associated neuropsychological impairment on everyday functioning. Journal of the International Neuropsychological Society 10(3):317-331.

IOM (Institute of Medicine). 2015. Psychological testing in the service of disability determination. Washington, DC: The National Academies Press.

Kershaw, M. M., and L. S. Webber. 2008. Assessment of financial competence. Psychiatry, Psychology and Law 15(1):40-55.

Lai, J. M., and J. Karlawish. 2007. Assessing the capacity to make everyday decisions: A guide for clinicians and an agenda for future research. The American Journal of Geriatric Psychiatry 15(2):101-111.

Lai, J. M., T. M. Gill, L. M. Cooney, E. H. Bradley, K. A. Hawkins, and J. H. Karlawish. 2008. Everyday decision-making ability in older persons with cognitive impairment. The American Journal of Geriatric Psychiatry 16(8):693-696.

Lazar, C. M., A. C. Black, T. J. McMahon, R. A. Rosenheck, R. Ries, D. Ames, and M. I. Rosen. 2015. All-data approach to assessing financial capability in people with psychiatric disabilities. Psychological Assessment [Epub ahead of print].

Loeb, P. 1996. Independent living scales. San Antonio, TX: Psychological Corporation.

Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×

Loewenstein, D. A., S. Argüelles, M. Bravo, R. Q. Freeman, T. Argüelles, A. Acevedo, and C. Eisdorfer. 2001. Caregivers’ judgments of the functional abilities of the Alzheimer’s disease patient: A comparison of proxy reports and objective measures. The Journals of Gerontology. Series B, Psychological Sciences and Social Sciences 56(2):78-84.

Marson, D. C., S. M. Sawrie, S. Snyder, B. McInturff, T. Stalvey, A. Boothe, T. Aldridge, A. Chatterjee, and L. E. Harrell. 2000. Assessing financial capacity in patients with Alzheimer disease: A conceptual model and prototype instrument. Archives of Neurology 57(6):877-884.

Marson, D. C., R. Savage, and J. Phillips. 2006. Financial capacity in persons with schizophrenia and serious mental illness: Clinical and research ethics aspects. Schizophrenia Bulletin 32(1):81-91.

Martin, R. C., K. Triebel, R. E. Kennedy, A. P. Nicholas, R. L. Watts, N. Stover, M. Brandon, and D. C. Marson. 2013. Impaired financial abilities in Parkinson’s disease patients with mild cognitive impairment and dementia. Parkinsonism & Related Disorders 19(11):986-990.

Mausbach, B. T., P. D. Harvey, A. E. Pulver, C. A. Depp, P. S. Wolyniec, M. H. Thornquist, J. R. Luke, J. A. McGrath, C. R. Bowie, and T. L. Patterson. 2010. Relationship of the Brief UCSD Performance-based Skills Assessment (UPSA-B) to multiple indicators of functioning in people with schizophrenia and bipolar disorder. Bipolar Disorders 12(1):45-55.

Mausbach, B. T., C. A. Depp, C. R. Bowie, P. D. Harvey, J. A. McGrath, M. H. Thronquist, J. R. Luke, P. S. Wolyniec, A. E. Pulver, and T. L. Patterson. 2011. Sensitivity and specificity of the UCSD Performance-based Skills Assessment (UPSA-B) for identifying functional milestones in schizophrenia. Schizophrenia Research 132(2-3):165-170.

McKibbin, C. L., J. S. Brekke, D. Sires, D. V. Jeste, and T. L. Patterson. 2004. Direct assessment of functional abilities: Relevance to persons with schizophrenia. Schizophrenia Research 72(1):53-67.

Patterson, T. L., S. Goldman, C. L. McKibbin, T. Hughs, and D. V. Jeste. 2001. UCSD Performance-Based Skills Assessment: Development of a new measure of everyday functioning for severely mentally ill adults. Schizophrenia Bulletin 27(2):235-245.

Rosen, M. I., A. C. Black, C. M. Lazar, R. A. Rosenheck, T. J. McMahon, and B. J. Rounsaville, in consultation with K. J. Conrad, R. Ries, and D. Ames. 2015. Financial Incapability Structured Clinical Assessment done Longitudinally (F.I.S.C.A.L) (Version 4/28/2015). New Haven, CT: Yale School of Medicine.

Sabbag, S., E. M. Twamley, L. Vella, R. K. Heaton, T. L. Patterson, and P. D. Harvey. 2011. Assessing everyday functioning in schizophrenia: Not all informants seem equally informative. Schizophrenia Research 131(1-3):250-255.

Sattler, J. M. 2014. Foundations of behavioral, social, and clinical assessment of children, 6th ed. La Mesa, CA: Jerome M. Sattler, Publisher, Inc.

Sherod, M. G., H. R. Griffith, J. Copeland, K. Belue, S. Krzywanski, E. Y. Zamrini, L. E. Harrell, D. G. Clark, J. C. Brockington, R. E. Powers, and D. C. Marson. 2009. Neurocognitive predictors of financial capacity across the dementia spectrum: Normal aging, mild cognitive impairment, and Alzheimer’s disease. Journal of the International Neuropsychological Society 15(2):258-267.

Sireci, S. G., and T. Sukin. 2013. Test validity. In APA handbook of testing and assessment in psychology, Vol. 1, edited by K. F. Geisinger, B. A. Bracken, J. F. Carlson, J. C. Hansen, N. R. Kuncel, S. P. Reise, and M. C. Rodriguez. Washington, DC: APA. Pp. 61-84.

Thomson, L. K. 1992. The Kohlman evaluation of living skills, 3rd ed. Rockville, MD: American Occupational Therapy Association.

Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×

Wallace, C. J., R. P. Liberman, R. Tauber, and J. Wallace. 2000. The Independent Living Skills Survey: A comprehensive measure of the community functioning of severely and persistently mentally ill individuals. Schizophrenia Bulletin 26(3):631-658.

Widera, E., V. Steenpass, D. Marson, and R. Sudore. 2011. Finances in the older patient with cognitive impairment: “He didn’t want me to take over.” Journal of the American Medical Association 305(7):698-706.

Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×

ANNEX TABLE 5-1 Characteristics of Common Instruments for Assessing Financial Capability

Instrument (authors, year) Domains/Itemsa Component of Financial Capability Measuredb Administration Properties
Financial Incapability Structured Clinical Assessment done Longitudinally (FISCAL) (Lazar et al., 2015) 4 global criteria for financial incapability, scored using algorithm to yield dichotomous incapability determination:

(A1) basic needs not met AND (A2) funds needed for basic needs were spent on something else;

(B) substantial funds spent on something that harmed the client; (C1) past misspending (not meeting basic needs) likely to continue; (C2) past misspending (on harmful things) likely to continue

Optional: contextual factors could be used to inform determination after algorithm yielded results

Financial performance; financial judgment Clinician rated

Financial Capacity Instrument (FCI) (Griffith et al., 2003; Marson et al., 2000)

9 domains (activities), 18 tasks, 2 total scores

(Marson et al., 2000: 6 domains, 14 tasks)

Financial knowledge; financial judgment

Training required to administer; 1 of the 9 domains requires collateral report

Can take 40-50 minutes to administer to someone with Alzheimer’s disease (AD)

Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×
Populations Studied Psychometric Properties Notes
118 adults (18-65 years; mean 46, standard deviation [SD] 10.5); ethnically diverse

Inclusion criteria: inpatients in psychiatric unit or in intensive outpatient program; current or past Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) substance use diagnosis; receiving ≥ $600 (Supplemental Security Income [SSI] or Social Security Disability Insurance [SSDI])/month; no rep. payee or conservator

Face validity (expert reviewed)

Interrater reliability: kappa = 0.77 (“very good”

Construct validity: convergent validity with measure of money mismanagement (r = 0.46); discriminant validity with measure of depression (Beck Depression Inventory-II [BDI-II]; r = 0.24)

Weak association with homelessness

Algorithm: A OR B met AND C met; optional use of contextual factors to inform determination; contextual factors invoked for 5 percent of cases (for disorganized behavior and impaired judgment)

Higher agreement among raters when raters more certain of accuracy of their determinations

Demographic characteristics did not differ between capable and incapable individuals

AD, mild cognitive impairment (MCI), Parkinson’s disease (Martin et al., 2013), healthy older adults (as comparison groups)

In Marson et al. (2000) study, good reliability in each of the 6 reliability domains

Developed to assess financial decline in AD and related dementias; used primarily in research; limited clinical utility

People with MCI (amnestic type) demonstrate significant impairments compared (in some but not all domains) with controls matched for age, sex, education, socioeconomic status (SES), and race; worse performance by MCI patients on practical application of concepts than on understanding of financial concepts

Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×
Instrument (authors, year) Domains/Itemsa Component of Financial Capability Measuredb Administration Properties

Financial Capacity Instrument-Short Form (FCI-SF) (Gerstenecker et al., 2015)

5 domains assessed: coin/currency knowledge, financial conceptual knowledge and problem solving, understanding/using a checkbook, and understanding/using a bank statement

Yields 5 scores (Mental Calculation, Financial Conceptual Knowledge, Single Checkbook/Register Task, Complex Checkbook/Register Task, and Using Bank Statement)

Total score (range of 0-74); higher scores indicate better financial skills

Also time-to-completion scores on a number of specific tasks and composite time scores

Financial knowledge

37 items,
15 minutes

Financial Capacity Assessment Instrument (FCAI) (Kershaw and Webber, 2008)

6 domain subscales, one total score

Subscales: everyday financial abilities, financial judgment, estate management, cognitive functioning related to financial tasks, debt management, support resources

Financial knowledge; financial judgment

38 items (original had 41 items; 3 removed)

Structured interview format; “objective scoring guidelines”

Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×
Populations Studied Psychometric Properties Notes

Cognitively normal, community-dwelling older adults (n = 1,344); ages 70-96

Developed from longer FCI; normative data available from original study (Gerstenecker et al., 2015); however, sample predominantly white

Age significantly associated with performance scores (total and 4/5 domains)

Education significantly associated with performance scores (total and all performance subtests)

Adults (n = 178, ages 18-91, mean 53) with cognitive impairment (defined by authors) due to acquired brain injury (n = 36), schizophrenia (n = 29), intellectual disability (n = 32), dementia (n = 22), and healthy controls (n = 59)

Internal consistency (Cronbach’s alpha for subscales ranged from 0.54 to 0.91)

Construct validity (positive correlations with other measures of financial competence)

Interrater reliability; average 89 percent agreement (kappa = 0.86)

Test-retest reliability high for 4/6 subscales and total score

People with a legally appointed administrator performed worse on all dimensions of FCAI compared with people without a legally appointed administrator

Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×
Instrument (authors, year) Domains/Itemsa Component of Financial Capability Measuredb Administration Properties

Money Mismanagement Measure (MMM) (Conrad et al., 2006)

Questions re difficulties meeting basic needs, budgeting, paying bills, and keeping track of funds over past month; assesses financial situation

Financial judgment; possibly financial performance

28 items; self-reported

Timeline Historical Review of Income and Financial Transactions (THRIFT) (Black et al., 2013)

Uses calendar (timeline follow-back method) re past-month income, transactions, expenses, debt, and account balances

Financial judgment; financial performance

Semistructured interview

Assessment of Capacity for Everyday Decision-making (ACED) (Lai and Karlawish, 2007; Lai et al., 2008)

Designed to assess capacity for “everyday” decisions (e.g., medication management); can be tailored to assess financial management

Financial judgment (understanding, appreciation, reasoning, expression of a choice)

Structured questionnaire consisting of 7 items, each scored 0, 1, or 2

Clinician rated and judged

15-20 minutes

a The “domains”listed in this column refer to those described by the authors, as opposed to the components of capability listed in the next column as determined by the committee.

b The components of financial capability measured by each instrument are listed here. These are based on the committee’s judgment of which components are actually measured by an instrument, according to the committee’s consensus definitions of financial capability, financial competence, financial knowledge, financial judgment, and financial performance (see Chapter 4).

Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×
Populations Studied Psychometric Properties Notes

Two samples of people with serious mental illness (SMI):

(1) (n = 186, mean age 43);

(2) (n = 184, mean age 46)

Rasch person reliability = 0.85 (in untreated sample; lower Rasch of 0.72 in original sample due to restriction of range)

Good construct validity (expected correlations with other measures)

Screening assessment of money mismanagement for people with SMI

Specifically developed with representative payee system in mind

People with SMI (n = 28)

1-week test-retest reliability: r = 0.77 (income); r = 0.91 (expenses); r = 0.99 (debt)

Available at www.behaviorchange.yale.edu

Patient-beneficiaries (n = 134)

Mental health clinicians (n = 78), identified by patients as people who “could best answer questions about them for past 6 months”

Correctly classified 73 percent of cases in combination with SSA method (when compared with method described in FISCAL)

Factor analysis supported expert-identified conceptual groupings of four subscales

Intended for use by treating clinician

Available at www.behaviorchange.yale.edu

Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×

This page intentionally left blank.

Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×
Page 125
Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×
Page 126
Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×
Page 127
Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×
Page 128
Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×
Page 129
Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×
Page 130
Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×
Page 131
Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×
Page 132
Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×
Page 133
Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×
Page 134
Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×
Page 135
Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×
Page 136
Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×
Page 137
Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×
Page 138
Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×
Page 139
Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×
Page 140
Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×
Page 141
Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×
Page 142
Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×
Page 143
Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×
Page 144
Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×
Page 145
Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×
Page 146
Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×
Page 147
Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×
Page 148
Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×
Page 149
Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×
Page 150
Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×
Page 151
Suggested Citation:"5 Methods and Measures for Assessing Financial Competence and Performance." National Academies of Sciences, Engineering, and Medicine. 2016. Informing Social Security's Process for Financial Capability Determination. Washington, DC: The National Academies Press. doi: 10.17226/21922.
×
Page 152
Next: 6 Conclusions and Recommendations »
Informing Social Security's Process for Financial Capability Determination Get This Book
×
Buy Paperback | $52.00 Buy Ebook | $41.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The U.S. Social Security Administration (SSA) provides benefits to disabled adults and children, offering vital financial support to more than 19 million disabled Americans. Of that group, approximately 5.5 million have been deemed – by virtue of youth or mental or physical impairment - incapable of managing or directing the management of their benefits. Hence, a representative payee has been appointed to receive and disburse SSA payments for these beneficiaries to ensure that their basic needs for shelter, food, and clothing are met. Periodically, however, concerns have been expressed about the accuracy of the process by which SSA determines whether beneficiaries are capable of managing their benefits, with some evidence suggesting that underdetection of incapable recipients may be a particular problem.

The importance of creating as accurate a process as possible for incapability determinations is underscored by the consequences of incorrectly identifying recipients either as incapable when they can manage their benefits or as capable when they cannot. Failure to identify beneficiaries who are incapable of managing their funds means abandoning a vulnerable population to potential homelessness, hunger, and disease.

Informing Social Security's Process for Financial Capability Determination considers capability determination processes used by other similar benefit programs, abilities required to manage, and direct the management of, benefits, and effective methods and measures for assessing capability. This report evaluates SSA's capability determination process for adult beneficiaries and provides recommendations for improving the accuracy and efficiency of the agency's policy and procedures for making these determinations.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!